Archive for category Postfix
These are from python.org server
maximal_backoff_time = 1000s
minimal_backoff_time = 300s
Every now and then, mail fails with “timed out while sending end of data — message may be sent more than once”, or with: “lost connection after DATA”. Network outages happen, systems crash. There isn’t much you can do about it. Usually the problem goes away by itself.
However, when you see mail deliveries fail consistently, you may have a different problem: broken path MTU discovery. Or it could be a broken PIX firewall.
Cisco PIX “fixup protocol smtp” bug
The Cisco PIX firewall has a bug when running software older than version 5.2(4) or 6.0(1).
The bug ID is CSCds90792. The “fixup protocol smtp” feature does not correctly handle the case where the “.” and the “CRLF” at the end of mail are sent in separate packets.
How does one recognize a mailer behind a Cisco PIX with “fixup protocol smtp” enabled? As of version 5.1 and later, the fixup protocol smtp command changes the characters in the SMTP banner to asterisks except for the “2”, “0” and “0 SPACE” characters.
When you connect to a mailer behind such a filter you see something like:
220 **************************************0******0*********20 ****200**0*********0*00
IP path MTU discovery
A little background is in order. With the SMTP protocol, the HELO, MAIL FROM and RCPT TO commands and responses are relatively short. When you’re talking to old versions of sendmail, every command and every response is sent as a separate packet, because sendmail didn’t implement ESMTP command pipelining until recently.
The message content, however, is sent as a few datagrams, each datagram typically a kbyte large or even bigger, depending on your local network MTU.
When mail fails consistently due to a timeout, I suspect that the sending machine runs a modern UNIX which implements path MTU discovery. That causes the machine to send packets as large as it would send over the LAN, with the IP DON’T FRAGMENT bit set, preventing intermediate routers from fragmenting the packets that are too big for their networks.
Depending on what network path a message follows, some router on the way responds with an ICMP MUST FRAGMENT message saying the packet is too big. Normally, the sending machine will re-send the data after chopping it up into smaller pieces.
However, things break when some router closer to the sending system is dropping such ICMP feedback messages, in a mistaken attempt to protect systems against certain attacks. In that case, the ICMP feedback message never reaches the sending machine, and the connection times out.
This is the same configuration problem that causes trouble with web servers behind a misconfigured packet filter: small images/files are sent intact, large images/files time out because the server does not see the MUST FRAGMENT ICMP feedback messages.
Workaround: at the sending machine, disable path MTU discovery. Mail will get out, but of course everyone else will still suffer. How to disable path MTU discovery? It depends. Solaris has an ndd command; other systems use different means such as sysctl to control kernel parameters on a running system.
Workaround: at the receiving machine, set a smaller MTU. For example, people using PPPoE (PPP over Ethernet) often have to choose an MTU lightly smaller than the default 1500 for ethernet.
Fix: find the router that drops the ICMP MUST FRAGMENT messages, and convince the person responsible for it to fix the configuration.
Postfix (smtpd daemon) can enforce a number of limits on incoming email. This will stop email flooding attacks.
A bot connects to your Postfix email server and sends garbage commands or spam, attempting to crash your server. You can limit:
=> The length of lines in a message and so on
=> The size of messages
=> The number of recipients for a single delivery
Try following directives in your postfix main.cf config file:
smtpd_error_sleep_time – The SMTP server response delay after a client has made more than $smtpd_soft_error_limit errors, and fewer than smtpd_hard_error_limit errors, without delivering mail.
smtpd_soft_error_limit : The number of errors a remote SMTP client is allowed to make without delivering mail before the Postfix SMTP server slows down all its responses.
smtpd_hard_error_limit : The maximal number of errors a remote SMTP client is allowed to make without delivering mail. The Postfix SMTP server disconnects when the limit is exceeded.
Open config file
# vi main.cf
Append following directives:
smtpd_error_sleep_time = 1s
smtpd_soft_error_limit = 10
smtpd_hard_error_limit = 20
Save and restart/reload postfix configuration
# /etc/init.d/postfix restart
Postfix waits one second before each error such as HELO command not provided or FQDN hostname does not exists etc After 10 such errors postfix will start to increase delay. If error limits touches 20 Postfix will disconnect client.
- Add user and group groupadd dspam useradd -g dspam -s/bin/false -c"DSPAM" dspam -Compile ( setenv CFLAGS '-O6 -march=i686' ;./configure \ --with-dspam-home=/var/dspam \ --with-dspam-home-mode=770 \ --with-dspam-home-owner=dspam \ --with-dspam-home-group=postdrop \ --with-dspam-mode=2510 \ --with-dspam-owner=dspam \ --with-dspam-group=postfix \ --with-delivery-agent=/usr/sbin/sendmail \ --with-storage-driver=mysql_drv \ --with-mysql-includes=/usr/local/include/mysql \ --with-mysql-libraries=/usr/local/lib/mysql \ --enable-preferences-extension \ --enable-virtual-users \ --enable-daemon \ --enable-large-scale ) make make install - DSPAM.CONF /usr/local/etc/dspam.confHome /var/dspam StorageDriver /usr/local/lib/libmysql_drv.so TrustedDeliveryAgent "/usr/sbin/sendmail" DeliveryHost 127.0.0.1 DeliveryPort 10026 DeliveryIdent localhost DeliveryProto SMTP OnFail error Trust root Trust httpd Trust dspam Trust postfix Trust mail Trust mailnull Trust smmsp Trust daemon TrainingMode teft TestConditionalTraining on Feature chained Feature whitelist Algorithm graham burton PValue graham Preference "spamAction=quarantine" Preference "signatureLocation=headers" # 'message' or 'headers' Preference "showFactors=on" AllowOverride localStore AllowOverride trainingMode AllowOverride spamAction spamSubject AllowOverride statisticalSedation AllowOverride enableBNR AllowOverride enableWhitelist AllowOverride signatureLocation AllowOverride showFactors AllowOverride optIn optOut AllowOverride whitelistThreshold MySQLServer /tmp/mysql.sock MySQLPort 3306 MySQLUser dspam MySQLPass ****** MySQLDb dspam MySQLCompress true MySQLVirtualTable dspam_virtual_uids MySQLVirtualUIDField uid MySQLVirtualUsernameField username MySQLUIDInSignature on -Modify master.cf# ========================================================================== # service type private unpriv chroot wakeup maxproc command + args # (yes) (yes) (yes) (never) (100) # ========================================================================== #smtp inet n - n - 40 smtpd -o content_filter=filter:dummy #smtp inet n - n - 30 smtpd -o content_filter=clamav:clamav smtp inet n - n - 30 smtpd -o content_filter=lmtp:unix:/tmp/dspam.sock#ALLOW CLAMAV TO WORK WITH DSPAM CLAMAV localhost:10026 inet n - n - 50 smtpd # -o cleanup_service_name=pre-cleanup -o content_filter=clamav:clamav -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks -o smtpd_helo_restrictions= -o smtpd_client_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o mynetworks=127.0.0.0/8 -o smtpd_authorized_xforward_hosts=127.0.0.0/8
With hundreds of Postfix processes, the kernel will eventually run out of file handles; after that, it will run out of sockets.
To set the following kernel parameters at boot time, add the following lines to the /boot/loader.conf file (this is verified with FreeBSD 4.4):
With FreeBSD 4.2, the last three parameters cannot be set from /boot/loader.conf. To set the open file limits, execute the following commands as root:
# sysctl -w kern.maxfiles=16384
# sysctl -w kern.maxfilesperproc=16384
With FreeBSD 4.2, kern.maxproc can be set only by recompiling the kernel with a different maxusers setting in the kernel configuration file.
Cool tips from Ralf Hildebrandt to tune your Postfix http://www.arschkrebs.de/postfix/
11. Stress Testing
How much mail will my box be able to handle?
To find out how much traffic your installation can handle, you need to perform some kind of stress testing. To put an adequate load on the server, you need a fast mail generator. Postfix comes with a pair of testing programs named smtp-source and smtp-sink for just this purpose. Here’s how they work:
This program connects to a host on a TCP port (port 25 by default) and sends one or more messages, either sequentially or in parallel. The program speaks both SMTP (default) or LMTP and is meant to aid in measuring server performance.
This test server listens on the named host (or address) and port. It recieves messages from the network and throws them away. You can measure client and network performance with this program.
Let’s start with smtp-source to stress-test your Postfix installation. The following example injects 100 total messages of size 5k each in 20 parallel sessions to a Postfix server running on localhost port 25. Because you’re also interested in how much time this takes, use the time command:
$ time ./smtp-source -s 20 (1) -l 5120 (2) -m 100 (3) -c (4) \
-f email@example.com (5) -t firstname.lastname@example.org (6) localhost:25 (7)
(1) 20 parallel sessions
(2) 5k message size
(3) 100 total messages
(4) display a counter
(5) envelope sender
(6) envelope recipient
(7) target SMTP server
In the example above, injection took 4.294s. However, you also want to know how long actual delivery takes? Check your logs for this, and also to verify that every last message arrived for <email@example.com> received.
Now let’s turn our attention to smtp-sink to find out how many messages per second your server can handle from your horrible mass mailing sofware. Postfix has to process each outgoing message even if the server on the other side throws it away (therefore, you can’t use this to test the raw performance of your mass mailer unless you connect your mailer directly to smtp-sink).
The following example sets up an SMTP listener on port 25 of localhost:
$ ./smtp-sink -c localhost:25 1000
Now you can run your client tests.
If you want to get an idea for how much overhead the network imposes and also get a control experiment to see what the theoretical maximum throughput for a mail server, you can make smtp-source and smtp-sink talk to each other. Open two windows. In the first, start up the dummy server like this:
# ./smtp-sink -c localhost:25 1000
With this in place, start throwing messages at this server with smtp-source in the other window:
$ time ./smtp-source -s 20 -l 5120 -m 100 -c \
-f firstname.lastname@example.org -t email@example.com localhost:25
This output shows that smtp-sink is much faster at accepting messages than Postfix. It took only 0.239 seconds to accept the messages, which is 18 times faster than the Postfix injection process. Now, wouldn’t it be nice if you could throw away all incoming email like this?
11.1. Disk I/O
Why do I see huge load, when no process is actually using the processor during stress testing?
When you run your stress testing, you might encounter huge load averages on your machine that seem out of place. Assuming that you don’t have any content filtering in place, Postfix is I/O bound, so your I/O subsystem could be saturated.
If the output of top shows a a high load such as 10.7, but none of your processes are actually using the CPU. In this particular case, your load is probably coming from the kernel using most of the CPU for I/O and not letting processes run. Furthermore, the reason that the kernel is doing so much I/O is that many more processes have requested I/O operations (and are now waiting for them).
Linux 2.6 kernels support iowait status in the top command. To see if this is the case on 2.4.x kernels (which don’t have a seperate means of displaying the iowait status), you can add a kernel module. Oliver Wellnitz wrote such a kernel module that you can download at ftp://ftp.ibr.cs.tu-bs.de/os/linux/people/wellnitz/programming/. This module calculates the load differently and gives you an interface in the /proc filesystem that you can see like this:
# cat /proc/loadavg-io
rq 0.30 0.23 0.14
io 0.08 0.31 0.27
In this example, rq is the number of processes, which are in the state TASK_RUNNING, while io is the number of processes, which are in the state TASK_UNINTERRUPTIBLE (waiting for I/O). The sum of those two is what the kernel usually calls load.
If you’re having problems like this, you need faster disks, or even a solution such as a SSD (a solid state disk, basically a battery backupped RAM disk) or a mirrored/striped RAID for the queue directory. See Section XX.XX[XREF, in the performance chapter] for more information. One other solution that may or may not work is to remove the synchronous updates for the queue directory. If you’re using an ext2 or ext3 filesystem, try this command:
# chattr -R -S /var/spool/postfix/
This setting is actually the default with recent Postfix installations.