HOWTO : Sendmail & Lotus Domino (on the same box)

The new DNSBL feature in Domino6 is definately a step forward in the fight against spam, however the lack of any built-in support for the $DNSBLSites tags in the Notes client makes the feature much less than useful than Lotus would have you believe. I'm a big fan of the open source anti-spam/anti-virus tools for sendmail -- SpamAssassin, ClamAV, etc. and really wanted to use them in front of Domino. The only problem was that I didn't have an extra box sitting around to use as an inbound relay in front of the Domino server. So I figured I'd just run sendmail on port 25 and the Domino SMTP listener task on port 26 and relay from one to the other. I'd already done pretty much the same thing using Symantec Anti-Virus for SMTP Gateways and Domino on the same Win2K box so I knew it could work...in theory.

I got it all configured without too much trouble but sendmail simply would not relay anything to localhost and kept generating this error:

553 mail.mydomain.com. config error: mail loops back to me (MX problem?)

I know it loops back to me -- that's the idea! I found plenty of helpful info on how to correct the MX error which usually is the cause of that error message. But I couldn't find any mention of howto ignore the error and relay anyway. I was certain sendmail could do it but I was having no luck finding that magic incantation. After reading some sendmail/anti-virus howtos, I contemplated writing a custom delivery agent to get the mail to Domino. I was slowly making my way through the chapter on delivery agents in the "bat book" when i found the answer to the loopback problem:

define(`SMTP_MAILER_FLAGS', `k')dnl

The k flag tells sendmail to ignore loopback errors and sure enough, once I added that line to my sendmail.mc, the mail relayed through sendmail to Domino flawlessly.

Here's the relevant lines from my sendmail.mc:

define(`SMART_HOST',`esmtp:[127.0.0.1]')dnl
define(`SMTP_MAILER_FLAGS', `k')dnl
define(`LOCAL_MAILER_FLAGS', `k')dnl
define(`confRELAY_MAILER', `esmtp')dnl
define(`RELAY_MAILER_ARGS', `TCP localhost 26')dnl
define(`SMTP_MAILER_ARGS', `TCP localhost 26')dnl
define(`ESMTP_MAILER_ARGS', `TCP localhost 26')dnl

I was originally using mailertable to do the relay for specific domains only, which is why the RELAY_MAILER lines are in there, but I ended up using SMART_HOST instead becuase it skips mailertable all together -- one less file to edit. Whichever you choose, smarthost or mailertable, you'll need the domains you want relayed to Domino in the relay-domains file and *not* in the local-host-names file.

In the next post I'll try installing and configuring MailScanner, which is a mail framework that wraps up SpamAssassin, your choice of anti-virus, DCC, Razor, and Pyzor, into a sort of "sendmail sandwich."