On Sun, Nov 3, 2013 at 11:49 AM, Private Sender <nobody@snovc.com> wrote:

I would recommend you go a step further and use DKIM, ADSP, and DMARC.

Don't do DMARC if you expect to have end-users forward emails, or subscribe to mailinglists. Despite the removal from the current DMARC spec, the original guidelines called for DMARC to be used for auto-generated transactional email, and expressly not for end-user generated content. -Jim P.

On 11/3/2013 8:11 PM, John Levine wrote:

...

I would recommend you go a step further and use DKIM, ADSP, and DMARC.

Using DKIM is a good idea. Do *not* use ADSP. It is a failed experiment which will provide no benefit and considerable pain.

+1

If you believe that your domain is heavily forged (which if you are not Paypal, Facebook, or a large bank or ISP, it almost certainly is not), you can set up a DMARC record to collect some statistics about what mail other people are getting that appears to be from you. Do not try to use DMARC to tell people to quarantine or reject your mail until you are really sure you understand the statistics you're getting.

+1 The 'reporting' function in DMARC appears to have wide applicability and substantial benefit. The handling (rejection, etc.) function has very narrow benefit. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net

On Nov 4, 2013, at 8:41 AM, Franck Martin <fmartin@linkedin.com> wrote:

www.maawg.org has published a sender BCP, please read it

You mean http://www.maawg.org/sites/maawg/files/news/MAAWG_Senders_BCP_Ver2a-updated.... Regards, -drc

* Private Sender <nobody@snovc.com>:

On 11/3/2013 8:39 AM, rwebb@ropeguru.com wrote:

...

I am looking for some info on current practice for an email server and SMTP delivery. It has been a while since I have had to setup an email server and I have been tasked with setting up a small one for a friend. My question centers around the server sending outgoing email and the current practices requirements for other servers to accept email Things like rDNS, SPF records, etc... [...] MX, PTR, and SPF are really all you need. I would recommend you go a step further and use DKIM, ADSP, and DMARC. It will help keep asshat spammers from flaming your domain all over the internet.

And while you are at it - why not implement DNSSEC for the domain in question and publish some DANE TLSA records? Stefan

On Sun, Nov 3, 2013 at 11:08 AM, Rich Kulawiec <rsk@gsp.org> wrote:

non-generic DNS/rDNS. ("non-generic" meaning something that looks like a host that should sending and receiving email. In other words, mailgw.example.net looks real. ip-137-12-16-164.example.com looks like a random host that's probably part of a botnet.)

This is a good idea, but you can use a generic hostname if you want. It's not a consensus that you need a non-generic hostname. There can be negative consequences for mail delivered to some recipients, because SpamAssassin and some other reasonably popular spamfilters, will use a regular expression based on a sending mail server's rDNS name to decide that "ip-137-12-16-14.example.com" is not a bonafide mail server, based on the hostname alone; this can increase the probability that legitimate mail you send will be miscategorized as spam, if you do choose to use a generic hostname for a mail server. http://svn.apache.org/repos/asf/spamassassin/branches/jm_bug_3852_two_level_... There are many ad-hoc rules that spam filters will subject messages and sending mail servers to, that are not part of the formal requirements for sending MTAs. Make sure that you HELO/EHLO as the same host -- unless there's

some good reason not to. There probably isn't.

The RFC contains a MUST NOT in regards to verifying the HELO name matches. So, the HELO can use any hostname --- as long as the hostname forward resolves to something; it should resolve to the IP address of one of your mail servers. Some mail servers provide service for many domains, and have many DNS names that could be placed in a HELO. The general rule here is: Look at what is common, what is the simplest and most likely way for mail server operators to present themselves. You don't want your mail server to look any different to the outside world, either in operation, naming, or forward and reverse DNS configuration, HELO messages, etc, from a majority of other legitimate mail servers.

SPF is worthless crap: don't bother. Use a real MTA, e.g., postfix

I do not believe that is the consensus of the community -- or the working groups behind the SPF-related RFCs. There are many mail servers and spam filters that will enforce SPF policies; there are also a number that will synthesize an "implicit" policy of Soft (or hard) Fail for any sending IP, except the MX or base A name IP, if you do not publish explicit SPF records. I have a spam filter that penalizes messages from domains with no SPF published, or not matching a high-granularity allow condition by scoring them as more likely to be spam; greylisting may be caused by a soft fail, and of course, messages will be rejected on a hard fail. If you are serious about running an outgoing mail server; I do not believe you have the luxury of completely ignoring SPF, DKIM / domainkeys / sender-id, and other established or emerging community standards. After all, they can be used as a tool to help reject some spam. e.g. spoofed @usps.com messages

Don't use a quarantine, it's a horrible idea. (Ask RSA how that worked

Message quarantines are great; they are helpful for mitigating the false positives of overly-agressive filters. A combination of quarantine and reject policies can be useful, for border cases. Especially, if there are some users that "want" spammy messages, which are actually HTML bulk-sent newsletters of some sort or another. out for them.) Make sure you don't backscatter. Make sure you don't

use SMTP "callouts", which are just as abusive as spam. Make sure you

Implemented appropriately; an SMTP callout (or "SMTP connect" verification) to the connecting IP is a great way to help resolve the suspiciousness level of the sending mail server. SMTP connect verification is a good thing to be using, when the sending server is using an envelope From domain, that has not yet bothered to publish the proper SPF records to verify them. There are bad broken SMTP callout implementations that should be avoided, and there are rather useful ones.

have working "postmaster" and "abuse" addresses. Make sure your MTA doesn't emit or respond to return-receipts. Read your logs (again).

Of course, read receipt requests should be suppressed when dealing with internet hosts. Having a working postmaster address is a requirement. Using the name abuse@ for the abuse contact is not required, and it is likely to be targetted by spammers. Working abuse and technical contacts should be published in the IP address and domain name WHOIS databases for the mail server's primary domain, and all envelope from domains that can be originated on that mail server. ---rsk

-- -JH

Thanks to everyone for all the tips and info. I think I have compiled plenty of info to get this done. I will probably start with some of the basics and see how things go. THen as needed start putting in some additional features as I see how things progress. Robert On Fri, 8 Nov 2013 07:37:40 -0500 Rich Kulawiec <rsk@gsp.org> wrote:

I suggest moving this to mailop, where it arguably belongs. But I'm going to follow up on a few points, anyway.

First, I forgot to mention two other highly effective mail system defense methods: geoblocking and passive OS fingerprinting.

Geoblocking: A mail server for a local construction business in Arizona is unlikely to require mail from Poland, Peru or Pakistan. So there's no reason to go with a default-permit model: use default-deny and only allow mail from places where legitimate mail might originate. (In this case, perhaps: the US, Mexico, and Canada.) Use the ranges from ipdeny.com. This will stop an astonishing amount of spam (and other SMTP-borne abuse) cold. And it can be done at the MTA or in the firewall: which is better depends on circumstances.

Obviously this doesn't work for everyone. Obviously this (like everything else) runs the risk of false positives -- but it's easy to mitigate that. Obviously it does require understanding the patterns in your mail traffic, but any competent mail system admin has long since performed detailed statistical analysis and has a pretty good idea what the characteristics of their incoming mail stream look like.

Passive OS fingerprinting: regard anything originating from an OS that fingerprints as Windows as dubious, at best. Possible actions vary: graylisting (more precisely, graylisting regardless of previous traffic) is one good option. Utilizing this in concert with geoblocking (above) works beautifully, e.g., "I'm in Arizona and something in Portugal that fingerprints as Windows is trying to send me SMTP traffic: the probability approaches unity that this is spam". When combined with rDNS information, this becomes a highly efficient mechanism with a FP rate that's ridiculously low. (In other words, if that same system has rDNS that looks like 123-45-67-8.example.com then it either really is a bot or it's a mail system run by someone with no clue.)

A few short points and one long one in response:

On Sun, Nov 03, 2013 at 12:00:23PM -0600, Jimmy Hess wrote:

...

The RFC contains a MUST NOT in regards to verifying the HELO name matches. So, the HELO can use any hostname --- as long as the hostname forward resolves to something; it should resolve to the IP address of one of your mail servers. Some mail servers provide service for many domains, and have many DNS names that could be placed in a HELO.

All true. But none of this argues against using the canonical hostname in the HELO. It's the simplest, easiest option (and quite often the one that software will pick by default).

...

...

SPF is worthless crap: don't bother. Use a real MTA, e.g., postfix

I do not believe that is the consensus of the community -- or the working groups behind the SPF-related RFCs.

I'm well aware it's not the consensus. It's my opinion.

Clue #1 that SPF is crap should have been its grandiose self-promoting announcement ("Spam as a technical problem is solved by SPF"). Clue #2 should have been the observation that -- by far -- the most prolific early adopters were spammers. When your enemy latches on to your new weapon much faster than you do, that should be a tipoff that maybe it's not what you hope it is. Clue #3 is available to anyone who deploys a sufficiently large and diverse set of spamtraps for several years and analyzes the data that arrives: SPF presence/absence or contents have no statistically useful anti-spam value in a properly-designed mail defense architecture.

Don't believe me? Okay. Fine. Set up a few thousand spamtraps, gather data for 3-5 years, see for yourself.

So yes, it's standardized; so what? So is (sort of) DNS forgery, see http://tools.ietf.org/html/draft-livingood-dns-redirect-03 for example. That's also crap, it just happens to be well-documented crap.

So: if you feel you must use something, use DKIM, which I think shows vastly more promise. Just don't expect it to be a panacea, because the current miserable state of security at *all* levels undercuts it badly. Not DKIM's fault, really, but it does impact its usefulness in the real world.

...

Message quarantines are great; they are helpful for mitigating the false positives of overly-agressive filters.

This one I'll spend more time on. Quarantines are a worst practice in mail systems engineering. Here are some assorted reasons why, briefly:

- One of the fundamental principles of mail system defense is that you should make your mistakes early, consistently, and loudly. (And you WILL make mistakes. Everyone does.) The point of doing this is that it enables all concerned, including you, to have a decent chance of noticing them, figuring out that they're mistakes, and correcting them. Quarantines hide, defer, and silence your mistakes, making it much more difficult to run your system properly. They're a lazy admin's coverup, not a fix.

- Quarantines present deadlock problems. Mail from user A to user B which is quarantined and causes B to eventually send a message to user A enquiring about the missing missive stands a decent chance of ending up in B's quarantine. The end result is that human time (which is a scarce and expensive commodity) is needlessly wasted figuring out the problem and then trying to evade it, usually by trying to tapdance a way past the filters on both sides. Contrast with a hard, immediate rejection, which -- coupled with an appropriate resolution mechanism -- facilitates quick, easy diagnosis and repair.

- Some people use quarantines because they're unduly concerned about their false positive rate. But slapping the ill-advised band-aid of a quarantine on an underlying problem like a high FP rate solves nothing. It merely obscures the mistake. The correct solution is to figure out why the FP rate is high, and fix that, because that's where the problem is.

- Users have spent the past two decades or so conclusively proving, beyond any possible argument, that they are utterly incapable of distinguishing spam from non-spam, phish from non-phish. (Consider carefully: if they could, en masse and accurately, would we have the scope and scale of the problem set we currently face? No. We would not.) "Educating users" is a complete failure as a strategy; see point #5 here:

The Six Dumbest Ideas in Computer Security http://www.ranum.com/security/computer_security/editorials/dumb/

Even personnel who *work in security* can't solve this classification problem correctly (which is why I said "ask RSA how that's working out for them"). Your users are no better. They're probably worse. So punting the task of correctly classifying incoming mail to them is reckless and irresponsible. With precious few and rare exceptions, they WILL fail. Freuqently. And sometimes (again: see RSA) the consequences of even a single failure can be catastrophic.

More bluntly, allowing users anywhere near the machinery of security, when they've proven for decades that they're absolutely unqualified to handle even its simplest aspects, is clearly unprofessional and highly negligent. Yes, this is somewhat a fascist, uncompromising, condescending, etc. attitude. It's also the only one that's been demonstrated to work in the real world. It's a shame, really, but this is not the 'net of 1983. You can be nice and flexible and accomodating or you can be (modestly) secure. Pick one.

- Disposition of quarantined mail is problematic -- unless of course the intention is to let it accumulate indefinitely, which is a dubious idea at best. Eventually, *something* has to be done. Delivering it to the user is probably unwise, which you should already know, otherwise you would have delivered it in the first place. Simply deleting it means that you lied to the delivering MTA back when you first accepted it -- and that's very poor operational practice. Not to mention quite rude. So what, *exactly*, do you plan to do with it? When? How?

- Informing users of quarantined mail is another problem. Anyone with any familiarity with user behavior knows that they'll click on anything and everything in a heartbeat without performing any kind of due diligence. So that well-crafted spear phish that you should have rejected outright but quarantined because it scored well enough on your content-screening system? Your user is going to pull that right back and open it as soon as they glance over the "Subject" line quickly in Monday morning's quarantine report, and you will be 0wned by lunchtime. Game over, man.

- On top of all of this, there's an even more fundamental strategic problem. Spam is (among many other things) an attack on the time and attention of recipients. Mail system operators who deploy quarantines are aiding and abetting that attack, because they're chewing up the same scarce and expensive resources: time and attention. One of the goals of a mail defense system is to reduce that time and attention to zero. Of course, under real-world conditions, that's never possible -- still, one should always strive to asymptotically approach it. But quarantines are clearly a massive step in the wrong direction, for what I trust are abundantly obvious reasons.

...

Implemented appropriately; an SMTP callout (or "SMTP connect" verification) to the connecting IP is a great way to help resolve the suspiciousness level of the sending mail server.

No, it's always abusive. As analysis carried out by the late Bruce Gingery, myself, and others on spam-l years ago (when Verizon was doing this) demonstrates, allowing others to cause your operation to generate outbound SMTP traffic to third parties of *their* choosing is a serious tactical error. It furnishes a free, scalable DDoS support service -- which, unfortunately, is not a hypothetical issue. It also facilitates third-party bypass of others' security mechanisms, which, depending on your legal jurisdiction, may be an issue: I would recommend consulting your attorneys before deploying it, and asking them what exposure you incur by doing so. And on top of all that, it has no anti-spam value whatsoever. (Note: self-directed callouts, to one's own internal mail infrastructure, are not abusive. They're inefficient, and much better methods nearly always exist for providing the same functionality, but they're not abusive.)

...

Using the name abuse@ for the abuse contact is not required, and it is likely to be targetted by spammers.

Of course it will be. So will "postmaster". So will every other role and non-role address. Any reasonably competent operation should have no problem dealing with that. IMHO, anyone who cannot manage the rudimentary task of managing their "abuse" mailbox is wholly unqualified to run a mail system. (Or a web server, or a network, or anything else exposed to the public Internet.)

---rsk

On Fri, Nov 08, 2013 at 08:37:32AM -0500, William Herrin wrote:

On Sun, Nov 3, 2013 at 11:39 AM, <rwebb@ropeguru.com> wrote:

...

I am looking for some info on current practice for an email server and SMTP delivery. It has been a while since I have had to setup an email server and I have been tasked with setting up a small one for a friend. My question centers around the server sending outgoing email and the current practices requirements for other servers to accept email Things like rDNS, SPF records, etc...

Hi Robert,

Current best practices are: don't run your own email server unless you're willing to spend the ongoing time and effort it takes to keep up with the current solutions to the spam, hacking and abuse problems. Corollary: when you get bored of doing so for a tiny mail server, stop running it and buy a service.

and yet, at the IETF this week, in the technical plenary, a call to diffuse the target space by running your own services. much harder to have your mail scrapped from your servers than from your providers. /bill

Other than that, the _changes_ of note in the last decade are:

1. The blacklist aggregators and IP reputation services have changed so you have to find the new ones, 2. There are email whitelist services now, some free others for a nominal cost. Use them. 3. Phishing and spear phishing are relatively sophisticated now, so your spam solution has to deal reasonably with it. 4. Relay from and to an external address without changing the envelope sender no longer functions reliably due to things like SPF enforcement and no mail servers I've noticed have such a translator built in.

Regards, Bill Herrin

-- William D. Herrin ................ herrin@dirtside.com bill@herrin.us 3005 Crane Dr. ...................... Web: <http://bill.herrin.us/> Falls Church, VA 22042-3004