On Tue, Oct 27, 2015 at 10:18:11AM -0400, Ian Smith wrote:
I'm not making any argument about the relation of SPF compliance to message quality or spam/ham ratio. You are no doubt correct that at this point in the game SPF doesn't matter with respect to message quality in a larger context, because these days messages that are not SPF compliant will simply never arrive, and therefore aren't sent.
No, what I'm trying to explain to you is that the presence or absence of SPF records, and the content of those records, has no bearing on whether not messages emitted are spam or nonspam. I have millions of messages that passed all SPF checks and are clearly spam; I have millions of messages that failed (or had none) and are clearly not spam. I have analyzed this data in ridiculous detail using a variety of statistical/pattern recognition techniques, and the bottom line vis-a-vis SPF is that it simply doesn't matter. It would be nice if it did; it would be nice if the fatuous claim made at SPF's introduction ("Spam as a technical problem is solved by SPF") were true. But it's not. It's worthless.
I'm saying that SPF helps prevent envelope header forgery, which is what it was designed to do.
When trying to stop spam, forgery (header and otherwise) is largely unimportant. These are separate-but-related problems, and it is a fundamental tactical error to conflate them.
The fact that NANOG isn't checking SPF (and it isn't, I tested) was exploited and resulted in a lot of spam to the list.
No, the fact that NANOG's mailing list mechanisms (the MTA, Mailman, etc.) aren't defended by *other* methods is the problem. There are much better ways to accomplish the goal than screwing around with SPF.
You can argue that envelope header forgery is irrelevant, and that corner cases don't matter. But I think this latest incident provides a good counterexample that it does matter.
It's an unimportant and isolated incident. I have a bazillion of those too. But using those, instead of looking at overall statistical trends, is a very poor way to craft mail system defenses. The correct strategy is to begin with those measures that: - have the lowest FP rate - have the lowest FN rate - take the least bandwidth - take the least memory - take the least CPU - are as close as possible to the source of abuse - rely the least on external resources - are simplest to understand - are simplest to implement - are easiest to monitor and evaluate - are easiest to maintain and modify - are the least susceptible to gaming - are the least susceptible to "flavor-of-the-moment" issues and then work down the list from there. This yields architectures that are effective, predictable, and accurate. Not perfect: there is no such thing; but certainly robust enough for production use. ---rsk