On Mon, Sep 06, 2004 at 12:04:45PM +0000, Peter Corlett wrote:

Henry Linneweh <hrlinneweh@sbcglobal.net> wrote:

...

This is not a good beginning http://www.eweek.com/article2/0,1759,1642848,00.asp

I'm an advocate of SPF, but not because it's the magic bullet that stops spam. It does however allow innocent domains to say "no, I didn't send that" and thus avoid the double-bounced backwash from a spammer forging their domain as the sender.

It's also a step towards making domain-based whitelists / blacklists more practical (and, as pointed out recently on spam-l, which might be a more appropriate place for this discussion, makes more aggressive filtering of non-whitelisted domains and domains without SPF records more possible). It should hopefully help with viruses that forge the sender-address and should help reduce bouncebacks due to spam and viruses with forged sender addresses. It can help make phishing scams more difficult to pull off. It makes it easier for someone to say "this domain will NEVER send any legitimate email traffic". Will spammers register tons of new domains, setting up SPF for each? Probably. Will they start spoofing other domains hosted by the same provider? . Will they register "look-alike" domains? Will viruses get smarter, and start sending themselves out via providers' SMTP servers? Probably. But all of these cases are still an improvement over the current situation, and help make life easier for existing email filtering / processing tools. I don't personally believe that "[s]pam as a technical problem is solved by SPF"[1], but I do think it has the potential to reduce some existing problems with email (some of which are related to spam). I'm cautiously optimistic that it /may/ be a good thing. Victor Duchovni made some interesting points about SPF on spam-l that are worth checking out if you can access the archives. Some excerpts (please edit attributions if you're quoting / replying to this - I didn't write this): What everyone is forgetting is that the biggest proponents of SPF are large mailbox providers, and their real motivation is actually not so much deterring spam, but lowering the administrative cost of maintaining white-lists! White-listing IP addresses loses, because legitimate bulk mailers (and some no so legitimate ones, but that is not the point) who are whitelisted by the ISPs occasionally move their outbound relays to new address pools. Also some providers host multiple sender domains, some that one wants to whitelist and some that one does not. [...] This does nothing to block spam, this merely decentralizes whitelist management. With more up-to-date (reliable?) whitelists, one can afford to spend more resources on aggressive filters of mail that is not white-listed, and not worry as much about false positives. [1] http://www.interesting-people.org/archives/interesting-people/200401/msg0003... -- "Since when is skepticism un-American? Dissent's not treason but they talk like it's the same..." (Sleater-Kinney - "Combat Rock")

On Mon, 6 Sep 2004, Edward B. Dreger wrote:

Yawn. "If the sender domain isn't forged, the mail isn't spam" is incredibly stupid logic.

No Kidding!

I suppose the next big news article will be that spammers also prefer forging domains that lack SPF records. (Will miracles never cease?)

Amazing :) I think SPF is an important step in getting rid of people pretending to be someone else. If you have SPF records, and they match the mail, chances are you are who you say you are. Finding out who you are behind domain records/etc, thats a different story...

On Tue, Sep 07, 2004 at 11:32:11AM +0100, Paul Jakma <paul@clubi.ie> wrote a message of 24 lines which said:

Also, SPF doesnt tell you whether it is spam.

Of course. It never pretended to do so.

Indeed, apparently majority of SPF-valid email at moment is spam!

No. Where did you find the figures?

On 09/07/04, Paul Jakma <paul@clubi.ie> wrote:

On Tue, 7 Sep 2004, Stephane Bortzmeyer wrote:

...

...

Also, SPF doesnt tell you whether it is spam.

Of course. It never pretended to do so.

Right, but a lot of seem people to be under the mistaken impression it will have some effect on spam.

And every time those people speak up, someone who knows better corrects them. Thus is wisdom gained. -- J.D. Falk "...one of the worst signs of our danger <jdfalk@cybernothing.org> is we can't imagine the route from here to utopia." -- Kim Stanley Robinson

On Tue, 7 Sep 2004, Paul Vixie wrote:

i don't agree. i think it's overengineered and that a simpler solution like the one at <http://sa.vix.com/~vixie/mailfrom.txt>

oh, hear hear. Then there's Sender-ID. Bulky XML in DNS, sigh.

should have been deployed years ago, but i don't think SPF, or things like SPF, are at all worthless.

every time someone forges one of my domains or e-mail addresses as a spam source, i get all kinds of bot-mail telling me that what the spammer tried to do didn't work. quite a lot of challenge/response nonsense. quite a few majordomo/etc listbot error messages. a whole pile of mailer-daemon@ errors.

True, but bounces, and anything else with NULL return path, can be taken care of with SRS. Bogus bounces are probably the most annoying non-spam email problem, and we do not need SPF to kill those. Hence, given a better solution to the only pressing problem we know SPF can solve, SPF is worthless. For the other problems, well, SPF just isnt going to solve them. So SPF will tell you that client.acme.net is indeed allowed to send mail from foobar.com, but that describes only trust between foobar.com->client.acme.net. I am no wiser at all as to whether foobar.com is worthy enough to send me email. And given that there are *millions* of domains, and they can be registered by anyone within minutes, I'm unlikely *ever* to be able to make any use of the knowledge that foobar.com allows client.acme.net to send mail on their behalf to discriminate between genuine and spam email. (other than whitelisting clients i trust - but i dont need SPF for that). Indeed, you've been saying this for years. ;) (which is largely how i've come to my own opinion ;) )

if all mailbots learned to speak something like SPF, and my domains all advertise the nec'y metadata to enable something like SPF, then i would find it far easier to filter the remaining drivel in my inbox, which would just be spam and e-mail (listed in order by volume) -- no more mailbot responses to messages i never sent.

See: http://www.libsrs2.org/ http://www.libsrs2.org/srs/srs.pdf http://asarian-host.net/srs/sendmailsrs.htm And be happy, and realise "SPF is worthless" ;)

the economic benefit that will actually cause something like SPF to come into wide use is different yet again -- it's not to make it easier to filter the remainder, and it's not to stop spam. it's to protect trademarks owned by large e-mail providers ("@hotmail.com" being one, "@yahoo.com" being another) from dilution.

Ah, ok. Yes, I've read you making above argument before and, aye, it's a very fair point. But, is it enough of a reason? It seems like a fallback reason, for use when other answers to "what actual real problems does SPF solve?" are not forthcoming. Is it really worth it for every domain owner on the planet (including spammers!) to implement SPF records in DNS, and the resulting forwarding breakage, simply to provide some fairly intangible "dilution protection" for, primarily, the very small subset of widely-known domains out there? It would prevent joe-jobs, yes. But how bothersome are those, given that the bounces can be dealt with with the far less intrusive SRS?

everything that happens on the internet these days happens for economics-related reasons. i'm glad that companies bigger and richer than i am find it in their own selfish best interests to push something like SPF -- that means it'll happen. that my own reasons differ from theirs is immaterial. that they have to mismarket it as a spamstopper to get corporate and investor support for it is also immaterial. the fact is, it's coming -- and

Well that depends. At the moment it looks like the clients will implement a standard that most of the servers will not! Also, I doubt I'll be implementing SPF myself. Indeed, to implement SPF I would have to list the MTAs of at least several irish ISPs, and probably more, as I have users who only receive email via my systems, but dont send it via systems. yes yes, MSA.. but I dont even know most of these people except as usernames in a password file, they're mostly non-technical, and I dont intend to track them down one by one and go visit them to reconfigure their MUAs for them. And even if i did, no doubt they also have /other/ email addresses, eg one from their ISP, and many popular, particularly older versions of, MUAs have problems with allowing one to configure SMTP/MSA according to From address, sigh.

it's useful, just not for the advertised reasons, or a universal reason.

Ah, absolutely yes. regards, -- Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A Fortune: It does not matter if you fall down as long as you pick up something from the floor while you get up.

"J.D. Falk" <jdfalk@cybernothing.org> wrote:

On 09/07/04, Paul Jakma <paul@clubi.ie> wrote:

...

Then there's Sender-ID. Bulky XML in DNS, sigh.

No, that was CallerID. SenderID uses a format that looks and smells almost exactly like SPF.

I only mention this to reduce the FUD.

Sender-ID requires the processing of a minimum of 10 TXT scripts that can reference dozens of A, AAAA, MX, PTR, CIDR notation for IPv4 & IPv6 addresses, construct labels from various message fields, invoke redirections, includes, "exists" macros, and specify both "Pass"/"Neutral" open-ended lists, where each script is provided a total of 20 seconds. In addition, the draft requires undocumented extensions be ignored, even at the released revision level. Such amounts of DNS lookups of "who knows what" may easily exceed normal mail traffic, and, with the 20 second timeout (200 second total), it is unlikely exponential UDP back-off will be obtained. XML would have made it worse, but that does not mean there is little reason for concern. By simply authenticating the MTA EHLO and establishing a name, a mailbox domain to MTA name relationship, as a name list, can be established within a single DNS lookup without the use of script. Importantly, the EHLO establishes a name that does not assume mail channel integrity to be useable for reputation assertions. Such a mailbox domain to MTA name list would not force the use of specific RFC2822 From mailbox domains, as it would be independent of the MTA authentication. Such a name list would not expose users to harm by both being spoofed and then having their mailbox domain blacklisted by reputation services mistakenly trusting Sender-ID. There is nothing within Sender-ID that indicates an MTA is shared, or that Sender-ID was checked. -Doug

On Mon, 6 Sep 2004, Sean Donelan wrote: Hrmmm, perhaps this hasn't been thought of yet, but this is a serious idea for things like spamassassin, or the like. For this list of domains, a decent twofold effort could happen: 1) A decent push on the part of pobox.com (previously, their focus has been on protecting lots of senders, like AOL, or Earthlink), rather than commonly-forged-phishers, to get these folks on board. 2) A big old warning (possibly for these domains themselves to opt into) as a "we know we're high risk but we have an SPF record, please check it" RDNSL. It could even be used in some cases with SpamAssassin to inject a link into the email for the location to report such forgeries. (Such info could be kept in the RDNSL, for example). Knowledge is Power. -Dan

Although SenderID (or whatever the final name is) is not completed yet, SPF has been around for a while and some people have been using it. But who? Do domains with SPF records have fewer phishing attacks? Fewer virus bounce-backs? Fewer spam forgiers?

According to the Anti-Phishing Working Group, these are the most phished companies. How many are using SPF? I checked the most obvious domain name for the companies (.COM and their country variant e.g. .CO.UK)

Company Name Has SPF TXT record

Citibank NO eBay NO US Bank NO Paypal NO Fleet NO LLoyds NO Barclays NO AOL YES Halifax NO Westpac NO FirstUSA NO VISA NO Earthlink YES e-gold NO Bank One NO Bendigo NO HSBC NO MBNA NO Suntrust NO Verizon NO

-- "there is no loyalty in the business, so we stay away from things that piss people off" -The Boss, November 12, 2002 --------Dan Mahoney-------- Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---------------------------

Niels Bakker <niels=nanog@bakker.net> wrote: [...]

No, SPF is not feasible for integration into SpamAssassin. Some links: http://www.apache.org/foundation/docs/sender-id-position.html http://www.debian.org/News/2004/20040904

Microsoft's encumbered Sender ID and SPF are not the same.

http://www.imc.org/ietf-mxcomp/mail-archive/msg03884.html

This URL confirms that Sender ID will not go into Exim, but there is already SPF support. --

...

That will happen 2 weeks after pigs fly. I hear the government has funded a jet-pig initiative... Well, thats one way for the politicians to justify more pork in the budget. - Various in alt.folklore.computers

On Mon, Sep 06, 2004 at 07:19:01PM -0400, Mark Jeftovic wrote:

I'm not sure the people behind this concept (SPF, RMX, et al) ever intended it to be the FUSSP, but a lot of the ensuing enthusiasm built it up to that.

Consider that the people behind SPF made this statement (upon introducing it): "Spam as a technical problem is solved by SPF." If, therefore, there is an overabundance of enthusiasm for that concept, then it seems to be very clear where full responsibility for that rests.

I've *never* viewed SPF as an "antispam" methodology, but considered it an inevitable utility of the DNS system. Other methods are evolving to deal with spam, don't confuse them with what SPF is, which is essentially an authentication/identification framework that has the ability to mitigate one of the more popularly used spam obfuscation techniques.

I'll agree with you that it may mitigate one of the more popularly used spammer obfuscation techniques, but that particular technique is a minor problem (considering spam/abuse as a whole) and not all that worth solving -- since *other* spammer obfuscation techniques which SPF (and DomainKeys and SenderID et.al.) don't address are already available and being used. (Why aren't they used more? Spammers haven't needed to. But if pressed, they will. Rapidly.) The bigger problem isn't the spammer obfuscation technique: it's the backscatter from all the mail systems which bounce instead of reject, Bouncing was not all *that* unreasonable until we started to operate in an environment with massive SMTP forgery (from spam/viruses/etc.) -- several years ago. It's now much more desirable to reject whenever possible, saving everyone bandwidth/cycles/grief. I don't think I like the idea of wallpapering over this problem with SPf/DomainKeys/etc.: I think I'd rather see those mail systems fixed to deal with the environment they find themselves in. [ Especially because the other spammer obfuscation techniques I referred to are available, and will be used if and when SPF or DomainKeys or any of these are widely deployed. Thus, mail systems will *still* inhabit an environment of massive forgery and should be prepared to deal with it as best they can...where I think one approach to that is "don't make it any worse". ] Yeah, that may be a lot of work to complete -- although there are a myriad of simple techniques available to at least mitigate it, if not eliminate it entirely, and any relief would be welcome.

That spammers are publishing SPF records is in no way indicative of an inherent flaw in SPF's objectives or a failure in its implementation, in fact, I welcome spammers who publish SPF data detailing the originating points of their email. If more "known spam domains" did this, a handy DNSBL could be constructed out of such data (with a few caveats of course, it would also potentially open the door to a type of DoS attack).

RHSBLs (i.e. DNSBLs which list domain names instead of IP addresses, thus "Right-Hand-Side BL's") have already been built. See www.surbl.org and www.ahbl.org, for example. But this tactic doesn't work -- as an anti-spam technique -- as well as we might hope, for three reasons: 1. Spammers have an [effectively] infinite supply of domains. This won't change because spammers who burn through domains rapidly (and thus need to purchase more) are some of the registrars' best customers. They're also early adopters of "obfuscated registration", so much so that it's becoming increasingly likely that any domain thus registered is declaring "intent to abuse". [1] 2. Spammers control a large -- as in tens of millions -- number of zombies. [2] This won't change because none of the three entities who could do anything about it (the zombies' former owners, consumer broadband ISPs, Microsoft) are willing to step up, admit there's a problem, and do whatever it takes to fix it. 3. Mail from a forged sender is operationally indistinguishable from mail from an unforged but unknown sender. [3] This won't change either. And because of #1, spammers have an essentially infinite number of domains to do it with, and because of #2, they have a large number of systems to do it from. And as a result, a *lot* of things that we could try, not just SPF/DomainKeys/et.al., just won't work. (Example? Oh, take the various "hashcash" ideas that have been floated: getting into a computing cycle contest with spammers is a guaranteed loss.) ---Rsk [1] For example, one group of pirate-software spammers appears to be burning through domains at the rate of one every 24-48 hours, and has been doing this for months. [2] It's hard to know how many systems are zombies, but "tens of millions" is probably the right order of magnitude. I did a back-of-the-envelope calculation a few months and came up with 10 to 20 million; Carl Hutzler (of AOL Policy Enforcement) provided an estimate of 50-100 million on Spam-L. We went back and forth a little bit about this, and while I don't want to try to speak for Carl, I think we agreed that the true number is probably in the middle: so let's say, 40 million just so we have a number to argue about. [3] This may not be clear, so let me illustrate by example. Consider two incoming pieces of SMTP traffic, which claim to be from: frooblebonker@yahoo.com and joe@56trf5.com respectively. If I tell you that the first one is forged -- and that the putative sender doesn't exist (or isn't really the person sending the message) and that the second one is unforged (and is actually correct) -- did I just do you any good *with respect to stopping spam*? For the first one: sure. You reject it, 'cause it's forged. Heck, even if it's not spam, and it may not be, you may decide to reject it anyway: using the anti-SMTP-forgery technlogy of your choice: SPF, DomainKeys, it doesn't matter. For the second one: no. Because knowing that sender is unforged/real doesn't, in and of itself, do you any good *with respect to stopping spam*. You ALSO need a way to know that 56trf5.com is a spam source domain. And you don't know that. Yet. So suppose I tell you that, too: "56trf5.com is probably controlled by a well-known spammer and so you might want to block traffic from it." Did I just do any good? Not really. Because tomorrow you will get SMTP traffic from: sam@78jh89.com and none of the things that you learned today *in and of themselves* are going to help you figure out what to do with that. Oh, it's unforged: it really IS from 78jh89.com. You can even confirm it with your favorite anti-SMTP-forgery technology, if you like. So what? It doesn't do you any good until I also tell you that 78jh89.com is another spammer-controlled domain. Lather, rinse, repeat hundreds of thousands (or more) times.

On Wed, 8 Sep 2004 13:52:59 +0100 <Michael.Dillon@radianz.com> asked:

I see that 56trf5.com is a real domain. Does this mean that the domain name registries and DNS are now being polluted with piles of garbage entries in the same way that Google searches have been polluted with tons of pages full of nothing but search keywords and ads?

Yes. Hadn't you noticed? Statistically speaking there are now more domains with fake contact records than there are with genuine contact records, and certain registrars have been allowing new domains to be registered using contact addresses that have previously been proved to be bogus. -- Richard Cox

...

I see that 56trf5.com is a real domain. Does this mean that the domain name registries and DNS are now being polluted with piles of garbage entries in the same way that Google searches have been polluted with tons of pages full of nothing but search keywords and ads?

Yes - and there's a list of such domains that we track published as the ob.surbl.org zonefile (http://www.surbl.org for details)

the way i can prove that this methodology is only employed by a minority of very-smart MTA operators is: "that it's effective." if it were widely used, such that it affected global spam volume and thus spammer revenue, then the spammers would switch to different tricks. MAPS RBL was intended to be immune to this reflexive failure mode because it targetted address space, which was a scarcer commodity than domain names. i recommend against deployment of anti-spam methodologies whose only guaranteed effect is to force spammers to have to be smarter. (they will!) -- Paul Vixie

Jeff Kell wrote:

I'm getting attacks from:

159.226.x.x 202.x.x.x 203.x.x.x

These /8s are shared between a whole lot of different ISPs in different countries. Do the machines trying this typically look like botnets, or open proxies? Do you notice any other traffic (malicious or otherwise) from these IPs immediately before or after these telnet probes? srs

[ Two replies in one. Last point has operational content. ] On Wed, Sep 08, 2004 at 01:52:59PM +0100, Michael.Dillon@radianz.com wrote:

I see that 56trf5.com is a real domain. Does this mean that the domain name registries and DNS are now being polluted with piles of garbage entries in the same way that Google searches have been polluted with tons of pages full of nothing but search keywords and ads?

Absolutely. As one example out of thousands, there are at least 350 domains names of the form: aaefelb.info abbbafd.info acdfiaj.info aclbkcdc.info adkehgi.info aeamdgi.info that have been burned through by one currently-active group of spammers. Another group has about 16,700 domains (and counting) that I'm aware of. Note also the relationship betwen this proliferation, the zombies, and rapidly-updating DNS -- see below. On Wed, Sep 08, 2004 at 01:26:27PM -0500, Robert Bonomi wrote:

I _do_ think that it is _a_step_ 'in the right direction'. I'd *love* to see SPF-type data returned on rDNS queries -- that would practically put the zombie spam-sending machines out of business.

Not even close, I'm afraid. Yes, it would deal, to some extent, with direct-to-MX spam from them (*if* all the domain they were forging cooperated), but: 1. Nothing stops those zombies from sending out spam via the mail servers on the networks on which they're located. (And in the process, forging either the address of the former owner of the zombie or another user on the same network.) Before you say "but the network operators would detect and fix that" let me point out that zombie-generated spam has been epidemic for going on two years and many -- MANY --ISPs have yet to perform basic network triage that could mitigate much of this very quickly. It's reaching, I think, to expect that those same ISPs, who by now have grown quite comfortable sitting on their hands, would do anything about this. (I recently speculated n Spam-L that I was willing to bet that at least one such ISP would respond by plugging in more mail servers in order to alleviate the resulting congestion. Bruce Gingery promptly pointed out that this is a sucker bet: it's already happened.) 2A. Nothing stops those zombies from embedding spam payloads in ordinary messages sent by their [putative] users. Mail grandma? Spam grandma. 2B. Nothing stops those zombies from accepting spam payloads on port XXXX and writing it directly to disk in the place and format expected by the end user's mail client. No SMTP. No DNS. And with optional forged headers "proving" SPF/DomainKeys/etc. validity, just in case tools for checking those are in use. 3. Spammers have been using rapidly-updating DNS for quite some time in order to spread out their zombie-hosted web sites. With today's change they can now extend that up a level: nothing is stopping them from, say, registering 1000 domains, using 100,000 zombies to host copies of the content, and using rapidly-updating DNS to distribute the traffic (as well as making shutting it all down tedious). And as if that won't be enough fun (and here's the operational bit): 4. This is the point that I think a lot of us tend to overlook: arguably, SMTP spam from those zombies is the *least* of our problems. Those systems are under the control of an unknown number of unknown persons, and can be put to many more uses -- and already have. They've already been observed hosting spamvertised web sites [1], probing for open proxies, and participating in DDoS attacks. They represent an enormous computing resource that's effectively in the hands of The Bad Guys. (To put this in perspective, compare the estimated size of the zombie farm to the much-vaunted Google cluster in terms of CPU count, aggregate bandwidth, and network diversity.) And as I said previously, none of the three entities who could do anything about it (the zombies' former owners, consumer broadband ISPs, Microsoft) are willing to step up, admit there's a problem, and do whatever it takes to fix it. There is thus no reason at all to expect the problem to decrease; on the contrary, there is every reason (given the miserable track records of all concerned) to expect it to increase. ---Rsk [1] Including some with content of interest to the FTC, DEA, FBI, RIAA, MPAA, BSA, SPA and other people who have lawyers, guns and/or money. Makes sense from spammy's point of view: it's free, it's fault-tolerant and scalable (thanks to rapidly-updating DNS), and maybe someone else will get clobbered for it.