From nanog-bounces+bonomi=mail.r-bonomi.com@nanog.org Fri Feb 19 22:32:48 2010 From: William Herrin <bill@herrin.us> Date: Fri, 19 Feb 2010 23:32:10 -0500 Subject: Re: Spamhaus... To: Larry Sheldon <LarrySheldon@cox.net> Cc: nanog@nanog.org
On Fri, Feb 19, 2010 at 8:35 PM, Larry Sheldon <LarrySheldon@cox.net> wrote:
On 2/19/2010 7:20 PM, William Herrin wrote:
"If an SMTP server has accepted the task of relaying the mail and later finds that the destination is incorrect or that the mail cannot be delivered for some other reason, then it MUST construct an "undeliverable mail" notification message and send it to the originator of the undeliverable mail (as indicated by the reverse-path)."
Does the RFC say what to do if the reverse-path has been damaged and now points to somebody who had nothing what ever to do with the email?
Hi Larry,
Re-reading the paragraph I quoted and you repeated, I'm going to say that the answer is "yes."
I'll bite. *HOW* do you send to the _originator_ (as *required* by the RFC you quoted) of the undeliverable mail, when the reverse path points to 'someone else'? Note well the exact lanugage used -- it does not say 'the party named in the reverse path', the 'claimed sender', 'putative sender' or any other similar equivocation that justifies sending to a forged address. It says "the originator". To me, that can be only iterpreted in _one_ way. To wit: as the party that _actually_ created and transmitted the message, _regardless_ of what the declared return path is. Since such a message is 'defective' (not RFC-compliant -- because the true point -of-origin is *NOT* in the reverse path, as it MUST be for an RFC-compliant message) on it's face, I will argue that there is no need to apply the 'required' handling for a 'proper' message to it.
Does the RFC say what to do if the reverse-path has been damaged and now points to somebody who had nothing what ever to do with the email? Do the TCP RFCs say what to do in response to a SYN packet, if the source IP address has been damaged, and now points to some source IP that has "nothing whatever" to do with the packet?
You can only do the best possible -- discard the packet, if something allows you to determine the SYN is obviously spoofed or invalid, sure, drop it, otherwise you have no real choice. With SMTP there are many cases where you have no choice but to 250 the message, and send a DSN/bounce back later. E-mail users expect that when they send someone a message, it gets there in 6 hours or less, or they get an error within 6 hours. The purpose of SMTP protocol is to provide reliable mail delivery, which includes reporting errors. Spurious DSNs are less harmful than missing DSNs. Spurious DSNs can be discarded easily by the mail server that knows it didn't pass that message. DSNs that were not generated cannot be recovered. Discarding is currently the responsibility of the mail server whose address has been forged. Just like it's the responsibility of a host whose source address was forged in a TCP transaction, to discard the "ACK" packet for a connection that resulted from a spoofed SYN. The mail server sending DSN for the fake message, or replying to a spoofed SYN is not a spammer in any way, they are actually a victim wasting their own bandwidth responding to a bogus message. They have no real choice in the matter -- Weaknesses in SMTP protocol and lack of SPF implementation forced them into this position, they can't tell if the "MAIL FROM" line is real, anymore than a host on the internet can look at a source IP on a packet and know it's real or not. In the general case, Basic DNS and SPF tests are basically all the receiving mail server has to work with in "validating" return path. And they should perform these tests, before responding 250. When you responded with "250 Message accepted for delivery", that says you were satisfied that the message was legitimate, and the RETURN PATH and TO address is verifiable as far as you can tell. You can't NOT send DSNs if a failure occurs after that point, that is contrary to the requirements for reliable mail delivery. Rliable storage and delivery of legitimate messages is just as important as suppression of noise. Without reliable delivery, we don't really have a such thing as "internet mail" anymore. And by the way, a backup MX cannot verify the recipient when the primary MX is down. Especially when the backup MX is hosted off-site by another provider. The job of the backup is to hold mail in queue until the main mail system is back in operation, so "recipient verification" cannot actually be guaranteed. The perimeter MX also has no idea, that the recipient's mailbox has run out of disk space and cannot store the message, or that the alias points to a catch-all address on a different provider's mail server, where the user's account has been deleted. Some backscatter is to be expected as long as we have a reliable mail system, and it relies on returning messages via DSN to unverifiable MAIL FROM addresses. I only really see three options here.... give up on reliable mail delivery (might as well abandon SMTP entirely then, and replace it with a simpler protocol), revise SMTP to allow authentication of 'MAIL FROM', Or revise SMTP to require somehow validating the entire delivery path before "250 OK" can be accepted for any RCPT TO line. As in, eliminating the ability for mail servers to 'queue' messages for delivery. Instead when you issue 'RCPT TO', your mail server must immediately connect to the next hop mail server, start the SMTP transaction, and get an OK for that 'RCPT TO' prior to returning "Recipient OK" back to you. So you getting 250 OK to "RCPT TO" means every mail server in the delivery path simultaneously has a port 25 connection open to the next hop, has just returned 250 to the same RCPT TO line. -- -Mysid
I'm really amazed the thread police haven't pulled this one over and hauled it off to jail. The questions of when/whether/and to who bounces should be sent is a debate for spam-l or nanae. IMO, the original question in this thread was on-topic, but unfortunately it got very little discussion before things devolved into "why are you sending bounces?" and "I suppose you can't read the RFCs." The original question, "what do you do (or have you done) when DNSBL-X approaches you saying that your network is hitting their public NS's too hard and wants you to pay for continued access?" is something I'd like to see some answers to. Despite the Subject:, Spamhaus is neither the only DNSBL currently doing this nor the first to watch statistics on their public NS's and approach networks asking for money and/or cutting off access if you don't pay. Maybe you run a mail cluster that uses DNSBL-X. Maybe you haven't even heard of it, but you have enough customers using it, and querying through your caching DNS servers that your network has come up on their radar as a "heavy user". Telling your heavy user customers to stop using your DNS cache probably won't help. I know at least some of these orgs aggregate queries either per RIR assigned CIDR or per ASN, so spreading the queries out isn't likely to get you around the issue. So, do you pay, and setup your own local copy of the zones? Let them block your servers/network and let those of your customers who care make their own arrangements for continued access? ---------------------------------------------------------------------- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
it off to jail. The questions of when/whether/and to who bounces should be sent is a debate for spam-l or nanae. I don't know about that. Bounce handling is not a question of spam filtering. Spam or not is orthogonal to the issue of forged return path. There really should be nothing to debate, except in the context of a
On Sat, Feb 20, 2010 at 6:25 PM, Jon Lewis <jlewis@lewis.org> wrote: protocol discussion, as the current internet standards are pretty clear and specifically inflexible on how internet mail hosts must handle error conditions. Just like TCP rfc793 is clear on how internet mail hosts must handle connection establishment.
cache probably won't help. I know at least some of these orgs aggregate queries either per RIR assigned CIDR or per ASN, so spreading the queries out isn't likely to get you around the issue.
When contacted by the DNSBl, perhaps you inform the end-users to make DNSBL queries directly against DNSBL servers, directly from the mail server which is in their SWIP'ed IP range. There is little else you can do, isn't there?
So, do you pay, and setup your own local copy of the zones? Let them block your servers/network and let those of your customers who care make their own arrangements for continued access? That depends on the importance of the DNSBL.
Spamhaus' description of rsync datafeed service on their web site appears to be incompatible with an ISP setting up a local copy and allowing customers to query. When setting up a local copy of the zone, you pay by "Number of e-mail users". See the problem? As an ISP serving a local copy of the zone to customer mail servers, you don't know how many mailboxes they have. Maybe i'm mistaken, but it appears each end user has to buy the service for their own mail servers, and the ISP isn't allowed to bypass that. For the purpose of the agreements with spamhaus, an ISP customer is probably considered a third party, and making a rbldns server available to them is disclosing spamhaus' secret DNSBL zone information.... -- -J
Maybe I'm mistaken, but it appears each end user has to buy the service for their own mail servers, and the ISP isn't allowed to bypass that. For the purpose of the agreements with spamhaus, an ISP customer is probably considered a third party, and making a rbldns server available to them is disclosing spamhaus' secret DNSBL zone information....
In my experience, they're pretty reasonable. I would talk to them (or one of their datafeed sales agents) before assuming that they won't sell you the service you need. R's, John
On Sun, 2010-02-21 at 06:27 +0000, John Levine wrote:
In my experience, they're pretty reasonable. I would talk to them (or one of their datafeed sales agents) before assuming that they won't sell you the service you need.
They are indeed. In my day job, a large group of related members of different institutions approached our umbrella networking organisation to speak to Spamhaus for the specific reason that we were concerned that; a) between us we were making millions (if not billions) of queries a day to the mirror servers, and b) collective negotiation would make a service available for all of us for far less than individual orgs paying for their own. We now have a "private" mirror, which is accessible only from within the same AS in which we all sit. The load is therefore not on the Spamhaus servers or public mirrors, and we're collectively paying for the service so the service is supported. Everyone wins. Unfortunately (for this discussion) I don't know how much it cost, but I would assume it wasn't much because the lead time between request and service implementation was pretty short. Personally I think Spamhaus are entirely correct to identify and block, or request payment, from heavy users of their _free_ service. A little like the organisations paying many other members of this list will do for heavy data users in a residential or mobile context, in fact - but that's far too controversial an issue to be conflated with this one (oh dear). Graeme
Jon Lewis wrote:
The original question, "what do you do (or have you done) when DNSBL-X approaches you saying that your network is hitting their public NS's too hard and wants you to pay for continued access?" is something I'd like to see some answers to. Despite the Subject:, Spamhaus is neither the only DNSBL currently doing this nor the first to watch statistics on their public NS's and approach networks asking for money and/or cutting off access if you don't pay.
As a matter of interest, who are the other current DNSBL's to do it? Michelle
On Sun, 21 Feb 2010, Michelle Sullivan wrote:
As a matter of interest, who are the other current DNSBL's to do it?
To the best of my knowledge, MAPS was the first to do it. Uribl.com currently does it (and does the sort of query aggregation across your entire? network) that I mentioned. ---------------------------------------------------------------------- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
On Sun, 21 Feb 2010, Jon Lewis wrote:
On Sun, 21 Feb 2010, Michelle Sullivan wrote:
As a matter of interest, who are the other current DNSBL's to do it?
To the best of my knowledge, MAPS was the first to do it. Uribl.com currently does it
And SURBL.org. Tony. -- f.anthony.n.finch <dot@dotat.at> http://dotat.at/ GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS. MODERATE OR GOOD.
Jon Lewis wrote:
On Sun, 21 Feb 2010, Michelle Sullivan wrote:
As a matter of interest, who are the other current DNSBL's to do it?
To the best of my knowledge, MAPS was the first to do it. Uribl.com currently does it (and does the sort of query aggregation across your entire? network) that I mentioned.
Can you access MAPS without a subscription at all? As far as SORBS goes, we monitor the individual DNS servers for excessive queries and ask any provider causing excessive queries to run their own local copy. We don't charge for any of it, we don't require them to run a public mirror (though sometimes we ask.) Regards, Michelle
On Sun, 21 Feb 2010, Michelle Sullivan wrote:
To the best of my knowledge, MAPS was the first to do it. Uribl.com currently does it (and does the sort of query aggregation across your entire? network) that I mentioned.
Can you access MAPS without a subscription at all?
At this point, I have no idea. Originally, yes. Even after they "went commercial", IIRC, they were still going to provide free access for "hobbyists" but not for business users. The quality and coverage of their service compared to others that were available became such that I stopped using it and didn't miss it. ---------------------------------------------------------------------- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
To the best of my knowledge, MAPS was the first to do it. Uribl.com currently does it (and does the sort of query aggregation across your entire? network) that I mentioned.
Can you access MAPS without a subscription at all?
No. A low level subscription is pretty cheap, and I used to have one, paying $140/yr for under 1000 users, but I dropped it when I found that MAPS wasn't providing anything I couldn't get elsewhere. R's, John
On 2/21/2010 12:32 PM, Jon Lewis wrote:
On Sun, 21 Feb 2010, Michelle Sullivan wrote:
To the best of my knowledge, MAPS was the first to do it. Uribl.com currently does it (and does the sort of query aggregation across your entire? network) that I mentioned.
Can you access MAPS without a subscription at all?
At this point, I have no idea. Originally, yes. Even after they "went commercial", IIRC, they were still going to provide free access for "hobbyists" but not for business users. The quality and coverage of their service compared to others that were available became such that I stopped using it and didn't miss it.
I also have no current information (except personal surprise that they were still around), but I got into anti-spam groups and lists, and learning "sendmail" when our mail started failing left and right. Long story short somebody (HP, our vendor? the previous secretive "admin"? I never did figure out who) had configured all of our sendmail instances to use MAPS, and MAPS had shut off the service--no warning that I know of, no alternative that I know of. I do recall that when we started developing our own tools (in the pre-Postini days) our catch rate went up and our FP rate plummeted. -- "Government big enough to supply everything you need is big enough to take everything you have." Remember: The Ark was built by amateurs, the Titanic by professionals. Requiescas in pace o email Ex turpi causa non oritur actio Eppure si rinfresca ICBM Targeting Information: http://tinyurl.com/4sqczs http://tinyurl.com/7tp8ml
Am 21.02.10 10:25, schrieb Michelle Sullivan:
As a matter of interest, who are the other current DNSBL's to do it?
dnswl.org currently does not do it, but bandwidth suckers are a pain. The work is considerable: log aggregation, log review, trying to find a responsible for the IPs and following up until they finally implement a local copy. We losely define 100k queries/24h to be acceptable. Above that, you should set up your local (private) mirror (and hey, rsync is free!). And there are some entities that do not even acknowledge that a problem exists -- most likely until you turn access off for them. Yes, I can completely understand Spamhaus & Co for limiting access to their public mirrors. (OTOH, blocking access to these abusers is hard since our infrastructure partly relies on donated, shared public DNSBL mirrors.) -- Matthias
On 2/20/2010 4:57 PM, James Hess wrote: For the purpose of the following two paragraphs, pretend for the moment that you operate a business selling stuff via an email address Sales@Example.Com. For dramatic effect, assume your children will starve if you are not able to sell anything. Further, pretend that you have really annoyed somebody--a competitor, perhaps. Suppose that your competitor has contracted with a real, genuine spammer to send email mmessages advertizing some trash at a rate of tens of thousands per second until the bot-net gets shut down using Sales@Example.Com as the Return-Path. Now. Read the two paragraphs.
Spurious DSNs are less harmful than missing DSNs. Spurious DSNs can be discarded easily by the mail server that knows it didn't pass that message. DSNs that were not generated cannot be recovered.
Discarding is currently the responsibility of the mail server whose address has been forged. Just like it's the responsibility of a host whose source address was forged in a TCP transaction, to discard the "ACK" packet for a connection that resulted from a spoofed SYN.
Anything about those two 'graphs you would like to reconsider? And by the way, when I was running a network, if I got very many errant SYN's from a particular source, that source would get a static route to a 500 ohm resistor.
The mail server sending DSN for the fake message, or replying to a spoofed SYN is not a spammer in any way, they are actually a victim wasting their own bandwidth responding to a bogus message.
Victim they may be, spammer they are, The definition of "spammer" does not include a "get even with the world" or "do unto others as was done unto you" clauses. -- "Government big enough to supply everything you need is big enough to take everything you have." Remember: The Ark was built by amateurs, the Titanic by professionals. Requiescas in pace o email Ex turpi causa non oritur actio Eppure si rinfresca ICBM Targeting Information: http://tinyurl.com/4sqczs http://tinyurl.com/7tp8ml
For the purpose of the following two paragraphs, pretend for the moment that you operate a business selling stuff via an email address Sales@Example.Com. For dramatic effect, assume your children will starve if you are not able to sell anything.
Further, pretend that you have really annoyed somebody--a competitor, perhaps. Suppose that your competitor has contracted with a real, genuine spammer to send email mmessages advertizing some trash at a rate of tens of thousands per second until the bot-net gets shut down using Sales@Example.Com as the Return-Path.
Lest anyone think this is a hypothetical argument, for a while I had annoyed some eastern European spammer enough that I was getting 400,000 blowback bounces a day on a server that never sent more than 10,000 outbound messages/day. I was able to deal with it via some custom patches in my SMTP daemon, since the blowback had technical characteristics that made it possible to recognize efficiently, but it wasn't pretty. R's, John
participants (9)
-
Graeme Fowler -
James Hess -
John Levine -
Jon Lewis -
Larry Sheldon -
Matthias Leisi -
Michelle Sullivan -
Robert Bonomi -
Tony Finch