/24 blocking by ISPs - Re: Problems sending mail to yahoo?
On Fri, Apr 11, 2008 at 1:22 AM, Raymond L. Corbin <rcorbin@hostmysite.com> wrote:
Yeah, but without them saying which IP's are causing the problems you can't really tell which servers in a datacenter are forwarding their spam/abusing Yahoo. Once the /24 block is in place then they claim to have no way of knowing who actually caused the block on the /24. The feedback loop would help depending on your network size.
Almost every large ISP does that kind of "complimentary upgrade" There are enough networks around, like he.net, Yipes, PCCW Global / Cais etc, that host huge amounts of "snowshoe" spammers - http://www.spamhaus.org/faq/answers.lasso?section=Glossary#233 (you know, randomly named / named after a pattern domains, with anonymous whois or probably a PO box / UPS store in the whois contact, DNS served by the usual suspects like Moniker..) a /27 or /26 in a /24 might generate enough spam to drown the volume of legitimate email from the rest of the /24, and that would cause this kind of /24 block In some cases, such as 63.217/16 on CAIS / PCCW, there is NOTHING except spam coming from several /24s (and there's a /20 and a /21 out of it in spamhaus), and practically zero traffic from the rest of the /16. Or there's Cogent with a similar infestation spread around 38.106/16 ISPs with virtual hosting farms full of hacked cgi/php scripts, forwarders etc just dont trigger /24 blocks at the rate that ISPs hosting snowshoe spammers do. /24 blocks are simply a kind of motivation for large colo farms to try choosing between hosting spammers and hosting legitimate customers. srs ..
It's not unusual to do /24 blocks, however Yahoo claims they do not keep any logs as to what causes the /24 block. If they kept logs and were able to tell us which IP address in the /24 sent abuse to their network we would then be able to investigate it. Their stance of 'it's coming from your network you should know' isn't really helpful in solving the problem. When an IP is blocked a lot of ISP's can tell you why. I would think when they block a /24 they would atleast be able to decipher who was sending the abuse to their network to cause the block and not simply say 'Were sorry our anti-spam measures do not conform with your business practices'. Logging into every server using a /24 is looking for needle in a haystack. -Ray ________________________________________ From: Suresh Ramasubramanian [ops.lists@gmail.com] Sent: Thursday, April 10, 2008 11:56 PM To: Raymond L. Corbin Cc: Chris Stone; nanog@merit.edu Subject: /24 blocking by ISPs - Re: Problems sending mail to yahoo? On Fri, Apr 11, 2008 at 1:22 AM, Raymond L. Corbin <rcorbin@hostmysite.com> wrote:
Yeah, but without them saying which IP's are causing the problems you can't really tell which servers in a datacenter are forwarding their spam/abusing Yahoo. Once the /24 block is in place then they claim to have no way of knowing who actually caused the block on the /24. The feedback loop would help depending on your network size.
Almost every large ISP does that kind of "complimentary upgrade" There are enough networks around, like he.net, Yipes, PCCW Global / Cais etc, that host huge amounts of "snowshoe" spammers - http://www.spamhaus.org/faq/answers.lasso?section=Glossary#233 (you know, randomly named / named after a pattern domains, with anonymous whois or probably a PO box / UPS store in the whois contact, DNS served by the usual suspects like Moniker..) a /27 or /26 in a /24 might generate enough spam to drown the volume of legitimate email from the rest of the /24, and that would cause this kind of /24 block In some cases, such as 63.217/16 on CAIS / PCCW, there is NOTHING except spam coming from several /24s (and there's a /20 and a /21 out of it in spamhaus), and practically zero traffic from the rest of the /16. Or there's Cogent with a similar infestation spread around 38.106/16 ISPs with virtual hosting farms full of hacked cgi/php scripts, forwarders etc just dont trigger /24 blocks at the rate that ISPs hosting snowshoe spammers do. /24 blocks are simply a kind of motivation for large colo farms to try choosing between hosting spammers and hosting legitimate customers. srs ..
On Fri, Apr 11, 2008 at 8:37 PM, Raymond L. Corbin <rcorbin@hostmysite.com> wrote:
It's not unusual to do /24 blocks, however Yahoo claims they do not keep any logs as to what causes the /24
We keep quite detailed logs. No comment about yahoo - I've never been at the other end of a /24 block from them srs
On 4/11/08, Raymond L. Corbin <rcorbin@hostmysite.com> wrote:
It's not unusual to do /24 blocks, however Yahoo claims they do not keep any logs as to what causes the /24 block. If they kept logs and were able to tell us which IP address in the /24 sent abuse to their network we would then be able to investigate it. Their stance of 'it's coming from your network you should know' isn't really helpful in solving the problem. When an IP is blocked a lot of ISP's can tell you why. I would think when they block a /24 they would atleast be able to decipher who was sending the abuse to their network to cause the block and not simply say 'Were sorry our anti-spam measures do not conform with your business practices'. Logging into every server using a /24 is looking for needle in a haystack.
*heh* And yet just last year, Yahoo was loudly dennounced for keeping logs that allowed the Chinese government to imprison political dissidents. Talk about damned if you do, damned if don't... I guess logs should only be kept as long as they can only be used for good, and not evil? Matt
-Ray
On Sat, Apr 12, 2008 at 09:36:43AM -0700, Matthew Petach wrote:
*heh* And yet just last year, Yahoo was loudly dennounced for keeping logs that allowed the Chinese government to imprison political dissidents. Talk about damned if you do, damned if don't...
But those are very different kinds of logs -- with personally identifiable information. I see a sharp difference between those and logs which record (let's say) SMTP abuse incidents/attempts by originating IP address. ---Rsk
(all opinions below my own... comments are intended to address a number of points made previously in this extended thread, by rick and others) are you saying you don't consider the sending ip address or the envelope sender or the envelope recipient to be a. useful for spam detection b. personally identifiable information having done quite a lot of spam filtering (and having worked on big mail before, e.g. on the original AOL internet gateways) i think they are in both categories. (the HELO strings can be pretty useful also)... the scale of mail at yahoo, gmail, hotmail, aol (maybe brightmail and postini, too) is well beyond the numbers anyone else here is citing. i can assure you there are lots of smart and caring people working on problems of mail abuse (both incoming from the internet and outgoing, too). both of these cost us a lot of money, and we know it. yahoo receives > 500M visitors per month, and collects about 25 TB of logs every day. analyze that! my understanding is the chinese govt has specific requirements regarding logging and log retention that are compulsory for any company with servers in china. europe and other countries are trying to promulgate laws about log retention. logs cut both ways, by the way. they can be exculpatory as well, particularly in the case of a phished or cracked account used for something illegal. with the ip addresses of the abuse, the defense can assert that the account owner was not whodunit. with no logs, it's much harder to substantially defend against the govt in such cases, presumption of innocence notwithstanding. on the original issue (as i work for yahoo, but in the security group, not in mail), we *do* try to follow the lists, at least as lurkers. as a big and public company, somewhat in the spotlight from time to time, we are restricted from making statements that could be misinterpreted as "speaking for the company" without going through various approval channels. i summarized the substantive bits of this thread for yahoo mail management for their comments, and particularly seconding the suggestion that yahoo provide more transparency to isps to make it possible for them to clean/keep clean their own houses. there is dialog going on about improving the process so it's more predictable and less frustrating for ISPs. the forms really do work, they tell me. (not fast enough for you, we hear clearly.) (i just hope more transparency doesn't make things easier for, say, the Russian Business Network or the Storm gang.) on the question of greylisting, you're right that there are delays imposed on senders of email who are perceived as spam senders but "first connect fails" greylisting is not used. the documentation could be improved. (all documentation, except guy steele's or mary claire van leunen's, could be improved.) unfortunately, we're all pretty much in the same boat on this one, so let's not fight about it (at least, don't fight with me...) On Apr 12, 2008, at 7:08 PM, Rich Kulawiec wrote:
On Sat, Apr 12, 2008 at 09:36:43AM -0700, Matthew Petach wrote:
*heh* And yet just last year, Yahoo was loudly dennounced for keeping logs that allowed the Chinese government to imprison political dissidents. Talk about damned if you do, damned if don't...
But those are very different kinds of logs -- with personally identifiable information. I see a sharp difference between those and logs which record (let's say) SMTP abuse incidents/attempts by originating IP address.
---Rsk
participants (5)
-
mark seiden-via mac -
Matthew Petach -
Raymond L. Corbin -
Rich Kulawiec -
Suresh Ramasubramanian