Criminals, The Network, and You [Was: Something Else]
Re-sending due to Merit's minor outage. - ferg ---------- Forwarded Message ---------- -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - -- Robert Blayzor <rblayzor@inoc.net> wrote:
The fact that they're rejecting on a 5xx error based on no DNS PTR is a=
bit harsh. While I'm all for requiring all hosts to have valid PTR records, there are times when transient or problem servers can cause a DNS lookup failure or miss, etc. If anything they should be returning a= 4xx to have the remote host"try again later".
Oh, wait till you realize that some of the HTTP returns are bogus altogether -- and actually still serve malware. It's pretty rampant right now. :-/ - - ferg -----BEGIN PGP SIGNATURE----- Version: PGP Desktop 9.6.2 (Build 2014) wj8DBQFGxR1lq1pz9mNUZTMRApQRAKCEOLpuu69A1+B4vCHQTZs+hHLKaACcD1Ak 9JNwl2i1mL08WNUQSlXBYGM=3D =3DffuN -----END PGP SIGNATURE----- -- "Fergie", a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/
My mail servers return 5xx on NXDOMAIN. If my little shop can spend not too much money for three-9s reliability in the DNS servers, other shops can as well. When I first deployed the system, the overwhelming majority of the rejects were from otherwise known spam locations (looking at Spamhaus, Spamcop, and a couple of other well-known DNSBLs). The number of false positives were so small that whitelisting was easy and simple to maintain. If a shop is not multihomed, they can contract with one or more DNS hosts to provide high-availability DNS, particularly for their in-addr.arpa zones. It's not hard. Nor expensive. Paul Ferguson wrote:
Re-sending due to Merit's minor outage.
- ferg
---------- Forwarded Message ----------
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
- -- Robert Blayzor <rblayzor@inoc.net> wrote:
The fact that they're rejecting on a 5xx error based on no DNS PTR is a=
bit harsh. While I'm all for requiring all hosts to have valid PTR records, there are times when transient or problem servers can cause a DNS lookup failure or miss, etc. If anything they should be returning a=
4xx to have the remote host"try again later".
Oh, wait till you realize that some of the HTTP returns are bogus altogether -- and actually still serve malware.
It's pretty rampant right now. :-/
- - ferg
-----BEGIN PGP SIGNATURE----- Version: PGP Desktop 9.6.2 (Build 2014)
wj8DBQFGxR1lq1pz9mNUZTMRApQRAKCEOLpuu69A1+B4vCHQTZs+hHLKaACcD1Ak 9JNwl2i1mL08WNUQSlXBYGM=3D =3DffuN -----END PGP SIGNATURE-----
-- "Fergie", a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/
Hi All, It seems to me reverse DNS just isn't an acceptable anti-spam measure. Too many broken reverses exist with smaller companies (try getting a 3rd party to fix it). It's not that hard for a bot to figure out a DSL's reverse entry and use that for its HELO. And there are a lot more effective pre-processing anti-spam measures, including greylisting (with its own problems) and reputation-based systems. Best Regards, Jason -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Stephen Satchell Sent: Wednesday, September 12, 2007 9:55 AM To: nanog@nanog.org Subject: Re: Criminals, The Network, and You [Was: Something Else] My mail servers return 5xx on NXDOMAIN. If my little shop can spend not too much money for three-9s reliability in the DNS servers, other shops can as well. When I first deployed the system, the overwhelming majority of the rejects were from otherwise known spam locations (looking at Spamhaus, Spamcop, and a couple of other well-known DNSBLs). The number of false positives were so small that whitelisting was easy and simple to maintain. If a shop is not multihomed, they can contract with one or more DNS hosts to provide high-availability DNS, particularly for their in-addr.arpa zones. It's not hard. Nor expensive. Paul Ferguson wrote:
Re-sending due to Merit's minor outage.
- ferg
---------- Forwarded Message ----------
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
- -- Robert Blayzor <rblayzor@inoc.net> wrote:
The fact that they're rejecting on a 5xx error based on no DNS PTR is a=
bit harsh. While I'm all for requiring all hosts to have valid PTR records, there are times when transient or problem servers can cause a DNS lookup failure or miss, etc. If anything they should be returning a=
4xx to have the remote host"try again later".
Oh, wait till you realize that some of the HTTP returns are bogus altogether -- and actually still serve malware.
It's pretty rampant right now. :-/
- - ferg
-----BEGIN PGP SIGNATURE----- Version: PGP Desktop 9.6.2 (Build 2014)
wj8DBQFGxR1lq1pz9mNUZTMRApQRAKCEOLpuu69A1+B4vCHQTZs+hHLKaACcD1Ak 9JNwl2i1mL08WNUQSlXBYGM=3D =3DffuN -----END PGP SIGNATURE-----
-- "Fergie", a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/
!SIG:46e80d6b62576097418713!
On Wed, Sep 12, 2007 at 10:13:00AM -0600, Jason J. W. Williams wrote:
Hi All,
It seems to me reverse DNS just isn't an acceptable anti-spam measure. Too many broken reverses exist with smaller companies (try getting a 3rd party to fix it). It's not that hard for a bot to figure out a DSL's
On this topic, I again solicit remarks from interested parties on the IETF dnsop reverse-mapping-considerations draft, which the editors currently think is baked. If you have remarks about it, now would be an excellent time to make them: http://tools.ietf.org/wg/dnsop/draft-ietf-dnsop-reverse-mapping-consideratio... Thanks! A -- Andrew Sullivan 204-4141 Yonge Street Afilias Canada Toronto, Ontario Canada <andrew@ca.afilias.info> M2P 2A8 jabber: ajsaf@jabber.org +1 416 646 3304 x4110
on Wed, Sep 12, 2007 at 10:13:00AM -0600, Jason J. W. Williams wrote:
It seems to me reverse DNS just isn't an acceptable anti-spam measure. Too many broken reverses exist with smaller companies (try getting a 3rd party to fix it). It's not that hard for a bot to figure out a DSL's reverse entry and use that for its HELO. And there are a lot more effective pre-processing anti-spam measures, including greylisting (with its own problems) and reputation-based systems.
Your first sentence and your third are in direct conflict, as are your first and your fourth - please understand that the use of rDNS (especially generic - as distinct from known dynamic or static) is an extremely effective tool against the botnets, and can itself be an extremely useful input into "reputation-based systems". As for your second sentence, well, you're right in saying that blocking solely on the perceived absence of, or generic nature of, rDNS naming is probably prone to false positives, but nonetheless it's not really my responsibility to ensure that you choose a decent service provider with the ability to provide proper and current and specific identification for your IP. If more ISPs dealt with abuse issues on their own networks, this wouldn't be such a big deal - but it's difficult for me to accept mail from, say, a host named 'dsl-static-pool.1.2.3.4.bigisp.example.net' when I've seen hundreds of thousands of abusive messages from hosts with that same naming convention, all bots. YMM, of course, V. As for the third, well, now you know why I use generic rDNS detection to defeat bots. As you say, "It's not that hard for a bot to figure out [any infected host]'s reverse entry and use that for its HELO". In fact, that's exactly what many of them do, when they're not forging well known services or sending unqualified/unresolvable strings in HELO/EHLO. And that, in itself, is responsible for over a fifth of our SMTP-time spam detections (and rejections, so there's no outscatter, unlike with a wide variety of "antispam" appliances, such as Barracudas). It's a sensible and sane perimeter defense tactic, far better than what I see most doing. If you're running a mail source, make damn sure it's got non-generic rDNS /and/ that it's configured to HELO with something that doesn't make it look like a "bot", and you'll stand a much better chance of delivering mail to me and my service's users. If you're not, well, the time is running short for you to fix that brokenness. Steve -- hesketh.com/inc. v: +1(919)834-2552 f: +1(919)834-2553 w: http://hesketh.com/ antispam news, solutions for sendmail, exim, postfix: http://enemieslist.com/
Hi Steve, In my opinion, the first and fourth statements are not necessarily in conflict. A reputation system based purely on reverses is pretty broken. Also, it is not necessary to use it as a factor in calculating a very reliable reputation. I'm having trouble seeing how the first and third are in conflict as well, but I may be indexing statements differently. Regarding the second, you're absolutely right. It's not your responsibility if a 3rd party doesn't have a rDNS entry (at all or non-generic), however the reality is you're going to have to deal with it anyway. If your customers allow you to tell the senders to buzz off and fix it, that's terrific. However, you're in a more authoritarian (IT-wise) environment than most I would suspect. Also, you risk hurting your customers. As an example, it's not a suitable answer to our law firm customers who are critically-dependent on receiving e-mail from hopelessly broken senders.
As for the third, well, now you know why I use generic rDNS detection to defeat bots. As you say, "It's not that hard for a bot to figure out [any infected host]'s reverse entry and use that for its HELO". In fact, that's exactly what many of them do, when they're not forging well known services or sending unqualified/unresolvable strings in HELO/EHLO. And that, in itself, is responsible for over a fifth of our SMTP-time spam detections (and rejections, so there's no outscatter, unlike with a wide variety of "antispam" appliances, such as Barracudas). It's a sensible and sane perimeter defense tactic, far better than what I see most doing.
It's not disputable that rejecting generic rDNS hits (or failures depending on your point of view) will gain you benefits. What I think is disputable, is the benefit to false positive ratio. About 60% of our botnet analyses show unqualified, or outright out-of-spec HELOs. One can catch the remaining 40% through correlation of certain SMTP factors with the results of content-analysis. Near real-time data mining of both informational inputs shouldn't be underplayed. Lastly, I fully agree one should reject as much as possible before the SMTP session ends. Whether or not rDNS is a good anti-spam measure for you entails a lot of factors. I posit from our own statistical analyses the benefit to pain ratio issue is not high enough. Particularly, when there so many other correlations you can run that have lower false positive rates. Best Regards, Jason
On Wed, 12 Sep 2007, Jason J. W. Williams wrote:
your customers. As an example, it's not a suitable answer to our law firm customers who are critically-dependent on receiving e-mail from hopelessly broken senders.
I've always wondered why the bad guys can't wrap postal packages correctly or spell. http://www.usps.com/postalinspectors/pos84.pdf On the other hand, if you get a package that looks like that, is it really worth taking the chance? Appearences can be important. Hopeless broken senders seem to figure out how to fix things when their critically important (sent via an unreliable e-mail) messages keep getting returned to sender.
On Wed, Sep 12, 2007 at 03:43:22PM -0600, Jason J. W. Williams wrote:
In my opinion, the first and fourth statements are not necessarily in conflict. A reputation system based purely on reverses is pretty broken.
Actually, it's amazingly reliable, when the patterns (literal string matches and regular expressions) are carefully compiled and cross-checked. One example of the thousands of patterns I block outright is *.fios.verizon.net. I do this (a) because I'm seeing lots of spam from generically-named hosts in that subdomain, e.g., within the last hour attempts have been noted from: relay=pool-71-164-205-139.dllstx.fios.verizon.net [71.164.205.139] relay=pool-71-170-104-162.dllstx.fios.verizon.net [71.170.104.162] relay=pool-71-170-70-195.dllstx.fios.verizon.net [71.170.70.195] relay=pool-71-172-239-30.nwrknj.fios.verizon.net [71.172.239.30] relay=pool-71-189-201-76.lsanca.fios.verizon.net [71.189.201.76] relay=pool-71-190-246-40.nycmny.fios.verizon.net [71.190.246.40] relay=pool-72-73-220-84.cmdnnj.fios.verizon.net [72.73.220.84] relay=pool-72-76-240-99.nwrknj.fios.verizon.net [72.76.240.99] relay=pool-72-84-85-199.nrflva.fios.verizon.net [72.84.85.199] relay=pool-72-89-97-187.bflony.fios.verizon.net [72.89.97.187] relay=static-71-183-52-18.nycmny.fios.verizon.net [71.183.52.18] (b) because I've yet to receive a single report of a false positive from this pattern after using it for a considerable period of time and (c) because Verizon seems disinclined to do anything about the problem other than have their paid professional spokesliars repeat the usual B.S., viz.: http://www.theregister.co.uk/2007/09/10/isps_ignore_strorm_worm_and_other_ma... ISPs turn blind eye to million-machine malware monster By Dan Goodin in San Francisco Published Monday 10th September 2007 06:02 GMT [...] Verizon turned down requests for an interview with a security engineer, but a spokeswoman said officials are aware of the rankings and are working to put new measures in place by the end of the year to curb the spam flowing out of its network. "We are concerned about it," the spokeswoman, Bobbi Hensen, said. "We don't like spam. We are aggressively working on that." [...] Uh-huh. This problem was reported to Verizon roughly FIVE YEARS ago, and should have, of course, been competently addressed withing a matter of days. It hasn't been. There is no sign that it will be. It's therefore been necessary for those of us enduring torrential quantities of Verizon-originated abuse to take appropriate defensive actions. Like rejecting all SMTP traffic from *.fios.verizon.net. And Verizon is merely one of the offenders -- the article cited above lists several others. I just happened to single them out for use as an example here because I found the contrast between their years-long history of utter negligence and their officially-stated position to be particularly striking. Comcast, Charter, SBCGlobal, Ameritech, Level3, SWBell, Nextgentel, Pacbell, and Qwest, just to name a few off the cuff, are equally culpable. Anyway: the use of generic rDNS patterns for outright rejection turns out to be quite effective with a very low FP rate.
Regarding the second, you're absolutely right. It's not your responsibility if a 3rd party doesn't have a rDNS entry (at all or non-generic), however the reality is you're going to have to deal with it anyway. If your customers allow you to tell the senders to buzz off and fix it, that's terrific. However, you're in a more authoritarian (IT-wise) environment than most I would suspect. Also, you risk hurting your customers. As an example, it's not a suitable answer to our law firm customers who are critically-dependent on receiving e-mail from hopelessly broken senders.
Any firm that is critically dependent on email (beyond an intranet environment) is being naive and foolish by relying on a known-unreliable communications medium. Connections fail. DNS breaks. Servers croak. Disks fill. Poor software is deployed. And the entire Internet-wide infrastructure for mail is under constant stress from spam, DoS attacks, misconfiguration, and outright stupidity (e.g. SAV). As to hopelessly-broken senders, we do not do them any favors by accomodating their brokeness. It is better in the long term for all of us to educate them about the not just the de jure, but the de facto minimum requirements for mail servers -- which in 2007 include not only functional rDNS, but rDNS pointing to non-generic hostnames. ---Rsk
On Tue, 18 Sep 2007, Rich Kulawiec wrote:
here because I found the contrast between their years-long history of utter negligence and their officially-stated position to be particularly striking. Comcast, Charter, SBCGlobal, Ameritech, Level3, SWBell, Nextgentel, Pacbell, and Qwest, just to name a few off the cuff, are equally culpable.
They all suck isn't very useful information. Although collectively they've probably fixed hundreds of thousands of customer computers over the years, like bad Boston drivers, there is always more. Instead of they suck, it might be more useful to highlight providers of similar scale which you think do a good job which others could emulate.
Anyway: the use of generic rDNS patterns for outright rejection turns out to be quite effective with a very low FP rate.
Some people think that users on dynamic addresses should be read-only, and not allowed to operate servers or send messages. Like most forms of red-lining, it tends to become self-fulling. Websites that only support Internet Explorer probably get very few false positives because people affected are used to working around that or just ignore them. Networks that don't update their Bogon lists probably get very few false positives because people learn to work around them or ignore them. And so on. Unlesss accept all those messages from those addresses and check them, you really don't know the false positive rate. You only know the self-reported complaint rate; which is usually a fraction of the actual false positive rate.
Instead of they suck, it might be more useful to highlight providers of similar scale which you think do a good job which others could emulate.
How about some smarter statistics. Instead of counting the spam emails from Network X, count the spam sources and divide that by the number of end user customers (or hosts) in Network X. By doing this you get a clearer picture of who is cleaning their house, and who is letting it slide. Think of a messy house. You say that there were 8 dirty plates in the living room, on the floor, the sofa the coffee table. Horrible, right? Not if there are 8 people living in the house. In that case it represents one evening of laziness, going to bed without cleaning up first. But if only one person lives in the house and there are no guests, then the 8 dirty plates represent a big mess. Whenever you scale up anything, small nits also grow in absolute magnitude. The small scale operator who ignores the nits is following the same practices as the large scale operator who ignores the nits. If there are lots of nits, I want to know if the large scale operator should be criticised for not adjusting their processes to deal with scaling up, or whether somebody really is being incompetent. There are different remedies to the two situations. Scaling issues can be solved by paying attention, education, installing tools/products/services. But incompetence generally requires replacing people, especially management who allow the incompetence to thrive.
Unlesss accept all those messages from those addresses and check them, you really don't know the false positive rate. You only know the self-reported complaint rate; which is usually a fraction of the actual false positive rate.
Yes. It is tempting to take numbers at their face value, but I find that whenever somebody has an axe to grind, their numbers are based on flawed reasoning or measuring the wrong things. --Michael Dillon
On Tue, Sep 18, 2007 at 02:42:43PM -0400, Sean Donelan wrote:
Instead of they suck, it might be more useful to highlight providers of similar scale which you think do a good job which others could emulate.
A first-order examination of the data suggests that Cox may be doing something pro-active. The number of IP addresses on their network engaged in recent spam delivery attempts is considerably less than that from others, e.g.: Comcast: 1240 Verizon: 1594 Cox: 25 Of course, that could be due to the relative sizes of the networks involved, but a second-order examination of the same data suggests something more: the abuse-emitting IP addresses on Cox's network do not appear to show up repeatedly over long periods of time. Contrast with those on Comcast and Verizon, where it's quite common to see the same ones in the logs for days/weeks/months. This suggests to me that Cox is actually paying attention to abuse outbound from their network and is either disconnecting or quarantining hosts which emit it.
Some people think that users on dynamic addresses should be read-only, and not allowed to operate servers or send messages. Like most forms of red-lining, it tends to become self-fulling.
Of course, I've never suggested any such thing. Users of such ISPs should normally be using their own ISPs outbound mail server(s), a solution which is perfectly adequate for the overwhelming majority of users. And it should be no problem for them to use other mail servers, using SMTP AUTH (with TLS or SSL) or via HTTP. But the days when direct-to-MX traffic is acceptable from all addresses are as gone as those when operation of an open SMTP relay was acceptable. Moreover, those who wish to operate servers (and whose ISPs permit that operation per their TOS) should have no difficulty in having the ISP assign them non-generic DNS/rDNS, and/or assign them static address space which is reserved for such users.
Unlesss accept all those messages from those addresses and check them, you really don't know the false positive rate. You only know the self-reported complaint rate; which is usually a fraction of the actual false positive rate.
Actually, I know quite a bit more than that. But since this is NANOG and not an anti-spam discussion list, I elected not to delve into the rather lengthy details of the methodology used to ascertain the FP rate. But *briefly* and glossing heavily, when a particular IP address (with generic rDNS, let's say): - fails to wait for the SMTP greeting - fails to send a QUIT - fails to retry when given a deferral response - retries immediately when given a deferral response - attempts delivery to an MX that hasn't been an MX for years - attempts delivery to a domain whose MX record hasn't been pointed to this MX for years - HELOs as ebay.com (or similar) - HELOs as a non-existent domain - HELOs as a very-recently-registered domain - HELOs as something different each time it connects - HELOs as *my* MX or a domain handle by it - attempts to send mail with a putative sender @paypal.com (or similar) - attempts delivery repeatedly with different putative senders/different putative sender domains - attempts to send mail with putative senders whose domain does not exist or does not resolve - attempts to send mail with a putative senders whose domain has A or MX records which resolve to invalid network space - attempts to send mail with a putative sender whose domain has very suspicious DNS records [1] - attempts to send mail from a putative sender using a known-spammer-owned domain - attempts to send mail from a putative sender using a domain which resolves to DROP-listed space. [2] - attempts to send mail from a putative sender using a very-recently-registered domain - attempts to send mail from a putative sender using *my* domain - attempts delivery to spamtraps - attempts delivery to never-existed addresses - attempts delivery to one-off addresses available only in SMTP reject notices - attempts delivery of messages whose headers contain known spamware signatures - attempts to send mail whose body-part contains URIs which match well-maintained lists of spammer-controlled URIs - has already been noted by other independent observers as attempting spam delivery to their MX's - passive-OS-fingerprints as running Windows - is listed in the A or NS records of a known spammer domain (see [1] again) - has been noticed carrying out other attacks, e.g., attempting exploitation via HTTP, SSH, etc. - and so on or multiple of the above (as is the case most of the time), then it's very, very unlikely that refusal of the traffic constitutes a FP. ---Rsk [1] A recent example: segron.com, queried yesterday (a query today will likely return different values, BTW): segron.com a 62.214.228.21 segron.com a 75.47.248.183 segron.com a 79.113.34.148 segron.com a 79.120.16.19 segron.com a 80.133.250.133 segron.com a 81.198.35.132 segron.com a 85.179.60.176 segron.com a 89.110.23.181 segron.com a 89.178.60.48 segron.com a 89.212.0.195 segron.com a 121.137.164.24 segron.com a 121.200.140.244 segron.com a 218.254.186.186 segron.com a 221.126.244.121 segron.com a 221.127.158.128 which resolve to: i3ED6E415.versanet.de adsl-75-47-248-183.dsl.lsan03.sbcglobal.net 79-113-34-148.rdsnet.ro p5085FA85.dip.t-dialin.net e179060176.adsl.alicedsl.de pppoe-181.23.110.89-adsl.spbnit.ru 89-178-60-48.broadband.corbina.ru 89-212-0-195.dynamic.dsl.t-2.net 244.140.200.121.megaegg.ne.jp cm218-254-186-186.hkcable.com.hk (5 addresses yield NXDOMAIN) Clearly, segron.com's "web site" is being hosted on hijacked systems. [2] See http://www.spamhaus.org/DROP/ for details, but the gist is that this short and very carefully maintained list enumerates network space which is either hijacked or 100% spammer-controlled or both.
On Wed, 19 Sep 2007, Rich Kulawiec wrote:
in the logs for days/weeks/months. This suggests to me that Cox is actually paying attention to abuse outbound from their network and is either disconnecting or quarantining hosts which emit it.
Its nice to see Cox getting some praise for a change. Last month people were castigating it for being too agressive at trying to block Bots. It seems like half the net is always criticizing ISPs for doing too little and half the net is always criticizing ISPs for doing too much. Cox blocks a lot of ports on its network (25, 80, 135-139, 445, 1900, 1433, 1434, 1900, subseven ports); blackholes networks and DNS names; firewall software that blocked sites with bad TCP software stacks such as Craigslist; and so on. Some people think Cox is being pro-active on the security front; other people think Cox is violating a sacred trust. ISPs are pretty much just damned. Why should an network user have to petition his or her ISP to authorize their use of a valid network protocol? Shouldn't application authorization occur at the application level instead of relying on the equivalent of .rlogin network-level checks. Companies like DynDNS show there is user demand to operate their own servers (including P2P servers, mail servers, web servers, dns servers, etc) on dynamic IP addresses without needing a special "static" IP address or different in-addr.arpa name. With Fast-Flux, it looks like the next network port that should be blocked on broadband/dialup connections is DNS tcp/udp 53.
or multiple of the above (as is the case most of the time), then it's very, very unlikely that refusal of the traffic constitutes a FP.
Until a false positive happens. I see 1-2 false positives a year using checks for "generic-looking" in-addr.arpa names; and a few more false positives for IP addresses without in-addr.arpa names. Nevertheless I still continue to use those checks because the false positive rate is below my pain threshold. But I don't pretend it never happens or may not be a concern to someone else. I also almost never get a valid e-mail to my postmaster account, just spam; but some people still think every mail server should accept mail to the postmaster account anyway no matter how rarely it gets legitimate email. They even set up RBLs of mail servers without postmaster accounts. Maybe we need a RBL of mail servers that don't accept mail from generic in-addr.arpa or dynamic IP addresses.
On Thu, Sep 20, 2007 at 01:31:41PM -0400, Sean Donelan wrote:
Why should an network user have to petition his or her ISP to authorize their use of a valid network protocol?
Because many (most?) ISPs have done such a poor job of controlling SMTP abuse outbound from their networks over the past decade that it's now a best practice to consider all mail from generic hostnames/dynamic IP space highly suspect -- at best. Those ISPs have repeatedly proven over many years that they aren't capable of detecting and squelching SMTP abuse sources on their own networks; [1] this leaves everyone else with a choice: either (a) put up with it or (b) devise measures to stuff a sock in it. And (a) simply isn't tenable for mail servers receiving abuse in torrential quantities. If any of those ISPs are unhappy with the choice of tactics encompassed by (b) then perhaps they should have anticipated that unhappiness years ago when they were first alerted to this problem. Had they taken even rudimentary steps to solve it (instead of merely having their spokesdroids repeat the bare-faced lie that they "take the spam problem seriously") then perhaps it would not have been necessary for others to devise methods to deal with their failures. If any network user is unhappy (and I can easily see why they would be), then he or she should take that up with their ISP, since it's quite likely that their own ISP has been a contributor to the problem.
Companies like DynDNS show there is user demand to operate their own servers (including P2P servers, mail servers, web servers, dns servers, etc) on dynamic IP addresses without needing a special "static" IP address or different in-addr.arpa name.
That model is no longer viable, unfortunately. I wish that weren't the case, but the combination of ISP and end-user negligence along with mass hijacking of end-user systems has rendered it so.
They even set up RBLs of mail servers without postmaster accounts. Maybe we need a RBL of mail servers that don't accept mail from generic in-addr.arpa or dynamic IP addresses.
You are certainly free to set up a DNSBL or RHSBL using any listing criteria you wish, but please be aware that if you set up one using that particular criteria, anyone using it will likely be refusing a LOT of valid mail, including that of some very large organizations, since (as I said above) blocking such traffic has long since been established as a best practice. There are multiple DNSBLs, RHSBLs, and static lists which enumerate such hosts; for example, consider the Spamhaus PBL: http://www.spamhaus.org/pbl/index.lasso which relies in part on input from the ISPs themselves, and is one of the zones included in the comprehensive "zen" DNSBL zone published by Spamhaus. ---Rsk [1] I still adhere to the quaint/outdated/antique concept that everyone is responsible for making sure that their networks are not an operational hazard to everyone else's networks, and that they should plan, budget, staff, build, operate and train accordingly.
My mail servers return 5xx on NXDOMAIN. If my little shop can spend not too much money for three-9s reliability in the DNS servers, other shops can as well. When I first deployed the system, the overwhelming majority of the rejects were from otherwise known spam locations (looking at Spamhaus, Spamcop, and a couple of other well-known DNSBLs). The number of false positives were so small that whitelisting was easy and simple to maintain.
If a shop is not multihomed, they can contract with one or more DNS hosts to provide high-availability DNS, particularly for their in-addr.arpa zones.
It's not hard. Nor expensive.
Well, if by "3 9's" you mean "99.9%", and that's acceptable to you, then fine. Otherwise, your self-measured uptime of your DNS servers is not that relevant, as the real question is what is the availability of your DNS servers as measured from whoever might be doing a lookup on your domain (or, more specifically, from whatever random mail server happens to be doing a domain lookup of your domain). I would be skeptical that it is easy for any organization to build a nameserver system that can actually reach 99.999% availability from random points on the Internet. Contracting to an outsourcer is no guarantee, as we've seen large-scale DDoS attacks against some of these. Outsourcers are actually riskier, since a DDoS against the nameservers of any of their customers is essentially a DDoS against your nameservers. Some combination of outsourced plus diverse self-managed servers probably lands you there, but it is neither easy nor without expense to make arrangements like this. Given the level of clue required to get truly rock solid DNS, it may be better to 4XX NXDOMAIN. Most spambots don't seem to retry on a 4XX anyways, so to a spambot, the 4XX *is* a 5XX, but to a real mail client, the 4XX is a 4XX, and that seems like it would be a more resilient choice. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net "We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
On Wednesday 12 September 2007 16:54, you wrote:
My mail servers return 5xx on NXDOMAIN. If my little shop can spend not too much money for three-9s reliability in the DNS servers, other shops can as well.
You get NXDOMAIN when an authoratitive servers says there is no such domain, it doesn't occur if the DNS servers aren't available. So I fail to see the connection to reliability of DNS servers. All well engineers mail services provide 4xx (or accept the email) on SERVFAIL (or other lookup failure), if they insist on checking DNS information as part of accepting email. One has to allow for the case where the mail servers can't speak to the DNS servers, which may include cases where the DNS servers are available, but say routing, or other parts of the DNS are fubar. Serious programmer(s?) spent a lot of time making sure the MTA we use does the right thing under all error conditions so far encountered, I'd consider altering that behaviour vandalism. I feel like some sort of clumsy cave man compared to the authors every time I configure it as it is.
participants (10)
-
Andrew Sullivan -
Jason J. W. Williams -
Joe Greco -
michael.dillon＠bt.com -
Paul Ferguson -
Rich Kulawiec -
Sean Donelan -
Simon Waters -
Stephen Satchell -
Steven Champeon