ken,

---snip--- 3) There was a thread a little while ago that talked about a way to cut down spam by simply restricting who you would accept SMTP traffic from. Unfortunately, I don't recall the details, but at the time it struck me as eminently sensible, and just required cooperation between ISPs to implement effectively. ---snip---

so what you are saying is that you would like to go back to the fidonet days, when site A had to agree to route site B's mail? a deny all, accept some rule for smtp would horribly break all that is good in humanity. am i missing something here? paul

One significant contributing factor to the lack of care or clue by mid and large size ISPs is the level 1 helldesk. I do not intend to insult anybody who is doing level 1 support, but you are not going to find people with serious network engineering expertise for $12/hour (or when outsourcing tech support for $5/hour to India). Far too many layer 1 people have to deal with clueless users who call in saying "Your mail server is haxxxing my firewall!". How do you seperate the legitimate abuse complaints from the chaff? That said, if somebody has a fast connection, hand-holding them through the process of using Windows Update by phone isn't terribly difficult. I think one of the smartest things a DSL/Cable ISP could do is negotiate a bulk license purchase with an anti-virus software vendor such as Kaspersky (makers of AVP), which can provide licenses for as little as $10 each in bulk. Is $10 per customer per year too much to pay for comprehensive auto-updating virus-scanning of client PCs?

1) Summarily fencing/sandboxing/disconnecting clients sending high volumes of spam, virii, etc. You might politely contact your commercial/static clients first, but anyone connecting a "bare" PC on a broadband circuit is too stupid to deserve coddling. The great majority of your clients would thank you profusely.

So far as I can see, detection of serious abusers should pretty straightforward. It wouldn't require any pretense at spam or virus filtering, per se; just pick off the clients that are flagrant sources of the plague of the month.

On Sun, 26 Oct 2003 kenw@kmsi.net wrote:

A few things that make sense to me (as a non-ISP network consultant) include:

Most ISPs are relatively secure. Yes, occasionally a backbone router shows up on some list with a password of "cisco." The major problems are in the systems managed and installed on non-ISP networks (i.e. end-users).

1) Summarily fencing/sandboxing/disconnecting clients sending high volumes of spam, virii, etc. You might politely contact your commercial/static clients first, but anyone connecting a "bare" PC on a broadband circuit is too stupid to deserve coddling. The great majority of your clients would thank you profusely.

Really? Most users are angry when their network connection is interrupted for any reason, including their own mistakes. Read some of the articles in the university newspapers when students were cut of from the network after not fixing their computers. How many people thank police officers when they are stopped for speeding, reckless driving, drunk driving, burned out taillights, etc? Or instead how many say things like the police should be out catching "real" criminals (i.e. anyone other than them)? As a non-ISP consultant, when a client asks you to configure their Exchange server do you always conduct a top-to-bottom security analysis of the client's entire business infrastructure and refuse to do business with them until after they have corrected every deficiency? Or does the client just say screw you, and hires a different consultant that will do what the client wants?

2) Notwithstanding the above, would it really be so hard to trap network packets bearing clear signatures of the "plague of the month"? Sure, it would create an extra load on routers or require special filtering hardware, but wouldn't it be worth it? Again, no need to be comprehensive; just blast the ones that are easy pickings.

Routers (especially high end routers) are barely stable just routing packets. Some high-end line cards don't support even a simple 2-line access control list. With the market currently heading to the lowest price possible, increasing costs doesn't appear to pay even for the niche markets that are interested. Instead the extra equipment is installed where there are clients willing to pay for it. The easiest, most effective place to catch packets is at the edge/end-users. An end-user firewall is $0-$50. Anywhere in the core is very difficult. What is the cost of a single OC192 firewall? Look at the post office, it doesn't try to find Anthrax in most of the mail. Instead a few locations at the edge of the postal system, e.g. the White House, Congress, etc, have added security precautions. The rest of the mail just flys through the system.

3) There was a thread a little while ago that talked about a way to cut down spam by simply restricting who you would accept SMTP traffic from. Unfortunately, I don't recall the details, but at the time it struck me as eminently sensible, and just required cooperation between ISPs to implement effectively.

Again, look the postal mail system. One proposal required everyone mail letters in person at the post office, and show id to the postal clerk. The problem is it really doesn't solve the problem. Third-party trust systems don't scale well beyone one or two degrees of separation. And there is only one major postal system. But it doesn't require cooperation from the ISP to accept mail from only people you know. You can do that today. The question is why don't more people do it? The ISP doesn't know who you know. Should ISPs require you to register your friends & family in order to receive mail? I don't know if it has come to that. And we all know how effective Caller-ID has been in cutting down telemarketing phone calls at dinner time. And the related caller-id blocking, and block caller-id blocking, and block block caller-id blocking, etc.

By the way, can anybody explain to me a legitimate use for port 135/137 traffic across the Internet, like it's somebody's private LAN? Seems to me anybody who still thinks that's legitimate is living in the past.

Bits on the wire using ports 135/137 are not intrinsically less safe than any other bits. And vendors have shown a willingness to add ways around port filters in the network, not by developing more secure protocols but by developing ways to send the same packets between insecure systems on other ports. Sendmail and BIND have more CERT/CC advisories than any other application, including NETBIOS. How many people are suggesting blocking port 53 and port 25?

So, the big question: why don't ISPs do more of this? Are they afraid of client reaction? Doesn't wash, for me: most clients would be highly grateful, and all it really takes for the remainder is fair warning. Cost? Again, you can judge for yourselves how low the fruit you choose to pick; the biggest gains have the best ROI.

Happy clients, liberated bandwidth, faster servers -- what's to loose?

Angry clients, increased bandwidth costs, slower servers doing more checks? ISPs are doing a lot to protect end-users. Some examples include Education campaigns Free anti-virus software Free personal firewall software Port filters (port 80 anyone?) Notification of compromised systems Incident Response Intrusion Detection/Intrusion Prevention Managed Security Services Unfortunately some of the argument is a bit like the old cries for public payphone companies were responsible for the drug dealers in poor neighborhoods. So they removed public payphones. The drug dealing problem wasn't solved.

Please bear in mind that much of this might be my take on viability, practicality, or past activity related to some of these suggestions. Moreover, this may not represent even my own opinions on the appropriate course of action. Inline... On Sun, Oct 26, 2003 at 06:01:09PM -0700, kenw@kmsi.net said something to the effect of:

..snip snip..>

1) Summarily fencing/sandboxing/disconnecting clients sending high volumes of spam, virii, etc. You might politely contact your commercial/static clients first, but anyone connecting a "bare" PC on a broadband circuit is too stupid to deserve coddling.

Unfortunately, you have described the standard home user. Would love to think that most have trusty firewalls guarding meticulously-patched (non-Windows) boxen, but... I expect our customers to watch their own backs, heed our admonitions, and go so far to hope that they find the appropriate clues to be responsible, but I can't imagine someone like Cox or CW or AOL or somesuch other alienating a prevailing chunk of their customer base with one nasty-gram followed by clobbering a switch port.

The great majority of your clients would thank you profusely.

I beg to differ on the last line. The happy ones would say nothing. And we may not be able to hear them if they did over the yowling from the unhappy ones who do not understand why we would do such a thing. I may be wrong, but it seems to me that most of us (service providers, that is) have some degree of a more involved customer notification process that must be followed before we can begin swinging the ax like that. Contractual obligation may often preclude a provider's harsh response to abusive or patently irresponsible network activity more often than failure to notice or give a flying fsck that it has transpired.

2) Notwithstanding the above, would it really be so hard to trap network packets bearing clear signatures of the "plague of the month"?

<not_sarcasm> Trap and do what with? </not_sarcasm>

Sure, it would create an extra load on routers or require special filtering hardware, but wouldn't it be worth it? Again, no need to be comprehensive; just blast the ones that are easy pickings.

Any other ISP that toyed with/deployed the filtering NetBIOS and/or 92-byte ICMP packets will remember how grossly unpopular we became for doing so. Accusations by customers and downstream engineers alike branded us fascist iconoclasts with no consideration for the aphoristic hands that feed us, and complete disregard for the needs of those who pay us for the evil service we restrict providing. Search the convocations (circa 08/2003) on this list even to get a feel for the dissent among the ranks regarding the feasibility and wired morality of such an action. Another trickier problem is fueled by the nature of the sploit-du-jour. If some lamer opts to poke surreptitious holes or generate static by way of a random, obscure, generally ignored port, the choice to filter is not as difficult a one, beyond *poof* conjuring up resources to reign in the resulting management nightmare. However, whether it be because of the inherently pregnable code or a malware creator's awareness of the impracticality (read: near impossibility) of recon and restricition of certain types of traffic, many of these evasive maneuvers can ride the clown car across critical service ports. Forget crippling a network by filtering them, which is frequently the case; we wind up in some seriously hot water for violating customer contracts, which forbid selective inhibition of legit traffic along with the anomolous. It becomes a dichotomy of casualties of war vs. curing the disease by killing the patient. (fwiw...i break more than my fair share of eggs...) To balance and consider both factions, providers are being tasked with stepping up to the plate and scrutinizing large streams of traffic to discern between the good and the bad based on other criteria. So now we're faced with other issues that require extremely intuitive and invasive packet inspection. Some of this is vaporware and is still skating around conference tables of pitch men all over the place, others are gaining credibility and becoming technologies many providers are striving toward actualizing. Intrusion and extrusion detection are fabulous...nay...will set you free...and ultimately necessary things, but some of what I'm seeing suggested here borders on impossible to put to work without absurd overhead or flirting with serious invasion of privacy. For the record, I am by no means disagreeing with the logic of the suggestion, but am merely playing experienced devil's advocate among those who may still be sporting the collective bruise from past actions of this sort.

3) There was a thread a little while ago that talked about a way to cut down spam by simply restricting who you would accept SMTP traffic from. Unfortunately, I don't recall the details, but at the time it struck me as eminently sensible, and just required cooperation between ISPs to implement effectively.

There have also been numerous threads from a little while ago flaming AOL and the like for deploying whitelist-type or no-auth-no-pass countermeasures for spam. Again, I don't know what the right answer here is, but I can see that, like every other coin, there are two diametrically opposed sides and teams going to bat over this issue. Perhaps the fault, dear Brutus, in this particular case continues to lie more inarguably and blatantly than ever with the end users and in smtp itself, which may honestly be dead. Clearly the protocol was not meant to withstand the rigors and abuse (no pun intended) to which it is nowadays being subjected. Its design over 20 years ago was intended to service a kinder, gentler, more honest, and less devious Internet. Perhaps, if we're looking to do the responsible thing and undo spam damage (and other similar ilks), we should belly up to the bar and work to bolster the protocols and technologies. That may be all the more hand we as technologists can have in the matter.

One problem for the average ISP would be the monitoring and updating of plague control infrastructure. It would probably be a lot easier with a bit of cooperation and sharing -- either that, or someone could get rich offering services to ISPs for a fee.

Average ISP, as in, what size? Tier 1? 2? 3? Managed service provider?

By the way, can anybody explain to me a legitimate use for port 135/137 traffic across the Internet, like it's somebody's private LAN? Seems to me anybody who still thinks that's legitimate is living in the past.

That would be Microsoft. NetBIOS has no business on the Internet. End of story.

So, the big question: why don't ISPs do more of this? Are they afraid of client reaction? Doesn't wash, for me: most clients would be highly grateful, and all it really takes for the remainder is fair warning. Cost?

Fair warning gives spammers time to migrate addresses and infected users time to infect countless others. And the customers will not be unusually happy; they have a right to expect exemplary service, as they are promised and pay for it. I don't imagine that most of our subscribers are phoning in to confirm and praise receipt of five-9s.

Again, you can judge for yourselves how low the fruit you choose to pick; the biggest gains have the best ROI.

One last time...I agree with you that ISPs need to keep an eye on their brood. I'm simply offering gentle explanations of why, perhaps, providers haven't been able to more neatly and elegantly (and swiftly) mitigate the burdens you note with the suggestions you raise. With that, regarding your last point... Be careful what you wish for. Let's do it! But...this will take everyone's participation to work. I can see it now...sleazier ISPs will sell 135-139 access wide open on the black market by way of some crazy out-of-the-way transit hole... ;)

Happy clients, liberated bandwidth, faster servers -- what's to loose?

Show me the first and I'll be much quicker to follow. I'd settle for the tooth fairy sometimes, as I'm not sure those happy clients really exist... ;) Best of luck to you. Let me know if you find the answer? I'll put cycles behind a worthy cause anyday... ymmv, --ra -- K. Rachael Treu, CISSP rara@navigo.com ..this blurb has been brought to you by the letters 'v' and 'i'.. -- I am an employee of, but do not necessarily represent herein, Global Crossing, Ltd. --