Daniel,

...

Short of turning off their network access, why won't users fix their computers when the computer is infected or needs a patch?

Hey, it's working! If it ain't broken ....

I doubt this. Recently, I worked with a couple of people that each had their PCs infected. Their own virtual neighborhood complained to them, and they surely were embaressed about the situation, but... They just did not know how to fix it, i.e. where to start. Call it cluelessness, call it lack of education.

Related question for network engineers: When did you have your last medical check-up? To what extent do you follow your physician's recommendations?

__ Erik-Jan.

Daniel Karrenberg wrote:

There is that too; but I have frequently observed people not doing it even when provided detailed step-by-step instructions. On the other hand they would proceed relatively quickly once "it stopped working", e.g. the Internet plug was pulled. Some of them would use the instructions provided, others would get help; but not before "it stopped owrking".

Indeed. It seems to be a motivation problem. "Also, using the net registering system we posted a virus alert and made information available," said Cunningham. "Most people probably skipped through it though." Obviously, this is by no means specific to computer patching. People are either "busy", lazy, apathetic, etc. Most don't pay attention until they're forced to; i.e., when their system stops working because a virus broke it or because their network access is shut off. You can ask nicely or post warnings a billion times to no avail. Human nature, perhaps. -Terry

At 8:02 PM -0400 10/3/03, Terry Baranski wrote:

Obviously, this is by no means specific to computer patching. People are either "busy", lazy, apathetic, etc. Most don't pay attention until

I've played the user-notification game myself in fighting hoaxes (do a search on wormalert@somewhere.com sometime--and consider what happens when tens of thousands of people add it to their address book and then forward the latest joke/hoax/virus to everyone in their address book). I used to send auto-replies debunking the hoax--but then they'd report them as spam to their ISP, and their ISP would block my domain. Others would just delete them. Often the only way to get their attention was to send mail to everyone they'd cc'd, and ask *them* to contact the offender. There is no question that people don't understand their computers. It's all magic to them. The idea that the energizer bunny will appear on their screen when they send mail to five friends is no less likely than the idea that dropping a file on their email icon will bring up a compose window. But in fairness to the users, this isn't all their fault. They've been told right and left not to open mail from strangers (a completely bogus concept, given that viruses tend to come from friends). What I found was that they take that quite literally. Mail from mailer-daemon (now there's a scary name), mail from postmaster, mail from anybody they don't personally know; gets deleted. And that includes mail from their ISP. They can't tell spam from purchase receipts from viruses from fake warnings from legitimate warnings. Consider the latest "microsoft patch" virus. That was a professional looking job. Do you really expect the user to know not to open that, but to know that the notification from their ISP about their machine being infected is legit? They either need to be contacted out of band, or their email software needs to support a secure channel of communications that they can really trust. -- Kee Hinckley http://www.messagefire.com/ Next Generation Spam Defense http://commons.somewhere.com/buzz/ Writings on Technology and Society I'm not sure which upsets me more: that people are so unwilling to accept responsibility for their own actions, or that they are so eager to regulate everyone else's.

At 2:11 AM +0000 10/5/03, Suresh Ramasubramanian wrote:

For more fun, consider that you are postmaster@somewhere.com, and get those

It's the anti-virus ones that drive me nuts. "Someone in your domain sent us a virus which always forges the from line, but we're going to tell you anyway because we'd like you to buy our software..."

Reply to that and you will, as likely as not, get your reply sent back to you and your upstreams as a spam complaint.

When I moved somewhere.com to a new ISP, the very first thing I did was contact the abuse desk there and warn them what to expect. That was helpful when Universal Studios tried to come after me because someone at somewhere.com (literally :-) had posted a stolen movie on usenet. (Only one?)

on that cc list for the same reason that you are - Outlook Express being set up to add all people that you reply to, to your address book.

...

been told right and left not to open mail from strangers (a completely bogus concept, given that viruses tend to come from friends). What I found was that they take that quite literally.

Say what? I have received virii from people I don't know from Adam, from countries where I don't know anyone at all.

Those of us who post widely get that. But your average "just use email to talk to friends and family" is more likely to get it from friends--unless of course they forwarded a joke to everyone in their address book, who forwarded it....

...

They either need to be contacted out of band, or their email software needs to support a secure channel of communications that they can really trust.

Hotmail, for example, clearly marks mail from hotmail staff (service announcements etc) with a different colored text in the inbox ... I guess if you control the client your user uses (using a custom built web interface is one way, a customized browser / mail client is another way) ...

But other than that, you could well ask for the moon.

Bringing this back to the more relevant topic. Is there something that ISPs could do to notify users and get in their face more without shutting off their connection? Perhaps a custom piece of notification software that only took signed messages, and made some attempt to keep its bits secure? Unfortunately I don't see much way to keep it from being subverted without OS support. If it became common enough, then the virus writers would just simulate messages from it and disable the real one. -- Kee Hinckley http://www.messagefire.com/ Next Generation Spam Defense http://commons.somewhere.com/buzz/ Writings on Technology and Society I'm not sure which upsets me more: that people are so unwilling to accept responsibility for their own actions, or that they are so eager to regulate everyone else's.

On Sun, 5 Oct 2003, Suresh Ramasubramanian wrote:

Kee Hinckley [05/10/03 00:57 -0400]:

...

Bringing this back to the more relevant topic. Is there something that ISPs could do to notify users and get in their face more without shutting off their connection? Perhaps a custom piece of

I have seen corporate and university networks that make every PC have PC Anywhere or its equivalent as part of the standard install, for activity to be monitored.

There are some differences between private networks and public networks. In a company, the company is the "owner" of the PCs and employees (in the US) have little expectation of privacy using company computers. On the public network, generally the customer owns the computer not the ISP. How far should an ISP go monitoring the activities of their customers? ISPs can and do notify customers by many methods such as popups, email, mail, phone calls, knocking on the door, etc. Notification doesn't seem to be the problem, but of the customer taking action. And even if the customer is willing, its difficult for them to tell if they have actually fixed their computers. Windows XP System Restore and anti-virus programs don't get along well. Booting Windows in "Safe Mode" requires dexterity. Most people don't have sniffers to check what their computers are transmitting. Sometimes it takes a non-expert several attempts to completely fix things. So from an ISPs point of view, is there a way for the ISP to quickly tell the customer if the particular computer is fixed without unduly intruding on the privacy of the customer? With home networks, there may be multiple computers behind a NAT/router/firewall. So a simple network scan doesn't always work.

On Sun, 5 Oct 2003, Suresh Ramasubramanian wrote:

...

So from an ISPs point of view, is there a way for the ISP to quickly tell the customer if the particular computer is fixed without unduly

Isolate his IP and have all outbound http redirected to a page that says "please call [escalated tech support number]" to get this fixed.

Seems to be the only reasonably foolproof way.

I think you missed the point. The problem isn't notification. Customer calls the escalated tech support number is swears the problem is fixed. Should the tech support person just take the customer's word that the problem is fixed and turn their connection back on? What happens a few hours later when you start getting complaints again about the same customer? Do you turn the connection off again. And then the customer again swears they have the problem fixed. How many times do you repeat the process? Other than taking the customer's word, is their any way for the ISP to verify the customer has fixed their computer before turning the connection on again?

Suresh Ramasubramanian wrote:

Sean Donelan [05/10/03 17:44 -0400]:

...

What happens a few hours later when you start getting complaints again about the same customer? Do you turn the connection off again. And

Sure, turn it off again. And again.

Sooner or later, it will dawn on the customer that no, his system is not fixed. And in the meantime, both his bandwidth quota (if any) and the ISP's pipes avoid getting saturated with worms.

We have a better way - first time they get turned off. Second time they get turned off and told if it happens again you will be told to get service elsewhere. Third time their account is deleted. I am yet to have one that has reached the third time - 85k users here. / Mat

On Mon, 06 Oct 2003 02:43:48 -0000, Suresh Ramasubramanian said:

Matthew Sullivan [06/10/03 11:38 +1000]:

...

Third time their account is deleted.

I am yet to have one that has reached the third time - 85k users here.

Let me guess - that'd mostly be dialup users, right? Or maybe simply email users? Not (say) T1 and larger users?

If it is mostly dialup users, it's all the more remarkable, as "conventional wisdom" has home users as being even less security-clued than the SOHO crowd or corporate sites...

The difference being campus machines are null routed rather than disconnected, and they are not reconnected until checked and clean.

And once again, the question: how do you know the machines have been checked and cleaned before they are reconnected? Do you take the customers word, or do you perform some other check yourself?

Network security is high priority here and it doesn't matter what machine is compromised, they are all disconnected in one way or another, and yet we still have to nuke machines occasionally because of suspicious (DDoS/scanning etc) traffic.

Seems like a re-active policy. Why don't you check the computers before they start exhibiting suspicious behavior, such as when they are first connected to the network? Waiting until after the computer is compromised is too late. Some companies require all new computers to pass a network scan (e.g. ISS, Nessus, Retina, etc) before getting assigned a routable address. Should commercial service providers have the same policy when new customers connect to the network? Or is it considered a bad thing to warn customers about vulnerabilities in their computers in advance. Instead waiting until after your receive a complaint about something exploiting those vulnerabilities before taking action?

At 12:57 AM 10/5/2003, you wrote:

At 2:11 AM +0000 10/5/03, Suresh Ramasubramanian wrote:

...

For more fun, consider that you are postmaster@somewhere.com, and get those

It's the anti-virus ones that drive me nuts. "Someone in your domain sent us a virus which always forges the from line, but we're going to tell you anyway because we'd like you to buy our software..."

What gets me is the moron admins who track down every "attack" they see. "Attacks" such as ICMP echo requests, Port 80 connections, etc. If they get huge logs that's one thing, but for four pings from a windows box or a mistyped IP address in a URL and they are worried about our "attack" These bogus reports outnumber legitimate complaints 4:1. -Robert Tellurian Networks - The Ultimate Internet Connection http://www.tellurian.com | 888-TELLURIAN | 973-300-9211 "Good will, like a good name, is got by many actions, and lost by one." - Francis Jeffrey

On Mon, 6 Oct 2003 Valdis.Kletnieks@vt.edu wrote:

My favorite:

"ntp-1.vt.edu is portscanning me very slowly with source port 123...."

The really sad ones are the ones who 3 days earlier dropped me a note to tell me they'll using our NTP server.....

Due to the propensity of people to configure NTP in various annoying ways, and then failing to respond to requests to fix their systems, the NTP developers added a new command packet to the protocol. To help reduce the level of spurious network traffic due to obsolete configuration files, a special control message called the kiss-o'-death packet has been implemented. If enabled and a packet is denied service or exceeds the client limits, a compliant server will send this message to the client. A compliant client will cease further transmission and send a message to the system log. See the Authentication Options page for further information. Should other protocols include the same feature? If someone sends you a Dynamic DNS update, could the protocol include a kiss-o'-death packet to tell clients to go away? If someone keeps probing your HTTP server, should HTTP include a kiss-o'-death packet to tell clients to go away? If someone connects to your SMTP server, should SMTP include a kiss-o'-death packet to tell clients to go away. Or is this such a widely needed feature, should we try to include it in a base protocol such as IP/ICMP? In a large number of these cases, its not malicious, its misconfigured. A host could respond to a connection with an ICMP Go Away packet. Even if the end-host doesn't comply, an edge device or firewall using ICMP snooping could detect the Go Away packet and change its configuration. By pushing the control out to the edges, we avoid overloading the core with one-to-one configuration changes. Using a four-way handshake, its possible to protect against most types of spoofing and denial of service even without encryption or keys distribution. If you think the root servers are attacking you, Ok. Send a Go Away packet, and you won't get any more packets from the root server. Why would you do this? I don't know, but that's your choice. It would get the ISP out of the business of deciding what should be considered network abuse, and what isn't. After the user stops shooting himself in the foot (or runs out of toes), maybe he'll fix the broken auto-firewall software which thinks everyone is attacking him. At NANOG in Chicago, if anyone would like to discuss it further let me know.

On Mon, 06 Oct 2003 02:11:22 EDT, Sean Donelan said:

to the client. A compliant client will cease further transmission and send a message to the system log. See the Authentication Options page for further information.

ntp-2.vt.edu used to be an alias for my workstation, until it got moved to a more production machine. Two years later, there was still a flux of 50 packets/ second from machines that thought that even though it had been unreachable for 2 years(*), maybe THIS time it would answer. (when the xntpd exploit came out a few years ago, we turned on logging on our border routers - inside of an hour we had trapped packets from some 6-8 hosts that were heading to an IP address that hadn't been an NTP server for over 8 years). The only reason this number is notable is because even when it was a production server, the packet flux was only 100-150 packets/second. So obviously, we can't trust users to get it right. The problem with a 'kiss-o-death' packet is that it needs to be authenticated. Otherwise, you can use spoofed packets to DoS somebody. How many lines are in your root-DNS hints? And even if we insist on the KoD packet having the query ID in it, that's a TINY address space. I can even feed you spam to force you to hit the DNS, trickle you some forged KoD packets, and within a day or so make you refuse to talk to any of the root nameservers... (Note that TCP connections are a lot more easily dealt with, as the 3-packet handshake adds a lot to the security. However, Wesel's numbers on "98% of the root DNS traffic is bogus" indicate that we really need this on the UDP side of the fence as well....) It's the same basic reason why the UCITA provisions for remote deactivation of software went over like a lead balloon..... (*) I originally Did The Right Thing and had ICMP Port Unreachables going back, but some lameware Windows set-your-clock program interpreted those as "Ask again and maybe it will answer", so it would ask about 50 times a seconds.. continuously (oddly enough, it *didnt* retransmit if it got NO answer). The 5th or 6th time some bozo installed this program in a lab of 40-80 machines, I gave up and filtered all responses.

Sean Donelan wrote:

Should other protocols include the same feature? If someone sends you a Dynamic DNS update, could the protocol include a kiss-o'-death packet to tell clients to go away? If someone keeps probing your HTTP server, should HTTP include a kiss-o'-death packet to tell clients to go away?

Erm, I can see a huge DoS hole waiting to happen to any protocol that doesn't in turn implement some sort of authentication of the server. The more protocols you allow to do this, the more potential for DoS of important (possibly) client information. Peter

Sean Donelan wrote:

Uhm, you are also aware that if the attacker can spoof the kiss-o'-death packets; the same attacker could spoof all sorts of other packets including the time protocol packets to change the clock on your computer.

"Yes but"... there is a strong likelyhood that less paranoid protocol implementors (not necessarily designers, just those coding stuff from spec) could simplify their lives and not check all the right conditions required to filter unwanted stuff. Bye bye farm. Oh, this has happened already ? Now, where is that "Windows Update" icon again ... Peter

E.B. Dreger wrote:

HTTP implementations have had vulnerabilities due to insufficient checking. Thus HTTP is a bad idea.

SMTP implementations have had vulnerabilities due to insufficient checking. Thus SMTP is a bad idea.

SNMP implementations have had vulnerabilities due to insufficient checking. Thus SNMP is a bad idea.

Come to think of it, IP stacks have had vulnerabilities due to insufficient checking. IP is a bad idea, too.

No, please do not twist my words; I referrred to poor implementations of good ideas. Nowhere did I say that the protocol is bad as a result of poor implementations. Peter

Robert Boyle [10/6/2003 9:42 AM] :

What gets me is the moron admins who track down every "attack" they see. "Attacks" such as ICMP echo requests, Port 80 connections, etc. If they get huge logs that's one thing, but for four pings from a windows box or a mistyped IP address in a URL and they are worried about our "attack" These bogus reports outnumber legitimate complaints 4:1.

99% of them autogenerated by "personal firewall products". That include *screenshots* of "attack reports". Those can be safely auto-trashed, 99.999% of them are completely bogus stuff like "your DNS server is hacking me!!!!" srs -- Suresh Ramasubramanian <suresh@outblaze.com> gpg# EDEDEFB9 Security and Antispam Operations Manager, Outblaze Limited

Sean Donelan wrote:

On Fri, 3 Oct 2003, Erik-Jan Bos wrote:

...

I doubt this. Recently, I worked with a couple of people that each had their PCs infected. Their own virtual neighborhood complained to them, and they surely were embaressed about the situation, but... They just did not know how to fix it, i.e. where to start. Call it cluelessness, call it lack of education.

Newspapers have published How-To instructions. In the US, even USA Today published How-To instructions. The USA Today newspaper is known as McPaper for a reason. ISPs sent out step-by-step directions, complete with pictures and screen shots. In addition to full-page newspaper ads Microsoft has an easy 3-steps to protect your computer.

I have not seen much information on this in Dutch newspapers, but perhaps I am not reading the right papers. I surely think that news papers worldwide should publish on this.

Ok, not everyone is a computer expert. If their TV, VCR or car started belching smoke and flames, and they didn't know how to fix it, what would they do? Take it to a repair shop? If you get a flat tire, pull off to the side of the road and either repair the tire or call the auto club for help. You don't continue drive down the highway on the tire rims hoping the noise and sparks will just go away.

Perhaps an "auto club" for PC-users: You call and within the next 24 or 48 hours, depending on your subscription, an expert would dial in or come by to get you on the virtual road again. __ Erik-Jan.

Sean,

Ok, not everyone is a computer expert. If their TV, VCR or car started belching smoke and flames, and they didn't know how to fix it, what would they do? Take it to a repair shop? If you get a flat tire, pull off to the side of the road and either repair the tire or call the auto club for help. You don't continue drive down the highway on the tire rims hoping the noise and sparks will just go away.

You've put your finger on it. ISPs have to help users understand that their machines are broken in a way that makes them unable to gain access to the Internet -- then most will take them to the shop PDQ, and hopefully get them back with some protection installed. Recently my ISP, Time-Warner Roadrunner sent me a letter (in the mail!) informing me that portscans were coming from my cable modem, and asking me to respond to them within 48 hours to tell them what action I had taken. I took care of it, and complimented rr.mn.com on their service in telling me about the problem. I don't know what RR's next step would have been had I not acted, but I hope they would have suspended my service promptly. That may seem harsh to some users, but they have to realize it when their machines are broken in a way that may not be obvious to them as users, just as, in some states, people are forced by law to spend real money to clean up auto emissions. The resulting widespread outrage might eventually result in better computer software. Over the last 30 years or so, new-car reliability has improved dramatically for a similar reason. My opinion only, not my employer's. -- John Renwick

Speaking on Deep Background, the Press Secretary whispered:

...

Short of turning off their network access, why won't users fix their computers when the computer is infected or needs a patch?

Hey, it's working! If it ain't broken ....

And when you DO patch it, then it REALLY breaks. And your paper is due. Now what does the average Art History major do? I have to wonder how big the support $$ would be at Farber College if they really offered the handholding Flounder and Neidermeyer need? Would it cost as much as the bandwidth? -- A host is a host from coast to coast.................wb8foz@nrk.com & no one will talk to a host that's close........[v].(301) 56-LINUX Unless the host (that isn't close).........................pob 1433 is busy, hung or dead....................................20915-1433

At 3:26 PM -1000 10/9/03, Michael Painter wrote:

http://www.wired.com/news/digiwood/0,1412,60613,00.html

"When students first register on the network, they are required to read about peer-to-peer networks and certify that they will not share copyright files. Icarus then scans their computer, detects any worms, viruses or programs that act as a server, such as Kazaa. Students are then given instructions on how to disable offending programs."

Kinda' does some of what you want done? <s>

Icarus. Just sort of scares me that some students might use a hair dryer on his wings.