FBI tells the public to call their ISP for help
If your call center volumes go up today... The fine people at the FBI are recommending people call their ISP for home computer technical support, even though most ISPs don't sell home computers, operating system software or application software. http://www.fbi.gov/page2/june07/botnet061307.htm First, if you believe your computer has been compromised, do not call the FBI directly. You should contact your Internet service provider. They can help you determine if your computer has been infected, and what steps to take to restore it. We are not in a position to provide technical assistance. BTW, 1 million compromised computers is probably a low estimate.
On Jun 13, 2007, at 11:49 AM, Sean Donelan wrote:
BTW, 1 million compromised computers is probably a low estimate.
Besides the 'call your ISP for technical help' blunder, there's actually more useful info, believe it or not, in the press release linked in the article: <http://www.fbi.gov/pressrel/pressrel07/botnet061307.htm> The FBI aren't claiming only 1 million infected machines, they're saying that this particular sweep involves up to a million botted hosts. It seems to me that the larger inference is that law enforcement are taking the botnet problem more seriously, which is what a lot of folks in the operational community have been advocating for a long time. While one aspect of the messaging is questionable, it seems to me that active national-level LEO involvement in this problem-space would be welcomed by many. It's just a first step, and those are always the hardest to take. ---------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // 408.527.6376 voice Equo ne credite, Teucri. -- Laocoön
On Wed, 13 Jun 2007, Roland Dobbins wrote:
It seems to me that the larger inference is that law enforcement are taking the botnet problem more seriously, which is what a lot of folks in the operational community have been advocating for a long time. While one aspect of the messaging is questionable, it seems to me that active national-level LEO involvement in this problem-space would be welcomed by many.
Its great to see FBI agents and the DOJ taking more interest in the problem of Bots and computer intrusions. I'm especially happy to see some arrests. The focus on home-grown bad guys was also good, instead of pointing the finger at some random other country. There are more than enough bad guys in more than enough countries to go around. If US law enforcement makes any progress at home, each other country can work on their native bad guys. Unfortunately, most FBI agents probably have about as much control over the FBI press office as most ISP security engineers have over their marketing departments. While FBI agents may be working with ISP security engineers, I suspect the FBI press office didn't bothered to vet or coordinate its press release with ISPs before issuing it. We've all cringed at one time or another at what our respective marketing teams come up with.
The fine people at the FBI are recommending people call their ISP for home computer technical support, even though most ISPs don't sell home computers, operating system software or application software.
No, the ISPs merely sell the channel through which the home computers get infected with worms and viruses, and through which the home computers vomit the reults of those infections. The biggest reason that we are making no progress against zombies is that everyone wants them to be someone else's problem. I entirely agree that the prime responsibility lies with the computer vendors and particularly with operating system vendors, such as one near Seattle, that year after year ship easily compromised software. Whenever someone tries to ask me a Windows question, I tell them to call Microsoft and demand they support the software they sell. But ISPs are not wholly without responsibility. If one of your customers reloaded Windows from CD and then needed to download all of the patches, do you provide a way for them to do it without getting re-wormed before the download is done? R's, John
On Wed, 14 Jun 2007, John Levine wrote:
But ISPs are not wholly without responsibility. If one of your customers reloaded Windows from CD and then needed to download all of the patches, do you provide a way for them to do it without getting re-wormed before the download is done?
Windows patches and updates are copyrighted intellectual property of Microsoft, and can not be re-distributed without written permission of Microsoft. Microsoft currently does not have an authorized way for general public ISPs to redistribute Microsoft updates except by a connection to the Internet. Institutional licenses, such as available for universities, are not licensed to ISPs. Since many Microsoft patches are only legally available via the Internet, and an ISP can not predict which servers Microsoft will use to distribute Microsoft patches, ISPs must enable essentially full Internet access which includes access for most worms. I have been down this road several times with Microsoft legal. And if an ISP wants to obey the law, there isn't a good answer for ISPs. If the ISP says to hell with the law, there are several technical options for redistributing Microsoft updates. If Microsoft changed its licensing policies for ISPs, there are several technical options for redistributing Microsoft updates.
Sean Donelan wrote: <snip>
Since many Microsoft patches are only legally available via the Internet, and an ISP can not predict which servers Microsoft will use to distribute Microsoft patches, ISPs must enable essentially full Internet access which includes access for most worms.
<snip> May I recommend developing an in house method for allowing the customer only access to your servers (web, dns, proxy, etc), and then apply filters for everything else except for tcp/80. If you wanted to be additionally paranoid, you could even allow only established tcp/80 connections back to the customer. Once updated, customer could establish contact to have filters removed, or an automated web process you be created. It's a ton of work, and there are any number of ways to do it. A lot depends on your network. It can be done, though. Jack
On Thu, 14 Jun 2007, Jack Bates wrote:
May I recommend developing an in house method for allowing the customer only access to your servers (web, dns, proxy, etc), and then apply filters for everything else except for tcp/80. If you wanted to be additionally paranoid, you could even allow only established tcp/80 connections back to the customer.
Once updated, customer could establish contact to have filters removed, or an automated web process you be created.
It's a ton of work, and there are any number of ways to do it. A lot depends on your network. It can be done, though.
I went down that road several times, and there are many issues with what you have described which won't work for how Microsoft distributes its updates and patches; and with the user. Microsoft has enabled Windows with enough features, users can infect their machine with only TCP/80. Please review the archives for details from several years ago, and at some point you will end up needing to violate the written Microsoft licenses. Its not a technical problem (although engineers seem to like to think everything is), its a legal issue with Microsoft's lawyer and licenses.
Wouldn't it be more appropriate if the FBI told people the phone number to Micr0$0ft? Owen
On Jun 14, 2007, at 8:03 AM, Jim Popovitch wrote:
On Thu, 2007-06-14 at 07:41 -0700, Owen DeLong wrote:
Wouldn't it be more appropriate if the FBI told people the phone number to Micr0$0ft?
It would lighten the load on the ISPs, but would it really achieve anything worthwhile?
-Jim P.
It would place more of the burden on the source of the problem. Hopefully, it would also allow Micr0$0ft to assist the people unfortunate enough to have purchased their products overcome the problems created by that decision, but, that might be asking too much. Owen
* Owen DeLong:
Wouldn't it be more appropriate if the FBI told people the phone number to Micr0$0ft?
No; most of them haven't got any existing contractual relationship with Microsoft. If you alluded to Microsoft's support lines: well, why should pay $90 (or what the single-call cost is) to fix a computer which is apparently working fine?
On Jun 15, 2007, at 12:42 PM, Florian Weimer wrote:
* Owen DeLong:
Wouldn't it be more appropriate if the FBI told people the phone number to Micr0$0ft?
No; most of them haven't got any existing contractual relationship with Microsoft.
If you alluded to Microsoft's support lines: well, why should pay $90 (or what the single-call cost is) to fix a computer which is apparently working fine?
Wrong... Most of them are subject to the problems they have because of their contractual relationship with Micr0$0ft. Specifically, they made the unfortunate mistake of purchasing software from Micr0$0ft, agreeing to the Micr0$0ft End User License Agreement (contractual relationship) and then running the Micr0$0ft software, which lead directly to their system getting owned (or pwn3d if you prefer) due to the enormous number of design flaws, well known exploits, and other deficiencies in the code purchased from Micr0$0ft. In what way, exactly, is this in any part the ISPs fault? Why should their ISP bear the brunt of the costs for Micr0$0ft's poorly written code? Owen
* Owen DeLong:
Wrong... Most of them are subject to the problems they have because of their contractual relationship with Micr0$0ft. Specifically, they made the unfortunate mistake of purchasing software from Micr0$0ft, agreeing to the Micr0$0ft End User License Agreement (contractual relationship) and then running the Micr0$0ft software, which lead directly to their system getting owned (or pwn3d if you prefer) due to the enormous number of design flaws, well known exploits, and other deficiencies in the code purchased from Micr0$0ft.
In most parts of the world, the Microsoft EULA is not enforceable. Most users don't buy their software from Microsoft, either. It's preinstalled on their PC, and Microsoft disclaims any support.
In what way, exactly, is this in any part the ISPs fault? Why should their ISP bear the brunt of the costs for Micr0$0ft's poorly written code?
Most ISPs recommend using Microsoft software or provide software for the Microsoft platform, and require to turn on JavaScript, which makes browsers much more vulnerable. (Obviously, this doesn't matter in practice, but still.) They don't exist in a vacuum. But the whole thing underlines a very difficult problem compromised end users face: they haven't got anyone to turn to. Someone quoted rates for some services, and these aren't acceptable (you can almost get a newer, faster PC for that price). Part of the problem is piracy, which makes it difficult to reinstall everything from scratch. Another one is the lack of an audit trail which would tell *why* the customer got infected, so that you could get some learning effect.
On Jun 15, 2007, at 3:06 PM, Florian Weimer wrote:
Most users don't buy their software from Microsoft, either. It's preinstalled on their PC, and Microsoft disclaims any support.
That is mostly true, except in the case of security issues - which is what I believe this thread is somehow still talking about. Microsoft gives no-charge phone support to all Windows users, regardless of where you received your license, if it's a security related problem. Viruses, spyware, intrusions, etc. Call 1-866- PCSAFETY in the US/Canada, or you can click at link at http:// support.microsoft.com/security to get information for other countries. I've never tried it, but I've heard that they've been surprisingly helpful, even in cases where it was obviously not Microsoft's fault (directly, anyway). I'm not 100% positive that their policy explicitly allows OEM license holders to use that number, but from those I've talked to that have used it - they don't ask for any license information at all. After they've verified it fits their definition of a security problem, you're handed over to a tech to help you clean it up. -- Kevin
On Jun 15, 2007, at 1:23 PM, Kevin Day wrote:
I've never tried it, but I've heard that they've been surprisingly helpful, even in cases where it was obviously not Microsoft's fault (directly, anyway). I'm not 100% positive that their policy explicitly allows OEM license holders to use that number, but from those I've talked to that have used it - they don't ask for any license information at all. After they've verified it fits their definition of a security problem, you're handed over to a tech to help you clean it up.
It is my understanding that they even support pirated software in this context; they figure it's better to fix the stuff and then figure out how to get the right stuff there than to wenge about its pedigree.
* Fred Baker: [Microsoft security updates]
It is my understanding that they even support pirated software in this context;
Their message on this message on this topic is rather mixed. The Office update used to display warnings that after a security update, pirated copies might cease to function. And the updates claimed that you need the original CD, which did not appear to be true, but still. For Windows Update, Microsoft has been quite successful in creating the impression that during the update check, your system is examined for pirated software. And finally, a major source of malware are sites which distribute cracks and product keys. 8-(
Its not a technical problem (although engineers seem to like to think everything is), its a legal issue with Microsoft's lawyer and licenses.
I realize it's not a technical problem, although I suspect there are some technical twiddles that could help, e.g., persuading Microsoft to put the update servers in their own ASN to make it easier to put them in a sandbox. And I realize that Microsoft's combination of arrogance and naivete can make them painful to deal with. So I guess I'm glad that the FBI has told people to call their ISPs, to remind ISPs that doing nothing is not costless, and to provide an incentive to keep pushing on MS and other providers of problem software to do something about it. R's, John
Once upon a time, John Levine <johnl@iecc.com> said:
I realize it's not a technical problem, although I suspect there are some technical twiddles that could help, e.g., persuading Microsoft to put the update servers in their own ASN to make it easier to put them in a sandbox. And I realize that Microsoft's combination of arrogance and naivete can make them painful to deal with.
$ dig download.windowsupdate.com ;download.windowsupdate.com. IN A download.windowsupdate.com. 3411 IN CNAME main.dl.wu.akadns.net. main.dl.wu.akadns.net. 111 IN CNAME dom.dl.wu.akadns.net. dom.dl.wu.akadns.net. 111 IN CNAME dl.wu.ms.edgesuite.net. dl.wu.ms.edgesuite.net. 8080 IN CNAME a26.ms.akamai.net. a26.ms.akamai.net. 20 IN A 216.180.86.39 a26.ms.akamai.net. 20 IN A 216.180.86.37 $ If you have Akamai servers, the IPs will be on your network (and of course shared with many other sites). You'd have to limit access with a limited DNS server (since few will use or even know IPs to visit) that only gives out DNS for certain hosts/domains. -- Chris Adams <cmadams@hiwaay.net> Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble.
On Jun 14, 2007, at 2:45 PM, Chris Adams wrote:
Once upon a time, John Levine <johnl@iecc.com> said:
I realize it's not a technical problem, although I suspect there are some technical twiddles that could help, e.g., persuading Microsoft to put the update servers in their own ASN to make it easier to put them in a sandbox. And I realize that Microsoft's combination of arrogance and naivete can make them painful to deal with.
$ dig download.windowsupdate.com ;download.windowsupdate.com. IN A download.windowsupdate.com. 3411 IN CNAME main.dl.wu.akadns.net. main.dl.wu.akadns.net. 111 IN CNAME dom.dl.wu.akadns.net. dom.dl.wu.akadns.net. 111 IN CNAME dl.wu.ms.edgesuite.net. dl.wu.ms.edgesuite.net. 8080 IN CNAME a26.ms.akamai.net. a26.ms.akamai.net. 20 IN A 216.180.86.39 a26.ms.akamai.net. 20 IN A 216.180.86.37 $
If you have Akamai servers, the IPs will be on your network (and of course shared with many other sites). You'd have to limit access with a limited DNS server (since few will use or even know IPs to visit) that only gives out DNS for certain hosts/domains.
Unfortunately, this is not always true. MS does not single-source. Users going to Windows Updates can and will be directed to a number of places, including Akamai, and Microsoft itself, depending on time of day, phase of moon, and whim of the content owner. In general, creating a sandbox where a computer can only reach $UPDATE_SERVER is very, very difficult. And, as much as I hate to admit it, MS OSes are not the only ones that can be compromised (he types on his black MacBook). That said, the majority of compromised computers do run some flavor of Redmond-Ware. (One can argue about the underlying cause - market share, quality of software, virus writer's preference, whatever - but the fact still stands that most compromised computers run Windows.) So getting a "windows update sandbox" would be very useful. -- TTFN, patrick
Patrick W. Gilmore wrote: [.]]
That said, the majority of compromised computers do run some flavor of Redmond-Ware. (One can argue about the underlying cause - market share, quality of software, virus writer's preference, whatever - but the fact still stands that most compromised computers run Windows.) So getting a "windows update sandbox" would be very useful.
You want to have a look at: http://technet.microsoft.com/en-us/wsus/ 8<---------------------------------------------------------------- Microsoft Windows Server Update Services Microsoft Windows Server Update Services (WSUS) enables information technology administrators to deploy the latest Microsoft product updates to computers running the Windows operating system. By using WSUS, administrators can fully manage the distribution of updates that are released through Microsoft Update to computers in their network. ----------------------------------------------------------------->8 Which is used in large organizations to deploy patches with ease. Requires some AD mumbojumbo of course. Really the information is out there, google knows, so can you :) Greets, Jeroen
On Thu, 14 Jun 2007, Jeroen Massar wrote:
You want to have a look at: http://technet.microsoft.com/en-us/wsus/
Which is used in large organizations to deploy patches with ease. Requires some AD mumbojumbo of course.
Really the information is out there, google knows, so can you :)
Read the Microsoft license agreement for WSUS, the information is out there. It works for institutional license holders, but not for public ISPs. Small ISPs without legions of lawyers may not worry about stuff like this, but unfortunately large ISPs have too. Its not a technical issue. If the Microsoft lawyers said ok, the engineers could come up with lots of ways to do this. I asked Microsoft's lawyers multiple times. But as always, you should consult with your own legal advisor. I keep hoping one day Microsoft will announce something like WSUS for ISPs. But its been several years.
On Jun 14, 2007, at 12:21 PM, Sean Donelan wrote:
Read the Microsoft license agreement for WSUS, the information is out there. It works for institutional license holders, but not for public ISPs.
Maybe I'm totally off-base, but I could've sworn I read something somewhere in the last year or so about Microsoft working with some or genning up a program to work with SPs in order to offer this functionality to their customers, if they so choose? Can anyone from Microsoft comment? ---------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // 408.527.6376 voice Equo ne credite, Teucri. -- Laocoön
Let me buy an appliance that handles that DNS/filtering/firewalling/updating/etc for owned machines, one that has MSFT's blessing, and that just requires policy-based routing and handing out special DNS server IPs. Frank -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Sean Donelan Sent: Thursday, June 14, 2007 2:22 PM To: Jeroen Massar Cc: nanog@nanog.org Subject: Re: FBI tells the public to call their ISP for help On Thu, 14 Jun 2007, Jeroen Massar wrote:
You want to have a look at: http://technet.microsoft.com/en-us/wsus/
Which is used in large organizations to deploy patches with ease. Requires some AD mumbojumbo of course.
Really the information is out there, google knows, so can you :)
Read the Microsoft license agreement for WSUS, the information is out there. It works for institutional license holders, but not for public ISPs. Small ISPs without legions of lawyers may not worry about stuff like this, but unfortunately large ISPs have too. Its not a technical issue. If the Microsoft lawyers said ok, the engineers could come up with lots of ways to do this. I asked Microsoft's lawyers multiple times. But as always, you should consult with your own legal advisor. I keep hoping one day Microsoft will announce something like WSUS for ISPs. But its been several years.
Frank Bulk wrote:
Let me buy an appliance that handles that DNS/filtering/firewalling/updating/etc for owned machines, one that has MSFT's blessing, and that just requires policy-based routing and handing out special DNS server IPs.
Please see one of: http://domino.research.ibm.com/comm/pr.nsf/pages/news.20060327_virus.html http://www.informationweek.com/story/showArticle.jhtml?articleID=14200013 http://www.ercim.org/publication/Ercim_News/enw56/riordan.html and various others. Billy Goats can do exactly at least the jailing part and most likely there are other similar services that provide the same functionality. The upgrade portion really depends on the installed software base of course. Without somebody actually doing the upgrade and most likely not even removing the virus/bot etc in place, not much can be done in that area, especially in non-ISP environments where you don't have root on the PC. This portion at least quarantines the box and then allows you to simply instruct the user in the common methods of battling the problem that the user has. Greets, Jeroen
The Billy Goat product only seems to detect and notify nefarious activity, but it does nothing for the owned clients. I want something that restricts my owned subscribers to downloading updates and tools while preventing them from spewing forth more spam and the like. Mirage Networks is the closest to it, from my limited knowledge. Frank -----Original Message----- From: Jeroen Massar [mailto:jeroen@unfix.org] Sent: Saturday, June 16, 2007 9:43 PM To: frnkblk@iname.com Cc: 'Sean Donelan'; nanog@nanog.org Subject: Re: FBI tells the public to call their ISP for help Frank Bulk wrote:
Let me buy an appliance that handles that DNS/filtering/firewalling/updating/etc for owned machines, one that has MSFT's blessing, and that just requires policy-based routing and handing out special DNS server IPs.
Please see one of: http://domino.research.ibm.com/comm/pr.nsf/pages/news.20060327_virus.html http://www.informationweek.com/story/showArticle.jhtml?articleID=14200013 http://www.ercim.org/publication/Ercim_News/enw56/riordan.html and various others. Billy Goats can do exactly at least the jailing part and most likely there are other similar services that provide the same functionality. The upgrade portion really depends on the installed software base of course. Without somebody actually doing the upgrade and most likely not even removing the virus/bot etc in place, not much can be done in that area, especially in non-ISP environments where you don't have root on the PC. This portion at least quarantines the box and then allows you to simply instruct the user in the common methods of battling the problem that the user has. Greets, Jeroen
Frank Bulk wrote:
The Billy Goat product only seems to detect and notify nefarious activity, but it does nothing for the owned clients.
I want something that restricts my owned subscribers to downloading updates and tools while preventing them from spewing forth more spam and the like.
A Billy Goat will nicely quarantine the host that is infected, that is the whole goal of the system. What access is still allowed when the host is in that quarantine is of course a matter of policy. Allowing them to access things like Windows Update and providing at least a good virusscanner + SpyBot Search&Destroy etc is most likely a good thing to do for these situations. IMHO ISPs should per default simply feed port 25 outbound through their own SMTP relays. BUT always have a very easy way (eg a Control Panel behind a user/pass on a website) to disable this kind of filtering. This is what XS4all does and it seems to have a lot of effect but still allows anybody who doesn't 'want' this protection to use the Internet the way they want it, and not the way that is prescribed before them. Of course, when they disable the filter it becomes very easy when something does go wrong to laugh at them and not allow them to turn the filter off unless they pay a fine or something similar ;) For that matter, why don't ISPs start doing that: Introduce a fine. When somebody gets infected, and thus doesn't take good care of his/her/it's computer fine them. Let them pay say $25 to get fully back on the Internet and only allow a very slow rate of traffic in the mean time. Of course, the argument most likely goes then that they will swap ISPs, but they will quickly run out of those and of course ISPs don't want to lose clients over it, as the ignorant are the ones that provide the simple cash.
Mirage Networks is the closest to it, from my limited knowledge.
As mentioned, there are most very likely different products in this area which can resolve your problem. Also one can always run your own(tm). Greets, Jeroen
On Sun, 17 Jun 2007, Jeroen Massar wrote:
For that matter, why don't ISPs start doing that: Introduce a fine. When somebody gets infected, and thus doesn't take good care of his/her/it's computer fine them. Let them pay say $25 to get fully back on the Internet and only allow a very slow rate of traffic in the mean time.
Please review the archives. ISPs have tried fining customers as far back as 1997. Past attempts to hold individuals responsible for the actions of their compromised computers result in bad press, because eventually the individuals will be little old grandmothers on fixed incomes whose only contact with the outside world is getting pictures of their grandchildren via the net. Newspapers love stories about big bad corporations picking on poor innocent grandmothers. http://www.networkworld.com/news/2004/091304widernetearthlink.html?page=1 This ISP flatfoot enjoys giving spammers the boot Bellyaching bad guys just part of the job. By Cara Garretson, Network World, 09/13/04
Assigning a fine doesn't win any friends. The customer is already miffed that: a) we talked to them and wasted their precious personal time b) 'accused' them of malicious activity c) that we took them offline d) that they'll now need to spend $100 at a computer shop or use up goodwill credits with computer-savvy friends to fix it up. No, fines don't help, at least for the majority of people. If they care in any way they will try to get it fixed ASAP, and if they don't care, well, we don't feel too bad then if we have to disconnect them. Again, that's rarely the case because 99% of people really do care. Regards, Frank -----Original Message----- From: Jeroen Massar [mailto:jeroen@unfix.org] Sent: Sunday, June 17, 2007 9:15 AM To: frnkblk@iname.com Cc: 'Sean Donelan'; nanog@nanog.org Subject: Quarantining infected hosts (Was: FBI tells the public to call their ISP for help) Frank Bulk wrote:
The Billy Goat product only seems to detect and notify nefarious activity, but it does nothing for the owned clients.
I want something that restricts my owned subscribers to downloading updates and tools while preventing them from spewing forth more spam and the like.
A Billy Goat will nicely quarantine the host that is infected, that is the whole goal of the system. What access is still allowed when the host is in that quarantine is of course a matter of policy. Allowing them to access things like Windows Update and providing at least a good virusscanner + SpyBot Search&Destroy etc is most likely a good thing to do for these situations. IMHO ISPs should per default simply feed port 25 outbound through their own SMTP relays. BUT always have a very easy way (eg a Control Panel behind a user/pass on a website) to disable this kind of filtering. This is what XS4all does and it seems to have a lot of effect but still allows anybody who doesn't 'want' this protection to use the Internet the way they want it, and not the way that is prescribed before them. Of course, when they disable the filter it becomes very easy when something does go wrong to laugh at them and not allow them to turn the filter off unless they pay a fine or something similar ;) For that matter, why don't ISPs start doing that: Introduce a fine. When somebody gets infected, and thus doesn't take good care of his/her/it's computer fine them. Let them pay say $25 to get fully back on the Internet and only allow a very slow rate of traffic in the mean time. Of course, the argument most likely goes then that they will swap ISPs, but they will quickly run out of those and of course ISPs don't want to lose clients over it, as the ignorant are the ones that provide the simple cash.
Mirage Networks is the closest to it, from my limited knowledge.
As mentioned, there are most very likely different products in this area which can resolve your problem. Also one can always run your own(tm). Greets, Jeroen
Indeed and there is no need to fine them. Simply quarantine them in a way that allows outbound WWW access and nothing else. Most customers will not notice anyway. You could also occasionally re-direct them to a forced-portal that tells them they are infected with something and describing how to fix it. Remember, they are victims too... -- Leigh Frank Bulk wrote:
Assigning a fine doesn't win any friends. The customer is already miffed that: a) we talked to them and wasted their precious personal time b) 'accused' them of malicious activity c) that we took them offline d) that they'll now need to spend $100 at a computer shop or use up goodwill credits with computer-savvy friends to fix it up.
No, fines don't help, at least for the majority of people. If they care in any way they will try to get it fixed ASAP, and if they don't care, well, we don't feel too bad then if we have to disconnect them. Again, that's rarely the case because 99% of people really do care.
Regards,
Frank
-----Original Message----- From: Jeroen Massar [mailto:jeroen@unfix.org] Sent: Sunday, June 17, 2007 9:15 AM To: frnkblk@iname.com Cc: 'Sean Donelan'; nanog@nanog.org Subject: Quarantining infected hosts (Was: FBI tells the public to call their ISP for help)
Frank Bulk wrote:
The Billy Goat product only seems to detect and notify nefarious activity, but it does nothing for the owned clients.
I want something that restricts my owned subscribers to downloading
updates
and tools while preventing them from spewing forth more spam and the like.
A Billy Goat will nicely quarantine the host that is infected, that is the whole goal of the system. What access is still allowed when the host is in that quarantine is of course a matter of policy. Allowing them to access things like Windows Update and providing at least a good virusscanner + SpyBot Search&Destroy etc is most likely a good thing to do for these situations.
IMHO ISPs should per default simply feed port 25 outbound through their own SMTP relays. BUT always have a very easy way (eg a Control Panel behind a user/pass on a website) to disable this kind of filtering. This is what XS4all does and it seems to have a lot of effect but still allows anybody who doesn't 'want' this protection to use the Internet the way they want it, and not the way that is prescribed before them. Of course, when they disable the filter it becomes very easy when something does go wrong to laugh at them and not allow them to turn the filter off unless they pay a fine or something similar ;)
For that matter, why don't ISPs start doing that: Introduce a fine. When somebody gets infected, and thus doesn't take good care of his/her/it's computer fine them. Let them pay say $25 to get fully back on the Internet and only allow a very slow rate of traffic in the mean time.
Of course, the argument most likely goes then that they will swap ISPs, but they will quickly run out of those and of course ISPs don't want to lose clients over it, as the ignorant are the ones that provide the simple cash.
Mirage Networks is the closest to it, from my limited knowledge.
As mentioned, there are most very likely different products in this area which can resolve your problem. Also one can always run your own(tm).
Greets, Jeroen
On 6/17/07, Jeroen Massar <jeroen@unfix.org> wrote:
IMHO ISPs should per default simply feed port 25 outbound through their own SMTP relays. BUT always have a very easy way (eg a Control Panel behind a user/pass on a website) to disable this kind of filtering. This
Y'know, port 25 is just the tip of the iceberg when it comes to what all an infected host can do .. which is why quite a lot of ISPs (Bell Canada is particularly good at it, as are some others) are getting good at deploying "Walled Gardens" - vlan the infected host into its own little sandbox from where it can access only windows update, AV update sites and the ISP's support pages, nothing else, on any port. The user has to fix (disinfect, reimage, whatever) his host before he contacts the ISP support desk and gets let back onto their network. --srs
Suresh Ramasubramanian wrote:
On 6/17/07, Jeroen Massar <jeroen@unfix.org> wrote:
IMHO ISPs should per default simply feed port 25 outbound through their own SMTP relays. BUT always have a very easy way (eg a Control Panel behind a user/pass on a website) to disable this kind of filtering. This
Y'know, port 25 is just the tip of the iceberg when it comes to what all an infected host can do ..
Of course, though 25 is (afaik ;) the most abused one that will annoy a lot of other folks with spam, phishings and virus distribution, though the latter seems to have come to a near halt from what I see.
which is why quite a lot of ISPs (Bell Canada is particularly good at it, as are some others) are getting good at deploying "Walled Gardens" - vlan the infected host into its own little sandbox from where it can access only windows update, AV update sites and the ISP's support pages, nothing else, on any port.
The user has to fix (disinfect, reimage, whatever) his host before he contacts the ISP support desk and gets let back onto their network.
That is IMHO really the only way to go. People who get hit by that once, or maybe even twice will make sure it doesn't happen the third time. Support costs will effectively sink because of such a system as it will avoid those hosts from infecting others hosts, to be part of bot nets, spam attacks etc etc etc. (Especially for managers: Lower your TCO! Drive Business! $buzzword!) I tip my hat to the Bell Canada folks for having such a system! Greets, Jeroen
On 6/18/07, Jeroen Massar <jeroen@unfix.org> wrote:
Of course, though 25 is (afaik ;) the most abused one that will annoy a lot of other folks with spam, phishings and virus distribution, though the latter seems to have come to a near halt from what I see.
Read these and weep, then - http://darkwing.uoregon.edu/~joe/port25.pdf http://darkwing.uoregon.edu/~joe/zombies.pdf As Joe says (and I agree), trying to fix infected hosts on your network by blocking port 25 is like treating lung cancer with cough syrup. srs -- Suresh Ramasubramanian (ops.lists@gmail.com)
On Mon, 18 Jun 2007, Suresh Ramasubramanian wrote:
On 6/18/07, Jeroen Massar <jeroen@unfix.org> wrote:
Of course, though 25 is (afaik ;) the most abused one that will annoy a lot of other folks with spam, phishings and virus distribution, though the latter seems to have come to a near halt from what I see.
Read these and weep, then - http://darkwing.uoregon.edu/~joe/port25.pdf http://darkwing.uoregon.edu/~joe/zombies.pdf
As Joe says (and I agree), trying to fix infected hosts on your network by blocking port 25 is like treating lung cancer with cough syrup.
The great thing about opinions is everyone has one. See also http://www.maawg.org/port25 Or how about http://www.securitymanagement.com/library/Sans_Ulrich1203.pdf http://www.networkworld.com/edge/news/2003/0908studyisps.html The best answer is probably paying for a strong ISP abuse team. But for whatever reasons, some ISPs prefer to invest in other areas.
On 6/18/07, Sean Donelan <sean@donelan.com> wrote:
The great thing about opinions is everyone has one. See also http://www.maawg.org/port25
MAAWG's port 25 management document is kind of based on consensus. Joe is a senior tech advisor at MAAWG. contributed substantially to that document .. and those two presentations were made at a maawg (san diego in 2005 if I remember right) so ..
The best answer is probably paying for a strong ISP abuse team. But for whatever reasons, some ISPs prefer to invest in other areas.
Bah. Not to underrate having a strong and clued abuse team. However, throwing more people at this is a non starter. You need to automate. -- Suresh Ramasubramanian (ops.lists@gmail.com)
On Mon, 18 Jun 2007, Suresh Ramasubramanian wrote:
The best answer is probably paying for a strong ISP abuse team. But for whatever reasons, some ISPs prefer to invest in other areas.
Bah. Not to underrate having a strong and clued abuse team. However, throwing more people at this is a non starter. You need to automate.
Automation is a non-starter unless you have people to deal with the exceptions. If you don't deal with exceptions, eventually problems with any automated system will overwhelm you. You can only hid behind IVR recordings "You call is very important to us" for so long.
On 6/18/07, Sean Donelan <sean@donelan.com> wrote:
Automation is a non-starter unless you have people to deal with the exceptions. If you don't deal with exceptions, eventually problems with any automated system will overwhelm you. You can only hid behind IVR recordings "You call is very important to us" for so long.
You're preaching to the choir there. That still doesnt underrate the importance of automating this. Throwing people at it simply doesnt scale. -- Suresh Ramasubramanian (ops.lists@gmail.com)
On Mon, 18 Jun 2007, Suresh Ramasubramanian wrote:
On 6/18/07, Sean Donelan <sean@donelan.com> wrote:
Automation is a non-starter unless you have people to deal with the exceptions. If you don't deal with exceptions, eventually problems with any automated system will overwhelm you. You can only hid behind IVR recordings "You call is very important to us" for so long.
You're preaching to the choir there. That still doesnt underrate the importance of automating this. Throwing people at it simply doesnt scale.
You need a both. The mistake engineers make is thinking technology is the solution. The mistake customer care makes is thinking a pleasent voice is the solution. The mistake law enforcement makes is thinking an arrest is the solution. The mistake legislators make is thinking a law is the solution. And so on. We need a mix of all those things, including people, technology, laws and physical arrests. The problem is not a naturally occuring phenomena. The opponents are intelligent people who react to anything we do. I've seen ISPs with very advanced automated systems that went unused becaused their customer care organizations couldn't cope with the scale of problem customers. I was building infected customer sandboxes a long time ago. Even if your automated systems handle 99% of the problem customers, that 1% can doom your plans if you don't understand it. ISPs looking for automation may consider these vendors or several free/open source alternatives. Simplicita: http://www.simplicita.com/ Bradbord: http://www.bradfordnetworks.com/ Motive: http://www.motive.com/ Cisco/Perfigo: http://www.cisco.com/en/US/products/ps6128/index.html F-Secure Network Control: http://www.f-secure.co.uk/enterprises/products/fsnc.html Trend Micro Intercloud: http://us.trendmicro.com/us/about/news/pr/article/20070123143622.html
On 6/18/07, Sean Donelan <sean@donelan.com> wrote:
Simplicita: http://www.simplicita.com/ Bradbord: http://www.bradfordnetworks.com/ Motive: http://www.motive.com/ Cisco/Perfigo: http://www.cisco.com/en/US/products/ps6128/index.html F-Secure Network Control: http://www.f-secure.co.uk/enterprises/products/fsnc.html Trend Micro Intercloud: http://us.trendmicro.com/us/about/news/pr/article/20070123143622.html
Add PerfTech - www.perftech.com to the list. I think Arbor and Sandvine have some kit for this as well. As for the rest - you're still preaching to the choir here, I dont see where we disagree on this -- Suresh Ramasubramanian (ops.lists@gmail.com)
On Mon, 2007-06-18 at 21:00 +0530, Suresh Ramasubramanian wrote:
On 6/18/07, Sean Donelan <sean@donelan.com> wrote:
Automation is a non-starter unless you have people to deal with the exceptions. If you don't deal with exceptions, eventually problems with any automated system will overwhelm you. You can only hid behind IVR recordings "You call is very important to us" for so long.
You're preaching to the choir there. That still doesnt underrate the importance of automating this. Throwing people at it simply doesnt scale.
Before you make it a technical or HR issue you first have to either find a way to make aggressive ISP policies profitable or introduce .gov-regulations that say you either operate according to some standard or not at all. //per
On 6/19/07, Per Heldal <heldal@eml.cc> wrote:
Before you make it a technical or HR issue you first have to either find a way to make aggressive ISP policies profitable or introduce .gov-regulations that say you either operate according to some standard or not at all.
Well - you have to have your management behind you on this one - it involves monitoring and a change or two all across your network, not just at the edge, or the core. Plus changes to support and other. -- Suresh Ramasubramanian (ops.lists@gmail.com)
Suresh Ramasubramanian wrote:
MAAWG's port 25 management document is kind of based on consensus. Joe is a senior tech advisor at MAAWG. contributed substantially to that document .. and those two presentations were made at a maawg (san diego in 2005 if I remember right) so ..
Joe also pointed out the biggest problem with blocking port 25; it pushes the abuse towards the smarthosts. This creates a lot of issues. Smarthosts have to be regulated more closely. Support must be increased to deal with customers that have legitimate large scale outbound needs and will need smarthost restrictions lifted. A certain amount of spam leakage must be expected out of the smarthost, but most recipients won't know or take the time to tell the difference. This leads to more blocking of the smarthosts, which causes more issues for a larger number of customers. I'd rather monitor and filter traffic patterns on port 25 (and the various other ports that are also often spewing other things) than block it. It's not unusual to see tcp/25 spewing at the same time as udp/135 and tcp/445 or even tcp/1025. A detection of both network scans and correlating inbound connections to outbound tcp/25 leads to a lot of good proactive automation. Spam abuse may be the most publicly annoying use of trojans/bots, but it's probably the least destructive use (debatable). Jack
On 6/18/07, Jack Bates <jbates@brightok.net> wrote:
Joe also pointed out the biggest problem with blocking port 25; it pushes the abuse towards the smarthosts. This creates a lot of issues. Smarthosts have to
So .. great. You have a huge spam problem that flew under your radar as it was spread across multiple /24s or far larger netblocks, now concentrated within far fewer servers that are part of the same cluster. That kind of makes your job a bit easier then .. half full glass v/s half empty glass, and all that.
I'd rather monitor and filter traffic patterns on port 25 (and the various other ports that are also often spewing other things) than block it. It's not unusual to see tcp/25 spewing at the same time as udp/135 and tcp/445 or even tcp/1025.
[...] Which is what a lot of the kit Sean posted about does .. srs -- Suresh Ramasubramanian (ops.lists@gmail.com)
Suresh Ramasubramanian wrote:
On 6/18/07, Jack Bates <jbates@brightok.net> wrote:
Joe also pointed out the biggest problem with blocking port 25; it pushes the abuse towards the smarthosts. This creates a lot of issues. Smarthosts have to
So .. great. You have a huge spam problem that flew under your radar as it was spread across multiple /24s or far larger netblocks, now concentrated within far fewer servers that are part of the same cluster. That kind of makes your job a bit easier then .. half full glass v/s half empty glass, and all that.
I'd rather monitor and filter traffic patterns on port 25 (and the various other ports that are also often spewing other things) than block it. It's not unusual to see tcp/25 spewing at the same time as udp/135 and tcp/445 or even tcp/1025.
[...]
Which is what a lot of the kit Sean posted about does ..
srs
We filter ALL udp/135 and tcp/445 or even tcp/1025 towards and from the Internet. Port 25 is only allowed to go through the smarthosts and other whitelisted mail servers. We have never had any complaints about the 135/445/1025 blocking and very few about the port25 stuff. Spambots are getting clever and they now use configured SMTP relays in thunderbird/outlook etc so the mail gateways get quite a bit of traffic. But we have lots of them (Ironports) behind load balancers so theres little problem there. -- Leigh Porter UK Broadband
On 6/18/07, Suresh Ramasubramanian <ops.lists@gmail.com> wrote:
On 6/18/07, Jeroen Massar <jeroen@unfix.org> wrote:
Of course, though 25 is (afaik ;) the most abused one that will annoy a lot of other folks with spam, phishings and virus distribution, though the latter seems to have come to a near halt from what I see.
[snip]
As Joe says (and I agree), trying to fix infected hosts on your network by blocking port 25 is like treating lung cancer with cough syrup.
Perhaps, but I think someone possibly misunderstood the goal behind blocking port 25. It doesn't "fix" an infected host, the point is to mitigate one of the attack vectors by which the infection could spread to new clean hosts, to reduce the range of possible attacks/spreading techniques infected host could launch -- in some cases, the spread will stop entirely, if the particular software spreads only by connecting to destination mail servers on port 25, and while the hosts may still be infected, there is much less harm (in terms of automatically spamming and spreading to other hosts) that will be possible, with port 25 blocked. Preventing hosts from just SMTP'ing out just anywhere they like creates a new hurdle for any infection to get over to spread; now any malware suddenly needs to figure out a SMTP server to use, and a username and password to use with SMTP authentication, and any other restrictions imposed by the ISP outgoing MTA. Think of it as having people infected with TB wearing masks while they are in public. It certainly doesn't cure them of the disease, that's not the point. It's for the protection of possible hosts not yet infected by the parasite. It's no guarantee that the disease doesn't ever spread to someone else, but the opportunity for airborne spread is slightly reduced, and that's the goal. -- -J
James Hess wrote:
Preventing hosts from just SMTP'ing out just anywhere they like creates a new hurdle for any infection to get over to spread; now any malware suddenly needs to figure out a SMTP server to use, and a username and password to use with SMTP authentication, and any other restrictions imposed by the ISP outgoing MTA.
This sounds great, except it doesn't scale. My router says there is no noticeable difference between tcp/25 and tcp/445, or udp/134 or udp/1434 or tcp/1025, or tcp/80. It asked if we should just block all ports and force people through proxy servers. Why mitigate one vector when you can take them all out? What makes SMTP so special a vector? Yes, my router speaks. Yours doesn't? Jack
Jack Bates wrote:
James Hess wrote:
Preventing hosts from just SMTP'ing out just anywhere they like creates a new hurdle for any infection to get over to spread; now any malware suddenly needs to figure out a SMTP server to use, and a username and password to use with SMTP authentication, and any other restrictions imposed by the ISP outgoing MTA.
This sounds great, except it doesn't scale. My router says there is no noticeable difference between tcp/25 and tcp/445, or udp/134 or udp/1434 or tcp/1025, or tcp/80. It asked if we should just block all ports and force people through proxy servers. Why mitigate one vector when you can take them all out? What makes SMTP so special a vector?
Yes, my router speaks. Yours doesn't?
Jack
You said it does not scale but then went on to describe a completely differant issue. Agreed, SMTP is not really a special vector, other than it's ovbious commercial spam use. So just block all the usual virus vector ports, block 25 and force people to use your own SMTP servers and the problem 9this particular one goes away.. -- Leigh
On 6/19/07, Leigh Porter <leigh.porter@ukbroadband.com> wrote:
Agreed, SMTP is not really a special vector, other than it's ovbious commercial spam use. So just block all the usual virus vector ports, block 25 and force people to use your own SMTP servers and the problem 9this particular one goes away..
No. the part of it you target (outbound spam) merely relocates itself, and your smtp servers become huge spam sinks. Filter all you want and you'll still leak spam unless you take those hosts down And in the meantime those hosts will also be launching dos attacks, hosting "fast flux" pills / warez / kiddy pr0n sites, carrying out id / card theft .. best to isolate and take them down. You can port block at your edge till you burst and you'll still be in a lot of hot water. -- Suresh Ramasubramanian (ops.lists@gmail.com)
On Jun 19, 2007, at 8:35 AM, Suresh Ramasubramanian wrote:
On 6/19/07, Leigh Porter <leigh.porter@ukbroadband.com> wrote:
Agreed, SMTP is not really a special vector, other than it's obvious commercial spam use. So just block all the usual virus vector ports, block 25 and force people to use your own SMTP servers and the problem [for] this particular one goes away..
No. the part of it you target (outbound spam) merely relocates itself, and your smtp servers become huge spam sinks. Filter all you want and you'll still leak spam unless you take those hosts down
And in the meantime those hosts will also be launching dos attacks, hosting "fast flux" pills / warez / kiddy pr0n sites, carrying out id / card theft .. best to isolate and take them down.
You can port block at your edge till you burst and you'll still be in a lot of hot water.
Web-site/browser vulnerabilities make ISP efforts largely futile. Infection rates easily overwhelm aggressive automated detection and wall-garden strategies. Nevertheless, blocking port 25 offers several benefits even for this seemingly failing effort. Messages can be rate limited, where delivery errors also provide direct clues as to which system are likely infected. Web related script vulnerabilities impact some of the largest online email providers! In the zeal to enable advertising, customer accounts are easily harvested. These accounts may also receive password updates from other accounts, placing even critical financial information at risk. Every compromised account is then able to impersonate owners, utilize their address book and entice further infections by offering malware related messages. The malware might appear as seemingly harmless links or documents. Email is a vector that must be watched carefully, however the greater danger is with web/browser vulnerabilities. Complacency permitting, and at times even promoting use of known defective products must end. The era of combining scripts and active code along with every piece of information conveyed must end. Unless the Internet industry responds effectively, legislators will likely to react in their own futile way. Less is more. A document MUST NOT require active code to convey information. -Doug
Douglas Otis wrote:
Complacency permitting, and at times even promoting use of known defective products must end. The era of combining scripts and active code along with every piece of information conveyed must end. Unless the Internet industry responds effectively, legislators will likely to react in their own futile way.
According to a recent article on Wired: /* SNIP */ It would make it unlawful for anyone to: "...engage in unfair or deceptive acts or practices in connection with specified conduct, including: (1) taking unsolicited control of the computer; (2) modifying computer settings; (3) collecting personally identifiable information [incl. using keystroke loggers]; (4) inducing the owner or authorized user to disclose personally identifiable information; (5) inducing the unsolicited installation of computer software; and (6) removing or disabling a security, anti-spyware, or anti-virus technology." http://blog.wired.com/27bstroke6/2007/06/house_passes_an.html http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=110_cong_bills&docid=f:h964eh.txt.pdf /* END SNIP */ Which leaves me wondering... Sometimes in order for someone to actually install something helpful, one might at times have to disable certain programs then re-enable them. Looking at the broad term "modifying computer settings" and "disabling a security..." one has to wonder whether an overzealous office running politician would use such a broad law for political purposes. Politics aside, reality is reality. This law is beyond broad in fact taken at face value, any ISP seeking to mitigate a problem on their network may somewhere down the line break a law. How can one argue they never were "induced the authorized owner to disclose their information" to someone say mitigating security when that person threw them on a "cleanroom vlan". Trollishness aside, laws are almost always taken at face value black and white until someone falls victim to an insanely dumb law and fights back. I'd hate to be scapegoated as an individual and would hate to see the business I'm working for get a bad rap for some congressperson's lack of understanding and zeal to gain higher power. -- ==================================================== J. Oquendo http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x1383A743 echo infiltrated.net|sed 's/^/sil@/g' "Wise men talk because they have something to say; fools, because they have to say something." -- Plato
Douglas Otis wrote:
On Jun 19, 2007, at 8:35 AM, Suresh Ramasubramanian wrote:
On 6/19/07, Leigh Porter <leigh.porter@ukbroadband.com> wrote:
Agreed, SMTP is not really a special vector, other than it's obvious commercial spam use. So just block all the usual virus vector ports, block 25 and force people to use your own SMTP servers and the problem [for] this particular one goes away..
No. the part of it you target (outbound spam) merely relocates itself, and your smtp servers become huge spam sinks. Filter all you want and you'll still leak spam unless you take those hosts down
And in the meantime those hosts will also be launching dos attacks, hosting "fast flux" pills / warez / kiddy pr0n sites, carrying out id / card theft .. best to isolate and take them down.
You can port block at your edge till you burst and you'll still be in a lot of hot water.
Web-site/browser vulnerabilities make ISP efforts largely futile. Infection rates easily overwhelm aggressive automated detection and wall-garden strategies. Nevertheless, blocking port 25 offers several benefits even for this seemingly failing effort. Messages can be rate limited, where delivery errors also provide direct clues as to which system are likely infected.
Web related script vulnerabilities impact some of the largest online email providers! In the zeal to enable advertising, customer accounts are easily harvested. These accounts may also receive password updates from other accounts, placing even critical financial information at risk. Every compromised account is then able to impersonate owners, utilize their address book and entice further infections by offering malware related messages. The malware might appear as seemingly harmless links or documents. Email is a vector that must be watched carefully, however the greater danger is with web/browser vulnerabilities.
Complacency permitting, and at times even promoting use of known defective products must end. The era of combining scripts and active code along with every piece of information conveyed must end. Unless the Internet industry responds effectively, legislators will likely to react in their own futile way.
Less is more. A document MUST NOT require active code to convey information.
-Doug
This is a great point Doug. Port based vulns are, IMO, starting to decline due to update of SP2 etc. There's still a lot there but in a few years it will be quite low as hopefully most people will either filter it or customers will have default on firewalls. Browsers and dumb customers opening emails are where it's at now. The only way to filter that is to look at ALL traffic using some horrid DPI box or proxy or something. life really sucks. -- Leigh
On Tue, 19 Jun 2007, Jack Bates wrote:
This sounds great, except it doesn't scale. My router says there is no noticeable difference between tcp/25 and tcp/445, or udp/134 or udp/1434 or tcp/1025, or tcp/80. It asked if we should just block all ports and force people through proxy servers. Why mitigate one vector when you can take them all out? What makes SMTP so special a vector?
Actually, yes ISPs should block them all. That's a bit provocative, but lets work through it. Its a question of what should be the "defaults" for various types of Internet connections. Almost every consumer grade router/modem (d-link, linksys, netgear, etc) now includes stateful packet firewalls. Almost all recent consumer operating systems now include a stateful packet firewall. While the ultimate decision should remain the subscriber's, ISPs still should adopt improved "default" settings for today's Internet. If a subscriber wants to surf the Internet naked, that should be their informed choice. If you use the principle of least surprise, most retail consumers don't initially expect their computers to be "open" to the Internet or for their computer to do things they didn't initiate. Individual "default settings" shouldn't be in the backbone, that doesn't scale. However, at the "edge" (or near edge, which might be on the provider side of the demarc or the user side of the demarc) may be a good spot for some things we've found we can't trust with hosts/applications. Most of the work on improving defaults for CPE and "near edge" defaults is not occuring in the IETF. Instead CableLabs and the DSL Forum have been the leaders in this area. While the "default" might be to block unexpected/unusual traffic, the ISP, edge, cpe, etc should all give the user the control until they abuse it. -- Caution: UltraDNS/Neustar's monthly rates aren't month to month.
That AD mumbo jumbo you blow off so blithely is HOW you get clients to use WSUS instead of whichever random IP Microsoft is pointing at today for updates. It requires Group Policy settings, and unless you want to force all your customers to make their machines part of an AD domain, which most can't join even if they were willing since they're running consumer machines with XP Home on them, you can't force them to use your local server. Jamie Bowden -- "It was half way to Rivendell when the drugs began to take hold" Hunter S Tolkien "Fear and Loathing in Barad Dur" Iain Bowen <alaric@alaric.org.uk> -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Jeroen Massar Sent: Thursday, June 14, 2007 3:14 PM To: Patrick W. Gilmore Cc: nanog@nanog.org Subject: Re: FBI tells the public to call their ISP for help Patrick W. Gilmore wrote: [.]]
That said, the majority of compromised computers do run some flavor of Redmond-Ware. (One can argue about the underlying cause - market share, quality of software, virus writer's preference, whatever - but the fact still stands that most compromised computers run Windows.) So getting a "windows update sandbox" would be very useful.
You want to have a look at: http://technet.microsoft.com/en-us/wsus/ 8<---------------------------------------------------------------- Microsoft Windows Server Update Services Microsoft Windows Server Update Services (WSUS) enables information technology administrators to deploy the latest Microsoft product updates to computers running the Windows operating system. By using WSUS, administrators can fully manage the distribution of updates that are released through Microsoft Update to computers in their network. ----------------------------------------------------------------->8 Which is used in large organizations to deploy patches with ease. Requires some AD mumbojumbo of course. Really the information is out there, google knows, so can you :) Greets, Jeroen
In general, creating a sandbox where a computer can only reach $UPDATE_SERVER is very, very difficult.
I believe it. Perhaps we could help Microsoft make it easier. The sandbox doesn't have to include all their servers, just enough of them to service the sandboxed users.
And, as much as I hate to admit it, MS OSes are not the only ones that can be compromised (he types on his black MacBook).
If we can get sandboxes for MS and Apple, we FreeBSD users are willing to take our chances. R's, John
Since many Microsoft patches are only legally available via the Internet, and an ISP can not predict which servers Microsoft will use to distribute Microsoft patches, ISPs must enable essentially full Internet access which includes access for most worms.
Has anybody tried a firewalling solution in which unpatched PCs are only able to access a special ISP-operated forwarding nameserver which is configured to only reply with A records for a list of known Microsoft update sites? And then have this specially patched nameserver also trigger the firewall to open up access to the addresses that it returns in A records? According to Microsoft, their list of "trusted sites" for MS Update is *.update.microsoft.com and download.windowsupdate.com. Even if they have some sort of CDN (Content Delivery Network) with varying IP addresses based on topology or load, this is still predictable enough for a software solution to provide a temporary walled garden. You don't need to make copies of their patch files. You don't need MS to provide an out-of-band list of safe IP addresses. As long as you are able to divert a subscriber's traffic through a special firewalled garden, an ISP can implement this with no special support from MS. Wrap this up with a GUI for your support-desk people to enable/disable the traffic diversion and you have a low-cost solution. You can even leverage the same technology to deal with botnet infestations although you would probably want a separate firewalled garden that allows access to a wider range of sites known to be safe, i.e. Google, Yahoo, ISP's own pages, etc. --Michael Dillon
On Thursday 14 June 2007 10:27, michael.dillon@bt.com wrote:
Since many Microsoft patches are only legally available via the Internet, and an ISP can not predict which servers Microsoft will use to distribute Microsoft patches, ISPs must enable essentially full Internet access which includes access for most worms.
Has anybody tried a firewalling solution in which unpatched PCs are only able to access a special ISP-operated forwarding nameserver which is configured to only reply with A records for a list of known Microsoft update sites? And then have this specially patched nameserver also trigger the firewall to open up access to the addresses that it returns in A records?
According to Microsoft, their list of "trusted sites" for MS Update is *.update.microsoft.com and download.windowsupdate.com. Even if they have some sort of CDN (Content Delivery Network) with varying IP addresses based on topology or load, this is still predictable enough for a software solution to provide a temporary walled garden.
You don't need to make copies of their patch files. You don't need MS to provide an out-of-band list of safe IP addresses. As long as you are able to divert a subscriber's traffic through a special firewalled garden, an ISP can implement this with no special support from MS. Wrap this up with a GUI for your support-desk people to enable/disable the traffic diversion and you have a low-cost solution. You can even leverage the same technology to deal with botnet infestations although you would probably want a separate firewalled garden that allows access to a wider range of sites known to be safe, i.e. Google, Yahoo, ISP's own pages, etc.
--Michael Dillon
There's a major problem with this - End-users won't take nicely to being restricted from going to specific websites, and will more than likely go to another ISP rather than to patch their computer as they see no benefit of patching themselves. We see the benefit of the patches, they don't nessasarily. Not to single anyone out but there will more than likely always be a careless (and/or clueless) ISP who doesn't care if over half their network is wormed, the customers from the ISPs who are cracking down on infected machines will simply go over to the ISP who doesn't care as there would be "less hassle". What needs to be done is ALL ISPs accross the board need to clean up their networks, thus cornering the lazy end-users into cleaning up their machines. To be honest: There's too few ISPs that would want to take up the responsibility of filtering worm'd customers, and as well, the instant an ISP starts filtering, they may even set themselves up for a lawsuit of the customer saying "I paid for the service, why aren't I getting it?!" And reguarding Microsoft and their patching licences: Those patches may be their precious "legal property" but it's their hording of legal rights that's damaging hundreds of thousands of computers. Microsoft is currently abusing their market share standings and giving insufficient patch distribution, (i.e. offline distibution) Therefore Microsoft should be held accountable for every computer that becomes infected with worms due to insufficient patching. To me, it sounds like Microsoft wants the power, but doesn't want the responsibility that comes with the power of great market share. It is time Microsoft be forced to take that responsibility.
On Thu, 2007-06-14 at 16:34 -0400, Kradorex Xeron wrote: [snip]
And reguarding Microsoft and their patching licences: Those patches may be their precious "legal property" but it's their hording of legal rights that's damaging hundreds of thousands of computers. Microsoft is currently abusing their market share standings and giving insufficient patch distribution, (i.e. offline distibution) Therefore Microsoft should be held accountable for every computer that becomes infected with worms due to insufficient patching. To me, it sounds like Microsoft wants the power, but doesn't want the responsibility that comes with the power of great market share. It is time Microsoft be forced to take that responsibility.
Regulation targeting software-vendors and service-providers has little effect as it is an attempt to disrupt the money-flow somewhere in the middle. It's usually more efficient to "attack" the source. I.e. authorities must hold computer users responsible and make them pay every penny (or millions;) it costs to investigate and clean up their mess. Mainstream OS'es as we know them would have been unsellable in today's market if users, ever since the internet was commercialised, had been held responsible. //per
In the 2+ years I have been working for an ISP I'm not aware of one customer that has gone over to one of our competitors because we identified and cut them off for an abuse issue. Most of them have been very grateful that we identified a problem and are earnest in resolving it. And for those who don't care? In a slight variation on an oft-quoted statement in this listserv, "I want my competitors to have them." Frank -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Kradorex Xeron Sent: Thursday, June 14, 2007 3:35 PM To: nanog@nanog.org Subject: Re: FBI tells the public to call their ISP for help On Thursday 14 June 2007 10:27, michael.dillon@bt.com wrote:
Since many Microsoft patches are only legally available via the Internet, and an ISP can not predict which servers Microsoft will use to distribute Microsoft patches, ISPs must enable essentially full Internet access which includes access for most worms.
Has anybody tried a firewalling solution in which unpatched PCs are only able to access a special ISP-operated forwarding nameserver which is configured to only reply with A records for a list of known Microsoft update sites? And then have this specially patched nameserver also trigger the firewall to open up access to the addresses that it returns in A records?
According to Microsoft, their list of "trusted sites" for MS Update is *.update.microsoft.com and download.windowsupdate.com. Even if they have some sort of CDN (Content Delivery Network) with varying IP addresses based on topology or load, this is still predictable enough for a software solution to provide a temporary walled garden.
You don't need to make copies of their patch files. You don't need MS to provide an out-of-band list of safe IP addresses. As long as you are able to divert a subscriber's traffic through a special firewalled garden, an ISP can implement this with no special support from MS. Wrap this up with a GUI for your support-desk people to enable/disable the traffic diversion and you have a low-cost solution. You can even leverage the same technology to deal with botnet infestations although you would probably want a separate firewalled garden that allows access to a wider range of sites known to be safe, i.e. Google, Yahoo, ISP's own pages, etc.
--Michael Dillon
There's a major problem with this - End-users won't take nicely to being restricted from going to specific websites, and will more than likely go to another ISP rather than to patch their computer as they see no benefit of patching themselves. We see the benefit of the patches, they don't nessasarily. Not to single anyone out but there will more than likely always be a careless (and/or clueless) ISP who doesn't care if over half their network is wormed, the customers from the ISPs who are cracking down on infected machines will simply go over to the ISP who doesn't care as there would be "less hassle". What needs to be done is ALL ISPs accross the board need to clean up their networks, thus cornering the lazy end-users into cleaning up their machines. To be honest: There's too few ISPs that would want to take up the responsibility of filtering worm'd customers, and as well, the instant an ISP starts filtering, they may even set themselves up for a lawsuit of the customer saying "I paid for the service, why aren't I getting it?!" And reguarding Microsoft and their patching licences: Those patches may be their precious "legal property" but it's their hording of legal rights that's damaging hundreds of thousands of computers. Microsoft is currently abusing their market share standings and giving insufficient patch distribution, (i.e. offline distibution) Therefore Microsoft should be held accountable for every computer that becomes infected with worms due to insufficient patching. To me, it sounds like Microsoft wants the power, but doesn't want the responsibility that comes with the power of great market share. It is time Microsoft be forced to take that responsibility.
On 6/17/07, Frank Bulk <frnkblk@iname.com> wrote:
In the 2+ years I have been working for an ISP I'm not aware of one customer that has gone over to one of our competitors because we identified and cut them off for an abuse issue. Most of them have been very grateful that we identified a problem and are earnest in resolving it.
I'm pretty sceptical of the notion that it's easier to change ISP than download a Windows update. If that's true for your network, perhaps you should tell us something about your provisioning/billing/CRM arrangements!
participants (23)
-
Alexander Harrowell -
Chris Adams -
Douglas Otis -
Florian Weimer -
Frank Bulk -
Fred Baker -
J. Oquendo -
Jack Bates -
James Hess -
Jamie Bowden -
Jeroen Massar -
Jim Popovitch -
John Levine -
Kevin Day -
Kradorex Xeron -
Leigh Porter -
michael.dillon＠bt.com -
Owen DeLong -
Patrick W. Gilmore -
Per Heldal -
Roland Dobbins -
Sean Donelan -
Suresh Ramasubramanian