Quarantine your infected users spreading malware
Many ISP's who do care about issues such as worms, infected users "spreading the love", etc. simply do not have the man-power to handle all their infected users' population. It is becoming more and more obvious that the answer may not be at the ISP's doorstep, but the ISP's are indeed a critical part of the solution. What their eventual role in user safety will be I can only guess, but it is clear (to me) that this subject is going to become a lot "hotter" in coming years. Aunty Jane (like Dr. Alan Solomon (drsolly) likes to call your average user) is your biggest risk to the Internet today, and how to fix the user non of us have a good idea quite yet. Especially since it's not quite one as I put in an Heinlein quote below. Some who are user/broadband ISP's (not say, tier-1 and tier-2's who would be against it: "don't be the Internet's Firewall") are blocking ports such as 139 and 445 for a long time now, successfully preventing many of their users from becoming infected. This is also an excellent first step for responding to relevant outbreaks and halting their progress. Philosophy aside, it works. It stops infections. Period. Back to the philosophy, there are some other solutions as well. Plus, should this even be done? One of them has been around for a while, but just now begins to mature: Quarantining your users. Infected users quarantine may sound a bit harsh, but consider; if a user is indeed infected and does "spread the joy" on your network as well as others', and you could simply firewall him (or her) out of the world (VLAN, other solutions which may be far better) letting him (or her) go only to a web page explaining the problem to them, it's pretty nifty. As many of us know, handling such users on tech support is not very cost-effective to ISP's, as if a user makes a call the ISP already losses money on that user. Than again, paying abuse desk personnel just so that they can disconnect your users is losing money too. Which one would you prefer? Jose (Nazario) points to many interesting papers on the subject on his blog: http://www.wormblog.com/papers/ Is it the ISP's place to do this? Should the ISP do this? Does the ISP have a right to do this? If the ISP is nice enough to do it, and users know the ISP might. Why not? This (as well as port blocking) is more true for organizations other than ISP's, but if they are indeed user/broadband ISP's, I see this as both the effective and the ethical thing to do if the users are notified this might happen when they sign their contracts. Then all the "don't be the Internet's firewall" debate goes away. I respect the "don't be the Internet's firewall issue", not only for the sake of the cause but also because friends such as Steven Bellovin and other believe in them a lot more strongly than I do. Bigger issues such as the safety of the Internet exist now. That doesn't mean user rights are to be ignored, but certainly so shouldn't ours, especially if these are mostly unaffected? I believe both are good and necessary solutions, but every organization needs to choose what is best for it, rather than follow some pre-determined blueprint. What's good for one may be horrible for another. "You don't approve? Well too bad, we're in this for the species boys and girls. It's simple numbers, they have more and every day I have to make decisions that send hundreds of people, like you, to their deaths." -- Carl Jenkins, Starship Trooper, the movie. I don't think the second part of the quote is quite right (to say the least), but I felt bad leaving it out, it's Heinlein after all... anyone who claims he is a fascist though will have to deal with me. :) This isn't only about users, it's about the bad guys and how they out-number us, too. They have far better cooperation to boot. There are several such products around and they have been discussed here on NANOG before, but I haven't tried them myself as of yet, so I can't really recommend any of them. Can you? I'll update on these as I find out more on: http://blogs.securiteam.com This write-up can be found here: http://blogs.securiteam.com/index.php/archives/312 Gadi. -- http://blogs.securiteam.com/ "Out of the box is where I live". -- Cara "Starbuck" Thrace, Battlestar Galactica.
On Mon, 20 Feb 2006 23:40:48 +0200, Gadi Evron said:
Many ISP's who do care about issues such as worms, infected users "spreading the love", etc. simply do not have the man-power to handle all their infected users' population.
It is becoming more and more obvious that the answer may not be at the ISP's doorstep, but the ISP's are indeed a critical part of the solution. What their eventual role in user safety will be I can only guess, but it is clear (to me) that this subject is going to become a lot "hotter" in coming years.
The ISPs will be a part of the solution. However, ISPs fall into two major categories: 1) The ones that read the types of lists that you posted this to 2) The ones that have the problem. You're preaching to the choir, Gadi - and if there's *one* thing I'd like a solution for, it's *that* problem. How do you get the unwashed masses of ISPs to join the choir so you can preach to them?
Valdis.Kletnieks@vt.edu wrote:
On Mon, 20 Feb 2006 23:40:48 +0200, Gadi Evron said:
Many ISP's who do care about issues such as worms, infected users "spreading the love", etc. simply do not have the man-power to handle all their infected users' population.
It is becoming more and more obvious that the answer may not be at the ISP's doorstep, but the ISP's are indeed a critical part of the solution. What their eventual role in user safety will be I can only guess, but it is clear (to me) that this subject is going to become a lot "hotter" in coming years.
The ISPs will be a part of the solution. However, ISPs fall into two major categories:
1) The ones that read the types of lists that you posted this to 2) The ones that have the problem.
You're preaching to the choir, Gadi - and if there's *one* thing I'd like a solution for, it's *that* problem. How do you get the unwashed masses of ISPs to join the choir so you can preach to them?
What products that answer this are out there, and how good, in your experience, are they? We discussed this here before non-conclusively and stayed on philosophy, anyone has new experience on the subject? Thanks.
On Tue, 21 Feb 2006, Gadi Evron wrote:
Many ISP's who do care about issues such as worms, infected users "spreading the love", etc. simply do not have the man-power to handle all their infected users' population.
The ISPs will be a part of the solution. However, ISPs fall into two major categories:
1) The ones that read the types of lists that you posted this to 2) The ones that have the problem.
You're preaching to the choir, Gadi - and if there's *one* thing I'd like a solution for, it's *that* problem. How do you get the unwashed masses of ISPs to join the choir so you can preach to them?
What products that answer this are out there, and how good, in your experience, are they?
We discussed this here before non-conclusively and stayed on philosophy, anyone has new experience on the subject?
Let's be clear in what we're addressing. Are we talking about an en masse quarantine of IP addresses sending the worm traffic, or identifying the C&C<->payload conversations and applying blocks accordingly? Where are the anti-virus and software firewall vendors in this conversation? To be plain, this obviously isn't a problem you can solve with some border filters. The complexity, and fallout, from trying to put those kinds of filtering in is just too great. It's cumbersome to manage manually and operational impact is too great. If we're going to philosophize about solutions, let's throw some ideas out. Where do concepts like ThreatNet fit into this notion? (http://ali.as/threatnet/) To save some reading, the idea behind ThreatNet is to establish a closed threat sharing network with trusted peers, sharing information about malcontents doing things on your network that they shouldn't be. If you can positively identify SSH brute force sources, port scan patterns, worm traffic, spam sources, etc, and report them to trusted peers in a collaborative fashion, it becomes easier to support intelligent and rapid traffic filtering concepts in your network designs, where appropriate, even if it's something as simple as putting together a business case for filtering entire netblocks or regions. (Yes, I write my own analyzers, and yes, I'm involved peripherally with this project.) ThreatNet is still pretty nascent, but conceptually it's got merit. I'll bring up MainNerve again since they're the only vendor I've worked with that's got tools for selectively filtering known troublemakers. As a potential solution, I bring both of these items up because they provide the ability to take good, distributed intelligence gathering and apply them to your network in a precision manner, if at all, in accordance with any unique policies you may have. The problem, as I see it, is that even if one ISP sees the bad behaviour, there's no communication amongst the community (that I can see) to relay or collate the history. It's like playing Mom off against Dad because they never talk to each other. For coming up with clear patterns of abuse and shenanigans, we're suffering from collective myopia because we're ignoring an aspect of of our favorite big ass communications medium. Or I'm completely off base, in which case tell me to shut up and I'll go back into my code coma. - billn
While i'm not being told to shut up because this is off topic (yet), I'm going to suggest that people interested in continuing this conversation contact me off list and coordinate something ad hoc. The amount of bullshit I've already recieved in response to thinking that this has operational merit when it comes to mitigating both risk and effects is pretty astounding, even by nanog standards. Thanks. - billn On Mon, 20 Feb 2006, Bill Nash wrote:
On Tue, 21 Feb 2006, Gadi Evron wrote:
Many ISP's who do care about issues such as worms, infected users "spreading the love", etc. simply do not have the man-power to handle all their infected users' population.
The ISPs will be a part of the solution. However, ISPs fall into two major categories:
1) The ones that read the types of lists that you posted this to 2) The ones that have the problem.
You're preaching to the choir, Gadi - and if there's *one* thing I'd like a solution for, it's *that* problem. How do you get the unwashed masses of ISPs to join the choir so you can preach to them?
What products that answer this are out there, and how good, in your experience, are they?
We discussed this here before non-conclusively and stayed on philosophy, anyone has new experience on the subject?
Let's be clear in what we're addressing. Are we talking about an en masse quarantine of IP addresses sending the worm traffic, or identifying the C&C<->payload conversations and applying blocks accordingly?
Where are the anti-virus and software firewall vendors in this conversation? To be plain, this obviously isn't a problem you can solve with some border filters. The complexity, and fallout, from trying to put those kinds of filtering in is just too great. It's cumbersome to manage manually and operational impact is too great.
If we're going to philosophize about solutions, let's throw some ideas out. Where do concepts like ThreatNet fit into this notion? (http://ali.as/threatnet/) To save some reading, the idea behind ThreatNet is to establish a closed threat sharing network with trusted peers, sharing information about malcontents doing things on your network that they shouldn't be. If you can positively identify SSH brute force sources, port scan patterns, worm traffic, spam sources, etc, and report them to trusted peers in a collaborative fashion, it becomes easier to support intelligent and rapid traffic filtering concepts in your network designs, where appropriate, even if it's something as simple as putting together a business case for filtering entire netblocks or regions. (Yes, I write my own analyzers, and yes, I'm involved peripherally with this project.) ThreatNet is still pretty nascent, but conceptually it's got merit.
I'll bring up MainNerve again since they're the only vendor I've worked with that's got tools for selectively filtering known troublemakers.
As a potential solution, I bring both of these items up because they provide the ability to take good, distributed intelligence gathering and apply them to your network in a precision manner, if at all, in accordance with any unique policies you may have. The problem, as I see it, is that even if one ISP sees the bad behaviour, there's no communication amongst the community (that I can see) to relay or collate the history. It's like playing Mom off against Dad because they never talk to each other. For coming up with clear patterns of abuse and shenanigans, we're suffering from collective myopia because we're ignoring an aspect of of our favorite big ass communications medium.
Or I'm completely off base, in which case tell me to shut up and I'll go back into my code coma.
- billn
How do you get the unwashed masses of ISPs to join the choir so you can preach to them?
Why not just bypass them and go direct to the unwashed masses of end users? Offer them a free windows infection blocker program that imposes the quarantine itself locally on the user's machine. This program would use stealth techniques to hide itself in the user's machine, just like viruses do. And this program would do nothing but register itself with an encoded registry, and listen for an encoded command to activate itself. Rather like a botnet except with the user's consent and with a positive goal. When the community of bot/worm researchers determines that this machine is infected, they inform the central registry using their own encoded signal. When enough "votes" have been collected, the registry sends the shutdown signal to the end user, thus triggering the blocker program to quarantine the user. At this point a friendly helpful webpage pops up and guides the user through the disinfection process. Unlike antivirus software, the application on the user's computer does not need to detect malware and it needs no database updates. It does only one thing and it relies on the collective intelligence of the anti-malware community. This won't stop worms or botnets, but it will slow them down and it will greatly speed the cleanup process. --Michael Dillon
----- Original Message ----- From: <Michael.Dillon@btradianz.com> Subject: Re: Quarantine your infected users spreading malware
Rather like a botnet except with the user's consent and with a positive goal.<<
Isn't this pretty much like how they were compromised in the first place? How do you differentiate this infection from the ones they've been preached to to avoid? "Trust me...I won't come in your mouth."
On Tue, 21 Feb 2006 13:05:35 GMT, Michael.Dillon@btradianz.com said:
How do you differentiate this infection from the ones they've been preached to to avoid?
The same way that people currently differentiate bad software from good software before they install something on their machines.
If people actually *knew* how to do this differentiation any better than flipping the quarter I have in my pocket, we wouldn't be having this discussion.
On Tue, 21 Feb 2006 Valdis.Kletnieks@vt.edu wrote:
If people actually *knew* how to do this differentiation any better than flipping the quarter I have in my pocket, we wouldn't be having this discussion.
Yep. Although it should have been obvious, a problem with quarantine systems is most users can't validate an inline "trusted path" if the host or something along the path may have been compromised. Even if it hasn't been totally compromised, the bad guys can impersonate the look and feel of your quarantine system to lead your users down the walled garden path of the bad guy's choosing. If you notify uses by e-mail, the bad guys can make their e-mail look very similar. If you notify users by web page interception, the bad guys can make their web page pop-ups look like your quarantine pages. And so on. So you are quickly back to out-of-band communication paths with the user. A couple of years ago I was a big fan of inline quarantine systems. And for some things it may still work such as initial registration and setup before an user's machine is compromised. But I've changed my mind, or rather the bad guys changed it for me, what the long term effectiveness of inline quarantine systems of compromised systems can be.
Michael.Dillon@btradianz.com wrote:
How do you get the unwashed masses of ISPs to join the choir so you can preach to them?
Why not just bypass them and go direct to the unwashed masses of end users? Offer them a free windows infection blocker program that imposes the quarantine itself locally on the user's machine. This program would use stealth techniques to hide itself in the user's machine, just like viruses do. And this program would do nothing but register itself with an encoded registry, and listen for an encoded command to activate itself. Rather like a botnet except with the user's consent and with a positive goal.
When the community of bot/worm researchers determines that this machine is infected, they inform the central registry using their own encoded signal. When enough "votes" have been collected, the registry sends the shutdown signal to the end user, thus triggering the blocker program to quarantine the user.
At this point a friendly helpful webpage pops up and guides the user through the disinfection process.
Unlike antivirus software, the application on the user's computer does not need to detect malware and it needs no database updates. It does only one thing and it relies on the collective intelligence of the anti-malware community.
This won't stop worms or botnets, but it will slow them down and it will greatly speed the cleanup process.
--Michael Dillon
Hi Michael, the only problem with that approach is that you think like a defender. As the defense is local to the user's machine, the attacker can just kick it away. -- http://blogs.securiteam.com/ "Out of the box is where I live". -- Cara "Starbuck" Thrace, Battlestar Galactica.
Offer them a free windows infection blocker program that imposes the quarantine itself locally on the user's machine. This program would use stealth techniques to hide itself in the user's machine, just like viruses do.
As the defense is local to the user's machine, the attacker can just kick it away.
How are they going to identify the code to throw away? I believe that the state of the art for AV software is to create randomly named EXE files so that attackers cannot delete the running process, and then the EXE file ensures that the installed program and startup config are not tampered with. If AV software can protect itself this way, why would anyone build an infection blocker using any less protection? --Michael Dillon
Michael.Dillon@btradianz.com wrote:
If AV software can protect itself this way, why would anyone build an infection blocker using any less protection?
AV software can *try* and protect itself in this and other ways, but that is OT to NANOG. I don't mind discussing it in private though if software protection reversing technology interests you. :) Gadi. -- http://blogs.securiteam.com/ "Out of the box is where I live". -- Cara "Starbuck" Thrace, Battlestar Galactica.
On 2/21/06, Michael.Dillon@btradianz.com <Michael.Dillon@btradianz.com> wrote:
Why not just bypass them and go direct to the unwashed masses of end users? Offer them a free windows infection blocker program that imposes the quarantine itself locally on the user's machine. This program would use stealth techniques to hide itself in the user's machine, just like viruses do. And this program would do nothing but register itself with an encoded registry, and listen for an encoded command to activate itself. Rather like a botnet except with the user's consent and with a positive goal.
Intruiging concept.. Why bother "hiding" itself though? Or is the idea to prevent itself from being removed by malware?
When the community of bot/worm researchers determines that this machine is infected, they inform the central registry using their own encoded signal. When enough "votes" have been collected, the registry sends the shutdown signal to the end user, thus triggering the blocker program to quarantine the user.
Isn't there a risk of DoS though? What's to prevent someone from "spoofing" those signals and shutting down other users? Relative precautions would need to be taken, but to be sure, the end-user needs the ability to override the system. Thus leaving us in the same situation as before. Firewall? I don't need no stinking firewall.. :)
Unlike antivirus software, the application on the user's computer does not need to detect malware and it needs no database updates. It does only one thing and it relies on the collective intelligence of the anti-malware community.
Sure it does.. It doesn't need to remove it, per se, but it will need to know what the infection is so it can give the correct disinfection instructions..
--Michael Dillon
-- Jason 'XenoPhage' Frisvold XenoPhage0@gmail.com
When enough "votes" have been collected, the registry sends the shutdown signal to the end user, thus triggering the blocker program to quarantine the user.
Isn't there a risk of DoS though? What's to prevent someone from "spoofing" those signals and shutting down other users?
The signal would be encoded using a unique key. I would also expect that the choice of listening port would be somehow randomized and registered in the central registry to make it less of a DOS target.
Relative precautions would need to be taken, but to be sure, the end-user needs the ability to override the system. Thus leaving us in the same situation as before. Firewall? I don't need no stinking firewall..
I see no reason why the user needs the ability to override or remove the software. After all, during normal operation it does nothing at all therefore it does not interfere in any way with machine operation. The intent is to make it virtually impossible to remove this software so that a virus or worm cannot remove it either.
Sure it does.. It doesn't need to remove it, per se, but it will need to know what the infection is so it can give the correct disinfection instructions..
If the quarantined state keeps open a port 443 connection to a specific trusted webserver run by the group of trusted security researchers then the specifics of combatting the worm can be made available on that site. If necessary the site could upload ActiveX controls to do malware scans or recommend the installation of such software. --Michael Dillon
On Tue, 21 Feb 2006, Michael.Dillon@btradianz.com wrote:
Why not just bypass them and go direct to the unwashed masses of end users? Offer them a free windows infection blocker program that imposes the quarantine itself locally on the user's machine. This program
Offering them free software won't work to the levels you want. At first, you'll get a response, because consumers always jump at free shiny things, until something happens that makes them not like it anymore, and then they'll dig in and never use it again. If you want to get this kind of filtering into your core, you have a need to get this to a compulsory level for access. I don't think there's any disagreement as to the roots of this problem: - Modern users are generally clueless. - Most don't have firewalls or even the most basic of protections. - Getting tools deployed where they need to be most is the hardest. With that said.. If you're talking about a compulsory software solution, why not, as an ISP, go back to authenticated activity? Distribute PPPOE clients mated with common anti-spyware/anti-viral tools. Pull down and update signatures *every time* the user logs in, and again periodically while the user is logged in (for those that never log out). Require these safeguards to be active before they can pass the smallest traffic. The change in traffic flow would necessitate some architecture kung fu, maybe even AOL style, but you'd have the option of selectively picking out reported malicious/infected users (*cough* ThreatNet *cough*) and routing them through packet inspection frameworks on a case by case basis. Quite possibly, you could even automate that and the users would never be the wiser. - billn
On 2/21/06, Bill Nash <billn@odyssey.billn.net> wrote:
If you're talking about a compulsory software solution, why not, as an ISP, go back to authenticated activity? Distribute PPPOE clients mated with common anti-spyware/anti-viral tools. Pull down and update signatures *every time* the user logs in, and again periodically while the user is logged in (for those that never log out). Require these safeguards to be active before they can pass the smallest traffic.
Cost prohibitive.. In order to do that you'll need licenses from the AV companies..
The change in traffic flow would necessitate some architecture kung fu, maybe even AOL style, but you'd have the option of selectively picking out reported malicious/infected users (*cough* ThreatNet *cough*) and routing them through packet inspection frameworks on a case by case basis. Quite possibly, you could even automate that and the users would never be the wiser.
And then the privacy zealots would be livid.. Silently re-routing traffic like that.. How dare you suggest such a ... wait.. hrm.. The internet basically does this already.. I wonder if the zealots are aware of that.. :)
- billn
-- Jason 'XenoPhage' Frisvold XenoPhage0@gmail.com
On Tue, 21 Feb 2006 10:42:20 EST, Jason Frisvold said:
On 2/21/06, Bill Nash <billn@odyssey.billn.net> wrote:
If you're talking about a compulsory software solution, why not, as an ISP, go back to authenticated activity? Distribute PPPOE clients mated with common anti-spyware/anti-viral tools. Pull down and update signatures *every time* the user logs in, and again periodically while the user is logged in (for those that never log out). Require these safeguards to be active before they can pass the smallest traffic.
Cost prohibitive.. In order to do that you'll need licenses from the AV companies..
Oddly enough, AOL and several other large providers seem to have no problems advertising some variant on 'free A/V software'.
On 2/21/06, Valdis.Kletnieks@vt.edu <Valdis.Kletnieks@vt.edu> wrote:
Oddly enough, AOL and several other large providers seem to have no problems advertising some variant on 'free A/V software'.
Key words there.. "Large Provider" .. I don't think A/V companies have any interest whatsoever in smaller providers.. Just not a big enough customer base I guess... It would be nice to see an A/V provider willing to take that first step and offer something like this to providers, regardless of size. No packaging needed, so there's a cost savings there for the vendor. I'm not familiar with how this works in AOL land.. Does the end-user need to subscribe to anything other than AOL? ie, are there any "hidden" fees? -- Jason 'XenoPhage' Frisvold XenoPhage0@gmail.com
No, just $24/month (or whatever it is now) for the whole service. You go to a "keyword" and it > does a web based installation widget. It is free as long as you remain a subscriber. I'm not familiar with how this works in AOL land.. Does the end-user need to subscribe to anything other than AOL? ie, are there any "hidden" fees?
-- Jason 'XenoPhage' Frisvold XenoPhage0@gmail.com
On Tuesday 21 February 2006 10:26, Jason Frisvold wrote:
On 2/21/06, Valdis.Kletnieks@vt.edu <Valdis.Kletnieks@vt.edu> wrote:
Oddly enough, AOL and several other large providers seem to have no problems advertising some variant on 'free A/V software'.
Key words there.. "Large Provider" .. I don't think A/V companies have any interest whatsoever in smaller providers.. Just not a big enough customer base I guess...
It would be nice to see an A/V provider willing to take that first step and offer something like this to providers, regardless of size. No packaging needed, so there's a cost savings there for the vendor.
I'm not familiar with how this works in AOL land.. Does the end-user need to subscribe to anything other than AOL? ie, are there any "hidden" fees?
The problem with discussing AOL and "large provider" in the same sentence is that the complete AOL (connection, desktop, tools, etc) function are AOL controlled (walled garden) so they have the capability of doing much more in that arena that other providers. Secondly, to the best of my knowledge, A/V vendors do make their products available to "any" provider - it is just that small to medium sized ISP's cannot justify the cost/benefit ratio and keep their pricing anywhere near competitive with the "big" boys. At ten copies a month you get little to no discount - at 10,000 copies per month you get quite a cut... -- Larry Smith SysAd ECSIS.NET sysad@ecsis.net
On 21 Feb 2006, at 16:26, Jason Frisvold wrote:
Key words there.. "Large Provider" .. I don't think A/V companies have any interest whatsoever in smaller providers.. Just not a big enough customer base I guess... It would be nice to see an A/V provider willing to take that first step and offer something like this to providers, regardless of size.
Anti-virus is already offered directly to end users ... for free ! http://free.grisoft.com/ And they don't care ! How is someone else telling them that they need a virus checker going to change anything ? -a
On 2/23/06, Andy Davidson <andy@nosignal.org> wrote:
And they don't care ! How is someone else telling them that they need a virus checker going to change anything ?
It's not. That's why services such as AOL integrate it with the system.. Granted, the user has to initially accept it, but it's a virtually painless process.. AOL's software does all the work. If a user has to download each individual program, install it, ensure it's updated, etc., then they tend to ignore the use of such a product. Even mostly-automated updates are a burden for them because messages pop up now and then telling them that they're not up to date, warnings about new outbreaks, etc. Most users don't care one way or the other and it's simpler for them to ignore the whole situation. For something like AVG, yes it's free. But, I don't think that includes allowing an ISP to package it up and distribute it as a value-added feature.. Most companies frown on that sort of thing. I believe even Microsoft's EULA forbids distributing SP2 without strict permission.
-a
-- Jason 'XenoPhage' Frisvold XenoPhage0@gmail.com
Andy Davidson wrote:
And they don't care ! How is someone else telling them that they need a virus checker going to change anything ?
We allowed users back online to run Housecall at trendmicro for free so they could get cleaned up and save some money. However, the resuspend rate was so high, we quickly changed to offline cleanup only. It will remain until we perfect our auto defense system. Customers just want things to work. They don't care if they are infected. It's amazing how many customers swear they aren't scanning or sending email, and refuse to understand that their computer is capable of doing things without them knowing. -Jack
--On February 23, 2006 8:02:31 AM -0600 Jack Bates <jbates@brightok.net> wrote:
We allowed users back online to run Housecall at trendmicro for free so they could get cleaned up and save some money. However, the resuspend rate was so high, we quickly changed to offline cleanup only. It will remain until we perfect our auto defense system.
Customers just want things to work. They don't care if they are infected. It's amazing how many customers swear they aren't scanning or sending email, and refuse to understand that their computer is capable of doing things without them knowing.
What doesn't help is the ISPs out there who are complete dolts and first don't verify reports and second false alarm. They'll cut a user off on a single complaint without any evidence or verification. Or worse they have some automated system that false alarms without any way to verify you're cleaned up. And if you can't get online you can't get cleaned up anyway. Catch 22.
Michael Loftis wrote:
What doesn't help is the ISPs out there who are complete dolts and first don't verify reports and second false alarm. They'll cut a user off on a single complaint without any evidence or verification. Or worse they have some automated system that false alarms without any way to verify you're cleaned up. And if you can't get online you can't get cleaned up anyway. Catch 22.
I don't really see how any ISP will terminate an account for just one complaint, after all, it's losing money.. We have seen a few good examples of pretty big ISP's who said here how quarantine works for them. Got an example on how ISP's are kicking users out?
--On February 23, 2006 9:09:26 PM +0200 Gadi Evron <ge@linuxbox.org> wrote:
I don't really see how any ISP will terminate an account for just one complaint, after all, it's losing money..
We have seen a few good examples of pretty big ISP's who said here how quarantine works for them.
Got an example on how ISP's are kicking users out?
Speakeasy suspended my service for a week over a single report from someone. The mail never even travelled through or via any of my systems, the header bit that was called in was forged. It took a week to get them to give me the information they'd gotten in complaint. There was a forged Received header (completely fabricated, including the 'Qostfix' MTA) and also a forged HELO or EHLO of a non-existent host when it actually relayed it off onto someone elses MTA. I can't remember the exact ISP...might've been RoadRunner or TW in Toronto, but a friend had her DSL or CableModem suspendded, ended up changing providors. There was an infection, it was cleaned, they were allowed back on, then the ISP either received an old/backlogged complaint or something and they cut them off again,, but the machines were all clean (indeed watching the network for traffic over several days revealede nothing that they claimed to be the problem). -- "Genius might be described as a supreme capacity for getting its possessors into trouble of all kinds." -- Samuel Butler
On Thu 23 Feb 2006 (11:18 -0600), Michael Loftis wrote:
--On February 23, 2006 8:02:31 AM -0600 Jack Bates <jbates@brightok.net> wrote:
We allowed users back online to run Housecall at trendmicro for free so they could get cleaned up and save some money. However, the resuspend rate was so high, we quickly changed to offline cleanup only. It will remain until we perfect our auto defense system.
Customers just want things to work. They don't care if they are infected. It's amazing how many customers swear they aren't scanning or sending email, and refuse to understand that their computer is capable of doing things without them knowing.
What doesn't help is the ISPs out there who are complete dolts and first don't verify reports and second false alarm. They'll cut a user off on a single complaint without any evidence or verification. Or worse they have some automated system that false alarms without any way to verify you're cleaned up. And if you can't get online you can't get cleaned up anyway. Catch 22.
www.quarantainenet.nl It puts them in a protected environment where they can get cleaned up on-line without serious risk of re-infection. They can pop their e-mail, reply via webmail, but they can't connect to anywhere except a list of update sites. It uses honeypots to avoid false positives. In short, it works. -- Jim Segrave jes@nl.demon.net
On Tue, 28 Feb 2006, Jim Segrave wrote:
www.quarantainenet.nl
It puts them in a protected environment where they can get cleaned up on-line without serious risk of re-infection. They can pop their e-mail, reply via webmail, but they can't connect to anywhere except a list of update sites.
there was little in the way of 'how' in the link above though :(
The simplest method is to issue a different gateway to a registry of known offenders, forcing their into a restrictive environment that blocks all ports, and uses network translation tricks to redirect all web traffic to a portal. For cable modems and bridged DSL, you can do this with DHCP, matching their MAC address. PPPOE/DSL or similiar, you match on user name. Issue RFC1918 space with a gateway to your quarantine network. The rest is NAT/PAT and w3proxy stunts. You could pull it off with something as simple as iptables and squid, after dealing with the DHCP or authentication servers (ala Radius) to issue to the correct credentials. - billn On Tue, 28 Feb 2006, Christopher L. Morrow wrote:
On Tue, 28 Feb 2006, Jim Segrave wrote:
www.quarantainenet.nl
It puts them in a protected environment where they can get cleaned up on-line without serious risk of re-infection. They can pop their e-mail, reply via webmail, but they can't connect to anywhere except a list of update sites.
there was little in the way of 'how' in the link above though :(
On Tue, 28 Feb 2006, Bill Nash wrote:
The simplest method is to issue a different gateway to a registry of known offenders, forcing their into a restrictive environment that blocks all ports, and uses network translation tricks to redirect all web traffic to a portal.
For cable modems and bridged DSL, you can do this with DHCP, matching their MAC address. PPPOE/DSL or similiar, you match on user name. Issue RFC1918 space with a gateway to your quarantine network.
The rest is NAT/PAT and w3proxy stunts. You could pull it off with something as simple as iptables and squid, after dealing with the DHCP or authentication servers (ala Radius) to issue to the correct credentials.
yes, I could dream up a few hundred ways to accomplish this, but the 'documentation' at the site referenced doesn't address even one way. So, saying 'it works' and 'it works for carriers' and 'yea us!' is not helpful, without some example of 'how' :(
- billn
On Tue, 28 Feb 2006, Christopher L. Morrow wrote:
On Tue, 28 Feb 2006, Jim Segrave wrote:
www.quarantainenet.nl
It puts them in a protected environment where they can get cleaned up on-line without serious risk of re-infection. They can pop their e-mail, reply via webmail, but they can't connect to anywhere except a list of update sites.
there was little in the way of 'how' in the link above though :(
On Tue 28 Feb 2006 (19:29 +0000), Christopher L. Morrow wrote:
On Tue, 28 Feb 2006, Bill Nash wrote:
The simplest method is to issue a different gateway to a registry of known offenders, forcing their into a restrictive environment that blocks all ports, and uses network translation tricks to redirect all web traffic to a portal.
For cable modems and bridged DSL, you can do this with DHCP, matching their MAC address. PPPOE/DSL or similiar, you match on user name. Issue RFC1918 space with a gateway to your quarantine network.
The rest is NAT/PAT and w3proxy stunts. You could pull it off with something as simple as iptables and squid, after dealing with the DHCP or authentication servers (ala Radius) to issue to the correct credentials.
yes, I could dream up a few hundred ways to accomplish this, but the 'documentation' at the site referenced doesn't address even one way. So, saying 'it works' and 'it works for carriers' and 'yea us!' is not helpful, without some example of 'how' :(
You did think of contacting them and asking? You know, e-mail, fax, telephone, that sort of thing? The first time I mentioned this company, I said that it is used to put infected customers into a virtual router where all their internet traffic is proxied via a server. which blocks unwanted addresses, answers web requests not to designated servers from an internal service - so going to google.com brings up the page explaining why your account is quarantined. The specifics of connecting it to your network, oddly enough, probably will depend on how your network is built, which is why you might need to contact them. I thought this was a network operator's mailing list, not a spoon-feeding session -- Jim Segrave jes@nl.demon.net
Jim Segrave <jes@nl.demon.net> writes:
On Tue, 28 Feb 2006, Bill Nash wrote:
The simplest method is to issue a different gateway to a registry of known offenders, forcing their into a restrictive environment that blocks all ports, and uses network translation tricks to redirect all web traffic to a portal.
You did think of contacting them and asking? You know, e-mail, fax, telephone, that sort of thing?
Yes, we did think of that sort of thing. Those of us with even the slightest notion of business and profitability constraints promptly discarded the idea of getting a human into the loop. Ideally you just automatically add them to the broken stuff database, notify/incent them to fix things (by adding them to the quarantine group), and have them take care of themselves by following the directions found therein, and NOT involving your call center. ---Rob
On Thu, Mar 02, 2006 at 07:57:14AM -0500, Robert E. Seastrom wrote:
Jim Segrave <jes@nl.demon.net> writes:
You did think of contacting them and asking? You know, e-mail, fax, telephone, that sort of thing?
Yes, we did think of that sort of thing. Those of us with even the slightest notion of business and profitability constraints promptly discarded the idea of getting a human into the loop. Ideally you just
I think what Jim meant was something else: that if you have questions about the Quarantainenet product, you contact the Quarantainenet people via e-mail, fax or telephone and ask. 'Them' was not referring to actual customers this time. -- Niels Raijer | "But in the pocket of my clothes niels@fusix.nl | was a single white rose http://www.fusix.nl | for Pierrette..."
--On Tuesday, February 28, 2006 14:07:36 -0500 Bill Nash <billn@odyssey.billn.net> wrote:
The simplest method is to issue a different gateway to a registry of known offenders, forcing their into a restrictive environment that blocks all ports, and uses network translation tricks to redirect all web traffic to a portal.
For cable modems and bridged DSL, you can do this with DHCP, matching their MAC address. PPPOE/DSL or similiar, you match on user name. Issue RFC1918 space with a gateway to your quarantine network.
The rest is NAT/PAT and w3proxy stunts. You could pull it off with something as simple as iptables and squid, after dealing with the DHCP or authentication servers (ala Radius) to issue to the correct credentials.
We a couple techniques at Carnegie Mellon, depending on the network scenario. The DHCP based technique outlined above requires no extra infrastructure, just extra configuration, so it is what we use for most of our campus wired networks. We use the same setup as our registration helper network, so our internal name for the DHCP based quarantine system is called QuickReg. An unknown or banned client gets an address in 1918 space and can only access our abuse tracking, patch download and network registration systems. But on our campus wireless network we use a inline filter system we call AuthBridge, based on ebtables and iptables, to filter & redirect any traffic from unknown/banned clients. This system provides a more seamless user experience, but requires a layer-2 aggregation point where you can pass the traffic through the filter host. Because our wireless is a single campus wide layer-2 network this is more feasible for that network. Both of these systems are integrated with CMU's DHCP & DNS Management system, NetReg. (not to be confused with Southwestern University's NetReg. Different systems...) The DHCP helper system is a builtin feature, while the AuthBridge system is an add on. (AuthBridge just went through a complete rewrite to use the standard ebtables/iptables in Linux 2.6, and a public release should be available soon...) For information on NetReg, QuickReg or AuthBridge, see: http://www.net.cmu.edu/netreg http://acs-wiki.andrew.cmu.edu/twiki/bin/view/Netreg/WebHome http://acs-wiki.andrew.cmu.edu/twiki/bin/view/Netreg/NetRegManualDesign#Qui ckReg http://acs-wiki.andrew.cmu.edu/twiki/bin/view/Netreg/AuthBridge (Our abuse tracking system also integrates with NetReg, so going from an external incident report to a machine suspension and email to the user & admins is as simple as dropping an IP and timestamp into a web form...) -David Nolan Network Software Designer Computing Services Carnegie Mellon University
--On Tuesday, February 28, 2006 14:39:37 -0500 David Nolan <vitroth+@cmu.edu> wrote:
We a couple techniques at Carnegie Mellon, depending on the network scenario.
The DHCP based technique outlined above requires no extra infrastructure, just extra configuration, so it is what we use for most of our campus wired networks. We use the same setup as our registration helper network, so our internal name for the DHCP based quarantine system is called QuickReg. An unknown or banned client gets an address in 1918 space and can only access our abuse tracking, patch download and network registration systems.
Following up my own post. I know, its always bad ettiquete, but I forgot to mention something. We're also using an active suspension mechanism for these networks to block clients with current valid DHCP leases instantly. We use Unicast Reverse Path Filtering (*) and /32 host routes injected into our OSPF cloud via quagga (ospf routing daemon on a unix server). This means a suspended host loses all network connectivity immediately, until they re-dhcp, at which point they'll have a rfc1918 address and have access to the quarantine network. This also handles the occasional statically configured host. We can also use this system to filter external hosts without having to manipulate border router acls frequently. (*): For anyone who doesn't know, URPF is essentially a way to do automatic acls, comparing the source IP of on an incoming packet to the routing table to verify the packet should have come from this interface. With the right hardware this is significantly cheaper then acl processing. And its certainly easier to maintain. And by injecting a /32 null route into the route table you can cause a host's local router to start discarding all traffic from that IP. -David Nolan Network Software Designer Computing Services Carnegie Mellon University
David Nolan wrote: <snip>
(*): For anyone who doesn't know, URPF is essentially a way to do automatic acls, comparing the source IP of on an incoming packet to the routing table to verify the packet should have come from this interface. With the right hardware this is significantly cheaper then acl processing. And its certainly easier to maintain. And by injecting a /32 null route into the route table you can cause a host's local router to start discarding all traffic from that IP.
<snip sig> Yeah, but it's not near as fun as dynamic acls updated via a script monitoring flow logs in real-time. It's definitely easier to implement, though. For people utilizing RBE/dhcp combo on Cisco routers, it is also possible to just remove the /32 route that was dynamically created which will kill traffic until the customer requests dhcp again, which will by that time place them in the quarantine. One advantage to temp route removal is that it requires no cleanup. Just make sure you don't wipe out your permanent static routes. -Jack
--On Wednesday, March 01, 2006 07:54:17 -0600 Jack Bates <jbates@brightok.net> wrote:
David Nolan wrote: <snip>
(*): For anyone who doesn't know, URPF is essentially a way to do automatic acls, comparing the source IP of on an incoming packet to the routing table to verify the packet should have come from this interface. With the right hardware this is significantly cheaper then acl processing. And its certainly easier to maintain. And by injecting a /32 null route into the route table you can cause a host's local router to start discarding all traffic from that IP.
<snip sig>
Yeah, but it's not near as fun as dynamic acls updated via a script monitoring flow logs in real-time. It's definitely easier to implement, though.
Interesting... Thats actually basically what we were doing before, but phased out in favor of the URPF & host routes approach. We felt the URPF approach was much cleaner, and more efficient. A routing table lookup is more efficient then a acl processing, particulary if you have significant numbers of rou and solved some problems we were having. It also solved some issues we had, including keeping dynamic acls synchronized betwen two redundant routers (HSRP pairs and/or redundant border routers). -David
On Wed, 1 Mar 2006, David Nolan wrote:
Yeah, but it's not near as fun as dynamic acls updated via a script monitoring flow logs in real-time. It's definitely easier to implement, though.
Interesting... Thats actually basically what we were doing before, but phased out in favor of the URPF & host routes approach. We felt the URPF approach was much cleaner, and more efficient. A routing table lookup is more efficient then a acl processing, particulary if you have significant numbers of rou and solved some problems we were having. It also solved some issues we had, including keeping dynamic acls synchronized betwen two redundant routers (HSRP pairs and/or redundant border routers).
I think when he said fun, he meant 'masochistic and nerve wracking, in a vaguely entertaining because we have scripts issuing and removing ACLs from our routing core kind of way.' I've built reactive firewalls before, but even I'd be leery of a reactive ACL implementation. /32 null route injection is far far easier to manage. =) - billn
Date: Tue, 28 Feb 2006 18:50:29 +0000 (GMT) From: Christopher L. Morrow <christopher.morrow@verizonbusiness.com> To: nanog@merit.edu Subject: Re: Quarantine your infected users spreading malware
On Tue, 28 Feb 2006, Jim Segrave wrote:
www.quarantainenet.nl
It puts them in a protected environment where they can get cleaned up on-line without serious risk of re-infection. They can pop their e-mail, reply via webmail, but they can't connect to anywhere except a list of update sites.
there was little in the way of 'how' in the link above though :(
From what I know (from presentations of the folk behind Qnet, and talks with people actually using it) is that they have a sort of "export" module, which allows you to either output the IP's, or parse
Well, it's very much dependant on your own network. them such that you get a crafted DHCP entry, or special MAC address based "alternate VLAN" statement for on a switch etc. They have templates for a bunch of things, but whether or not one of those templates is applicable or even useful in your own network remains te be seen each and every time. The main strength of Qnet is the detection, and even better, the way of allowing people to clean themselves, and then get back on the net. Having a helpdesk tell (different) people the same line over and over again gets tedious. Putting the effort into making a nice explanatory webpage get so much more "return on investment"... ;) Kind regards, JP Velders
On Wed, 1 Mar 2006, JP Velders wrote:
Date: Tue, 28 Feb 2006 18:50:29 +0000 (GMT) From: Christopher L. Morrow <christopher.morrow@verizonbusiness.com> To: nanog@merit.edu Subject: Re: Quarantine your infected users spreading malware
On Tue, 28 Feb 2006, Jim Segrave wrote:
www.quarantainenet.nl
It puts them in a protected environment where they can get cleaned up on-line without serious risk of re-infection. They can pop their e-mail, reply via webmail, but they can't connect to anywhere except a list of update sites.
there was little in the way of 'how' in the link above though :(
From what I know (from presentations of the folk behind Qnet, and talks with people actually using it) is that they have a sort of "export" module, which allows you to either output the IP's, or parse
Well, it's very much dependant on your own network. them such that you get a crafted DHCP entry, or special MAC address based "alternate VLAN" statement for on a switch etc.
which is fabulous for those of you with ethernet... without ethernet most of these solutions fall on their faces and die the horrid death of an enterprise product :( Now, they say: "Works great on carrier networks"... my question was "how" and "perhaps with a little less hand-waviness please?"
They have templates for a bunch of things, but whether or not one of those templates is applicable or even useful in your own network remains te be seen each and every time.
and none of these so called templates is available or described on their public documentation :( There are a few ways to skin this cat, depending upon architecture one might even work. Without knowing the possible methodologies available it's not helpful :(
The main strength of Qnet is the detection, and even better, the way of allowing people to clean themselves, and then get back on the net. Having a helpdesk tell (different) people the same line over and over again gets tedious. Putting the effort into making a nice explanatory webpage get so much more "return on investment"... ;)
agreed, punting this problem to the helpdesk makes the helpdesk manager grab his gun(s) and find the security wonk that put a hurtin' on his numbers :) Also, it costs lots of money, which isn't generally a good plan.
Christopher L. Morrow wrote: <snip>
agreed, punting this problem to the helpdesk makes the helpdesk manager grab his gun(s) and find the security wonk that put a hurtin' on his numbers :) Also, it costs lots of money, which isn't generally a good plan.
Do you find that web redirection actually stems the flow of calls to the helpdesk? We find that anything out of the normal usually results in a customer calling the helpdesk just because they weren't expecting it. We found this to be true of email notifications as well. The other issue is, of course, differing what we are doing with those thousands of annoying ads that make users believe they are infected. -Jack
--On Wednesday, March 01, 2006 11:42:01 -0600 Jack Bates <jbates@brightok.net> wrote:
Do you find that web redirection actually stems the flow of calls to the helpdesk? We find that anything out of the normal usually results in a customer calling the helpdesk just because they weren't expecting it. We found this to be true of email notifications as well.
We believe it does help to an extent. But more importantly to us the same system that sent the notices and quarantined the host also is tracking the incident. Its visible to the help desk staff and the security staff, and searching there first when a user contacts us is standard procedure. Prior to this system we were keeping track of suspended machines by hand or via email. In the summer of 2003, when the big windows RPC vulnerability was out, and both Blaster and Welchia happened, we knew right away that we needed a system to track the *hundreds* of suspend/restore requests we were processing. First it was just a tracking system, then it became a full automated notification and suspension system. One of the things we do is send vulnerability notices for large scale OS vulnerabilities. For example, for the Windows Print Spooler vulnerability, MS05-043, we scan our network multiple times a day and send notices to the owners of vulnerable machines. The user/admin then has 24 hours to patch the machine and use the web app to tell us they did. If they don't do so the machine is suspended. Once suspended they can still use the web app to restore themselves. However if we find a machine is still unpatched after we've been told it was patched we immediately suspend it.
The other issue is, of course, differing what we are doing with those thousands of annoying ads that make users believe they are infected.
Well, once they're quarantined they should stop getting those ads and just get your quarantine notice, so that should be different, right? -David
On Wed, 1 Mar 2006, Jack Bates wrote:
Christopher L. Morrow wrote: <snip>
agreed, punting this problem to the helpdesk makes the helpdesk manager grab his gun(s) and find the security wonk that put a hurtin' on his numbers :) Also, it costs lots of money, which isn't generally a good plan.
Do you find that web redirection actually stems the flow of calls to the helpdesk? We find that anything out of the normal usually results in a
don't know, we don't do it except for some internal things I think... I just know what our customer support folks do if I screw up and make a bunch of customers call in :)
On Wed 01 Mar 2006 (11:42 -0600), Jack Bates wrote:
Christopher L. Morrow wrote: <snip>
agreed, punting this problem to the helpdesk makes the helpdesk manager grab his gun(s) and find the security wonk that put a hurtin' on his numbers :) Also, it costs lots of money, which isn't generally a good plan.
Do you find that web redirection actually stems the flow of calls to the helpdesk? We find that anything out of the normal usually results in a customer calling the helpdesk just because they weren't expecting it. We found this to be true of email notifications as well. The other issue is, of course, differing what we are doing with those thousands of annoying ads that make users believe they are infected.
Yes, it reduces, but does not stop the number of calls. More importantly, because the customer can still access sites such as MS update, Norton, McAfee, Housecall etc, even while quarantined, those people who call the helpdesk can get directed to the "how to fix it page" rapidly, so the calls stay shorter. -- Jim Segrave jes@nl.demon.net
On Wed 01 Mar 2006 (16:33 +0000), Christopher L. Morrow wrote:
On Wed, 1 Mar 2006, JP Velders wrote:
Date: Tue, 28 Feb 2006 18:50:29 +0000 (GMT) From: Christopher L. Morrow <christopher.morrow@verizonbusiness.com> To: nanog@merit.edu Subject: Re: Quarantine your infected users spreading malware
On Tue, 28 Feb 2006, Jim Segrave wrote:
www.quarantainenet.nl
It puts them in a protected environment where they can get cleaned up on-line without serious risk of re-infection. They can pop their e-mail, reply via webmail, but they can't connect to anywhere except a list of update sites.
there was little in the way of 'how' in the link above though :(
From what I know (from presentations of the folk behind Qnet, and talks with people actually using it) is that they have a sort of "export" module, which allows you to either output the IP's, or parse
Well, it's very much dependant on your own network. them such that you get a crafted DHCP entry, or special MAC address based "alternate VLAN" statement for on a switch etc.
which is fabulous for those of you with ethernet... without ethernet most of these solutions fall on their faces and die the horrid death of an enterprise product :( Now, they say: "Works great on carrier networks"... my question was "how" and "perhaps with a little less hand-waviness please?"
You could have answered your own questions, for your own network, in the same amount of time as writing these postings to nanog, by asking the company. -- Jim Segrave jes@nl.demon.net
On Tue, 21 Feb 2006, Valdis.Kletnieks@vt.edu wrote:
If you're talking about a compulsory software solution, why not, as an ISP, go back to authenticated activity? Distribute PPPOE clients mated with common anti-spyware/anti-viral tools. Pull down and update signatures *every time* the user logs in, and again periodically while the user is logged in (for those that never log out). Require these safeguards to be active before they can pass the smallest traffic.
Cost prohibitive.. In order to do that you'll need licenses from the AV companies..
Oddly enough, AOL and several other large providers seem to have no problems advertising some variant on 'free A/V software'.
When referring to AOL customers, though, you're talking about a target market that is accustomed to being offered a bundled package, and for lack of a better term, doing what it's told. Largely, AOL users aren't the problem. Comcast, Cox, Adelphia, and similiar providers with raw IP consumers are the problem.[1] A la carte services are all good and well for the end user, but it's a double edged sword in that they're good for the botnet crews, too. I used to sneer at offerings like AOL or Compuserv, because they weren't what I needed. Now, I'm actually kind of glad they exist because some users clearly need the training wheels. This is as much of a social problem as it is a technical one. I'm starting to understand the perspective of a legislative heavy federal government that has to pass laws to protect folks who are pretty much ignorant of the problem. - billn [1] I don't point those out because of specific problems, I point them out to describe service offering styles and network architecture. I have no interest in detailing why provider X sucks, or talking to your lawyers about it.
On Tue, 21 Feb 2006, Jason Frisvold wrote:
On 2/21/06, Bill Nash <billn@odyssey.billn.net> wrote:
If you're talking about a compulsory software solution, why not, as an ISP, go back to authenticated activity? Distribute PPPOE clients mated with common anti-spyware/anti-viral tools. Pull down and update signatures *every time* the user logs in, and again periodically while the user is logged in (for those that never log out). Require these safeguards to be active before they can pass the smallest traffic.
Cost prohibitive.. In order to do that you'll need licenses from the AV companies..
Big deal. You're talking about volume licensing at that point, and offering vendors an opportunity to compete to get on every desktop in your customer base. That's a big stick to negotiate with, especially if you're an Earthlink or AOL.
The change in traffic flow would necessitate some architecture kung fu, maybe even AOL style, but you'd have the option of selectively picking out reported malicious/infected users (*cough* ThreatNet *cough*) and routing them through packet inspection frameworks on a case by case basis. Quite possibly, you could even automate that and the users would never be the wiser.
And then the privacy zealots would be livid.. Silently re-routing traffic like that.. How dare you suggest such a ... wait.. hrm.. The internet basically does this already.. I wonder if the zealots are aware of that.. :)
Yeah, the privacy zealots, of which I'm one, don't have much of a leg to stand on, since as the direct service provider, you'd be directly within AUP/Contractually provided rights to do so, under that particular service model. They can't ding you for being active in your *response* to complaints about malicious activity sourced from your network, and taking the time to verify it. So long as you're keeping their personal information out of the hands of others, they don't have much to bitch about. The ISPs win because they've got ready means to tie complaints directly back to an active customer, AND verify the complaint. Consumers win because they've got cheap anti-virus they still don't have to do anything about. The internet wins because ISPs are sharing non-personally identifying information about naughty behaviour and maybe increasing the mean TTL for new Windows machines. In the long term, privacy advocates win because networks have implemented active responses to attacks that routinely lead to identity theft. The biggest hole I see in this concept is home routers that do NAT (linksys, linux boxes, etc). While capable of PPPOE, you can't quite mandate the A/V clients. You still have the option of doing packet inspection, which is still better than nothing. - billn
On 2/21/06, Bill Nash <billn@odyssey.billn.net> wrote:
Big deal. You're talking about volume licensing at that point, and offering vendors an opportunity to compete to get on every desktop in your customer base. That's a big stick to negotiate with, especially if you're an Earthlink or AOL.
Agreed. And with that, the little guys go away.
Yeah, the privacy zealots, of which I'm one, don't have much of a leg to stand on, since as the direct service provider, you'd be directly within AUP/Contractually provided rights to do so, under that particular service model. They can't ding you for being active in your *response* to complaints about malicious activity sourced from your network, and taking the time to verify it. So long as you're keeping their personal information out of the hands of others, they don't have much to bitch about.
Agreed, but without publishing the exact procedures, protocols, etc, they can always complain that something might be happening.. Don't get me wrong, I'm just as much for privacy as most of the "zealots", but there is a point at which there has to be an acceptable risk.
The ISPs win because they've got ready means to tie complaints directly back to an active customer, AND verify the complaint. Consumers win because they've got cheap anti-virus they still don't have to do anything about. The internet wins because ISPs are sharing non-personally identifying information about naughty behaviour and maybe increasing the mean TTL for new Windows machines. In the long term, privacy advocates win because networks have implemented active responses to attacks that routinely lead to identity theft.
I wish everyone had this view. Fixing, or at least patching, this problem would help out a lot in the long run. But there's a lot to be done to handle it. An ISP can deal with it themselves or, more often than not, can ignore it. As I was saying before, if there were some sort of standards body that set forth a best practices guide of some sort, that might go a long way. Education for the end-user is key here too. Educate them to understand what precautions are in place at the ISP level, and what they can do to protect themselves. I think it's gotten better in recent years, despite the increase in viral activity. I think the increase is due to better propogation techniques rather then hordes of dumb users.
The biggest hole I see in this concept is home routers that do NAT (linksys, linux boxes, etc). While capable of PPPOE, you can't quite mandate the A/V clients. You still have the option of doing packet inspection, which is still better than nothing.
Hrm.. Unless some sort of shim was required on the end-user computer.. something transparent that merely identified itself in the background to the central authority and verified signatures and the like..
- billn
-- Jason 'XenoPhage' Frisvold XenoPhage0@gmail.com
Heya, Sorry about continuing this thread... I noticed a few people discussing this topic and wondering about new ways to look at quarantining hosts. There's a working group within the US Internet2 community that's been working on a generalized architecture and set of white-papers that our member institutions can share. If you're interested, check out the two drafts that we have so far (SALSA-Netauth working group): Architecture for Automating Network Policy (PDF) http://security.internet2.edu/netauth/docs/internet2-salsa-netauth-architect... Strategies for Automating Network Policy Enforcement http://security.internet2.edu/netauth/docs/internet2-salsa-netauth-policy-en... We'd welcome any thoughts, criticism, complaints, praise, etc... Eric :)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Bill Nash wrote:
On Tue, 21 Feb 2006, Michael.Dillon@btradianz.com wrote:
Why not just bypass them and go direct to the unwashed masses of end users? Offer them a free windows infection blocker program that imposes the quarantine itself locally on the user's machine. This program
Offering them free software won't work to the levels you want. At first, you'll get a response, because consumers always jump at free shiny things, until something happens that makes them not like it anymore, and then they'll dig in and never use it again. If you want to get this kind of filtering into your core, you have a need to get this to a compulsory level for access.
I don't think there's any disagreement as to the roots of this problem: - Modern users are generally clueless. - Most don't have firewalls or even the most basic of protections. - Getting tools deployed where they need to be most is the hardest.
With that said..
If you're talking about a compulsory software solution, why not, as an ISP, go back to authenticated activity? Distribute PPPOE clients mated with common anti-spyware/anti-viral tools. Pull down and update signatures *every time* the user logs in, and again periodically while the user is logged in (for those that never log out). Require these safeguards to be active before they can pass the smallest traffic.
The change in traffic flow would necessitate some architecture kung fu, maybe even AOL style, but you'd have the option of selectively picking out reported malicious/infected users (*cough* ThreatNet *cough*) and routing them through packet inspection frameworks on a case by case basis. Quite possibly, you could even automate that and the users would never be the wiser.
- ----------------- - From my past discussion at nanog sessions, it appears this sink-hole like process has been extremely helpful for AOL. Maybe Vijay from AOL could chime in and enlighten us or folks could look at the archives. regards, /virendra
- billn
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFD+4sWpbZvCIJx1bcRAq2oAJ4z9xmrBYwppdTpYTtLkNow+N17ZQCeJsnE xr6y99lCbEAnO60SUEtv9Xk= =av1X -----END PGP SIGNATURE-----
We're one of those user/broadband ISPs, and I have to agree with the other commentary that to set up an appropriate filtering system (either user, port, or conversation) across all our internet access platforms would be difficult. Put it on the edge and you miss the intra-net traffic, put it in the core and you need a box on every router, which for a larger or graphically distributed ISPs could be cost-prohibitive. In relation to that ThreatNet model, we just could wish there was a place we could quickly and accurately aggregate information about the bad things our users are doing -- a combination of RBL listings, abuse@, SenderBase, MyNetWatchman, etc. We don't have our own traffic monitoring and analysis system in place, and even if we did, I'm afraid our work would still be very reactionary. And for the record, we are one of those ISPs that blocks ports 139 and 445 on our DSLAM and CMTS, and we've not received one complaint, but I'm confident it has cut down on a host of infections. Frank -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Gadi Evron Sent: Monday, February 20, 2006 3:41 PM To: nanog@merit.edu Subject: Quarantine your infected users spreading malware Many ISP's who do care about issues such as worms, infected users "spreading the love", etc. simply do not have the man-power to handle all their infected users' population. It is becoming more and more obvious that the answer may not be at the ISP's doorstep, but the ISP's are indeed a critical part of the solution. What their eventual role in user safety will be I can only guess, but it is clear (to me) that this subject is going to become a lot "hotter" in coming years. Aunty Jane (like Dr. Alan Solomon (drsolly) likes to call your average user) is your biggest risk to the Internet today, and how to fix the user non of us have a good idea quite yet. Especially since it's not quite one as I put in an Heinlein quote below. Some who are user/broadband ISP's (not say, tier-1 and tier-2's who would be against it: "don't be the Internet's Firewall") are blocking ports such as 139 and 445 for a long time now, successfully preventing many of their users from becoming infected. This is also an excellent first step for responding to relevant outbreaks and halting their progress. Philosophy aside, it works. It stops infections. Period. Back to the philosophy, there are some other solutions as well. Plus, should this even be done? One of them has been around for a while, but just now begins to mature: Quarantining your users. Infected users quarantine may sound a bit harsh, but consider; if a user is indeed infected and does "spread the joy" on your network as well as others', and you could simply firewall him (or her) out of the world (VLAN, other solutions which may be far better) letting him (or her) go only to a web page explaining the problem to them, it's pretty nifty. As many of us know, handling such users on tech support is not very cost-effective to ISP's, as if a user makes a call the ISP already losses money on that user. Than again, paying abuse desk personnel just so that they can disconnect your users is losing money too. Which one would you prefer? Jose (Nazario) points to many interesting papers on the subject on his blog: http://www.wormblog.com/papers/ Is it the ISP's place to do this? Should the ISP do this? Does the ISP have a right to do this? If the ISP is nice enough to do it, and users know the ISP might. Why not? This (as well as port blocking) is more true for organizations other than ISP's, but if they are indeed user/broadband ISP's, I see this as both the effective and the ethical thing to do if the users are notified this might happen when they sign their contracts. Then all the "don't be the Internet's firewall" debate goes away. I respect the "don't be the Internet's firewall issue", not only for the sake of the cause but also because friends such as Steven Bellovin and other believe in them a lot more strongly than I do. Bigger issues such as the safety of the Internet exist now. That doesn't mean user rights are to be ignored, but certainly so shouldn't ours, especially if these are mostly unaffected? I believe both are good and necessary solutions, but every organization needs to choose what is best for it, rather than follow some pre-determined blueprint. What's good for one may be horrible for another. "You don't approve? Well too bad, we're in this for the species boys and girls. It's simple numbers, they have more and every day I have to make decisions that send hundreds of people, like you, to their deaths." -- Carl Jenkins, Starship Trooper, the movie. I don't think the second part of the quote is quite right (to say the least), but I felt bad leaving it out, it's Heinlein after all... anyone who claims he is a fascist though will have to deal with me. :) This isn't only about users, it's about the bad guys and how they out-number us, too. They have far better cooperation to boot. There are several such products around and they have been discussed here on NANOG before, but I haven't tried them myself as of yet, so I can't really recommend any of them. Can you? I'll update on these as I find out more on: http://blogs.securiteam.com This write-up can be found here: http://blogs.securiteam.com/index.php/archives/312 Gadi. -- http://blogs.securiteam.com/ "Out of the box is where I live". -- Cara "Starbuck" Thrace, Battlestar Galactica.
Frank Bulk wrote:
We're one of those user/broadband ISPs, and I have to agree with the other commentary that to set up an appropriate filtering system (either user, port, or conversation) across all our internet access platforms would be difficult. Put it on the edge and you miss the intra-net traffic, put it in the core and you need a box on every router, which for a larger or graphically distributed ISPs could be cost-prohibitive.
I have a question here, do you have repeat offenders in your abuse desk who are of the malware-sort rather than bad people? Can these be put in a specific group?
In relation to that ThreatNet model, we just could wish there was a place we could quickly and accurately aggregate information about the bad things our users are doing -- a combination of RBL listings, abuse@, SenderBase, MyNetWatchman, etc. We don't have our own traffic monitoring and analysis system in place, and even if we did, I'm afraid our work would still be very reactionary.
And for the record, we are one of those ISPs that blocks ports 139 and 445 on our DSLAM and CMTS, and we've not received one complaint, but I'm confident it has cut down on a host of infections.
Would you happen to have statistics on how far it did/didn't help reduce abuse reports, tech support calls, etc.? Thanks!
Frank
Gadi.
-----Original Message----- From: Gadi Evron [mailto:ge@linuxbox.org] Sent: Monday, February 20, 2006 7:35 PM To: frnkblk@iname.com Cc: nanog@merit.edu Subject: Re: Quarantine your infected users spreading malware Frank Bulk wrote:
We're one of those user/broadband ISPs, and I have to agree with the other commentary that to set up an appropriate filtering system (either user, port, or conversation) across all our internet access platforms would be difficult. Put it on the edge and you miss the intra-net traffic, put it in the core and you need a box on every router, which for a larger or graphically distributed ISPs could be cost-prohibitive.
I have a question here, do you have repeat offenders in your abuse desk who are of the malware-sort rather than bad people? Can these be put in a specific group? FB> Most of the repeat offenders tend to be people who lack the ability to choose website judiciously, to put it kindly. But when we encourage them to get a pop-up blocker, update their antivirus (either the whole program or definitions), and install a firewall (Windows XP or cheap NAT router), the problem usually fades away. Most "just didn't know" that their computer was spewing forth spam or viruses, being used as a proxy, or part of some kind of botnet.
In relation to that ThreatNet model, we just could wish there was a place we could quickly and accurately aggregate information about the bad things our users are doing -- a combination of RBL listings, abuse@, SenderBase, MyNetWatchman, etc. We don't have our own traffic monitoring and analysis system in place, and even if we did, I'm afraid our work would still be very reactionary.
And for the record, we are one of those ISPs that blocks ports 139 and 445 on our DSLAM and CMTS, and we've not received one complaint, but I'm confident it has cut down on a host of infections.
Would you happen to have statistics on how far it did/didn't help reduce abuse reports, tech support calls, etc.? FB> We don't look at the logs for entries regarding ports 139/445, but when we last looked it was a few unique IP addresses per day. And due our size, we have no idea how much it reduced abuse reports. It's been in place for several years.
Frank
Gadi.
On Mon, 2006-02-20 at 23:40:48 +0200, Gadi Evron proclaimed... [snip]
I'll update on these as I find out more on: http://blogs.securiteam.com
This write-up can be found here: http://blogs.securiteam.com/index.php/archives/312
Ah yes, the old self-promotion trick. You know, I get some ads for C1@lis that sound pretty good until I have to click on thier link to get more information. Moderators: doesn't this border on spam?
eric-list-nanog@catastrophe.net wrote:
On Mon, 2006-02-20 at 23:40:48 +0200, Gadi Evron proclaimed...
[snip]
I'll update on these as I find out more on: http://blogs.securiteam.com
This write-up can be found here: http://blogs.securiteam.com/index.php/archives/312
Ah yes, the old self-promotion trick. You know, I get some ads for C1@lis that sound pretty good until I have to click on thier link to get more information.
The information, quite a bit of it, comes before the link. If you'd like I can send it you you again. Thanks! Gadi. -- http://blogs.securiteam.com/ "Out of the box is where I live". -- Cara "Starbuck" Thrace, Battlestar Galactica.
On Tue, Feb 21, 2006 at 07:17:38AM +0200, Gadi Evron wrote:
eric-list-nanog@catastrophe.net wrote:
On Mon, 2006-02-20 at 23:40:48 +0200, Gadi Evron proclaimed...
[snip]
I'll update on these as I find out more on: http://blogs.securiteam.com
This write-up can be found here: http://blogs.securiteam.com/index.php/archives/312
Ah yes, the old self-promotion trick. You know, I get some ads for C1@lis that sound pretty good until I have to click on thier link to get more information.
The information, quite a bit of it, comes before the link. If you'd like I can send it you you again. Thanks!
Gadi.
It appears the quality of nanog mailing list is becoming on the par with that of Full-Disclosure. James
participants (23)
-
Andy Davidson -
Bill Nash -
Christopher L. Morrow -
David Nolan -
Eric Gauthier -
eric-list-nanog＠catastrophe.net -
Frank Bulk -
Gadi Evron -
Jack Bates -
James -
Jason Frisvold -
Jim Segrave -
JP Velders -
Larry Smith -
Michael Loftis -
Michael Painter -
Michael.Dillon＠btradianz.com -
Niels Raijer -
PC -
Robert E.Seastrom -
Sean Donelan -
Valdis.Kletnieks＠vt.edu -
Vicky Røde