On 8 Feb 2000, Sean Donelan wrote:
Date: 8 Feb 2000 03:25:36 -0800 From: Sean Donelan <sean@donelan.com> To: nanog@merit.edu Subject: Yahoo! Lessons Learned
As much as I enjoy finding out about Yahoo & GlobalCenter issues by reading the newswires, I wonder if there are any lessons we can learn from these events. Or was this not big enough to get attention of upper management?
Possibly.
Was there something Yahoo!, GlobalCeneter or other providers could have done, either individually or in cooperation, to prevent the
problem?
Yes. One of the emails sent in, mentioned that a network they work with or for was being utilized as an amplifier. Each network that have gateway routers should ensure that they disallow IP broadcasts. It was mentioned that this was a co-ordinated attack. That meant a bit of planning and access to various machines. As to the number of attackers only Yahoo's internal people may know. Even then it may have only been one individual with a script that accessed many locations at one time and initiated the commands. There is the ability to do such an attack. The reality of "stay connected 24/7" at the household level with highspeed internet, makes the possibility of this attack more of a multi level victom attack. Home users do not know that they are leaving the door open to exploitation with simple Window's shares. Savy people gain access to the cable and dsl modem user's PCs and then launch their attacks. Small utilities are put in place to make it easier to find the exploited machines. Thus creating a network of available attack, harder to track connections. Education is a tool that can be used to inform customers. If each node on the Internet takes care of it's own doors then there will be less available launching pads. Thus making it a bit simpler to track an attack. Who or what will do the education is a question. Who are the responsible parties if no education is taken or given? To me, the responsiblity question is a nitemare at best. I just hope Yahoo's unfortunate incident opens some eyes, some lines of communication and education. K. Graham Network Analyst, CCNA kim@penguin-power.com
"K. Graham" wrote:
On 8 Feb 2000, Sean Donelan wrote:
Date: 8 Feb 2000 03:25:36 -0800 From: Sean Donelan <sean@donelan.com> To: nanog@merit.edu Subject: Yahoo! Lessons Learned
As much as I enjoy finding out about Yahoo & GlobalCenter issues by reading the newswires, I wonder if there are any lessons we can learn from these events. Or was this not big enough to get attention of upper management?
Possibly.
Was there something Yahoo!, GlobalCeneter or other providers could have done, either individually or in cooperation, to prevent the
problem?
Yes. One of the emails sent in, mentioned that a network they work with or for was being utilized as an amplifier. Each network that have gateway routers should ensure that they disallow IP broadcasts.
Please refer to RFC2644/BCP34 on the subject of directed broadcasts. This RFC recommends router vendors disable directed broadcasts by default. It also recommends ISPs disable directed broadcast on ALL routers. In light of the recent events, it would be good to see a concerted effort made by everyone to ensure this has been done. Of course as Paul has mentioned, we wrote RFC 2267 several years ago to address this very issue. I strongly encourage folks to take a hard look at ingress filtering. Hardware vendors have implemented features in dialup servers and routers which can help. While implementing these measures may not directly benefit your network, doing so may thwart an attack against someone else's net. Tomorrow, the roles could be reversed. As with many areas of managing the Internet, cooperation is key. -- ----------------------------------------------------------------- Daniel Senie dts@senie.com Amaranth Networks Inc. http://www.amaranthnetworks.com
At Tuesday 11:01 PM 2/8/00 , Daniel Senie wrote:
Please refer to RFC2644/BCP34 on the subject of directed broadcasts. This RFC recommends router vendors disable directed broadcasts by default. It also recommends ISPs disable directed broadcast on ALL routers. In light of the recent events, it would be good to see a concerted effort made by everyone to ensure this has been done.
I recall that SprintLink had some, uhm, plans to put ingress (and egress?) filters on all interfaces facing dedicated customers that were not multi-homed. This came after realization that education of the end-user was a fruitless and herculian task: Network smarts are virtually non-existent in IT departments, and even loads of smaller ISPs everywhere. Whatever became of this project ? At what traffic level (across the entire box) do Cisco 7{0;2;5}00 routers with RSP{2;4} cards fall over and die because of CPU load?
Of course as Paul has mentioned, we wrote RFC 2267 several years ago to address this very issue. I strongly encourage folks to take a hard look at ingress filtering. Hardware vendors have implemented features in dialup servers and routers which can help.
Without wanting to bash my favorite NAS vendor: I have asked for 'ip verify unicast reverse path' in their boxes as much as 2+ years ago. They recently admitted to having no record of this request, and it has just now become a request for engineering. Vendors do not have their focus on security, just like most everyone else in the Internet "industry". Skating on thin ice has a price...
While implementing these measures may not directly benefit your network, doing so may thwart an attack against someone else's net. Tomorrow, the roles could be reversed. As with many areas of managing the Internet, cooperation is key.
Like the kind of cooperation that is making people close their open SMTP relays voluntarily because closed relays are A Good Thing <tm> or are a BCP? That always and only worked with threats of loss of connectivity or humiliation through public exposure. Some networks have taken it upon themselves to shield their customers from such well-deserved scrutiny from the outside (Hi Dave!). Nothing will change until Yahoo decides that the legitimate operators of the Trinoo/Tribe/whatever slaves have acted with reckless neglect by not keeping their system secured with vendor-issued patches. But when they do, duck and cover for the wave of lawyers hitting like an Ion-storm. bye,Kai
Kai Schlichting writes:
I recall that SprintLink had some, uhm, plans to put ingress (and egress?) filters on all interfaces facing dedicated customers that were not multi-homed. This came after realization that education of the end-user was a fruitless and herculian task: Network smarts are virtually non-existent in IT departments, and even loads of smaller ISPs everywhere. Whatever became of this project ?
At what traffic level (across the entire box) do Cisco 7{0;2;5}00 routers with RSP{2;4} cards fall over and die because of CPU load?
This becomes rather difficult to do properly when a large percentage of an ISPs customer base are multihomed. The unicast RPF check knob does not handle this situation so you are left generating access lists which has different scaling issues to contend with. Still doable but more difficult. -Hank
At Tuesday 11:01 PM 2/8/00 , Daniel Senie wrote:
Please refer to RFC2644/BCP34 on the subject of directed broadcasts. This RFC recommends router vendors disable directed broadcasts by default. It also recommends ISPs disable directed broadcast on ALL routers. In light of the recent events, it would be good to see a concerted effort made by everyone to ensure this has been done.
I recall that SprintLink had some, uhm, plans to put ingress (and egress?) filters on all interfaces facing dedicated customers that were not multi-homed. This came after realization that education of the end-user was a fruitless and herculian task: Network smarts are virtually non-existent in IT departments, and even loads of smaller ISPs everywhere. Whatever became of this project ?
If you sell a customer a circuit and they do nothing more than default to you with address space you provide, this is easy. If a customer talks BGP to you and you require them to submit prefixs to you for filtering (which should generally be the policy if you want any kind of protection against having 7 coppies of the internet routing tables in your network), this is also easy. You already know which netblocks can be sourced from that connection. If the CPU can handle it, there is no good reason not to do it. ---------------------------------------------------------------------- Wayne Bouchard [Immagine Your ] web@typo.org [Company Name Here] Network Engineer ----------------------------------------------------------------------
The reality of "stay connected 24/7" at the household level with highspeed internet, makes the possibility of this attack more of a multi level victom attack. Home users do not know that they are leaving the door open to exploitation with simple Window's shares. Savy
The other thing to note is that average BW per computer is rising rapidly. Which means that even a single box in someone's home (windows OR UNIX) can be "usefull" as a source if its compromised. The problem here is that there really is nothing that can be done to help this since its a matter of keeping the server secure. Do you know anyone who wants to train 250 million people in computer security? And of those 250 million, how many of them even care? ---------------------------------------------------------------------- Wayne Bouchard [Immagine Your ] web@typo.org [Company Name Here] Network Engineer ----------------------------------------------------------------------
participants (6)
-
Dan Hollis -
Daniel Senie -
Henry Kilmer -
K. Graham -
Kai Schlichting -
Wayne Bouchard