Regarding today's Charlie Foxtrot: The problem that occured today appears to be caused by someone without clues redistributing their BGP (which contains a set of routes of people clueless enough to trust them to run an exchange point of sorts) into their IGP, back into the BGP tables. It is also possible that the lack of clues was involved in choosing a vendor that exhibited a bug similar to this. This caused the AS paths to be lost, allowing loopish things to occur. This caused router tables to flare, and improper paths to be propogated and used. Lots of churn for a stressed Internet. It seems to have propogated rather evilly by hiding in edge ASes and seeping back in via edge ASes to other providers (ie it comes in provider 1, and then goes out provider 2). This also seems to have been propogated by subasish behaviour of certain providers running confederations and multiple ASes. RSes mail re: email not being an effective mode for conversing is somewhat correct, nonetheless, I continued to receive email from NANOG through the issue. As well, several larger ISP folk were on the phone diagnosing the problem and taking action to minimize the effect. Enke's mail stating that filters could have prevented this is somewhat accurate, but it is not apparent that filtering folk were not at risk to this event. I'd be curious to hear what ANS saw. The duration appears to have been ~11:30 AM EDT to ~2:00 PM EDT. At no time was this a catestrophic Internet breaking event, so relax Bob. This could have been prevented by two things: 1/ Strict adherence to peer-peer filters. 2/ Someone not doing something stupid. I think we are getting too big to rely on 2 not happening. Yet the sense of liberty yearns for 1 not to happen. But it will. This will help accelerate it. -alan