To circle back to the original post... Level 3 does have multiple routes out of Tampa. They just apparently don't use them all for their transit service. Why not? ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com ----- Original Message ----- From: "valdis kletnieks" <valdis.kletnieks@vt.edu> To: "Baldur Norddahl" <baldur.norddahl@gmail.com> Cc: nanog@nanog.org Sent: Sunday, May 20, 2018 5:43:42 PM Subject: Re: Curiosity about AS3356 L3/CenturyLink network resiliency (in general) On Sun, 20 May 2018 09:16:25 +0200, Baldur Norddahl said:
He is complaining about AS3356 in specific and claiming they COULD reroute around it but choose not to. This leads me to assume there are alternatives. Two places, Miami and Texas, are mentioned and that a double fault, one in Miami and another in Texas would bring down the network. I am from Europe, but am I to believe that Miami and Texas (or anywhere between those two) are served by only two fiber conduits?
There's a difference between "route around it by flipping some BGP magic" and "route around it by digging a ditch to a third city". The fact that other places have other conduits doesn't change the fact that a given city may only have two physical conduits handy. Often, there are other *possible* paths that could be built out, but other providers have looked at the cost of digging a ditch from the city, out a third path, to their closest POP, and decided it's not economically feasible. You can only route across the fiber that's actually there and lit up. You're from Europe? OK, consider this setup: Andorra. Two providers, one of who backhaul that path all the way to Madrid, and the other that backhauls to Marseilles. Sure, there's other cities along the way, but there's no fiber path from where you are to there. For instance, the fiber path may run from Madrid to Zaragoza, where it splits 3 ways to Pamplona, Andorra, and Barcelona - but if Barcelona and Pamplona don't provide alternate paths out to the net, you're still going to Madrid. Meanwhile, other companies may provide service to lots of smaller places along the border on the Spain side, and other companies provide service to lots of places on the French side, but not into Andorra itself. You don't like that, consider any one of the many European cities that are in a deep river valley, so the only realistic ways to the outside world are "upstream" and "downstream".
The question was if downtime on a transit provider of many hours is unacceptable. I am offering my experience that this happens to all of them. Some of them can have problems that last days not hours. Do not ever assume that a so called "tier 1" network is good as your only transit.
The gotcha here is the very high danger than with only two paths out of the city, your second and third choices are fate-sharing with that Tier 1. If you're in Andorra, and you have 8 providers that share a path through a tunnel to Toulouse, and another 6 that share a bridge to Barcelona, you still have a problem. (That, and anybody who buys transit only from one Tier 1 is going to have a really hard time getting routes to the *rest* of the internet...)