Christopher Morrow wrote:
To be fair, it looks like FB has 4 /32's (and 4 /128's) for their DNS authoritatives. All from different /24's or /48's, so they should have decent routing diversity. They could choose to announce half/half from alternate pops, or other games such as this.
Yup.
I don't know that that would have solved any of the problems last week nor any problems in the future.
There are various solutions. For example, if FB had relied on, instead of route withdrawal, standard DNS expire mechanism, FB should have noticed that FB needed another zone for stable data for maintenance servers, I think.
I think Bill's slide 30 is pretty much what FB has/had deployed:
It seems to me that he assumes transit providers and cloud providers are different entities. FB, instead, operate their own transit network and clouds within its domain and clouds are connected only by FB transit (there aren't multiple (red and green) transit).
it's also not clear that FB is connecting their CDN to single points in any provider... I'd guess there are some cases of that,
That is bad enough, if FB wants to "optimize" their traffic for the cases by killing DNS redundancy to put all the name servers in single POP, which is my concern. Masataka Ohta