Looks like Google recently posted their post-mortem of their outage on the 12th https://status.cloud.google.com/incident/storage/19002 On Thu, Mar 14, 2019 at 1:21 PM Suresh Ramasubramanian <ops.lists@gmail.com> wrote:
That's a 2010 outage that someone dug out and was doing the rounds as a new one
--srs
------------------------------ *From:* NANOG <nanog-bounces@nanog.org> on behalf of cosmo < clinton.mielke@gmail.com> *Sent:* Thursday, March 14, 2019 9:50 PM *To:* Bryan Holloway *Cc:* nanog@nanog.org *Subject:* Re: FB?
Facebook pushed an update to their code that manages cookies, that had a rather severe bug in it that resulted in a large flood of requests to their database servers. To deal with this load, they had to prevent all writes and then slowly allow people back on.
I saw the writeup for it last night but cannot seem to find it now! Grrr. Did I dream it?
On Thu, Mar 14, 2019 at 8:42 AM Bryan Holloway <bryan@shout.net> wrote:
On 3/14/19 9:06 AM, Tom Beecher wrote:
As much as I wanted to crack jokes because I cannot stand Facebook (the product), much love to all you FB engineers that went through (and are probably still going through) much hell.
+1 on both counts.
We've all been there; no bueno.