When it even happens to the best of us
Huge, huge props to Cloudflare. THAT is what a good RFO report should look like. Sucks when it happens, but spotting it quickly, backing out the change, and identifying how it happened is crucial to helping ensure it won't happen that way again. Thank you again to Tom, Bryton, and the rest of the Cloudflare team for the clear and comprehensive analysis of the outage. There's some really good learning lessons for all of us that are implementing automation across large networks. We don't all have to put our hands on the hot stove, one after another. ;) Thanks! Matt On Sun, Jan 25, 2026, 21:25 Hank Nussbacher via NANOG <nanog@lists.nanog.org> wrote:
https://blog.cloudflare.com/route-leak-incident-january-22-2026/
-Hank
_______________________________________________ NANOG mailing list
https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/5JDWQBCU...
On Mon, Jan 26, 2026 at 2:15 AM, Matthew Petach <nanog@lists.nanog.org> wrote:
Huge, huge props to Cloudflare. THAT is what a good RFO report should look like.
+many lots. Big complicated networks are big and complicated…. And sometimes there WILL be issues. When this occurs, documenting what happened in a full, complete and transparent mechanism makes customers trust the network so much more than anodyne "There was an issue. We fixed it!" Sucks when it happens, but spotting it quickly, backing out the change, and
identifying how it happened is crucial to helping ensure it won't happen that way again.
Thank you again to Tom, Bryton, and the rest of the Cloudflare team for the clear and comprehensive analysis of the outage. There's some really good learning lessons for all of us that are implementing automation across large networks.
Yes, indeed, thank you. Much respect. W We don't all have to put our hands on the hot stove, one after another. ;)
Thanks!
Matt
On Sun, Jan 25, 2026, 21:25 Hank Nussbacher via NANOG <nanog@lists.nanog. org> wrote:
https://blog.cloudflare.com/route-leak-incident-january-22-2026/
-Hank
_______________________________________________ NANOG mailing list
https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/ 5JDWQBCULLABBVZ5TMEEKGH7RLKZMNZH/
_______________________________________________ NANOG mailing list https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/ IJYI7LW7PK6TM53QDGJXJ6KFK5T234V3/
Huge, huge props to Cloudflare. THAT is what a good RFO report should look like.
+1 as well. And a side note that this is one of the nice parts of operating in an unregulated market - staff have the freedom to publish stuff like this. In contrast, due to regulation and risk of regulatory sanction, entities in regulated markets often can only concede there was an outage. JL
participants (4)
-
Hank Nussbacher -
Livingood, Jason -
Matthew Petach -
Warren Kumari