internet - at least texas
what in the world is going on in texas? i'm in south texas (san antonio) and all 4 of my internet connections went way down! i disabled cogent bgp and already looking better on my other transit links (non-cogent) -- -Aaron
Did you check the Outages list. It could be just you. -mel via cell
On Nov 25, 2025, at 3:33 PM, Aaron Gould via NANOG <nanog@lists.nanog.org> wrote:
what in the world is going on in texas? i'm in south texas (san antonio) and all 4 of my internet connections went way down! i disabled cogent bgp and already looking better on my other transit links (non-cogent)
-- -Aaron
_______________________________________________ NANOG mailing list https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/3KIJECRT...
Seeing some loss on Cogent in Austin. Drained Cogent, others appear to be fine. June Slater On Tue, Nov 25, 2025 at 3:32 PM Aaron Gould via NANOG <nanog@lists.nanog.org> wrote:
what in the world is going on in texas? i'm in south texas (san antonio) and all 4 of my internet connections went way down! i disabled cogent bgp and already looking better on my other transit links (non-cogent)
-- -Aaron
_______________________________________________ NANOG mailing list
https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/3KIJECRT...
Yeah, I'm getting a bunch of alerts for Texas too. Seems like it might affect multiple areas and multiple carriers, but I don't really have a sense of exactly what's going on yet. I am seeing a lot of Cogent, but I'm seeing alerts for other carriers too. Curious if anyone else has anything. On Tue, Nov 25, 2025 at 6:49 PM June Slater via NANOG <nanog@lists.nanog.org> wrote:
Seeing some loss on Cogent in Austin. Drained Cogent, others appear to be fine.
June Slater
On Tue, Nov 25, 2025 at 3:32 PM Aaron Gould via NANOG < nanog@lists.nanog.org> wrote:
what in the world is going on in texas? i'm in south texas (san antonio) and all 4 of my internet connections went way down! i disabled cogent bgp and already looking better on my other transit links (non-cogent)
-- -Aaron
_______________________________________________ NANOG mailing list
https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/3KIJECRT... _______________________________________________ NANOG mailing list
https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/6XXD4ALE...
No issues here in Houston or Dallas on cogent. Not seeing any complaint volume on outages list, either. May be localized.
On Nov 25, 2025, at 17:32, Aaron Gould via NANOG <nanog@lists.nanog.org> wrote:
what in the world is going on in texas? i'm in south texas (san antonio) and all 4 of my internet connections went way down! i disabled cogent bgp and already looking better on my other transit links (non-cogent)
-- -Aaron
_______________________________________________ NANOG mailing list https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/3KIJECRT...
Observed routing loops on Cogent out of McAllen. Reachability cleaned up after turning down peering to them. Cogent has an entry on their Network Status page now. https://ecogent.cogentco.com/network-status ________________________________ From: Lee Fawkes via NANOG <nanog@lists.nanog.org> Sent: Tuesday, November 25, 2025 4:20 PM To: North American Network Operators Group <nanog@lists.nanog.org> Cc: Lee Fawkes <nanog@digital-oasis.com> Subject: Re: internet - at least texas [EXTERNAL EMAIL] DO NOT CLICK links or attachments unless you recognize the sender and know the content is safe. Yeah, I'm getting a bunch of alerts for Texas too. Seems like it might affect multiple areas and multiple carriers, but I don't really have a sense of exactly what's going on yet. I am seeing a lot of Cogent, but I'm seeing alerts for other carriers too. Curious if anyone else has anything. On Tue, Nov 25, 2025 at 6:49 PM June Slater via NANOG <nanog@lists.nanog.org> wrote:
Seeing some loss on Cogent in Austin. Drained Cogent, others appear to be fine.
June Slater
On Tue, Nov 25, 2025 at 3:32 PM Aaron Gould via NANOG < nanog@lists.nanog.org> wrote:
what in the world is going on in texas? i'm in south texas (san antonio) and all 4 of my internet connections went way down! i disabled cogent bgp and already looking better on my other transit links (non-cogent)
-- -Aaron
_______________________________________________ NANOG mailing list
Dear Cogent Customer, We are currently experiencing an outage on our network. We are working to resolve this as quickly as possible. Please for updates on this event. If you need to open an individual ticket on this event, please e-mail support@cogentco.com and reference outage HD305673827. Impacted Region: Texas, USA / Mexico Order(s) Impacted: 1-300427262 Outage Ticket: HD305673827 Status: Get Outlook for iOS<https://aka.ms/o0ukef> ________________________________ From: Tim Burke via NANOG <nanog@lists.nanog.org> Sent: Tuesday, November 25, 2025 5:25:08 PM To: nanog@lists.nanog.org <nanog@lists.nanog.org> Cc: nanog@lists.nanog.org <nanog@lists.nanog.org>; Tim Burke <tim@mid.net> Subject: Re: internet - at least texas No issues here in Houston or Dallas on cogent. Not seeing any complaint volume on outages list, either. May be localized.
On Nov 25, 2025, at 17:32, Aaron Gould via NANOG <nanog@lists.nanog.org> wrote:
what in the world is going on in texas? i'm in south texas (san antonio) and all 4 of my internet connections went way down! i disabled cogent bgp and already looking better on my other transit links (non-cogent)
-- -Aaron
Can confirm we are affected in Austin. We have pulled the plug to Cogent until it recovers. Sent from my Galaxy -------- Original message --------From: Robert DeVita via NANOG <nanog@lists.nanog.org> Date: 11/25/25 4:49 PM (GMT-08:00) To: North American Network Operators Group <nanog@lists.nanog.org> Cc: nanog@lists.nanog.org, Robert DeVita <radevita@mejeticks.com> Subject: Re: internet - at least texas Dear Cogent Customer,We are currently experiencing an outage on our network. We are working to resolve this as quickly as possible. Please for updates on this event.If you need to open an individual ticket on this event, please e-mail support@cogentco.com and reference outage HD305673827.Impacted Region: Texas, USA / MexicoOrder(s) Impacted: 1-300427262Outage Ticket: HD305673827Status:Get Outlook for iOS<https://aka.ms/o0ukef>________________________________From: Tim Burke via NANOG <nanog@lists.nanog.org>Sent: Tuesday, November 25, 2025 5:25:08 PMTo: nanog@lists.nanog.org <nanog@lists.nanog.org>Cc: nanog@lists.nanog.org <nanog@lists.nanog.org>; Tim Burke <tim@mid.net>Subject: Re: internet - at least texasNo issues here in Houston or Dallas on cogent. Not seeing any complaint volume on outages list, either. May be localized.> On Nov 25, 2025, at 17:32, Aaron Gould via NANOG <nanog@lists.nanog.org> wrote:>> what in the world is going on in texas? i'm in south texas (san antonio) and all 4 of my internet connections went way down! i disabled cogent bgp and already looking better on my other transit links (non-cogent)>> --> -Aaron>> _______________________________________________> NANOG mailing list https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.nanog.org%2Farchives%2Flist%2Fnanog%40lists.nanog.org%2Fmessage%2F3KIJECRT6TZWEICWHLH4G4DSY4BXXW6L%2F&data=05%7C02%7Cradevita%40mejeticks.com%7Ce8738236aec24220fd7f08de2c824a23%7Cf4cf98f2d77845638d5b5fd2fa70ff2f%7C0%7C0%7C638997135290815962%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=43PT9%2FQZjUc5gYf8dlwn4%2FYgrbmUwnCMVNOh43a1mU8%3D&reserved=0<https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/3KIJECRT6TZWEICWHLH4G4DSY4BXXW6L/>_______________________________________________NANOG mailing listhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.nanog.org%2Farchives%2Flist%2Fnanog%40lists.nanog.org%2Fmessage%2FQ5SJZNSAVV735EGV7OSF43HJ5RKDRH2E%2F&data=05%7C02%7Cradevita%40mejeticks.com%7Ce8738236aec24220fd7f08de2c824a23%7Cf4cf98f2d77845638d5b5fd2fa70ff2f%7C0%7C0%7C638997135290836571%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=8py0AgliuL0NtteKqQUlb3lors1Mkkf%2BqMD6DDyuTK0%3D&reserved=0<https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/Q5SJZNSAVV735EGV7OSF43HJ5RKDRH2E/>_______________________________________________NANOG mailing list https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/TRP7DGXI...
I've seen "grey failure" issues like this a few times with Cogent in the south. There will be an issue on their backbone somewhere, and they'll continue to advertise our prefixes and blackhole our traffic. Is this because they've got some kind of setup where their control plane is separated from their data plane, so that their iBGP peers stay up when there's no path for the data to take to the site? Or is it a lack of failover capacity and extreme congestion, where enough traffic is able to get through that their iBGP sessions stay up, but best-effort customer traffic is getting tossed on the floor? On Tue, Nov 25, 2025 at 8:34 PM Michael J McCafferty via NANOG < nanog@lists.nanog.org> wrote:
Can confirm we are affected in Austin. We have pulled the plug to Cogent until it recovers. Sent from my Galaxy -------- Original message --------From: Robert DeVita via NANOG < nanog@lists.nanog.org> Date: 11/25/25 4:49 PM (GMT-08:00) To: North American Network Operators Group <nanog@lists.nanog.org> Cc: nanog@lists.nanog.org, Robert DeVita <radevita@mejeticks.com> Subject: Re: internet - at least texas Dear Cogent Customer,We are currently experiencing an outage on our network. We are working to resolve this as quickly as possible. Please for updates on this event.If you need to open an individual ticket on this event, please e-mail support@cogentco.com and reference outage HD305673827.Impacted Region: Texas, USA / MexicoOrder(s) Impacted: 1-300427262Outage Ticket: HD305673827Status:Get Outlook for iOS<https://aka.ms/o0ukef>________________________________From: Tim Burke via NANOG <nanog@lists.nanog.org>Sent: Tuesday, November 25, 2025 5:25:08 PMTo: nanog@lists.nanog.org <nanog@lists.nanog.org>Cc: nanog@lists.nanog.org <nanog@lists.nanog.org>; Tim Burke <tim@mid.net>Subject: Re: internet - at least texasNo issues here in Houston or Dallas on cogent. Not seeing any complaint volume on outages list, either. May be localized.> On Nov 25, 2025, at 17:32, Aaron Gould via NANOG <nanog@lists.nanog.org> wrote:>> what in the world is going on in texas? i'm in south texas (san antonio) and all 4 of my internet connections went way down! i disabled cogent bgp and already looking better on my other transit links (non-cogent)>> --> -Aaron>> _______________________________________________> NANOG mailing list https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.nanog.org%2Farchives%2Flist%2Fnanog%40lists.nanog.org%2Fmessage%2F3KIJECRT6TZWEICWHLH4G4DSY4BXXW6L%2F&data=05%7C02%7Cradevita%40mejeticks.com%7Ce8738236aec24220fd7f08de2c824a23%7Cf4cf98f2d77845638d5b5fd2fa70ff2f%7C0%7C0%7C638997135290815962%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=43PT9%2FQZjUc5gYf8dlwn4%2FYgrbmUwnCMVNOh43a1mU8%3D&reserved=0 < https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/3KIJECRT6TZWEICWHLH4G4DSY4BXXW6L/>_______________________________________________NANOG mailing listhttps:// nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.nanog.org%2Farchives%2Flist%2Fnanog%40lists.nanog.org%2Fmessage%2FQ5SJZNSAVV735EGV7OSF43HJ5RKDRH2E%2F&data=05%7C02%7Cradevita%40mejeticks.com%7Ce8738236aec24220fd7f08de2c824a23%7Cf4cf98f2d77845638d5b5fd2fa70ff2f%7C0%7C0%7C638997135290836571%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=8py0AgliuL0NtteKqQUlb3lors1Mkkf%2BqMD6DDyuTK0%3D&reserved=0 < https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/Q5SJZNSAVV735EGV7OSF43HJ5RKDRH2E/>_______________________________________________NANOG mailing list https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/TRP7DGXI... _______________________________________________ NANOG mailing list
https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/VAINR3HY...
This is vaguely familiar here in Toronto Canada. (https://lists.nanog.org/archives/list/nanog@lists.nanog.org/thread/HXXCJEIRW...) Cogent will have some equal cost multipath (typically seen by me to AS577) among say 3 transport links, and one physical link is down so out of a subnet, every 3rd or fourth IP wont work because the BGP doesnt know one leg is down. I would normally be able to triangulate around the issue by varying the source IP the traceroute was done from. e.g. if the advertised CIDR block is say 192.168.1.0/24, a traceroute with a source addr of192.168.1.1 might fail, but 192.168.1.2 and 192.168.3 would work based on how the forwarding plan picks (typically a static hash) which leg to punt the packet through. ---Mike On 11/26/2025 11:59 AM, Lee Fawkes via NANOG wrote:
I've seen "grey failure" issues like this a few times with Cogent in the south. There will be an issue on their backbone somewhere, and they'll continue to advertise our prefixes and blackhole our traffic. Is this because they've got some kind of setup where their control plane is separated from their data plane, so that their iBGP peers stay up when there's no path for the data to take to the site? Or is it a lack of failover capacity and extreme congestion, where enough traffic is able to get through that their iBGP sessions stay up, but best-effort customer traffic is getting tossed on the floor?
On Tue, Nov 25, 2025 at 8:34 PM Michael J McCafferty via NANOG < nanog@lists.nanog.org> wrote:
Can confirm we are affected in Austin. We have pulled the plug to Cogent until it recovers. Sent from my Galaxy -------- Original message --------From: Robert DeVita via NANOG < nanog@lists.nanog.org> Date: 11/25/25 4:49 PM (GMT-08:00) To: North American Network Operators Group <nanog@lists.nanog.org> Cc: nanog@lists.nanog.org, Robert DeVita <radevita@mejeticks.com> Subject: Re: internet - at least texas Dear Cogent Customer,We are currently experiencing an outage on our network. We are working to resolve this as quickly as possible. Please for updates on this event.If you need to open an individual ticket on this event, please e-mail support@cogentco.com and reference outage HD305673827.Impacted Region: Texas, USA / MexicoOrder(s) Impacted: 1-300427262Outage Ticket: HD305673827Status:Get Outlook for iOS<https://aka.ms/o0ukef>________________________________From: Tim Burke via NANOG <nanog@lists.nanog.org>Sent: Tuesday, November 25, 2025 5:25:08 PMTo: nanog@lists.nanog.org <nanog@lists.nanog.org>Cc: nanog@lists.nanog.org <nanog@lists.nanog.org>; Tim Burke <tim@mid.net>Subject: Re: internet - at least texasNo issues here in Houston or Dallas on cogent. Not seeing any complaint volume on outages list, either. May be localized.> On Nov 25, 2025, at 17:32, Aaron Gould via NANOG <nanog@lists.nanog.org> wrote:>> what in the world is going on in texas? i'm in south texas (san antonio) and all 4 of my internet connections went way down! i disabled cogent bgp and already looking better on my other transit links (non-cogent)>> --> -Aaron>> _______________________________________________> NANOG mailing list https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.nanog.org%2Farchives%2Flist%2Fnanog%40lists.nanog.org%2Fmessage%2F3KIJECRT6TZWEICWHLH4G4DSY4BXXW6L%2F&data=05%7C02%7Cradevita%40mejeticks.com%7Ce8738236aec24220fd7f08de2c824a23%7Cf4cf98f2d77845638d5b5fd2fa70ff2f%7C0%7C0%7C638997135290815962%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=43PT9%2FQZjUc5gYf8dlwn4%2FYgrbmUwnCMVNOh43a1mU8%3D&reserved=0 < https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/3KIJECRT6TZWEICWHLH4G4DSY4BXXW6L/>_______________________________________________NANOG mailing listhttps:// nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.nanog.org%2Farchives%2Flist%2Fnanog%40lists.nanog.org%2Fmessage%2FQ5SJZNSAVV735EGV7OSF43HJ5RKDRH2E%2F&data=05%7C02%7Cradevita%40mejeticks.com%7Ce8738236aec24220fd7f08de2c824a23%7Cf4cf98f2d77845638d5b5fd2fa70ff2f%7C0%7C0%7C638997135290836571%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=8py0AgliuL0NtteKqQUlb3lors1Mkkf%2BqMD6DDyuTK0%3D&reserved=0 < https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/Q5SJZNSAVV735EGV7OSF43HJ5RKDRH2E/>_______________________________________________NANOG mailing list https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/TRP7DGXI... _______________________________________________ NANOG mailing list
https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/VAINR3HY...
NANOG mailing list https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/JHL6GUAK...
participants (9)
-
Aaron Gould -
Ben Jahner -
June Slater -
Lee Fawkes -
Mel Beckman -
Michael J McCafferty -
mike tancsa -
Robert DeVita -
Tim Burke