Issues connecting to Cloudflare over IPv4 in Nashville
A client reported issues connecting to various websites this evening and I've narrowed it down to cloudflare hosted sites specifically on ipv4. My traceroutes go through my upstream all the way to Comcast in Nashville where they die. IPv6 cloudflare addresses work just fine, but several v4 addresses I tested fail. Is anybody else seeing similar issues, or anybody from cloudflare here able to help me troubleshoot? Thanks! Landy
Do you have some example trace routes? -mel
On Jan 26, 2025, at 12:10 PM, Landy Bible <landy@ljb2of3.net> wrote:
A client reported issues connecting to various websites this evening and I've narrowed it down to cloudflare hosted sites specifically on ipv4. My traceroutes go through my upstream all the way to Comcast in Nashville where they die.
IPv6 cloudflare addresses work just fine, but several v4 addresses I tested fail.
Is anybody else seeing similar issues, or anybody from cloudflare here able to help me troubleshoot?
Thanks! Landy
I don't have them handy on my phone, but luckily the issue had resolved itself by the time I got up Saturday morning, so no need to dig further. On Sun, Jan 26, 2025, 15:08 Mel Beckman <mel@beckman.org> wrote:
Do you have some example trace routes?
-mel
On Jan 26, 2025, at 12:10 PM, Landy Bible <landy@ljb2of3.net> wrote:
A client reported issues connecting to various websites this evening and I've narrowed it down to cloudflare hosted sites specifically on ipv4. My traceroutes go through my upstream all the way to Comcast in Nashville where they die.
IPv6 cloudflare addresses work just fine, but several v4 addresses I tested fail.
Is anybody else seeing similar issues, or anybody from cloudflare here able to help me troubleshoot?
Thanks! Landy
We saw similar a few months ago over a period with our customers transiting Chicago through Lumen. We found a Cloudflare maintenance was going on at the time, but it seems something went wrong between them and Lumen. We could not ping Cloudflare DNS such as 1.1.1.1 or reach other Cloudflare sites over that period. We had to disable our Lumen link through Chicago to recover. We never established whose fault it was. -Ian From: NANOG <nanog-bounces+iflanders=springnet.net@nanog.org> on behalf of Landy Bible <landy@ljb2of3.net> Date: Tuesday, January 28, 2025 at 7:19 PM To: Mel Beckman <mel@beckman.org> Cc: nanog@nanog.org <nanog@nanog.org> Subject: Re: Issues connecting to Cloudflare over IPv4 in Nashville I don't have them handy on my phone, but luckily the issue had resolved itself by the time I got up Saturday morning, so no need to dig further. On Sun, Jan 26, 2025, 15:08 Mel Beckman <mel@beckman.org<mailto:mel@beckman.org>> wrote: Do you have some example trace routes? -mel
On Jan 26, 2025, at 12:10 PM, Landy Bible <landy@ljb2of3.net<mailto:landy@ljb2of3.net>> wrote:
A client reported issues connecting to various websites this evening and I've narrowed it down to cloudflare hosted sites specifically on ipv4. My traceroutes go through my upstream all the way to Comcast in Nashville where they die.
IPv6 cloudflare addresses work just fine, but several v4 addresses I tested fail.
Is anybody else seeing similar issues, or anybody from cloudflare here able to help me troubleshoot?
Thanks! Landy
[City Utilities] [SpringNet]<http://www.springnet.net> Sales 417.575.7000 | Support 417.874.8000 | springnet.net<http://www.springnet.net>
We (Cloudflare) had a 15mins connectivity incident in Chicago on 2024-11-12, maybe that's the one you're referring to. On Tue, Jan 28, 2025 at 8:34 PM Ian Flanders <iflanders@springnet.net> wrote:
We saw similar a few months ago over a period with our customers transiting Chicago through Lumen. We found a Cloudflare maintenance was going on at the time, but it seems something went wrong between them and Lumen. We could not ping Cloudflare DNS such as 1.1.1.1 or reach other Cloudflare sites over that period. We had to disable our Lumen link through Chicago to recover. We never established whose fault it was.
-Ian
*From: *NANOG <nanog-bounces+iflanders=springnet.net@nanog.org> on behalf of Landy Bible <landy@ljb2of3.net> *Date: *Tuesday, January 28, 2025 at 7:19 PM *To: *Mel Beckman <mel@beckman.org> *Cc: *nanog@nanog.org <nanog@nanog.org> *Subject: *Re: Issues connecting to Cloudflare over IPv4 in Nashville
I don't have them handy on my phone, but luckily the issue had resolved itself by the time I got up Saturday morning, so no need to dig further.
On Sun, Jan 26, 2025, 15:08 Mel Beckman <mel@beckman.org> wrote:
Do you have some example trace routes?
-mel
On Jan 26, 2025, at 12:10 PM, Landy Bible <landy@ljb2of3.net> wrote:
A client reported issues connecting to various websites this evening and I've narrowed it down to cloudflare hosted sites specifically on ipv4. My traceroutes go through my upstream all the way to Comcast in Nashville where they die.
IPv6 cloudflare addresses work just fine, but several v4 addresses I tested fail.
Is anybody else seeing similar issues, or anybody from cloudflare here able to help me troubleshoot?
Thanks! Landy
[image: City Utilities]
[image: SpringNet] <http://www.springnet.net>
Sales 417.575.7000 | Support 417.874.8000 | springnet.net <http://www.springnet.net>
Thank you for your reply and the information. The event we experienced lasted several hours on September 5, 2024, and coincided with this maintenance: https://community.cloudflare.com/t/ord-chicago-on-2024-09-05/707550. Ultimately, we had to take down our Lumen connection through Chicago to restore customers' ability to access Cloudflare-related sites and services. Here is an article we found about someone experiencing a similar issue shortly after ours: https://www.bleepingcomputer.com/news/technology/cloudflare-outage-cuts-off-.... Interestingly, this article mentions Nashville, similar to the original discussion. We are unsure if our event was due to Lumen failing to reroute traffic correctly during Cloudflare’s maintenance or another underlying issue. A ticket was opened with Lumen, and they confirmed seeing link state changes with Cloudflare, but they were unable to provide details regarding the associated routing. It was difficult to reach Cloudflare during or after the event to obtain information. Thank you, -Ian From: NANOG <nanog-bounces+iflanders=springnet.net@nanog.org> On Behalf Of Jérôme Fleury Sent: Friday, January 31, 2025 11:20 AM To: nanog@nanog.org Subject: Re: Issues connecting to Cloudflare over IPv4 in Nashville We (Cloudflare) had a 15mins connectivity incident in Chicago on 2024-11-12, maybe that's the one you're referring to. On Tue, Jan 28, 2025 at 8:34 PM Ian Flanders <iflanders@springnet.net<mailto:iflanders@springnet.net>> wrote: We saw similar a few months ago over a period with our customers transiting Chicago through Lumen. We found a Cloudflare maintenance was going on at the time, but it seems something went wrong between them and Lumen. We could not ping Cloudflare DNS such as 1.1.1.1 or reach other Cloudflare sites over that period. We had to disable our Lumen link through Chicago to recover. We never established whose fault it was. -Ian From: NANOG <nanog-bounces+iflanders=springnet.net@nanog.org<mailto:springnet.net@nanog.org>> on behalf of Landy Bible <landy@ljb2of3.net<mailto:landy@ljb2of3.net>> Date: Tuesday, January 28, 2025 at 7:19 PM To: Mel Beckman <mel@beckman.org<mailto:mel@beckman.org>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> <nanog@nanog.org<mailto:nanog@nanog.org>> Subject: Re: Issues connecting to Cloudflare over IPv4 in Nashville I don't have them handy on my phone, but luckily the issue had resolved itself by the time I got up Saturday morning, so no need to dig further. On Sun, Jan 26, 2025, 15:08 Mel Beckman <mel@beckman.org<mailto:mel@beckman.org>> wrote: Do you have some example trace routes? -mel
On Jan 26, 2025, at 12:10 PM, Landy Bible <landy@ljb2of3.net<mailto:landy@ljb2of3.net>> wrote:
A client reported issues connecting to various websites this evening and I've narrowed it down to cloudflare hosted sites specifically on ipv4. My traceroutes go through my upstream all the way to Comcast in Nashville where they die.
IPv6 cloudflare addresses work just fine, but several v4 addresses I tested fail.
Is anybody else seeing similar issues, or anybody from cloudflare here able to help me troubleshoot?
Thanks! Landy
[City Utilities] [SpringNet]<http://www.springnet.net/> Sales 417.575.7000 | Support 417.874.8000 | springnet.net<http://www.springnet.net/> [City Utilities] [SpringNet]<http://www.springnet.net/> Sales 417.575.7000 | Support 417.874.8000 | springnet.net<http://www.springnet.net/>
participants (4)
-
Ian Flanders -
Jérôme Fleury -
Landy Bible -
Mel Beckman