Lima, OH Spectrum/Charter Severe Node/Hop Latency Issues
Hello all, One of my NetOps engineers resides in Lima, Ohio and they are receiving terrible bufferbloat, packet loss, and random disconnects. I have been pinging 24.33.160.213 (Lima, OH Spectrum/Chart Node) and it's rejecting a ton of packets. This has been going on for weeks. Node having problems: lag-1.limaohid01h.netops.charter.com NOC seems like they don't care, same with OSP in the field. There is no reason why this hop (#13) should have up to 613ms ping times. Thank you, Austin [cid:d929237e-d932-4060-9dbb-f4b99b12afa4]
Austin, If you run MTRs or traceroutes through the node, is there any other additional packet loss seen in the path, and at the destination? What does the reverse MTR or traceroute look like? The attached image was stripped out by the mailing list system. Bufferbloat is controlled at the firewall level, which is different from packet loss and disconnects. Ryan From: NANOG <nanog-bounces+ryan=rkhtech.org@nanog.org> On Behalf Of Austin Ayers via NANOG Sent: Tuesday, February 7, 2023 1:49 PM To: nanog@nanog.org Subject: Lima, OH Spectrum/Charter Severe Node/Hop Latency Issues Hello all, One of my NetOps engineers resides in Lima, Ohio and they are receiving terrible bufferbloat, packet loss, and random disconnects. I have been pinging 24.33.160.213 (Lima, OH Spectrum/Chart Node) and it's rejecting a ton of packets. This has been going on for weeks. Node having problems: lag-1.limaohid01h.netops.charter.com NOC seems like they don't care, same with OSP in the field. There is no reason why this hop (#13) should have up to 613ms ping times. Thank you, Austin
ICMP response time from a router/device is not a great way to judge if there is an issue or not. The devices generally have control plane policing and responding to ICMP is not prioritized at all. I would suggest your engineer setup something on their end of the connection that you can ping, and start there. Leverage something like smoke ping to something on their LAN, or even the public IP on their RG/Modem. Kevin On Tue, Feb 7, 2023 at 15:01 Austin Ayers via NANOG <nanog@nanog.org> wrote:
Hello all,
One of my NetOps engineers resides in Lima, Ohio and they are receiving terrible bufferbloat, packet loss, and random disconnects.
I have been pinging 24.33.160.213 (Lima, OH Spectrum/Chart Node) and it's rejecting a ton of packets. This has been going on for weeks.
Node having problems: lag-1.limaohid01h.netops.charter.com
NOC seems like they don't care, same with OSP in the field.
There is no reason why this hop (#13) should have up to 613ms ping times.
Thank you, Austin
For those who haven't seen it (i.e. Austin), here is "the guide" on how to troubleshoot correctly with traceroute: https://archive.nanog.org/meetings/nanog47/presentations/Sunday/RAS_Tracerou... ICMP is deprioritized by any normal router. Non-cascading loss does not indicate a problem of any kind. The NOC doesn't care because nothing is wrong, and the OSP team definitely doesn't care because ICMP is several layers above OSP and is therefore not their problem. On Tue, Feb 7, 2023 at 5:11 PM Kevin Shymkiw <kshymkiw@gmail.com> wrote:
ICMP response time from a router/device is not a great way to judge if there is an issue or not. The devices generally have control plane policing and responding to ICMP is not prioritized at all.
I would suggest your engineer setup something on their end of the connection that you can ping, and start there.
Leverage something like smoke ping to something on their LAN, or even the public IP on their RG/Modem.
Kevin
On Tue, Feb 7, 2023 at 15:01 Austin Ayers via NANOG <nanog@nanog.org> wrote:
Hello all,
One of my NetOps engineers resides in Lima, Ohio and they are receiving terrible bufferbloat, packet loss, and random disconnects.
I have been pinging 24.33.160.213 (Lima, OH Spectrum/Chart Node) and it's rejecting a ton of packets. This has been going on for weeks.
Node having problems: lag-1.limaohid01h.netops.charter.com
NOC seems like they don't care, same with OSP in the field.
There is no reason why this hop (#13) should have up to 613ms ping times.
Thank you, Austin
participants (4)
-
Austin Ayers -
Kevin Shymkiw -
Ross Tajvar -
Ryan Hamel