Neel, Sounds like buffer bloat. Run a speed test, whatever is your maximum for your download and upload take 10% away from it, and setup traffic shaping in OPNsense (https://docs.opnsense.org/manual/shaping.html) with those values. If the issue goes away, then you're exceeding the buffer of CenturyLink's device with the bursts of traffic. Ryan -----Original Message----- From: NANOG <nanog-bounces+ryan=rkhtech.org@nanog.org> On Behalf Of Neel Chauhan Sent: Monday, November 1, 2021 6:44 PM To: nanog@nanog.org Subject: CenturyLink Fiber Latency Issues (Seattle, WA) Hi NANOG Mailing List, I don't know if any of you work at CenturyLink/Lumen, very less on their Fiber network in Seattle, WA. However, here's my story. If I attempt to run certain applications that use 1000, or 10000 TCP connections, I get latency spikes. It is based on how many connections, but also how much bandwidth is used. This means certain things like Tor relays are off limits to me (which I wish to run). On an idle connection, the PingPlotter outputs look like this: https://centurylinklatencyissues.com/image-000.png If I attempt to run BitTorrent with 1000 connections in Deluge, PingPlotter looks like this: https://centurylinklatencyissues.com/image-002.png Getting support, or even executive contacts to admit the issue hasn't worked. They all love to blame my equipment or applications, when CL routers also show the issue when I run the same things whereas my same exact OPNsense box on Google Fiber Webpass running Tor at another address had no issues whatsoever, and I can ping other Tor relays on CenturyLink AS209 just fine (from a VPS). The most competent person I dealt with was actually one tech. He told me there was "capacity issues" in our neighborhood, and that's the reason for the issues. However, nothing was done about it afterwards, I'm guessing since I turned off my Tor relay after the visit to avoid complaints from family members. On an AT&T forum, people have said GPON gives latency spikes/packet loss on congestion: https://www.dslreports.com/forum/r33242889-How-rare-is-GPON-XGSPON-saturatio n The capacity managers in Seattle are literally dragging their feet: it's 100x worse than AT&T's 802.1X. I know AT&T and CenturyLink don't compete, but if I had to choose between AT&T Fiber and CenturyLink, I'll take AT&T in a heartbeat, no ifs, no buts, even if I have to use AT&T's crappy router instead of my OPNsense box. Going back, do any of you who work at CenturyLink/Lumen can get me to the right people, hopefully the capacity managers in Seattle? I could go with Comcast, but it's either (a) 35 Mbps uploads or (b) $329/mo for "Gigabit Pro" with a 2-year contract and a steep install fee. I am seriously considering Gigabit Pro even if it breaks the bank, but hope I won't have to go there. I don't need 2 Gbps and would rather pay $65 than $329. 300-500 Mbps uploads when I need it is the sweet spot for me (even without Tor) which CL GPON should easily handle without a sweat. I also don't exactly **trust** Comcast, they're a horrible company in many metrics, but in some ways Comcast is more competent than CenturyLink. Best, Neel Chauhan