[NANOG] Verizon (Fios) Packet Loss | Philadelphia Metro Area | Possible Routing Issue?

Hello, fellow NANOG members, Is there anyone from Verizon on the list that could take a look at a possible routing issue on the Fios network that could be causing loss? I have two locations, each with 1G service. Location 1 is in Quakertown, PA & has a dynamic IP on the 96.227.54.0/24 block Location 2 is in Chalfont, PA & has a dynamic IP on the 108.16.195.0/24 block They are ~11miles apart as the crow flies. I have some EdgeConnect SDWAN appliances connected at these locations and they're measuring a variable amount of loss between them (more during peak hours). I have checked everything I can on my end and posting here as a last resort. Traceroute from Quakertown to Chalfont seems to hop through two LCR nodes in Washington, DC. 1. dog-coresw01.xxxxxxxxx.net 2. ixl0-dog-opnsense01.xxxxxxxxx.NET 3. lo0-100.PHLAPA-VFTTP-346.verizon-gni.net 4. B3346.PHLAPA-LCR-21.verizon-gni.net B3346.PHLAPA-LCR-22.verizon-gni.net 5. G1-7-2-4.WASHDC-LCR-21.verizon-gni.net G1-7-2-3.WASHDC-LCR-21.verizon-gni.net 6. Chalfont, PA 108.16.195.x Unless the PTR records on hop #5 are incorrect, it seems odd that traffic from Quakertown, PA has to travel all the way down to Washington DC, then come back up to Pennsylvania. Traceroute from the opposite direction (Chalfont to Quakertown) does not show the WASHDC LCR nodes. 1. 192.168.138.1 2. ixl0-wpv-opnsense.xxxxxxxxx.NET 3. lo0-100.PHLAPA-VFTTP-354.verizon-gni.net 4. (waiting for reply) 5. Bundle-Ether1000.PHLAPA-LCR-21.VERIZON-GNI.NET Bundle-Ether1000.PHLAPA-LCR-22.VERIZON-GNI.NET Bundle-Ether1200.PHLAPA-LCR-22.VERIZON-GNI.NET HundredGigE2-0-0-3.PHLAPA-LCR-21.VERIZON-GNI.NET HundredGigE2-0-0-0.PHLAPA-LCR-21.VERIZON-GNI.NET Bundle-Ether800.PHLAPA-LCR-22.VERIZON-GNI.NET HundredGigE2-0-0-3.PHLAPA-LCR-22.VERIZON-GNI.NET 6. ae204-0.PHLAPA-VFTTP-346.verizon-gni.net ae203-0.PHLAPA-VFTTP-346.verizon-gni.net 7. Quakertown, PA 96.227.54.x I'm out of ideas on what could be causing the packet loss. If someone from Verizon could take a look, that would be GREATLY appreciated. Thank you!!!

Sounds like congestion if it's worse at peak times. Looking at the latency numbers will tell you if it's actually going to DC and back. On Sat, Mar 29, 2025 at 2:46 PM alexander.lembesis--- via NANOG < nanog@lists.nanog.org> wrote:
Hello, fellow NANOG members,
Is there anyone from Verizon on the list that could take a look at a possible routing issue on the Fios network that could be causing loss? I have two locations, each with 1G service. Location 1 is in Quakertown, PA & has a dynamic IP on the 96.227.54.0/24 block Location 2 is in Chalfont, PA & has a dynamic IP on the 108.16.195.0/24 block They are ~11miles apart as the crow flies. I have some EdgeConnect SDWAN appliances connected at these locations and they're measuring a variable amount of loss between them (more during peak hours). I have checked everything I can on my end and posting here as a last resort.
Traceroute from Quakertown to Chalfont seems to hop through two LCR nodes in Washington, DC. 1. dog-coresw01.xxxxxxxxx.net 2. ixl0-dog-opnsense01.xxxxxxxxx.NET 3. lo0-100.PHLAPA-VFTTP-346.verizon-gni.net 4. B3346.PHLAPA-LCR-21.verizon-gni.net B3346.PHLAPA-LCR-22.verizon-gni.net 5. G1-7-2-4.WASHDC-LCR-21.verizon-gni.net G1-7-2-3.WASHDC-LCR-21.verizon-gni.net 6. Chalfont, PA 108.16.195.x
Unless the PTR records on hop #5 are incorrect, it seems odd that traffic from Quakertown, PA has to travel all the way down to Washington DC, then come back up to Pennsylvania.
Traceroute from the opposite direction (Chalfont to Quakertown) does not show the WASHDC LCR nodes. 1. 192.168.138.1 2. ixl0-wpv-opnsense.xxxxxxxxx.NET 3. lo0-100.PHLAPA-VFTTP-354.verizon-gni.net 4. (waiting for reply) 5. Bundle-Ether1000.PHLAPA-LCR-21.VERIZON-GNI.NET Bundle-Ether1000.PHLAPA-LCR-22.VERIZON-GNI.NET Bundle-Ether1200.PHLAPA-LCR-22.VERIZON-GNI.NET HundredGigE2-0-0-3.PHLAPA-LCR-21.VERIZON-GNI.NET HundredGigE2-0-0-0.PHLAPA-LCR-21.VERIZON-GNI.NET Bundle-Ether800.PHLAPA-LCR-22.VERIZON-GNI.NET HundredGigE2-0-0-3.PHLAPA-LCR-22.VERIZON-GNI.NET 6. ae204-0.PHLAPA-VFTTP-346.verizon-gni.net ae203-0.PHLAPA-VFTTP-346.verizon-gni.net 7. Quakertown, PA 96.227.54.x
I'm out of ideas on what could be causing the packet loss. If someone from Verizon could take a look, that would be GREATLY appreciated.
Thank you!!! _______________________________________________ NANOG mailing list
https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/2DD4GF7H...
participants (2)
-
alexander.lembesis@gmail.com
-
Ross Tajvar