Thanks for the folks who looked at this -- things are looking better this morning: traceroute to 76.113.128.1 (76.113.128.1), 30 hops max, 40 byte packets 1 69.65.40.62 (69.65.40.62) 0.858 ms 0.840 ms 0.838 ms 2 so2-0-0-0.er1.Chi1.Servernap.net (69.39.239.169) 1.876 ms 1.878 ms 1.875 ms 3 ge-6-20.car1.Chicago1.Level3.net (4.79.65.49) 1.854 ms 1.858 ms 1.855 ms 4 ae-2-54.edge1.Chicago2.Level3.net (4.68.101.115) 60.047 ms 60.068 ms 60.067 ms 5 COMCAST-IP.edge1.Chicago2.Level3.net (4.71.248.26) 3.045 ms 3.051 ms 3.049 ms 6 te-0-2-0-5-ar03.roseville.mn.minn.comcast.net (68.87.174.73) 12.172 ms 12.267 ms 12.250 ms 7 te-2-1-ur01.sims.mn.minn.comcast.net (68.87.174.74) 11.717 ms * * 8 te-8-3-ur02.sims.mn.minn.comcast.net (68.87.174.78) 11.940 ms * * 9 te-2-1-ur01.newport.mn.minn.comcast.net (68.87.174.82) 12.224 ms * * 10 c-76-113-128-1.hsd1.mn.comcast.net (76.113.128.1) 12.203 ms 12.203 ms 12.045 ms -Eric Eric Spaeth wrote:

For the last couple weeks there has been a route stuck in the Chicago wan/core that is directing some Minnesota-bound traffic through Denver, even though Chicago and the Roseville, MN aggregation remain up and directly connected. This has the dual benefit of unnecessarily increasing the load on Comcast's internal backbone as well as increasing latency for Minnesota subscribers connecting to "east of the Mississippi" destinations by ~20ms.

I'm hoping Comcast engineers read this list, or someone in the carrier community can help poke one of their Comcast contacts to help get this resolved.

Thanks in advance! "Wedged" route - 76.113.128.0/17 Correct route - 69.180.128.0/18

Example trace from Chicago source to 76.113.128.0/17: ========================================= traceroute to 76.113.128.1 (76.113.128.1), 30 hops max, 40 byte packets 1 69.65.40.62 (69.65.40.62) 0.542 ms 0.511 ms 0.508 ms 2 so2-0-0-0.er1.Chi1.Servernap.net (69.39.239.169) 1.632 ms 1.642 ms 2.121 ms 3 ge-6-20.car1.Chicago1.Level3.net (4.79.65.49) 1.605 ms 1.608 ms 1.619 ms 4 ae-2-54.edge1.Chicago2.Level3.net (4.68.101.115) 1.604 ms 1.602 ms 1.600 ms 5 COMCAST-IP.edge1.Chicago2.Level3.net (4.71.248.26) 2.735 ms 2.741 ms 2.739 ms 6 pos-0-8-0-0-cr01.denver.co.ibone.comcast.net (68.86.85.114) 27.284 ms 27.398 ms 27.387 ms 7 te-9-4-ar02.roseville.mn.minn.comcast.net (68.86.91.154) 44.177 ms * * 8 te-0-2-0-5-ar03.roseville.mn.minn.comcast.net (68.87.174.73) 28.352 ms 28.352 ms 28.349 ms 9 te-2-1-ur01.sims.mn.minn.comcast.net (68.87.174.74) 28.826 ms * * 10 te-8-3-ur02.sims.mn.minn.comcast.net (68.87.174.78) 28.959 ms * * 11 te-2-1-ur01.newport.mn.minn.comcast.net (68.87.174.82) 29.267 ms * te-2-1-ur01.newport.mn.minn.comcast.net (68.87.174.82) 28.700 ms 12 c-76-113-128-1.hsd1.mn.comcast.net (76.113.128.1) 28.638 ms 28.673 ms 28.667 ms =========================================

Example trace from Chicago source to working route 69.180.128.0/18 ========================================= traceroute to 69.180.130.1 (69.180.130.1), 30 hops max, 40 byte packets 1 69.65.40.62 (69.65.40.62) 0.482 ms 0.450 ms 0.446 ms 2 so2-0-0-0.er1.Chi1.Servernap.net (69.39.239.169) 1.595 ms 2.082 ms 2.082 ms 3 ge-6-20.car1.Chicago1.Level3.net (4.79.65.49) 1.568 ms 1.569 ms 1.579 ms 4 ae-2-52.edge1.Chicago2.Level3.net (4.68.101.51) 1.562 ms 1.563 ms 1.560 ms 5 COMCAST-IP.edge1.Chicago2.Level3.net (4.71.248.22) 2.708 ms 2.713 ms 2.711 ms 6 te-0-1-0-7-ar03.roseville.mn.minn.comcast.net (68.87.174.21) 13.144 ms 11.919 ms 11.877 ms 7 68.87.174.22 (68.87.174.22) 11.824 ms * * 8 te-8-3-ur02.brooklynpark.mn.minn.comcast.net (68.87.174.26) 12.333 ms * * 9 te-2-1-ur01.newhope.mn.minn.comcast.net (68.87.174.30) 12.012 ms * * 10 c-3-0-ubr02.newhope.mn.minn.comcast.net (69.180.130.1) 11.963 ms 12.018 ms 11.973 ms =========================================

-Eric