Possible Comcast Load Balancing issues in Chicago?
It’s possible I’m missing something, but it looks like traffic from Comcast business customers in the Minneapolis/St. Paul area is not all making it through Chicago to my VPN endpoints. Testing to some of my address space from one of the Comcast remote VPN sites, I can get to some numbers and not others (odds/evens switch roles depending on the block I trace) fall apart hopping though Chicago. Testing to the same addresses from the HE looking glass, everything seems fine. Any input or even knowledge that it’s being worked on would be appreciated. Traceroutes below: Fails: HUGO-CH-2921#trace 199.249.109.13 Type escape sequence to abort. Tracing the route to vpn.metro-inet.us (199.249.109.13) VRF info: (vrf in name/id, vrf out name/id) 1 75-149-158-38-minnesota.hfc.comcastbusiness.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 12 msec 12 msec 16 msec 3 te-4-2-ur02.whitebear.mn.minn.comcast.net (68.85.167.37) 16 msec 16 msec 16 msec 4 te-2-1-ur02.oakdale.mn.minn.comcast.net (68.87.174.138) 16 msec 16 msec 16 msec 5 te-8-3-ur01.oakdale.mn.minn.comcast.net (68.87.174.109) 20 msec 12 msec 16 msec 6 te-0-3-0-5-ar01.crosstown.mn.minn.comcast.net (69.139.176.61) 16 msec 16 msec 20 msec 7 pos-0-0-0-0-ar01.roseville.mn.minn.comcast.net (68.87.174.194) 16 msec 16 msec 12 msec 8 he-1-12-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.77) 24 msec 24 msec 28 msec 9 * * * 10 * Gets Through: HUGO-CH-2921#trace 199.249.109.14 Type escape sequence to abort. Tracing the route to 199.249.109.14 VRF info: (vrf in name/id, vrf out name/id) 1 75-149-158-38-minnesota.hfc.comcastbusiness.net (75.149.158.38) 4 msec 0 msec 0 msec 2 73.237.168.1 8 msec 12 msec 8 msec 3 te-4-2-ur01.whitebear.mn.minn.comcast.net (68.85.167.33) 12 msec 12 msec 12 msec 4 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 12 msec 12 msec 5 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 12 msec 8 msec 8 msec 6 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 12 msec 12 msec 12 msec 7 he-1-11-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.73) 28 msec 20 msec 24 msec 8 ix-7-0-3-0.tcore1.CT8-Chicago.as6453.net (64.86.137.33) 16 msec 16 msec 20 msec 9 206.82.141.90 20 msec 20 msec 20 msec 10 hurricane-ic-124397-chi-bb1.c.telia.net (213.248.104.214) 24 msec 24 msec 28 msec 11 100ge13-1.core1.msp1.he.net (184.105.223.178) 36 msec 24 msec 28 msec 12 city-of-roseville.gigabitethernet3-8.core1.msp1.he.net (184.105.249.218) 28 msec 28 msec 24 msec 13 199.249.109.227 28 msec 24 msec 28 msec Gets Through: HUGO-CH-2921#trace 67.88.181.25 Type escape sequence to abort. Tracing the route to ip67-88-181-25.z181-88-67.customer.algx.net(67.88.181.25) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 8 msec 8 msec 8 msec 3 te-4-2-ur01.whitebear.mn.minn.comcast.net (68.85.167.33) 12 msec 12 msec 8 msec 4 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 8 msec 12 msec 5 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 12 msec 12 msec 8 msec 6 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 8 msec 12 msec 12 msec 7 he-1-12-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.77) 24 msec 20 msec 16 msec 8 ix-2-3-0-0.tcore1.CT8-Chicago.as6453.net (64.86.137.41) 24 msec 16 msec 40 msec 9 p5-1.ir1.chicago2-il.us.xo.net (206.111.2.33) 68 msec 20 msec 80 msec 10 207.88.14.193.ptr.us.xo.net (207.88.14.193) 32 msec 32 msec 36 msec 11 ae0d0.mcr1.minneapolis-mn.us.xo.net (216.156.0.170) 28 msec 28 msec 28 msec Fails: HUGO-CH-2921#trace 67.88.181.26 Type escape sequence to abort. Tracing the route to webvpn.metro-inet.us (67.88.181.26) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 8 msec 8 msec 8 msec 3 te-4-2-ur02.whitebear.mn.minn.comcast.net (68.85.167.37) 8 msec 12 msec 8 msec 4 te-2-1-ur02.oakdale.mn.minn.comcast.net (68.87.174.138) 8 msec 12 msec 20 msec 5 te-8-3-ur01.oakdale.mn.minn.comcast.net (68.87.174.109) 8 msec 8 msec 12 msec 6 te-0-3-0-5-ar01.crosstown.mn.minn.comcast.net (69.139.176.61) 12 msec 12 msec 12 msec 7 pos-0-1-0-0-ar01.roseville.mn.minn.comcast.net (68.87.174.2) 8 msec 8 msec 12 msec 8 he-1-13-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.81) 24 msec 24 msec 20 msec 9 * * * 10 * * * 11 * * * 12 * * * Gets Through: HUGO-CH-2921#trace 67.88.181.27 Type escape sequence to abort. Tracing the route to ip67-88-181-27.z181-88-67.customer.algx.net(67.88.181.27) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 4 msec 0 msec 4 msec 2 73.237.168.1 8 msec 12 msec 8 msec 3 te-4-2-ur02.whitebear.mn.minn.comcast.net (68.85.167.37) 12 msec 12 msec 8 msec 4 te-8-3-ur01.whitebear.mn.minn.comcast.net (68.87.174.133) 12 msec 12 msec 4 msec 5 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 12 msec 8 msec 6 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 16 msec 8 msec 8 msec 7 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 12 msec 12 msec 16 msec 8 he-1-13-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.81) 20 msec 20 msec 20 msec 9 ix-2-2-1-0.tcore1.CT8-Chicago.as6453.net (64.86.137.37) 16 msec 16 msec 16 msec 10 p5-1.ir1.chicago2-il.us.xo.net (206.111.2.33) 32 msec 72 msec 96 msec 11 207.88.14.193.ptr.us.xo.net (207.88.14.193) 36 msec 32 msec 32 msec 12 ae0d0.mcr1.minneapolis-mn.us.xo.net (216.156.0.170) 28 msec 28 msec 32 msec 13 * * * Fails: HUGO-CH-2921#trace 67.88.181.28 Type escape sequence to abort. Tracing the route to ip67-88-181-28.z181-88-67.customer.algx.net(67.88.181.28) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 12 msec 8 msec 12 msec 3 te-4-2-ur01.whitebear.mn.minn.comcast.net (68.85.167.33) 8 msec 8 msec 8 msec 4 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 8 msec 12 msec 5 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 12 msec 8 msec 8 msec 6 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 12 msec 12 msec 12 msec 7 he-1-11-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.73) 20 msec 20 msec 20 msec 8 * * * 9 * * * 10 * Mark Mayfield City of Roseville – AS 54371 Network Systems Engineer 2660 Civic Center Drive Roseville, MN 55113 651-792-7098 Office
Looking into. -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Metro-Inet Netops Sent: Thursday, May 15, 2014 11:10 AM To: nanog@nanog.org Subject: Possible Comcast Load Balancing issues in Chicago? It’s possible I’m missing something, but it looks like traffic from Comcast business customers in the Minneapolis/St. Paul area is not all making it through Chicago to my VPN endpoints. Testing to some of my address space from one of the Comcast remote VPN sites, I can get to some numbers and not others (odds/evens switch roles depending on the block I trace) fall apart hopping though Chicago. Testing to the same addresses from the HE looking glass, everything seems fine. Any input or even knowledge that it’s being worked on would be appreciated. Traceroutes below: Fails: HUGO-CH-2921#trace 199.249.109.13 Type escape sequence to abort. Tracing the route to vpn.metro-inet.us (199.249.109.13) VRF info: (vrf in name/id, vrf out name/id) 1 75-149-158-38-minnesota.hfc.comcastbusiness.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 12 msec 12 msec 16 msec 3 te-4-2-ur02.whitebear.mn.minn.comcast.net (68.85.167.37) 16 msec 16 msec 16 msec 4 te-2-1-ur02.oakdale.mn.minn.comcast.net (68.87.174.138) 16 msec 16 msec 16 msec 5 te-8-3-ur01.oakdale.mn.minn.comcast.net (68.87.174.109) 20 msec 12 msec 16 msec 6 te-0-3-0-5-ar01.crosstown.mn.minn.comcast.net (69.139.176.61) 16 msec 16 msec 20 msec 7 pos-0-0-0-0-ar01.roseville.mn.minn.comcast.net (68.87.174.194) 16 msec 16 msec 12 msec 8 he-1-12-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.77) 24 msec 24 msec 28 msec 9 * * * 10 * Gets Through: HUGO-CH-2921#trace 199.249.109.14 Type escape sequence to abort. Tracing the route to 199.249.109.14 VRF info: (vrf in name/id, vrf out name/id) 1 75-149-158-38-minnesota.hfc.comcastbusiness.net (75.149.158.38) 4 msec 0 msec 0 msec 2 73.237.168.1 8 msec 12 msec 8 msec 3 te-4-2-ur01.whitebear.mn.minn.comcast.net (68.85.167.33) 12 msec 12 msec 12 msec 4 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 12 msec 12 msec 5 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 12 msec 8 msec 8 msec 6 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 12 msec 12 msec 12 msec 7 he-1-11-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.73) 28 msec 20 msec 24 msec 8 ix-7-0-3-0.tcore1.CT8-Chicago.as6453.net (64.86.137.33) 16 msec 16 msec 20 msec 9 206.82.141.90 20 msec 20 msec 20 msec 10 hurricane-ic-124397-chi-bb1.c.telia.net (213.248.104.214) 24 msec 24 msec 28 msec 11 100ge13-1.core1.msp1.he.net (184.105.223.178) 36 msec 24 msec 28 msec 12 city-of-roseville.gigabitethernet3-8.core1.msp1.he.net (184.105.249.218) 28 msec 28 msec 24 msec 13 199.249.109.227 28 msec 24 msec 28 msec Gets Through: HUGO-CH-2921#trace 67.88.181.25 Type escape sequence to abort. Tracing the route to ip67-88-181-25.z181-88-67.customer.algx.net(67.88.181.25) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 8 msec 8 msec 8 msec 3 te-4-2-ur01.whitebear.mn.minn.comcast.net (68.85.167.33) 12 msec 12 msec 8 msec 4 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 8 msec 12 msec 5 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 12 msec 12 msec 8 msec 6 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 8 msec 12 msec 12 msec 7 he-1-12-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.77) 24 msec 20 msec 16 msec 8 ix-2-3-0-0.tcore1.CT8-Chicago.as6453.net (64.86.137.41) 24 msec 16 msec 40 msec 9 p5-1.ir1.chicago2-il.us.xo.net (206.111.2.33) 68 msec 20 msec 80 msec 10 207.88.14.193.ptr.us.xo.net (207.88.14.193) 32 msec 32 msec 36 msec 11 ae0d0.mcr1.minneapolis-mn.us.xo.net (216.156.0.170) 28 msec 28 msec 28 msec Fails: HUGO-CH-2921#trace 67.88.181.26 Type escape sequence to abort. Tracing the route to webvpn.metro-inet.us (67.88.181.26) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 8 msec 8 msec 8 msec 3 te-4-2-ur02.whitebear.mn.minn.comcast.net (68.85.167.37) 8 msec 12 msec 8 msec 4 te-2-1-ur02.oakdale.mn.minn.comcast.net (68.87.174.138) 8 msec 12 msec 20 msec 5 te-8-3-ur01.oakdale.mn.minn.comcast.net (68.87.174.109) 8 msec 8 msec 12 msec 6 te-0-3-0-5-ar01.crosstown.mn.minn.comcast.net (69.139.176.61) 12 msec 12 msec 12 msec 7 pos-0-1-0-0-ar01.roseville.mn.minn.comcast.net (68.87.174.2) 8 msec 8 msec 12 msec 8 he-1-13-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.81) 24 msec 24 msec 20 msec 9 * * * 10 * * * 11 * * * 12 * * * Gets Through: HUGO-CH-2921#trace 67.88.181.27 Type escape sequence to abort. Tracing the route to ip67-88-181-27.z181-88-67.customer.algx.net(67.88.181.27) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 4 msec 0 msec 4 msec 2 73.237.168.1 8 msec 12 msec 8 msec 3 te-4-2-ur02.whitebear.mn.minn.comcast.net (68.85.167.37) 12 msec 12 msec 8 msec 4 te-8-3-ur01.whitebear.mn.minn.comcast.net (68.87.174.133) 12 msec 12 msec 4 msec 5 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 12 msec 8 msec 6 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 16 msec 8 msec 8 msec 7 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 12 msec 12 msec 16 msec 8 he-1-13-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.81) 20 msec 20 msec 20 msec 9 ix-2-2-1-0.tcore1.CT8-Chicago.as6453.net (64.86.137.37) 16 msec 16 msec 16 msec 10 p5-1.ir1.chicago2-il.us.xo.net (206.111.2.33) 32 msec 72 msec 96 msec 11 207.88.14.193.ptr.us.xo.net (207.88.14.193) 36 msec 32 msec 32 msec 12 ae0d0.mcr1.minneapolis-mn.us.xo.net (216.156.0.170) 28 msec 28 msec 32 msec 13 * * * Fails: HUGO-CH-2921#trace 67.88.181.28 Type escape sequence to abort. Tracing the route to ip67-88-181-28.z181-88-67.customer.algx.net(67.88.181.28) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 12 msec 8 msec 12 msec 3 te-4-2-ur01.whitebear.mn.minn.comcast.net (68.85.167.33) 8 msec 8 msec 8 msec 4 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 8 msec 12 msec 5 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 12 msec 8 msec 8 msec 6 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 12 msec 12 msec 12 msec 7 he-1-11-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.73) 20 msec 20 msec 20 msec 8 * * * 9 * * * 10 * Mark Mayfield City of Roseville – AS 54371 Network Systems Engineer 2660 Civic Center Drive Roseville, MN 55113 651-792-7098 Office
Seems to have cleared up about 30 minutes ago. Thanks, Mark Mayfield City of Roseville – AS 54371 Network Systems Engineer 2660 Civic Center Drive Roseville, MN 55113 651-792-7098 Office -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Smith, Courtney Sent: Thursday, May 15, 2014 10:32 AM To: Metro-Inet Netops Cc: nanog@nanog.org Subject: RE: Possible Comcast Load Balancing issues in Chicago? Looking into. -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Metro-Inet Netops Sent: Thursday, May 15, 2014 11:10 AM To: nanog@nanog.org Subject: Possible Comcast Load Balancing issues in Chicago? It’s possible I’m missing something, but it looks like traffic from Comcast business customers in the Minneapolis/St. Paul area is not all making it through Chicago to my VPN endpoints. Testing to some of my address space from one of the Comcast remote VPN sites, I can get to some numbers and not others (odds/evens switch roles depending on the block I trace) fall apart hopping though Chicago. Testing to the same addresses from the HE looking glass, everything seems fine. Any input or even knowledge that it’s being worked on would be appreciated. Traceroutes below: Fails: HUGO-CH-2921#trace 199.249.109.13 Type escape sequence to abort. Tracing the route to vpn.metro-inet.us (199.249.109.13) VRF info: (vrf in name/id, vrf out name/id) 1 75-149-158-38-minnesota.hfc.comcastbusiness.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 12 msec 12 msec 16 msec 3 te-4-2-ur02.whitebear.mn.minn.comcast.net (68.85.167.37) 16 msec 16 msec 16 msec 4 te-2-1-ur02.oakdale.mn.minn.comcast.net (68.87.174.138) 16 msec 16 msec 16 msec 5 te-8-3-ur01.oakdale.mn.minn.comcast.net (68.87.174.109) 20 msec 12 msec 16 msec 6 te-0-3-0-5-ar01.crosstown.mn.minn.comcast.net (69.139.176.61) 16 msec 16 msec 20 msec 7 pos-0-0-0-0-ar01.roseville.mn.minn.comcast.net (68.87.174.194) 16 msec 16 msec 12 msec 8 he-1-12-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.77) 24 msec 24 msec 28 msec 9 * * * 10 * Gets Through: HUGO-CH-2921#trace 199.249.109.14 Type escape sequence to abort. Tracing the route to 199.249.109.14 VRF info: (vrf in name/id, vrf out name/id) 1 75-149-158-38-minnesota.hfc.comcastbusiness.net (75.149.158.38) 4 msec 0 msec 0 msec 2 73.237.168.1 8 msec 12 msec 8 msec 3 te-4-2-ur01.whitebear.mn.minn.comcast.net (68.85.167.33) 12 msec 12 msec 12 msec 4 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 12 msec 12 msec 5 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 12 msec 8 msec 8 msec 6 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 12 msec 12 msec 12 msec 7 he-1-11-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.73) 28 msec 20 msec 24 msec 8 ix-7-0-3-0.tcore1.CT8-Chicago.as6453.net (64.86.137.33) 16 msec 16 msec 20 msec 9 206.82.141.90 20 msec 20 msec 20 msec 10 hurricane-ic-124397-chi-bb1.c.telia.net (213.248.104.214) 24 msec 24 msec 28 msec 11 100ge13-1.core1.msp1.he.net (184.105.223.178) 36 msec 24 msec 28 msec 12 city-of-roseville.gigabitethernet3-8.core1.msp1.he.net (184.105.249.218) 28 msec 28 msec 24 msec 13 199.249.109.227 28 msec 24 msec 28 msec Gets Through: HUGO-CH-2921#trace 67.88.181.25 Type escape sequence to abort. Tracing the route to ip67-88-181-25.z181-88-67.customer.algx.net(67.88.181.25) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 8 msec 8 msec 8 msec 3 te-4-2-ur01.whitebear.mn.minn.comcast.net (68.85.167.33) 12 msec 12 msec 8 msec 4 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 8 msec 12 msec 5 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 12 msec 12 msec 8 msec 6 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 8 msec 12 msec 12 msec 7 he-1-12-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.77) 24 msec 20 msec 16 msec 8 ix-2-3-0-0.tcore1.CT8-Chicago.as6453.net (64.86.137.41) 24 msec 16 msec 40 msec 9 p5-1.ir1.chicago2-il.us.xo.net (206.111.2.33) 68 msec 20 msec 80 msec 10 207.88.14.193.ptr.us.xo.net (207.88.14.193) 32 msec 32 msec 36 msec 11 ae0d0.mcr1.minneapolis-mn.us.xo.net (216.156.0.170) 28 msec 28 msec 28 msec Fails: HUGO-CH-2921#trace 67.88.181.26 Type escape sequence to abort. Tracing the route to webvpn.metro-inet.us (67.88.181.26) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 8 msec 8 msec 8 msec 3 te-4-2-ur02.whitebear.mn.minn.comcast.net (68.85.167.37) 8 msec 12 msec 8 msec 4 te-2-1-ur02.oakdale.mn.minn.comcast.net (68.87.174.138) 8 msec 12 msec 20 msec 5 te-8-3-ur01.oakdale.mn.minn.comcast.net (68.87.174.109) 8 msec 8 msec 12 msec 6 te-0-3-0-5-ar01.crosstown.mn.minn.comcast.net (69.139.176.61) 12 msec 12 msec 12 msec 7 pos-0-1-0-0-ar01.roseville.mn.minn.comcast.net (68.87.174.2) 8 msec 8 msec 12 msec 8 he-1-13-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.81) 24 msec 24 msec 20 msec 9 * * * 10 * * * 11 * * * 12 * * * Gets Through: HUGO-CH-2921#trace 67.88.181.27 Type escape sequence to abort. Tracing the route to ip67-88-181-27.z181-88-67.customer.algx.net(67.88.181.27) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 4 msec 0 msec 4 msec 2 73.237.168.1 8 msec 12 msec 8 msec 3 te-4-2-ur02.whitebear.mn.minn.comcast.net (68.85.167.37) 12 msec 12 msec 8 msec 4 te-8-3-ur01.whitebear.mn.minn.comcast.net (68.87.174.133) 12 msec 12 msec 4 msec 5 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 12 msec 8 msec 6 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 16 msec 8 msec 8 msec 7 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 12 msec 12 msec 16 msec 8 he-1-13-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.81) 20 msec 20 msec 20 msec 9 ix-2-2-1-0.tcore1.CT8-Chicago.as6453.net (64.86.137.37) 16 msec 16 msec 16 msec 10 p5-1.ir1.chicago2-il.us.xo.net (206.111.2.33) 32 msec 72 msec 96 msec 11 207.88.14.193.ptr.us.xo.net (207.88.14.193) 36 msec 32 msec 32 msec 12 ae0d0.mcr1.minneapolis-mn.us.xo.net (216.156.0.170) 28 msec 28 msec 32 msec 13 * * * Fails: HUGO-CH-2921#trace 67.88.181.28 Type escape sequence to abort. Tracing the route to ip67-88-181-28.z181-88-67.customer.algx.net(67.88.181.28) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 12 msec 8 msec 12 msec 3 te-4-2-ur01.whitebear.mn.minn.comcast.net (68.85.167.33) 8 msec 8 msec 8 msec 4 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 8 msec 12 msec 5 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 12 msec 8 msec 8 msec 6 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 12 msec 12 msec 12 msec 7 he-1-11-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.73) 20 msec 20 msec 20 msec 8 * * * 9 * * * 10 * Mark Mayfield City of Roseville – AS 54371 Network Systems Engineer 2660 Civic Center Drive Roseville, MN 55113 651-792-7098 Office
Mark, Faulty circuit was removed from path. NOC is working with TATA(AS6453) to repair. Thanks. Courtney Smith Network Engineer Comcast http://www.comcastwholesale.com/ -----Original Message----- From: Mark Mayfield [mailto:Mark.Mayfield@metro-inet.us] Sent: Thursday, May 15, 2014 12:04 PM To: Smith, Courtney; Metro-Inet Netops Cc: nanog@nanog.org Subject: RE: Possible Comcast Load Balancing issues in Chicago? Seems to have cleared up about 30 minutes ago. Thanks, Mark Mayfield City of Roseville – AS 54371 Network Systems Engineer 2660 Civic Center Drive Roseville, MN 55113 651-792-7098 Office -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Smith, Courtney Sent: Thursday, May 15, 2014 10:32 AM To: Metro-Inet Netops Cc: nanog@nanog.org Subject: RE: Possible Comcast Load Balancing issues in Chicago? Looking into. -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Metro-Inet Netops Sent: Thursday, May 15, 2014 11:10 AM To: nanog@nanog.org Subject: Possible Comcast Load Balancing issues in Chicago? It’s possible I’m missing something, but it looks like traffic from Comcast business customers in the Minneapolis/St. Paul area is not all making it through Chicago to my VPN endpoints. Testing to some of my address space from one of the Comcast remote VPN sites, I can get to some numbers and not others (odds/evens switch roles depending on the block I trace) fall apart hopping though Chicago. Testing to the same addresses from the HE looking glass, everything seems fine. Any input or even knowledge that it’s being worked on would be appreciated. Traceroutes below: Fails: HUGO-CH-2921#trace 199.249.109.13 Type escape sequence to abort. Tracing the route to vpn.metro-inet.us (199.249.109.13) VRF info: (vrf in name/id, vrf out name/id) 1 75-149-158-38-minnesota.hfc.comcastbusiness.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 12 msec 12 msec 16 msec 3 te-4-2-ur02.whitebear.mn.minn.comcast.net (68.85.167.37) 16 msec 16 msec 16 msec 4 te-2-1-ur02.oakdale.mn.minn.comcast.net (68.87.174.138) 16 msec 16 msec 16 msec 5 te-8-3-ur01.oakdale.mn.minn.comcast.net (68.87.174.109) 20 msec 12 msec 16 msec 6 te-0-3-0-5-ar01.crosstown.mn.minn.comcast.net (69.139.176.61) 16 msec 16 msec 20 msec 7 pos-0-0-0-0-ar01.roseville.mn.minn.comcast.net (68.87.174.194) 16 msec 16 msec 12 msec 8 he-1-12-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.77) 24 msec 24 msec 28 msec 9 * * * 10 * Gets Through: HUGO-CH-2921#trace 199.249.109.14 Type escape sequence to abort. Tracing the route to 199.249.109.14 VRF info: (vrf in name/id, vrf out name/id) 1 75-149-158-38-minnesota.hfc.comcastbusiness.net (75.149.158.38) 4 msec 0 msec 0 msec 2 73.237.168.1 8 msec 12 msec 8 msec 3 te-4-2-ur01.whitebear.mn.minn.comcast.net (68.85.167.33) 12 msec 12 msec 12 msec 4 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 12 msec 12 msec 5 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 12 msec 8 msec 8 msec 6 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 12 msec 12 msec 12 msec 7 he-1-11-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.73) 28 msec 20 msec 24 msec 8 ix-7-0-3-0.tcore1.CT8-Chicago.as6453.net (64.86.137.33) 16 msec 16 msec 20 msec 9 206.82.141.90 20 msec 20 msec 20 msec 10 hurricane-ic-124397-chi-bb1.c.telia.net (213.248.104.214) 24 msec 24 msec 28 msec 11 100ge13-1.core1.msp1.he.net (184.105.223.178) 36 msec 24 msec 28 msec 12 city-of-roseville.gigabitethernet3-8.core1.msp1.he.net (184.105.249.218) 28 msec 28 msec 24 msec 13 199.249.109.227 28 msec 24 msec 28 msec Gets Through: HUGO-CH-2921#trace 67.88.181.25 Type escape sequence to abort. Tracing the route to ip67-88-181-25.z181-88-67.customer.algx.net(67.88.181.25) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 8 msec 8 msec 8 msec 3 te-4-2-ur01.whitebear.mn.minn.comcast.net (68.85.167.33) 12 msec 12 msec 8 msec 4 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 8 msec 12 msec 5 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 12 msec 12 msec 8 msec 6 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 8 msec 12 msec 12 msec 7 he-1-12-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.77) 24 msec 20 msec 16 msec 8 ix-2-3-0-0.tcore1.CT8-Chicago.as6453.net (64.86.137.41) 24 msec 16 msec 40 msec 9 p5-1.ir1.chicago2-il.us.xo.net (206.111.2.33) 68 msec 20 msec 80 msec 10 207.88.14.193.ptr.us.xo.net (207.88.14.193) 32 msec 32 msec 36 msec 11 ae0d0.mcr1.minneapolis-mn.us.xo.net (216.156.0.170) 28 msec 28 msec 28 msec Fails: HUGO-CH-2921#trace 67.88.181.26 Type escape sequence to abort. Tracing the route to webvpn.metro-inet.us (67.88.181.26) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 8 msec 8 msec 8 msec 3 te-4-2-ur02.whitebear.mn.minn.comcast.net (68.85.167.37) 8 msec 12 msec 8 msec 4 te-2-1-ur02.oakdale.mn.minn.comcast.net (68.87.174.138) 8 msec 12 msec 20 msec 5 te-8-3-ur01.oakdale.mn.minn.comcast.net (68.87.174.109) 8 msec 8 msec 12 msec 6 te-0-3-0-5-ar01.crosstown.mn.minn.comcast.net (69.139.176.61) 12 msec 12 msec 12 msec 7 pos-0-1-0-0-ar01.roseville.mn.minn.comcast.net (68.87.174.2) 8 msec 8 msec 12 msec 8 he-1-13-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.81) 24 msec 24 msec 20 msec 9 * * * 10 * * * 11 * * * 12 * * * Gets Through: HUGO-CH-2921#trace 67.88.181.27 Type escape sequence to abort. Tracing the route to ip67-88-181-27.z181-88-67.customer.algx.net(67.88.181.27) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 4 msec 0 msec 4 msec 2 73.237.168.1 8 msec 12 msec 8 msec 3 te-4-2-ur02.whitebear.mn.minn.comcast.net (68.85.167.37) 12 msec 12 msec 8 msec 4 te-8-3-ur01.whitebear.mn.minn.comcast.net (68.87.174.133) 12 msec 12 msec 4 msec 5 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 12 msec 8 msec 6 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 16 msec 8 msec 8 msec 7 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 12 msec 12 msec 16 msec 8 he-1-13-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.81) 20 msec 20 msec 20 msec 9 ix-2-2-1-0.tcore1.CT8-Chicago.as6453.net (64.86.137.37) 16 msec 16 msec 16 msec 10 p5-1.ir1.chicago2-il.us.xo.net (206.111.2.33) 32 msec 72 msec 96 msec 11 207.88.14.193.ptr.us.xo.net (207.88.14.193) 36 msec 32 msec 32 msec 12 ae0d0.mcr1.minneapolis-mn.us.xo.net (216.156.0.170) 28 msec 28 msec 32 msec 13 * * * Fails: HUGO-CH-2921#trace 67.88.181.28 Type escape sequence to abort. Tracing the route to ip67-88-181-28.z181-88-67.customer.algx.net(67.88.181.28) VRF info: (vrf in name/id, vrf out name/id) 1 pubwp.comcast.net (75.149.158.38) 0 msec 0 msec 0 msec 2 73.237.168.1 12 msec 8 msec 12 msec 3 te-4-2-ur01.whitebear.mn.minn.comcast.net (68.85.167.33) 8 msec 8 msec 8 msec 4 te-2-1-ur02.shoreview.mn.minn.comcast.net (68.87.174.129) 8 msec 8 msec 12 msec 5 te-8-3-ur01.shoreview.mn.minn.comcast.net (68.87.174.125) 12 msec 8 msec 8 msec 6 te-0-3-0-6-ar01.roseville.mn.minn.comcast.net (68.87.174.178) 12 msec 12 msec 12 msec 7 he-1-11-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.94.73) 20 msec 20 msec 20 msec 8 * * * 9 * * * 10 * Mark Mayfield City of Roseville – AS 54371 Network Systems Engineer 2660 Civic Center Drive Roseville, MN 55113 651-792-7098 Office
participants (3)
-
Mark Mayfield -
Metro-Inet Netops -
Smith, Courtney