At 08:27 AM 31/07/2004, Krichbaum, Eric wrote:
If it's every 60 seconds, I'd suspect the BGP timer is the root. They probably forgot to use next-hop self or a static route to a peer. The end result being that the route to the bgp peer is learned via bgp itself...
Maybe. Hard to say if thats what their default hold time is. I am still seeing the odd hit in their network. For the sites we connect with inside Bell, the tunnel LQR expire is 10 seconds and we have seen 2 big bounces since routing around this morning. I emailed their noc, but no response. The Bell looking glass doesnt seem to have any flap statistics so I dont know if things are bouncing inside :( It is looking different once again. Via Cogent to me, 194# traceroute 199.212.134.1 traceroute to 199.212.134.1 (199.212.134.1), 64 hops max, 44 byte packets 1 HSE-MTL-ppp12931.qc.sympatico.ca (209.226.183.241) 266.986 ms 229.511 ms 209.679 ms 2 Hamilton-ppp278329.sympatico.ca (206.172.130.250) 419.439 ms 219.614 ms 199.657 ms 3 kitcorr01-fe0-0-0.15.in.bellnexxia.net (206.47.229.8) 219.461 ms 239.605 ms 187.514 ms 4 badBellDNS (64.230.241.125) 221.605 ms 209.597 ms 199.655 ms 5 badBellDNS (64.230.242.194) 219.439 ms 227.155 ms 202.135 ms 6 core2-chicago23-pos10-0.in.bellnexxia.net (206.108.103.118) 229.386 ms 249.620 ms 346.174 ms 7 bx1-chicago23-pos11-0.in.bellnexxia.net (206.108.103.125) 228.434 ms 234.041 ms 219.645 ms 8 p13-0.core01.ord01.atlas.cogentco.com (154.54.11.29) 249.438 ms 239.589 ms 199.658 ms 9 p15-0.core02.ord01.atlas.cogentco.com (66.28.4.62) 239.441 ms 249.618 ms 229.679 ms 10 p5-0.core01.yyz01.atlas.cogentco.com (66.28.4.214) 249.409 ms 237.480 ms 231.808 ms 11 g0-1.na01.b011027-0.yyz01.atlas.cogentco.com (66.250.14.230) 251.504 ms 259.618 ms 219.669 ms 12 1572534Ontario.demarc.cogentco.com (38.112.5.166) 239.467 ms 249.593 ms 219.660 ms 13 tor-hespler-360-dslgate.sentex.ca (64.7.143.43) 229.489 ms 249.606 ms 214.908 ms 14 hespler-tor-360-i4.sentex.ca (64.7.143.46) 241.707 ms 229.615 ms 219.680 ms 15 ns.sentex.ca (199.212.134.1) 259.436 ms 239.613 ms 215.514 ms Hops 6 and 8 were coming back as * * * on the traceroute a few hrs ago, but packets were getting to and from me. Hopefully someone from Bell will pipe up on or offlist as to what the problem was / is and if its resolved. Telus is my main transit, and I dont like having to use such a blunt approach to working around this issue :( ---Mike
Eric Krichbaum, Chief Engineer MCSE, CCNP, CCDP, CCSP, CCIP
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Mike Tancsa Sent: Saturday, July 31, 2004 7:52 AM To: nanog@nanog.org Subject: Next hop issues inside AS577 to AS852?
Unfortunately, I am not a direct customer of AS577 otherwise I would open a ticket with them, but we have a lot of sites inside Bell Canada that need to reach us.
Starting suspiciously at maintenance window time, we were seeing sporadic reachability issues coming at us from Bell. I am pretty sure its to us, and not the other way around as exiting out, I always prefer my GT/360 link and depending on the source IP it always works. The path back to me was via AS852 (telus) but I had to massively prepend to force it via someone else to get things working.
But here are 2 traceroutes from inside AS577 (Bell) back to me
Traceroute a) 194# traceroute -n 64.7.153.1 traceroute to 64.7.153.1 (64.7.153.1), 64 hops max, 44 byte packets 1 209.226.183.241 266.863 ms 219.712 ms 199.588 ms 2 206.172.130.250 219.408 ms 219.700 ms 209.619 ms 3 206.47.229.8 239.433 ms 195.376 ms 193.984 ms 4 64.230.241.125 229.460 ms 199.675 ms 209.669 ms 5 64.230.242.150 209.415 ms 189.707 ms 199.627 ms 6 154.11.3.25 239.440 ms 219.684 ms 208.091 ms 7 154.11.6.17 241.001 ms 217.379 ms 231.976 ms 8 64.7.143.44 229.456 ms 199.648 ms 189.635 ms 9 64.7.143.45 224.784 ms 194.333 ms 189.678 ms 10 64.7.153.1 229.431 ms 209.654 ms 209.672 ms
traceroute b) 194# traceroute -n 64.7.153.1 traceroute to 64.7.153.56 (64.7.153.56), 64 hops max, 44 byte packets 1 209.226.183.241 278.871 ms 221.986 ms 189.687 ms 2 206.172.130.250 219.433 ms 229.680 ms 339.650 ms 3 206.47.229.19 239.447 ms 189.621 ms 195.621 ms 4 64.230.241.121 223.483 ms 209.663 ms 199.695 ms 5 64.230.242.97 329.445 ms 209.667 ms 229.666 ms 6 64.230.242.181 239.463 ms 196.798 ms 192.568 ms 7 * * * 8 * * *
Every 60 seconds or so the path back to me inside AS577 would change back and forth between a) and b). I dont know what Hop 7 on b) is. It could be another peer to AS852 (Telus) or just another internal router at Bell (AS577). Suffice to say, when taking path b) packets never get back to me.
To work around it, I had to prepend out my AS852 link so that Bell comes back at me via GT/360 or Cogent.
Anyone from Bell or Telus around to clarify where the problem is? Sadly, this is a holiday long weekend here in Canada :( The wheels fell off around 4:30 AM EST.
---Mike
-------------------------------------------------------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet since 1994 www.sentex.net Cambridge, Ontario Canada www.sentex.net/mike