Hi, Yeah, I was just seeing some issues through TATA (AS6453) with routes being blackholed in Newark, or at least that was where the packets stopped. I had to shut my peer with them and I just finished opening a trouble ticket. A traceroute to 209.167.35.0/24 from a source addr in 205.211.164.0/24 byte packets 1 teleglobe-vl38-tor (205.211.165.121) 0.259 ms 0.465 ms 0.488 ms 2 if-2-3-513.core4.TNK-Toronto.as6453.net (209.58.16.21) 0.987 ms 0.981 ms 0.565 ms 3 if-14-0-0.mcore3.TTT-Scarborough.as6453.net (63.243.172.2) 53.390 ms 2.973 ms 2.991 ms 4 if-3-1-0-0.tcore1.NJY-Newark.as6453.net (216.6.98.34) 20.481 ms 62.938 ms 20.967 ms 5 if-2-2.tcore2.NJY-Newark.as6453.net (66.198.70.2) 20.986 ms 21.059 ms 20.871 ms 6 * * * Not sure if it was after them or coming back to them. Same source addr through AS174 is fine ---Mike On 8/3/2011 3:27 PM, James Smallacombe wrote:
I'm having issues through Verizon too...I have a server colocated in Vancouver...could it be a Canadian thing with Verizon?
2 l100.phlapa-vfttp-60.verizon-gni.net (98.114.95.1) 8.910 ms 8.760 ms 7.026 ms 3 g3-0-2-860.phlapa-lcr-08.verizon-gni.net (130.81.139.120) 10.711 ms 8.466 ms 10.698 ms 4 * * * 5 so-13-2-0-0.res-bb-rtr2.verizon-gni.net (130.81.19.118) 14.937 ms 15.975 ms 15.148 ms 6 0.ae2.br2.iad8.alter.net (152.63.34.73) 14.346 ms 13.943 ms 14.833 ms 7 * * * 8 * * *
I can ssh to the box from other networks, and here's a traceroute back to my Verzon FIOS IP...other Verizon customers (DSL, etc) report same problem:
2 static-209-17-142-114.gtcust.grouptelecom.net (209.17.142.114) 0.744 ms 0.642 ms 0.620 ms 3 static-66-38-255-9.gtcust.grouptelecom.net (66.38.255.9) 0.750 ms 0.657 ms 0.649 ms 4 GE3-0.PEERA-VANCBC.IP.GROUPTELECOM.NET (66.59.190.6) 0.730 ms 0.694 ms 0.693 ms 5 bx4-vancouver_G1-1-6.net.bell.ca (67.69.199.105) 0.847 ms 0.836 ms 0.828 ms 6 core4-vancouver_ge8-0-0.net.bell.ca (64.230.183.109) 4.637 ms core3-vancouver_ge8-0-0.net.bell.ca (64.230.183.105) 113.794 ms 17.328 ms 7 core2-seattle_pos4-0-0_core.net.bell.ca (64.230.144.97) 23.686 ms core1-seattle_pos6-0-0_core.net.bell.ca (64.230.144.89) 4.672 ms core2-seattle_pos4-0-0_core.net.bell.ca (64.230.144.97) 5.720 ms 8 bx2-seattle_POS10-0-0.net.bell.ca (64.230.186.22) 4.358 ms bx2-seattle_POS11-0-0.net.bell.ca (64.230.186.26) 4.396 ms bx2-seattle_POS10-0-0.net.bell.ca (64.230.186.22) 4.353 ms 9 Comcast-peering.net.bell.ca (67.69.246.198) 4.991 ms 4.795 ms 4.795 ms 10 pos-0-4-0-0-cr01.seattle.wa.ibone.comcast.net (68.86.86.137) 8.285 ms 5.258 ms 4.919 ms 11 pos-0-6-0-0-cr01.denver.co.ibone.comcast.net (68.86.87.49) 46.892 ms 46.901 ms 46.952 ms 12 pos-0-13-0-0-cr01.chicago.il.ibone.comcast.net (68.86.85.245) 57.243 ms 57.359 ms 57.333 ms 13 pos-2-13-0-0-cr01.newyork.ny.ibone.comcast.net (68.86.87.25) 85.080 ms 85.020 ms 85.069 ms 14 te-2-1-pe01.philadelphia.pa.ibone.comcast.net (68.86.84.194) 87.264 ms 86.372 ms 86.453 ms 15 75.149.230.250 (75.149.230.250) 86.548 ms 86.489 ms 86.439 ms
On Mon, 28 Feb 2011, Mike Tancsa wrote:
I was just looking at an issue between 701 in Toronto. Seems to be resolved now-- at least the issue I was seeing.
the bad traceroute, looked like .... 3 if-3-0-2.core3.TNK-Toronto.as6453.net (64.86.81.3) 0.988 ms 0.978 ms 1.578 ms 4 209.58.94.10 (209.58.94.10) 1.902 ms 71.416 ms 3.472 ms 5 if-3-1-0-0.tcore1.NJY-Newark.as6453.net (216.6.98.34) 22.286 ms 21.957 ms 29.472 ms 6 if-12-0.mcore3.NJY-Newark.as6453.net (66.198.70.14) 67.961 ms if-3-0-0.mcore3.NJY-Newark.as6453.net (216.6.57.121) 21.449 ms 20.956 ms 7 if-10-0.core3.NTO-NewYork.as6453.net (216.6.57.66) 21.975 ms 22.467 ms 21.977 ms 8 Vlan1297.icore1.NTO-NewYork.as6453.net (209.58.26.49) 20.977 ms * 30.520 ms 9 0.ae20.BR2.NYC4.ALTER.NET (204.255.168.173) 21.478 ms 21.458 ms 21.477 ms 10 * 11 *
Now its working
.... 3 if-3-0-2.core3.TNK-Toronto.as6453.net (64.86.81.3) 0.975 ms 4 209.58.94.10 (209.58.94.10) 1.975 ms 5 if-3-1-0-0.tcore1.NJY-Newark.as6453.net (216.6.98.34) 21.445 ms 6 if-12-0.mcore3.NJY-Newark.as6453.net (66.198.70.14) 54.472 ms 7 if-10-0.core3.NTO-NewYork.as6453.net (216.6.57.66) 112.426 ms 8 Vlan1297.icore1.NTO-NewYork.as6453.net (209.58.26.49) 22.964 ms 9 0.ae20.BR2.NYC4.ALTER.NET (204.255.168.173) 21.455 ms 10 0.ae2.XL4.NYC4.ALTER.NET (152.63.3.117) 21.466 ms 11 0.so-5-1-0.XT2.TOR2.ALTER.NET (152.63.128.121) 34.958 ms 12 0.POS7-1.GW2.TOR2.ALTER.NET (152.63.131.205) 33.958 ms
When it was not working, packets would not get from my AS (11647) to the target in IP in AS701. But packets from 701 would get back to my AS. The AS path in both directions are 701-6453-11647 and 11647 6453 701... I saw a similar outage to VPNs I have in AS15290 which I see as 11647 6453 701 15290. However, I did not have time to check if it was the same behaviour with loss being in one direction. In both cases, IPs that follow 11647 174 701 and 701 174 11647 and 11647 174 7018 15290 and 15290 7018 174 11647 were not impacted.
---Mike
On 2/28/2011 9:53 PM, ML wrote:
Seeing some packet loss via Cogent.
www.internetpulse.net seems to be lighting up.
-- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/
James Smallacombe PlantageNet, Inc. CEO and Janitor up@3.am http://3.am =========================================================================
-- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/