Verizon Issues? East Coast US
Seeing some packet loss via Cogent. www.internetpulse.net seems to be lighting up.
I'm noticing it too. POP in Grand Rapids, circuit to Detroit. Packet loss, but no total loss of connectivity. -----Original Message----- From: ML [mailto:ml@kenweb.org] Sent: Monday, February 28, 2011 9:53 PM To: NANOG Subject: Verizon Issues? East Coast US Seeing some packet loss via Cogent. www.internetpulse.net seems to be lighting up.
Seeing some packet loss via Cogent. www.internetpulse.net seems to be lighting up. I'm noticing it too. POP in Grand Rapids, circuit to Detroit. Packet loss, but no total loss of connectivity.
Interesting. I began to notice trouble reaching google about a week ago from 2 different east-cost Verizon FiOS lines -- one business and one residential -- but my problem appears to be congestion at the AS701 <> AS15169 border. I'm not noticing any loss across AS701. These problems began around 9am Eastern on Saturday Feb 19th -- see attached smokeping graph. You can see the diurnal pattern in the loss -- packet loss peaks between 9a-4p ET. Of course, I can get toipv6.google.com without any loss at all via my HE tunnel! In both cases, mtr shows ~50% loss beginning at google-gw.customer.alter.net (152.179.50.62), the first hop in AS15169. It's clear that I must be losing more ICMP than TCP packets given that google webpages come up fairly quickly, but youtube videos hang ever since this started. Anybody else seeing this? Thanks, -Chris -- Chris Tracy <ctracy@es.net> Energy Sciences Network (ESnet) Lawrence Berkeley National Laboratory
From our experience and smoke pings on Verizon's alternet, they ALWAYS have issues. Which is why we moved away from them. -Bret On Mar 1, 2011, at 9:47 AM, Chris Tracy wrote:
Seeing some packet loss via Cogent. www.internetpulse.net seems to be lighting up. I'm noticing it too. POP in Grand Rapids, circuit to Detroit. Packet loss, but no total loss of connectivity.
Interesting. I began to notice trouble reaching google about a week ago from 2 different east-cost Verizon FiOS lines -- one business and one residential -- but my problem appears to be congestion at the AS701 <> AS15169 border. I'm not noticing any loss across AS701.
These problems began around 9am Eastern on Saturday Feb 19th -- see attached smokeping graph. You can see the diurnal pattern in the loss -- packet loss peaks between 9a-4p ET. Of course, I can get toipv6.google.com without any loss at all via my HE tunnel!
In both cases, mtr shows ~50% loss beginning at google-gw.customer.alter.net (152.179.50.62), the first hop in AS15169. It's clear that I must be losing more ICMP than TCP packets given that google webpages come up fairly quickly, but youtube videos hang ever since this started.
Anybody else seeing this?
Thanks, -Chris
-- Chris Tracy <ctracy@es.net> Energy Sciences Network (ESnet) Lawrence Berkeley National Laboratory
<Screen shot 2011-03-01 at Mar 1 11.44.55 AM.jpg>
On Tue, 1 Mar 2011 11:47:39 -0500, Chris Tracy wrote
In both cases, mtr shows ~50% loss beginning at google- gw.customer.alter.net (152.179.50.62), the first hop in AS15169. It's clear that I must be losing more ICMP than TCP packets given that google webpages come up fairly quickly, but youtube videos hang ever since this started.
Anybody else seeing this?
I've been seeing ~50% packet loss to google from FiOS (WDC area) for a while now. Youtube completely unusable during the day for the most part, but that has been going on for months to tell you the truth. ~Randy
On 2/28/2011 9:53 PM, ML wrote:
Seeing some packet loss via Cogent.
www.internetpulse.net seems to be lighting up.
Looking at from Level3 via San Jose, NLayer via Chicago, Cogent via NY. Seems like the trouble starts after: 0.ge-5-0-0.CHI01-BB-RTR1.ALTER.NET Substitute CHI for NY, SJC etc.
I was just looking at an issue between 701 in Toronto. Seems to be resolved now-- at least the issue I was seeing. the bad traceroute, looked like .... 3 if-3-0-2.core3.TNK-Toronto.as6453.net (64.86.81.3) 0.988 ms 0.978 ms 1.578 ms 4 209.58.94.10 (209.58.94.10) 1.902 ms 71.416 ms 3.472 ms 5 if-3-1-0-0.tcore1.NJY-Newark.as6453.net (216.6.98.34) 22.286 ms 21.957 ms 29.472 ms 6 if-12-0.mcore3.NJY-Newark.as6453.net (66.198.70.14) 67.961 ms if-3-0-0.mcore3.NJY-Newark.as6453.net (216.6.57.121) 21.449 ms 20.956 ms 7 if-10-0.core3.NTO-NewYork.as6453.net (216.6.57.66) 21.975 ms 22.467 ms 21.977 ms 8 Vlan1297.icore1.NTO-NewYork.as6453.net (209.58.26.49) 20.977 ms * 30.520 ms 9 0.ae20.BR2.NYC4.ALTER.NET (204.255.168.173) 21.478 ms 21.458 ms 21.477 ms 10 * 11 * Now its working .... 3 if-3-0-2.core3.TNK-Toronto.as6453.net (64.86.81.3) 0.975 ms 4 209.58.94.10 (209.58.94.10) 1.975 ms 5 if-3-1-0-0.tcore1.NJY-Newark.as6453.net (216.6.98.34) 21.445 ms 6 if-12-0.mcore3.NJY-Newark.as6453.net (66.198.70.14) 54.472 ms 7 if-10-0.core3.NTO-NewYork.as6453.net (216.6.57.66) 112.426 ms 8 Vlan1297.icore1.NTO-NewYork.as6453.net (209.58.26.49) 22.964 ms 9 0.ae20.BR2.NYC4.ALTER.NET (204.255.168.173) 21.455 ms 10 0.ae2.XL4.NYC4.ALTER.NET (152.63.3.117) 21.466 ms 11 0.so-5-1-0.XT2.TOR2.ALTER.NET (152.63.128.121) 34.958 ms 12 0.POS7-1.GW2.TOR2.ALTER.NET (152.63.131.205) 33.958 ms When it was not working, packets would not get from my AS (11647) to the target in IP in AS701. But packets from 701 would get back to my AS. The AS path in both directions are 701-6453-11647 and 11647 6453 701... I saw a similar outage to VPNs I have in AS15290 which I see as 11647 6453 701 15290. However, I did not have time to check if it was the same behaviour with loss being in one direction. In both cases, IPs that follow 11647 174 701 and 701 174 11647 and 11647 174 7018 15290 and 15290 7018 174 11647 were not impacted. ---Mike On 2/28/2011 9:53 PM, ML wrote:
Seeing some packet loss via Cogent.
www.internetpulse.net seems to be lighting up.
-- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/
I'm having issues through Verizon too...I have a server colocated in Vancouver...could it be a Canadian thing with Verizon? 2 l100.phlapa-vfttp-60.verizon-gni.net (98.114.95.1) 8.910 ms 8.760 ms 7.026 ms 3 g3-0-2-860.phlapa-lcr-08.verizon-gni.net (130.81.139.120) 10.711 ms 8.466 ms 10.698 ms 4 * * * 5 so-13-2-0-0.res-bb-rtr2.verizon-gni.net (130.81.19.118) 14.937 ms 15.975 ms 15.148 ms 6 0.ae2.br2.iad8.alter.net (152.63.34.73) 14.346 ms 13.943 ms 14.833 ms 7 * * * 8 * * * I can ssh to the box from other networks, and here's a traceroute back to my Verzon FIOS IP...other Verizon customers (DSL, etc) report same problem: 2 static-209-17-142-114.gtcust.grouptelecom.net (209.17.142.114) 0.744 ms 0.642 ms 0.620 ms 3 static-66-38-255-9.gtcust.grouptelecom.net (66.38.255.9) 0.750 ms 0.657 ms 0.649 ms 4 GE3-0.PEERA-VANCBC.IP.GROUPTELECOM.NET (66.59.190.6) 0.730 ms 0.694 ms 0.693 ms 5 bx4-vancouver_G1-1-6.net.bell.ca (67.69.199.105) 0.847 ms 0.836 ms 0.828 ms 6 core4-vancouver_ge8-0-0.net.bell.ca (64.230.183.109) 4.637 ms core3-vancouver_ge8-0-0.net.bell.ca (64.230.183.105) 113.794 ms 17.328 ms 7 core2-seattle_pos4-0-0_core.net.bell.ca (64.230.144.97) 23.686 ms core1-seattle_pos6-0-0_core.net.bell.ca (64.230.144.89) 4.672 ms core2-seattle_pos4-0-0_core.net.bell.ca (64.230.144.97) 5.720 ms 8 bx2-seattle_POS10-0-0.net.bell.ca (64.230.186.22) 4.358 ms bx2-seattle_POS11-0-0.net.bell.ca (64.230.186.26) 4.396 ms bx2-seattle_POS10-0-0.net.bell.ca (64.230.186.22) 4.353 ms 9 Comcast-peering.net.bell.ca (67.69.246.198) 4.991 ms 4.795 ms 4.795 ms 10 pos-0-4-0-0-cr01.seattle.wa.ibone.comcast.net (68.86.86.137) 8.285 ms 5.258 ms 4.919 ms 11 pos-0-6-0-0-cr01.denver.co.ibone.comcast.net (68.86.87.49) 46.892 ms 46.901 ms 46.952 ms 12 pos-0-13-0-0-cr01.chicago.il.ibone.comcast.net (68.86.85.245) 57.243 ms 57.359 ms 57.333 ms 13 pos-2-13-0-0-cr01.newyork.ny.ibone.comcast.net (68.86.87.25) 85.080 ms 85.020 ms 85.069 ms 14 te-2-1-pe01.philadelphia.pa.ibone.comcast.net (68.86.84.194) 87.264 ms 86.372 ms 86.453 ms 15 75.149.230.250 (75.149.230.250) 86.548 ms 86.489 ms 86.439 ms On Mon, 28 Feb 2011, Mike Tancsa wrote:
I was just looking at an issue between 701 in Toronto. Seems to be resolved now-- at least the issue I was seeing.
the bad traceroute, looked like .... 3 if-3-0-2.core3.TNK-Toronto.as6453.net (64.86.81.3) 0.988 ms 0.978 ms 1.578 ms 4 209.58.94.10 (209.58.94.10) 1.902 ms 71.416 ms 3.472 ms 5 if-3-1-0-0.tcore1.NJY-Newark.as6453.net (216.6.98.34) 22.286 ms 21.957 ms 29.472 ms 6 if-12-0.mcore3.NJY-Newark.as6453.net (66.198.70.14) 67.961 ms if-3-0-0.mcore3.NJY-Newark.as6453.net (216.6.57.121) 21.449 ms 20.956 ms 7 if-10-0.core3.NTO-NewYork.as6453.net (216.6.57.66) 21.975 ms 22.467 ms 21.977 ms 8 Vlan1297.icore1.NTO-NewYork.as6453.net (209.58.26.49) 20.977 ms * 30.520 ms 9 0.ae20.BR2.NYC4.ALTER.NET (204.255.168.173) 21.478 ms 21.458 ms 21.477 ms 10 * 11 *
Now its working
.... 3 if-3-0-2.core3.TNK-Toronto.as6453.net (64.86.81.3) 0.975 ms 4 209.58.94.10 (209.58.94.10) 1.975 ms 5 if-3-1-0-0.tcore1.NJY-Newark.as6453.net (216.6.98.34) 21.445 ms 6 if-12-0.mcore3.NJY-Newark.as6453.net (66.198.70.14) 54.472 ms 7 if-10-0.core3.NTO-NewYork.as6453.net (216.6.57.66) 112.426 ms 8 Vlan1297.icore1.NTO-NewYork.as6453.net (209.58.26.49) 22.964 ms 9 0.ae20.BR2.NYC4.ALTER.NET (204.255.168.173) 21.455 ms 10 0.ae2.XL4.NYC4.ALTER.NET (152.63.3.117) 21.466 ms 11 0.so-5-1-0.XT2.TOR2.ALTER.NET (152.63.128.121) 34.958 ms 12 0.POS7-1.GW2.TOR2.ALTER.NET (152.63.131.205) 33.958 ms
When it was not working, packets would not get from my AS (11647) to the target in IP in AS701. But packets from 701 would get back to my AS. The AS path in both directions are 701-6453-11647 and 11647 6453 701... I saw a similar outage to VPNs I have in AS15290 which I see as 11647 6453 701 15290. However, I did not have time to check if it was the same behaviour with loss being in one direction. In both cases, IPs that follow 11647 174 701 and 701 174 11647 and 11647 174 7018 15290 and 15290 7018 174 11647 were not impacted.
---Mike
On 2/28/2011 9:53 PM, ML wrote:
Seeing some packet loss via Cogent.
www.internetpulse.net seems to be lighting up.
-- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/
James Smallacombe PlantageNet, Inc. CEO and Janitor up@3.am http://3.am =========================================================================
Please disregard my reply...I used pine for the first time in months and although this was tagged as a New message, I didn't see the date was from months ago. However, I AM seeing problems right now as described below...anybody aware of any Verizon issues? On Wed, 3 Aug 2011, James Smallacombe wrote:
I'm having issues through Verizon too...I have a server colocated in Vancouver...could it be a Canadian thing with Verizon?
2 l100.phlapa-vfttp-60.verizon-gni.net (98.114.95.1) 8.910 ms 8.760 ms 7.026 ms 3 g3-0-2-860.phlapa-lcr-08.verizon-gni.net (130.81.139.120) 10.711 ms 8.466 ms 10.698 ms 4 * * * 5 so-13-2-0-0.res-bb-rtr2.verizon-gni.net (130.81.19.118) 14.937 ms 15.975 ms 15.148 ms 6 0.ae2.br2.iad8.alter.net (152.63.34.73) 14.346 ms 13.943 ms 14.833 ms 7 * * * 8 * * *
I can ssh to the box from other networks, and here's a traceroute back to my Verzon FIOS IP...other Verizon customers (DSL, etc) report same problem:
2 static-209-17-142-114.gtcust.grouptelecom.net (209.17.142.114) 0.744 ms 0.642 ms 0.620 ms 3 static-66-38-255-9.gtcust.grouptelecom.net (66.38.255.9) 0.750 ms 0.657 ms 0.649 ms 4 GE3-0.PEERA-VANCBC.IP.GROUPTELECOM.NET (66.59.190.6) 0.730 ms 0.694 ms 0.693 ms 5 bx4-vancouver_G1-1-6.net.bell.ca (67.69.199.105) 0.847 ms 0.836 ms 0.828 ms 6 core4-vancouver_ge8-0-0.net.bell.ca (64.230.183.109) 4.637 ms core3-vancouver_ge8-0-0.net.bell.ca (64.230.183.105) 113.794 ms 17.328 ms 7 core2-seattle_pos4-0-0_core.net.bell.ca (64.230.144.97) 23.686 ms core1-seattle_pos6-0-0_core.net.bell.ca (64.230.144.89) 4.672 ms core2-seattle_pos4-0-0_core.net.bell.ca (64.230.144.97) 5.720 ms 8 bx2-seattle_POS10-0-0.net.bell.ca (64.230.186.22) 4.358 ms bx2-seattle_POS11-0-0.net.bell.ca (64.230.186.26) 4.396 ms bx2-seattle_POS10-0-0.net.bell.ca (64.230.186.22) 4.353 ms 9 Comcast-peering.net.bell.ca (67.69.246.198) 4.991 ms 4.795 ms 4.795 ms 10 pos-0-4-0-0-cr01.seattle.wa.ibone.comcast.net (68.86.86.137) 8.285 ms 5.258 ms 4.919 ms 11 pos-0-6-0-0-cr01.denver.co.ibone.comcast.net (68.86.87.49) 46.892 ms 46.901 ms 46.952 ms 12 pos-0-13-0-0-cr01.chicago.il.ibone.comcast.net (68.86.85.245) 57.243 ms 57.359 ms 57.333 ms 13 pos-2-13-0-0-cr01.newyork.ny.ibone.comcast.net (68.86.87.25) 85.080 ms 85.020 ms 85.069 ms 14 te-2-1-pe01.philadelphia.pa.ibone.comcast.net (68.86.84.194) 87.264 ms 86.372 ms 86.453 ms 15 75.149.230.250 (75.149.230.250) 86.548 ms 86.489 ms 86.439 ms
On Mon, 28 Feb 2011, Mike Tancsa wrote:
I was just looking at an issue between 701 in Toronto. Seems to be resolved now-- at least the issue I was seeing.
the bad traceroute, looked like .... 3 if-3-0-2.core3.TNK-Toronto.as6453.net (64.86.81.3) 0.988 ms 0.978 ms 1.578 ms 4 209.58.94.10 (209.58.94.10) 1.902 ms 71.416 ms 3.472 ms 5 if-3-1-0-0.tcore1.NJY-Newark.as6453.net (216.6.98.34) 22.286 ms 21.957 ms 29.472 ms 6 if-12-0.mcore3.NJY-Newark.as6453.net (66.198.70.14) 67.961 ms if-3-0-0.mcore3.NJY-Newark.as6453.net (216.6.57.121) 21.449 ms 20.956 ms 7 if-10-0.core3.NTO-NewYork.as6453.net (216.6.57.66) 21.975 ms 22.467 ms 21.977 ms 8 Vlan1297.icore1.NTO-NewYork.as6453.net (209.58.26.49) 20.977 ms * 30.520 ms 9 0.ae20.BR2.NYC4.ALTER.NET (204.255.168.173) 21.478 ms 21.458 ms 21.477 ms 10 * 11 *
Now its working
.... 3 if-3-0-2.core3.TNK-Toronto.as6453.net (64.86.81.3) 0.975 ms 4 209.58.94.10 (209.58.94.10) 1.975 ms 5 if-3-1-0-0.tcore1.NJY-Newark.as6453.net (216.6.98.34) 21.445 ms 6 if-12-0.mcore3.NJY-Newark.as6453.net (66.198.70.14) 54.472 ms 7 if-10-0.core3.NTO-NewYork.as6453.net (216.6.57.66) 112.426 ms 8 Vlan1297.icore1.NTO-NewYork.as6453.net (209.58.26.49) 22.964 ms 9 0.ae20.BR2.NYC4.ALTER.NET (204.255.168.173) 21.455 ms 10 0.ae2.XL4.NYC4.ALTER.NET (152.63.3.117) 21.466 ms 11 0.so-5-1-0.XT2.TOR2.ALTER.NET (152.63.128.121) 34.958 ms 12 0.POS7-1.GW2.TOR2.ALTER.NET (152.63.131.205) 33.958 ms
When it was not working, packets would not get from my AS (11647) to the target in IP in AS701. But packets from 701 would get back to my AS. The AS path in both directions are 701-6453-11647 and 11647 6453 701... I saw a similar outage to VPNs I have in AS15290 which I see as 11647 6453 701 15290. However, I did not have time to check if it was the same behaviour with loss being in one direction. In both cases, IPs that follow 11647 174 701 and 701 174 11647 and 11647 174 7018 15290 and 15290 7018 174 11647 were not impacted.
---Mike
On 2/28/2011 9:53 PM, ML wrote:
Seeing some packet loss via Cogent.
www.internetpulse.net seems to be lighting up.
-- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/
James Smallacombe PlantageNet, Inc. CEO and Janitor up@3.am http://3.am =========================================================================
James Smallacombe PlantageNet, Inc. CEO and Janitor up@3.am http://3.am =========================================================================
On 8/3/2011 3:31 PM, James Smallacombe wrote:
However, I AM seeing problems right now as described below...anybody aware of any Verizon issues?
I was told by TATA one of their core routers in NY is not reachable. So perhaps some inadvertent black hole routing between them / by them. ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/
On 2011-08-03, at 3:50 PM, Mike Tancsa wrote:
On 8/3/2011 3:31 PM, James Smallacombe wrote:
However, I AM seeing problems right now as described below...anybody aware of any Verizon issues?
I was told by TATA one of their core routers in NY is not reachable. So perhaps some inadvertent black hole routing between them / by them.
Do you have a ticket number, Mike? Seems like they are still blackholing traffic.
On 8/3/2011 5:32 PM, Jason Lixfeld wrote:
On 2011-08-03, at 3:50 PM, Mike Tancsa wrote:
On 8/3/2011 3:31 PM, James Smallacombe wrote:
However, I AM seeing problems right now as described below...anybody aware of any Verizon issues?
I was told by TATA one of their core routers in NY is not reachable. So perhaps some inadvertent black hole routing between them / by them.
Do you have a ticket number, Mike? Seems like they are still blackholing traffic.
I will send the ticket offlist. The last update I got from them an hr ago ---------------- Dear Customer, Our TAC team investigated and found still our router in New York is facing issue. We have to emergency upgraded the ios and clear the issue. We will update after activity is completed. ---------------- ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/
Hi, Yeah, I was just seeing some issues through TATA (AS6453) with routes being blackholed in Newark, or at least that was where the packets stopped. I had to shut my peer with them and I just finished opening a trouble ticket. A traceroute to 209.167.35.0/24 from a source addr in 205.211.164.0/24 byte packets 1 teleglobe-vl38-tor (205.211.165.121) 0.259 ms 0.465 ms 0.488 ms 2 if-2-3-513.core4.TNK-Toronto.as6453.net (209.58.16.21) 0.987 ms 0.981 ms 0.565 ms 3 if-14-0-0.mcore3.TTT-Scarborough.as6453.net (63.243.172.2) 53.390 ms 2.973 ms 2.991 ms 4 if-3-1-0-0.tcore1.NJY-Newark.as6453.net (216.6.98.34) 20.481 ms 62.938 ms 20.967 ms 5 if-2-2.tcore2.NJY-Newark.as6453.net (66.198.70.2) 20.986 ms 21.059 ms 20.871 ms 6 * * * Not sure if it was after them or coming back to them. Same source addr through AS174 is fine ---Mike On 8/3/2011 3:27 PM, James Smallacombe wrote:
I'm having issues through Verizon too...I have a server colocated in Vancouver...could it be a Canadian thing with Verizon?
2 l100.phlapa-vfttp-60.verizon-gni.net (98.114.95.1) 8.910 ms 8.760 ms 7.026 ms 3 g3-0-2-860.phlapa-lcr-08.verizon-gni.net (130.81.139.120) 10.711 ms 8.466 ms 10.698 ms 4 * * * 5 so-13-2-0-0.res-bb-rtr2.verizon-gni.net (130.81.19.118) 14.937 ms 15.975 ms 15.148 ms 6 0.ae2.br2.iad8.alter.net (152.63.34.73) 14.346 ms 13.943 ms 14.833 ms 7 * * * 8 * * *
I can ssh to the box from other networks, and here's a traceroute back to my Verzon FIOS IP...other Verizon customers (DSL, etc) report same problem:
2 static-209-17-142-114.gtcust.grouptelecom.net (209.17.142.114) 0.744 ms 0.642 ms 0.620 ms 3 static-66-38-255-9.gtcust.grouptelecom.net (66.38.255.9) 0.750 ms 0.657 ms 0.649 ms 4 GE3-0.PEERA-VANCBC.IP.GROUPTELECOM.NET (66.59.190.6) 0.730 ms 0.694 ms 0.693 ms 5 bx4-vancouver_G1-1-6.net.bell.ca (67.69.199.105) 0.847 ms 0.836 ms 0.828 ms 6 core4-vancouver_ge8-0-0.net.bell.ca (64.230.183.109) 4.637 ms core3-vancouver_ge8-0-0.net.bell.ca (64.230.183.105) 113.794 ms 17.328 ms 7 core2-seattle_pos4-0-0_core.net.bell.ca (64.230.144.97) 23.686 ms core1-seattle_pos6-0-0_core.net.bell.ca (64.230.144.89) 4.672 ms core2-seattle_pos4-0-0_core.net.bell.ca (64.230.144.97) 5.720 ms 8 bx2-seattle_POS10-0-0.net.bell.ca (64.230.186.22) 4.358 ms bx2-seattle_POS11-0-0.net.bell.ca (64.230.186.26) 4.396 ms bx2-seattle_POS10-0-0.net.bell.ca (64.230.186.22) 4.353 ms 9 Comcast-peering.net.bell.ca (67.69.246.198) 4.991 ms 4.795 ms 4.795 ms 10 pos-0-4-0-0-cr01.seattle.wa.ibone.comcast.net (68.86.86.137) 8.285 ms 5.258 ms 4.919 ms 11 pos-0-6-0-0-cr01.denver.co.ibone.comcast.net (68.86.87.49) 46.892 ms 46.901 ms 46.952 ms 12 pos-0-13-0-0-cr01.chicago.il.ibone.comcast.net (68.86.85.245) 57.243 ms 57.359 ms 57.333 ms 13 pos-2-13-0-0-cr01.newyork.ny.ibone.comcast.net (68.86.87.25) 85.080 ms 85.020 ms 85.069 ms 14 te-2-1-pe01.philadelphia.pa.ibone.comcast.net (68.86.84.194) 87.264 ms 86.372 ms 86.453 ms 15 75.149.230.250 (75.149.230.250) 86.548 ms 86.489 ms 86.439 ms
On Mon, 28 Feb 2011, Mike Tancsa wrote:
I was just looking at an issue between 701 in Toronto. Seems to be resolved now-- at least the issue I was seeing.
the bad traceroute, looked like .... 3 if-3-0-2.core3.TNK-Toronto.as6453.net (64.86.81.3) 0.988 ms 0.978 ms 1.578 ms 4 209.58.94.10 (209.58.94.10) 1.902 ms 71.416 ms 3.472 ms 5 if-3-1-0-0.tcore1.NJY-Newark.as6453.net (216.6.98.34) 22.286 ms 21.957 ms 29.472 ms 6 if-12-0.mcore3.NJY-Newark.as6453.net (66.198.70.14) 67.961 ms if-3-0-0.mcore3.NJY-Newark.as6453.net (216.6.57.121) 21.449 ms 20.956 ms 7 if-10-0.core3.NTO-NewYork.as6453.net (216.6.57.66) 21.975 ms 22.467 ms 21.977 ms 8 Vlan1297.icore1.NTO-NewYork.as6453.net (209.58.26.49) 20.977 ms * 30.520 ms 9 0.ae20.BR2.NYC4.ALTER.NET (204.255.168.173) 21.478 ms 21.458 ms 21.477 ms 10 * 11 *
Now its working
.... 3 if-3-0-2.core3.TNK-Toronto.as6453.net (64.86.81.3) 0.975 ms 4 209.58.94.10 (209.58.94.10) 1.975 ms 5 if-3-1-0-0.tcore1.NJY-Newark.as6453.net (216.6.98.34) 21.445 ms 6 if-12-0.mcore3.NJY-Newark.as6453.net (66.198.70.14) 54.472 ms 7 if-10-0.core3.NTO-NewYork.as6453.net (216.6.57.66) 112.426 ms 8 Vlan1297.icore1.NTO-NewYork.as6453.net (209.58.26.49) 22.964 ms 9 0.ae20.BR2.NYC4.ALTER.NET (204.255.168.173) 21.455 ms 10 0.ae2.XL4.NYC4.ALTER.NET (152.63.3.117) 21.466 ms 11 0.so-5-1-0.XT2.TOR2.ALTER.NET (152.63.128.121) 34.958 ms 12 0.POS7-1.GW2.TOR2.ALTER.NET (152.63.131.205) 33.958 ms
When it was not working, packets would not get from my AS (11647) to the target in IP in AS701. But packets from 701 would get back to my AS. The AS path in both directions are 701-6453-11647 and 11647 6453 701... I saw a similar outage to VPNs I have in AS15290 which I see as 11647 6453 701 15290. However, I did not have time to check if it was the same behaviour with loss being in one direction. In both cases, IPs that follow 11647 174 701 and 701 174 11647 and 11647 174 7018 15290 and 15290 7018 174 11647 were not impacted.
---Mike
On 2/28/2011 9:53 PM, ML wrote:
Seeing some packet loss via Cogent.
www.internetpulse.net seems to be lighting up.
-- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/
James Smallacombe PlantageNet, Inc. CEO and Janitor up@3.am http://3.am =========================================================================
-- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/
participants (8)
-
Bret Palsson -
Chris Tracy -
James Smallacombe -
Jason Lixfeld -
Matt Taber -
Mike Tancsa -
ML -
Randy McAnally