Verizon/UU.net/Alternet Routing issue
At about 4:24pm EDT, I lost connectivity from Verizon to destinations in New York, Seattle and others. Came back up (4:46pm) while composing this email. Anyone else notice? Major problem or minor routing issue? Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. localrouter 67.6% 395 0.6 1.6 0.5 18.8 2.3 2. 10.1.41.15 0.0% 395 5.7 5.1 1.8 306.0 17.4 3. P4-2.LCR-02.WASHDC.verizon-g 0.0% 395 7.4 2.7 1.2 19.0 2.5 4. 130.81.29.218 0.0% 395 6.0 3.8 1.8 40.9 4.2 5. 152.63.39.177 0.0% 395 8.6 6.8 3.9 71.3 4.4 152.63.36.213 6. 152.63.69.113 71.6% 395 120.7 44.0 31.2 186.7 30.3 7. POS7-0-0.GW4.IND6.ALTER.NET 30.7% 395 1179. 133.3 121.3 1179. 79.5 8. 152.63.67.250 93.9% 395 121.5 125.4 121.0 186.2 13.0 9. POS6-0-0.GW4.IND6.ALTER.NET 53.0% 395 318.9 217.7 206.8 722.0 43.3 10. 152.63.67.250 96.2% 395 211.1 211.1 209.0 215.7 1.8 11. POS6-0-0.GW4.IND6.ALTER.NET 67.0% 395 422.1 305.9 294.9 692.1 37.5 12. 152.63.67.250 97.5% 394 295.1 298.0 295.1 303.6 2.5 13. POS6-0-0.GW4.IND6.ALTER.NET 73.5% 394 523.9 391.5 382.1 523.9 17.7 14. 152.63.67.250 98.7% 392 388.5 386.6 381.9 389.5 3.1 15. POS6-0-0.GW4.IND6.ALTER.NET 82.6% 392 632.9 481.2 468.6 632.9 22.2 16. 152.63.67.250 99.2% 388 472.7 472.2 470.2 473.6 1.8 17. POS6-0-0.GW4.IND6.ALTER.NET 85.8% 388 737.0 573.3 559.4 737.0 27.8 18. 152.63.67.250 99.2% 387 560.5 562.0 560.5 565.1 2.7 19. POS6-0-0.GW4.IND6.ALTER.NET 89.6% 387 839.0 664.8 644.9 839.0 38.6 20. 152.63.67.250 99.2% 387 649.3 649.6 649.3 649.9 0.3 21. POS6-0-0.GW4.IND6.ALTER.NET 94.8% 383 946.4 763.8 734.6 946.4 48.5 22. 152.63.67.250 99.7% 376 735.5 735.5 735.5 735.5 0.0 23. POS6-0-0.GW4.IND6.ALTER.NET 92.5% 376 895.4 842.2 819.1 909.0 26.8 24. ??? 25. POS6-0-0.GW4.IND6.ALTER.NET 96.7% 365 1153. 955.9 908.9 1153. 78.7 26. ??? 27. POS6-0-0.GW4.IND6.ALTER.NET 96.6% 328 1261. 1057. 998.8 1261. 86.8 28. 152.63.67.250 99.6% 245 999.3 999.3 999.3 999.3 0.0 29. POS6-0-0.GW4.IND6.ALTER.NET 98.8% 245 1189. 1123. 1086. 1189. 57.5 30. ??? Beckman --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ ---------------------------------------------------------------------------
Yes, all our traffic was dying over at UUNet, yet it was still being announced it. It just came up before I could send anything to outages. -----Original Message----- From: Peter Beckman [mailto:beckman@angryox.com] Sent: Wednesday, November 12, 2008 13:48 To: nanog@nanog.org Subject: Verizon/UU.net/Alternet Routing issue At about 4:24pm EDT, I lost connectivity from Verizon to destinations in New York, Seattle and others. Came back up (4:46pm) while composing this email. Anyone else notice? Major problem or minor routing issue? Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. localrouter 67.6% 395 0.6 1.6 0.5 18.8 2.3 2. 10.1.41.15 0.0% 395 5.7 5.1 1.8 306.0 17.4 3. P4-2.LCR-02.WASHDC.verizon-g 0.0% 395 7.4 2.7 1.2 19.0 2.5 4. 130.81.29.218 0.0% 395 6.0 3.8 1.8 40.9 4.2 5. 152.63.39.177 0.0% 395 8.6 6.8 3.9 71.3 4.4 152.63.36.213 6. 152.63.69.113 71.6% 395 120.7 44.0 31.2 186.7 30.3 7. POS7-0-0.GW4.IND6.ALTER.NET 30.7% 395 1179. 133.3 121.3 1179. 79.5 8. 152.63.67.250 93.9% 395 121.5 125.4 121.0 186.2 13.0 9. POS6-0-0.GW4.IND6.ALTER.NET 53.0% 395 318.9 217.7 206.8 722.0 43.3 10. 152.63.67.250 96.2% 395 211.1 211.1 209.0 215.7 1.8 11. POS6-0-0.GW4.IND6.ALTER.NET 67.0% 395 422.1 305.9 294.9 692.1 37.5 12. 152.63.67.250 97.5% 394 295.1 298.0 295.1 303.6 2.5 13. POS6-0-0.GW4.IND6.ALTER.NET 73.5% 394 523.9 391.5 382.1 523.9 17.7 14. 152.63.67.250 98.7% 392 388.5 386.6 381.9 389.5 3.1 15. POS6-0-0.GW4.IND6.ALTER.NET 82.6% 392 632.9 481.2 468.6 632.9 22.2 16. 152.63.67.250 99.2% 388 472.7 472.2 470.2 473.6 1.8 17. POS6-0-0.GW4.IND6.ALTER.NET 85.8% 388 737.0 573.3 559.4 737.0 27.8 18. 152.63.67.250 99.2% 387 560.5 562.0 560.5 565.1 2.7 19. POS6-0-0.GW4.IND6.ALTER.NET 89.6% 387 839.0 664.8 644.9 839.0 38.6 20. 152.63.67.250 99.2% 387 649.3 649.6 649.3 649.9 0.3 21. POS6-0-0.GW4.IND6.ALTER.NET 94.8% 383 946.4 763.8 734.6 946.4 48.5 22. 152.63.67.250 99.7% 376 735.5 735.5 735.5 735.5 0.0 23. POS6-0-0.GW4.IND6.ALTER.NET 92.5% 376 895.4 842.2 819.1 909.0 26.8 24. ??? 25. POS6-0-0.GW4.IND6.ALTER.NET 96.7% 365 1153. 955.9 908.9 1153. 78.7 26. ??? 27. POS6-0-0.GW4.IND6.ALTER.NET 96.6% 328 1261. 1057. 998.8 1261. 86.8 28. 152.63.67.250 99.6% 245 999.3 999.3 999.3 999.3 0.0 29. POS6-0-0.GW4.IND6.ALTER.NET 98.8% 245 1189. 1123. 1086. 1189. 57.5 30. ??? Beckman ------------------------------------------------------------------------ --- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ ------------------------------------------------------------------------ ---
Same here... I'm a Verizon FiOS customer... My Vonage line went dead in the middle of a call... On Wed, 12 Nov 2008, Darryl Dunkin wrote: ddunki>Yes, all our traffic was dying over at UUNet, yet it was still being ddunki>announced it. It just came up before I could send anything to outages. ddunki> ddunki>-----Original Message----- ddunki>From: Peter Beckman [mailto:beckman@angryox.com] ddunki>Sent: Wednesday, November 12, 2008 13:48 ddunki>To: nanog@nanog.org ddunki>Subject: Verizon/UU.net/Alternet Routing issue ddunki> ddunki>At about 4:24pm EDT, I lost connectivity from Verizon to destinations in ddunki>New York, Seattle and others. Came back up (4:46pm) while composing ddunki>this ddunki>email. Anyone else notice? Major problem or minor routing issue? ddunki> ddunki> Packets Pings ddunki> Host Loss% Snt Last Avg Best Wrst ddunki>StDev ddunki> 1. localrouter 67.6% 395 0.6 1.6 0.5 18.8 ddunki>2.3 ddunki> 2. 10.1.41.15 0.0% 395 5.7 5.1 1.8 306.0 ddunki>17.4 ddunki> 3. P4-2.LCR-02.WASHDC.verizon-g 0.0% 395 7.4 2.7 1.2 19.0 ddunki>2.5 ddunki> 4. 130.81.29.218 0.0% 395 6.0 3.8 1.8 40.9 ddunki>4.2 ddunki> 5. 152.63.39.177 0.0% 395 8.6 6.8 3.9 71.3 ddunki>4.4 ddunki> 152.63.36.213 ddunki> 6. 152.63.69.113 71.6% 395 120.7 44.0 31.2 186.7 ddunki>30.3 ddunki> 7. POS7-0-0.GW4.IND6.ALTER.NET 30.7% 395 1179. 133.3 121.3 1179. ddunki>79.5 ddunki> 8. 152.63.67.250 93.9% 395 121.5 125.4 121.0 186.2 ddunki>13.0 ddunki> 9. POS6-0-0.GW4.IND6.ALTER.NET 53.0% 395 318.9 217.7 206.8 722.0 ddunki>43.3 ddunki>10. 152.63.67.250 96.2% 395 211.1 211.1 209.0 215.7 ddunki>1.8 ddunki>11. POS6-0-0.GW4.IND6.ALTER.NET 67.0% 395 422.1 305.9 294.9 692.1 ddunki>37.5 ddunki>12. 152.63.67.250 97.5% 394 295.1 298.0 295.1 303.6 ddunki>2.5 ddunki>13. POS6-0-0.GW4.IND6.ALTER.NET 73.5% 394 523.9 391.5 382.1 523.9 ddunki>17.7 ddunki>14. 152.63.67.250 98.7% 392 388.5 386.6 381.9 389.5 ddunki>3.1 ddunki>15. POS6-0-0.GW4.IND6.ALTER.NET 82.6% 392 632.9 481.2 468.6 632.9 ddunki>22.2 ddunki>16. 152.63.67.250 99.2% 388 472.7 472.2 470.2 473.6 ddunki>1.8 ddunki>17. POS6-0-0.GW4.IND6.ALTER.NET 85.8% 388 737.0 573.3 559.4 737.0 ddunki>27.8 ddunki>18. 152.63.67.250 99.2% 387 560.5 562.0 560.5 565.1 ddunki>2.7 ddunki>19. POS6-0-0.GW4.IND6.ALTER.NET 89.6% 387 839.0 664.8 644.9 839.0 ddunki>38.6 ddunki>20. 152.63.67.250 99.2% 387 649.3 649.6 649.3 649.9 ddunki>0.3 ddunki>21. POS6-0-0.GW4.IND6.ALTER.NET 94.8% 383 946.4 763.8 734.6 946.4 ddunki>48.5 ddunki>22. 152.63.67.250 99.7% 376 735.5 735.5 735.5 735.5 ddunki>0.0 ddunki>23. POS6-0-0.GW4.IND6.ALTER.NET 92.5% 376 895.4 842.2 819.1 909.0 ddunki>26.8 ddunki>24. ??? ddunki>25. POS6-0-0.GW4.IND6.ALTER.NET 96.7% 365 1153. 955.9 908.9 1153. ddunki>78.7 ddunki>26. ??? ddunki>27. POS6-0-0.GW4.IND6.ALTER.NET 96.6% 328 1261. 1057. 998.8 1261. ddunki>86.8 ddunki>28. 152.63.67.250 99.6% 245 999.3 999.3 999.3 999.3 ddunki>0.0 ddunki>29. POS6-0-0.GW4.IND6.ALTER.NET 98.8% 245 1189. 1123. 1086. 1189. ddunki>57.5 ddunki>30. ??? ddunki> ddunki>Beckman ddunki>------------------------------------------------------------------------ ddunki>--- ddunki>Peter Beckman Internet ddunki>Guy ddunki>beckman@angryox.com ddunki>http://www.angryox.com/ ddunki>------------------------------------------------------------------------ ddunki>--- ddunki> ddunki> Best, Alan
Yes, we saw the same thing and all seems to be better now. Was on hold and hung up. Mike Walter, MCP Systems Administrator 3z.net a PCD Company http://www.3z.net "When Success is the Only Solution think 3z.net" -----Original Message----- From: Peter Beckman [mailto:beckman@angryox.com] Sent: Wednesday, November 12, 2008 4:48 PM To: nanog@nanog.org Subject: Verizon/UU.net/Alternet Routing issue At about 4:24pm EDT, I lost connectivity from Verizon to destinations in New York, Seattle and others. Came back up (4:46pm) while composing this email. Anyone else notice? Major problem or minor routing issue? Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. localrouter 67.6% 395 0.6 1.6 0.5 18.8 2.3 2. 10.1.41.15 0.0% 395 5.7 5.1 1.8 306.0 17.4 3. P4-2.LCR-02.WASHDC.verizon-g 0.0% 395 7.4 2.7 1.2 19.0 2.5 4. 130.81.29.218 0.0% 395 6.0 3.8 1.8 40.9 4.2 5. 152.63.39.177 0.0% 395 8.6 6.8 3.9 71.3 4.4 152.63.36.213 6. 152.63.69.113 71.6% 395 120.7 44.0 31.2 186.7 30.3 7. POS7-0-0.GW4.IND6.ALTER.NET 30.7% 395 1179. 133.3 121.3 1179. 79.5 8. 152.63.67.250 93.9% 395 121.5 125.4 121.0 186.2 13.0 9. POS6-0-0.GW4.IND6.ALTER.NET 53.0% 395 318.9 217.7 206.8 722.0 43.3 10. 152.63.67.250 96.2% 395 211.1 211.1 209.0 215.7 1.8 11. POS6-0-0.GW4.IND6.ALTER.NET 67.0% 395 422.1 305.9 294.9 692.1 37.5 12. 152.63.67.250 97.5% 394 295.1 298.0 295.1 303.6 2.5 13. POS6-0-0.GW4.IND6.ALTER.NET 73.5% 394 523.9 391.5 382.1 523.9 17.7 14. 152.63.67.250 98.7% 392 388.5 386.6 381.9 389.5 3.1 15. POS6-0-0.GW4.IND6.ALTER.NET 82.6% 392 632.9 481.2 468.6 632.9 22.2 16. 152.63.67.250 99.2% 388 472.7 472.2 470.2 473.6 1.8 17. POS6-0-0.GW4.IND6.ALTER.NET 85.8% 388 737.0 573.3 559.4 737.0 27.8 18. 152.63.67.250 99.2% 387 560.5 562.0 560.5 565.1 2.7 19. POS6-0-0.GW4.IND6.ALTER.NET 89.6% 387 839.0 664.8 644.9 839.0 38.6 20. 152.63.67.250 99.2% 387 649.3 649.6 649.3 649.9 0.3 21. POS6-0-0.GW4.IND6.ALTER.NET 94.8% 383 946.4 763.8 734.6 946.4 48.5 22. 152.63.67.250 99.7% 376 735.5 735.5 735.5 735.5 0.0 23. POS6-0-0.GW4.IND6.ALTER.NET 92.5% 376 895.4 842.2 819.1 909.0 26.8 24. ??? 25. POS6-0-0.GW4.IND6.ALTER.NET 96.7% 365 1153. 955.9 908.9 1153. 78.7 26. ??? 27. POS6-0-0.GW4.IND6.ALTER.NET 96.6% 328 1261. 1057. 998.8 1261. 86.8 28. 152.63.67.250 99.6% 245 999.3 999.3 999.3 999.3 0.0 29. POS6-0-0.GW4.IND6.ALTER.NET 98.8% 245 1189. 1123. 1086. 1189. 57.5 30. ??? Beckman ------------------------------------------------------------------------ --- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ ------------------------------------------------------------------------ ---
Peter Beckman wrote:
At about 4:24pm EDT, I lost connectivity from Verizon to destinations in New York, Seattle and others. Came back up (4:46pm) while composing this email. Anyone else notice? Major problem or minor routing issue?
Beckman
I'm still on hold with Verizon -- has anyone heard anything official from them? I'm hesitant to turn that circuit back up... Andrew Cruse
Confirmed here as well; Saw loss on DS3s between 424 and 440 EST. BGP survived but routing didnt .. No RCA yet from VZN (on hold). On Wed, Nov 12, 2008 at 3:47 PM, Peter Beckman <beckman@angryox.com> wrote:
At about 4:24pm EDT, I lost connectivity from Verizon to destinations in New York, Seattle and others. Came back up (4:46pm) while composing this email. Anyone else notice? Major problem or minor routing issue?
Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. localrouter 67.6% 395 0.6 1.6 0.5 18.8 2.3 2. 10.1.41.15 0.0% 395 5.7 5.1 1.8 306.0 17.4 3. P4-2.LCR-02.WASHDC.verizon-g 0.0% 395 7.4 2.7 1.2 19.0 2.5 4. 130.81.29.218 0.0% 395 6.0 3.8 1.8 40.9 4.2 5. 152.63.39.177 0.0% 395 8.6 6.8 3.9 71.3 4.4 152.63.36.213 6. 152.63.69.113 71.6% 395 120.7 44.0 31.2 186.7 30.3 7. POS7-0-0.GW4.IND6.ALTER.NET 30.7% 395 1179. 133.3 121.3 1179. 79.5 8. 152.63.67.250 93.9% 395 121.5 125.4 121.0 186.2 13.0 9. POS6-0-0.GW4.IND6.ALTER.NET 53.0% 395 318.9 217.7 206.8 722.0 43.3 10. 152.63.67.250 96.2% 395 211.1 211.1 209.0 215.7 1.8 11. POS6-0-0.GW4.IND6.ALTER.NET 67.0% 395 422.1 305.9 294.9 692.1 37.5 12. 152.63.67.250 97.5% 394 295.1 298.0 295.1 303.6 2.5 13. POS6-0-0.GW4.IND6.ALTER.NET 73.5% 394 523.9 391.5 382.1 523.9 17.7 14. 152.63.67.250 98.7% 392 388.5 386.6 381.9 389.5 3.1 15. POS6-0-0.GW4.IND6.ALTER.NET 82.6% 392 632.9 481.2 468.6 632.9 22.2 16. 152.63.67.250 99.2% 388 472.7 472.2 470.2 473.6 1.8 17. POS6-0-0.GW4.IND6.ALTER.NET 85.8% 388 737.0 573.3 559.4 737.0 27.8 18. 152.63.67.250 99.2% 387 560.5 562.0 560.5 565.1 2.7 19. POS6-0-0.GW4.IND6.ALTER.NET 89.6% 387 839.0 664.8 644.9 839.0 38.6 20. 152.63.67.250 99.2% 387 649.3 649.6 649.3 649.9 0.3 21. POS6-0-0.GW4.IND6.ALTER.NET 94.8% 383 946.4 763.8 734.6 946.4 48.5 22. 152.63.67.250 99.7% 376 735.5 735.5 735.5 735.5 0.0 23. POS6-0-0.GW4.IND6.ALTER.NET 92.5% 376 895.4 842.2 819.1 909.0 26.8 24. ??? 25. POS6-0-0.GW4.IND6.ALTER.NET 96.7% 365 1153. 955.9 908.9 1153. 78.7 26. ??? 27. POS6-0-0.GW4.IND6.ALTER.NET 96.6% 328 1261. 1057. 998.8 1261. 86.8 28. 152.63.67.250 99.6% 245 999.3 999.3 999.3 999.3 0.0 29. POS6-0-0.GW4.IND6.ALTER.NET 98.8% 245 1189. 1123. 1086. 1189. 57.5 30. ???
Beckman --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ ---------------------------------------------------------------------------
-- ..!google!arpa.com!j
Same here. Saw the issue from Los Angeles, and from New York. Traces were dropping a few hops into the Verizon cloud. BGP stayed up, but routing went nowhere. Paul ________________________________ From: jamie rishaw [mailto:j@arpa.com] Sent: Wed 11/12/2008 3:14 PM To: Peter Beckman Cc: nanog@nanog.org Subject: Re: Verizon/UU.net/Alternet Routing issue Confirmed here as well; Saw loss on DS3s between 424 and 440 EST. BGP survived but routing didnt .. No RCA yet from VZN (on hold). On Wed, Nov 12, 2008 at 3:47 PM, Peter Beckman <beckman@angryox.com> wrote:
At about 4:24pm EDT, I lost connectivity from Verizon to destinations in New York, Seattle and others. Came back up (4:46pm) while composing this email. Anyone else notice? Major problem or minor routing issue?
Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. localrouter 67.6% 395 0.6 1.6 0.5 18.8 2.3 2. 10.1.41.15 0.0% 395 5.7 5.1 1.8 306.0 17.4 3. P4-2.LCR-02.WASHDC.verizon-g 0.0% 395 7.4 2.7 1.2 19.0 2.5 4. 130.81.29.218 0.0% 395 6.0 3.8 1.8 40.9 4.2 5. 152.63.39.177 0.0% 395 8.6 6.8 3.9 71.3 4.4 152.63.36.213 6. 152.63.69.113 71.6% 395 120.7 44.0 31.2 186.7 30.3 7. POS7-0-0.GW4.IND6.ALTER.NET 30.7% 395 1179. 133.3 121.3 1179. 79.5 8. 152.63.67.250 93.9% 395 121.5 125.4 121.0 186.2 13.0 9. POS6-0-0.GW4.IND6.ALTER.NET 53.0% 395 318.9 217.7 206.8 722.0 43.3 10. 152.63.67.250 96.2% 395 211.1 211.1 209.0 215.7 1.8 11. POS6-0-0.GW4.IND6.ALTER.NET 67.0% 395 422.1 305.9 294.9 692.1 37.5 12. 152.63.67.250 97.5% 394 295.1 298.0 295.1 303.6 2.5 13. POS6-0-0.GW4.IND6.ALTER.NET 73.5% 394 523.9 391.5 382.1 523.9 17.7 14. 152.63.67.250 98.7% 392 388.5 386.6 381.9 389.5 3.1 15. POS6-0-0.GW4.IND6.ALTER.NET 82.6% 392 632.9 481.2 468.6 632.9 22.2 16. 152.63.67.250 99.2% 388 472.7 472.2 470.2 473.6 1.8 17. POS6-0-0.GW4.IND6.ALTER.NET 85.8% 388 737.0 573.3 559.4 737.0 27.8 18. 152.63.67.250 99.2% 387 560.5 562.0 560.5 565.1 2.7 19. POS6-0-0.GW4.IND6.ALTER.NET 89.6% 387 839.0 664.8 644.9 839.0 38.6 20. 152.63.67.250 99.2% 387 649.3 649.6 649.3 649.9 0.3 21. POS6-0-0.GW4.IND6.ALTER.NET 94.8% 383 946.4 763.8 734.6 946.4 48.5 22. 152.63.67.250 99.7% 376 735.5 735.5 735.5 735.5 0.0 23. POS6-0-0.GW4.IND6.ALTER.NET 92.5% 376 895.4 842.2 819.1 909.0 26.8 24. ??? 25. POS6-0-0.GW4.IND6.ALTER.NET 96.7% 365 1153. 955.9 908.9 1153. 78.7 26. ??? 27. POS6-0-0.GW4.IND6.ALTER.NET 96.6% 328 1261. 1057. 998.8 1261. 86.8 28. 152.63.67.250 99.6% 245 999.3 999.3 999.3 999.3 0.0 29. POS6-0-0.GW4.IND6.ALTER.NET 98.8% 245 1189. 1123. 1086. 1189. 57.5 30. ???
Beckman --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ ---------------------------------------------------------------------------
-- ..!google!arpa.com!j The information contained in this e-mail and any attached documents may be privileged, confidential and protected from disclosure. If you are not the intended recipient you may not read, copy, distribute or use this information. If you have received this communication in error, please notify the sender immediately by replying to this message and then delete it from your system
At about 4:24pm EDT, I lost connectivity from Verizon to destinations in New York, Seattle and others. Came back up (4:46pm) while composing
I was told this was a Level3 router leaking bad routes into Verizon, and was told the problem is now resolved. Mike -----Original Message----- From: Paul Jasa [mailto:pjasa@univision.net] Sent: Wednesday, November 12, 2008 2:27 PM To: jamie rishaw; Peter Beckman Cc: nanog@nanog.org Subject: RE: Verizon/UU.net/Alternet Routing issue Same here. Saw the issue from Los Angeles, and from New York. Traces were dropping a few hops into the Verizon cloud. BGP stayed up, but routing went nowhere. Paul ________________________________ From: jamie rishaw [mailto:j@arpa.com] Sent: Wed 11/12/2008 3:14 PM To: Peter Beckman Cc: nanog@nanog.org Subject: Re: Verizon/UU.net/Alternet Routing issue Confirmed here as well; Saw loss on DS3s between 424 and 440 EST. BGP survived but routing didnt .. No RCA yet from VZN (on hold). On Wed, Nov 12, 2008 at 3:47 PM, Peter Beckman <beckman@angryox.com> wrote: this
email. Anyone else notice? Major problem or minor routing issue?
Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. localrouter 67.6% 395 0.6 1.6 0.5 18.8 2.3 2. 10.1.41.15 0.0% 395 5.7 5.1 1.8 306.0 17.4 3. P4-2.LCR-02.WASHDC.verizon-g 0.0% 395 7.4 2.7 1.2 19.0 2.5 4. 130.81.29.218 0.0% 395 6.0 3.8 1.8 40.9 4.2 5. 152.63.39.177 0.0% 395 8.6 6.8 3.9 71.3 4.4 152.63.36.213 6. 152.63.69.113 71.6% 395 120.7 44.0 31.2 186.7 30.3 7. POS7-0-0.GW4.IND6.ALTER.NET 30.7% 395 1179. 133.3 121.3 1179. 79.5 8. 152.63.67.250 93.9% 395 121.5 125.4 121.0 186.2 13.0 9. POS6-0-0.GW4.IND6.ALTER.NET 53.0% 395 318.9 217.7 206.8 722.0 43.3 10. 152.63.67.250 96.2% 395 211.1 211.1 209.0 215.7 1.8 11. POS6-0-0.GW4.IND6.ALTER.NET 67.0% 395 422.1 305.9 294.9 692.1 37.5 12. 152.63.67.250 97.5% 394 295.1 298.0 295.1 303.6 2.5 13. POS6-0-0.GW4.IND6.ALTER.NET 73.5% 394 523.9 391.5 382.1 523.9 17.7 14. 152.63.67.250 98.7% 392 388.5 386.6 381.9 389.5 3.1 15. POS6-0-0.GW4.IND6.ALTER.NET 82.6% 392 632.9 481.2 468.6 632.9 22.2 16. 152.63.67.250 99.2% 388 472.7 472.2 470.2 473.6 1.8 17. POS6-0-0.GW4.IND6.ALTER.NET 85.8% 388 737.0 573.3 559.4 737.0 27.8 18. 152.63.67.250 99.2% 387 560.5 562.0 560.5 565.1 2.7 19. POS6-0-0.GW4.IND6.ALTER.NET 89.6% 387 839.0 664.8 644.9 839.0 38.6 20. 152.63.67.250 99.2% 387 649.3 649.6 649.3 649.9 0.3 21. POS6-0-0.GW4.IND6.ALTER.NET 94.8% 383 946.4 763.8 734.6 946.4 48.5 22. 152.63.67.250 99.7% 376 735.5 735.5 735.5 735.5 0.0 23. POS6-0-0.GW4.IND6.ALTER.NET 92.5% 376 895.4 842.2 819.1 909.0 26.8 24. ??? 25. POS6-0-0.GW4.IND6.ALTER.NET 96.7% 365 1153. 955.9 908.9 1153. 78.7 26. ??? 27. POS6-0-0.GW4.IND6.ALTER.NET 96.6% 328 1261. 1057. 998.8 1261. 86.8 28. 152.63.67.250 99.6% 245 999.3 999.3 999.3 999.3 0.0 29. POS6-0-0.GW4.IND6.ALTER.NET 98.8% 245 1189. 1123. 1086. 1189. 57.5 30. ???
Beckman
------------------------------------------------------------------------ ---
Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/
------------------------------------------------------------------------ ---
-- .!google!arpa.com!j The information contained in this e-mail and any attached documents may be privileged, confidential and protected from disclosure. If you are not the intended recipient you may not read, copy, distribute or use this information. If you have received this communication in error, please notify the sender immediately by replying to this message and then delete it from your system -- THIS E-MAIL MESSAGE AND ANY FILES TRANSMITTED HEREWITH, ARE INTENDED SOLELY FOR THE USE OF THE INDIVIDUAL(S) ADDRESSED AND MAY CONTAIN CONFIDENTIAL, PROPRIETARY OR PRIVILEGED INFORMATION. IF YOU ARE NOT THE ADDRESSEE INDICATED IN THIS MESSAGE (OR RESPONSIBLE FOR DELIVERY OF THIS MESSAGE TO SUCH PERSON) YOU MAY NOT REVIEW, USE, DISCLOSE OR DISTRIBUTE THIS MESSAGE OR ANY FILES TRANSMITTED HEREWITH. IF YOU RECEIVE THIS MESSAGE IN ERROR, PLEASE CONTACT THE SENDER BY REPLY E-MAIL AND DELETE THIS MESSAGE AND ALL COPIES OF IT FROM YOUR SYSTEM.
At about 4:24pm EDT, I lost connectivity from Verizon to destinations in New York, Seattle and others. Came back up (4:46pm) while composing
Anyone else still seeing routing issues from Verizon's network still? We are getting intermittent routing to/from our IP space. Sincerely, Eric Tykwinski TrueNet, Inc. P: 610-429-8300 F: 610-429-3222 -----Original Message----- From: Braun, Mike [mailto:MBraun@firstam.com] Sent: Wednesday, November 12, 2008 6:06 PM To: nanog@nanog.org Subject: RE: Verizon/UU.net/Alternet Routing issue I was told this was a Level3 router leaking bad routes into Verizon, and was told the problem is now resolved. Mike -----Original Message----- From: Paul Jasa [mailto:pjasa@univision.net] Sent: Wednesday, November 12, 2008 2:27 PM To: jamie rishaw; Peter Beckman Cc: nanog@nanog.org Subject: RE: Verizon/UU.net/Alternet Routing issue Same here. Saw the issue from Los Angeles, and from New York. Traces were dropping a few hops into the Verizon cloud. BGP stayed up, but routing went nowhere. Paul ________________________________ From: jamie rishaw [mailto:j@arpa.com] Sent: Wed 11/12/2008 3:14 PM To: Peter Beckman Cc: nanog@nanog.org Subject: Re: Verizon/UU.net/Alternet Routing issue Confirmed here as well; Saw loss on DS3s between 424 and 440 EST. BGP survived but routing didnt .. No RCA yet from VZN (on hold). On Wed, Nov 12, 2008 at 3:47 PM, Peter Beckman <beckman@angryox.com> wrote: this
email. Anyone else notice? Major problem or minor routing issue?
Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. localrouter 67.6% 395 0.6 1.6 0.5 18.8 2.3 2. 10.1.41.15 0.0% 395 5.7 5.1 1.8 306.0 17.4 3. P4-2.LCR-02.WASHDC.verizon-g 0.0% 395 7.4 2.7 1.2 19.0 2.5 4. 130.81.29.218 0.0% 395 6.0 3.8 1.8 40.9 4.2 5. 152.63.39.177 0.0% 395 8.6 6.8 3.9 71.3 4.4 152.63.36.213 6. 152.63.69.113 71.6% 395 120.7 44.0 31.2 186.7 30.3 7. POS7-0-0.GW4.IND6.ALTER.NET 30.7% 395 1179. 133.3 121.3 1179. 79.5 8. 152.63.67.250 93.9% 395 121.5 125.4 121.0 186.2 13.0 9. POS6-0-0.GW4.IND6.ALTER.NET 53.0% 395 318.9 217.7 206.8 722.0 43.3 10. 152.63.67.250 96.2% 395 211.1 211.1 209.0 215.7 1.8 11. POS6-0-0.GW4.IND6.ALTER.NET 67.0% 395 422.1 305.9 294.9 692.1 37.5 12. 152.63.67.250 97.5% 394 295.1 298.0 295.1 303.6 2.5 13. POS6-0-0.GW4.IND6.ALTER.NET 73.5% 394 523.9 391.5 382.1 523.9 17.7 14. 152.63.67.250 98.7% 392 388.5 386.6 381.9 389.5 3.1 15. POS6-0-0.GW4.IND6.ALTER.NET 82.6% 392 632.9 481.2 468.6 632.9 22.2 16. 152.63.67.250 99.2% 388 472.7 472.2 470.2 473.6 1.8 17. POS6-0-0.GW4.IND6.ALTER.NET 85.8% 388 737.0 573.3 559.4 737.0 27.8 18. 152.63.67.250 99.2% 387 560.5 562.0 560.5 565.1 2.7 19. POS6-0-0.GW4.IND6.ALTER.NET 89.6% 387 839.0 664.8 644.9 839.0 38.6 20. 152.63.67.250 99.2% 387 649.3 649.6 649.3 649.9 0.3 21. POS6-0-0.GW4.IND6.ALTER.NET 94.8% 383 946.4 763.8 734.6 946.4 48.5 22. 152.63.67.250 99.7% 376 735.5 735.5 735.5 735.5 0.0 23. POS6-0-0.GW4.IND6.ALTER.NET 92.5% 376 895.4 842.2 819.1 909.0 26.8 24. ??? 25. POS6-0-0.GW4.IND6.ALTER.NET 96.7% 365 1153. 955.9 908.9 1153. 78.7 26. ??? 27. POS6-0-0.GW4.IND6.ALTER.NET 96.6% 328 1261. 1057. 998.8 1261. 86.8 28. 152.63.67.250 99.6% 245 999.3 999.3 999.3 999.3 0.0 29. POS6-0-0.GW4.IND6.ALTER.NET 98.8% 245 1189. 1123. 1086. 1189. 57.5 30. ???
Beckman
------------------------------------------------------------------------ ---
Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/
------------------------------------------------------------------------ ---
-- .!google!arpa.com!j The information contained in this e-mail and any attached documents may be privileged, confidential and protected from disclosure. If you are not the intended recipient you may not read, copy, distribute or use this information. If you have received this communication in error, please notify the sender immediately by replying to this message and then delete it from your system -- THIS E-MAIL MESSAGE AND ANY FILES TRANSMITTED HEREWITH, ARE INTENDED SOLELY FOR THE USE OF THE INDIVIDUAL(S) ADDRESSED AND MAY CONTAIN CONFIDENTIAL, PROPRIETARY OR PRIVILEGED INFORMATION. IF YOU ARE NOT THE ADDRESSEE INDICATED IN THIS MESSAGE (OR RESPONSIBLE FOR DELIVERY OF THIS MESSAGE TO SUCH PERSON) YOU MAY NOT REVIEW, USE, DISCLOSE OR DISTRIBUTE THIS MESSAGE OR ANY FILES TRANSMITTED HEREWITH. IF YOU RECEIVE THIS MESSAGE IN ERROR, PLEASE CONTACT THE SENDER BY REPLY E-MAIL AND DELETE THIS MESSAGE AND ALL COPIES OF IT FROM YOUR SYSTEM.
Eric Tykwinski wrote:
Anyone else still seeing routing issues from Verizon's network still? We are getting intermittent routing to/from our IP space.
Sincerely,
Eric Tykwinski TrueNet, Inc. P: 610-429-8300 F: 610-429-3222
I've gotten word that some people on Verizon DSL in the Philly area are having trouble reaching our network, but haven't been able to confirm yet. Andrew
participants (9)
-
Alan Halachmi -
andrew2＠one.net -
Braun, Mike -
Darryl Dunkin -
Eric Tykwinski -
jamie rishaw -
Mike Walter -
Paul Jasa -
Peter Beckman