- NANOG - lists.nanog.org

Re: Sandy seen costing telco, cable hundreds of millions of dollars
by Alex Rubenstein 08 Nov '12

08 Nov '12

Probably ATT. Many areas of NJ had zero service from them for days. ----- Original Message ----- From: Jima <nanog(a)jima.tk> To: nanog <nanog(a)nanog.org> Sent: Wed Nov 07 09:32:25 2012 Subject: RE: Sandy seen costing telco, cable hundreds of millions of dollars On Tuesday, 2012-11-06, Frank Bulk wrote: > So which wireless carrier is bringing down the average to 81%? A quick skim of the article (again, http://www.reuters.com/article/2012/11/01/storm-sandy-telecoms-idUSL1E8M1L9… ) makes me suspect AT&T. They're mentioned twice in other context, but there's not a sites-online statistic for them. I suppose it's worth noting that this wouldn't be the first time they've caught flak for their (in)ability to cover NYC sufficiently. Jima

2 1

MTU issues s0.wp.com
by Seth Mos 08 Nov '12

08 Nov '12

Hi, Since about a week or so it's become impossible to reach wp.com content over IPv6. IPv4 content does work fine, using the IPv6 literal returns a 404 which is small enough to fit in a smaller 1480 byte MTU. I have another test site that has a clean 1500 byte mtu and I can fetch the s0.wp.com page from there. It looks like tunneled IPv6 users might be in hurt here. Is anyone else experiencing similar issues? My traceroute shows they are employing a CDN for s0.wp.com, so not everyone might be affected. 7 asd2-rou-1022.NL.eurorings.net (2001:680:0:800f::291) 6.460 ms 6.203 ms 6.188 ms 8 asd2-rou-1044.eurorings.net (2001:680::134:222:85:63) 6.447 ms 6.494 ms 6.495 ms 9 adm-b5-link.telia.net (2001:2000:3080:6f::1) 6.818 ms 6.936 ms 6.891 ms 10 ldn-b3-v6.telia.net (2001:2000:3018:5::1) 15.290 ms 27.481 ms 15.380 ms 11 edgecast-ic-147468-ldn-b3.c.telia.net (2001:2000:3080:378::2) 15.116 ms 15.174 ms 15.176 ms 12 2606:2800:234:1922:15a7:17bf:bb7:f09 (2606:2800:234:1922:15a7:17bf:bb7:f09) 15.496 ms 15.327 ms 15.460 ms Kind regards, Seth

7 6

route-views.eqix DC METRO AREA IX RENUMBERING
by John Kemp 08 Nov '12

08 Nov '12

We have the renumber interface enabled and configured for any known peers to make the transition for the RouteViews EQUINIX ASHBURN route collector. OLD PEERING ADDRESS: 206.223.115.142 NEW PEERING ADDRESS: 206.126.236.142 Current v4 peer list looks like below. If you need to check, telnet to route-views.eqix.routeviews.org Thanks, John Kemp (kemp(a)routeviews.org) > Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down > State/PfxRcd > 206.126.236.10 4 4589 0 0 0 0 0 never > Active > 206.126.236.12 4 2914 0 0 0 0 0 never > Active > 206.126.236.19 4 3257 0 0 0 0 0 never > Active > 206.126.236.24 4 11666 0 0 0 0 0 never > Active > 206.126.236.25 4 6079 0 0 0 0 0 never > Active > 206.126.236.26 4 16559 0 0 0 0 0 never > Active > 206.126.236.37 4 6939 0 0 0 0 0 never > Active > 206.126.236.47 4 19151 0 0 0 0 0 never > Active > 206.126.236.52 4 4565 0 0 0 0 0 never > Active > 206.126.236.58 4 32098 0 0 0 0 0 never > Active > 206.126.236.60 4 4436 0 0 0 0 0 never > Active > 206.126.236.61 4 4436 0 0 0 0 0 never > Active > 206.126.236.76 4 5769 0 0 0 0 0 never > Active > 206.126.236.81 4 6453 0 0 0 0 0 never > Active > 206.126.236.109 4 19166 0 0 0 0 0 never > Active > 206.126.236.120 4 41095 0 0 0 0 0 never > Active > 206.126.236.156 4 7795 0 0 0 0 0 never > Active > 206.126.236.181 4 8781 0 0 0 0 0 never > Active > 206.223.115.10 4 4589 168086 1454 0 0 0 1d00h12m > 428226 > 206.223.115.12 4 2914 335570 1454 0 0 0 1d00h12m > 422014 > 206.223.115.19 4 3257 306479 2886 0 0 0 1d00h12m > 421751 > 206.223.115.24 4 11666 350048 2886 0 0 0 1d00h12m > 427034 > 206.223.115.25 4 6079 127539 1454 0 0 0 1d00h12m > 421048 > 206.223.115.26 4 16559 150834 1454 0 0 0 1d00h12m > 422475 > 206.223.115.37 4 6939 280514 1454 0 0 0 1d00h12m > 426934 > 206.223.115.47 4 19151 179133 1454 0 0 0 1d00h12m > 424028 > 206.223.115.52 4 4565 3061 2886 0 0 0 > 1d00h12m 2058 > 206.223.115.58 4 32098 5855 1454 0 0 0 > 1d00h12m 957 > 206.223.115.60 4 4436 235866 2886 0 0 0 1d00h12m > 422375 > 206.223.115.61 4 4436 237259 2886 0 0 0 1d00h12m > 422375 > 206.223.115.76 4 5769 205324 1454 0 0 0 1d00h12m > 422482 > 206.223.115.81 4 6453 0 0 0 0 0 never > Active > 206.223.115.109 4 19166 0 0 0 0 0 never > Active > 206.223.115.120 4 41095 166604 1454 0 0 0 1d00h12m > 422118 > 206.223.115.156 4 7795 3176 1454 0 0 0 > 1d00h12m 191 > 206.223.115.181 4 8781 6157 1454 0 0 0 > 1d00h12m 764 > -- John Kemp (kemp(a)routeviews.org) RouteViews Engineer NOC: noc(a)routeviews.org MAIL: help(a)routeviews.org WWW: http://www.routeviews.org

1 0

Looking for a outside plant contact Zayo/AboveNet Manhattan
by Christopher J. Pilkington 08 Nov '12

08 Nov '12

We're looking at some emergency office space in Manhattan and we identified an AboveNet/Zayo fiber panel in the space. Would like to see if someone could confirm if it is viable. Anyone from Abovenet lurking? Thanks, -cjp

1 0

Call for Presentations: NANOG 57 in Orlando, FL
by David Temkin 08 Nov '12

08 Nov '12

NANOG Community, I know that we all just left Dallas after NANOG 56, but the NANOG Program Committee is already hard at work preparing for NANOG 57 in Orlando! The North American Network Operators' Group (NANOG) will hold their 57th meeting in Orlando, FL on February 4th through the 6th. Of special note, this is the first meeting that will have a fully Monday through Wednesday agenda. Our host, CyrusOne is eagerly awaiting welcoming you to the Renaissance Orlando at SeaWorld. The NANOG Program Committee is now seeking proposals for presentations, panels, tutorials, tracks sessions, and keynote materials for the NANOG 57 program. We invite presentations highlighting issues relating to technology already deployed or soon-to-be deployed in the Internet. Vendors are encouraged to work with operators to present real-world deployment experiences with the vendor's products and interoperability. NANOG 57 submissions are welcome at http://pc.nanog.org For further information on what the Program Committee is seeking, please see http://www.nanog.org/meetings/nanog57/callforpresentations.html This will also be our first meeting after the 2012 WCIT in early December, and we expect topical and timely presentations regarding the results When considering submitting a presentation, keep these important dates in mind: Presentation Abstracts and Draft Slides Due: 10-December-2012 Final Slides Due: 7-January-2013 Draft Program Published: 14-January-2013 Final Agenda Published: 18-January-2013 Please submit your materials to http://pc.nanog.org Looking forward to seeing everyone in Orlando! -Dave Temkin

1 0

Re: Looking for recommendation on 10G Ethernet switch
by Andrew Latham 08 Nov '12

08 Nov '12

On Fri, Nov 2, 2012 at 2:38 PM, Kevin L. Karch <Kevinkarch(a)vackinc.com> wrote: > Andrew > > We offer several solutions that meet your initial requirements. Can you tell me if this is a multi rack deployment and a few more details? > > If you would like we could have a call with one of our applications engineers and get a budgetary quote assembled. > > Please let me know how you would like to proceed. > > Thank you, > > Kevin L. Karch > Network Specialist > > Direct: 847-833-8810 > Fax: 847-985-5550 > Email: kevinkarch(a)vackinc.com > Web: www.vackinc.com > The Optical Network Specialists Kevin, no thank you, I did not start this thread. If I ever need products I reach out to my contacts at each manufacturer or distributor. It would be much less embarrassing for you if the website in your signature actually finished loading the images containing the text for your company. -- ~ Andrew "lathama" Latham lathama(a)gmail.com http://lathama.net ~

3 2

NJ impact
by Alex Rubenstein 07 Nov '12

07 Nov '12

I had to summarize this recently for a news article I was interviewed for, so I figured I forward: -- Of our three datacenters, this is what we saw: Parsippany 1 (OCT) - The worst we saw here was several sub-second power hits. UPS's held without problem, and we did not transfer to generator at all yet. Parsippany 2 (WBR) - Transferred to generator at about 7:55 PM EST Monday as a precautionary measure due to ongoing utility power hits. However, shortly after transfer, utility voltage went to 0 on all phases; around 10p power returned, but abnormally high (seeing about 550 volts on 480 volt bus). We retransferred last night as utility voltage settled down. Cedar Knolls 1 (MMU) - Briefly transferred to generator around 7:10, then back to utility. We then force transferred to generator around 8pm and stayed until this morning. Returned to utility and all systems are normal. We have accommodated many customers (and non-customers) by allowing them to use our conference rooms and some offices, mainly because they lost power at their buildings, lost access to their buildings, or generators ran out of fuel. We literally have card tables setup in hallways and common areas where people sit and use Wifi to conduct business. There has been no service interruption of any sort to any of our datacenter customers, as of this writing. We lost one span of fiber optics between MMU and 165 Halsey, Newark (NWR), network healed in less than 1 second. It was imperceptible. That was early though, Monday afternoon. This has since restored today. We have sales people onsite who are actively taking sales calls, and there is a measurable amount of customers moving equipment into our facilities as I write this. Many are moving equipment from offices with no power and no ETR in sight. We have electricians onsite to help accommodate. The facilities department has been manning all locations around the clock. All sites remain accessible by road. There was never a situation where you could not over the last 48 hours. We have no equipment in 111 8th, so I cannot offer any telemetry there. However, our gear in 60 Hudson St (Tel-X 9th floor) is still online. We saw no interruption at 165 Halsey, except for some very brief utility hits. UPS's held. We are seeing hundreds, if not thousands, of DSL / T1 / T3 / Ethernet customers down - presumably mostly power loss on the CPE. We lost our connection to Covad, presumably they have gear in a building which lost power. These are only just starting to come back. We have seen substantial interruption to the cellular networks; in many cases, ATT Wireless (who NAC uses) is unable to complete a call. In certain areas, there is no signal at all. Where I live, in 07874 (Byram), many roads are impassable and most everyone does not have power. Trees are down everywhere. I heard that a Byram squad car had a tree land on it while responding to a call. Schools are closed, Halloween has been "rescheduled" according to our governor. My cable is down, cable modem is down, POTS service is out - yet I still have xDSL (loop by Verizon, IP by NAC) and I run on that successfully. I am unable to get any ATT cell service at my house (usually at least three bars). I am on generator at my house, consuming about 2 gal/hour (propane), with a little over 500 gallons left. Gasonline is becoming an issue. As I understand, having talked to our oil supplier, there is 0 refinery capacity in NJ at this time. All gasoline is being imported from PA, DE, MD, etc. Most stations don't have power. Ones that do are generally out of gas. If they have gas, they have mile long lines and are rationing. Diesel is unavailable if you had not pre-arranged for it. Widespread power outages remain. I am running on generator at my house (Generac QT048) on a 1000 gallon tank of propane, have been since Monday about 7pm. It is estimated at least 7 to 10 days before power restoration. I have yet to see one JCPL truck anywhere near where I live. That is all I have. This is the road leading to the Byram Police Dept - I took this yesterday morning: https://sphotos-a.xx.fbcdn.net/hphotos-prn1/12696_4311181250657_900557870_n… This is typically what Northern NJ looks like: https://sphotos-a.xx.fbcdn.net/hphotos-ash3/548544_4311185050752_1011136048… https://sphotos-b.xx.fbcdn.net/hphotos-ash3/61115_4317167440308_289655722_n… (not the top of the telephone pole hanging from the wires)

2 2

Indonesian ISP Moratel announces Google's prefixes
by Anurag Bhatia 07 Nov '12

07 Nov '12

Another case of route hijack - http://blog.cloudflare.com/why-google-went-offline-today-and-a-bit-about I am curious if big networks have any pre-defined filters for big content providers like Google to avoid these? I am sure internet community would be working in direction to somehow prevent these issues. Curious to know developments so far. Thanks. -- Anurag Bhatia anuragbhatia.com Linkedin <http://in.linkedin.com/in/anuragbhatia21> | Twitter<https://twitter.com/anurag_bhatia>| Google+ <https://plus.google.com/118280168625121532854>

9 14

AS 1668 BGP contact - possible prefix hijacking
by Ben Bartsch 07 Nov '12

07 Nov '12

Hi: Is there anyone here who can help us with a possible prefix hijacking situation through ATDN? Please contact me off list if you (or you know somebody) that can help us. I've tried the ATDN NOC and Vikas, but they have been no help whatsoever. The hijacked prefix appears to be sent to AS 1668, then propagated from there. Thanks. Ben AS 32440

2 2

Sandy seen costing telco, cable hundreds of millions of dollars
by Roy 07 Nov '12

07 Nov '12

http://www.reuters.com/article/2012/11/01/storm-sandy-telecoms-idUSL1E8M1L9…

3 2