- Test - lists.nanog.org

Re: Diversity - was: Fiber cut in SF area
by joel.mercado＠verizon.net 16 Apr '09

16 Apr '09

Hopefully none of these customers had service and protect ckts that went down... I would be pissed as a ceo if that happen to my company. Hopefully level3's new service offering is 100@percent redundant as stated The new service offerings include: - Protected Wavelengths: Level 3 now provides automatic protection-switching to a dedicated diversely routed wavelength in the event of a network failure. The protection switch, fully automated and managed by Level 3, happens at switching speeds approaching SONET restoration times. The single interface to the customer requires no additional capital cost for customer optical ports, and the diverse restoration path is fixed and fully known to the customer. These features allow customers to achieve fast restoration with predictable performance in their network without adding significant cost and routing complexity. - ------Original Message------ From: Wallace Keith To: nanog(a)nanog.org Subject: Diversity - was: Fiber cut in SF area Sent: Apr 14, 2009 7:06 PM -----Original Message----- From: Roy [mailto:r.engehausen@gmail.com] Sent: Tuesday, April 14, 2009 6:43 PM To: Gino Villarini Cc: nanog(a)nanog.org Subject: Re: Fiber cut in SF area Gino Villarini wrote: > Here in my area most of business outfits that require maximum > availability of Internet or WAN conenctions have implemented dual > connections from dual providers, most with a fiber/copper main and a > fixed wireless backup. This trend goes from banks to Mcdonalds > > > Gino A. Villarini > gav(a)aeronetpr.com > Aeronet Wireless Broadband Corp. > tel 787.273.4143 fax 787.273.4145 > > >A large company in the affected area had a T3 supplied by AT&T and a >wireless link to another ISP that was fed by two metro-ethernet links by >companies other than AT&T. >All three uplinks were lost. So much for having backups, This just goes to emphasize that when creating a diversity or backup scenario, you need to get full disclosure from Provider B that they do not use Provider A's facilities, including shared sheath, duct, etc in any way. Also, there is the need to avoid the same telco buildings, regen huts, etc. and in some cases, entire cities. Any telecom/datacom manager who has done their homework should be able to map out their paths back to critical diverse infrastructure. -Keith Sent on the Now Network� from my Sprint® BlackBerry

14 20

Re: Looking for AT&T / Verizon / Sprint WWAN service impressions- onor off-list replies welcome
by joel.mercado＠verizon.net 16 Apr '09

16 Apr '09

I am 100 percent with you on this. Some techs arrive to our data center with no tools and they have the same response I just thought it was a simple install. I know they have different levels for techs but you should not have to wait another couple of days to complete a install. They should send another tech same day but that's not the case. ------Original Message------ From: Charles Wyble To: Crooks, Sam Cc: nanog(a)nanog.org Subject: Re: Looking for AT&T / Verizon / Sprint WWAN service impressions- onor off-list replies welcome Sent: Apr 15, 2009 7:56 PM What is it about the bloody telcos. You want to spend money, but yet you can't reach the right people to get your questions answered or schedule the service. Gah. I experienced this recently, trying to have some inside wiring work done at my house. They rolled a tech, but then he claimed he "wasn't prepared" to do the work. What exactly was he prepared to do, on an inside wiring call? It took multiple calls / disconnects to SBC to get to the right dept and have a tech deployed who was actually prepared to install the jack. Do these places take courses on anti sustainability? It's amazing they are still in business. Crooks, Sam wrote: > > After much hassle and several false starts and disconnects in getting in > touch with the right department in Sprint, I spoke to a woman in > technical support in the group that supports 3G data cards. > > She said: > > - public IP addresses are used > - static IP available for $3/mo additional > - maximum 3G data plan is 5GB/mo of data transfer, retail reate, $60/mo > (which is typically cheaper than your typical ADSL/IDSL or ISDN service > cost) > > > Sam Crooks > > Sent on the Now Network� from my Sprint® BlackBerry

1 0

Re: Looking for AT&T / Verizon / Sprint WWAN service impressions - on or off-list replies welcome
by Seth Mattinen 15 Apr '09

15 Apr '09

Charles Wyble wrote: > > > Crooks, Sam wrote: >> I'm considering use of AT&T / Verizon / Sprint WWAN services and the >> Cisco 3G router interface cards/integrated module in C880 routers for >> primary or backup WAN network connectivity for routers. >> > > I haven't used the integrated cards with cisco gear. However I do have > 300+ cards deployed throughout the United States (EVDO USB modems on > Linux boxes). > > >> I'm looking for information from users of these services on the >> following: >> - addressing - Do these WWAN services use dynamic, PPPoE or static IP >> assignment typically? Any of the 3? All? >> - is static IP assignment available? > > We have static IP assignment for our Verizon cards. Sprint cards aren't > static. > I received an offlist response indicating Sprint now offers static. ~Seth

3 2

tcptraceroute, traceroute and IP addresses [was]Re: Level3 funkiness
by Scott Weeks 15 Apr '09

15 Apr '09

> # traceroute level3.net When diagnosing things like this try using the IP address and tcptraceroute or some similar tool. NOT plain old traceroute and a DNS name. Especially when writing to a list with participants as technically involved as those on NANOG. scott

2 1

Level3 funkiness
by J. Oquendo 15 Apr '09

15 Apr '09

Anyone else experience sporadic funkiness via Level3? I can't even reach the main website from who knows how many networks I've tried. Also friends and former colleagues have tried to reach the site to no avail. One of my machines on AT&T: # traceroute level3.net traceroute to level3.net (63.211.236.36), 30 hops max, 40 byte packets 4 cr1.n54ny.ip.att.net (12.122.105.58) 11.285 ms 21.702 ms 21.477 ms 5 ggr2.n54ny.ip.att.net (12.122.131.141) 12.712 ms 10.194 ms 16.393 ms 6 so-8-0-0.car3.NewYork1.Level3.net (4.68.127.149) 9.975 ms 10.019 ms 10.833 ms 7 vlan79.csw2.NewYork1.Level3.net (4.68.16.126) 10.162 ms 10.189 ms 14.474 ms 8 ae-71-71.ebr1.NewYork1.Level3.net (4.69.134.69) 15.763 ms 11.166 ms 9.725 ms 9 ae-3-3.ebr4.Washington1.Level3.net (4.69.132.93) 16.139 ms 30.616 ms 16.275 ms 10 ae-64-64.csw1.Washington1.Level3.net (4.69.134.178) 15.684 ms ae-74-74.csw2.Washington1.Level3.net (4.69.134.182) 21.870 ms ae-84-84.csw3.Washington1.Level3.net (4.69.134.186) 28.729 ms 11 ae-92-92.ebr2.Washington1.Level3.net (4.69.134.157) 17.035 ms ae-62-62.ebr2.Washington1.Level3.net (4.69.134.145) 17.041 ms ae-72-72.ebr2.Washington1.Level3.net (4.69.134.149) 21.940 ms 12 ae-2-2.ebr2.Chicago2.Level3.net (4.69.132.69) 31.671 ms 42.407 ms 45.774 ms 13 ae-1-100.ebr1.Chicago2.Level3.net (4.69.132.113) 31.922 ms 32.115 ms 38.135 ms 14 ae-3.ebr2.Denver1.Level3.net (4.69.132.61) 75.265 ms 67.528 ms 67.937 ms 15 ge-9-0.hsa1.Denver1.Level3.net (4.68.107.35) 62.587 ms !H ge-9-1.hsa1.Denver1.Level3.net (4.68.107.99) 62.543 ms !H ge-9-2.hsa1.Denver1.Level3.net (4.68.107.163) 75.797 ms !H (From Texas through Above.net) $ traceroute level3.net|tail -n 1 traceroute to level3.net (63.211.236.36), 64 hops max, 40 byte packets 11 ge-6-2.hsa1.Denver1.Level3.net (4.68.107.131) 21.473 ms !H * ge-6-0.hsa1.Denver1.Level3.net (4.68.107.3) 21.547 ms !H Confirmed it can't be reached from Travelers Ins, The Hartford, none of my connections. Anyone else seeing issues? I'm seeing drop off from clients going through their Atlanta interconnects with Charter and two other providers, which I can't make sense of. I DO KNOW they experienced some sort of issue with a TDM switch or so they said... Very broad statements: "We know teh interwebs are down please stand by" I know websites are one thing, but the chances of the website going down, a TDM switch being wacky and now clients traversing their networks complaining all at once seems a little out of the ordinary. =+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ J. Oquendo SGFA, SGFE, C|EH, CNDA, CHFI, OSCP "It takes 20 years to build a reputation and five minutes to ruin it. If you think about that, you'll do things differently." - Warren Buffett 227C 5D35 7DCB 0893 95AA 4771 1DCE 1FD1 5CCD 6B5E http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x5CCD6B5E

13 18

RE: Level3 funkiness
by Dave Larter 15 Apr '09

15 Apr '09

Tracing to www.level3.net (4.68.95.28) dies at 4.68.94.1 for me as well. -----Original Message----- From: Andy Vance [mailto:avance@hq.speakeasy.net] Sent: Wednesday, April 15, 2009 3:55 PM To: Dave Larter; J. Oquendo; nanog(a)nanog.org Subject: RE: Level3 funkiness I'm not having any issues from either east coast or west coast in reaching Level-3. traceroute to www.level3.net (4.68.95.28), 30 hops max, 38 byte packets 1 5.ge-0-2-0.cr1.wdc1.speakeasy.net (66.92.159.252) 0.337 ms 0.269 ms 0.255 ms 2 100.ge-0-0-0.cr2.wdc1.speakeasy.net (69.17.83.66) 0.278 ms 0.264 ms 0.256 ms 3 ge-4-1-440.car2.Washington1.Level3.net (166.90.148.49) 0.932 ms 0.858 ms 0.822 ms 4 vlan89.csw3.Washington1.Level3.net (4.68.17.190) 14.051 ms 0.975 ms 1.001 ms 5 ae-82-82.ebr2.Washington1.Level3.net (4.69.134.153) 1.340 ms 1.127 ms 1.385 ms 6 ae-2-2.ebr2.Chicago2.Level3.net (4.69.132.69) 18.646 ms 18.612 ms 18.519 ms 7 ae-1-100.ebr1.Chicago2.Level3.net (4.69.132.113) 18.925 ms 19.119 ms 18.712 ms 8 ae-3.ebr2.Denver1.Level3.net (4.69.132.61) 47.165 ms 53.793 ms 54.239 ms 9 ge-9-0.hsa1.Denver1.Level3.net (4.68.107.35) 43.107 ms 43.287 ms ge-9-1.hsa1.Denver1.Level3.net (4.68.107.99) 42.558 ms 10 4.68.94.1 (4.68.94.1) 43.116 ms 43.128 ms 43.224 ms Cheers, Andy -----Original Message----- From: Dave Larter [mailto:dave@stayonline.com] Sent: Wednesday, April 15, 2009 12:46 PM To: J. Oquendo; nanog(a)nanog.org Subject: RE: Level3 funkiness Yes, I die on your hop14 with TWTelecom -----Original Message----- From: J. Oquendo [mailto:sil@infiltrated.net] Sent: Wednesday, April 15, 2009 3:36 PM To: nanog(a)nanog.org Subject: Level3 funkiness Anyone else experience sporadic funkiness via Level3? I can't even reach the main website from who knows how many networks I've tried. Also friends and former colleagues have tried to reach the site to no avail. One of my machines on AT&T: # traceroute level3.net traceroute to level3.net (63.211.236.36), 30 hops max, 40 byte packets 4 cr1.n54ny.ip.att.net (12.122.105.58) 11.285 ms 21.702 ms 21.477 ms 5 ggr2.n54ny.ip.att.net (12.122.131.141) 12.712 ms 10.194 ms 16.393 ms 6 so-8-0-0.car3.NewYork1.Level3.net (4.68.127.149) 9.975 ms 10.019 ms 10.833 ms 7 vlan79.csw2.NewYork1.Level3.net (4.68.16.126) 10.162 ms 10.189 ms 14.474 ms 8 ae-71-71.ebr1.NewYork1.Level3.net (4.69.134.69) 15.763 ms 11.166 ms 9.725 ms 9 ae-3-3.ebr4.Washington1.Level3.net (4.69.132.93) 16.139 ms 30.616 ms 16.275 ms 10 ae-64-64.csw1.Washington1.Level3.net (4.69.134.178) 15.684 ms ae-74-74.csw2.Washington1.Level3.net (4.69.134.182) 21.870 ms ae-84-84.csw3.Washington1.Level3.net (4.69.134.186) 28.729 ms 11 ae-92-92.ebr2.Washington1.Level3.net (4.69.134.157) 17.035 ms ae-62-62.ebr2.Washington1.Level3.net (4.69.134.145) 17.041 ms ae-72-72.ebr2.Washington1.Level3.net (4.69.134.149) 21.940 ms 12 ae-2-2.ebr2.Chicago2.Level3.net (4.69.132.69) 31.671 ms 42.407 ms 45.774 ms 13 ae-1-100.ebr1.Chicago2.Level3.net (4.69.132.113) 31.922 ms 32.115 ms 38.135 ms 14 ae-3.ebr2.Denver1.Level3.net (4.69.132.61) 75.265 ms 67.528 ms 67.937 ms 15 ge-9-0.hsa1.Denver1.Level3.net (4.68.107.35) 62.587 ms !H ge-9-1.hsa1.Denver1.Level3.net (4.68.107.99) 62.543 ms !H ge-9-2.hsa1.Denver1.Level3.net (4.68.107.163) 75.797 ms !H (From Texas through Above.net) $ traceroute level3.net|tail -n 1 traceroute to level3.net (63.211.236.36), 64 hops max, 40 byte packets 11 ge-6-2.hsa1.Denver1.Level3.net (4.68.107.131) 21.473 ms !H * ge-6-0.hsa1.Denver1.Level3.net (4.68.107.3) 21.547 ms !H Confirmed it can't be reached from Travelers Ins, The Hartford, none of my connections. Anyone else seeing issues? I'm seeing drop off from clients going through their Atlanta interconnects with Charter and two other providers, which I can't make sense of. I DO KNOW they experienced some sort of issue with a TDM switch or so they said... Very broad statements: "We know teh interwebs are down please stand by" I know websites are one thing, but the chances of the website going down, a TDM switch being wacky and now clients traversing their networks complaining all at once seems a little out of the ordinary. =+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ J. Oquendo SGFA, SGFE, C|EH, CNDA, CHFI, OSCP "It takes 20 years to build a reputation and five minutes to ruin it. If you think about that, you'll do things differently." - Warren Buffett 227C 5D35 7DCB 0893 95AA 4771 1DCE 1FD1 5CCD 6B5E http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x5CCD6B5E

2 1

SIP - perhaps botnet? anyone else seeing this?
by Leland E. Vandervort 15 Apr '09

15 Apr '09

Hi All, Over the past couple of days we have been seeing an exponential increase (about 200-fold) in the amount of UDP SIP Control traffic in our netflow data. The past 24 hours, for example, has shown a total of nearly 300 GB of this traffic incoming and over 400 GB outgoing -- this despite the fact that we do not host any SIP services ourselves, and currently to my knowledge, we have no hosting customers running any kind of SIP services. (Total RTP traffic for 24 hours is only in the region of 150 Kb -- so a vast inbalance between control and RTP) The local sources/destinations of the traffic are within our hosting space, but are spread across a wide range of hosts (i.e. nothing really related to a single or handful of hosts). Additionally over the past couple of days we have seen an increase of mails to our abuse desk for "brute force" attempts against a number of SIP services... possibly directly related to this traffic. Is anyone aware of a new variant or modus-operandi of botnets in circulation in the past couple of days which attempt to exploit SIP services? Has anyone else notice a significant increase in this kind of traffic? Thanks Leland

8 11

Re: Network SLA
by Saqib Ilyas 15 Apr '09

15 Apr '09

Hmmm. Good point. Perhaps the Internet traffic gets only a small share of the link capacity and the rest is reserved for corporate clients' VPN traffic etc. I was thinking more along the lines of corporate SLAs, not for Internet traffic. On Wed, Apr 15, 2009 at 4:05 PM, Rod Beck <Rod.Beck(a)hiberniaatlantic.com>wrote: > Congestion is more common than you think. And by the way, if congestion > is not a problem in Pakistan, then why is the VoIP qualit so poor? > > :) > > Roderick S. Beck > Director of European Sales > Hibernia Atlantic > 13-15, rue Sedaine, 75011 Paris > http://www.hiberniaatlantic.com > Wireless: 1-212-444-8829. > French Landline: 33+1+4355+8224 > French Wireless: 33-6-14-33-48-97. > AOL Messenger: GlobalBandwidth > rod.beck(a)hiberniaatlantic.com > rodbeck(a)erols.com > ``Unthinking respect for authority is the greatest enemy of truth.'' Albert > Einstein. > > > > -----Original Message----- > From: Saqib Ilyas [mailto:msaqib@gmail.com <msaqib(a)gmail.com>] > Sent: Wed 4/15/2009 11:22 AM > To: nanog(a)nanog.org > Subject: Re: Network SLA > > I talked to the NOC personnel at a small (compared to North American > standards) ISP in Pakistan. They said that their core links are operating > at > less than 50% utilization most of the time. Under such conditions, > violating > SLA conditions in the core is unlikely. If such is also the case with most > service providers in the North America as well, then why would they even > use > active measurement such as iPerf or BRIX or Cisco IP SLAs before signing an > SLA? > Thanks and best regards > > On Thu, Feb 19, 2009 at 8:50 PM, Saqib Ilyas <msaqib(a)gmail.com> wrote: > > > Greetings > > I am curious to know about any tools/techniques that a service provider > > uses to assess an SLA before signing it. That is to say, how does an > > administrator know if he/she can meet what he is promising. Is it based > on > > experience? Are there commonly used tools for this? > > Thanks and best regards > > -- > > Muhammad Saqib Ilyas > > PhD Student, Computer Science and Engineering > > Lahore University of Management Sciences > > > > > > -- > Muhammad Saqib Ilyas > PhD Student, Computer Science and Engineering > Lahore University of Management Sciences > > -- Muhammad Saqib Ilyas PhD Student, Computer Science and Engineering Lahore University of Management Sciences

3 2

ACLs vs. full firewalls
by Michael Helmeste 15 Apr '09

15 Apr '09

Hi all, One of the duties of my current place of employ is reorganizing the network. We have a few Catalyst 6500 series L3 switches, but currently do all packet filtering (and some routing) using a software based firewall. Don't ask me, I didn't design it :) Current security requirements are only based on TCP and non-stateful UDP src/dst net/port filtering, and so my suggestion was to use ACLs applied on the routed interface of each VLAN. There was some talk of using another software based firewall or a Cisco FWSM card to filter traffic at the border, mostly for management concerns. We expect full 1 gig traffic levels today, and 10 gig traffic levels in the future. I view ACLs as being a cheap, easy to administrate solution that scales with upgrades to new interface line speeds, where a full stateful firewall isn't necessary. However, I wanted to get other opinions of what packet filtering solutions people use in the border and in the core, and why. What's out there, and why do you guys use it? How do you feel about the scalability, performance, security, and manageability of your solution? What kind of traffic levels do you put through it?

13 14

Re: Fiber cut in SF area
by Murphy, Brian S CTR USAF ACC 83 NOS/Det 4 15 Apr '09

15 Apr '09

I haven't seen any mention of the possible use of FSO (Free Space Optics) by the provider to restore some reasonable amount of connectivity during an outage due to a fiber cut. I would expect that having 2 or 3 pairs of FSO boxes to provide a "reduced failover capacity" in metro areas would be a reasonable measure to ensure service for extended physical (fiber break, cut, backhoe) outages - although not necessarily for power. Yes, it would take some time to roll them out and set them up, but less time than the crew working the splices, and the folks handling the FSO boxes should be different from the fiber splice truck roll crew. Note that a power outage would not allow microwave to be an effective remediation method either. Plus, FSO's use of lasers (vice microwaves) means no issues with spectrum (AFAIK). Granted, they have limited distance and require LoS, but using two or more pairs can probably handle the 80% situation in the metro (unless there is data to indicate otherwise). murph ------------------------------------- Date: Tue, 14 Apr 2009 15:57:52 -0700 From: Roy <r.engehausen(a)gmail.com> Subject: Re: Fiber cut in SF area To: JC Dill <jcdill.lists(a)gmail.com> Cc: nanog(a)nanog.org Message-ID: <49E514F0.7000009(a)gmail.com> Content-Type: text/plain; charset=ISO-8859-1 JC Dill wrote: > Gino Villarini wrote: >> Good points, some variables are dependant on the network infrastructure >> of the wireless provider. Localy, the main 2 providers have a >> "copper/fiber independent" networks. >> >> > I'm pretty sure the WISPs in the Santa Cruz and Gilroy/Morgan Hill > areas were all also taken offline due to the fiber cut. (Roy, can you > verify, for south county?) Anyone in those areas who relied on a WISP > as a backup to their fiber/copper link found that their "redundant" > system wasn't really redundant after all. > > You may want to check (verify) how your 2 main providers handle their > backhaul. > > jc > > It based on where the WISP fiber feed was located but in general they were all down. There were some special edge cases that stayed up fed from distant mountain tops. It didn't seem to matter who your upstream ISP was, they were all gone.

3 2