- Test - lists.nanog.org

Re: Spring 1999 NOC contact drill
by Sean Donelan 21 Apr '99

21 Apr '99

wb8foz(a)nrk.COM (David Lesher) writes: >Putting asides the spamhaus "noDood(a)Bif.gov" ones, a large number >of the addresses appear to be just plain obsolete; the sysadm >moved on, etc. I've gotten a few questions about how I collected the contact information I'm using. Since the Internet is doubling every xxxx days, some folks may not remember this from last time. I originally used sources such as WHOIS, Manning's in-addr.arpa TXT records and various random web pages. But was informed by a number of providers that was unfair, because ARIN/NSI/whoever sometimes couldn't keep the updated information in their databases or because the phone number some providers prefer to list in their WHOIS records is for their sales department, not their NOC. Or they don't accept calls from non-customers. So I changed. The contact information I now use is based on the information each provider gave when we signed an agreement or set up our interconnect with them. According to almost every peering agreement, it is the responsibility of the peer to provide correct updated contact information, even after being 'borged' by another company. In all cases the contact information was provided directly by the other ISP to me. Some providers have given me direct phone numbers to their NOC, others have given me phone numbers to their customer service departments. I don't really care, its whatever phone number you think is most likely to work 24x7. I generally discourage personal cell-phone and pager numbers though. The test is really to reach the provider's crew, not an individual. If you provide a direct-dial telephone number, it doesn't even cost you the 1-800 charge. The requirement for being listed is a) a provider connected to a major US exchange point (mae-east, mae-west, aads, paix, pacbell, sprint) or represent a major piece of Internet infrastructure (root-server, radb). Everyone else I assume should be reached via their upstream provider. b) provided me a working contact telephone number and e-mail c) agree to take my call d) and confirmed the information when I did called. It would also be nice if you agreed to peer with me, but that isn't a strong requirement. However, when I can't reach someone I don't peer with, I just delete them from the list without following up. I assume calls should be directed through their upstream provider. I don't intend to check 6,000 different ISPs, just the 150 or so ISPs meeting those conditions above. And no, I do not believe or use Boardwatch's classification of ISPs, which seems based on who advertises in their publication more than any other factor. Heck, they've listed ISPs who didn't even have working routers at the time as "backbone providers." -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

1 0

Spring 1999 NOC contact drill
by Sean Donelan 21 Apr '99

21 Apr '99

The oldest and largest NOC contact drill is back. For the next week I will attempt to use the telephone number provided by each NOC to contact that provider. With the number of ISPs involved, it usually takes me about three days to call all of them, assuming something doesn't go wrong. All calls should be during normal USA business hours, so after-hour contacts won't be verified. Sometimes I make a mistake, and get the time zone of the ISP wrong, my apologies if that happens. I also try to check there are no major Internet outages in progress before calling so I didn't interrupt extreme NOC chaos. But again, I may make a mistake and call at the same time as a major outage. Again, my apologies. Let me know you are dealing with a crisis and I'll re-schedule the calls for later. In the script I use, I identify myself, identify DRA, and ask to confirm the telephone number, including a non-1-800/888/877 number, and e-mail address of the ISP's network operation center. In today's world of fancy phone switches, I always tell them what number I called. I've found many NOC's are unaware of what numbers are used to reach them, or when they change. Its amazing how often the person says "You must have called the right number because you reached us." And when I tell him what number I actually dialed, he has no idea how I reached their NOC using it. Everything from area code changes to companies merging NOCs and temporarily forwarding old NOC phone numbers happen. This seems especially bad for those ISPs being merged into Telcos. Telcos tend to have the worst problem with 'special' phone numbers for their NOCs which don't work outside their service area or even the country. But the problem does affect everyone to some degree. Voice-mail systems are a special case. I attempt to choose the 'obvious' prompts. But I'm often stymied by systems which require special codes before allowing contact with any human. If I can't figure the system out in less than five minutes I move on to the next provider. If necessary I explained what a network operations center is, or ask who I should call to report a network problem. I don't require access to the network engineering staff. A customer service person is Ok as long as they can confirm the correct telephone number for reporting network problems and e-mail address for the NOC. You are welcome to inform your network operators or customer service reps I may be calling. However, in the past I've found this doesn't help much. Corporate culture tends to overwhelm those efforts. As usual I will only identify those ISPs which I was able to confirm the contact information worked. [I wonder if I should identify those I can't successfully reach, since it seems to be the same four or five ISPs every time.] If you want to, you can consider this part of a Y2K contingency test. Even though I think having good contact information is needed all year long, and worrying about contingency planning for only one night misses the point. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

2 1

etwork Solutions competitors announced
by Rodney Joffe 21 Apr '99

21 Apr '99

For those who care, and missed it... ICANN announced the names of the 5 testbed registrars a short while ago.... They are: America Online CORE (Internet Council of Registrars) France Telecom/Oléane Melbourne IT register.com The complete release is at: http://www.icann.org/icann-pr21apr99.htm /rlj

1 0

Re: The future of NAPs & IXPs
by Vadim Antonov 20 Apr '99

20 Apr '99

Stephen -- i simply attached a date after the title. BTW - backplane capacity is not the same as user goodput capacity; and ethernet switch is not the same as a router. Cisco products historically demonstrated that quite clearly. And i'm working for a cisco competitor now (after the acquisition of GeoTel) and have no incentive to promote newer cisco's products :) --vadim Stephen Sprunk <ssprunk(a)cisco.com> wrote: >Perhaps you could update this paper to reflect current products, since you >specifically name vendors and their products' limitations without explicitly >listing any model numbers or dates? >For instance, the Cisco GSR (aka 12000) router has a non-blocking backplane >capacity of 40Gbit/s, where you list a maximum backplane capacity of >0.7Gbit/s. Also, the Cisco 6500 switch has a non-blocking backplane >capacity of 256Gbit/s and can currently hold up to 130 GigE ports. This >shows two orders of magnitude growth in capacity since your paper was >written, and that's not counting the products I can't tell you about yet :) >While I understand that the actual numbers are mostly irrelevant to the >paper, it would be appreciated if you'd either update the numbers or put in >a footnote recognizing that your numbers are out of date. >Stephen > | | Stephen Sprunk, K5SSS, CCIE #3723 > :|: :|: NSA, Network Consulting Engineer > :|||: :|||: 14875 Landmark Blvd #400; Dallas, TX >.:|||||||:..:|||||||:. Pager: 800-365-4578 / 800-901-6078 >C I S C O S Y S T E M S Email: ssprunk(a)cisco.com

1 0

Re: NETOPS PLANNING: april 23-25 in Washington DC
by Sean Donelan 20 Apr '99

20 Apr '99

dredd(a)megacity.ORG (Derek Balling) writes: >God I love this country. :) What? You want access to your personal >property? Too bad, the SS, I mean Secret Service says "Nein!". Well, there was a smiley in there. Before this veers too far off-topic, I would make this comment. After being on the other side during the Pope's visit to St. Louis in January trying to move VIPs around the city (the Secret Service has very cool credentials which melt and turn colors if you try to photocopy them), I've seen how hard the US Secret Service tries to assure property owners, tenents, etc access to their property. But sometimes it takes a bit of explaining before the officer manning the barricade understands why a person from Missouri needs to wheel a Cisco GSR router through the middle of a secure area in Washington DC in the middle of the night. If nothing else, the officer would question his sanity. Any sane person would want to drive directly into the gated garage :-) ObOpsPlanning: If something does go down over the weekend, we'll have it back up as fast as we can. There may be a few 'force majure' reasons why it takes a bit longer than normal. Thank you for your understanding. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

1 0

NETOPS PLANNING: april 23-25 in Washington DC
by Sean Donelan 20 Apr '99

20 Apr '99

Those ISPs without equipment in Washington DC may not be aware of it, but from April 23-25 the city will be a bit chaotic. The NATO meeting will have 42 ministerial delegations in town on those dates. What's this got to do with ISPs? Hopefully not much. However, its been a bit difficult getting accurate information from the US Secret Service about exactly what streets will be closed, and precisely which areas of the city will be in the exclusion zone. For example, it is unclear whether we will be able to enter a building that faces a closed street. Several major Internet circuits and facilities happen to run through and adjacent to these zones. A backhoe incident is unlikely. However, many facilities will be running unattended. It may not be possible for us or other ISPs to get into the area or our facilities at 3am to reboot routers. Please have patience, if they go down, we will get them back up as soon as we can. I may have a bit of trouble trying to explain why a person with a Missouri driver's license needs to get into the area. If you are not familiar with Washington, DC; Mae-east is located well away from the excitement and should be unaffected. However, other ISPs may also have equipment in the central part of the district, and may have similar problems to us. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

2 1

Re: The future of NAPs & IXPs
by Vadim Antonov 20 Apr '99

20 Apr '99

All is well but you missed one of the most critical issues with the current IXPs: lack of scalability. The private point-to-point interconnects are at least as fast as backbones. Fixing IXP scalability issues requires somewhat radical departure from the current router architecture; such as being done by terabit router vendors. In other words, even if multi-party IXPs are more cost-effective, they are currently (and in the near-term future) unable to handle the load. Also, the number of interconnects (IXPs or direct) cannot be large because of flap-amplification properties of the inter-backbone connections. (BTW, O(5) can be an arbitrarily large fixed number, simply speaking :) --vadim William B. Norton <wbn(a)equinix.com> wrote: >For what it's worth...I just finished a paper that highlights the trade >offs between the direct circuit interconnect model and the exchange point >interconnection model for ISPs. The paper discusses the operations and financial models (taking into account the circuit costs, cost of exchange participation, cost of dark fiber, etc.) and the implications of these strategies across the # of interconnection participants and bandwidth utilization between the participants. To cut to the chase, the major points from the paper: >1) For ISP interconnection, direct circuit interconnection is financially >attractive for low #s of connections (O(5)) of relatively low bandwidth >(DS-3/OC-3). >2) As the bandwidth and # of interconnections grow, the exchange point >interconnection model proves much more scalable for two reasons: First, as >bandwidth grows between participants, ISPs are able to aggregate >interconnection traffic over increasingly large pipe back to their cloud, >yielding potentially significant economies of scale. The direct circuit >interconnection does not provide for this aggregation since the pipes are >destined to different plances.

3 2

DOD mulls Internet disconnect (Re: The future of NAPs and IXPs)
by Sean Donelan 19 Apr '99

19 Apr '99

>> providers to troubleshoot, etc. The greater the dependence on >> interconnection, the more hardened you want that interconnection environment. > >It's worth pointing out, as a sidebar to that, that the greater the >dependence on interconnection, the more hardened it's _feasible_ to make >that interconnection environment, assuming you've rolled your numbers >right. Staying on the proper side of that curve is important... Don't fall into the trap of thinking you can harden anything enough. The Internet was built on the assumption diversity buys reliability better than hardening. To this day, it seems to be a hard thing to do in practice. It is much easier to adopt the bunker mentality, and try to protect it all behind one big wall. Which doesn't work much better in practice, but pays consultants a lot better. Normally, I wouldn't assume Lt. Generals understand technical matters very well, prefering instead to listen to majors and below. I have no clue what the Lt. General is thinking changing .mil to .com will do for security. http://www.fcw.com/pubs/fcw/1999/0419/fcw-newsdod-4-19-99.html I wonder if the Lt. General realizes the damage he may end up doing. Some of us "outsiders" have long relied on information from the military web sites to help us improve the design of our own infrastructures. Which in turn the government relies on to carry out its mission. It is a fallicy to think you can provide the information needed to protect the network to just one or two "cleared" companies. I'm not "cleared," yet I still need provide information assurance services to my clients. It would be a shame if a hacker chopped three "zeros" off a defense spending bill. I thought the government had figured there is more interdependence between all levels after all its critical infrastructure studies in the last couple of years. Sorry, I just had to vent. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

2 1

Re: The future of NAPs & IXPs
by William B. Norton 19 Apr '99

19 Apr '99

Ed- Sorry for the delay in posting a response; there were issues with the nanog-post list. Who created that thing anyway ;-) At 03:09 PM 4/15/99 -0400, etignor(a)rhsmith.umd.edu wrote: > >I would like to know what net professionals think about the future of NAPs & >IXPs. For what it's worth...I just finished a paper that highlights the trade offs between the direct circuit interconnect model and the exchange point interconnection model for ISPs. The paper discusses the operations and financial models (taking into account the circuit costs, cost of exchange participation, cost of dark fiber, etc.) and the implications of these strategies across the # of interconnection participants and bandwidth utilization between the participants. To cut to the chase, the major points from the paper: 1) For ISP interconnection, direct circuit interconnection is financially attractive for low #s of connections (O(5)) of relatively low bandwidth (DS-3/OC-3). This is due to the fact that ISPs typically pay only half of the cost for the direct circuits between each other while they pay the full freight for the big pipe into the exchange. ISPs may want to use the exchange for a POP, but barring use like that, if all you want to do is exchange traffic with these other 4 folks into the foreseeable future, the direct circuit interconnect model makes some sense. 2) As the bandwidth and # of interconnections grow, the exchange point interconnection model proves much more scalable for two reasons: First, as bandwidth grows between participants, ISPs are able to aggregate interconnection traffic over increasingly large pipe back to their cloud, yielding potentially significant economies of scale. The direct circuit interconnection does not provide for this aggregation since the pipes are destined to different plances. Secondly, operationally, fewer backhoes in an exchange, fewer local loop providers to troubleshoot, etc. The greater the dependence on interconnection, the more hardened you want that interconnection environment. 3) Several of the exchanges also provide for the centralization of content allowing additional transit sales revenue that potentially dwarf the cost savings highlighted above for interconnection. The direct circuit interconnect model doesn't allow for this additional revenue opportunity. 4) The analysis finds that facilities-based ISPs win big since (being able to exploit WDM technologies) they can seamlessly grow the pipe(s) into the exchange while the direct circuit interconnection model can't take advantage of this level of aggregation. IMHO, these are the most compelling reasons the IXP will be around for some time. I can make the white paper with the plots and data available to you if you like. >Given UUNET's inclination to peer only with national backbone providers, what is >the future of settlement free peering arrangements? > >Will non-profit, cooperative organizations that require multilateral peering to >participate (LINX) survive or thrive going forward, or will we see more for >profit NAPs run by Sprint and co. run the non-profits out of business? There are some scale issues that come into play that non-profits have to grapple with. For example, the resources required to build and operate a commercial grade operations environment from scratch (building close to fiber, HVAC, Generators, UPSes, Power, Security Systems, 24/7 Staffing, etc.) with sufficient redundancy excludes most non-profits from at least the high end of the market. Part of the challenge is achieving the critical mass to take advantage of economies of scale to compete. Another issue is, even if a non-profit could, what would be the motivation to expend the capital? Is it in their charter for a non-profit organization to do so, and why would that non-profit organization want to compete against a for-profit doing the same thing? In the US, IRS regulatory issues pop up too, and the tax exempt status may be challenged if too much revenue is generated in this new line of the business. >Will co-operative IXPs proliferate in regional markets? Will this give small >ISPs, acting collectively, more leverage with the larger carriers (Exodus and >GTE )? > >I know these aren't truly operational question, but peering and transit is an >issue that will have an impact on the international infrastructure and may >result in increased regulation in the future. Are there other lists where >people might have some thoughts about these issues? The answers to these two are too long (and contentious) to answer here. Since you're in Maryland anyway, I am chairing a panel (with Dave Diaz, Bill Manning, and Dave Rand) in a few weeks at ISPCON in Baltimore that will discuss several of these topics. We'll be focusing on definitions and highlighting experiences in the field. This might be of interest to you and we can talk off-line there if you want. Bill ---------------------------------------------------------------- William B. Norton <wbn(a)equinix.com> +1 650.298.0400 x2225 Equinix Director of Business Development

2 1

Call before you dig
by Sean Donelan 18 Apr '99

18 Apr '99

woody(a)zocalo.NET (Bill Woodcock) writes: >Okay, so basically the deal was that there were a zillion (like 200) >different "call before you dig" hotlines for different areas, and you had >to track down the correct one, and there weren't a lot of penalties for >not having one for an area, et cetera. Then in 1997, there was a federal >house bill to unify it all, it carried over to 1998, and then actually >passed around May or June sometime, as part of the Omnibus Federal >Transportation act. The upshot of that is that there's one 888 number for >the whole country now, and states lose a portion of their federal highway >funds if they don't get the utilities which their PUCs regulate to >participate in the program. Its still a bit early to tell if the One-Call legislation made any difference. Only two quarters of information has been reported, and one quarter was down, and the following quarter was up. However, there are still a lot of cases where the excavator did call, but the utility either didn't mark, marked the wrong place, or didn't mark all the lines. As Bill points out, the Department of Transportation is the federal agency in charge of One-Call. States can get a $1M-$5M federal grant for setting up a One-Call coordinator for their state. But there is no national GIS database, nor any truely common way of describing where everything is. After getting a call, the One-Call people will forward the information to the listed contact for all registered utilities in a particular geographic grid. It is then up to each of those contacts to go out and mark their clients' lines. Sometimes a group of utilties will hire a common contractor to mark all of their lines, other utilties send their own people. Sometimes a utiity will forget they had lines in a particular area, and fail to register that grid with the One-Call people. Some companies consider their maps proprietary, and won't allow anyone else to see or use them. Other company maps are just plain wrong. With deregulation, there is no longer just "the phone company" or just "the water company." Multiple companies may have the same type of lines in an area. The marking contractor for one company won't necessarily mark the lines for a different company (or even a different subsidary of the same company), or even alert the excavator he know's there is another company's lines he didn't mark. Its kinda of like another ISP trying to get the SPRINT NOC to open a trouble ticket. There is a bit of tit-for-tat going on. Even after you have all the colors marked on the ground, you are never really sure just what's missing until you start digging. National Call Before You Dig = 888-258-0808 -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

2 1