March 1998 - Test - lists.nanog.org

NANOG 13 INFO
by Pam Ciesla 23 Mar '98

23 Mar '98

The room block for the Hyatt Regency Dearborn will be available on Wednesday. Please mention the NANOG Meeting when reserving rooms. The NANOG rate is $125 per night. The direct phone number is 313-593-1234, or 800-233-1234, the fax number is 313-593-3366. If you have any problems reserving a room please contact me. Oh by the way, once you get passed Detroit Metro Airport :*) Dearborn has much to offer. There is the internationally famous Greenfield Village and Henry Ford Museum, plenty of good local restaurants, Detroit Greek Town (about 10 miles) and the Casino's in Windsor, Canada (about 20 minutes). Hope to see you there. xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Pam Ciesla 734-936-0172 - phone Merit Network, Inc. 734-647-3185 - fax University of Michigan 4251 Plymouth Road Bldg. I, Suite C Ann Arbor, MI 48105-2785 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

1 0

Re: What do we do with clueless ISPs (fwd)
by John Bradshaw 23 Mar '98

23 Mar '98

I would be interested in knowing who was spoken to at UUNET because I have two on-call engineers who are very clueful on the smurf attack and how to block it. There are issues being discussed at this time with the dostrack program and current revs of Cisco code beign distributed so tracking would not have been feasible (esp. since the site as the target and not the amplification site). Only if an amplification site was ID'd as a UUNET customer could a track to the perpetrator hav been performed. UUNET Security Engineers will alleviate the attack until it subsides or identification (and subsequent halting of the attack) occurs. The staff is not manned 24 hours (just like MCI) but my people are paged when these type of events occur (just like MCI). Hopefully, someone can provide me with the ticket # of the incident of the engineer who handled it. John > Joe Shaw said: > > From mail-maint(a)UU.NET Sun Mar 22 14:10:32 1998 > > Date: Sun, 22 Mar 1998 12:47:52 -0600 (CST) > > From: Joe Shaw <jshaw(a)insync.net> > > To: Randy Bush <rbush(a)bainbridge.verio.net> > > cc: Hank Nussbacher <hank(a)ibm.net.il>, nanog(a)merit.edu > > Subject: Re: What do we do with clueless ISPs > > In-Reply-To: <13588.7105.10059.574316(a)rip.psg.com> > > Message-ID: <Pine.SOL.3.96.980322122920.21775A-100000(a)vellocet.insync.net> > > MIME-Version: 1.0 > > Content-Type: TEXT/PLAIN; charset=US-ASCII > > Sender: owner-nanog(a)merit.edu > > > > > > Last weekend we had one host on our network as the target of a smurf > > attack. When I reported it to both our upstreams (UUNet and Time Warner > > who reported it to MCI), we got two stories. MCI, whom I'm not even a > > direct customer of started tracking the attack as soon as they were > > informed. UUNet took an hour to get a security person on the phone who > > then told me that there was nothing they could do, period. > > > > My question is this: When will UUNet have security types on duty 7 days a > > week, and will said people be clueful enough to track this sort of thing > > down? I told the people at UUNet that we were under smurf attack, and > > then I had to go through a 10 minute explanation of what a smurf attack > > was and what it was doing. I would expect a worldwide NSP to keep up with > > things like this, especially when a regional like myself can. > > > > I had logged all ICMP traffic coming into our network via an access list, > > and could give them all the information they needed to get to the > > offending networks, so it's not like they had such a hard job ahead of > > them. > > > > Joe Shaw - jshaw(a)insync.net > > NetAdmin - Insync Internet Services > > > > On Sat, 21 Mar 1998, Randy Bush wrote: > > > > > > How does one send "samples" of a Smurf > > > > > > When BBN's NOC handed one to our NOC yesterday, or was it the day before, > > > they sent a cut and paste of > > > o configuring their edge cisco to detect and log > > > o the log > > > which both documented the problem and, if our NOC did not have smurf clue, > > > gave a clue on how to track. > > > > > > [ aside: it was tracked to the perp and stomped ] > > > > > > randy > > > > > > -- ===================================================================================== John Bradshaw, Manager, Security & Fraud Support (800)900-0241 (F) (703)645-4424 UUNET Technologies, Inc., 3060 Williams Dr. Fairfax, VA 22031; http://www.uu.net security(a)uu.net - Security Incidents; fraud(a)uu.net - Massmail, spam-complaint(a)uu.net =====================================================================================

1 0

How to repair fiber cuts...
by Alex Bligh 23 Mar '98

23 Mar '98

Being a European, I was wondering if some enlightened US person could give me some operational info on how fiber cuts are repaired in the US. Specifically what I am interested in those caused by train derailments. As I understand it in the US a lot of fiber runs alongside tracks, or is buried not very deep. Presumably when a train is derailed, it can be going at a reasonable speed, and is likely to (a) shred a few hundred meters of fiber, and (b) possibly fall on top of it making it difficult to physically access. Also one presumes that the affected area is then covered in cranes, winches, emergency services etc. who are likely to cause more damage. I am therefore presuming that what happens is someone splices around the cut with a (reasonably long) piece of fiber that they keep for this sort of eventuality, and later when things have calmed down the fiber is respliced to go along its original path. Is this correct? So: 1. How long after such a disastrous cut occurs is it normal to have connectivity back? 2. Do operators in general keep spare fiber? If so, how do they get it there? By road or by rail (both might be difficult)? 3. How long does the splicing operation itself take? 4. Can the resplice in general be done without much service interruption? It's (1) I'm most interested in - is the cut normally repaired (for instance) within 24 hours? In case you were wondering, the reason I'm asking this is because I have a theory I want to check out, which is that being spoilt with SDH over here, the operators have less (or indeed no) proper disaster plan for fiber cuts (varies operator to operator), and thus they get really caught out either when there are two cuts, when their SDH doesn't do the right thing, or when they forget that certain things aren't SDH. (Yes Peter, you know who I mean). -- Alex Bligh GX Networks (formerly Xara Networks)

2 1

Re: Building a NOC
by Howard C. Berkowitz 23 Mar '98

23 Mar '98

>Floor drains are good. But once again, you need to have a chat with >your designer & inspector about what, why, when. Anti-backflow valves, >and floor sink traps are mandatory. Automatic trap primers seem to lead >to flooding. For example, St. Louis County recently opened a new jail. >However it was delayed three months because the week before opening when >they discovered if all the toilets were flushed at the same time on the >eighth floor, as prisoners are known to do, the sinks exploded on the >bottom floor. The plumbing designer had followed normal industry practice >for commercial buildings, but forgot jails house a user population that >is different from a normal commercial building. Hmmmm...reminiscent of a NANOG meeting? > >Like most disasters, you may not be able to prevent flooding, but you >can mitigate some of its affects. Hit the E.P.O. and unfurl the plastic >tarps you have stored & handy for just this emergency. Good point. After one flood from upper-story firefighting, the US Senate Computer Center did something along these lines I haven't seen elsewhere but wasn't a bad idea. They got some large rolls of plastic, and mounted the rolls on bars suspended from the ceiling -- sort of like giant toilet paper rolls. The leading edges of the rolls were folded over a strip of wood, and a rope tied to that. In the event of a fire/flood alarm, the operators could grab a handle at the end of the rope, and pull the plastic sheet over the top of equipment in under a minute. Far faster than unfolding tarps. > >>Putting equipment on high, rather than low, floors can protect it. > >Pick your disaster scenario, low-flying airplanes or high-flowing water. How about a lost Forest Service water bomber? No statistics available, but I would suspect water flowing down is far more common than airplanes flowing down. >>I wonder if any of the TEMPEST manuals have been declassified? THey never >>were at a terribly high level, and had some very nice general engineering >>in them. > >Not yet. Although I've been trying to figure out how to buy the old >MCAUTO (McDonnell-Douglas Automation) Data Center and turn it in to the >Mid-West high-tech co-location space. In addition to a three-story, >multi-football field long data center, and a backup power plant a small >city would envy, it is rumored to have so-called black-project rooms with >TEMPEST/EMP/EMF/RFI/TLA... shielding. InterNIC and PSI could co-locate >in the same building, and not run into each other even in the parking >lot. Is there a place that would be good for Fleming? >I've only seen one cafeterria in the building, so that could be >a problem :-). This may be the same data center that Bob Courtney, the retired security chief for IBM, talked about when he told the story of the Rustproof 360/195 in A Large Unnamed Aerospace Company in St. Louis. Apparently, they had a 195, a supercomputer for the time, in a magnificent computer room. Fire protection up the wazoo. Every kind of power and grounding anyone could think of. What no one did think of was that there were two paint shops in the same building, and the paint tanks were in one of them, with piping to the other. Guess what the overhead pipes ran over. Late one night, a pipe blew and dumped several hundred gallons of zinc chromate primer into the CPU cabinet. The resulting hardened mass gave new meaning to the term, "solid state." Moral: check for hazardous materials elsewhere in the building, and materials that might not be hazardous to people but will do the nasty to equipment.

2 1

Re: IP over SONET considered harmful?
by Sean M. Doran 22 Mar '98

22 Mar '98

| There's LQM in PPP. I would prefer to eliminate PPP. | Better yet, | what i do with splicing traffic into many channels effectively | provides IP-invisible restoration w/o loss of useful capacity. | Just route those channels into diverse physical paths :) Um, unfortunately I missed your presentation at NANOG and there is as yet no sign of the real video archives that I can find, so going by memory and your web pages, what I remember is that you are planning on using interfaces roughly comparable to STS-1 and STS-3c, and using telco WAN gear to do the work of add/drop multiplexing. I thought this was rather clever, actually, particularly given the accuracy of the prediction that non-facilities-owners would have trouble getting access to anything faster than DS3s in the near run. If you have changed your mind and are building your own fibre muxes, that would be neat to know. :) | > SDH/SONET is also a really keen way of sharing a network | > among IP and other things on the wide-area side, and | > an even keener way of having access to relatively low-speed | > customer data on the more local side. | | On customer-access side even ATM seems to be fine :) It | definitely is a huge lot cheaper than SONET gear. It is pretty easy to pluck out VCs/SPEs from SDH/SONET. The general thought was to have the equivalent of a CT3 (cisco device, takes T3 in on BNC pair, pulls out DS1s) that can extract from, say, an OC3 or OC12, DS3s, SPEs containing T1s and E1s, and perhaps even individual DS0s. (DS0s would be neat as with clever load-balancing that gives you a granularity that is competitive with the use of ATM as a fine-grained TDM system). Essentially, an ADM on a card. I don't see that as being prohibitively expensive to engineer, and ironically I think it would be easier for you than for some of your competitors to make switching work, since they may be forced to simulate a hierarchy of routers on the card. | Ciena folks told me some other compelling reasons for dropping | SONET, which i do not feel i'm at liberty to discuss here. Interesting. Aim them at me or Peter if it doesn't have to do with SONET's speed ceilings. | I also expect it :) My current favourite for framing is | 32-to-33 bit encoding, with flag being one of "malformed words", | one word header (2 bytes for payload length, 2 bytes for tag), and | 32-bit CRC at the end. I.e. the per-packet overhead is 12 bytes + 3.1% + | rounding to 4 bytes for odd-sized packets. Make all 1s and all 0s | and chess patterns to be invalid words, and loss-of-carrier becomes | easy to detect. Well, I still prefer the idea of a synchronous byte stream to a clever HDLC on steroids... :) Sean.

2 1

Re: IP over SONET considered harmful?
by Mike Trest (In Japan) 22 Mar '98

22 Mar '98

At 06:32 AM 3/21/98 -0800, Sean M. Doran wrote: >| Native IP over fiber is not a religion; it's a solid engineering. >snip< >SDH/SONET is also a really keen way of sharing a network >among IP and other things on the wide-area side, and >an even keener way of having access to relatively low-speed >customer data on the more local side. Both in US and In Japan, SONET/SDH systems are used to deliver such non-IP traffic as CATV/Video over the same physical facilities as IP traffic. Indeed, I can walk you into a top-rated commercial television station using a large Unix box with six ATM-over-SONET interfaces to drive video broadcast streams to the stations' live broadcast to millions of viewers. The same interfaces also transport TCP-IP application control data between Unix and the various off-line editing systems and the graphic development engines (MAC/W'95). You should not think that the prospect of internet based viewers are outside the planning horizion of commercial broadcast industry, even if that industry consists of predominately time critical (non-IP traffic) today. Some major players in this industry agree that the mixture of SONET/SDH with both IP and ATM traffic is "really keen". ..mike..

2 1

Re: Building a NOC
by Sean Donelan 22 Mar '98

22 Mar '98

>Yes. And your consultant must know local practices as well. If you are >building a facility in Fairfax County, Virginia (a major technology suburb >of Washington DC), avoid, in your plans or anything the building inspector >may see, the term "computer room." "Communications room," "network room," >etc., all prevent the problem: if they see "computer room," they will >demand a mainframe-style central emergency power off control, which greatly >increases electrical wiring cost. The psychology of dealing with building inspectors would fill an entire book, however this is a good example of understanding the "Why." If building inspectors and computer room designers were logical, they would realize why this requirement exists. They would also realize when the requirement didn't apply. The reason why the requirement exists is computer room wiring often doesn't meet all the safety and fire retardation requirements of normal occupancy space because the entire computer room is treated as if it was a restricted access area and fire retarding space. Hence the requirements for a seperate HVAC system, fire-rated walls, and a single disconnection means (i.e. EPO) for the entire room INCLUDING the UPS(s). On the other hand, if you followed normal occupancy wiring practices, one would expect the space could be treated as normal occupancy space by the building inspector. I know, that's not how inspectors think. I'm not sure eliminating the EPO would really be cheaper. Its not simply a matter of eliminating the EPO, but also replacing much of the raised floor cabling with conduit, plenum rated cable, and other changes. Not just power cables, but also communications cables, such as the cable connecting a disk cabinet to the CPU cabinet. And worse, from the building inspector's point of view, having some assurance someone won't install some cable next week without having it inspected. I realize in most offices there are more computers on people's desktops than in the computer room. At this point I get to critize designers for how they think. Electronic equipment is spread throughout the organization, and the infrastructure to support it can't be limited just to the computer room. But the building codes are always going to be playing catchup, so designers and inspectors have their negotiating work cut out for them. >Wasn't thinking of MILSTDs as much as some of the DISA documents, and often >relating to base practices. The DoD manual on grounding, bonding, and >shielding is excellent. This was precisely the one I was thinking about. Since the IEEE Emerald book was published in 1992, gounding, bounding and shielding practices for electronic equipment have been getting updated. There is a LOT of techno-babble floating around about computer room grounding (pun intended). Make sure you have the updated manual. There are some old ideas about practices such as isolated grounds for electronic equipment which are dangerous, and have been revised/clarified in the updates. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

2 1

Building a NOC
by John_W_Connors＠res.raytheon.com 22 Mar '98

22 Mar '98

Would anyone have information or knowledge of books, WWW Sites, or other means that would provide details on constructing a state of the art NOC to support 75000+ nodes in a LAN/MAN/WAN environment Thanks in advance JC

9 8

Re: IP over SONET considered harmful?
by Sean M. Doran 22 Mar '98

22 Mar '98

| Sounds reasonable to me. My point was that decrementing the IP TTL, | whether it be at each LSR or only at egress, should be a decision | that can be made by the provider. Well, in that case you also have to not copy the IP TTL into the tag TTL, but rather synthesize a new tag TTL, assuming the goal is not to end up with traceroute stars. I think this is insane, personally, but then Sprintlink in the past fell victim to devastating forwarding loops, both tight and loose; the latter were particularly bad, and so maybe I am "over sensitive". If you _really_ want to conceal one's network, why not just outright lie -- at your ingress LSR look for packets with low TTLs, and for ones that won't make it to their destinations, synthesize an ICMP time exceeded with your choice of source addresses. (cf the last paragraph of 2.14 in the current MPLS arch draft) Where this is not possible, you generate the lie as close as possible to the point where the ttl timeout would happen. Sean.

1 0

Re: Building a NOC
by Sean Donelan 22 Mar '98

22 Mar '98

Both good ideas. But be sure to learn and understand when, where and why the operational experiences given in these sources apply to building a NOC. One difficulty in designing a NOC for an ISP is they operate across several different problem domains, telecommunications, data processing, lan, wan, etc. Be careful if you hire a consultant with expertise in one particular area. NEBS and the like have several good ideas, but they also have a built-in set of assumptions. If you, or your consultant, don't understand when the assumptions apply and don't apply, when you apply the solutions to a different problem domain, you can actually make things worse. For example, the National Electrical Code doesn't apply to communications equipment located inside a communication utility building. But the building inspector will get very grouchy if an ISP's electrical wiring doesn't meet NEC standards. Also, there have been changes in 'standard' practices in the last 5 years. If your sources for designing data processing centers were written prior to 1990, I would look for a second source. The military has been updating its standards to reflect revised commercial practices, but the civilian sellers of the MILSTDs don't always show when an old practice has been superceded. My basic problem with the original question is I don't really understand what kind of NOC the person is trying to build. Are we talking about how build a NOC staff, or a NOC building to monitor a network of 70,000 nodes? Are we talking about building a NOC theatre so management/customers can view CNN on the projection TV wall when they visit, or a lights-out server/equipment room designed for limited human occupation? Are we trying to manage 70,000 remote systems, or just the core network connecting the sites. >Excellent observation. There was a text, _Engineering and Operations in >the Bell System_, that gave some good operational experience. With >divestiture, I'm not sure who took it over. I vaguely remember that >_Transmission Systems for Communications_ went to AT&T Bell Labs, and E&O >went to Bellcore. > >I'd also consider looking through some of the military communications >manuals. Unfortunately, I'm in the midst of a computer conversion and have >to recover some bookmark files, but you can search for Army Field Manuals >online -- they are at the TRADOC site at Ft. Monroe. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

4 3