- Test - lists.nanog.org

RE: Building a NOC
by Bill Unsworth 26 Mar '98

26 Mar '98

At 08:52 AM 3/23/98 -0500, you wrote: >> Pocket protectors? Slide rules? Any other items nerd/engineers carry >> in their shirt pocket that falls out when they are head-first into the >> raised floor. > >We've found socks and cigarette butts. Needless to say the perp is no >longer with us :) > >> Like most disasters, you may not be able to prevent flooding, but you >> can mitigate some of its affects. Hit the E.P.O. and unfurl the plastic >> tarps you have stored & handy for just this emergency. > >We also put drip pans which cover the entire ceiling over the data center - >we were burnt 2 years ago in a pre-war office building in NYC. (avoid space >with wc's above!) The pans drain outside of the machine room into a >high-traffic hallway (with a drain). Any drips get noticed very quickly, >and we don't have to test the water alarm. > I guess that flooding by the mains water is seriousl worth thinking about. For example we just realised that because the "mens room" (and the "ladies) lavatory cistern overflows go into the room (they are in the middle of the building) if something goes wrong we have a little flood. So any mains water installations (dishwashers, kitchens, showers) are a potential hazzard - solution a) install mains water cutoff valves with sensors in the floors b) get everyone not involved in the NOC out of the building. We are just doing both! I suppose that all the wiring could be in the ceiling although that would not help in multistorey building. Rambling a bit but what about the film Inferno with all that water stored at the top of the building. :-)) Bill > -- U-NET Ltd "Just Internet" mailto:bill@u-net.net http://www.u-net.net/ Tel 01925 484444 Fax 01925 484466

6 6

Re: Building a NOC
by Sean Donelan 25 Mar '98

25 Mar '98

>Some urban archaeologist is going to take up a research project that >investigates why certain artifacts predominate under raised floors. In Pocket protectors? Slide rules? Any other items nerd/engineers carry in their shirt pocket that falls out when they are head-first into the raised floor. >Do think of underfloor drains. They are your friend. In close to 30 years >of working with computer rooms, with all the emphasis on fire protection, >I've never had a computer room fire. I have, however, had several major >floods related to fire protection -- a couple from water running downhill >from a fire fought on a higher floor, two from burst sprinkler lines >(improperly protected from freezing), and more from ordinary plumbing >emergencies such as clogged and overflowing toilets. Floor drains are good. But once again, you need to have a chat with your designer & inspector about what, why, when. Anti-backflow valves, and floor sink traps are mandatory. Automatic trap primers seem to lead to flooding. For example, St. Louis County recently opened a new jail. However it was delayed three months because the week before opening when they discovered if all the toilets were flushed at the same time on the eighth floor, as prisoners are known to do, the sinks exploded on the bottom floor. The plumbing designer had followed normal industry practice for commercial buildings, but forgot jails house a user population that is different from a normal commercial building. Like most disasters, you may not be able to prevent flooding, but you can mitigate some of its affects. Hit the E.P.O. and unfurl the plastic tarps you have stored & handy for just this emergency. >Putting equipment on high, rather than low, floors can protect it. Pick your disaster scenario, low-flying airplanes or high-flowing water. >Another local factor that MUST be considered in certain cities, and can be >far worse to deal with than building inspectors, is the local electrical >union. Some are great craftsmen and incredibly helpful. But there are >also cities where a union may perceive the data people running low-voltage >circuits as under their jurisdiction...and cables have been known to be >cut. Some of my New York friends have worked out very specific personal >procedures to get on good terms with the shop steward Local 3 of the IBEW >before starting a specific project. Competent union electricians are a joy to work with. Incompetent craftsmen are not. Incompetent union craftsmen are [expletive deleted]. Just because the NEC includes Article 800, doesn't mean the typical electrician has a clue how to properly install communication wiring. On the other hand, I've found the same thing to be true of a lot of data people. A fun factoid, in 100% of the power audits conducted by American Power Conversion (http://www.apcc.com/) they found violations of the NEC. Admitly this is a self-selected sample. People don't normally have a power quality audit done, unless they are having power problems. But it does show using union electricians, and passing a building inspection doesn't always mean a top-quality job. >I wonder if any of the TEMPEST manuals have been declassified? THey never >were at a terribly high level, and had some very nice general engineering >in them. Not yet. Although I've been trying to figure out how to buy the old MCAUTO (McDonnell-Douglas Automation) Data Center and turn it in to the Mid-West high-tech co-location space. In addition to a three-story, multi-football field long data center, and a backup power plant a small city would envy, it is rumored to have so-called black-project rooms with TEMPEST/EMP/EMF/RFI/TLA... shielding. InterNIC and PSI could co-locate in the same building, and not run into each other even in the parking lot. I've only seen one cafeterria in the building, so that could be a problem :-). -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

3 2

Internet Diameter and TTL decrementing Followup
by alan＠isi.net 25 Mar '98

25 Mar '98

I took the liberty of summarizing interesting comments by folks on this latest thread "IP over SONET considered Harmful?" My comments and questions are inline with these notes. > yakov - look at putting knob support in or get concensus Yes, this is indeed the answer. I'd like to see the MPLS group dictate that the MPLS standard require the ability for the user to specify MPLS and IP TTL interaction. I'll go away and leave you to your other conversations when MPLS people and the big MPLS vendors say they will do this in this forum. > bill@canarie - doing physical mesh Too expensive. We can afford it but it just isn't an optimal solution. WDM switching is WAAAY off. > havard - ttls under 64 are broken (per rfc) Slow automobile drivers are bad. Yet they exist. And must be considered. > jmalcolm - atm+fr nspnets hide l2 paths already, non mpls ttl > decrementing@l2/l3 hops desired from vendors Agreed. Eloquently stated as always. > pee@fro - if you want l2/mpls tracing, build it w/out ttl use Would have been nice if those smart MPLS people had thought of that, wouldn't it? > l3 ttl nullification too complex and scarey one idea would be to remove the TTL decrementation (is that a word?) from certain router interfaces. However, why build pits to fall into when we can change the path, hopefully. > pferguso@cisco - mpls ttl discussed ad nauseum, knob suggested, No concensus on this should equal no standard. I truely think it's that big of a deal. > atm+fr nspnets may prefer to hide l2 hops > - internet wider than in nsfnet days > - ss7 not involved in this problem (re: hannigan@xcom) > - leave decision about ttl decrementing to provider > nshen@mci - nspnets can hide l2 or l3 if they choose, maybe put > path tracing into cos bits or elsewhere than ttl exp. Ideas on how to do path tracing (to make this ttl for path viewing argument go away) would be appreciated. > kwe@geo - world didn't end when nsfnet had 2 ttl decrements per hop. > back in 1994. a long time ago. much smaller > internet (sorry for editorializing) -- harmful not > required Doesn't address this problem. Arguing by analogy is not helpful. > scudder - ttl complaints are product of low moral evolution I really like him. But I don't agree. > smd - people who don't want ip ttl decremented at each l3 hop must > not be concerned about loops Not so concerned with loops as usability. Loops are easier to fix and prevent than windows 95 ttls. > - all ISPs should display full frontal nudity and show all > internal toplogy ISPs will hide things. However, having once been a big ISP serf, I really don't think the ISP gives a whoot if you see their topology. By that I mean they won't actively work to prevent you from finding out. Certainly they won't actively work to help you understand anything, but it's really not a conspiracy. More of a financial decision (do I spend money==time on showing them what's under my kimono or do I make things better so my customers like me so I make more money for my shareholders?). > - use different ICMP for ttl unreachables at l3 and mpls Would be cool. How? Where in the packets and standards do we stuff this idea? > - okay to give isps tools to hurt themselves if they > are aware of risk > - wants vadim to send ciena folks to him and lothberg > about stuff Thanks for this broadcast. Maybe if you find a good restaurant in SFO you can let me know. And also tell pee(a)frontiernet.net too. > - same email as last 2 years about moving ip into telco gear Good message to hammer home. However, it would be cool to see IP subsume ATM stuffs. And see apps and consumer products accelerate towards IP. Then TDM stuff naturally is optimized for IP more quickly. But the routers can handle it for the next 6-18 months. > - really scared of loops because they bit him at sl/1239 Haven't seen loops as a terrible problem. May be a difference in engineering style. > - cost of ttl-able path important in ttl-able decision good point; sort of supports the user-configurable knob. > vadim - ng ip boxes obviate atm Not for a while. I think at NANOG you said we couldn't order your product yet? But I still have that ticket for that plane to mars... > - shameless plug for pluris causing clustering antiquity > - NSPnet hub number decreasing Don't really buy this; physical space requirements and constraints demand more ubiquitous bw and geographic indifference. Whereas in the past people would consolidate and backhaul in few hubs, now folks are succesful and run out of space/power/foundation support. Maybe someday, but not for 2-3 years, longer out than the next wave of ip buildouts. > - wdm vendors don't like sonet Bet they don't tell Telco PL folks that :-) > - ip over fiber solid engineering Agreed. Esp. with this nifty SONET on Router stuffs. > - inet eng. harder than telco eng. as utilization is higher > in inet eng. Agreed. Can't hammer this home hard enough. > hannigan@xcom - ss7 bypass (???) > stan@netmerc - atm not dead, still needs for cem and tdm stuffs > swb@newbridge - in '94 people had to patch unix hosts to raise ttl > of ip packets Right. My fear is we build all these nifty SOIP (sonet over IP?) networks and noone shows up for the party, opting instead for expensive clunky IP over ATM providers because their TTLs are shiney and new. (obvious disclaimer - IP over ATM providers means IPonly networks that use ATM as a closed system transport mechanism for IP. I think the gang that thinks CUGs and NNI NSAP transport are growing will be shocked at the velocity of IP growth) > - traceroute was hack > - not everything in physical path must respond to trt So, all in all, we seem to have an implied concensus that: * The width of the Internet poses a problem for 'regular' IP traffic and conventions with an increase in IP hops. * NSP Network Providers should have the ability to unlink MPLS and IP TTls * IP is growing So, if we want this stuff to be useful (and prevent a cidr crisis) please help influence wgs to solve this problem in advance. Only you can help prevent TTL crises. -a

1 0

RE: Building a NOC
by Andrew Metcalf 24 Mar '98

24 Mar '98

On Saturday, March 21, 1998 2:31 PM, Howard C. Berkowitz [SMTP:hcb@clark.net] wrote: > At 14:51 -0500 3/16/98, Charles Sprickman wrote: > >I think I'd disagree with that. Where would all you folks be if you never > >picked up a book? You wouldn't be reading this list. Seriously, while > >there's not an ORA "How to Build a NOC" book with some cute animal on the > >cover, there is some information out there worth reading. <>snip<> > Excellent observation. There was a text, _Engineering and Operations in > the Bell System_, that gave some good operational experience. With > divestiture, I'm not sure who took it over. I vaguely remember that > _Transmission Systems for Communications_ went to AT&T Bell Labs, and E&O > went to Bellcore. The Bellcore information superstore has a lot of planning titles: http://bigbird.bellcore.com/ido/ andrew It is in the Reeds nature to bend, it is in the Oaks nature not to bend.

2 1

Re: IP over SONET considered harmful?
by Sean M. Doran 24 Mar '98

24 Mar '98

| A more flexible 'knob' would be cool, something like combining the 'cos' | bits in the labels with the decision to decremening or not. This will give | the management station the power to 'trace' even the regular trace packet | can't do. No, no, no, no, no. You are confusing TTL expiration with "visibility". You have a trade off between size of forwarding loop and TTL decrement. The longer it takes to decrement a TTL to 0 in a forwarding loop, the worse the loop's effects will be. What you want -- and I think you are very strange for wanting it, by the way, but I will supply you a solution anyway -- is a knob that modifies the ICMP ttl exceeded message that is sent back to the source. Sean.

1 0

Digex
by Rick Payne 24 Mar '98

24 Mar '98

Could someone clueful from Digex's NOC please contact me? Thanks, Rick -- Rick Payne, Senior Network Admin | Taxation is about extracting the NETCOM Internet Ltd. | maximum amount of milk with the rickp(a)corp.netcom.net.uk | minimum of moo.

1 0

Re: Internic hosage (fwd)
by Joe Provo - Network Architect 24 Mar '98

24 Mar '98

[Disclaim, disclaim: association listed for identifiaction, not endorsement, etc etc.] Dean Anderson <dean(a)av8.com> took a long pull from his crack pipe and wrote: [snip] > Ah. Only girls with lowcut [...] [snip] If your point is that periodicly lusers and abusers will have similar thoguhts or profess similar motivations to others that aren't lusers, then congratulations. You just discovered that no-one "owns" an idea or position. Even lunatics can believe in gravity; does that make it less valid? Does the presence of Jim Fleming (or you, his un-funhouse mirror) on this list make this list invalid? [snip] > You're smarter than that. I think. Besides, I react badly to coercion. So > do most people. Thats why we need laws. Sensible ones at that. Funny, but the enforcement of laws are based on ...coercion. I think you need to re-visit your dictionary. I'm not going to coerce you in any way shape or form when I plonk you into my nanog-procmail filter after this letter. Because I choose not to hear you on nanog makes me somehow abusive? Anyway, social workings behind law enforcement and the rationale for anarchism are on the dance card here, so... > Anyway, I don't think it is just spammers. NOC's don't put terribly good > information in whois because general public lusers will call them, instead > of the help desk. Most companies don't list off-hours numbers. etc. The presence of bad data is not justification for further bad data. To remove spam from the argumnet, since it appears to cause you to lose all ability to reason, let's use BGP. Anyone doing BGP should have 24x7 numbers available; if your peers don't, request they do so by (mumble date) or you'll have to cut them off. If someone can't handle the calls they might generate due to error or whatever, they shouldn't be in the deep end of the pool. Similarly with folks dumping postmaster to /dev/null. There are simple, sane things to do that are responsible. They are codified into various RFCs so that guides exist for newbies and folks wanting to do Right have something to point at when talking to the Suits. I am curious at you WAG that "Most companies don't list off-hours numbers. etc."; how long were you workinmg at the InterNIC grinding the database to determine this? > I don't think I protest enough. Spam actually runs only 2% of email volume. Again, with the stats of no consequence. 2% of _whose_ mail volume? How measured? etc etc... It is a hell of a lot more than 2% in my neck of the woods. > On the other hand, maybe I have been a bit hasty, and haven't considered > every option. Vix, will you put porn sites into the RBL? I'll tell you [snip] Bzzt - content is irrelevant. The RBL is behavior modification for folks with broken behavior or broken configurations. It works; I know since I had a server getting abused, no resources allocated for what I knew needed to be done, and I got RBLd. Suddenly, I had the resources to complete POP-before-SMTP and anti-relaying within the time it took me to explain what went on. It was also a suprise due to ...bad WHOIS data! Wow, I feel like James Burke; here we are back at the start of the conversation. Goodbye Dean, stand alongside Fleming in my ruleset. Learn how to be funny from him, ok?

1 0

What do we do with clueless ISPs
by Hank Nussbacher 23 Mar '98

23 Mar '98

I opened a trouble ticket with a major ISP in the USA about a Smurf originating at one of their customer nets targeted at me. I gave them the URL about Smurfing just in case they never heard about it. This is the email I got after 8 hours from their NOC: >***Note #1 03-20-1998 08:51:05 GMT Author: arsmith > > More info. requested > > Please send any samples along with the following info ASAP... > 1. Is it presently still going on ? > 2. Was your site used as a launching point for attacks on other entities ? How does one send "samples" of a Smurf (I gave them the originating IPs, destination IP, time, frequency, pkt size, etc.)? If this is how major ISPs handle Smurf storms, we can expect much more congestion. It is as if the ISP NOCs assume Smurf is Spam! -Hank

8 7

NANOG 13 INFO
by Pam Ciesla 23 Mar '98

23 Mar '98

The room block for the Hyatt Regency Dearborn will be available on Wednesday. Please mention the NANOG Meeting when reserving rooms. The NANOG rate is $125 per night. The direct phone number is 313-593-1234, or 800-233-1234, the fax number is 313-593-3366. If you have any problems reserving a room please contact me. Oh by the way, once you get passed Detroit Metro Airport :*) Dearborn has much to offer. There is the internationally famous Greenfield Village and Henry Ford Museum, plenty of good local restaurants, Detroit Greek Town (about 10 miles) and the Casino's in Windsor, Canada (about 20 minutes). Hope to see you there. xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Pam Ciesla 734-936-0172 - phone Merit Network, Inc. 734-647-3185 - fax University of Michigan 4251 Plymouth Road Bldg. I, Suite C Ann Arbor, MI 48105-2785 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

1 0

Re: What do we do with clueless ISPs (fwd)
by John Bradshaw 23 Mar '98

23 Mar '98

I would be interested in knowing who was spoken to at UUNET because I have two on-call engineers who are very clueful on the smurf attack and how to block it. There are issues being discussed at this time with the dostrack program and current revs of Cisco code beign distributed so tracking would not have been feasible (esp. since the site as the target and not the amplification site). Only if an amplification site was ID'd as a UUNET customer could a track to the perpetrator hav been performed. UUNET Security Engineers will alleviate the attack until it subsides or identification (and subsequent halting of the attack) occurs. The staff is not manned 24 hours (just like MCI) but my people are paged when these type of events occur (just like MCI). Hopefully, someone can provide me with the ticket # of the incident of the engineer who handled it. John > Joe Shaw said: > > From mail-maint(a)UU.NET Sun Mar 22 14:10:32 1998 > > Date: Sun, 22 Mar 1998 12:47:52 -0600 (CST) > > From: Joe Shaw <jshaw(a)insync.net> > > To: Randy Bush <rbush(a)bainbridge.verio.net> > > cc: Hank Nussbacher <hank(a)ibm.net.il>, nanog(a)merit.edu > > Subject: Re: What do we do with clueless ISPs > > In-Reply-To: <13588.7105.10059.574316(a)rip.psg.com> > > Message-ID: <Pine.SOL.3.96.980322122920.21775A-100000(a)vellocet.insync.net> > > MIME-Version: 1.0 > > Content-Type: TEXT/PLAIN; charset=US-ASCII > > Sender: owner-nanog(a)merit.edu > > > > > > Last weekend we had one host on our network as the target of a smurf > > attack. When I reported it to both our upstreams (UUNet and Time Warner > > who reported it to MCI), we got two stories. MCI, whom I'm not even a > > direct customer of started tracking the attack as soon as they were > > informed. UUNet took an hour to get a security person on the phone who > > then told me that there was nothing they could do, period. > > > > My question is this: When will UUNet have security types on duty 7 days a > > week, and will said people be clueful enough to track this sort of thing > > down? I told the people at UUNet that we were under smurf attack, and > > then I had to go through a 10 minute explanation of what a smurf attack > > was and what it was doing. I would expect a worldwide NSP to keep up with > > things like this, especially when a regional like myself can. > > > > I had logged all ICMP traffic coming into our network via an access list, > > and could give them all the information they needed to get to the > > offending networks, so it's not like they had such a hard job ahead of > > them. > > > > Joe Shaw - jshaw(a)insync.net > > NetAdmin - Insync Internet Services > > > > On Sat, 21 Mar 1998, Randy Bush wrote: > > > > > > How does one send "samples" of a Smurf > > > > > > When BBN's NOC handed one to our NOC yesterday, or was it the day before, > > > they sent a cut and paste of > > > o configuring their edge cisco to detect and log > > > o the log > > > which both documented the problem and, if our NOC did not have smurf clue, > > > gave a clue on how to track. > > > > > > [ aside: it was tracked to the perp and stomped ] > > > > > > randy > > > > > > -- ===================================================================================== John Bradshaw, Manager, Security & Fraud Support (800)900-0241 (F) (703)645-4424 UUNET Technologies, Inc., 3060 Williams Dr. Fairfax, VA 22031; http://www.uu.net security(a)uu.net - Security Incidents; fraud(a)uu.net - Massmail, spam-complaint(a)uu.net =====================================================================================

1 0