- NANOG - lists.nanog.org

The wyrms and the law.
by Roeland Meyer 26 Sep '01

26 Sep '01

Life just got more interesting for the crackers. http://www.securityfocus.com/news/257 Yesterday a new virus was announced on the news. It's only an email virus, but in light of the above, the timing is interesting. The bar is being raised. Between Code Red and Nimda, the past few months have been most interesting, network-wise. By any stretch of the imagination, wyrm spawning falls well within the boundary of "terrorist acts". The idea is that; to coerce behavior, via threat of harm or direct action to commit harm, is an act of terrorism. There goes the argument that crackers are doing us all a favor by exploiting security holes. -- R O E L A N D M J M E Y E R Managing Director Morgan Hill Software Company tel: +1 925 373 3954 cel: +1 925 352 3615 fax: +1 925 373 9781 http://www.mhsc.com

7 8

RE: Points of Failure (was Re: National infrastructure asset) (fw d)
by Roeland Meyer 26 Sep '01

26 Sep '01

|> From: Grant A. Kirkwood [mailto:grant@virtical.net] |> Sent: Tuesday, September 25, 2001 7:56 PM |> Rafi Sadowsky wrote: |> > <http://www.thebunker.net/index.htm> |> Anybody want to partner up in a business venture with me? I've always |> thought it'd be cool to buy one of those missile silos in Wyoming and |> build a datacenter like this. ;) I looked into this once. There was a similar site available in Oakland, CA, for a while (I don't know if it still is). All of the above plus it was built to handle shake'n'bake (8.3 Richter).

1 0

Re: Points of Failure (was Re: National infrastructure asset) (fwd)
by George William Herbert 26 Sep '01

26 Sep '01

David Lesher <wb8foz(a)nrk.com> wrote: > George William Herbert leaked to the press: The press are here? I thought we were safer than that ;-) >> On top of that, none of these facilities are sufficiently hardened. >> What takes a backhoe operator ten minutes by accident would be no more >> than an hours work by hand of a sufficiently educated attacker. >> None of these telco buildings are hardened in the traditional >> anti-terrorist sense of the word. There are still co-loc facilities >> in buildings shared with offices of unrelated companies, etc., >> there are still co-loc facilities in buildings with windows into >> server rooms, etc. I could go on but will stop now. The situation >> is hopeless in many areas. > >Fixing THAT is a far far harder issue. We did tackle it once. >It's called variously "Continuity Of Government" or "ATT >Underground" or "L4".. During our last Cold War, ATT spend a few >zillion rate-payer dollars building hardened underground facilities >for the #5 Transcontinental Coaxial Cable. The #5 was buried deep >{~48"} from coast to coast. The underground facilities were solid; >some rated for 10psi shock waves. (The crapper is hung on springs...) >Air filters; generators; airlocks. There was also hardened microwave; >check out the "dish in the hillside" at Mt. Weather. > >Most have been sold off, but not all. (DSN is based in one, as is >GEP.) We gave up when things like MIRVS made it obvious that >the undergrounds could be direct targets. > >Plus, I strongly doubt any facility other than Cheyenne Mountain >or Site R would survive a 767 hit. We don't really need all our coloc buildings, telco switching centers, cable landings to resist a nearby nuclear blast or direct hit from a jumbo jet. Even at the WTC, only a fraction of the damage to telco gear was in buildings the jets hit. Most of the damage was not-hardened buildings nearby getting plastered, or buildings far enough away that they were structurally fine which suffered systems failures running independently in the long-term blackout that followed. Let's look at the "worst credible case" though, jet aircraft. Nuclear reactor domes are rated to survive jumbo jet hits. Jet aircraft are remarkably poor penetrators, in military terminal ballistics terms. Ten feet of good reinforced concrete, at $400/cu yd or so installed, will do the job nicely, though there may be some spalling inside the aircraft won't penetrate. For a 3 story, 120,000 sq ft 200x200x30 ft building that would be about $9.3 million, about $77.70/ft^2. That's a Big Deal but within reason for total building costs, and if your threat isn't quite as serious as "jumbo jets being rammed into building" but then you can have porportionally thinner walls on your bunker/building. Or you can substitute going underground or piling an earth mound on top for some of that. Not that I'm recommending every telco facility be hardened to resist a direct jumbo jet hit. But it's not that ridiculous a task, and lesser hardening (half or 1/3 that level) would be downright reasonable compared to the other costs of making these sorts of buildings for these roles. A 2 foot thick reinforced concrete wall, for example, might well be extremely reasonable. >> Longer term, we all need to think about multi-level hardening of facilities >> and connectivity to avoid "cheap kills" due to accident or malicious attack. >> This gets into traditional datacenter design issues and beyond, into building >> hardening (the new standards for Federal buildings, for example, or even >> better the new standards for US Embassies...). > >Note the biggest safety measure for both the above is also the >hardest to get & keep.... stand-off distance. It's especially >hard to maintain same when you have an active airport's flight >path going overhead. > >And the biggest impediment is the oldest one; they co$t. >LOTS. Who is going to pay? Stand-off distance is a big part of it, and critical if we're worried about carbombs and truckbombs. It's useless if we're worried about aircraft. And won't help at all if the company we leased 400 square feet of cage to brings 2 tons of Semtex in inside Sun E5500 cases. Think of the network firewalls analogy; hard exterior but vulnerable interior is only so good... not all the threats are external. Hell, there are buildings with natural gas service inside the same structural enclosure as the coloc machine room. That can turn into a bomb entirely by accident, much less malicious intent. Or what happens if someone knocks over one of your tanks of fire supression agent with a forklift, and it bounces around the machine room like a scuba tank does when the regulator's knocked off, only 100 times larger. Those can go through foot thick concrete walls or better. Or someone drills into your wet pipe sprinkler system by accident. There are many many angles to all of this. -george william herbert gherbert(a)retro.com

4 3

RE: Points of Failure (was Re: National infrastructure asset) (fw d)
by Roeland Meyer 26 Sep '01

26 Sep '01

|> From: Rafi Sadowsky [mailto:rafi-nanog@meron.openu.ac.il] |> Sent: Tuesday, September 25, 2001 5:30 PM |> If you're paranoid & want your equipment to sustain a |> direct nuclear jit |> on the CoLo facility probably want to have a look at "The Bunker" |> (In the UK - sorry ;-) |> |> <http://www.thebunker.net/index.htm> |> |> It seems "The Bunker" is designed to survive a direct |> nuclear hit & you |> can get your own sealed room ( I guess it's not cheap though ...) |> |> A on-site supply of _3 months_ of generator fuel , etc .... ... and it's run by A.L. Digital, of Apache-SSL and OpenSSL fame. Physical AND Data security.

1 0

Re: Points of Failure (was Re: National infrastructure asset)
by George William Herbert 25 Sep '01

25 Sep '01

Ray writes: >> I know it's difficult to refrain from comment, but let's try to remember >> that the bad guys read this list too. While they may not have the knowledge >> of critical communication infrastructure points, they can certainly find and >> target them if we point them in the right direction. > >I'd actually argue the opposite. It's difficult to face this, but we know >we ARE vulnerable. The important long term solution is that we need to >address our weaknesses. By acknowleding where the critical points are, >AND PLANNING TO DEAL WITH THEIR LOSS, we make the system that much harder to >defeat. Exactly. The short term situation is we're vulnerable; how do we deal with those vulnerabilities being exploited (or accidentally exposed). The long term situation is how to we reduce or eliminate as many of those as possible. Part of the insidiousness of all this is that currently there is insufficient information available to a telco line end user to properly plan for that sort of loss. Right now, it is nigh-on impossible to get, verify, and keep on a permanent basis truly widely separated leased lines / bandwidth from point A to point B. Because of the increasing cooperation, shared facilities, etc. in the telco and fiber arenas, with many providers you're really getting someone else's service for part of the connection. Recall the train tunnel fire from not that long ago, now seemingly trivial, but at the time a huge disaster... On top of that, none of these facilities are sufficiently hardened. What takes a backhoe operator ten minutes by accident would be no more than an hours work by hand of a sufficiently educated attacker. None of these telco buildings are hardened in the traditional anti-terrorist sense of the word. There are still co-loc facilities in buildings shared with offices of unrelated companies, etc., there are still co-loc facilities in buildings with windows into server rooms, etc. I could go on but will stop now. The situation is hopeless in many areas. What we have learned and need to deal with is that we are, and will remain for the forseeable future, vulnerable to large chunks of "stuff" dropping away, possibly permanently. Be that buildings, bandwidth, higher level protocols, the immediate response has to be to be prepared to replace or route around something. And by that I mean *anything*. If your NOC burns down or is blown up (or hit by a tornado, knock on wood...) do you have adequate personel and facilities elsewhere to recover your network management? Eliminate all fiber links from city A to city B, and can your network still function? If all your facilities in metropolitan area Z all go completely offline, what are you able to do about it? Longer term, we all need to think about multi-level hardening of facilities and connectivity to avoid "cheap kills" due to accident or malicious attack. This gets into traditional datacenter design issues and beyond, into building hardening (the new standards for Federal buildings, for example, or even better the new standards for US Embassies...). This is a bad time for people who run fiber, but maybe it's a good time for them to consider how they run that fiber and should run it in the future. Raw cable in shallow trenches may be in the long term more expensive (if we include accidents, and in particular vulnerability to intentional attack) than deeper and/or better protected cables. Using random rights-of-way may be a mistake; it may make more sense to use ROW which are known and controlled or patrolled to some degree already. I had brought up the idea of using modern oil-well drilling technology to go horizontally deep under rivers and city centers a month or so ago; that also introduces structural hardening against intentional or accidental attack on the fibers. The key here is *think* about it. There is probably some bad guy out there who already is, though he may never decide to execute on those thoughts on you (or anyone). Try and get to any conclusion he might first, and at the very least list our your known vulnerabilities at every level you can think of, so that you can work on reducing them over time and conceptually be prepared to deal with them even if you can't afford to do detailed plans for everything that might go wrong. -george william herbert gherbert(a)retro.com

3 3

Our Qwest Testimonial: qwestoutage.com
by Allen Gwinn 25 Sep '01

25 Sep '01

Greetings, Simply put, they've put us through hell, so I've put a website up to tell our story at qwestoutage.com. I'd be interested in feedback--especially anyone that has had similar or different experiences than we've had. I've heard from another customer (DS-3 customer) who has had fairly good experience. His comment was, "the hardest thing to do is to get their attention--but once you do that, they're fairly responsive." I heard from one more that is decommissioning their Qwest circuits nationwide and moving back to UUNET. Thanks, Allen

10 9

RE: AT&T network recovery preparations
by Matt Zito 25 Sep '01

25 Sep '01

> -----Original Message----- > From: Sean Donelan [mailto:sean@donelan.com] > Sent: Friday, September 21, 2001 6:57 PM > To: nanog(a)merit.edu > Subject: AT&T network recovery preparations > > > > > As far as I know, AT&T is the only carrier which maintained > the ability to rebuild an entire central office from a "smoking > hole in the ground." Last year the program budget was in danger > of being cut, and the program eliminated because it had never > been needed. It was an expense, which couldn't be justified. > I spoke with one of the managers last year, and he said "It > would only take one catastrophe to pay for my entire budget." > I don't believe he was thinking of this. > > http://www.att.com/network/recovery/preparing.html > > > Along those lines: I don't know if all of the colos are reacting this way, but at AT&T's datacenter @ 811 10th Avenue in Manhattan, the security there has gone into crisis mode. The steel doors are down to the main entrance - everyone must enter through a side door, where your bags are thoroughly searched and IDs checked to the Nth degree. Something that's always been interesting about that facility as well are the "Duress lights" at many of the corner junctions near security outposts. There's a sign right next to these lights that reads "If the adjacent light is lit, a security condition may exist around this corner. Entering may constitute a serious risk to your person", or something like that. Nice to know that they take security seriously. Matt -- Matthew J. Zito Systems Engineer Register.com, Inc., 11th Floor, 575 8th Avenue, New York, NY 10018 Ph: 212-798-9205 PGP Key Fingerprint: 4E AC E1 0B BE DD 7D BC D2 06 B2 B0 BF 55 68 99

28 41

Re: Points of Failure (was Re: National infrastructure asset)
by George William Herbert 25 Sep '01

25 Sep '01

Sean writes: >Travis Pugh wrote: >> I was able to procure extremely detailed route information from multiple >> CLECs and Verizon in the Boston area when researching both dark fiber and >> SONET purchases, up to and including street-level maps. Long-haul providers >> also are able to provide detailed route information, if you're willing to >> ask -- the key is to be an educated buyer. > >Getting the maps is the "easy" part. Keeping track of where your >circuit is after six months is the hard part. I've spoken to people >who even had "dark fiber" groomed into a single path. > >When even NORAD and VISA have had problems maintaining diversity, I >find it hard to believe it is a simple as saying the key is to be >an educated buyer. Heck, I know major, major carriers which have >wiped out all the circuits to their own NOC. > >If it was so simple, everyone would do it. I think Travis does have a point, though, I bet there are a lot of people out there who didn't even bother to do the first try at it, partly out of lazyness, partly out of despair knowing that the maps will change eventually and tracking it may be impractical at some levels. I suspect more people need to make that effort, and the more people making that effort the easier it may become to actually track and maintain the diversity over time. That the tracking may not be in realtime and may not be 100% accurate doesn't mean that more info more of the time isn't an improvement. -george william herbert gherbert(a)retro.com

1 0

Re: Points of Failure (was Re: National infrastructure asset) (fwd)
by George William Herbert 25 Sep '01

25 Sep '01

Some of this is drifting off topic. I have also juggled the order of some of David's points in order to address them in order of most-on-topic to least-on-topic for this list. David committed to electrons: >Unnamed Administration sources reported that George William Herbert said: >> There are many many angles to all of this. > >And who will pay? In the end the customers will, of course. For any specific protective measure, there's a balance between risk and cost to protect against it, and for all risks there's a total balance of how much the customer is willing to pay overall period. We now know that the risk spectrum is both different and wider than it was. That now has to be assessed both technically and financially (cost of protecting, costs and risks of sustaining losses, costs customers are willing to bear) by all of us in the business now. The answers will not be the same for each business. The answer probably will be more push towards more hardening the more central to the net infrastructure a particular facility or business is, though, as a sweeping generalization. We've now come very close to losing major exchange points or telco centers of equivalent criticality several times. The consequences need to be addressed. The answer is obviously not going to be every colocation facility all the way out to your POP in Elko, NV being armored enough to withstand a nearby nuclear detonation. But how we do things structurally probably needs to shift a lot as a result. Too many people in the industries have blown off physical risk too much for too long. >> Not that I'm recommending every telco facility be hardened to >> resist a direct jumbo jet hit. But it's not that ridiculous >> a task, and lesser hardening (half or 1/3 that level) would be >> downright reasonable compared to the other costs of making these >> sorts of buildings for these roles. A 2 foot thick reinforced >> concrete wall, for example, might well be extremely reasonable. > >But every n of wall thinkness is f(n^2) less interior space; >f(n^3) more cost to build, and f(n^8) more crap from local >pols who really want retail space downtown instead of tombs. For small n compared to building dimentions, it's not n^2 but only n additional ground footprint. Ground footprint of my theoretical building without armor is 200x200=40,000 ft^2; with it, it's 220x220=48,000 ft^2. 20% more total. Thicker reinforced concrete walls are actually cheaper on a per pound basis than thinner ones are, believe it or not. Forms don't care how much space is between them, and form prep work is a lot of the cost. The actual rebar inside is porportional to volume/mass, as is the concrete cost, roughly (they both get marginally cheaper as you get to larger jobs and scale improves). If the local pols care, then you can either not build your data centers downtown, or put darkened glass windows on the outside of your concrete building to make it look like offices (this is actually reasonably good disguise and has been used in actual hardened facilities before). Another option is building-within-a-building; have offices around the perimiter, with the datacenter behind an internal solid concrete wall with few openings into the datacenter. >> Let's look at the "worst credible case" though, jet aircraft. >> Nuclear reactor domes are rated to survive jumbo jet hits. > >Better check this one out; there is considerable dispute >of late on this very point.. A F4 !=747 after all. The F-4 was used for one particular demonstration and test (the famous jet-on-rocket-sled-into-concrete-block video clip). The design criterion was a moderately loaded 747. We now diverge into well and truly off topic analysis... Looking at the problem, you have a 20x200 foot fuselage weighing about 500,000 lbs with fuel and baggage, a (4-0.5)x200 foot wing weighing 200,000 lbs with fuel, and 4 8 foot diameter engines which weigh about 12,000 lbs each with fairing and mount hardware. The only parts that are really "hard" are the engine shafts and the main landing gear. The rest is pretty weak. The concrete wall mass density is around 2.3 tons/cu meter, or around 7 tons per square meter of outer wall. That works out to 1,400 lbs/ft^2. The jetliner body at 500,000 lbs and about 315 square feet has a mass loading of about 1,590 lbs/ ft^2. That will crumple in over an airframe crumple distance of 200 feet, roughly. The engines have a mass loading of around 240 lbs/ft^2, but will concentrate most of their mass over about 10% of their area briefly and are only about 15 feet long. At an airspeed of 250 m/s, (562 mph, 500 knots) the body will impact over about 0.25 second with typical forces of around 50 million lbf, which is trying to shear a 20 foot diameter 10 foot thick reinforced concrete plug out of the larger wall structure (to use the simplest but generally quite accurate penetration model). That's 630 square feet or about 90,720 square inches, or a shear force of about 550 PSI. Good reinforced concrete will withstand more like 2,500 PSI minimum, and some of the fiber strand reinforced varieties along with rebar / mesh reinforcing are several times stronger. The center secton of the wing hitting at mid-fuselage is the largest single impact transient in the collision event, but even those don't penetrate. The engine impacts are significant primarily because they are largely focused via the engine shaft; they will be decellerating over about 3 meters in 0.025 second, with force loading on the wall of 12 million lbf over a roughly 8 foot diameter plug with surface area around 250 square feet or 36,000 square in. The shear forces on the plug walls are about 333 PSI. With much of the engine inertia effectively transferred forwards via the central shaft, this can transiently be 10 times more focused for parts of the impact, to 3330 PSI, approaching the failure point for the wall and perhaps exceeding it for limited areas of traditional reinforced concrete. The vast bulk of the engine mass should be stopped by the wall, however, and higher strength reinforced concrete will resist it adequately to prevent penetration. >> Jet aircraft are remarkably poor penetrators, in military terminal >> ballistics terms. Ten feet of good reinforced concrete, at >> $400/cu yd or so installed, will do the job nicely, though there >> may be some spalling inside the aircraft won't penetrate. For a 3 >> story, 120,000 sq ft 200x200x30 ft building that would be about >> $9.3 million, about $77.70/ft^2. > >No help. You'll have intact but long since roasted into >atoms building, hardware and employees. How many KWH of heat >from the 57,285 gallons of kerosene? Not a problem... Most of those KWH go up in the fireball and atmosphere, and the 40,000 ton concrete shell is a great heat sink and insulator to absorb and resist the rest. Much thinner walled bunkers are used for nuclear weapons installations at airfields, which have to survive having loaded bombers and fighters crash on the bunker and not cook the weapons inside, for example. As a minor aside, the example building I posted was woefully mis-optimized if you're worried about such hardening issues. You want a building that more closely resembles a sphere or cube (minimum surface area for contained space) rather than a relatively flat slab. Something more like 120x120x120, 5 stories up 3 below ground with 15 foot floor heights, would be much better. -george william herbert gherbert(a)retro.com

1 0

Need AT&T Broadband contact in Atlanta
by Jim Popovitch 25 Sep '01

25 Sep '01

If there is an AT&T Broadband engineer, with access to equipment in Atlanta, please contact me. There is an invalid route or subnet on the 24.31.98.0/255.255.254.0 subnet in Atlanta. The 'upper half' (24.31.99.1 - 24.31.99.255) is inaccessible from the public Internet, and users on that section cannot reach the default gateway of 24.31.98.1. The 'lower half' of the 24.31.98.0/255.255.254.0 subnet (24.31.98.1 - 24.31.98.255) is accessible via the public Internet. -Jim P. 404-277-8892 __________________________________________________ Do You Yahoo!? Get email alerts & NEW webcam video instant messaging with Yahoo! Messenger. http://im.yahoo.com

1 0