October 1999 - Test - lists.nanog.org

Peering: Sprint & CW
by jlewis＠lewis.org 13 Oct '99

13 Oct '99

I do alot of stuff on a system sitting on a T1 to CW from my home which (through work) gets to the net via frac-T3 to Sprint and UUNet. Suddenly, yesterday connectivity from work/home to CW began to suck horribly. Today, it's no better. I called the Sprintlink NOC yesterday, and their response was it was an overloaded peering point in Washington, and don't hold my breath waiting for it to improve. Has anyone else noticed / been affected by this? Between the packet loss and the latency, a 300bps modem would be preferable. I can get about half a line ahead of pine while typing, waiting for chars to show up so I can see my typos. Maybe we should cancel the increase we just ordered in our Sprint T3 and order a CW frac-T3 instead. Hostname %Loss Rcv Snt Best Avg Worst 1. maui.atlantic.net 0% 19 19 1 3 26 2. border1-s2-1.Orlando-WCom.atlantic.n 0% 19 19 21 40 99 3. sl-gw3-orl-6-0-1.sprintlink.net 0% 19 19 23 55 192 4. sl-bb10-orl-0-3.sprintlink.net 0% 19 19 23 38 98 5. sl-bb10-rly-1-0.sprintlink.net 0% 19 19 40 55 105 6. sl-bb2-dc-4-0-0.sprintlink.net 0% 19 19 42 58 100 7. core7-hssi0-0-0.Washington.cw.net 12% 15 18 1024 1083 1150 8. corerouter1.WashingtonEast.cw.net 6% 16 18 1038 1081 1144 9. acr1-loopback.Restonrst.cw.net 0% 17 18 1031 1079 1159 10. 206.24.179.2 95% 0 17 0 0 0 11. pride.se.cw.net 95% 0 17 0 0 0 12. webserver.ie.cw.net 0% 16 17 1014 1086 1161 ---------------------------------------------------------------------- Jon Lewis *jlewis(a)lewis.org*| Spammers will be winnuked or System Administrator | nestea'd...whatever it takes Atlantic Net | to get the job done. _________http://www.lewis.org/~jlewis/pgp for PGP public key__________

2 1

Re: Regarding global BGP community values
by Alex P. Rudnev 13 Oct '99

13 Oct '99

See http://noc.bn.ru/~alex/MON I know about 5 - 10 installation here. On Wed, 13 Oct 1999, Craig A. Haney wrote: > Date: Wed, 13 Oct 1999 09:55:47 -0400 > From: Craig A. Haney <craig(a)seamless.kludge.net> > To: Alex P. Rudnev <alex(a)virgin.relcom.eu.net> > Subject: Re: Regarding global BGP community values > > >Hmm. > > > >In the 'SNMPSTATD' monitoring I use (it's in the near plans) to monitor > >not _FREE MEMORY but _THE BIGGEST PIECE OF THE FREE MEMORY_. It's the most > >critical argument. If it is less than 128K, 'config' command do not work; > >if it is less than 1500 bytes, nothing works at all -:) > > someone still uses SNMPSTATD? > > -craig > > Aleksei Roudnev, Network Operations Center, Relcom, Moscow (+7 095) 194-19-95 (Network Operations Center Hot Line),(+7 095) 230-41-41, N 13729 (pager) (+7 095) 196-72-12 (Support), (+7 095) 194-33-28 (Fax)

1 0

Re: Regarding global BGP community values
by Vadim Antonov 13 Oct '99

13 Oct '99

Aw, Tony, you're talking to a guy who made his name programming discrete transistor computers with core memory (i mean _real_ core memory, and real germanium transistors :) I bet he knows more than anyone else in this list about squeezing bits :) Cheers, --vadim PS Old-timers are entitled to being grumpy about how wasteful the modern computing is. After all, how do you feel about operating systems which just have to have at least 30 Mb of RAM to be useful? :) From: Tony Li <tony1(a)home.net> "Alex P. Rudnev" wrote: > The growth itself do not cause the problems, but in conjunction with the > poor router implementation (which cause 60,000 routes to use 30 MB of the > RAM - that means 500 bytes for every prefix -:) and numerous memory leaks > in the router implementation cause the problem. If we look around, we'll > see existing computers (including embedded ones) have not CPU and memory > problems, and all problems we see with the routers are mainly caused by > the bad implemented text. I, and the rest of the Internet community, would like to invite you to start a router company and show us how it can be done with far less memory. ;-) More seriously, you might take a look around and note that there are not a great deal of difference in the amount of memory needed to support a prefix across the various well-known implementations. Which is not to say that we're blameless, just that a lot of good people have worked hard and are all equally incompetent at conserving memory while simultaneously producing a scalable, stable, feature-rich implementation. Regards, Tony

2 1

Re: Regarding global BGP community values
by Danny McPherson 13 Oct '99

13 Oct '99

> To go forward, I'd like to say here that the new draft are prepared now > concerning the global communities. While it's clear that a considerable amount of disagreement exists regarding transitive communities dynamically doing things, it's extremely simple for providers to just not pay attention to them. Another potential application for global transitive communities, which is likely even more debatable than path selection issues, is using them in conjunction with MEDs and "more specifics" of provider aggregates (to fix some of the brokenness of aggregates and MEDs) in order to provide a safety net for potential route leaking. This could be advantageous for several reasons. I think msot of us agree that a mechanism to re-introduce intelligence to "best-exit" type routing configurations is a good idea. It's a good idea not only because some providers want to perform "best-exit" as a value-add to their services, but also because it makes sense in order to provide the ability to compensate a peer (who's fussing about settlements and traffic asymmetries) by carrying the traffic longer on your network. It could also assist in more optimally regionalizing traffic exchange between networks, especially with the ever-growing geographically distributed inter-connectivty provided by direct interconnections. The offshoots with providing more specifics to peers are obvious, I believe. One problem is potential significant growth in routing and forwarding tables sizes, which was one of the primary drivers for aggregation techniques in the first place. If this is a problem, the provider can always opt to not accept the more specifics from the peer. The other problem I can think of at the moment, which is likley more of a concern for most folks, is wrt providers leaking more specifics, either via BGP customers, or directly. This could be a concern because perceived "clue" of a peer, as well as simple errors in configurations, etc... This can be addressed to some extent by providing drafts that discuss these issues. Then, there are the problems those accepting MEDs have regarding a networks ability to associate intelligent values with MEDs, or provide a *only* reasonable number of prefixes (versus "more specifics" expanding to thousands of /24s and longer). Of course, a large piece of this would be reliant upon a decent IP allocation plan that at worst provides router-based aggregates for more specifics, and preferrably PoP-based. This is difficult, of course, with all the older networks, acquisitions, etc... Anyways, if a set of transitive communities were defined to provide a safety net that could catch the more specifcs, or some other mechanism were created to provide the same capabilities, I'd be interested. I believe AboveNet and a few others actually have experience with accepting more specifics, and since I missed the BOF in Montreal (and no information is available on the web server as of yet?), I'd be interested in hearing what folks oinions are regarding this. -danny

3 5

Re: MCI WorldCom fiber cut in White Plains, NY
by Danny McPherson 12 Oct '99

12 Oct '99

> APS is really a useless feature. Even when you get the contract to say there > are diverse paths, they will later groom the circuits to put them in the > same bundle. APS, just like other things, can be employed less-than-intelligently. That doesn't make it broke. Believe it or not, more than a few times I've actually seen circuits switch to protect work when a fiber cut occurs on the primary path .. and, of course, we didn't read about those on NANOG :-) Given, had the protect path been on the same fiber that extra nanosecond likely wouldn't of provided much protection. Obviously, they need to be engineered and provisioned correctly. I do agree that grooming or lack of correct provisioning needs to be addressed, contractually, with clearly defined significant penalties associated with failure to meet the obligation. Once this occurs, I believe folks will begin to give more attention to such things that seem negligible in comparison to the commission they're about to receive. -danny

2 1

Re: McLeodUSA Fiber cut in Chicago, IL
by Charley Kline 11 Oct '99

11 Oct '99

I wrote: > One of our OC-3's went into path AIS at 2:55pm CDT today. McLeodUSA > reports they have a cut fiber in the Chicago area. The repair crew > has been on site since 5pm CDT, but they didn't give me an ETR. This was finally restored about 8pm CDT, but the sad epilogue is that the same contractor cut the same fibers 500 feet north of the first cut on Saturday morning around 10:30am CDT. That wasn't restored until around 6pm CDT Saturday evening. Sigh. /cvk

1 0

Re: MCI WorldCom fiber cut in White Plains, NY
by Paul A Vixie 11 Oct '99

11 Oct '99

scottd(a)cloud9.net (Scott Drassinower) writes: > MCI WorldCom has a fiber cut in White Plains, NY, as of 0902 EDT this > morning. MFS service seems to be decimated, although the Brooks network > is totally intact. i guess fibre is just not being laid in rings any more? why on earth should a single backhoe cut be able to take ANY circuit out? for the "exchange level peering" PAIX and Ames are setting up, PAIX is using NextLink as the dark fibre provider. but PAIX is not buying a path, we are buying a strand. it's ~250 miles long and goes all the way around the Bay. we're going to interrupt it in two places (PAIX and Ames) with Sonet-capable gear. when the inevitable backhoe comes, only one of the two ways around this ring will be sliced. my hope is that we can restore that cut before the other backhoe comes and slices at the other radian. but i've got to say, i didn't invent this topology -- it's what i've been told is the only way to do something like this, so i'm doing it like i was told to. apparently noone told MFS that this was the only way to do something like this? why pay the complexity and cost overhead of SONET if you're not building rings? -- Paul Vixie <vixie(a)mibh.net> >> But what *IS* the internet? > It's the largest equivalence class in the reflexive transitive > symmetric closure of the relationship "can be reached by an IP > packet from". --Seth Breidbart

5 7

Re: MCI WorldCom fiber cut in White Plains, NY
by Sean Donelan 11 Oct '99

11 Oct '99

> i guess fibre is just not being laid in rings any more? why on earth > should a single backhoe cut be able to take ANY circuit out? I've wondered about this for a few years. The only answer I've been able to come up with is "implementation." Its the same issue anytime the real world and theory collide. Of course, the salespeople give the most optimistic prediction for the performance of their product. But engineers aren't always much better. There are a lot of assumptions, guesstimates, and business trade-offs along the way to the final product. The biggest assumption is the ring is in fact a "ring," and not a "string" wrapped back on itself. Eventually even the accounts catch on to the fact you can sell almost double the fiber miles by selling both halves of the ring. If you see Nx1 protect, run. Another big assumption is the fiber equipment along the ring itself will work. There are a lot of active elements along the ring. The assumption only one thing will fail at a time (especially when humans are involved) isn't a safe assumption. Your protect circuit may have failed a long time ago, but your transport layer won't find out until it needs it. Your ring isn't interrupted just twice, at PAIX and AMES. But most likely passes through many active and passive points around the ring, most of which you won't know about, until they fail. If you have enough clout, walk the circuit with the carrier from demarc to demarc. Most will give you a song and dance about security concerns, but it helps to have the people who hand out security clearances out your side :-) I did once, and what was amazing wasn't that circuit was screwed up, but that the circuit worked at all. Not that it will really help in the long term, because the carrier will just re-groom the circuit a few weeks or months later. Lots of people have suggestions such as asking the carrier for actual route maps, design layout records, and even penalities. But I haven't found anything that really works. And its not just my incompetance, other people have had problems too. Both NORAD and the FAA had triple-redundant circuits cut. Southwestern Bell had three different fiber cuts in the past six months take out service for three to eighteen counties, including 9-1-1 and SS7 links. And even if you have done everything right, there is still a final dirty word "Pre-emption." If you kept your network up, and someone with a higher priority circuit didn't, the carrier can take your working circuit to restore the higher priority circuit. If there is a technical solution, I would love to hear it. But it seems to involve more than just technology.

1 0

Fiber won't work without electricity
by Scott Drassinower 09 Oct '99

09 Oct '99

At 1pm, MCI WorldCom had all MFS service restored that was in or passed through White Plains. It turns out it wasn't a fiber cut. The building they are located in (One North Broadway, which in addition to being a regular office tower includes POPs for pre-WorldCom MCI, MFS, and seemingly Nextel) was having electrical work performed by the building management. They had to shut down the entire building's power to get this work done. I stopped by at 12pm, and the building's security guards said management notified tenants last month they they would have no electricity on 10/9. At 7 or 8am the power went off, and the MFS batteries failed at 9am. It took four hours to get power back on. I guess they thought it was a fiber cut since the whole area simply went dark. At around 11am, they determined there was no electricity, and that the fiber was probably unhurt. Our C&W Internet service, which passes through the pre-WorldCom MCI POP in the same building, was unaffected. So either that facility has better batteries, or they did something to make sure that things would keep running today. -- Scott M. Drassinower scottd(a)cloud9.net Cloud 9 Consulting, Inc. White Plains, NY +1 914 696-4000 http://www.cloud9.net

2 2

MCI WorldCom fiber cut in White Plains, NY
by Scott Drassinower 09 Oct '99

09 Oct '99

MCI WorldCom has a fiber cut in White Plains, NY, as of 0902 EDT this morning. MFS service seems to be decimated, although the Brooks network is totally intact. At least one wireless provider (OmniPoint) has no inbound service available to customers in the area. No ideas as to where exactly the outage is or how long it will take for service to be restored. -- Scott M. Drassinower scottd(a)cloud9.net Cloud 9 Consulting, Inc. White Plains, NY +1 914 696-4000 http://www.cloud9.net

1 0