September 2007 - NANOG - lists.nanog.org

Re: shameful-cabling gallery of infamy - does anybody know where it went?
by Scott Weeks 12 Sep '07

12 Sep '07

--- streiner(a)cluebyfour.org wrote: ------------- From: "Justin M. Streiner" <streiner(a)cluebyfour.org> Note that telcos are not immune to shoddy cabling/installation work. <snip> http://www.cluebyfour.org/~streiner/mbr-pop-2000-ladder.JPG ---------------------------------------------------- Do that at the telco in Hawaii and you won't be working here very long. ;-) The installation work and wiring here is something to swoon over. scott

14 14

Re: Route table growth and hardware limits...talk to the filter
by Forrest 12 Sep '07

12 Sep '07

> From: owner-nanog(a)merit.edu on behalf of Jared Mauch > Sent: Sat 9/8/2007 8:17 AM > To: William Allen Simpson > Cc: nanog(a)nanog.org > Subject: Re: Route table growth and hardware limits...talk to the filter > > > I think this is the most important point so far. There are a lot > of providers that think that their announcements need to be global > to manage link/load balancing with their peers/upstreams. Proper use > of no-export (or similar) on the more specifics and the aggregate > being sent out will reduce the global noise significantly. > > Perhaps some of the providers to these networks will nudge them > a bit more to use proper techniques. > > I'm working on routing leaks this month. There have already been > over 2600 leak events today that could have been prevented with as-path > filters of some sort, either on a cutomer or peer. (this would obviously > be in-addition to prefix-list filters). > > - Jared > Maybe this is a dumb question, but why isn't there a BGP option to just filter more specific routes that have the same AS path as the larger aggregate? This would allow the networks that announce more specifics for traffic engineering to still accomplish that, while throwing away the garbage from someone else that decides to announce their /19 as 33 routes for no apparent reason. Sure, this would fail if a network decided to only announce /24's for example without a larger aggregate, but how many networks are really doing that? Forrest

17 25

Re: Good Stuff [was] Re: shameful-cabling gallery of infamy - does anybody know where it went?
by Scott Weeks 11 Sep '07

11 Sep '07

--- streiner(a)cluebyfour.org wrote: Some of the local old-school Bell Atlantic/Verizon techs also did very clean work, but most of them took the early retirement packages that were offered 4-5 years ago. ----------------------------------------- That's what's happening here. A lot of the old-school folks are moving on and the younger folks (I'm in between the two) don't take as much care and don't understand why guys like me grump about some of the new wiring. "I got it installed really fast, didn't I?" It's hard to watch art be trashed with graffiti. scott

1 0

2007 Elections Reminder
by Betty J. Burke 11 Sep '07

11 Sep '07

NANOG Community Members:> A friendly reminder! Reference 2007 Elections page at http://www.nanog.org/elections07.html Nominations for the NANOG Steering Committee, are due Today, Tuesday, 9-11-07. If you have not yet nominated someone and wish to do so, or if you have been asked to serve and have not yet accepted, please do so by sending a note to nominations(a)nanog.org. Finally, if you have accepted a nomination, but have not yet sent your bio information, please do so by sending the needed information to nominations(a)nanog.org. As always any questions, comments or concerns may also be sent to me!:> All best. Betty Burke Merit/NANOG SC Member

1 0

Using Mobile Phone email addys for monitoring
by Rick Kunkel 11 Sep '07

11 Sep '07

Hello folks, First off, apologies if this is off topic. I'm hoping that system and network monitoring tip are enough of a common issue that this falls under the group's charter. We've traditionally used mobile phone email addresses for system notifications, but over the past 6-12 months, it seems to have become increasingly sketchy. For instance, if an application fails to contact a certain service on a certain server, it sends an email (through it's own SMTP service, to avoid a chicken-and-egg prob if/when our main SMTP service fails) to 1234567890(a)tmomail.net. (Obviously, that was a fake number.) More and more, I'm getting less and less of these notifications. It seems especially prevalent when MANY things are sent at once; if, for example, a central piece fails, and dependent pieces suddenly fail as well. I try to telnet to mailx.tmomail.net port 25 and get sometimes good, sometimes laggy, and sometimes no response. T-Mobile, support levels all the way up to 3 tell me that it's not them, and everything should work wonderfully. Is SMTP to a mobile phone a fundamentally flawed way to do this? Anyone else have any issues, past or present, with this kind of thing? Thanks, Rick Kunkel

31 37

Re: shameful-cabling gallery of infamy - does anybody know where it went?
by Leigh Porter 10 Sep '07

10 Sep '07

Well, this one time, at band-camp .... Blake Pfankuch wrote: > One of my clients has their 34U cabinet under the stairs of the main > entrance to the facility. Mind you wood plank stairs, trafficked by > hundreds if not thousands of people a day. Over the top of the cabinet > hangs an extra long shower curtain, at an angle so that all water or mud > that falls through the wood slats actually drips down the "shower > curtain" and then runs down the drain, 3 feet away, to the sewer system. > > -----Original Message----- > From: owner-nanog(a)merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of > Warren Kumari > Sent: Monday, September 10, 2007 8:56 AM > To: Leigh Porter > Cc: Patrick Muldoon; Vinny Abello; nanog(a)merit.edu > Subject: Re: shameful-cabling gallery of infamy - does anybody know > where it went? > > > One of the places where I worked had a bunch of networking gear and > around 12x1U servers all squeezed into a shower stall.... There was a > cardboard sign hanging from the faucet saying "WARNING!!! Do not turn > on" > > W > > > > On Sep 10, 2007, at 11:38 AM, Leigh Porter wrote: > > >> We used to have a POP under somebodys stairs in Bristol in the UK and >> another POP in the loft of a friend of one of the employees. They sold >> their house and the POP stayed there and the new owners knew nothing >> about it, imagine their surprise when a telco engineer turned up >> wanting >> to fix a fibre fault ;-) >> >> -- >> Leigh >> >> >> Patrick Muldoon wrote: >> >>> On Sep 10, 2007, at 12:34 AM, Vinny Abello wrote: >>> >>> >>>> One of the stranger things a field tech of ours encountered wasn't >>>> necessarily bad wiring (although it's not great), but the fact that >>>> the demarc was located next to the toilet in the bathroom. >>>> Naturally, >>>> the constant humidity caused bad corrosion problems and other issues >>>> with their telco services. :) So as a general rule of thumb, avoid >>>> putting your telco and/or network gear next to the crapper or the >>>> services the equipment is meant to provide might also stink >>>> >>> I know of one ISP that had their local POP in a small rural town, >>> the >>> bathroom of a local store, sitting on a shelf in rather close >>> proximity to the sink (Sorry don't have pictures). So Router, modem >>> bank and a couple T1's. The kicker was they had it all plugged into >>> an extension cord that ran to another part of a back room. More >>> than >>> 1 time we (as the local telco) had to go out there cause they where >>> certain it was a problem with the Ts, When in fact someone had either >>> tripped over the power cord or unplugged it somehow. >>> >>> -Patrick >>> >>> -- >>> Patrick Muldoon >>> Network/Software Engineer >>> INOC (http://www.inoc.net) >>> PGPKEY (http://www.inoc.net/~doon) >>> Key ID: 0x370D752C >>> >>> NOTICE: alloc: /dev/null: filesystem full >>>

1 0

Re: shameful-cabling gallery of infamy - does anybody know where
by Frank Coluccio 10 Sep '07

10 Sep '07

The Valuation of a Disconnect During the 1968-1971 time frame, the new line installations at New York Telephone Company [now Verizon] all but came to a grinding halt in its busiest commercial districts of New York City. No, not because of labor actions or a strike (although a strike actually did ensue late during this period, as described in a moment), but because some genius at headquarters came to a startling realization that there was no money to be made in disconnects. A disconnect, for clarity, is an order to remove a line, or to remove portions of a line that were rearranged or upgraded. Consequently, the main distribution frames within central offices (some of which were L-shaped and a city block and a half long) became swamped with copper wires to the point that technicians could no longer find the termination blocks on which "soldering lugs" were located to affix new circuits, for the mountain of singles, pairs, triplets and quads that had accumulated and literally buried several of the larger frames. Fortunately for the telco, the union did eventually strike in 1971, but for better pay and working conditions and obviously not due to reasons sated above, although the strike nevertheless allowed the telco's management to call in legions of management staffers from the other 22 Affiliated Bell Operating Companies across the nation (which they'd not ordinarily do in the presence of union workers during normal times) to assist in its rescue that was effected through a massive initiative aimed at removing untold tons of dead wiring. During this period the telco was shielded from the burdens of new line installations by virtue of the strike as it feigned being understaffed (until the mess was cleared, and then new lines began going in like a charm, which gave management the opportunity to gloat over how fast things could "really" be done!). But the lesson had been learned the hard way that disconnects actually do matter, and they need to be administered in much the same way as do new line orders or any other "profitable" offering. I once had access but no longer do to a set of photos that illustrated the "waterfall" effects of the cables overflowing NY Tel MDFs at the W50th and E56th St COs. If anyone here has a link to those photos or a copy of them, kindly post or send offlist. TIA. Frank -------- On Mon Sep 10 1:14 , Jay Hennigan sent: > >Vinny Abello wrote: > >> One of the stranger things a field tech of ours encountered wasn't necessarily bad wiring (although it's not great), but the fact that the demarc was located next to the toilet in the bathroom. Naturally, the constant humidity caused bad corrosion problems and other issues with their telco services. :) So as a general rule of thumb, avoid putting your telco and/or network gear next to the crapper or the services the equipment is meant to provide might also stink. >> >> http://users.tellurian.com/vabello/bathroom-demarc.jpg > >On the plus side, they didn't have to go far for a ground. > >-- >Jay Hennigan - CCIE #7880 - Network Engineering - jay(a)impulse.net >Impulse Internet Service - http://www.impulse.net/ >Your local telephone and internet company - 805 884-6323 - WB6RDV

3 2

Re: Route table growth and hardware limits...talk to the filter
by Randy Bush 10 Sep '07

10 Sep '07

> Despite Tim's comment, this is not the result of "non-standard" > policy languages. In fact, it's the result of truly expressive > policy languages. i am less sure of that. tim's metarouting algebras do not seem to lack expressiveness. they just allow us to define policy languages that do not let us express stupid things. > Any language that forces all of us to make globally consistent > choices is not going to mandate aligned policies, which is going to > necessarily result in deeply restricted systems. i am not sure global consistency requires global homogeneity. it is more likely to require some visibility/transparency which maybe the folk doing partial-reveal crypto have a leg on. but, interesting as this is, i fear we have drifted a bit afield from describing hacks to allow us more control of what moves from our ears to our ribs and then what fibs we tell. randy

1 0

Re: Route table growth and hardware limits...talk to the filter
by Randy Bush 10 Sep '07

10 Sep '07

>> how about a filter between in-rib and what you actually crank >> through the churning clothes washer? pass on the in-rib, calc on >> the phyltered data. so when shorter prefix is withdrawn, you can >> look for next best candidate. > Possible, but decidedly non-trivial. Recall that if you're going to > phylter before the RIB, then you'll have to reflect that in your BGP > advertisements too. Any route caught in the phylter must be > withdrawn if it has been advertised. i am not happy with just announce from input, but note that it would remove this problem. it would also throw the best path baby out with the bathwater. i don't think we are ready for that. >> note that my original proposal/case some years back allowed a >> number of flavors of phylter, longer+same-next-hop, >> longer+same-as-path, longer+same_origin-as. > Full time employment for BGP geeks. ;-) On the flip side, the number > of variants of expert systems that you're proposing suggests that > there be one configurable mechanism. Complexity++; ;-( and even more hidden policy, which uncle tim warns us will lead to a wsj article some day (appended). but i am slightly more fearful of the cost of routing table growth and table churn. slightly. randy --- appended is a position statement from wired 2003. Tim Griffin ---------------------------------------------------------- ROUTING POLICY LANGUAGES MUST BE DESIGNED AND STANDARDIZED ---------------------------------------------------------- The following scenario MUST take place within the next few years: The Interdomain routing system will enter a state of non-convergence that is so disruptive as to effectively bring down large portions of the Internet. The problem will be due to unforeseen global interactions of locally defined routing policies. Furthermore, no one ISP will have enough knowledge to identify and debug the problem. It will take nearly a week to fix and cost the world economy billions of dollars. The world press will learn that the internet engineering community had known about this lurking problem all along.... So, we better have a solution! I'll argue the only way to effectively solve this problem is to define routing policy languages that are guaranteed to be globally sane, no matter the what local policies are defined. Then these languages need to be standardized and BGP speakers MUST be forced to use them. This raises many interesting research problems. Is it possible to design such languages? How can we find the right balance between local policy expressiveness and global sanity? What exactly do we mean by "autonomy" of routing policy? Do we need additional protocols to enforce global sanity conditions? How can we enforce compliance of policy language usage? -30-

1 0

Re: Route table growth and hardware limits...talk to the filter
by Forrest 09 Sep '07

09 Sep '07

> From: owner-nanog(a)merit.edu on behalf of Jared Mauch > Sent: Sat 9/8/2007 8:17 AM > To: William Allen Simpson > Cc: nanog(a)nanog.org > Subject: Re: Route table growth and hardware limits...talk to the filter > > > I think this is the most important point so far. There are a lot > of providers that think that their announcements need to be global > to manage link/load balancing with their peers/upstreams. Proper use > of no-export (or similar) on the more specifics and the aggregate > being sent out will reduce the global noise significantly. > > Perhaps some of the providers to these networks will nudge them > a bit more to use proper techniques. > Could a partial solution to this problem be something as simple as the routing vendors implementing something that by default analyzes the routes that you're announcing to your neighbors and automatically flags the more specifics that are covered by an identical larger aggregate with no-export? Rather than relying on the announcing network to make an effort to use no-export, make it a default that can be turned off if the need arises. If the cause of alot of this garbage is indeed caused by cluelessness or apathy, it seems like this would help. If people are too lazy or lacking the knowledge to fix what they're announcing, they're likely too lazy or lacking the knowledge to disable a default setting that flags extra unnecessary routes with no-export. Forrest

1 0