- Test - lists.nanog.org

Re: Regarding global BGP community values
by Danny McPherson 13 Oct '99

13 Oct '99

> To go forward, I'd like to say here that the new draft are prepared now > concerning the global communities. While it's clear that a considerable amount of disagreement exists regarding transitive communities dynamically doing things, it's extremely simple for providers to just not pay attention to them. Another potential application for global transitive communities, which is likely even more debatable than path selection issues, is using them in conjunction with MEDs and "more specifics" of provider aggregates (to fix some of the brokenness of aggregates and MEDs) in order to provide a safety net for potential route leaking. This could be advantageous for several reasons. I think msot of us agree that a mechanism to re-introduce intelligence to "best-exit" type routing configurations is a good idea. It's a good idea not only because some providers want to perform "best-exit" as a value-add to their services, but also because it makes sense in order to provide the ability to compensate a peer (who's fussing about settlements and traffic asymmetries) by carrying the traffic longer on your network. It could also assist in more optimally regionalizing traffic exchange between networks, especially with the ever-growing geographically distributed inter-connectivty provided by direct interconnections. The offshoots with providing more specifics to peers are obvious, I believe. One problem is potential significant growth in routing and forwarding tables sizes, which was one of the primary drivers for aggregation techniques in the first place. If this is a problem, the provider can always opt to not accept the more specifics from the peer. The other problem I can think of at the moment, which is likley more of a concern for most folks, is wrt providers leaking more specifics, either via BGP customers, or directly. This could be a concern because perceived "clue" of a peer, as well as simple errors in configurations, etc... This can be addressed to some extent by providing drafts that discuss these issues. Then, there are the problems those accepting MEDs have regarding a networks ability to associate intelligent values with MEDs, or provide a *only* reasonable number of prefixes (versus "more specifics" expanding to thousands of /24s and longer). Of course, a large piece of this would be reliant upon a decent IP allocation plan that at worst provides router-based aggregates for more specifics, and preferrably PoP-based. This is difficult, of course, with all the older networks, acquisitions, etc... Anyways, if a set of transitive communities were defined to provide a safety net that could catch the more specifcs, or some other mechanism were created to provide the same capabilities, I'd be interested. I believe AboveNet and a few others actually have experience with accepting more specifics, and since I missed the BOF in Montreal (and no information is available on the web server as of yet?), I'd be interested in hearing what folks oinions are regarding this. -danny

3 5

Re: MCI WorldCom fiber cut in White Plains, NY
by Danny McPherson 12 Oct '99

12 Oct '99

> APS is really a useless feature. Even when you get the contract to say there > are diverse paths, they will later groom the circuits to put them in the > same bundle. APS, just like other things, can be employed less-than-intelligently. That doesn't make it broke. Believe it or not, more than a few times I've actually seen circuits switch to protect work when a fiber cut occurs on the primary path .. and, of course, we didn't read about those on NANOG :-) Given, had the protect path been on the same fiber that extra nanosecond likely wouldn't of provided much protection. Obviously, they need to be engineered and provisioned correctly. I do agree that grooming or lack of correct provisioning needs to be addressed, contractually, with clearly defined significant penalties associated with failure to meet the obligation. Once this occurs, I believe folks will begin to give more attention to such things that seem negligible in comparison to the commission they're about to receive. -danny

2 1

Re: McLeodUSA Fiber cut in Chicago, IL
by Charley Kline 11 Oct '99

11 Oct '99

I wrote: > One of our OC-3's went into path AIS at 2:55pm CDT today. McLeodUSA > reports they have a cut fiber in the Chicago area. The repair crew > has been on site since 5pm CDT, but they didn't give me an ETR. This was finally restored about 8pm CDT, but the sad epilogue is that the same contractor cut the same fibers 500 feet north of the first cut on Saturday morning around 10:30am CDT. That wasn't restored until around 6pm CDT Saturday evening. Sigh. /cvk

1 0

Re: MCI WorldCom fiber cut in White Plains, NY
by Paul A Vixie 11 Oct '99

11 Oct '99

scottd(a)cloud9.net (Scott Drassinower) writes: > MCI WorldCom has a fiber cut in White Plains, NY, as of 0902 EDT this > morning. MFS service seems to be decimated, although the Brooks network > is totally intact. i guess fibre is just not being laid in rings any more? why on earth should a single backhoe cut be able to take ANY circuit out? for the "exchange level peering" PAIX and Ames are setting up, PAIX is using NextLink as the dark fibre provider. but PAIX is not buying a path, we are buying a strand. it's ~250 miles long and goes all the way around the Bay. we're going to interrupt it in two places (PAIX and Ames) with Sonet-capable gear. when the inevitable backhoe comes, only one of the two ways around this ring will be sliced. my hope is that we can restore that cut before the other backhoe comes and slices at the other radian. but i've got to say, i didn't invent this topology -- it's what i've been told is the only way to do something like this, so i'm doing it like i was told to. apparently noone told MFS that this was the only way to do something like this? why pay the complexity and cost overhead of SONET if you're not building rings? -- Paul Vixie <vixie(a)mibh.net> >> But what *IS* the internet? > It's the largest equivalence class in the reflexive transitive > symmetric closure of the relationship "can be reached by an IP > packet from". --Seth Breidbart

5 7

Re: MCI WorldCom fiber cut in White Plains, NY
by Sean Donelan 11 Oct '99

11 Oct '99

> i guess fibre is just not being laid in rings any more? why on earth > should a single backhoe cut be able to take ANY circuit out? I've wondered about this for a few years. The only answer I've been able to come up with is "implementation." Its the same issue anytime the real world and theory collide. Of course, the salespeople give the most optimistic prediction for the performance of their product. But engineers aren't always much better. There are a lot of assumptions, guesstimates, and business trade-offs along the way to the final product. The biggest assumption is the ring is in fact a "ring," and not a "string" wrapped back on itself. Eventually even the accounts catch on to the fact you can sell almost double the fiber miles by selling both halves of the ring. If you see Nx1 protect, run. Another big assumption is the fiber equipment along the ring itself will work. There are a lot of active elements along the ring. The assumption only one thing will fail at a time (especially when humans are involved) isn't a safe assumption. Your protect circuit may have failed a long time ago, but your transport layer won't find out until it needs it. Your ring isn't interrupted just twice, at PAIX and AMES. But most likely passes through many active and passive points around the ring, most of which you won't know about, until they fail. If you have enough clout, walk the circuit with the carrier from demarc to demarc. Most will give you a song and dance about security concerns, but it helps to have the people who hand out security clearances out your side :-) I did once, and what was amazing wasn't that circuit was screwed up, but that the circuit worked at all. Not that it will really help in the long term, because the carrier will just re-groom the circuit a few weeks or months later. Lots of people have suggestions such as asking the carrier for actual route maps, design layout records, and even penalities. But I haven't found anything that really works. And its not just my incompetance, other people have had problems too. Both NORAD and the FAA had triple-redundant circuits cut. Southwestern Bell had three different fiber cuts in the past six months take out service for three to eighteen counties, including 9-1-1 and SS7 links. And even if you have done everything right, there is still a final dirty word "Pre-emption." If you kept your network up, and someone with a higher priority circuit didn't, the carrier can take your working circuit to restore the higher priority circuit. If there is a technical solution, I would love to hear it. But it seems to involve more than just technology.

1 0

Fiber won't work without electricity
by Scott Drassinower 10 Oct '99

10 Oct '99

At 1pm, MCI WorldCom had all MFS service restored that was in or passed through White Plains. It turns out it wasn't a fiber cut. The building they are located in (One North Broadway, which in addition to being a regular office tower includes POPs for pre-WorldCom MCI, MFS, and seemingly Nextel) was having electrical work performed by the building management. They had to shut down the entire building's power to get this work done. I stopped by at 12pm, and the building's security guards said management notified tenants last month they they would have no electricity on 10/9. At 7 or 8am the power went off, and the MFS batteries failed at 9am. It took four hours to get power back on. I guess they thought it was a fiber cut since the whole area simply went dark. At around 11am, they determined there was no electricity, and that the fiber was probably unhurt. Our C&W Internet service, which passes through the pre-WorldCom MCI POP in the same building, was unaffected. So either that facility has better batteries, or they did something to make sure that things would keep running today. -- Scott M. Drassinower scottd(a)cloud9.net Cloud 9 Consulting, Inc. White Plains, NY +1 914 696-4000 http://www.cloud9.net

2 2

MCI WorldCom fiber cut in White Plains, NY
by Scott Drassinower 09 Oct '99

09 Oct '99

MCI WorldCom has a fiber cut in White Plains, NY, as of 0902 EDT this morning. MFS service seems to be decimated, although the Brooks network is totally intact. At least one wireless provider (OmniPoint) has no inbound service available to customers in the area. No ideas as to where exactly the outage is or how long it will take for service to be restored. -- Scott M. Drassinower scottd(a)cloud9.net Cloud 9 Consulting, Inc. White Plains, NY +1 914 696-4000 http://www.cloud9.net

1 0

The Cidr Report
by Tony Bates 08 Oct '99

08 Oct '99

This is an auto-generated mail on Fri Oct 8 12:00:00 PDT 1999 It is not checked before it leaves my workstation. However, hopefully you will find this report interesting and will take the time to look through this to see if you can improve the amount of aggregation you perform. The report is split into sections: 0) General Status List the route table history for the last week, list any possibly bogus routes seen and give some status on ASes. 1) Gains by aggregating at the origin AS level This lists the "Top 30" players who if they decided to aggregate their announced classful prefixes at the origin AS level could make a significant difference in the reduction of the current size of the Internet routing table. This calculation does not take into account the inclusion of holes when forming an aggregate so it is possible even larger reduction should be possible. 2) Weekly Delta A summary of the last weeks changes in terms of withdrawn and added routes. Please note that this is only a snapshot but does give some indication of ASes participating in CIDR. Clearly, it is generally a good thing to see a large amont of withdrawls. 3) Interesting aggregates Interesting here means not an aggregate made as a set of classful routes. Thanks to xara.net for giving me access to their routing tables once a day. Please send any comments about this report directly to me. Check http://www.employees.org/~tbates/cidr-report.html for a daily update of this report. ------------------------------------------------------------------------------ CIDR REPORT for 08Oct99 0) General Status Table History ------------- Date Prefixes 011099 64019 021099 64011 031099 64104 041099 64203 051099 64273 061099 64275 071099 64665 081099 64852 Check http://www.employees.org/~tbates/cidr.plot.html for a plot of the table history. Possible Bogus Routes --------------------- AS Summary ---------- Number of ASes in routing system: 5768 Number of ASes announcing only one prefix: 3053 (1673 cidr, 1380 classful) Largest number of cidr routes: 570 announced by AS701 Largest number of classful routes: 929 announced by AS701 1) Gains by aggregating at the origin AS level --- 08Oct99 --- ASnum NetsNow NetsCIDR NetGain % Gain Description AS271 384 144 240 62.5% BCnet Backbone AS719 731 566 165 22.6% LANLINK autonomous system AS2609 127 13 114 89.8% EUnet-TN AS9269 137 24 113 82.5% Hong Kong CTI AS1221 618 509 109 17.6% TELSTRA-AS AS174 583 490 93 16.0% Performance Systems International AS7046 341 255 86 25.2% UUNET-CUSTOMER AS4293 226 141 85 37.6% Internal ASN for C&W AS577 261 177 84 32.2% Bell Canada Backbone AS6944 97 15 82 84.5% RapidSite, Inc. AS7545 140 60 80 57.1% TPG Internet Pty Ltd AS7657 243 165 78 32.1% The Internet Group Limited AS7496 108 31 77 71.3% Power Up AS3749 140 64 76 54.3% TECNET AS4200 181 109 72 39.8% AGIS (Apex Global Information Ser AS4740 343 290 53 15.5% Ozemail Pty Ltd (ASN-OZEMAIL) AS209 310 261 49 15.8% Qwest Communications AS2493 138 91 47 34.1% iSTAR Internet, Inc. AS684 92 46 46 50.0% Manitoba Regional Network Backbon AS816 225 182 43 19.1% UUNET Canada (ASN-UUNETCA-AS4) AS701 929 888 41 4.4% Alternet AS6528 45 7 38 84.4% WORLD-LYNX AS549 200 162 38 19.0% ONet Backbone AS4755 103 68 35 34.0% Videsh Sanchar Nigam Ltd. India AS3737 99 66 33 33.3% PenTeleData Inc. (ASN-PTD) AS7568 45 13 32 71.1% C.S. Communications Co., Ltd. AS1785 236 204 32 13.6% NYSERNet Backbone AS1 492 460 32 6.5% GTE Internetworking AS9453 33 2 31 93.9% Soongsil University AS7745 40 9 31 77.5% BIGSKY-AS For the rest of the previous weeks gain information please see http://www.employees.org:80/~tbates/cidr-report.html 2) Weekly Delta Please see http://www.employees.org:80/~tbates/cidr-report.html for this part of the report 3) Interesting aggregates Please see http://www.employees.org:80/~tbates/cidr-report.html for this part of the report

1 0

Regarding global BGP community values
by Aleksi Suhonen 08 Oct '99

08 Oct '99

Hello, I've also come to think that there should be global communities for the basic purposes that most autonomous systems have built their own communities for. My approach is probably a little different to those of the other contributers to this discussion. Here is a sketch of the new well-known communities I have come up with: 0xFFFn:arg where n designates a function and arg designates a parameter to the function. These could be the functions: n=0 Never announce to AS <arg>. n=1 May announce to AS <arg>. n=2 If you announce to AS <arg>, prepend your own AS once. n=3 If you announce to AS <arg>, prepend your own AS twice. : : n=7 If you announce to AS <arg>, prepend your own AS six times. n=8 When announcing to AS <arg>, change this community to "local-AS" n=9 When announcing to AS <arg>, change this community to "no-advertise" n=10 When announcing to AS <arg>, change this community to "no-export" n=11 Reserved for brainstorming. n=12 Reserved for brainstorming. n=13 Reserved for brainstorming. n=14 Reserved for brainstorming. n=15 This contains the old well-known communities. Perhaps this could also double as "set local preference to <arg>?" 0xFFF0 is 65520, so 0xFFFn is 65520+n. Now I'm not saying these are all useful. I for one don't like to see anyone prepend more than three times and setting "no-advertise" at a distant location can also have quite nasty consequences. They may still have use in some special circumstances. Further brainstorming: an arg of 0 could be interpreted as "all transits" and 65535 as "all peers." If these special cases were to be hard coded into the BGP implementation, then there should be a way to mark sessions as transit/peering/other. If on the other hand support for these communities would be simply via auto-generated route-maps, you would only have to tell the generator the type of the session, and since the data for the generator would most probably come from a database of some kind the generator would get the type from there automagically. These communities should be able to travel through multiple autonomous systems. They should be usable to influence not only your transits' transits' routing decisions, but your transits' peers', your transits' transits' peers', your transits' transits' transits' peers' and so on. Getting all involved parties to turn on send-community on all involved BGP sessions is of course a challenge, but a one that has already been overcome with the existing "transit provider proprietary" communities where needed. Randy Bush pointed out an underlying problem with trusting preference hints from distant autonomous systems. I would like to argue the case against him and for letting everyone influence one's routing decisions. Suppose I have a transit T, a peer P, a customer C and a distant third party D. Suppose also that D has contractual agreements that result in me hearing D's routes from all of T, P and C. By default I would prefer D's routes heard from C. However, if D wishes me to use T or P to reach D instead, D can do so without the use of any communities. D can lower the preference of a path through P or C by arranging it so that I don't hear D from P or C, and D can increase the preference of a path by making me hear D's routes as more specifics from there. Both of these practises I frown upon. I would therefore see that it would be in my best interest to let D influence my routing decisions directly instead of littering my routing table or eating at the redundancy between my customers and D. Furthermore it should be in D's best interest not to give hints to use a bad path between D and me. If D would choose to maliciously abuse my trust (whether via the kludges or via the preference hints presented) I can always work to override or filter the bad routes and try to contact D's upstreams or D directly to resolve the issue. Alex Bligh noted that main stream IOS has insufficient means of manipulating communities. The bleeding edge versions have had methods for removing only some communities while preserving others for a good time now. I do agree though that routing policies can often be much more complicated than what is implementable with route-maps no matter how modern. -- Aleksi Suhonen Here is a sample shell script that does simple automagic. If anyone would use it, it would need to be modified to implement one's own routing policy too. It is included only as an example and proof of concept. I have not tested that what it produces actually works. #!/bin/sh # Usage: this-program my-as their-as their-ip comm-list session-type # comm-list: index of first available "ip community-list" # session-type: (T)ransit (P)eer (C)ustomer my_as=$1 their_as=$2 their_ip=$3 comm_deny=$4 comm_pre1=`echo 1+$4|bc` comm_pre2=`echo 2+$4|bc` comm_pre3=`echo 3+$4|bc` comm_pre4=`echo 4+$4|bc` comm_pre5=`echo 5+$4|bc` comm_pre6=`echo 6+$4|bc` comm_loas=`echo 7+$4|bc` comm_noad=`echo 8+$4|bc` comm_noex=`echo 9+$4|bc` # these can be reused for all sessions comm_local_pref_80=1 comm_local_pref_90=2 comm_local_pref_110=3 comm_remove=4 # set these to your values transit_route_tag=${my_as}:666 peer_route_tag=${my_as}:555 customer_route_tag=${my_as}:777 route_map_name_prefix=axu-${their_as} case $5 in T*) announce_community=$customer_route_tag denounce_community="$transit_route_tag $peer_route_tag" tag_community=$transit_route_tag default_preference=90 their_class=0 ;; P*) announce_community=$customer_route_tag denounce_community="$transit_route_tag $peer_route_tag" tag_community=$peer_route_tag default_preference=100 their_class=65535 ;; *) announce_community="$transit_route_tag $peer_route_tag $customer_route_t ag" denounce_community="" tag_community=$customer_route_tag default_preference=101 their_class=65534 ;; esac denounce_community="$denounce_community 65520:${their_as} 65520:${their_class}" cat <<EOF ! reused community-lists ip community-list $comm_local_pref_80 deny 65535:100 ip community-list $comm_local_pref_80 permit 65535:80 ip community-list $comm_local_pref_90 deny 65535:100 ip community-list $comm_local_pref_90 permit 65535:90 ip community-list $comm_local_pref_110 deny 65535:100 ip community-list $comm_local_pref_110 permit 65535:110 ip community-list $comm_remove permit 65535:80 65535:90 65535:100 65535:110 ip community-list $comm_remove permit $transit_route_tag $peer_route_tag ip community-list $comm_remove permit $customer_route_tag ! new community-lists ip community-list $comm_deny permit $denounce_community ip community-list $comm_pre1 permit 65522:${their_as} ip community-list $comm_pre1 permit 65522:${their_class} ip community-list $comm_pre2 permit 65523:${their_as} ip community-list $comm_pre2 permit 65523:${their_class} ip community-list $comm_pre3 permit 65524:${their_as} ip community-list $comm_pre3 permit 65524:${their_class} ip community-list $comm_pre4 permit 65525:${their_as} ip community-list $comm_pre4 permit 65525:${their_class} ip community-list $comm_pre5 permit 65526:${their_as} ip community-list $comm_pre5 permit 65526:${their_class} ip community-list $comm_pre6 permit 65527:${their_as} ip community-list $comm_pre6 permit 65527:${their_class} ip community-list $comm_loas permit 65528:${their_as} ip community-list $comm_loas permit 65528:${their_class} ip community-list $comm_noad permit 65529:${their_as} ip community-list $comm_noad permit 65529:${their_class} ip community-list $comm_noex permit 65530:${their_as} ip community-list $comm_noex permit 65530:${their_class} route-map ${route_map_name_prefix}-in permit 10 match community $comm_local_pref_80 set local-preference 80 set comm-list $comm_remove delete set community $tag_community additive ! route-map ${route_map_name_prefix}-in permit 20 match community $comm_local_pref_90 set local-preference 90 set comm-list $comm_remove delete set community $tag_community additive ! route-map ${route_map_name_prefix}-in permit 30 match community $comm_local_pref_110 set local-preference 110 set comm-list $comm_remove delete set community $tag_community additive ! route-map ${route_map_name_prefix}-in permit 40 set local-preference $default_preference set comm-list $comm_remove delete set community $tag_community additive ! route-map ${route_map_name_prefix}-out deny 10 match community $comm_deny ! route-map ${route_map_name_prefix}-out permit 20 match community $comm_pre1 set as-path prepend ${my_as} ! route-map ${route_map_name_prefix}-out permit 30 match community $comm_pre2 set as-path prepend ${my_as} ${my_as} ! route-map ${route_map_name_prefix}-out permit 40 match community $comm_pre3 set as-path prepend ${my_as} ${my_as} ${my_as} ! route-map ${route_map_name_prefix}-out permit 50 match community $comm_pre4 set as-path prepend ${my_as} ${my_as} ${my_as} ${my_as} ! route-map ${route_map_name_prefix}-out permit 60 match community $comm_pre5 set as-path prepend ${my_as} ${my_as} ${my_as} ${my_as} ${my_as} ! route-map ${route_map_name_prefix}-out permit 70 match community $comm_pre6 set as-path prepend ${my_as} ${my_as} ${my_as} ${my_as} ${my_as} ${my_as} ! route-map ${route_map_name_prefix}-out permit 80 match community $comm_loas set comm-list $comm_loas delete set community local-AS additive ! route-map ${route_map_name_prefix}-out permit 90 match community $comm_noad set comm-list $comm_noad delete set community no-advertise additive ! route-map ${route_map_name_prefix}-out permit 100 match community $comm_noex set comm-list $comm_noex delete set community no-export additive ! route-map ${route_map_name_prefix}-out permit 110 router bgp ${my_as} neighbor ${their_ip} remote-as ${their_as} neighbor ${their_ip} next-hop-self neighbor ${their_ip} send-community neighbor ${their_ip} remove-private-AS neighbor ${their_ip} route-map ${route_map_name_prefix}-in in neighbor ${their_ip} route-map ${route_map_name_prefix}-out out EOF

4 4

Network Diagrams
by Steve Baker 08 Oct '99

08 Oct '99

I was wondering, if anyone had, or knew where might gain access to network diagrams for MAE-East and MAE-West. Thanks -S

1 0