- NANOG - lists.nanog.org

Re: multi-homing fixes
by Paul Vixie 02 Sep '01

02 Sep '01

pete(a)kruckenberg.com (Pete Kruckenberg) writes: > ... Customers pay for the network to work end-to-end. More choices mean > better performance, more reliability. ... Not necessarily. Sure, the potential is there. But more than half the time in my experience of multihoming, the pessimal path is chosen. This includes a lot of anycast DNS experiences where each DNS server gives its own close-by mirror server's address as an answer to www.$FOO.com, thus using UDP performance into a predictor of TCP performance. Not only ain't it nec'ily so, it is nec'ily not so. The trivial case of multihoming is inside a campus where a file server, say, might be connected to more than one LAN. Unless you're very careful, your clients will end up talking to the file server through a gateway, that is, to some connection that it has to some LAN other than the client's LAN. This pessimality scales amazingly well to the larger Internet. Don't leave Murphy's Law out of your reliability calculations. More choices can simply mean more points of failure or more opportunities to make bad choices. -- Paul Vixie <vixie(a)eng.paix.net> President, PAIX.Net Inc. (MFNX.O)

2 1

RE: multi-homing fixes
by Roeland Meyer 01 Sep '01

01 Sep '01

|> From: Randy Bush [mailto:randy@psg.com] |> Sent: Monday, August 27, 2001 4:58 PM |> |> >>> The whole problem seems to me to be a lack of a |> micro-allocation policy, |> >>> and an agreement from providers that they will not |> filter that space. |> >> judicious use of this might be helpful. the problem is |> the 'judge' in |> >> judicious. |> > indeed. but it would still disassociate the space conservation and |> > table growth problems to some extent; the "judge" being the same |> > as today. |> |> ahhh, somebody understood the comment. |> |> the problem is that there is no obvious detent on the knob. |> to start the |> usual posturing and flamage, i propose min 2xDS3 multihomed. I would sincerely like to know why not 2xDS1 multi-homed. Most businesses can't cost-justify dual DS3's and many (UNited Airlines) gate their entire company through a T1 (not counting the ecommerce stuff, which is colo'd as it should be).

14 20

Re: multi-homing fixes
by smd＠clock.org 01 Sep '01

01 Sep '01

| > I'm pretty sure I need further explanation to "get it"./ I probably still don't get it, but let me see if I understand the mechanism. First, assign a prefix to a particular non-topological "locus", such as a metropolitan area, or a continent. Second, networks inside that locus will announce only the prefix, but with these exception bits. [Implied, but not stated: third, all these networks will exchange full information so as to be able to generate these exception bits]. Fourth, receivers of these prefixes, with the exception bits, will expand the longest-match trie (a Patricia tree is a compact representation of a trie, in common use when you have data with many nodes with just one child) so that lookups will only match in the case where there is no exception. If I understand you, what you are trying to do is to reduce the requirement for EVERY network operating within the aggregate to carry traffic to the ENTIRE aggregate at all times. This ordinarily would require announcing more specifics. So you propose a scheme where you use an attribute instead of the more specifics. Unfortunately, your attribute will cause the same behaviour in a receiver as would the list of more specifics, and therefore is merely a compression of the representation on the line that is somewhat better than, say, gzip. IOW, I think you are solving the wrong problem. We really have nearly zero experience with aggregates containing disjoint topology (i.e., non-provieder-based aggregation), largely because there is no obvious way to contain an explosion of more specifics when complete internal connectivity and complete transit break down. Steve Deering does propose a (partial) solution for this, but (in my opinion) it involves a complete reversal of current financial arrangements to work, in that a sender would have to compensate a transit network for carrying its traffic to anything within that aggregate, rather than the transit network collecting from the other (or both) parties. This is only a partial solution, since even where there is an incentive to maintain complete interconnectivity and carry traffic to all the consitituent subnets of the aggregate, failures will still cause black holes to arise even though other valid paths exist. Your scheme does let one warn of black holes in this eventuality, takes a bit less bandwith on the line, probably allows for the "slosh" to happen all at once rather than in dribs and drabs, and so forth, but it represents the same amount of work for the routers processing the attribute. That is, those routers are effectively brought inside the abstraction boundary of the "locus", and as a result the goal of hiding information from those routers is not met. My gut feeling is that for any sizable "locus", almost all of what we consider the core of the global routing system would be contained within the new abstraction boundary, so we're no better off than not aggregating in the first place. That is, we are MUCH better off with PA addressing. Sean.

5 8

RE: multi-homing fixes
by Martin, Christian 01 Sep '01

01 Sep '01

> but this is not really necessary either, if you use an IGP. (But some > networks use iBGP rather than an IGP to carry customer routes > internally.) I think it is more than 'some.' Carrying anything in your IGP aside from the links that make up the network and the nodes themselves presents several scaling issues, IMHO. --chris >

1 0

Cisco and Juniper Inter-Operability Issues ???
by Devon True 01 Sep '01

01 Sep '01

All: Our company is purchasing a couple of Junipers and we are going to run OC-3 circuits off of them. We current have Cisco 7507 routers that will connect to the Junipers (via FastEthernet and later GigE). We run OSPF and BGP in our network. Any gotchas we should watch out for and any tips/tricks? I read the NANOG archive and saw someone mention OSPF problems with Juniper and to run ISIS. Is that still the case? Thanks! ----- Devon

5 4

The Cidr Report
by CIDR Report 01 Sep '01

01 Sep '01

This is an auto-generated mail on Fri Aug 31 23:00:00 PDT 2001 It is not checked before it leaves my workstation. However, hopefully you will find this report interesting and will take the time to look through this to see if you can improve the amount of aggregation you perform. Check http://www.employees.org/~tbates/cidr-report.html for a daily update of this report. NEW: Check http://www.employees.org/~tbates/cidr-report-region.html for the regional version of this report. NEW: Check http://www.employees.org/~tbates/autnums.html for a complete list of autonomous system number to name mappings as used by the CIDR-Report. The report is split into sections: 0) General Status List the route table history for the last week, list any possibly bogus routes seen and give some status on ASes. 1) Gains by aggregating at the origin AS level This lists the "Top 30" players who if they decided to aggregate their announced classful prefixes at the origin AS level could make a significant difference in the reduction of the current size of the Internet routing table. This calculation does not take into account the inclusion of holes when forming an aggregate so it is possible even larger reduction should be possible. 2) Weekly Delta A summary of the last weeks changes in terms of withdrawn and added routes. Please note that this is only a snapshot but does give some indication of ASes participating in CIDR. Clearly, it is generally a good thing to see a large amont of withdrawls. 3) Interesting aggregates Interesting here means not an aggregate made as a set of classful routes. Thanks to GX Networks for giving me access to their routing tables once a day. Please send any comments about this report directly to CIDR Report <cidr-report(a)cisco.com>. ------------------------------------------------------------------------------ CIDR REPORT for 31Aug01 0) General Status Table History ------------- Date Prefixes 240801 103170 250801 103304 260801 103160 270801 102884 280801 103189 290801 103298 300801 103283 310801 103087 Check http://www.employees.org/~tbates/cidr.plot.html for a plot of the table history. Possible Bogus Routes --------------------- *** Bogus 82.105.0.0/16 from AS3269 *** Bogus 82.105.128.0/17 from AS3269 AS Summary ---------- Number of ASes in routing system: 11502 Number of ASes announcing only one prefix: 6853 (3906 cidr, 2947 classful) Largest number of cidr routes: 849 announced by AS701 Largest number of classful routes: 1398 announced by AS701 1) Gains by aggregating at the origin AS level --- 31Aug01 --- ASnum NetsNow NetsCIDR NetGain % Gain Description AS1221 1270 964 306 24.1% Telstra Pty Ltd AS701 1398 1187 211 15.1% UUNET Technologies, Inc. AS4151 295 166 129 43.7% USDA AS577 314 193 121 38.5% Bell Advanced Communications Inc. AS6595 169 60 109 64.5% DoD Education Activity Network As AS4293 381 272 109 28.6% Cable & Wireless USA AS16473 177 74 103 58.2% Bell South AS13999 113 11 102 90.3% Mega Cable S.A. de C.V. AS705 244 145 99 40.6% UUNET Technologies, Inc. AS6429 198 104 94 47.5% AT&T Chile Internet S.A. AS7018 756 663 93 12.3% AT&T AS4755 212 125 87 41.0% Videsh Sanchar Nigam Ltd. Autonom AS12302 96 26 70 72.9% MobiFon S.A. AS4200 150 82 68 45.3% Apex Global Information Services, AS7046 323 257 66 20.4% UUNET Technologies, Inc. AS6471 128 62 66 51.6% ENTEL CHILE S.A. AS1239 464 399 65 14.0% Sprint AS724 209 149 60 28.7% DLA Systems Automation Center AS16758 63 6 57 90.5% IKON Office Solutions AS703 310 255 55 17.7% UUNET Technologies, Inc. AS3908 321 266 55 17.1% Supernet, Inc. AS3464 144 91 53 36.8% Alabama SuperComputer Network AS209 289 236 53 18.3% Qwest AS6413 65 13 52 80.0% Southern Online Systems, Inc. AS226 145 93 52 35.9% Los Nettos AS19632 60 9 51 85.0% Metropolis Intercom S.A. AS17561 113 62 51 45.1% Internet service provision to Wes AS1 545 494 51 9.4% BBN Planet AS17557 110 62 48 43.6% Pakistan Telecom AS4323 241 194 47 19.5% Time Warner Communications, Inc. For the rest of the previous weeks gain information please see http://www.employees.org:80/~tbates/cidr-report.html 2) Weekly Delta Please see http://www.employees.org:80/~tbates/cidr-report.html for this part of the report 3) Interesting aggregates Please see http://www.employees.org:80/~tbates/cidr-report.html for this part of the report

1 0

RE: Is anyone using GILAT's "Spacenet" service
by Rowland, Alan D 01 Sep '01

01 Sep '01

The data has to bounce off the satellite 22,600 miles up, a not trivial distance ( ~1000-2000). Course they could lower that to ~500-1000 if they cache on the sat. :) -Al -----Original Message----- From: Dennis Dayman [mailto:dennis@thenose.net] Sent: Thursday, August 23, 2001 3:05 PM To: nanog(a)merit.edu Subject: RE: Is anyone using GILAT's "Spacenet" service > 500ms RTT which I do not believe. Try more like 500-2000 I've been on both Telstar 7 and GE satellites in their network and it's bad. NOC/CS is a pain to work with. Major problem though. Not only do you have to deal with RAIN fade at your locations, but they only have one uplink facility and when they get "covered" everyone in the network is dead. They have no back-up's. You have to RPA/HPA (proxy) all your HTML traffic and E-Mail, FTP, TELNET are VERY SLOW since it doesn't use their HTML proxy. --------------------------- Dennis Dayman dennis(a)thenose.net Voice 972-772-7890 FAX 972-772-3337 ------------------------------------------------------------------------ ------ What goes up, must come down. Ask any system administrator. ------------------------------------------------------------------------ ------

2 1

Re: What is the limit? (was RE: multi-homing fixes)
by Larry Sheldon 01 Sep '01

01 Sep '01

I have been trying to avoid saying this, but this is just nonsense with a little srinkle of "doesn't matter" here and there. > Actually, talking about water/electricity interaction in refrigerators, > lab equipment, etc, misses the very simple point - a probability of a leak > is proportional (as a good approximation :) to the number of moveable > components (PCBs, connectors, etc) in the system. In a typical CO it's > tens or hundreds of thousands. I don't remember seeing anybody talking about lab equipment and refrigerators here. We (some of us) were talking about liquid-cooled electronics, some of them were talking about actual experience--mine with water-cooled CPU's. > PS. Water is not a good coolant, and even distilled and deionized > water tends to pick ions from metal parts rather quickly. Spirits > are flammable; CFCs are bad for environment. There's also an > issue of toxicity. Water is one of the better coolants--particularly when cost is considered. The rest of that is one of the sprinkles I mentioned--water can in fact be toxic, see http://www.dhmo.org/ In the real, actual world, the coolant is in its own circulation system (like houses, engines, and such), so cruft in the water is no more than a maintenance issue every some large number of hours. > Of course, everything is doable, but what is the cost? > Off-the-shelf components are all designed for air cooling. > Switching to liquid cooling means a lot of custom stuff; which is > expensive and which takes a long time to design and manufacture. As I recall the electronic components in a water cooled 1100/90 CPU were as "off-the-shelf" as anything else in the machine, including most of the water system. No more "custom stuff" any "branded" equipment, I'd say. > PPS It is not voltage which matters, it's current :) Even if leaks do > not cause shorts the moisture accumulation may corrode parts > leading to mechanical shorts, it also may cause excessive > cross-talk. Sporadically malfunctioning equipment is much worse > than flat-out burned out. Not sure what this all means. Yes, we did have failures due to water leaks. Idjits on the floor above us neglected to put the plun in the bottom of their water tanks and pour a lot of water on us! > PPS Finally, getting your hands wet makes your changes to get killed > by electricity _much_ higher. 48V won't do you any harm if your > hands are dry, it may kill if they're wet. Als true--but then we were always careful to dry our hands before leaving the restroom. And checking the folks upstairs if we saw water bottles on the dock.

5 4

Re: multi-homing fixes
by smd＠clock.org 31 Aug '01

31 Aug '01

| Assigning a prefix to a continent wouldn't be a good idea, because | that way every regional ISP has to Well, that's what's been happening for a few years now... But consider this: | announce the very large bitmap for the entire continent, while most of it | contains just zeros. Per metro area would be better. But two ISPs that | have many multi-homing customers in common could use a prefix for just the | two of them, regardless of geography. Instead of a bit-vector with lots of zeros, do run-length encoding of the bits you don't have, or the bits you do have. Now compare that to doing automatic aggregation, so that instead of 16 contiguous 1 bits you introduce a prefix of x.x.x.x/(n-4). See what I mean by your bit-vector just being another representation (possibly more efficient on the wire) of what's already there? That is, you are noticing that for any given abstraction boundary that is non-topological, exception routes (more specifics or described with your bit-vector) have to be introduced when a black hole would otherwise result, and may be desirable to optimize routing. This is why the RIRs' prefixes are announced as relatively long chunks of /20s or so rather than the /8s or shorter as they are gotten from IANA/ICANN/ASO/thin air/whatever. If you try to take the same approach and have a "SRIR" (s = sub), you will see a carving up of the prefixes over time, in the same fashion. The ONLY way to avoid this right now is to have addressing exactly match topology. The problem here of course is that nobody will agree on which N entities are "top level". | I'm mostly trying to solve the memory problem, but it should also help | with (but certaintly not completely solve) the processing problem. It doesn't really do either. You have made a few time-vs-space tradeoffs which move the problems around a bit, but don't actually eliminate them. | Since an updated bitmap is always the same size and it updates many routes | at a time, it should take less CPU power to process the updates. Parsing the received updates is not the major worry, and your fix is targetted on the syntax of the updates, rather than their semantics, which is where the problem lies. | I think the only way to really know what the processing benefits of all of | this are is implementing it, or run detailed simulations, but those | require pretty much an implementation as well. Well, I won't stop you from testing an implementation or two. Write back to the list (or better yet, the IDR list) when you're done. Let me take some of your text out of context and agree with it fully: [A permanently stable holes-in-CIDR-blocks environment exists] | Only if there is a limit on the number of ISPs. I don't think there | is such a [practical rather than absolute] limit The problem with metro-based addressing is that the statement above is equally true for it as for PA addressing. Sean.

2 1

RE: Cisco and Juniper Inter-Operability Issues ???
by Stanley, Jon 31 Aug '01

31 Aug '01

To clarify on the last point: "commit confirmed <number>" specifies the number of miniutes that can pass without another commit before the changes will be automatically rolled back. I use this ALL the time on firewall policies (something like commit confirmed 2 to verify that I'm accomplishing what I want to). It really is a lifesaver. -----Original Message----- From: John Kristoff [mailto:jtk@depaul.edu] Sent: Friday, August 31, 2001 3:59 PM To: nanog(a)merit.edu Subject: Re: Cisco and Juniper Inter-Operability Issues ??? Devon True wrote: > Any gotchas we should watch out for and any tips/tricks? I read the > NANOG archive and saw someone mention OSPF problems with Juniper and to > run ISIS. Is that still the case? I haven't seen any OSPF compatibility problems, including the use of authentication and changes in the default reference-bandwidth between Junipers and Cisco. Tested using JunOS 4.4 and 4.3. As far as the OC3c goes, You may want to preconfigure all the VPIs you'll ever have in advance if you can. Anytime you reconfigure them, the Juniper does a reset of the SAR to reallocate memory to the VPIs. This will cause all your ATM circuits on that interface to bounce. If you setup your peers under a BGP group statement, you will also need a 'peer-as' statement at the group level. You can give it a bogus/private one there just to satisfy the config. I don't think they have fixed this wasn't fixed as of JunOS 4.4. And finally, 'commit confirmed' is your friend. :-) John

1 0