October 1998 - Test - lists.nanog.org

Sunday evening NANOG tutorials
by Susan R. Harris 15 Oct '98

15 Oct '98

We're pleased to announce two tutorials for Sunday evening, November 8, in Atlanta: 7:15 - 8:45 "Good ISPs Have No Class - Addressing Nuances and Nuisances" Presented by Howard Berkowitz A review of some of the more subtle points of CIDR, aggregation, and renumbering. Included are tricks and techniques that the newer ISP might need, including pure address administration and procedures for submitting address space justifications. Hank Nussbacher's CIDR FAQ is a prerequisite for the session: http://www.ibm.net.il/~hank/cidr.html 8:45 - 10:45 "External Route Selection: Tips and Techniques for ISPs" Presented by Avi Freedman, Net Access Tips for ISPs on external route selection, including the BGP MED and LOCAL_PREF attributes, peering at multiple locations; backup transit; and how to mix transit, public, and private peering. We've also added several other presentations to the Mon. - Tues. schedule: http://www.nanog.org/mtg-9811/topics.html ------------------------------------------------------------------------- Susan R. Harris, Ph.D. Merit Network, Inc. Phone: 734.936.2100 srh(a)merit.edu Senior Science Writer Fax: 734.647.3185 -------------------------------------------------------------------------

1 0

Re: IGPs in use
by Vince Fuller 15 Oct '98

15 Oct '98

Not to nit-pick, but one correction and a few comments: Let's start with OSPF: ... Timers must be consistent among all neighbours, so if you want to make a change to your hello/dead timers to deal with delays somewhere (in links or routers), then you have to configure your entire network all at once. This isn't true. Timers must be consistant between adjacent routers, not across the entire network. This means, for example, that you must change both ends of a point-to-point link at the same time or must change all of the routers on a common multi-access network at the same time. ... The "scaling" feature of multicasting on a multicast-capable subnet saves you only 1/2 of your traffic, and LSA traffic still grows as your neighbour count grows. This is thanks to the unicast ACKs. The ACKs should be smaller than the distributed LSA's, so the savings should be a bit better than 1/2... ... The means of repairing a level-2 (backbone area) parition is, to put it politely, completely fucked in the head. I'm not sufficiently familiar with how IS-IS works to know how it solves this problem. But I wouldn't describe the OSPF virtual link mechanism as "completely fucked in the head"; think of a virtual link as a tunnel used only by the routing system... ... There are some drawbacks to IS-IS: ... IS-IS uses an underlying traffic exchange which is based on OSI/CLNS. This introduces requirements for OSI addressing and CLNS implementation which are otherwise useless in an IP-centric network. IMHO, this represents a substantial bit of operational complexity (obtaining CLNS addresses, teaching operations/engineering staff how to use them and interpret them while debugging, etc., etc...) On a pragmatic note, though, the relative successes of IS-IS and OSPF in the large provider marketplace probably has more to do with the relative competence of the cisco's original OSPF and IS-IS implementors than anything else (only someone else who suffered through OSPF's growing pains way back in the 9.0-9.1 days can really appreciate this comment...) --Vince

1 0

Re: IGPs in use
by Tony Li 15 Oct '98

15 Oct '98

scharf(a)vix.com (Jerry Scharf) writes: > One thing about IS-IS that is quite different from OSPF and all the other > IGPs is that the neighbor traffic is not in IP but instead is done at > layer 2. Some say this is a good thing. I have never run into a case > where I couldn't get it to do what I wanted, but always suspected there > would be. Would this force ATM links to move to the less efficient SNAP > based encoding (if so, there's 4% of your bandwidth)? Yes, however, let's be accurate here: the problem is NOT that IS-IS runs over the link layer. The problem is that it doesn't run over IP. Even if it ran over another network layer protocol, you'd still require an additional demux layer. One might point out that SNAP is perhaps not the most efficient of all encapsulations anyhow. Nor is ATM. Tony

1 0

Re: IGPs in use
by Tony Li 15 Oct '98

15 Oct '98

smd(a)clock.org (Sean M. Doran) writes: > There are some drawbacks to IS-IS: its per-link metrics are not > rich as currently defined, the LSP space is limited (but can be > hacked at), and you are stuck with no more than 255 pseudonodes. > All of these are likely to be hacked at and some seem pretty > fixable by locking Henk and Tony (and probably Dave Katz, who did > a good public analysis of IGP scaling at NANOG in Phoenix and who > has done even better private analyses and gripefests --:) ) in a > room for a few hours. In fact, I believe that it is not necessary to lock us in a room. A preferable alternative is if you lock others out, and throw in bandwidth, cycles, pizza, Scotch, diet Coke, and cash. ;-) ;-) ;-) > Frankly, though, I'm hoping that rather than locking themselves > in the room to "improve" IS-IS, the Obvious People lock themselves > into a room and build something else instead (which they might simply > call IS-IS, even if it isn't. Been there, done that, called it BGP). It's not clear that locking these people in a room would produce something fundamentally different than IS-IS anyway... so.... I believe that a more accurate modus operandi will be to extend IS-IS as much as possible within the bounds of the existing protocol. Then, when pressing problems drive us beyond those borders, to do more major work on the protocol. Whether that final result is called IS-IS or not (or even OSPF ;-) is largely a political question, not a technical one. Tony

1 0

Re: IGPs in use
by Tony Li 15 Oct '98

15 Oct '98

smd(a)clock.org (Sean M. Doran) writes: > BGP is fundamentally rate-limited by virtue of running on TCP. While that's necessary, it turns out to not be sufficient. A competent implementation must also meter out the changes that it sends to a peer in some sensical fashion. IOS, for example, had a hack in it so that if it was blowing out memory, it would rate limit the number of updates that it would send to a peer. > By contrast, IGPs are *not* fundamentally rate-limited -- > retransmissions are not congestion avoiding in any IGP that > I know of. In addition to the specifications that Henk has noted, it's become very clear that an IGP implementation that does NOT rate limit itself is very likely to become unstable. Tony

1 0

Re: IGPs in use
by Sean M. Doran 15 Oct '98

15 Oct '98

| > most large nsps use is-is, but usually only to carry the routes of the bgp | > speaking interfaces. | | Really?? Not that I desire to express any prejudices, but I honestly | thought nobody used it. Everywhere that Peter went... It wouldn't surprise me to learn that everywhere that Andrew went does it too. | Does it just work better for very large networks? "Better" is difficult to nail down. It works acceptably. Few people have the experience of switching from one to the other with networks the size that they are today, and probably only they are in a position to make that kind of qualitative judgement. While IS-IS and OSPF are really similar, and Dr Li was right that on a theoretical level they both ought to scale the same way, since they are both two-level IGPs, there are some fundamental differences which may hurt during failures and may hurt going into the future. Let's start with OSPF: The per-neighbour state in OSPF is large. The lifetime field counts upwards (I have heard this decision was made just because IS-IS counts downwards). This turns out to be dumb: you can't increase the refresh interval, so dumping a full routing table into OSPF means you have to refresh one LSA every 30ish milliseconds around the clock, assuming you don't have any other instabilities, and before considering adding extra data (like AS paths) and the multiplicative effects of flooding across NBMAs and the like. Since we count upwards, we cannot reduce the refresh load. Timers must be consistent among all neighbours, so if you want to make a change to your hello/dead timers to deal with delays somewhere (in links or routers), then you have to configure your entire network all at once. The "scaling" feature of multicasting on a multicast-capable subnet saves you only 1/2 of your traffic, and LSA traffic still grows as your neighbour count grows. This is thanks to the unicast ACKs. The means of repairing a level-2 (backbone area) parition is, to put it politely, completely fucked in the head. Because OSPF was designed to be "really fast" on machines with 32-bit register operations, everything is beautifully 32-bit, which is not the greatest in terms of granularity. One of the sacrifices were fields that might let one extend the protocol, other than adding new LSA types. Whee! IS-IS on the other hand is considerably more extensible. The existence of integrated IS-IS for IP is a pleasant demonstration; the work by Henk Smit of Cisco also underlines how IS-IS can be brought forwards towards the future. Unlike in OSPF, IS-IS LSP traffic is constant independent of neighbour count. The LSP refresh timer is down-counting, and so you can make the refresh interval as long as 18 hours. This is good, because when you have a stable network, you might not want to fill it with unnecessary LSPs. The hello and dead timers are independent of each other -- you can tune them in parts of your network, rather than needing to tune them everywhere in your network all at once. The per-neighbour state in IS-IS is quite small. There are some drawbacks to IS-IS: its per-link metrics are not rich as currently defined, the LSP space is limited (but can be hacked at), and you are stuck with no more than 255 pseudonodes. All of these are likely to be hacked at and some seem pretty fixable by locking Henk and Tony (and probably Dave Katz, who did a good public analysis of IGP scaling at NANOG in Phoenix and who has done even better private analyses and gripefests --:) ) in a room for a few hours. Of course, as the number of vendors developing IS-IS grows, it too may develop fun change control issues, but it's unlikely that many changes will be made that will lead to a lack of interoperability with previous (or other) implementations. Doing it in the IETF (as with BGP) is probably not so bad an idea, either. Frankly, though, I'm hoping that rather than locking themselves in the room to "improve" IS-IS, the Obvious People lock themselves into a room and build something else instead (which they might simply call IS-IS, even if it isn't. Been there, done that, called it BGP). None of the current IGPs is free of bad scaling problems and building one that is will be hard, so don't get ready to throw out your iBGP hacks yet. Sean.

3 2

Re: IGPs in use
by Tony Li 15 Oct '98

15 Oct '98

hcb(a)clark.net (Howard C. Berkowitz) writes: > When I gave my OSPF tutorial at NANOG in June, I stressed OSPF shouldn't be > thought of purely as a 2-level hierarchy with a routing domain consisting > of an area 0 and a set of nonzero areas. Some of the OSPF scaling > problems I see, and these are probably equally likely in IS-IS, come from > people trying to put everything into a single OSPF routing domain. Aside > from performance issues, this can become a network administration nightmare. > > Splitting the interior network into several IGP routing domains, and > linking these with a backbone-of-backbones, helps both performance and > administration. The backbone group doesn't need to be concerned with LAN > installations in a POP or customer site. Depending on the particular > network, you might link IGP routing domains with: > > -- static routes > -- iBGP, putting all IGPs in a single AS > -- iBGP and eBGP in a confederation > -- Hybrid layer 2/3 techniques, such as linking IGP-routed domains > to internal layer 2 "superhubs" > > How much IGP support you need will depend on your network. A large > enterprise, or a provider of both connectivity and content, will probably > need more IGP stuff than a pure connectivity provider. Howard, Yes, those sound like a list of administrative nightmares. ;-) Wouldn't it be much easier to make use of a three or four level hierarchical IGP? Tony

2 1

Re: IGPs in use
by Sean M. Doran 15 Oct '98

15 Oct '98

Jared Mauch wrote: | I've heard arguments about the amount of cpu time | it takes to do both, but simply put: BGP is fundamentally rate-limited by virtue of running on TCP. If the receiving router or the underlying connectivity fabric is congested (slow CPU, busy or lossy network) then convergence time slows down. Also, if there is a burst of too much routing information to exchange, TCP and other mechanisms tend to smooth things out. A dramatic slow-down of convergence time is just fine if you can mitigate the effects of being unconverged (unreachability and forwarding loops, the latter of which is liable to be Bad). By contrast, IGPs are *not* fundamentally rate-limited -- retransmissions are not congestion avoiding in any IGP that I know of. So, routing information heading into a congested link or router adds to congestion, which can lead to congestion collapse (and possibly consequent re-routing, moving the problem to another point). Thus, the iBGP hack is good for you CPU-wise, since it slows down when your CPU is too busy. IS-IS and the overload bit are the only mechanism I can think of that is similar, and the overload bit is not exactly a graceful response to being (temporarily) busy. Finally, as a side-note, TCP also gives BGP the proper data-stream ordering and dramatically improves the odds that the receiving process will get exactly what was sent by the transmitting process across the network. Many IGPs have really bad behaviours in the presence of lost frames/packets, and some are very prone to processing corrupted data without realizing it's been corrupted. These behaviours can also lead to huge consumptions of CPU time. Now, design wise: | 1) If you put your backbone connected interfaces (loopbacks | serials, etc.. in your IGP [isis, ospf, whatever]) [and other good points] In other words, all you need to do is make sure that all the NEXT_HOP addresses your iBGP speakers know about are reachable with information gained independently from BGP. This means: the NEXT_HOP for any given route is directly connected, is statically configured, or is learned through another dynamic protocol. If you are clever in your address-allocation, you can have a single prefix per router, and set next-hop-self. This aggregates all the NEXT_HOP addresses on that box into one single prefix/address which its iBGP neighbours need to know how to reach. Sean.

2 1

Re: IGPs in use
by Henk Smit 14 Oct '98

14 Oct '98

Tony Li wrote: >From a theoretical perspective, they both scale exactly the same. To wit: >a two level hierarchy, with SPF within each level. Given ideal conditions, >one could reasonably build a 160,000 router network with either protocol >today without too much difficulty (400 nodes per area). > >From a practical perspective, the implementation differences and topology >issues dominate their practical applicability. When one compares the complexity of the SPF calculation itself, both OSPF and ISIS scale in a similar way. The way routing information is packaged in LSAs and LSPs is different. They way those LSAs and LSPs are flooded is different too. I think these difference are more important when one compares scalability between ISIS and OSPF. Henk.

1 0

Re: IGPs in use
by Danny McPherson 14 Oct '98

14 Oct '98

> it probably depends on what you mean by "large". If you mean "the 5 largest" > then yes, most use ISIS (but not all). If you mean the 50 largest, then > OSPF becomes much more prominent (that is, hardly anyone but the largest > ISPs use ISIS, at least from my discussions with dozens at my previous > and current employer). Likely because their networks have been 'round the longest and ISIS was the "IGP of choice" not long ago. Also, some have/had requirements for ISO CLNS ISIS routing support as well. The use of ISIS shouldn't lead folks to believe that OSPF doesn't scale well though. -danny

7 6