- Test - lists.nanog.org

Re: Links on the blink - reprise
by Sean Doran 21 Nov '95

21 Nov '95

The jury is still very much out on whether layering IP on top of frame relay with frame relay switches providing the principal long-haul switching fabric and routers making decisions about which PVCs to direct IP datagrams into is a reasonable, scalable approach. There are several advantages to the approach, vis. greater port density, especially in terms of ports-per-dollar on most FR devices than on high-end Ciscos; the (perhaps safe) bet that a medium-size user and channel of routers might have more of an influence over the development and engineering of an FR vendor's Internet-oriented products than over Cisco; that an organization with expenditure limits on bandwidth and relatively long minimum term committment lengths on cross-country circuits could do better traffic management, particularly in the presence of multiple parallel DS1s, than a similar setup using either inverse multiplexors or load balancing; and the fact that using a mesh along these lines is something that readily can interoperate with a migration to clear-channel DS3s or other technologies, or from a melting-down cross-country ethernet-over-ATM or FR-over-ATM backbone. As well, there is the probability that the users of the model you're asking about have lots and lots of points-of-presence used in an effort to reduce backhaul costs, which get quite substantial as the numbers of customers grow. Also, having lots of POPs interconnected at varying bandwidths can make designing a topology that is friendly to routers a bit more difficult in some ways. The model generally also very neatly hides the underlying topology from the IP layer, rendering traceroute essentially useless as a means of diagnosing weird time asymmetries and the like. There may be 13 physical hops between A and B on a network using this model, but traceroute might reveal only two. This has some obvious advantages to the provider keen on hiding the design of its network from "outsiders", or even from its own customers. Key to the decision is the gamble that the switching ability of FR switches will compare with the switching ability of high-end routers in the same environment. This is not a gamble I am comfortable with, particularly as I have very strong ideas of how well an SSE-equipped 7000 can switch packets in a backbone environment, and so far have only early impressions about one variety of popular FR switch in a similar environment. Also key to the decision is a bet on how well the FR-switch-based networks will scale beyond multiple DS1s and DS3s, especially in comparison to routers. There are substantial disadvantages, too. In order to take advantage of the greater port-density(-per-dollar) on FR switches right now, the end user has to use FR, which is not always practical or desirable. The monitoring and management systems of all FR switching products with which I am familiar are not router-geek-friendly. PVCs can do strange things, as can underlying circuits, which one really wants the IP-switching layer to recognize and adapt to; at present there is effectively no practical means to do this in anything but a failure of a PVC. There is a serious management nightmare in maintaining anything approaching a full mesh in a large network, and it worsens if one pushes the routing decisions right out to the edges of a given network. Maintaining lots of PVCs has many of the well-known disadvantages of maintaining lots of standard iBGP neighbours or any other system in which involves configuration length, complexity or both approaching or equivalent to N**2. There are a number of other deeply technical questions about the model which remain the subject of occasionally heated speculation, but the bottom line is that nobody really knows how well it will work as this type of network grows really big, or even as heavily trafficked as SprintLink and InternetMCI. Finally, pushing the envelope of technology is hard work at all times; I believe a pragmatic approach is to push only n things at any given time where n is ideally the smallest possible number. Adding frame relay switching technology into the mix of things that backbone NSPs have to deal with already strikes me as asking for trouble. | PSI and UUNET have apparently been following this strategy with their | backbones for some time. Netcom has also come on board. Your information may be better than mine, but I gathered that UUNET has just recently migrated over to this model from the previous MFS-supplied technologies they had been using. Frankly, neither netcom nor PSI does sufficient traffic to learn anything interesting about how the model behaves under stressful conditions. Alternet's experience over the next few months will be enlightening. Sean.

3 2

Re: Links on the blink - what will/should mci & sprint do?
by Sean Doran 20 Nov '95

20 Nov '95

| Can a 7500 being fed a single routing view switch IP packts among 6 OC3 AIP | cards running point-to-point AIP-AIP OC3 links to other 7500s at reasonable | speeds, one wonders. If you change "Can" to "Will", this is unclear right now. What apparently is clear (at least to me) after much beating up of the 7500 series beta boxes we got to play with in ICM land and the 7500 series software in the Cisco test lab, is that SprintLink's plan to use 7500s in the four new cities (Atlanta, Kansas City, Cheyenne and Seattle) that will be on-line imminently is reasonable. Firstly the 7500s, even without the smarter technology cards in development by Cisco will currently handle the load requirements of those cities. Secondly since everything currently is done in the RSPs, we have opportunities to try quick and experimental changes to the IOS software at very low-levels that don't exist with the 7000+SSP (but presumably if the proofs-of-concept work really well we can get a port done). Thirdly, there are some very cunning ideas being proposed by various hardware people for making the 7500 do interesting things on the switching front that very probably will get the router to deal with the kinds of loads you're asking about. Finally, the key advantage of the 7500 is the much better ports-per-dollar and ports-per-rack numbers over the 7000s using current technology, and even better numbers as Cisco rolls out the type of channelizing hardware we have been asking for. Specific performance questions wrt the Cisco 7500 series should probably be asked through your favourite Cisco information channels. I'm pretty sure everyone's going to find out how well 7500s work in practice, backbone-wise, Real Soon Now. The big risk, naturally, is that the platform just doesn't hold up in practice, which would force us to do a redesign along the lines of what Gordon has been asking about. Frankly, though, I'm not too worried. Many very smart people have put alot of work into making sure the 7500 doesn't become a complete corporate embarassment for Cisco. Sean.

4 3

Re: Links on the blink - what will/should mci & sprint do?
by Sean Doran 20 Nov '95

20 Nov '95

| Sounds like there is a need for a good ip switch. Something simple, | very fast, and low cost that you can download "static" routes to. It's called an SSP. Sean.

4 4

Re: Links on the blink - reprise
by Sean Doran 20 Nov '95

20 Nov '95

Curtis - | The brick wall is that a particular piece of equipment from a particular | vendor that a lot of service providers have made a large investment in | doesn't really perform all that well in the real world. Please allow me to mitigate your politics with a dose of reality. sl-dc-8.sprintlink.net, a now-fairly-old Cisco 7000 with one of the first four 2MB SSP boards ever shipped outside Cisco's doors has been observed to switch 125kpps through several interfaces over a 15 minute period several times in the past three weeks. The bottleneck is not in terms of switching capacity nor is it in terms of throughput across its backplane at present. The latter issue is looming, but we're simply not there yet. There have been substantial problems with respect to convergence times. Many of these have been ameliorated with experimental code now deployed throughout SprintLink and ICM, which does selective packet dropping to assist convergence rather than having the box keel over dead process-switching packets when the SSE cache is being completely repopulated. We are no longer hovering close to the practical limits of the current limitation, and are not very near the reasonable maximum for the current platform. This is not to say that we have all that much breathing-room, but this and other developments in the works does and will buy us much more time than we would gain by moving towards a system of the kind other providers appear to favour. The immediate danger is still in terms of BGP routing on defaultless routers, and we are all now keenly aware of that and I believe that even you have accepted that despite available alternatives like dedicated route servers, we must CIDRize or die. I am shocked and surprised to see you insinuating that my colleagues here and elsewhere and I simply don't know what the hell we're doing or what we're installing in our backbones. We are all too keenly aware that the growth of much of the Internet is outpacing the ability to build correctly-sized equipment and is forcing most of us to design plans for the deployment of large, scalable and for now over-engineered routers to cope with the current trends in traffic increases which do not look like they're going to shrink before the end of 1997. Not all of us are in the enviable position of being able to sit back and enjoy the air of superiority that seems to accompany being the chief commentator for a once-important and formerly leading-edge backbone. I think that you will find that at least two organizations who aren't in that position have been pushing Cisco very hard on the rather ingenious idea of developing an ISP test lab wherein real problems can be reproduced and studied, and where the characteristics of the current and in-development router technologies with respect to heavy-duty ISP environments can better be understood. You'll probably say that this is all familiar and old-hat, but kindly put your sneer aside and remember that the NSFNET backbone service ended a doubling and a quarter ago with respect to the size of the Internet then and now. These problems are difficult, nobody has complete answers, and "gee, just turn off a whole bunch of customers and you'll be fine" doesn't strike me as a plausible solution for Sprint, MCI, or AlterNet. Sean. P.S.: You might want to consider some "Sprint-did-it-firsts" which developed both within ICM and SprintLink vis a vis Cisco and general router technology deployment: 7000s, 64Mb RPs, BGP4, SSPs, 2MB SSPs, 7500s, reprioritization of forwarding vs other tasks, selective packet drop, and so on and so forth. Vadim Antonov and Peter Lothberg were and are never idle, and I fully intend to carry on the tradition of pushing useful new technology into the field as fast as it is available, because quite frankly, we need it all. DS3? OC3? Hah. You ain't seen nothing yet, baby.

2 1

Kind Words of Caution
by Tim Bass 20 Nov '95

20 Nov '95

Nanog, Before we begin to retire into the slow holiday mode of operation, I would like to offer this simple thought as a word of caution for those engaged in internetworking discussions. Last week, someone asked a valid question about pending and potential problems with aggregation and address space portability issues. The initial response was a statement validating that concern with hopes of a developing a fruit bearing discussion on this very controversial " in the closet " topic. The public process then followed the common disturbing direction that always clouds the issues (especially on the potential non-technical problems of aggregation) and effectively destroyed any prospect of meaningful interchange. I kindly caution many on this list, as internetworking topics are discussed related to Internet operations, *not* to be seduced by those skillful at pedantic discussions that obfuscate any controversial position that they disagree. Especially on the Internet, many are seduced by those who obfuscate the facts by pedantic side discussions and this important list is no exception. For those of you who are not familar with the English word "pedant" I kindly include a short blurb from Websters dictionary. pedant, n. 1. a person who makes an excessive or inappropriate show of learning. 2. a person who overemphasizes rules or minor details.... I kindly caution those unfamiliar with pedantic postings and the use of pedanticism on mailing lists to understand when to recognize these email postings and responses that tend to be pedantic in nature. Pedanticism can be used very effectively to seduce the public into refocusing the discussion from a logical argument that cannot be won nor argued convincingly, to one that is tangential to the topic. Often these pedantic responses clearly and effectively allow the poster to change the subject and then demonstrate their "knowledge" in a tangential or related subject area which they are well versed. As you can see, this clearly obfuscates the original concerns and issues. As we approach 1996, I only caution out of kind regard for tangible progress in building internet services to practice recognizing when 'hawks and eagles' and other high flying posters enjoy easily seducing the general audience with pedanticism, either intentional or unintenional. Practicing, in your own mind, to discern and recognize pedantic email discussions helps all of us who care about enjoying 'meaningful, tangible discussions' to think with clear comprehension. It is very well documented in civilized history that pure intellect and logic without attempting to include elements of patience and kindness, is *not* wisdom. Wisdom is the pursuit and practice of intellect with patience and understanding of others. For all of us, especially me, the pursuit of changing raw technical excellence to refined wisdom is a very challenging life pursuit. I work daily to improve my self-awarenedd in this area because, as an electrical engineer, I once believed that the 'intellect of building systems' was more important than the 'less intelligent ' society we engineer systems to benefit. This manner of thinking is clearly flawed and those, including myself, who are attracted to 'higher education' and intellectual pursuits have a nature human tendency to be seduced by this incomplete position. What is less common is to find those same individuals who recognize this and tend to pursuit wisdom as well as intellect. Kind Regards, Tim PS: Please enjoy safe travel during the holiday weekend. I'm heading out to stay with family in Tennessee and face a long 10 hour drive as well. +--------------------------------------------------------------------------+ | Tim Bass | #include<campfire.h> | | Principal Network Systems Engineer | for(beer=100;beer>1;beer++){ | | The Silk Road Group, Ltd. | take_one_down(); | | | pass_it_around(); | | http://www.silkroad.com/ | } | | | back_to_work(); /*never reached */ | +--------------------------------------------------------------------------+

3 2

Re: Kind Words of Caution
by Sean Doran 19 Nov '95

19 Nov '95

WhatEVER. Sean.

1 0

RE: Links on the blink - reprise
by Paul Ferguson 19 Nov '95

19 Nov '95

At 09:17 AM 11/19/95 -0500, Jeffrey P. Oliveto wrote: > >DS3/DS1 Backbone/Trunk capacity planning principles, whether across a Frame >Relay Backbone or Cisco 7000 hdlc trunk network are still the same. It's >just as easy to over configure DS3/DS1 Cisco HDLC trunks as Frame Relay >trunks. > I would have a tendency to disagree with you here, but que sera, sera. >Potentially at issue here is not Frame Relay networks as a transport but that >a Cisco 7000 can not scale properly to support 120+ end-users. :-) > >Modern Frame Relay switches: > >1) have sub-msec latency >2) can support multiple trunks at DS3+ (to include ATM) >3) are not burdened with processing any of the IP layer 3 nor routing overhead >4) because of 3 have a cost per port that is 300 to 400% less than a Cisco 7000 >5) can have it's backbone shared across multiple services thereby reducing both >capitalization and bandwidth expense >6) allow ISP to pass the cost savings on to customers > > I partially agree with your points above, but still maintain that it is much easier to sloppily engineer a frame-relay network than one consisting of point-to-point links. My $.02. - paul

1 0

Re: Links on the blink - what will/should mci & sprint do?
by sob＠academ.com 19 Nov '95

19 Nov '95

Sean Doran writes: > No, what a disaster would mean is replacing backbone 7500s > with 7000s Sean, are there actually backbone 7500s in production? I thought that was yet to come. Did I misread something here? -- Stan | Academ Consulting Services |internet: sob(a)academ.com Olan | For more info on academ, see this |uucp: {mcsun|amdahl}!academ!sob Barber | URL- http://www.academ.com/academ |Opinions expressed are only mine.

2 1

RE: Links on the blink - reprise
by Jeffrey P. Oliveto 19 Nov '95

19 Nov '95

Paul; You state: At 05:47 PM 11/17/95 -0500, Sean Doran wrote: > >There are substantial disadvantages, too. > >In order to take advantage of the greater >port-density(-per-dollar) on FR switches right now, the >end user has to use FR, which is not always practical >or desirable. > One of the reasons why end users find frame-relay undesireable is that they cannot be assured that their provider is not grossly oversubscribed on PVC-per-port density. When you buy a T1 private line, you can be assured that you're not sharing it with 120 other end-users. :-) DS3/DS1 Backbone/Trunk capacity planning principles, whether across a Frame Relay Backbone or Cisco 7000 hdlc trunk network are still the same. It's just as easy to over configure DS3/DS1 Cisco HDLC trunks as Frame Relay trunks. Potentially at issue here is not Frame Relay networks as a transport but that a Cisco 7000 can not scale properly to support 120+ end-users. :-) Modern Frame Relay switches: 1) have sub-msec latency 2) can support multiple trunks at DS3+ (to include ATM) 3) are not burdened with processing any of the IP layer 3 nor routing overhead 4) because of 3 have a cost per port that is 300 to 400% less than a Cisco 7000 5) can have it's backbone shared across multiple services thereby reducing both capitalization and bandwidth expense 6) allow ISP to pass the cost savings on to customers ----------------------------------------------------------------------------- Jeff Oliveto | Phone: +1.703.760.1764 Sr.Mgr Opns Technical Services | Fax: +1.703.760.3321 Cable & Wireless, Inc | Email: joliveto(a)cwi.net 1919 Gallows Road | URL: http://www.cwi.net/ Vienna, VA 22182 | NOC: +1.800.486.9999 -----------------------------------------------------------------------------

1 0

Re: Links on the blink - what will/should mci & sprint do?
by Sean Doran 19 Nov '95

19 Nov '95

icm-dc-3.icp.net is a 7505, as is one of its counterparts on the other side of the Atlantic. All routers in Kansas City, Seattle, Cheyenne and Atlanta are 7513s, although they are not yet up and operational. There are a largeish number of 7513s deployed in another parallel network that is engineered out of our group. No points for guessing what that's for. :) Sean.

1 0