Re: links on the blink (fwd)
Bill Manning wrote: Just for grins, how fast do the LAN interconnects need to be? A FDDI is about as fast as a single DS-3. So it makes little sense to connect mode than one DS-3 to a BB box, since there's no way to get that traffic thru a LAN to customer access boxes in the cluster! Ideally, the capacity of LAN *attachments* (not the total bandwidth of the LAN switch!) should be about the same as the capacity of backbone links, multiplied by the number of backbone links attached to each backbone router. Of course, there are tricks to partially offload traffic from the cluster's LAN (like connecting high-speed customers directly to backbone boxes, and arranging BB topology so the transit traffic won't cross the LAN -- this works in case of "duplicate" backbone). Anyway, that boils down to the LAN switch being the single point of failure. There's no reasonable way to do "parallel" LANs with load sharing, as the number of destinations in intra-POP traffic is relatively small. Aggregations makes things even worse. All in all, the backbone router should be scalable to the point that it does not need any clustering; and redundancy is provided by "internal" duplication. --vadim
I have no idea how those two blank messages got out. My apologies.
Ideally, the capacity of LAN *attachments* (not the total bandwidth of the LAN switch!) should be about the same as the capacity of backbone links, multiplied by the number of backbone links attached to each backbone router.
Agreed, which is why the GIGAswitch works and why, once Cisco has a reasonable 100BaseT interface processor, a Grand Junction will work.
Anyway, that boils down to the LAN switch being the single point of failure.
Statistically speaking, the things that go wrong with modern machines that have no moving parts are usually software or configuration related. Since the LAN switch is mostly hardware with a little bit of bridging and spanning tree stuff, it is a lot less complicated than the average router. I think router software and configuration errors are going to pretty much drive the failure rates for the next few years. I'm not worried about the LAN switches since there are so many worse/nearer things to worry about.
All in all, the backbone router should be scalable to the point that it does not need any clustering; and redundancy is provided by "internal" duplication.
Agreed. I had a conversation over coffee a few months ago and this topic came up; my conclusion was that we were in for a really rough ride since each previous time that the Internet backbone (or the average high end customer) needed more bandwidth, there was something available from the datacom folks to fit the bill. (9.6 analog, 56K digital, 1.5M digital, 45M digital.) Furthermore, the various framing and tariff issues moved right along and have made it possible to provision Internet service in ways that would have made no sense just a few years ago (witness frame relay and SMDS.) But this time, we are well and truly screwed. The datacom people have gradually moved to the "one world" model of ATM, and have put all their effort behind the mighty Cell. They thought their grand unification of voice, data, VOD, etc, was finally a way to stop dinking around with lots of incompatible systems and oddball gateways (those of you who have tried to get inter-LATA ISDN or international T1/E1 working know what I mean). So the datacom community's answer to the Internet community is "if you're not able to get what you need from T3, use ATM which will be infinitely scalable and can be used as an end-to-end system." Who knows? If DEC can make a 90MHz NVAX chip after some PhD somewhere "proved" that the multibyte instruction encoding made certain kinds of parallism impossible and that the fastest VAX would run at 60MHz, and if Intel can make a sewing machine's CPU into a 64-bit monster with three kinds of virtual memory, then perhaps the ATM folks can figure out how to do all the lookasides and exceptions they need to do in their little itty bitty hundred-picosecond window. But I don't think so. I think we are going to have to get clever, and that we have used up our bank account of times when brute force coming out of the sky can save us. This time, gentlemen, the calvary is not coming. As much as I despise the topic Matt and Vadim have been discussing today, I think we're looking at some kind of load sharing. Probably static load sharing with no adaptation to traffic type or flow, since as Jerry pointed out a few weeks back, that's a slippery slope and a lot of people have died on it.
On Thu, 9 Nov 1995, Paul A Vixie wrote:
sky can save us. This time, gentlemen, the calvary is not coming. As much as I despise the topic Matt and Vadim have been discussing today, I think we're looking at some kind of load sharing. Probably static load sharing with no adaptation to traffic type or flow, since as Jerry pointed out a few weeks back, that's a slippery slope and a lot of people have died on it.
By "load sharing" I presume you mean some sort of TDM where you have n real lines and every n'th packet goes out on any particular line. I suppose this would be even simpler to do at the byte level if we assume that all n lines go to the same endpoint. Or do you mean something different? Michael Dillon Voice: +1-604-546-8022 Memra Software Inc. Fax: +1-604-542-4130 http://www.memra.com E-mail: michael@memra.com
By "load sharing" I presume you mean some sort of TDM where you have n real lines and every n'th packet goes out on any particular line. I suppose this would be even simpler to do at the byte level if we assume that all n lines go to the same endpoint.
Or do you mean something different?
There's one nice thing about the brick wall we're headed for, and that's that it'll nail every part of our system simultaneously. We're running out of bus bandwidth, link capacity, route memory, and human brain power for keeping it all straight -- and we'll probably hit all the walls in the same week. But no, to answer your question, I mean something different. Ganging links and doing reverse aggregation would still require unifying the flows inside each switching center. (ATM avoids this by only putting the flows back together at endpoints.) One important wall we're about to hit is in the capacity of the switching centers (routers). It's not just DS3 that's not enough; if I ask a Cisco 75xx to route among four OC12*'s it'll up and die. The boxes who are doing well at higher bit rates are the ones who don't do anything complicated. Thus the GIGAswitch and the Cascade, which each do their (limited) jobs well even though they have many more bits flowing when they're full and busy than a 75xx can handle without getting dizzy. So, what I think I mean by static load balancing would look (at the BGP level) like a bazillion NAPs and a bazillion**2 MED's to keep local traffic local. It means doing star topologies with DS3's rather than buying a virtual star via SMDS or FR or ATM from some company who doesn't have a physical star to implement it with. This assumes that we can handle 100 or 500 views of 30,000 routes inside a commonly occuring switching center. Right now we can't. If pressured, I think Sean would admit that this is at the root of his desire for "CIDR uber alles" and a six-entry routing table in his core routers. I don't consider that goal achievable for IPv4 and the allocation plans I've seen for IPv6 do not give me cause for hope. So we are going to see Ncubed enter the routing business with 1GB routers and 256 RP's and an SSE in every IP. It will cost $1M to provision a superhub and a lot of the smaller folks will just go under or become second tier customers of the few who can afford to run a defaultless core.
participants (3)
-
Michael Dillon -
Paul A Vixie -
Vadim Antonov