Re: NAP/ISP Saturation WAS: Re: Exchanges that matter...
From: Jim Van Baalen <vansax@atmnet.net> I have a question that fits this topic. Why does everybody seem to be so sold on Gigaswitch based Xchange points?
A pretty good reason: it worked a lot better than everything else tried! As a bit of history, the only NAP that worked was with ethernet, which seemed a lot faster to most folks than the T1 links coming into the NAPs. But, the implementation was terribly flaky as a MAN technology. Then, ethernet got too congested, and folks moved to FDDI. Then, FDDI got congested and GigaSwitches were put in. They were the only thing available, and they made everything work better. My guess is that the next step will toss the LAN/MAN model for exchange GigaRouters directly connected by OC-3c/STM-1 and OC-12c/STM-4 WAN links -- but that's only a guess. It seems to be working. We are driven by things that actually work.
Based on membership and traffic it appears that there is still a stigma associated with Xchanges (PBnap and AADS for example) that have chosen different architectures. It was also my impression that people were much more critical of these "other" NAPS at the recent NANOG than SprintNAP and the MAEs.
That's because the ATM NAPs were started at the same time as MAE-East, but didn't work! They all took more than an extra year to get working. Oh, yeah, they cost a lot of money for some projects that really couldn't afford it, and it all went down the drain. Folks have a tendency to be critical of stuff with a history of failure.
In addition, with new line cards due out early next year, the BPXs will support ABR and, relatively speaking, huge buffers at high density OC3 and 2 port OC12.
Well, since they don't exist, why would anyone bank on deploying them? Let us know how well they work in a year or two. Meanwhile, PPP/SONET has been deployed for over 6 months at these speeds, and we are already getting experience with it. Yep, needs bigger buffers on those Ciscos, and PMC-Sierra didn't follow the SONET spec exactly, but ... experience is a much better teacher than promises. WSimpson@UMich.edu Key fingerprint = 17 40 5E 67 15 6F 31 26 DD 0D B9 9B 6A 15 2C 32 BSimpson@MorningStar.com Key fingerprint = 2E 07 23 03 C5 62 70 D3 59 B1 4F 5E 1D C2 C1 A2
From: Jim Van Baalen <vansax@atmnet.net> I have a question that fits this topic. Why does everybody seem to be so sold on Gigaswitch based Xchange points?
A pretty good reason: it worked a lot better than everything else tried!
As a bit of history, the only NAP that worked was with ethernet, which seemed a lot faster to most folks than the T1 links coming into the NAPs. But, the implementation was terribly flaky as a MAN technology.
Then, ethernet got too congested, and folks moved to FDDI. Then, FDDI got congested and GigaSwitches were put in. They were the only thing available, and they made everything work better.
Makes sense.
My guess is that the next step will toss the LAN/MAN model for exchange GigaRouters directly connected by OC-3c/STM-1 and OC-12c/STM-4 WAN links -- but that's only a guess. It seems to be working. We are driven by things that actually work.
Which leads to my question, since the Gigaswitch NAPs are not scaling to today's traffic shouldn't we be looking for something new that can? It may not be an ATM solution, but that is one obvious alternative and I think that there now exist tested ATM NAPs that, though they are not carrying as much traffic today, should scale more gracefully.
Based on membership and traffic it appears that there is still a stigma associated with Xchanges (PBnap and AADS for example) that have chosen different architectures. It was also my impression that people were much more critical of these "other" NAPS at the recent NANOG than SprintNAP and the MAEs.
That's because the ATM NAPs were started at the same time as MAE-East, but didn't work! They all took more than an extra year to get working.
Oh, yeah, they cost a lot of money for some projects that really couldn't afford it, and it all went down the drain.
I don't question that ATM was a much less mature technology than FDDI a couple of years ago. On the other hand, ATM is much more scalable and has matured considerably in the past few years.
Folks have a tendency to be critical of stuff with a history of failure.
In addition, with new line cards due out early next year, the BPXs will support ABR and, relatively speaking, huge buffers at high density OC3 and 2 port OC12.
Well, since they don't exist, why would anyone bank on deploying them? Let us know how well they work in a year or two.
Meanwhile, PPP/SONET has been deployed for over 6 months at these speeds, and we are already getting experience with it. Yep, needs bigger buffers on those Ciscos, and PMC-Sierra didn't follow the SONET spec exactly, but ...
experience is a much better teacher than promises.
Yes, and we have a lot more experience with OC3 and above devices running ATM than PPP/SONET. Jim
Let me add one more word to this discussion. In case of simple and solid FDDI or Ethernet protocol it's not big chance if the whole switch would be failed down by one crazy interface card or one crazy router. In case of ATM nobody can protect the whole ATM system from being failed down due to some software bug since 1 / 5 after system was installed, because the number of different featires in ATM network and ATM interface is more then 10 - 100 times more in comparasion with FDDI. This means - if your FDDI switch works fine just now, it's more than 99% it'll work next 2 years (may be it'll be saturated but would not crash totally); in case of ATM it's more than 20% the whole system would be crashed by some new neighboar with some new ATM software... It's not my experience, but one of our partners are debbugging simple direct ATM link just about 3 months - there is 3 vendors (ATM provider, ATM's provider vendor, CISCO) and it's not possible to determine why this link loss more than 50% of packets sometimes. This is due to ATM is too complex system... --- Aleksei Roudnev, Network Operations Center, Relcom, Moscow (+7 095) 194-19-95 (Network Operations Center Hot Line),(+7 095) 239-10-10, N 13729 (pager) (+7 095) 196-72-12 (Support), (+7 095) 194-33-28 (Fax)
Let me add one more word to this discussion. In case of simple and solid FDDI or Ethernet protocol it's not big chance if the whole switch would be failed down by one crazy interface card or one crazy router.
In some sense isn't this what is occuring with head of line blocking?
In case of ATM nobody can protect the whole ATM system from being failed down due to some software bug since 1 / 5 after system was installed, because the number of different featires in ATM network and ATM interface is more then 10 - 100 times more in comparasion with FDDI. This means - if your
In that case we should get rid of all of our routers. They certainly support much more complexity that ATM switches.
FDDI switch works fine just now, it's more than 99% it'll work next
Really? I am surprised that everybody seems to be of the opinion that the Gigaswitches are not causing any problems. This doesn't seem to me to be consistent with early discussions from this thread.
2 years (may be it'll be saturated but would not crash totally); in case of ATM it's more than 20% the whole system would be crashed by some new neighboar with some new ATM software...
ATM PVCs, as used by the current ATM NAPs, don't require much sophistication. They are basically just hardwired circuits.
It's not my experience, but one of our partners are debbugging simple direct ATM link just about 3 months - there is 3 vendors (ATM provider, ATM's provider vendor, CISCO) and it's not possible to determine why this link loss more than 50% of packets sometimes. This is due to ATM is too complex system...
I have not had this experience, but I would guess that it is due to a lack of debugging tools. These exist, but good ones are expensive. Good switches do provide some helpful debugging, but there is certainly work to be done on this front. I would add that FDDI was pretty lousy technology 5 or so years ago. Jim
> > Let me add one more word to this discussion. In case of simple and solid > > FDDI or Ethernet protocol it's not big chance if the whole switch would be > > failed down by one crazy interface card or one crazy router. > > In some sense isn't this what is occuring with head of line blocking? Line blocking is a throughput problem, it's not protocol incompability. > > In case of ATM nobody can protect the whole ATM system from being failed down due > > to some software bug since 1 / 5 after system was installed, because > > the number of different featires in ATM network and ATM interface is more > > then 10 - 100 times more in comparasion with FDDI. This means - if your > > In that case we should get rid of all of our routers. They certainly support > much more complexity that ATM switches. You are right, and if you'll install some new router (not Cisco, not Gigarouter based on the old, solid and well tested gated) into Internet core backbone - you'll get good chance to see your router crashes every 10 minutes. We had this few times ago when we mixed different routers into internet core network. Now we are using CISCO's, and If some fluktuation in the Internet would make clean new IOS's bug - it would be not my router which crashes first, but the the router not far from the source of fluktuation. It's amazing but it's true. > > FDDI switch works fine just now, it's more than 99% it'll work next > > Really? I am surprised that everybody seems to be of the opinion that the > Gigaswitches are not causing any problems. This doesn't seem to me to be > consistent with early discussions from this thread. I have not opinion about Gigaswitch, but if FDDI protocol is described by 20 sheets of paper and ATM by 20,000 sheets - the number of hidden bugs have the same proportion (software bugs I mean). > > 2 years (may be it'll be saturated but would not crash totally); in case > > of ATM it's more than 20% the whole system would be crashed by some new > > neighboar with some new ATM software... > > ATM PVCs, as used by the current ATM NAPs, don't require much sophistication. > They are basically just hardwired circuits. I am not shure was is PVC or some other .VC in the case I have described, but I suspect it was PVC; but it have caused a lot of problems anyway. Hope nobody promise LANE to be used by NAP's? Shurely PTT's and great vendors who just wast a lot of money for ATM solution would not allow ATM to die, and I hope it'll be one of (just as Gigabit Ethernet, and so on) solutions for the future NAP's. Sorry, there in Russia we have not filled even FDDI (not GigaSwitch) yet, and this do not allow me to make any predictions... But if protocol 1 is 1000 times more complex than protocol 2, and we;ll mix 10 different providers at the NAP-1 based on protocol 1 and NAP-2 based on the protocol 2 - guess how NAP-1 and NAP-2 will work? --- Aleksei Roudnev, Network Operations Center, Relcom, Moscow (+7 095) 194-19-95 (Network Operations Center Hot Line),(+7 095) 239-10-10, N 13729 (pager) (+7 095) 196-72-12 (Support), (+7 095) 194-33-28 (Fax)
> > > > Let me add one more word to this discussion. In case of simple and solid > > > FDDI or Ethernet protocol it's not big chance if the whole switch would be > > > failed down by one crazy interface card or one crazy router. > > > > In some sense isn't this what is occuring with head of line blocking? > Line blocking is a throughput problem, it's not protocol incompability. > > > > In case of ATM nobody can protect the whole ATM system from being failed down due > > > to some software bug since 1 / 5 after system was installed, because > > > the number of different featires in ATM network and ATM interface is more > > > then 10 - 100 times more in comparasion with FDDI. This means - if your > > > > In that case we should get rid of all of our routers. They certainly support > > much more complexity that ATM switches. > You are right, and if you'll install some new router (not Cisco, not Gigarouter > based on the old, solid and well tested gated) into Internet core backbone > - you'll get good chance to see your router crashes every > 10 minutes. We had this few times ago when we mixed different routers into > internet core network. Now we are using CISCO's, and If some > fluktuation in the Internet > would make clean new IOS's bug - it would be not my router which crashes first, > but the > the router not far from the source of fluktuation. > > It's amazing but it's true. > > > > FDDI switch works fine just now, it's more than 99% it'll work next > > > > Really? I am surprised that everybody seems to be of the opinion that the > > Gigaswitches are not causing any problems. This doesn't seem to me to be > > consistent with early discussions from this thread. > I have not opinion about Gigaswitch, but if FDDI protocol is described by > 20 sheets of paper and ATM by 20,000 sheets - the number of hidden bugs > have the same proportion (software bugs I mean). > > > > 2 years (may be it'll be saturated but would not crash totally); in case > > > of ATM it's more than 20% the whole system would be crashed by some new > > > neighboar with some new ATM software... > > > > ATM PVCs, as used by the current ATM NAPs, don't require much sophistication. > > They are basically just hardwired circuits. > I am not shure was is PVC or some other .VC in the case I have described, > but I suspect it was PVC; but it have caused a lot of problems anyway. > Hope nobody promise LANE to be used by NAP's? > > Shurely PTT's and great vendors who just wast a lot of money for ATM solution > would not allow ATM to die, and I hope it'll be one of (just as Gigabit Ethernet, > and so on) solutions for the future NAP's. Sorry, there in Russia we have > not filled even FDDI (not GigaSwitch) yet, and this do not allow me to > make any predictions... But if protocol 1 is 1000 times more complex than > protocol 2, and we;ll mix 10 different providers at the NAP-1 based > on protocol 1 and NAP-2 based on the protocol 2 - guess how NAP-1 and > NAP-2 will work? If you remove UNI, PNNI, LANE, CLIP, etc. from the equation ATM gets pretty simple. Some degree of testing in a live environment has already occured. As was pointed out earlier there were many interoperabily issues between Cisco ATM interfaces (including the lack of DS3 ATM) and the ATM NAPs early on. My questions are: 1) Are the Gigaswitches failing at the most congested NAPs? 2) Do any viable alternatives exist today? 3) If the answers to 1 and 2 are yes and no respectively are we prepared to abandon NAPs all together. Jim
participants (3)
-
alex＠relcom.EU.net -
Jim Van Baalen -
William Allen Simpson