Dear Mr. Antonov, Thanks for taking time to enter this little spat about Internet collapses, the growing importance of NANOG, and my cluelessness. You wrote:
I was in a backbone engineer's skin for quite a few years, and "hops" per se never were a problem. In fact, store-and-forward delays are a mere fraction of wire propagation delays -- do a traceroute coast-to-coast, look at delays and calculate how it relates to distance divided by speed of light. Indeed, you're the first person concerned with the growth of diameter (which is, BTW, logarithmic to size of the network).
Perhaps I am confusing terms here. How can it be a fact that "store-and-forward delays are a mere fraction of wire propagation delays?" I don't think so. Check me on this: Packets travel over wires at large fractions of the speed of light, but then sadly at each hop they must be received, checked, routed, and then queued for forwarding. Do I have that right? Forget checking, routing, and queueing (ha!), and you get, I think, that store and forward delay is roughly proportional to the number of hops times packet length divided by circuit speed (N*P/C). For 10 hops of a thousand bit packet at Ethernet speed, that would be 1 ms, or a couple hundred miles of prop delay. Check me on this, one of us might be off by several orders of magnitude. But at 30 hops of thousand byte packets at T1 speeds, that's, what? 4,000 miles of prop delay. A mere fraction? OK, maybe soon the entire Internet backbone(s) will be ATM at 622Mbps, which would certainly knock some of the wind out of N, P, and C. Soon? But of course, getting back to 1996, N*P/C doesn't count checking, routing, and queueing -- queueing gets to be a major multiple with loading. Oh, I forgot retransmission delays too, at each hop. And I forgot the increasing complications of route propagation as hops increase... If I am, as you say, the first person to be concerned with the growth of Internet diameter, which I doubt, then I deserve a medal. Or is my arithmetic wrong? Ease my cluelessness. /Bob Metcalfe, InfoWorld At 6:44 PM 4/2/96, Vadim Antonov wrote:
Received: by ccmail from lserver.infoworld.com
From avg@postman.ncube.com X-Envelope-From: avg@postman.ncube.com Received: from postman.ncube.com by lserver.infoworld.com with smtp (Smail3.1.29.1 #12) id m0u4IvH-000wq4C; Tue, 2 Apr 96 19:07 PST Received: from butler.ncube.com by postman.ncube.com (4.1/SMI-4.1) id AA19923; Tue, 2 Apr 96 18:42:20 PST Received: from skynet.ncube.com by butler.ncube.com (5.0/SMI-SVR4) id AA02534; Tue, 2 Apr 1996 18:40:46 +0800 Date: Tue, 2 Apr 1996 18:40:46 +0800 From: avg@postman.ncube.com (Vadim Antonov) Message-Id: <9604030240.AA02534@butler.ncube.com> To: bob_metcalfe@infoworld.com, jerry@fc.net Subject: RE: NANOG Cc: letters@infoworld.com, nanog@merit.edu Content-Length: 2913
Bob Metcalfe wrote:
Note, I have never predicted "the death of the Internet," only catastrophic collapse(s) during 1996, which is "a good calibration" of the rest of your objections (below).
One does not need to be Nostradamus to predict that s*t happens. It happened in the past, many times, too. Like when me and Sean installed a just-baked SSE into a DC box and it looked fine but screwed nearly all connectivity to Europe for few hours when we were trying to figure out what was going on. Or when FIX-E<->ICM-DC Bell Atlantic's DS-3 was flapping like mad when moon was in the wrong phase and BA did nothing to fix it for months. Or when some sequence of 1s and 0s was triggering some bulls*t alarms in Sprint fiber network so causing shutdowns on the entire OC-24 trunk. Or when a bogus static route in a Sprint's box was causing ANS's version of gated to go banana and drop BGP sessions. Or many many more occasions when "Bysantine-mode failure" becomes ugly reality in the middle of the night so causing more than few people to be dragged out of beds.
As long as Internet technology is freaking bleeding edge and operators are in the "code of the day" club catastrophes are bound to happen.
Jerry, Jerry, Jerry, the problem is not that the Internet's chief 100 engineers, whoever they are, fail to report their problems to me, it's that they (you?) fail to report them to anybody, including to each other, which is half our problem.
That is simply not true. The backbone engineering society is tightly knit and quite often backbone engineers are simply personal friends. I certainly never had a problem with people refusing to fix problems within their domains (well, PSI's TWD is not an operational problem). The organization-level corrdination is often broken at operators level, but that is merely a function of severe shortage of qualified personnel and inadequate compensation for the high-stress job.
Settlements, "wrong on the face?" Or are you just too busy busy busy defensive to argue?
Before you talk of settlements answer the simple question -- a packet travelled from provider A to provider B. Who should pay to whom? Then, please, stop perpetuating nonsense.
So, you say, increasing Internet diameters (hops) are only of concern to whiners like me? There are no whiners LIKE me. I am THE whiner. And hops ARE a first class problem, Jerry, or are you clueless about how store-and-forward packet switching actually really works?
I was in a backbone engineer's skin for quite a few years, and "hops" per se never were a problem. In fact, store-and-forward delays are a mere fraction of wire propagation delays -- do a traceroute coast-to-coast, look at delays and calculate how it relates to distance divided by speed of light. Indeed, you're the first person concerned with the growth of diameter (which is, BTW, logarithmic to size of the network).
--vadim
______________________________________________ ______________________________________________ Dr. Robert M. ("Bob") Metcalfe Executive Correspondent, InfoWorld and VP Technology, International Data Group Internet Messages: bob_metcalfe@infoworld.com Voice Messages: 617-534-1215 Conference Chairman for ACM97: The Next 50 Years of Computing San Jose Convention Center March 1-5, 1997 ______________________________________________ ______________________________________________
Actually, router hops are a problem when the packet times (your P/C) are on the order of bit propagation times. So, for 1.5 Mbps and approx. 1.5 Kbit packets, P/C is 1 msec. for 45 Mbps, P/C is 33 usec. You can see that routers for T1 and below hops introduce a (minimum) latency on the order of prop delay. T3 and beyond should not really be a problem. So, Vadim, you are right in the context of the Sprintlink backbone, and other DS3 and higher backbones. Of course, most large continental NSP's have DS3 backbones. Also, these networks are relatively small (in number of hops). However, a lot of leaf connectivity is not at DS3 and the more such hops you have in your path, the higher your path delay becomes. (A separate discussion is whether and how much this additional latency hurts.) As an aside, the NSFnet T1 backbone introduced 6 msec. or so of per hop latency, limiting each router to approx. 150-160 pps. Additional NSS hops really hurt us there! If a router is to switch DS3 at line speeds, it has to process 2*45 Mbps, or 90 Mbps per adapter. At 1.5 Kbit/packet, this is approx. 16 usec./packet. You can see why on-card embedded systems came into vogue ... Cheers, -- Bilal (who kinda remembers that backbone engineer skin thing - 1987-1992.)
Dear Mr. Antonov,
Thanks for taking time to enter this little spat about Internet collapses, the growing importance of NANOG, and my cluelessness.
You wrote:
I was in a backbone engineer's skin for quite a few years, and "hops" per se never were a problem. In fact, store-and-forward delays are a mere fraction of wire propagation delays -- do a traceroute coast-to-coast, look at delays and calculate how it relates to distance divided by speed of light. Indeed, you're the first person concerned with the growth of diameter (which is, BTW, logarithmic to size of the network).
Perhaps I am confusing terms here. How can it be a fact that "store-and-forward delays are a mere fraction of wire propagation delays?" I don't think so. Check me on this:
Packets travel over wires at large fractions of the speed of light, but then sadly at each hop they must be received, checked, routed, and then queued for forwarding. Do I have that right?
Forget checking, routing, and queueing (ha!), and you get, I think, that store and forward delay is roughly proportional to the number of hops times packet length divided by circuit speed (N*P/C).
For 10 hops of a thousand bit packet at Ethernet speed, that would be 1 ms, or a couple hundred miles of prop delay. Check me on this, one of us might be off by several orders of magnitude.
But at 30 hops of thousand byte packets at T1 speeds, that's, what? 4,000 miles of prop delay. A mere fraction?
OK, maybe soon the entire Internet backbone(s) will be ATM at 622Mbps, which would certainly knock some of the wind out of N, P, and C. Soon?
But of course, getting back to 1996, N*P/C doesn't count checking, routing, and queueing -- queueing gets to be a major multiple with loading. Oh, I forgot retransmission delays too, at each hop. And I forgot the increasing complications of route propagation as hops increase...
If I am, as you say, the first person to be concerned with the growth of Internet diameter, which I doubt, then I deserve a medal. Or is my arithmetic wrong? Ease my cluelessness.
/Bob Metcalfe, InfoWorld
At 6:44 PM 4/2/96, Vadim Antonov wrote:
Received: by ccmail from lserver.infoworld.com
From avg@postman.ncube.com X-Envelope-From: avg@postman.ncube.com Received: from postman.ncube.com by lserver.infoworld.com with smtp (Smail3.1.29.1 #12) id m0u4IvH-000wq4C; Tue, 2 Apr 96 19:07 PST Received: from butler.ncube.com by postman.ncube.com (4.1/SMI-4.1) id AA19923; Tue, 2 Apr 96 18:42:20 PST Received: from skynet.ncube.com by butler.ncube.com (5.0/SMI-SVR4) id AA02534; Tue, 2 Apr 1996 18:40:46 +0800 Date: Tue, 2 Apr 1996 18:40:46 +0800 From: avg@postman.ncube.com (Vadim Antonov) Message-Id: <9604030240.AA02534@butler.ncube.com> To: bob_metcalfe@infoworld.com, jerry@fc.net Subject: RE: NANOG Cc: letters@infoworld.com, nanog@merit.edu Content-Length: 2913
Bob Metcalfe wrote:
Note, I have never predicted "the death of the Internet," only catastrophic collapse(s) during 1996, which is "a good calibration" of the rest of your objections (below).
One does not need to be Nostradamus to predict that s*t happens. It happened in the past, many times, too. Like when me and Sean installed a just-baked SSE into a DC box and it looked fine but screwed nearly all connectivity to Europe for few hours when we were trying to figure out what was going on. Or when FIX-E<->ICM-DC Bell Atlantic's DS-3 was flapping like mad when moon was in the wrong phase and BA did nothing to fix it for months. Or when some sequence of 1s and 0s was triggering some bulls*t alarms in Sprint fiber network so causing shutdowns on the entire OC-24 trunk. Or when a bogus static route in a Sprint's box was causing ANS's version of gated to go banana and drop BGP sessions. Or many many more occasions when "Bysantine-mode failure" becomes ugly reality in the middle of the night so causing more than few people to be dragged out of beds.
As long as Internet technology is freaking bleeding edge and operators are in the "code of the day" club catastrophes are bound to happen.
Jerry, Jerry, Jerry, the problem is not that the Internet's chief 100 engineers, whoever they are, fail to report their problems to me, it's that they (you?) fail to report them to anybody, including to each other, which is half our problem.
That is simply not true. The backbone engineering society is tightly knit and quite often backbone engineers are simply personal friends. I certainly never had a problem with people refusing to fix problems within their domains (well, PSI's TWD is not an operational problem). The organization-level corrdination is often broken at operators level, but that is merely a function of severe shortage of qualified personnel and inadequate compensation for the high-stress job.
Settlements, "wrong on the face?" Or are you just too busy busy busy defensive to argue?
Before you talk of settlements answer the simple question -- a packet travelled from provider A to provider B. Who should pay to whom? Then, please, stop perpetuating nonsense.
So, you say, increasing Internet diameters (hops) are only of concern to whiners like me? There are no whiners LIKE me. I am THE whiner. And hops ARE a first class problem, Jerry, or are you clueless about how store-and-forward packet switching actually really works?
I was in a backbone engineer's skin for quite a few years, and "hops" per se never were a problem. In fact, store-and-forward delays are a mere fraction of wire propagation delays -- do a traceroute coast-to-coast, look at delays and calculate how it relates to distance divided by speed of light. Indeed, you're the first person concerned with the growth of diameter (which is, BTW, logarithmic to size of the network).
--vadim
______________________________________________ ______________________________________________
Dr. Robert M. ("Bob") Metcalfe Executive Correspondent, InfoWorld and VP Technology, International Data Group
Internet Messages: bob_metcalfe@infoworld.com Voice Messages: 617-534-1215
Conference Chairman for ACM97: The Next 50 Years of Computing San Jose Convention Center March 1-5, 1997 ______________________________________________ ______________________________________________
participants (2)
-
bac＠serendip.sdsc.edu -
bob_metcalfe＠infoworld.com