... It was pointed out that with no delay in the tests, the delay bandwidth product of the TCP flows was near zero and it was asserted (by me actually) that results from such testing is not useful since real TCP flows going through a NAP have considerable delays. I disagree that the lack of delay makes the tests 'not useful'. Yes, the tests were much weaker than tests with delay. But such tests could still be useful, since they demonstrate the presence or absence of problems other than
Curtis, Thanks for helping Stan build more comprehensive notes on what was, for me at least, the most interesting RegionalTechs/NANOG meeting in recent memory. I disagree with one statement you make several times... (In the context of the Ameritech presentation) the problems caused by the bursty nature of wide-area high-speed TCP flows. Actually, my criticism of the Ameritech presentation was that there was an insufficiently clear statement of what they *had* learned. There were allusions to problems with Cisco and workstations, but they were not specific enough to be as helpful as they might have been.
PacBell NAP Status--Frank Liu
[ ... ]
Again, no delay was added. Measured delay (ping time) was said to be 3 msec (presumably due to switching or slow host response). Again - It was pointed out that with no delay in the tests, the delay bandwidth product of the TCP flows was near zero and asserted that results from such testing is not useful.
ANS on performance --- Curtis Vallamizar
The difficulty in carrying TCP traffic is proportional to the delay bandwidth product of the traffic, not just the bandwidth. Adding delay makes the potential for bursts sustained over a longer period. Real networks have delay. US cross continent delay is 70 msec.
ANSNET results were given using improved software which improved buffer capacity, intentionally crippled software (artificially limited buffering), and software which included Random Early Detection (RED - described in a IEEE TON paper by Floyd and Jacobson). Sustained goodput rates of up to 40-41 Mb/s were acheived using ttcp and 1-8 TCP flows. Some pathelogical cases were demonstrated in which much worse performace was acheived. These case mostly involve too little buffering at the congestion point (intentionally crippled router code was used to demostrate this) or using a single TCP flow and setting the TCP window much too large (3-5 times the delay bandwidth product). The latter pathelogic case can be avoided if the routers implement RED. The conclusions were: 1) routers need buffer capacity as large as the delay bandwidth product and 2) routers should impement RED.
Only a 20 msec delay was added to the prototype NAP testing. Results with the prototype NAP and 20 msec delay were very poor compared to the performance of unchannelized DS3. Prototype NAP testing results were poor compared to Ameritec and Pacbell results due to the more realistic delay bandwidth product. Worse results can be expected with a 70 msec delay and may be better indications of actual performance when forwarding real traffic. More testing is needed after fixes to ADSUs are applied. A sufficient bottleneck can not be created at the switch until ADSU problems are addressed.
There was some discussion (at various times during the presentations) of what this all means for the NAPs. If I may summarize - On the positive side the Ameritec switch has more buffering than the Fore used in the Bellcore prototype NAP. On the negative side, Ameritec didn't include any delay in their testing. NAP testing results (both positive results from Amertic, mixed results from PacBel and negative results from ANS) are inconclusive so far. Right. I really hope that everyone at the NANOG meeting took your results to heart. It will be very important for the success of the multi-backbone NSFnet world. I agree that all three sets of tests should currently be regarded as partial and, to some extent, inconclusive as you say. I hope that all those, including BellCore, Ameritech, PacBell, and ANS continue with these tests and with the sharing of the testing results. One quibble: pathelogical should be spelled 'pathological'. The root word is pathos, from which we also get the word sympathy. I guess
Again, I disagree. Even in the absence of delay, some problems, including the Kentrox bit-rate mismatch, could be observed and the loads that caused packet loss could be observed. that those engaged in testing pathological ATM NAP situations should have more sympathy for each other. Cheers, -- Guy