- NANOG - lists.nanog.org

Questions about Internet Packet Losses
by Bob Metcalfe 14 Jan '97

14 Jan '97

Hello, and best wishes for what's left of 1997. Now, if you would, ... Below are some questions I hope you'll help me answer about packet loss on the Internet. Here are two paragraphs taken from: http://www.merit.edu/~ipma/netnow/docs/info.html: "Early experiments with NetNow show that 30% packet loss between public exchange points is common for major Internet service providers during peak usage periods. The initial investigation also suggests that loss rates are closely related to bandwidth usage and congestion problems. Although some of the packet loss is inadvertent, a large percentage of the public exchange point connectivity problems reflect intentional engineering decisions by Internet service providers based on commercial settlement issues. "The high packet loss may not generally reflect problems seen by the majority of customers of the larger network service providers. In fact, increasing levels of Internet traffic are not traversing the public exchange points. Instead, many large service providers are migrating their inter-provider traffic to private exchange points, or direct connections to other providers. Merit is working closely with providers to develop tools and infrastructure that more closely reflect Internet performance as observed by the majority of backbone customers." Questions: Are you familiar with this packet loss data from Merit? If not, please see above URL. Is Merit's packet loss data (NetNow) credible? Do packet losses in the Internet now average between 2% and 4% daily? Are 30% packet losses common during peak periods? Is there any evidence that Internet packet losses are trending up or down? If Merit's data is not correct, where has Merit gone wrong? Where is there better data? Were Merit's data correct, what would be the impact of 30% packet losses on opening up TCP connections? On TCP throughput, say through a 28.8Kbps modem? On Web throughput, since so many TCP connections are involved? On DNS look-ups? On email transport? How big a problem is HTTP's opening of so many TCP connections? Does TCP need to operate differently than it does now when confronted routinely with 30% packet losses and quarter-second transit delays? What is the proper response of an IP-based protocol, like TCP, as packet losses climb? Try harder or back off or what? How robust are various widespread TCP/IP implementations in the face of 30% packet loss and quarter-second transit delays? Is the Internet's sometimes bogging down due mainly to packet losses or busy servers or what, or does the Internet not bog down? What fraction of Internet traffic still goes through public exchange points and therefore sees these kinds of packet losses? What fraction of Internet traffic originates and terminates within a single ISP? Where is the data on packet losses experienced by traffic that does not go through public exchange points? If 30% loss impacts are noticeable, what should be done to eliminate the losses or reduce their impacts on Web performance and reliability? Are packet losses due mainly to transient queue buffer overflows of user traffic or to discards by overburdened routing processors or something else? What does Merit mean when they say that some of these losses are intentional because of settlement issues? Are ISPs cooperating intelligently in the carriage of Internet traffic, or are ISPs competing destructively, to the detriment of them and their customers? Any help you can offer on these questions would be appreciated. /Bob Metcalfe, InfoWorld ______________________________________________ ______________________________________________ Dr. Robert M. ("Bob") Metcalfe VP Technology, International Data Group InfoWorld columns: www.infoworld.com Mail: metcalfe(a)infoworld.com Telephone: 617-534-1215 Conference Chairman for ACM97: The Next 50 Years of Computing San Jose Convention Center, March 1-5, 1997 Registration information: www.acm.org/acm97 Register now at 1-800-342-6626 ______________________________________________ ______________________________________________

10 11

FYI: IP NetMonitor for MacOS
by Tom Easterday 14 Jan '97

14 Jan '97

FYI: If you have a Mac there is a tool called IP NetMonitor by Peter Sichel that has whois, tracroute, ping, and a subnet calculator. It is very nice and long overdue (IMHO). I have no ties to it other than I like it. You can try it for free, and it is only $10 to register... http://www.ultranet.com/~psichel/ We now return you to your regularly scheduled program of name calling and finger pointing.. Tom

5 4

NANOG Meeting
by Pam Ciesla 14 Jan '97

14 Jan '97

Other hotels within walking distance to the Grand Hyatt (415- 398-1234). Sir Francis Drake 415-392-7755 $169 Queen, $179 2 Doubles $189 King Cartwright 415-421-2865 Queen or 2 Doubles $109. Kensington Park 415-788-6400 $119 Queen or 2 Doubles. I do not have an answer yet for the room rate at the Grand Hyatt for 2/11, I will let you know ASAP. Pam Ciesla

2 1

RE: Questions about Internet Packet Losses
by Peter Cole 14 Jan '97

14 Jan '97

People want/need information, Lots of it and lots of different kinds of it. They don't/won't wait on it. Packet loose limits the growth of the Internet by reducing the desire to use the Internet to move different kinds of information. The better it works the more we will all use it, until we get tired of waiting for it. Then we back off and: place a phone call, send a fax, turn on the radio, go to the library or the post office, turn on the TV, read a magazine or newspaper, Install a lease line, etc. ISPs that provides better quality of service will attract content providers and content consumers that want more from the net. As consumers become aware of the differences in service, low performance ISPs will decline in popularity for demanding consumers. Try and find a customer that says they like to wait 30 seconds for a web page. Several of the dialup nationwide systems are 30 second slow. As customers of slow ISPs see faster response from higher quality of service ISPs they will switch or continue to wait. As new solutions are made available to consumers like personal video conferencing, consumers will tend to higher performance ISPs. Even if a carrier delivers ever packet they receive to the Gigaswitch at an exchange point the customer doesn't care where the packet was lost. The increase in private crossconnects sends a message to the exchange points, while giving the exchange point some breathing room. In summation packet loose is here to stay. The amount will be determined by the free market system. P.S. If someone can prove they have better access than what I am buying now I'll plan on switching. AS1702

1 0

Grand Hyatt for NANOG Meeting
by Pam Ciesla 14 Jan '97

14 Jan '97

Now has the room rate for 2/11 at $159.00. If you have already made reservations at the higher rate call and have them change it. Pam

1 0

Re: Questions about Internet Packet Losses
by William Allen Simpson 14 Jan '97

14 Jan '97

Yes, Bakul, keeping a central RTT cache per destination is a good idea. Most good stacks use it already. I think it was recommended in Host Requirements circa 1989. Keeping a per destination cache of Path RTT, Path MTU, and a Quality measurement was required in my initial IPng Neighbor Discovery design several years ago, before that was destroyed in the rewrite by committee. > From: Bakul Shah <bakul(a)torrentnet.com> > [Thinking aloud here...] > Perhaps a part of the TCP congestion avoidance algorithm can be > factored out in some sort of a `traffic central' module that tries > to give you the best bandwidth/packet loss estimate it has for a > given route provided you keep it updated with what you learn (i.e. > TCP tells it when a packet is lost etc). A new TCP connection can > then immediately start off with a bigger window (and won't open the > window too wide too quickly). Multiple connections between two > hosts can avoid what would be largely redundant estimate > computation. Even a UDP app. can try to benefit from this (such as > for communication where bounded delay is more critical than packet > loss). Other `traffic conditions' input can also be fed into this > module [perhaps as part of some future routing protocol]. Combining > this `quality' of a route aspect into routing protocols may make > sense in the long run.... > WSimpson(a)UMich.edu Key fingerprint = 17 40 5E 67 15 6F 31 26 DD 0D B9 9B 6A 15 2C 32 BSimpson(a)MorningStar.com Key fingerprint = 2E 07 23 03 C5 62 70 D3 59 B1 4F 5E 1D C2 C1 A2

1 0

Re: Questions about Internet Packet Losses
by Vern Paxson 14 Jan '97

14 Jan '97

> So you'll note that there are very clear spikes at 552 and 576 total packet > size. Further, note that the curve falls off _extremely_ rapidly. From > that, I argue that case (2) _cannot_ be the case because any type of random > distribution would be much smoother. (2) is a random distribution plus some spikes, plus some modality due to TCP's that can only generate packets of <= N bytes for different popular MSS's. So I think it still works (but will save more detailed analysis till later ....). > 2) What OS is using a 512 MSS? >From a bunch of (~ 20,000) traces in my study, looks like Irix 4.0/5.3, BSDI 1.1/2.0, OSF/1 3.0/3.2, and often but not always SunOS use either 512 or 536, or sometimes 500, 524, or 548 (BSDI). Never anything higher. This doesn't mean they *can't* use anything higher, just that they never found an opportunity to do so. 512 is far and away the most common MSS in the traces. (All the TCP's in the traces are BSD variants except for Solaris and Linux.) > 256? My best guess is SLIP links. > 4) 41 bytes is pretty obviously interactive traffic. Is the intuition > correct? That's my guess too. > What's so special about 44, 52, 48 and 56? Ack/SYN/FIN's with options. > What do people do > with 4, 8, 12 and 16 bytes of data? And why not any of the odd values? Because option sizes are rounded up to multiples of 4 bytes. Vern

2 1

Re: Questions about Internet Packet Losses
by Vern Paxson 14 Jan '97

14 Jan '97

> indicate that 9% of the packets had a 512 byte *payload*. > > I already subtracted off the 40 bytes for the headers. I meant MSS. (By payload, I meant data, i.e., not including headers, same as you ...) > However, > the spikes in the packet percentages are clearly due to hosts using that > MSS. You would otherwise expect to see a "smooth" distribution across > packet sizes, which is not at ALL what's happening. Right, spikes correspond to MSS's (and other effects like acks & SYN/FIN's). Are you saying the data shows significant spikes at other, non-standard MSS values? If it does, then (modulo some thinking about the particular numbers) I agree that they're likely MSS candidates and support the notion that weird MSS's are being used. But if the data has a few spikes at things like 512 but no strong spikes at non-standard values, then that doesn't help distinguish between (1) lots of weird TCP's using weird MSS's, vs (2) lots of packets that are less than full-size because the mainstream TCP didn't have enough bytes to fill them. If your data is available, I'd love to take a gander at it. Vern

3 3

Re: Questions about Internet Packet Losses
by Vern Paxson 14 Jan '97

14 Jan '97

> Isn't it unsurpsrising that >40% of packets are small? ACKs? > > If one made the gross oversimplifying approximation that everything is > unidirectional TCP traffic, then you'd expect to see one ACK per two data > packets. This is actually not at all a gross oversimplying approximation. Much TCP traffic unidirectional or request/response. The latter looks a whole lot like two back-to-back unidirectional connections, with only one chance to piggyback acks at the switchover. > Thus, we'd expect to see 33% at 40 bytes. It is the additional > 7+% that's surprising. Throw in frequent SYN/FIN/RST's because of small Web connections, SYN retransmissions because busy Web servers have full listen queues (or are being SYN-flooded :-), TCP's that ack every packet (Linux 1.0, I believe, and maybe later ones), TCP's that wind up acking every packet because their delay timer is less than an MSS prop time across a slow link (Solaris, for links < 10KB/sec), and dup acks due to sequence holes, and you might handwave the 7%. Vern

2 1

Re: 30%, huh? (Re: Questions about Internet Packet Losses )
by Vadim Antonov 14 Jan '97

14 Jan '97

Paul A Vixie <paul(a)vix.com> wrote: >I wish I had a background in "fractal mathematics" or whatever, but instead >I've just got my gut feelings to go by. They say that 30% loss translates >loosely to 30% overcommitment. I wish it was as simple as that. My educated guess is that 30% packet loss corresponds to about 1000% overcommitment. TCP actively reduces packet loss by doing congestion avoidance, so the results are more than a bit skewed. Strictly speaking, your assumption would be correct if packet arrivals were independent (i.e. Poisson). >As Tony and others have remarked here today, >TCP does not overcommit enough once it's up and running to account for 30% >loss. Just a little bit to test water :) But that's steady state. We're talking about the system where TCP doesn't really have a chance to establich proper window, mostly because of HTTP silliness. >And given slow start, I'm not sure it could overcommit that much even >given HTTP 1.0's tendancy to start a lot of new connections. HTTP barf barf barf. That's the poster case of shortcomings of the computer science education which prefers to neglect the gory details of what happens in the hardware or "lower levels" in general. >So if we are seeing 30% overcommit, my gut feeling is that most TCP endpoints >aren't using slow start. No, 30% loss can easily be caused by the initial TCP SYNs. The average flow duration is something like 12 packets, yes? TCP works well only when there's enough slack to handle initial SYNs. Actually, that's one argument against RED. It is interesting to explore how the network will behave if SYNs are given priority over the "regular" packets. >IETF had a huge flame war over whether slow start should be a recommended >standard or not since it was just an implementation detail or so said some. I'd say IETF should be abolished. If there's an argument about _that_ there's a serious doubt about overall sanity of the organization. >TCP creates a prisoner's dilemma problem for implementors, and it's sad that >more than one has said "let's not cooperate so that we can get more bit times." Well, no. Non-compliant TCP implementation is like ping -f. It should be a sufficient reason for disconnecting from the network. >I saw an ad for one company who advertised that their TCP stack was 200% >faster than the rest of them. Yep. The big problem is the sad state of cluefulness of the general user society. This ad sounds awfully like "buy our gun, earn $1000 in 30 minutes!". >As a good friend likes to tell me: "The world is full of clowns." Indeed. --vadim

1 0