Date: Tue, 09 Apr 2002 16:03:53 -0400 From: Richard A Steenbergen <ras@e-gerbil.net> Subject: Re: packet reordering at exchange points

To transfer 1Gb/s across 100ms I need to be prepared to buffer at least 25MB of data. According to pricewatch, I can pick up a high density 512MB PC133 DIMM for $70, and use $3.50 of it to catch that TCP stream. Throw in $36 for a GigE NIC, and we're ready to go for under $40. Yeah I know thats cheapest garbage you can get, but this is just to prove a point. :) I might only be able to get 800Mbit across a 32bit/33mhz PCI bus, but whatever.

Of course, in reality, things like choice of NIC can matter tremendously when it comes to going even moderately fast, which is why people continue to pay a premium for high performance NICs such as those by Syskonnect. (When you see vendors touting near-gigabit throughput for inexpensive gig NICs, beware: you're probably looking at benchmarks consisting of multiple streams sent with jumbo frames between two machines connected virtually back to back rather than "real world" performance associated with a single wide area tcp flow across a 1500 byte MTU link).

The problem isn't the lack of hardware, it's a lack of good software (both on the receiving side and probably more importantly the sending side), a lot of bad standards coming back to bite us (1500 byte packets is about as far from efficient as you can get), a lack of people with enough know-how to actually build a network that can transport it all (heck they can't even build decent networks to deliver 10Mbit/s, @Home was the closest), and just a general lack of things for end users to do with that much bandwidth even if they got it.

In the university community, it is routine for students in residence halls to have access to switched 10 (or even switched 100 Mbps) ethernet; of course, at that point, the issue isn't a lack of things for end users to do with that much (potential) bandwidth, it is the *cost* of provisioning wide area commodity bandwidth to support the demand that that speedy local infrastructure can generate that becomes the binding constraint. And speaking of University users, if you look at Internet2's excellent weekly reports (see, for example: http://netflow.internet2.edu/weekly/20020401/#fastest ) you'll see that wide area TCP non-measurement single flows *are* occuring at and above the 100Mbps mark, at least across that (admittedly rather atypical) network infrastructure. [Maybe not as commonly as we'd all like to hope, but they are happening.] Regards, Joe