On Fri, 15 Oct 2004, Alex Bligh wrote:
I can support what Iljisch said.
In a former life I ran extensive tests on the effect of CAR on TCP (no longer have the data to publish, but it's out there), and it's "just plain broken" - if your purpose is to simulate a lower amount of bandwidth with or without a burst. In a nutshell the problem is that the sliding window algorithm expects RTT to gradually increase with congestion, to find the optimum window size - the increased RTT stops the window growing. With rate-limiting that does not also shape (i.e. buffer the packets - this is true of token based systems such as CAR), the window size just keeps on expanding in leaps and bounds until there's a packet drop, whereupon it shrinks right down, rinse and repeat, so you get a sawtooth effect. Adding burst sizes just moves the problem around - you don't see the effect until later in the stream - because the excess of traffic over committed rate just sits there using up the burst and there is no signal to slow down; it /somewhat/ hides the effect in a lab if you are using short single requests (e.g. short HTTP) but not if you aggregate multiple parallel requests.
If you want to simulate lower bandwidths through a high bandwidth interface, and you want to be TCP friendly, you HAVE to use shaping. That means buffering (delaying) packets, and at gigabit line rates, with multiple clients, you need BIG buffers (but set sensible buffer limits per client).
You can reasonably trivially do the above test with ftp, ethereal, a bit of perl, and something to graph sequence numbers and throughput.
In the testing I have done, it works great. You get the bandwidth configured. Our customers would be outside with burning stakes and pitchforks otherwise. The vast majority of people who complain about CAR have horribly misconfigured limits. If your burst size is off, you get nothing resembling the configured throughput. Yes, there is sawtoothing, but the excessive burst accomodates that to a large degree, allowing for an overall correct transfer rate. The time spent in slow-start is accomodated by dropping packets later than you normally would, resulting in a correct rate over time. The sawtoothing doesn't really matter if you're selling a 512kbps ethernet connection, and the customer can always get both 512kbps in a single stream and aggregated across multiple streams. If the desire is to provide a simulated circuit with "x" bandwidth, CAR does a great job, IFF you correctly size the burst: 1.5x/8 for the normal burst, 3x/8 for the max burst. The aggregate rate of the transfer is "x" in all the testing I've done. How can you ask for more than the configured line rate? In my testing, I noticed a pronounced saw-tooth effect with incorrectly configured bursts, but with correctly configured bursts, the saw-toothing affect did not prevent delivery of the configured throughput. This holds up with multiple concurrent transfers. The customer gets whatever bandwidth is available under their cap, and when I look at their bandwidth graphs, they have the ability to saturate their bandwidth to 100% of the configured rate. Works for me and mine. Andy --- Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972 ---