In a message written on Fri, Jan 27, 2012 at 11:30:14PM +0100, bas wrote:
While your reasoning holds truth it does not explain why the expensive chassis solution (good) makes my customers happy, and the cheaper TOR solution makes my customers unhappy.....
Bufferbloat does not matter to them as jitter and latency does not matter. As long as the TCP window size negotioation is not reset the total amount of bit/sec increases for them.
I obviously don't know your application. The bufferbloat problem exists for 99.99% of the standard applications in the world. There are, however, a few corner cases. For instance, if you want to move a _single_ TCP stream at more than 1Gbps you need deep buffers. Dropping a single packet slows throughput too much due to a slow-start event. For most of the world with hundreds or thousands of TCP streams across a single port, such problems never occur.
If deep buffers are bad I would expect high-end chassis solutions not to offer them either. But the market seems to offer expensive deep buffer chassis solutions and cheap (per 10GE) TOR solutions.
The margin on a top-of-rack switch is very low. 48 port gige with 10GE uplinks are basically commodity boxes, with plenty of competition. Saving $100 on the bill of materials by cutting out some buffer makes the box more competitive when it's at a $2k price point. In contrast, large, modular chasses have a much higher margin. They are designed with great flexability, to take things like firewall modules and SSL accelerator cards. There are configs where you want some (not much) buffer due to these active appliances in the chassis, plus it is easier to hide an extra $100 of RAM in a $100k box. Also, as was pointed out to me privately, it is also important to loook at adaptive queue management features. The most famous is WRED, but there are other choices. Having a queue management solution on your routers and switches that works in concert with the congestion control mechanism used by the end stations always results in better goodput. Many of the low end switches have limited or no AQM choices, while the higher end switches with fancier ASICs can default to something like WRED. Be sure it is the deeper buffers that are making the difference, and not simply some queue management. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/