It will not be easy to get a straight answer, I would say more about your environ and applications. So if you considered the classical TCP algorithm ignoring latency it is large buffer, yet what about microburst? LG ________________________________ From: NANOG <nanog-bounces+lobna_gouda=hotmail.com@nanog.org> on behalf of William Herrin <bill@herrin.us> Sent: Friday, April 9, 2021 1:07 PM To: Mike Hammett <nanog@ics-il.net> Cc: nanog@nanog.org <nanog@nanog.org> Subject: Re: Trident3 vs Jericho2 On Fri, Apr 9, 2021 at 6:05 AM Mike Hammett <nanog@ics-il.net> wrote:
What I've observed is that it's better to have a big buffer device when you're mixing port speeds. The more dramatic the port speed differences (and the more of them), the more buffer you need.
If you have all the same port speed, small buffers are fine. If you have 100G and 1G ports, you'll need big buffers wherever the transition to the smaller port speed is located.
When a network is behaving well (losing few packets to data corruption), TCP throughput is is impacted by exactly two factors: 1. Packet round trip time 2. The size to which the congestion window has grown when the first packet is lost Assuming the sender has data ready, it will (after the initial negotiation) slam out 10 packets back to back at the local wire speed. Those 10 packets are the initial congestion window. After sending 10 packets it will wait and wait and wait until it hits a timeout or the other side responds with an acknowledgement. So those initial packets start out crammed right at the front of the round trip time with lots of empty afterwards. The receiver gets the packets in a similar burst and sends its acks. As the sender receives acknowledgement for each of the original packets, it sends two more. This doubling effect is called "slow start," and it's slow in the sense that the sender doesn't just throw the entire data set at the wire and hope. So, having received acks for 10 packets, it sends 20 more. These 20 have spread out a little bit, more or less based on the worst link speed in the path, but they're still all crammed up in a bunch at the start of the round trip time. Next round trip time it doubles to 40 packets. Then 80. Then 160. All crammed up at the start of the round trip time causing them to hit that one slowest link in the middle all at once. This doubling continues until one of the buffers in the middle is too small to hold the trailing part of the burst of packets while the leading part is sent. With a full buffer, a packet is dropped. Whatever the congestion window size is when that first packet is dropped, that number times the round trip time is more or less the throughput you're going to see on that TCP connection. The various congestion control algorithms for TCP do different things after they see that first packet drop. Some knock the congestion window in half right away. Others back down more cautiously. Some reduce growth all the way down to 1 packet per round trip time. Others will allow faster growth as the packets spread out over the whole round trip time and demonstrate that they don't keep getting lost. But in general, the throughput you're going to see on that TCP connection has been decided as soon as you lose that first packet. So, TCP will almost always get better throughput with more buffers. The flip side is latency: packets sitting in a buffer extend the time before the receiver gets them. So if you make a buffer that's 500 milliseconds long and then let a TCP connection fill it up, apps which work poorly in high latency environments (like games and ssh) will suffer. Regards, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/