On 9/6/23 11:20, Benny Lyne Amorsen wrote:
TCP looks quite different in 2023 than it did in 1998. It should handle packet reordering quite gracefully; in the best case the NIC will reassemble the out-of-order TCP packets into a 64k packet and the OS will never even know they were reordered. Unfortunately current equipment does not seem to offer per-packet load balancing, so we cannot test how well it works.
I ran per-packet load balancing on a Juniper LAG between 2015 - 2016. Let's just say I won't be doing that again. It balanced beautifully, but OoO packets made customers' lives impossible. So we went back to adaptive load balancing.
It is possible that per-packet load balancing will work a lot better today than it did in 1998, especially if the equipment does buffering before load balancing and the links happen to be fairly short and not very diverse.
Switching back to per-packet would solve quite a lot of problems, including elephant flows and bad hashing.
I would love to hear about recent studies.
2016 is not 1998, and certainly not 2023... but I've not heard about any improvements in Internet-based applications being better at handling OoO packets. Open to new info. 100Gbps ports has given us some breathing room, as have larger buffers on Arista switches to move bandwidth management down to the user-facing port and not the upsteam router. Clever Trio + Express chips have also enabled reasonably even traffic distribution with per-flow load balancing. We shall revisit the per-flow vs. per-packet problem when 100Gbps starts to become as rampant as 10Gbps did. Mark.