On Sat, Dec 26, 2020 at 5:41 PM Mikael Abrahamsson <swmike@swm.pp.se> wrote:
On Sat, 26 Dec 2020, Baldur Norddahl wrote:

> That is why. The RTT to the source can not be larger than the minimum
> buffer size in the transport path. Otherwise the speed will start
> decreasing.

This is no longer correct. There has been lots of TCP innovation since
this was true.

Please stop repeating it.

It is true there have been TCP improvements but you can very easily verify for yourself that it is very hard to get anywhere near 1 Gbps of actual transfer speed to destinations just 10 ms away. Try the nlnog ring network like this:

gigabit@gigabit01:~$ iperf -c netnod01.ring.nlnog.net
------------------------------------------------------------
Client connecting to netnod01.ring.nlnog.net, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 185.24.168.23 port 50632 connected with 185.42.136.5 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   452 MBytes   379 Mbits/sec

And that is a direct peer of ours.

In general you will have trouble with any server that has a NIC > 1G. If you find a server that has a 1G NIC this happens instead:

gigabit@gigabit01:~$ iperf -c bahnhof01.ring.nlnog.net
------------------------------------------------------------
Client connecting to bahnhof01.ring.nlnog.net, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 185.24.168.23 port 56412 connected with 195.178.185.171 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.08 GBytes   930 Mbits/sec

Why? Because the 1G NIC server naturally will pace the traffic at maximum 1G and therefore not fill any buffers in the transfer path. The 10G servers on the other hand WILL fill the buffers and experience packet loss.

Regards,

Baldur