xplornet contact or any experience with their satellite service?
A friend of mine just recently got Xplornet satellite service at his rural home. I'm well aware of the latency issues with satellite although frankly his latency is much better than I had feared it would be and is around 600-700ms. But what seems to be worse than the latency is the "burstiness" of the traffic and I am just wondering if that is normal/expected for satellite service in general, and/or expected from Xplornet's service, or if what I am seeing is not expected at all (i.e. not an artifact of the satellite signal but rather a network management issue). Here's iperf3 for 30 seconds sending data (i.e. upload speed): [ ID] Interval Transfer Bitrate [ 5] 0.00-1.21 sec 12.9 KBytes 87.4 Kbits/sec [ 5] 1.21-2.00 sec 6.47 KBytes 67.2 Kbits/sec [ 5] 2.00-3.00 sec 22.0 KBytes 180 Kbits/sec [ 5] 3.00-4.00 sec 41.4 KBytes 339 Kbits/sec [ 5] 4.00-5.00 sec 41.4 KBytes 339 Kbits/sec [ 5] 5.00-6.00 sec 55.6 KBytes 456 Kbits/sec [ 5] 6.00-7.00 sec 69.9 KBytes 572 Kbits/sec [ 5] 7.00-8.00 sec 89.3 KBytes 731 Kbits/sec [ 5] 8.00-9.00 sec 120 KBytes 986 Kbits/sec [ 5] 9.00-10.00 sec 86.7 KBytes 710 Kbits/sec [ 5] 10.00-11.00 sec 133 KBytes 1.09 Mbits/sec [ 5] 11.00-12.00 sec 184 KBytes 1.51 Mbits/sec [ 5] 12.00-13.00 sec 186 KBytes 1.53 Mbits/sec [ 5] 13.00-14.00 sec 159 KBytes 1.30 Mbits/sec [ 5] 14.00-15.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 15.00-16.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 16.00-17.00 sec 93.2 KBytes 763 Kbits/sec [ 5] 17.00-18.00 sec 264 KBytes 2.16 Mbits/sec [ 5] 18.00-19.00 sec 124 KBytes 1.02 Mbits/sec [ 5] 19.00-20.00 sec 157 KBytes 1.28 Mbits/sec [ 5] 20.00-21.00 sec 120 KBytes 986 Kbits/sec [ 5] 21.00-22.00 sec 86.7 KBytes 710 Kbits/sec [ 5] 22.00-23.00 sec 369 KBytes 3.02 Mbits/sec [ 5] 23.00-24.00 sec 197 KBytes 1.61 Mbits/sec [ 5] 24.00-25.00 sec 90.6 KBytes 741 Kbits/sec [ 5] 25.00-26.00 sec 193 KBytes 1.58 Mbits/sec [ 5] 26.00-27.00 sec 192 KBytes 1.57 Mbits/sec [ 5] 27.00-28.00 sec 189 KBytes 1.55 Mbits/sec [ 5] 28.00-29.00 sec 193 KBytes 1.58 Mbits/sec [ 5] 29.00-30.00 sec 179 KBytes 1.46 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-32.20 sec 4.41 MBytes 1.15 Mbits/sec 388 sender [ 5] 0.00-30.00 sec 3.57 MBytes 998 Kbits/sec receiver which averaged the overall prescribed "upload" speed, but notice that it's not 1Mb/s in any kind of a steady stream but rather bursts of higher than 1Mb/s speed followed by low/no speed. At one point it was 2 seconds with no transfer at all even. and here's receiving (i.e. "download"): [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.35 sec 46.6 KBytes 283 Kbits/sec 0 12.9 KBytes [ 5] 1.35-2.00 sec 0.00 Bytes 0.00 bits/sec 0 12.9 KBytes [ 5] 2.00-3.00 sec 67.3 KBytes 551 Kbits/sec 0 37.5 KBytes [ 5] 3.00-4.00 sec 46.6 KBytes 382 Kbits/sec 0 40.1 KBytes [ 5] 4.00-5.00 sec 105 KBytes 858 Kbits/sec 0 44.0 KBytes [ 5] 5.00-6.00 sec 88.0 KBytes 721 Kbits/sec 0 54.3 KBytes [ 5] 6.00-7.00 sec 141 KBytes 1.16 Mbits/sec 0 69.9 KBytes [ 5] 7.00-8.00 sec 124 KBytes 1.02 Mbits/sec 0 101 KBytes [ 5] 8.00-9.00 sec 186 KBytes 1.53 Mbits/sec 0 146 KBytes [ 5] 9.00-10.00 sec 248 KBytes 2.04 Mbits/sec 0 206 KBytes [ 5] 10.00-11.00 sec 311 KBytes 2.54 Mbits/sec 0 257 KBytes [ 5] 11.00-12.00 sec 0.00 Bytes 0.00 bits/sec 43 194 KBytes [ 5] 12.00-13.00 sec 0.00 Bytes 0.00 bits/sec 75 199 KBytes [ 5] 13.00-14.00 sec 435 KBytes 3.56 Mbits/sec 0 199 KBytes [ 5] 14.00-15.00 sec 0.00 Bytes 0.00 bits/sec 34 114 KBytes [ 5] 15.00-16.00 sec 0.00 Bytes 0.00 bits/sec 34 140 KBytes [ 5] 16.00-17.00 sec 373 KBytes 3.05 Mbits/sec 0 149 KBytes [ 5] 17.00-18.00 sec 0.00 Bytes 0.00 bits/sec 0 162 KBytes [ 5] 18.00-19.00 sec 373 KBytes 3.05 Mbits/sec 0 168 KBytes [ 5] 19.00-20.00 sec 0.00 Bytes 0.00 bits/sec 0 171 KBytes [ 5] 20.00-21.00 sec 373 KBytes 3.05 Mbits/sec 0 172 KBytes [ 5] 21.00-22.00 sec 0.00 Bytes 0.00 bits/sec 14 141 KBytes [ 5] 22.00-23.00 sec 0.00 Bytes 0.00 bits/sec 0 120 KBytes [ 5] 23.00-24.00 sec 373 KBytes 3.05 Mbits/sec 0 131 KBytes [ 5] 24.00-25.00 sec 0.00 Bytes 0.00 bits/sec 1 146 KBytes [ 5] 25.00-26.00 sec 0.00 Bytes 0.00 bits/sec 14 104 KBytes [ 5] 26.00-27.00 sec 0.00 Bytes 0.00 bits/sec 0 104 KBytes [ 5] 27.00-28.00 sec 373 KBytes 3.05 Mbits/sec 0 107 KBytes [ 5] 28.00-29.00 sec 0.00 Bytes 0.00 bits/sec 0 119 KBytes [ 5] 29.00-30.00 sec 373 KBytes 3.05 Mbits/sec 0 123 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 3.94 MBytes 1.10 Mbits/sec 215 sender [ 5] 0.00-30.80 sec 3.13 MBytes 853 Kbits/sec receiver Again, very bursty with periods of 1-2 seconds with no transfer. As you can imagine, the bursiness of this makes for horrible video conferencing since that cannot "buffer" the way single-direction streams like streaming video can and the codec ends up using the "worst case" dips as the speed of the connection and encodes for that. Cheers, b.
Brian, Satellite services are shared bandwidth broadcast systems. The behavior you’re seeing is pretty common at times when you’re competing for access with other users. Just like the regular Internet, there are times of day when people tend to move more data, and because of latency and other limitations on bidirectional traffic, packets get delivered in batches. It’s not possible to interleave bytes, or even packets themselves. So when you see low or no throughput, it’s because the transponder is addressing packets to other users. -mel beckman
On Apr 21, 2020, at 6:53 AM, Brian J. Murrell <brian@interlinx.bc.ca> wrote:
A friend of mine just recently got Xplornet satellite service at his rural home. I'm well aware of the latency issues with satellite although frankly his latency is much better than I had feared it would be and is around 600-700ms.
But what seems to be worse than the latency is the "burstiness" of the traffic and I am just wondering if that is normal/expected for satellite service in general, and/or expected from Xplornet's service, or if what I am seeing is not expected at all (i.e. not an artifact of the satellite signal but rather a network management issue).
Here's iperf3 for 30 seconds sending data (i.e. upload speed):
[ ID] Interval Transfer Bitrate [ 5] 0.00-1.21 sec 12.9 KBytes 87.4 Kbits/sec [ 5] 1.21-2.00 sec 6.47 KBytes 67.2 Kbits/sec [ 5] 2.00-3.00 sec 22.0 KBytes 180 Kbits/sec [ 5] 3.00-4.00 sec 41.4 KBytes 339 Kbits/sec [ 5] 4.00-5.00 sec 41.4 KBytes 339 Kbits/sec [ 5] 5.00-6.00 sec 55.6 KBytes 456 Kbits/sec [ 5] 6.00-7.00 sec 69.9 KBytes 572 Kbits/sec [ 5] 7.00-8.00 sec 89.3 KBytes 731 Kbits/sec [ 5] 8.00-9.00 sec 120 KBytes 986 Kbits/sec [ 5] 9.00-10.00 sec 86.7 KBytes 710 Kbits/sec [ 5] 10.00-11.00 sec 133 KBytes 1.09 Mbits/sec [ 5] 11.00-12.00 sec 184 KBytes 1.51 Mbits/sec [ 5] 12.00-13.00 sec 186 KBytes 1.53 Mbits/sec [ 5] 13.00-14.00 sec 159 KBytes 1.30 Mbits/sec [ 5] 14.00-15.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 15.00-16.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 16.00-17.00 sec 93.2 KBytes 763 Kbits/sec [ 5] 17.00-18.00 sec 264 KBytes 2.16 Mbits/sec [ 5] 18.00-19.00 sec 124 KBytes 1.02 Mbits/sec [ 5] 19.00-20.00 sec 157 KBytes 1.28 Mbits/sec [ 5] 20.00-21.00 sec 120 KBytes 986 Kbits/sec [ 5] 21.00-22.00 sec 86.7 KBytes 710 Kbits/sec [ 5] 22.00-23.00 sec 369 KBytes 3.02 Mbits/sec [ 5] 23.00-24.00 sec 197 KBytes 1.61 Mbits/sec [ 5] 24.00-25.00 sec 90.6 KBytes 741 Kbits/sec [ 5] 25.00-26.00 sec 193 KBytes 1.58 Mbits/sec [ 5] 26.00-27.00 sec 192 KBytes 1.57 Mbits/sec [ 5] 27.00-28.00 sec 189 KBytes 1.55 Mbits/sec [ 5] 28.00-29.00 sec 193 KBytes 1.58 Mbits/sec [ 5] 29.00-30.00 sec 179 KBytes 1.46 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-32.20 sec 4.41 MBytes 1.15 Mbits/sec 388 sender [ 5] 0.00-30.00 sec 3.57 MBytes 998 Kbits/sec receiver
which averaged the overall prescribed "upload" speed, but notice that it's not 1Mb/s in any kind of a steady stream but rather bursts of higher than 1Mb/s speed followed by low/no speed. At one point it was 2 seconds with no transfer at all even.
and here's receiving (i.e. "download"):
[ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.35 sec 46.6 KBytes 283 Kbits/sec 0 12.9 KBytes [ 5] 1.35-2.00 sec 0.00 Bytes 0.00 bits/sec 0 12.9 KBytes [ 5] 2.00-3.00 sec 67.3 KBytes 551 Kbits/sec 0 37.5 KBytes [ 5] 3.00-4.00 sec 46.6 KBytes 382 Kbits/sec 0 40.1 KBytes [ 5] 4.00-5.00 sec 105 KBytes 858 Kbits/sec 0 44.0 KBytes [ 5] 5.00-6.00 sec 88.0 KBytes 721 Kbits/sec 0 54.3 KBytes [ 5] 6.00-7.00 sec 141 KBytes 1.16 Mbits/sec 0 69.9 KBytes [ 5] 7.00-8.00 sec 124 KBytes 1.02 Mbits/sec 0 101 KBytes [ 5] 8.00-9.00 sec 186 KBytes 1.53 Mbits/sec 0 146 KBytes [ 5] 9.00-10.00 sec 248 KBytes 2.04 Mbits/sec 0 206 KBytes [ 5] 10.00-11.00 sec 311 KBytes 2.54 Mbits/sec 0 257 KBytes [ 5] 11.00-12.00 sec 0.00 Bytes 0.00 bits/sec 43 194 KBytes [ 5] 12.00-13.00 sec 0.00 Bytes 0.00 bits/sec 75 199 KBytes [ 5] 13.00-14.00 sec 435 KBytes 3.56 Mbits/sec 0 199 KBytes [ 5] 14.00-15.00 sec 0.00 Bytes 0.00 bits/sec 34 114 KBytes [ 5] 15.00-16.00 sec 0.00 Bytes 0.00 bits/sec 34 140 KBytes [ 5] 16.00-17.00 sec 373 KBytes 3.05 Mbits/sec 0 149 KBytes [ 5] 17.00-18.00 sec 0.00 Bytes 0.00 bits/sec 0 162 KBytes [ 5] 18.00-19.00 sec 373 KBytes 3.05 Mbits/sec 0 168 KBytes [ 5] 19.00-20.00 sec 0.00 Bytes 0.00 bits/sec 0 171 KBytes [ 5] 20.00-21.00 sec 373 KBytes 3.05 Mbits/sec 0 172 KBytes [ 5] 21.00-22.00 sec 0.00 Bytes 0.00 bits/sec 14 141 KBytes [ 5] 22.00-23.00 sec 0.00 Bytes 0.00 bits/sec 0 120 KBytes [ 5] 23.00-24.00 sec 373 KBytes 3.05 Mbits/sec 0 131 KBytes [ 5] 24.00-25.00 sec 0.00 Bytes 0.00 bits/sec 1 146 KBytes [ 5] 25.00-26.00 sec 0.00 Bytes 0.00 bits/sec 14 104 KBytes [ 5] 26.00-27.00 sec 0.00 Bytes 0.00 bits/sec 0 104 KBytes [ 5] 27.00-28.00 sec 373 KBytes 3.05 Mbits/sec 0 107 KBytes [ 5] 28.00-29.00 sec 0.00 Bytes 0.00 bits/sec 0 119 KBytes [ 5] 29.00-30.00 sec 373 KBytes 3.05 Mbits/sec 0 123 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 3.94 MBytes 1.10 Mbits/sec 215 sender [ 5] 0.00-30.80 sec 3.13 MBytes 853 Kbits/sec receiver
Again, very bursty with periods of 1-2 seconds with no transfer.
As you can imagine, the bursiness of this makes for horrible video conferencing since that cannot "buffer" the way single-direction streams like streaming video can and the codec ends up using the "worst case" dips as the speed of the connection and encodes for that.
Cheers, b.
Although I don't know of a way to solve this for videoconferencing, historically one way to mitigate the radio/vsat "batchiness" issue and its effect on end-to-end latency was to use a caching proxy server connected to a, er, "real" network somewhere, preferably as near as possible to the headend/uplink station. The modern web's move to TLS also means this technique is becoming pointless even for HTTP/S (although MITMing remains a way around that - many a HOWTO abounds, describing how to do this with Squid). FWIW, some very old radio systems behaved similarly... albeit not with 600+msec latency :-/. Some of the really old asymmetric TV systems (dial-up for uplink, CATV for downlink) exhibited similar characteristics and were similarly difficult to mitigate. Good luck! Adam Thompson Consultant, Infrastructure Services [[MERLIN LOGO]] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athompson@merlin.mb.ca<mailto:athompson@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> ________________________________ From: NANOG <nanog-bounces@nanog.org> on behalf of Mel Beckman <mel@beckman.org> Sent: Tuesday, April 21, 2020 9:05:20 AM To: Brian J. Murrell Cc: nanog@nanog.org Subject: Re: xplornet contact or any experience with their satellite service? Brian, Satellite services are shared bandwidth broadcast systems. The behavior you’re seeing is pretty common at times when you’re competing for access with other users. Just like the regular Internet, there are times of day when people tend to move more data, and because of latency and other limitations on bidirectional traffic, packets get delivered in batches. It’s not possible to interleave bytes, or even packets themselves. So when you see low or no throughput, it’s because the transponder is addressing packets to other users. -mel beckman
On Apr 21, 2020, at 6:53 AM, Brian J. Murrell <brian@interlinx.bc.ca> wrote:
A friend of mine just recently got Xplornet satellite service at his rural home. I'm well aware of the latency issues with satellite although frankly his latency is much better than I had feared it would be and is around 600-700ms.
But what seems to be worse than the latency is the "burstiness" of the traffic and I am just wondering if that is normal/expected for satellite service in general, and/or expected from Xplornet's service, or if what I am seeing is not expected at all (i.e. not an artifact of the satellite signal but rather a network management issue).
Here's iperf3 for 30 seconds sending data (i.e. upload speed):
[ ID] Interval Transfer Bitrate [ 5] 0.00-1.21 sec 12.9 KBytes 87.4 Kbits/sec [ 5] 1.21-2.00 sec 6.47 KBytes 67.2 Kbits/sec [ 5] 2.00-3.00 sec 22.0 KBytes 180 Kbits/sec [ 5] 3.00-4.00 sec 41.4 KBytes 339 Kbits/sec [ 5] 4.00-5.00 sec 41.4 KBytes 339 Kbits/sec [ 5] 5.00-6.00 sec 55.6 KBytes 456 Kbits/sec [ 5] 6.00-7.00 sec 69.9 KBytes 572 Kbits/sec [ 5] 7.00-8.00 sec 89.3 KBytes 731 Kbits/sec [ 5] 8.00-9.00 sec 120 KBytes 986 Kbits/sec [ 5] 9.00-10.00 sec 86.7 KBytes 710 Kbits/sec [ 5] 10.00-11.00 sec 133 KBytes 1.09 Mbits/sec [ 5] 11.00-12.00 sec 184 KBytes 1.51 Mbits/sec [ 5] 12.00-13.00 sec 186 KBytes 1.53 Mbits/sec [ 5] 13.00-14.00 sec 159 KBytes 1.30 Mbits/sec [ 5] 14.00-15.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 15.00-16.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 16.00-17.00 sec 93.2 KBytes 763 Kbits/sec [ 5] 17.00-18.00 sec 264 KBytes 2.16 Mbits/sec [ 5] 18.00-19.00 sec 124 KBytes 1.02 Mbits/sec [ 5] 19.00-20.00 sec 157 KBytes 1.28 Mbits/sec [ 5] 20.00-21.00 sec 120 KBytes 986 Kbits/sec [ 5] 21.00-22.00 sec 86.7 KBytes 710 Kbits/sec [ 5] 22.00-23.00 sec 369 KBytes 3.02 Mbits/sec [ 5] 23.00-24.00 sec 197 KBytes 1.61 Mbits/sec [ 5] 24.00-25.00 sec 90.6 KBytes 741 Kbits/sec [ 5] 25.00-26.00 sec 193 KBytes 1.58 Mbits/sec [ 5] 26.00-27.00 sec 192 KBytes 1.57 Mbits/sec [ 5] 27.00-28.00 sec 189 KBytes 1.55 Mbits/sec [ 5] 28.00-29.00 sec 193 KBytes 1.58 Mbits/sec [ 5] 29.00-30.00 sec 179 KBytes 1.46 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-32.20 sec 4.41 MBytes 1.15 Mbits/sec 388 sender [ 5] 0.00-30.00 sec 3.57 MBytes 998 Kbits/sec receiver
which averaged the overall prescribed "upload" speed, but notice that it's not 1Mb/s in any kind of a steady stream but rather bursts of higher than 1Mb/s speed followed by low/no speed. At one point it was 2 seconds with no transfer at all even.
and here's receiving (i.e. "download"):
[ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.35 sec 46.6 KBytes 283 Kbits/sec 0 12.9 KBytes [ 5] 1.35-2.00 sec 0.00 Bytes 0.00 bits/sec 0 12.9 KBytes [ 5] 2.00-3.00 sec 67.3 KBytes 551 Kbits/sec 0 37.5 KBytes [ 5] 3.00-4.00 sec 46.6 KBytes 382 Kbits/sec 0 40.1 KBytes [ 5] 4.00-5.00 sec 105 KBytes 858 Kbits/sec 0 44.0 KBytes [ 5] 5.00-6.00 sec 88.0 KBytes 721 Kbits/sec 0 54.3 KBytes [ 5] 6.00-7.00 sec 141 KBytes 1.16 Mbits/sec 0 69.9 KBytes [ 5] 7.00-8.00 sec 124 KBytes 1.02 Mbits/sec 0 101 KBytes [ 5] 8.00-9.00 sec 186 KBytes 1.53 Mbits/sec 0 146 KBytes [ 5] 9.00-10.00 sec 248 KBytes 2.04 Mbits/sec 0 206 KBytes [ 5] 10.00-11.00 sec 311 KBytes 2.54 Mbits/sec 0 257 KBytes [ 5] 11.00-12.00 sec 0.00 Bytes 0.00 bits/sec 43 194 KBytes [ 5] 12.00-13.00 sec 0.00 Bytes 0.00 bits/sec 75 199 KBytes [ 5] 13.00-14.00 sec 435 KBytes 3.56 Mbits/sec 0 199 KBytes [ 5] 14.00-15.00 sec 0.00 Bytes 0.00 bits/sec 34 114 KBytes [ 5] 15.00-16.00 sec 0.00 Bytes 0.00 bits/sec 34 140 KBytes [ 5] 16.00-17.00 sec 373 KBytes 3.05 Mbits/sec 0 149 KBytes [ 5] 17.00-18.00 sec 0.00 Bytes 0.00 bits/sec 0 162 KBytes [ 5] 18.00-19.00 sec 373 KBytes 3.05 Mbits/sec 0 168 KBytes [ 5] 19.00-20.00 sec 0.00 Bytes 0.00 bits/sec 0 171 KBytes [ 5] 20.00-21.00 sec 373 KBytes 3.05 Mbits/sec 0 172 KBytes [ 5] 21.00-22.00 sec 0.00 Bytes 0.00 bits/sec 14 141 KBytes [ 5] 22.00-23.00 sec 0.00 Bytes 0.00 bits/sec 0 120 KBytes [ 5] 23.00-24.00 sec 373 KBytes 3.05 Mbits/sec 0 131 KBytes [ 5] 24.00-25.00 sec 0.00 Bytes 0.00 bits/sec 1 146 KBytes [ 5] 25.00-26.00 sec 0.00 Bytes 0.00 bits/sec 14 104 KBytes [ 5] 26.00-27.00 sec 0.00 Bytes 0.00 bits/sec 0 104 KBytes [ 5] 27.00-28.00 sec 373 KBytes 3.05 Mbits/sec 0 107 KBytes [ 5] 28.00-29.00 sec 0.00 Bytes 0.00 bits/sec 0 119 KBytes [ 5] 29.00-30.00 sec 373 KBytes 3.05 Mbits/sec 0 123 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 3.94 MBytes 1.10 Mbits/sec 215 sender [ 5] 0.00-30.80 sec 3.13 MBytes 853 Kbits/sec receiver
Again, very bursty with periods of 1-2 seconds with no transfer.
As you can imagine, the bursiness of this makes for horrible video conferencing since that cannot "buffer" the way single-direction streams like streaming video can and the codec ends up using the "worst case" dips as the speed of the connection and encodes for that.
Cheers, b.
Hi, I used to work for a satellite ISP, and if I'm not mistaken Xplornet is buying bandwidth of my previous employer. That does not mean that your friend is on the same service. Where I worked, phy transmissions are scheduled based on tokens. A UT must have a token to transmit data. If there is no congestion, a token will be available and the UT or ground station may transmit. Congestion does not need to exist in the ground network or even the transponder. It can even be in the spectrum of that geographical area. To overcome the latency, a few things are done: - prefetching on the UT - WAN acceleration on the UT and in the network (basically syn/ack spoofing) and a few other proprietary things. Satellite is obviously not the optimal medium for video conferencing, but I would recommend that your friend tries to ratelimit their transmissions. The reason why your latency is higher than you expect, is because you expect latency to be equivalent to the round-trip time from earth to GEO and back. However, most of the time your UT will not be with the shortest line-of-sight, meaning the distance is longer. Also, the satellite merely acts as a smart mirror, and the traffic must still be backhauled from a groundstation to a processing location. They can also be several tens of milliseconds away. Then they must follow their normal path along the internet. Ping me off-list if you have specific questions. Thanks, Sabri ----- On Apr 21, 2020, at 4:58 AM, Brian J. Murrell brian@interlinx.bc.ca wrote:
A friend of mine just recently got Xplornet satellite service at his rural home. I'm well aware of the latency issues with satellite although frankly his latency is much better than I had feared it would be and is around 600-700ms.
But what seems to be worse than the latency is the "burstiness" of the traffic and I am just wondering if that is normal/expected for satellite service in general, and/or expected from Xplornet's service, or if what I am seeing is not expected at all (i.e. not an artifact of the satellite signal but rather a network management issue).
Here's iperf3 for 30 seconds sending data (i.e. upload speed):
[ ID] Interval Transfer Bitrate [ 5] 0.00-1.21 sec 12.9 KBytes 87.4 Kbits/sec [ 5] 1.21-2.00 sec 6.47 KBytes 67.2 Kbits/sec [ 5] 2.00-3.00 sec 22.0 KBytes 180 Kbits/sec [ 5] 3.00-4.00 sec 41.4 KBytes 339 Kbits/sec [ 5] 4.00-5.00 sec 41.4 KBytes 339 Kbits/sec [ 5] 5.00-6.00 sec 55.6 KBytes 456 Kbits/sec [ 5] 6.00-7.00 sec 69.9 KBytes 572 Kbits/sec [ 5] 7.00-8.00 sec 89.3 KBytes 731 Kbits/sec [ 5] 8.00-9.00 sec 120 KBytes 986 Kbits/sec [ 5] 9.00-10.00 sec 86.7 KBytes 710 Kbits/sec [ 5] 10.00-11.00 sec 133 KBytes 1.09 Mbits/sec [ 5] 11.00-12.00 sec 184 KBytes 1.51 Mbits/sec [ 5] 12.00-13.00 sec 186 KBytes 1.53 Mbits/sec [ 5] 13.00-14.00 sec 159 KBytes 1.30 Mbits/sec [ 5] 14.00-15.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 15.00-16.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 16.00-17.00 sec 93.2 KBytes 763 Kbits/sec [ 5] 17.00-18.00 sec 264 KBytes 2.16 Mbits/sec [ 5] 18.00-19.00 sec 124 KBytes 1.02 Mbits/sec [ 5] 19.00-20.00 sec 157 KBytes 1.28 Mbits/sec [ 5] 20.00-21.00 sec 120 KBytes 986 Kbits/sec [ 5] 21.00-22.00 sec 86.7 KBytes 710 Kbits/sec [ 5] 22.00-23.00 sec 369 KBytes 3.02 Mbits/sec [ 5] 23.00-24.00 sec 197 KBytes 1.61 Mbits/sec [ 5] 24.00-25.00 sec 90.6 KBytes 741 Kbits/sec [ 5] 25.00-26.00 sec 193 KBytes 1.58 Mbits/sec [ 5] 26.00-27.00 sec 192 KBytes 1.57 Mbits/sec [ 5] 27.00-28.00 sec 189 KBytes 1.55 Mbits/sec [ 5] 28.00-29.00 sec 193 KBytes 1.58 Mbits/sec [ 5] 29.00-30.00 sec 179 KBytes 1.46 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-32.20 sec 4.41 MBytes 1.15 Mbits/sec 388 sender [ 5] 0.00-30.00 sec 3.57 MBytes 998 Kbits/sec receiver
which averaged the overall prescribed "upload" speed, but notice that it's not 1Mb/s in any kind of a steady stream but rather bursts of higher than 1Mb/s speed followed by low/no speed. At one point it was 2 seconds with no transfer at all even.
and here's receiving (i.e. "download"):
[ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.35 sec 46.6 KBytes 283 Kbits/sec 0 12.9 KBytes [ 5] 1.35-2.00 sec 0.00 Bytes 0.00 bits/sec 0 12.9 KBytes [ 5] 2.00-3.00 sec 67.3 KBytes 551 Kbits/sec 0 37.5 KBytes [ 5] 3.00-4.00 sec 46.6 KBytes 382 Kbits/sec 0 40.1 KBytes [ 5] 4.00-5.00 sec 105 KBytes 858 Kbits/sec 0 44.0 KBytes [ 5] 5.00-6.00 sec 88.0 KBytes 721 Kbits/sec 0 54.3 KBytes [ 5] 6.00-7.00 sec 141 KBytes 1.16 Mbits/sec 0 69.9 KBytes [ 5] 7.00-8.00 sec 124 KBytes 1.02 Mbits/sec 0 101 KBytes [ 5] 8.00-9.00 sec 186 KBytes 1.53 Mbits/sec 0 146 KBytes [ 5] 9.00-10.00 sec 248 KBytes 2.04 Mbits/sec 0 206 KBytes [ 5] 10.00-11.00 sec 311 KBytes 2.54 Mbits/sec 0 257 KBytes [ 5] 11.00-12.00 sec 0.00 Bytes 0.00 bits/sec 43 194 KBytes [ 5] 12.00-13.00 sec 0.00 Bytes 0.00 bits/sec 75 199 KBytes [ 5] 13.00-14.00 sec 435 KBytes 3.56 Mbits/sec 0 199 KBytes [ 5] 14.00-15.00 sec 0.00 Bytes 0.00 bits/sec 34 114 KBytes [ 5] 15.00-16.00 sec 0.00 Bytes 0.00 bits/sec 34 140 KBytes [ 5] 16.00-17.00 sec 373 KBytes 3.05 Mbits/sec 0 149 KBytes [ 5] 17.00-18.00 sec 0.00 Bytes 0.00 bits/sec 0 162 KBytes [ 5] 18.00-19.00 sec 373 KBytes 3.05 Mbits/sec 0 168 KBytes [ 5] 19.00-20.00 sec 0.00 Bytes 0.00 bits/sec 0 171 KBytes [ 5] 20.00-21.00 sec 373 KBytes 3.05 Mbits/sec 0 172 KBytes [ 5] 21.00-22.00 sec 0.00 Bytes 0.00 bits/sec 14 141 KBytes [ 5] 22.00-23.00 sec 0.00 Bytes 0.00 bits/sec 0 120 KBytes [ 5] 23.00-24.00 sec 373 KBytes 3.05 Mbits/sec 0 131 KBytes [ 5] 24.00-25.00 sec 0.00 Bytes 0.00 bits/sec 1 146 KBytes [ 5] 25.00-26.00 sec 0.00 Bytes 0.00 bits/sec 14 104 KBytes [ 5] 26.00-27.00 sec 0.00 Bytes 0.00 bits/sec 0 104 KBytes [ 5] 27.00-28.00 sec 373 KBytes 3.05 Mbits/sec 0 107 KBytes [ 5] 28.00-29.00 sec 0.00 Bytes 0.00 bits/sec 0 119 KBytes [ 5] 29.00-30.00 sec 373 KBytes 3.05 Mbits/sec 0 123 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 3.94 MBytes 1.10 Mbits/sec 215 sender [ 5] 0.00-30.80 sec 3.13 MBytes 853 Kbits/sec receiver
Again, very bursty with periods of 1-2 seconds with no transfer.
As you can imagine, the bursiness of this makes for horrible video conferencing since that cannot "buffer" the way single-direction streams like streaming video can and the codec ends up using the "worst case" dips as the speed of the connection and encodes for that.
Cheers, b.
On Tue, 2020-04-21 at 11:11 -0700, Sabri Berisha wrote:
Hi,
Hi,
Where I worked, phy transmissions are scheduled based on tokens. A UT must have a token to transmit data. If there is no congestion, a token will be available and the UT or ground station may transmit. Congestion does not need to exist in the ground network or even the transponder. It can even be in the spectrum of that geographical area.
Interesting. So basically as Mel said, over-sold network. :-(
To overcome the latency,
Latency (AFAIU) is not really his primary issue. it's the lack of consistency in bandwidth. Periods of a second or two even where there is no transmission of anything at all followed by a second or two of transmission bursting even beyond his subscribed "rate". This effects his subscribed rate but in a really bad way for real-time traffic such as live/two-way video. He'd much, much more rather get a consistent pipe at his prescribed rate rather than an average of it over longer periods of time because then the codec would not have be encoding for those super bad periods of time where there are 1-2 seconds of no bandwidth at all.
Satellite is obviously not the optimal medium for video conferencing,
Indeed.
but I would recommend that your friend tries to ratelimit their transmissions.
He doesn't need to. The over-congested network is doing that for him. :-( In any case, I don't know that he has any way to put a rate limit on the tools he is using.
The reason why your latency is higher than you expect,
It actually isn't. It's nowhere near as high as I had come to (anecdotally -- I'd never had reason to do the math on the latency before now) believe it would be. Fortunately he might be a candidate for Xplornet (or others') WISP services. Hopefully they are a bit more stable. Cheers, b.
It’s not really oversold bandwidth. It’s just that the turnaround time for a bolus of data is too long for two-way video conferencing to be smooth or reliable. It’s like video conferencing using post cards :) -mel
On Apr 21, 2020, at 11:36 AM, Brian J. Murrell <brian@interlinx.bc.ca> wrote:
On Tue, 2020-04-21 at 11:11 -0700, Sabri Berisha wrote:
Hi,
Hi,
Where I worked, phy transmissions are scheduled based on tokens. A UT must have a token to transmit data. If there is no congestion, a token will be available and the UT or ground station may transmit. Congestion does not need to exist in the ground network or even the transponder. It can even be in the spectrum of that geographical area.
Interesting. So basically as Mel said, over-sold network. :-(
To overcome the latency,
Latency (AFAIU) is not really his primary issue. it's the lack of consistency in bandwidth. Periods of a second or two even where there is no transmission of anything at all followed by a second or two of transmission bursting even beyond his subscribed "rate". This effects his subscribed rate but in a really bad way for real-time traffic such as live/two-way video. He'd much, much more rather get a consistent pipe at his prescribed rate rather than an average of it over longer periods of time because then the codec would not have be encoding for those super bad periods of time where there are 1-2 seconds of no bandwidth at all.
Satellite is obviously not the optimal medium for video conferencing,
Indeed.
but I would recommend that your friend tries to ratelimit their transmissions.
He doesn't need to. The over-congested network is doing that for him. :-( In any case, I don't know that he has any way to put a rate limit on the tools he is using.
The reason why your latency is higher than you expect,
It actually isn't. It's nowhere near as high as I had come to (anecdotally -- I'd never had reason to do the math on the latency before now) believe it would be.
Fortunately he might be a candidate for Xplornet (or others') WISP services. Hopefully they are a bit more stable.
Cheers, b.
On 4/21/20 2:54 PM, Mel Beckman wrote:
It’s not really oversold bandwidth. It’s just that the turnaround time for a bolus of data is too long for two-way video conferencing to be smooth or reliable. It’s like video conferencing using post cards :)
Are consumer satellite provider networks TDD or FDD? I guess I had assumed the latter, but maybe not? Assuming FDD, I don't see why the downstream needs to necessarily have problems with extremely long user-data-striping periods unless there's some factor I am just totally not considering. The upstream is of course more of a problem. I assume it's TDMA, and the terminals have imperfect clocks. -- Brandon Martin
On Tue, 2020-04-21 at 18:54 +0000, Mel Beckman wrote:
It’s not really oversold bandwidth. It’s just that the turnaround time for a bolus of data is too long for two-way video conferencing to be smooth or reliable. It’s like video conferencing using post cards :)
Except that videoconferencing is just the victim of the problem, and the problem is bursty bandwidth not latency. In fact, the back-and- forth of conversation is actually surprisingly decent for satellite. Not as much "talking over" as I would have suspected. But put the victim application aside, the real data is in the iperf3 results I posted, demonstrating how bursty the throughput is. The problem with that of course is that the "lowest" bandwidth "valleys" becomes the "constant bandwidth" that the codec uses rather than the average -- which of course cannot be used for real-time VC. Cheers, b.
On 4/21/20 2:35 PM, Brian J. Murrell wrote:
Interesting. So basically as Mel said, over-sold network. :-(
This is pretty typical of consumer VSAT and such. You can of course get better performance...if you're willing to pay for it. If you find the right carrier/re-seller, you can perhaps find one whose "system" (to include ground station, terminals, and smarts on the bird) can respect DSCP and flag at least your voice traffic appropriately (probably EF) to perhaps lower the jitter and make conferencing more palatable. Finding competent folks at a typical consumer provider's helpdesk to talk to about such things is probably the limiting factor. The higher up the foodchain you go towards the folks who "own" the spectrum rather than re-sell will perhaps get you more luck on something like this though probably also at higher MRC. Unfortunately, it's just what happens when you spread an already limited resource (transponder bandwidth) out over essentially an entire continent or at least substantial portions of it. Imagine if you had a cable provider with a single node for an entire, say, US state. -- Brandon Martin
On 21.04.2020 20:59, Brandon Martin wrote:
On 4/21/20 2:35 PM, Brian J. Murrell wrote:
Interesting. So basically as Mel said, over-sold network. :-( This is pretty typical of consumer VSAT and such. You can of course get better performance...if you're willing to pay for it. If you find the right carrier/re-seller, you can perhaps find one whose "system" (to include ground station, terminals, and smarts on the bird) can respect DSCP and flag at least your voice traffic appropriately (probably EF) to perhaps lower the jitter and make conferencing more palatable. Finding competent folks at a typical consumer provider's helpdesk to talk to about such things is probably the limiting factor. The higher up the foodchain you go towards the folks who "own" the spectrum rather than re-sell will perhaps get you more luck on something like this though probably also at higher MRC.
Unfortunately, it's just what happens when you spread an already limited resource (transponder bandwidth) out over essentially an entire continent or at least substantial portions of it. Imagine if you had a cable provider with a single node for an entire, say, US state.
Could be interesting to do one-way-delay measurements in parallel to see how the packets are travelling and where and how big the queues are (Bufferbloat). Likeways if you have a unix system you could look at tcp statistics to see if you have packet retransmits (netstat -s). As previously noted TDMA between uploading or even ACK'ing users could be a bottleneck for the download speed. Olav
participants (6)
-
Adam Thompson -
Brandon Martin -
Brian J. Murrell -
Mel Beckman -
Olav Kvittem -
Sabri Berisha