Would anyone mind sharing with me their first-hand experiences with residential satellite internet? Right now I am evaluating HughesNet Gen4 and ViaSat Exede and I'm thinking specifically as a sysadmin who needs to use the uplink for work, not surf. What are your experiences with the following applications? -SSH, (specifically interactive CLI shell access) -RDP -SIP over SSL -IPSec Tunneling (should be a non-starter due to latency) -GRE Tunneling Thank you, -Nicholas
On Mon, Jun 22, 2015 at 4:39 PM, Nicholas Oas <nicholas.oas@gmail.com> wrote:
Would anyone mind sharing with me their first-hand experiences with residential satellite internet?
Hi Nicholas, Two-way satellite systems based on SV's in geostationary orbit (like the two you're considering) have high latency. 22,000 miles out, another 22,000 miles back and do it again for the return packet. You'll start around 500ms latency and go up from there. Any kind of interactive session (like SSH and RDP) will be excruciating. I'm not aware of any low earth orbit systems providing residential Internet. Iridium tries to but the bandwidth is pathetic (2400bps or so). LEO vehicles are only 500-1000 miles up. Latency is high compared to wireline systems but within usable bounds for interactive sessions. Regards, Bill Herrin -- William Herrin ................ herrin@dirtside.com bill@herrin.us Owner, Dirtside Systems ......... Web: <http://www.dirtside.com/>
On Jun 22, 2015, at 3:11 PM, William Herrin <bill@herrin.us> wrote:
Two-way satellite systems based on SV's in geostationary orbit (like the two you're considering) have high latency. 22,000 miles out, another 22,000 miles back and do it again for the return packet. You'll start around 500ms latency and go up from there. Any kind of interactive session (like SSH and RDP) will be excruciating.
It is indeed. This is first-hand in the sense that I once worked for an earth station manufacturer and did a fair bit of work related to this environment, and second-hand in that my sister, for a while, used VSAT connectivity to her home. The trick in the context is what's called a "performance-enhancing proxy", or PEP. What it does, in concept, is terminate a TCP connection at each earth station and use some form of private protocol over the bird. Cisco RBSCP (which maps TCP connections to SCTP sessions over the bird, http://www.cisco.com/c/en/us/td/docs/ios-xml/ios/interface/configuration/15-...) is an example of such a technology. The obvious benefit of a PEP is that it can convince a TCP sender to keep enough data in flight to make good use of the throughput rate of the satellite - you have a start-up issue with the first RTT, but after that it has essentially figured out what the effective window should be and makes that happen. The downside of a PEP is when the application is itself interactive (it's not about rate, it's about responsiveness clocked by end-to-end RTT) or the protocol in question isn't TCP (noting that TCP in IPsec ESP isn't TCP to the PEP - it's IPsec ESP, and can't be goosed along). In my sister's case, her description of the service was somewhat colorful, and included the word "slow".
On Jun 22, 2015 6:14 PM, "William Herrin" <bill@herrin.us> wrote:
Two-way satellite systems based on SV's in geostationary orbit (like the two you're considering) have high latency. 22,000 miles out, another 22,000 miles back and do it again for the return packet.
Just a minor nitpick - that's 22,300 miles above the equator at sea level. You're probably closer to 22,500 miles away from the bird (as could your uplink). That's just rough math adding the tangent of 1500 miles from the equator in my head (plus the tangent of the curve distance from that base line and angle of the bird :) ).
On Fri, Jun 26, 2015 at 1:19 PM, shawn wilson <ag4ve.us@gmail.com> wrote:
On Jun 22, 2015 6:14 PM, "William Herrin" <bill@herrin.us> wrote:
Two-way satellite systems based on SV's in geostationary orbit (like the two you're considering) have high latency. 22,000 miles out, another 22,000 miles back and do it again for the return packet.
Just a minor nitpick - that's 22,300 miles above the equator at sea level. You're probably closer to 22,500 miles away from the bird (as could your uplink). That's just rough math adding the tangent of 1500 miles from the equator in my head (plus the tangent of the curve distance from that base line and angle of the bird :) ).
Typically further than that because you're not only not at the same latitude as the bird, you're not at the same longitude either. If you want to nitpick. ;) -Bill -- William Herrin ................ herrin@dirtside.com bill@herrin.us Owner, Dirtside Systems ......... Web: <http://www.dirtside.com/>
SIP will suck. VPN will suck. RDP will suck. Have you looked to see if you have any local wireless ISPs in your area? Hit me up offlist if you want me to check for you. -Mike On Mon, Jun 22, 2015 at 1:39 PM, Nicholas Oas <nicholas.oas@gmail.com> wrote:
Would anyone mind sharing with me their first-hand experiences with residential satellite internet?
Right now I am evaluating HughesNet Gen4 and ViaSat Exede and I'm thinking specifically as a sysadmin who needs to use the uplink for work, not surf.
What are your experiences with the following applications? -SSH, (specifically interactive CLI shell access) -RDP -SIP over SSL -IPSec Tunneling (should be a non-starter due to latency) -GRE Tunneling
Thank you,
-Nicholas
-- Mike Lyon 408-621-4826 mike.lyon@gmail.com http://www.linkedin.com/in/mlyon
Interesting that you say that about sip. We had a client that would use it for sip on ships all the time. It wasn't the best but it worked. Ping times were between 500-700ms. Regards, Dovid -----Original Message----- From: Mike Lyon <mike.lyon@gmail.com> Sender: "NANOG" <nanog-bounces@nanog.org>Date: Mon, 22 Jun 2015 15:33:43 To: Nicholas Oas<nicholas.oas@gmail.com>; NANOG<nanog@nanog.org> Subject: Re: Residential VSAT experiences? SIP will suck. VPN will suck. RDP will suck. Have you looked to see if you have any local wireless ISPs in your area? Hit me up offlist if you want me to check for you. -Mike On Mon, Jun 22, 2015 at 1:39 PM, Nicholas Oas <nicholas.oas@gmail.com> wrote:
Would anyone mind sharing with me their first-hand experiences with residential satellite internet?
Right now I am evaluating HughesNet Gen4 and ViaSat Exede and I'm thinking specifically as a sysadmin who needs to use the uplink for work, not surf.
What are your experiences with the following applications? -SSH, (specifically interactive CLI shell access) -RDP -SIP over SSL -IPSec Tunneling (should be a non-starter due to latency) -GRE Tunneling
Thank you,
-Nicholas
-- Mike Lyon 408-621-4826 mike.lyon@gmail.com http://www.linkedin.com/in/mlyon
I never had good luck with VSAT and SIP. Maybe you had a better kit than I did :) -Mike On Mon, Jun 22, 2015 at 3:49 PM, Dovid Bender <dovid@telecurve.com> wrote:
Interesting that you say that about sip. We had a client that would use it for sip on ships all the time. It wasn't the best but it worked. Ping times were between 500-700ms.
Regards,
Dovid
-----Original Message----- From: Mike Lyon <mike.lyon@gmail.com> Sender: "NANOG" <nanog-bounces@nanog.org>Date: Mon, 22 Jun 2015 15:33:43 To: Nicholas Oas<nicholas.oas@gmail.com>; NANOG<nanog@nanog.org> Subject: Re: Residential VSAT experiences?
SIP will suck. VPN will suck. RDP will suck.
Have you looked to see if you have any local wireless ISPs in your area? Hit me up offlist if you want me to check for you.
-Mike
On Mon, Jun 22, 2015 at 1:39 PM, Nicholas Oas <nicholas.oas@gmail.com> wrote:
Would anyone mind sharing with me their first-hand experiences with residential satellite internet?
Right now I am evaluating HughesNet Gen4 and ViaSat Exede and I'm thinking specifically as a sysadmin who needs to use the uplink for work, not surf.
What are your experiences with the following applications? -SSH, (specifically interactive CLI shell access) -RDP -SIP over SSL -IPSec Tunneling (should be a non-starter due to latency) -GRE Tunneling
Thank you,
-Nicholas
-- Mike Lyon 408-621-4826 mike.lyon@gmail.com
-- Mike Lyon 408-621-4826 mike.lyon@gmail.com http://www.linkedin.com/in/mlyon
Personally, 500-700ms of delay is well within distinguishable range and causes challenges in verbal communication. If the speakers are both expecting and accustomed to delay like that (e.g. sailors that are used to being hundreds/thousands of miles away from anywhere and any other comms solution sucks anyway), it could be workable. For regular consumer/business voice applications, 100ms and lower is decent, but above that starts to get into various degrees of suckage. Just my 2c. -- Hugo On Mon 2015-Jun-22 15:54:49 -0700, Mike Lyon <mike.lyon@gmail.com> wrote:
I never had good luck with VSAT and SIP. Maybe you had a better kit than I did :)
-Mike
On Mon, Jun 22, 2015 at 3:49 PM, Dovid Bender <dovid@telecurve.com> wrote:
Interesting that you say that about sip. We had a client that would use it for sip on ships all the time. It wasn't the best but it worked. Ping times were between 500-700ms.
Regards,
Dovid
-----Original Message----- From: Mike Lyon <mike.lyon@gmail.com> Sender: "NANOG" <nanog-bounces@nanog.org>Date: Mon, 22 Jun 2015 15:33:43 To: Nicholas Oas<nicholas.oas@gmail.com>; NANOG<nanog@nanog.org> Subject: Re: Residential VSAT experiences?
SIP will suck. VPN will suck. RDP will suck.
Have you looked to see if you have any local wireless ISPs in your area? Hit me up offlist if you want me to check for you.
-Mike
On Mon, Jun 22, 2015 at 1:39 PM, Nicholas Oas <nicholas.oas@gmail.com> wrote:
Would anyone mind sharing with me their first-hand experiences with residential satellite internet?
Right now I am evaluating HughesNet Gen4 and ViaSat Exede and I'm thinking specifically as a sysadmin who needs to use the uplink for work, not surf.
What are your experiences with the following applications? -SSH, (specifically interactive CLI shell access) -RDP -SIP over SSL -IPSec Tunneling (should be a non-starter due to latency) -GRE Tunneling
Thank you,
-Nicholas
-- Mike Lyon 408-621-4826 mike.lyon@gmail.com
-- Mike Lyon 408-621-4826 mike.lyon@gmail.com
I too have had customers in a previous life where the 500ms delay really didn't cause any big issue. Same with SSH and even heavier stuff like SMB. Sure, it was slower than expected, but I could still saturate the pipe pretty good. Thing is...the kind of setups where you're getting 500ms delay with little jitter is stupidly expensive. Those are generally going to be an SCPC (single carrier per channel) uplink with hopefully something like IP over DVB providing a large pool of downlink bandwidth. Expect to pay over 4k per Megahertz (roughly translated to 1 Mbps unidirectional depending your link budgets) of bandwidth (sometimes substantially more, depending on what bird and provider you're using). O3B looks really interesting. I'm not aware of what they're current state of deployment is, but they've got a MEO (I think) constellation planned which will help a lot of with that latency. Viasat had something that looked promising too. I mean..if you're looking at doing sysadmin type stuff where you're already going to be pulling out your hair at times, doing so over hughesnet is going to suuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuck. On Mon, Jun 22, 2015 at 4:04 PM, Hugo Slabbert <hugo@slabnet.com> wrote:
Personally, 500-700ms of delay is well within distinguishable range and causes challenges in verbal communication. If the speakers are both expecting and accustomed to delay like that (e.g. sailors that are used to being hundreds/thousands of miles away from anywhere and any other comms solution sucks anyway), it could be workable.
For regular consumer/business voice applications, 100ms and lower is decent, but above that starts to get into various degrees of suckage.
Just my 2c.
-- Hugo
On Mon 2015-Jun-22 15:54:49 -0700, Mike Lyon <mike.lyon@gmail.com> wrote:
I never had good luck with VSAT and SIP. Maybe you had a better kit than I did :)
-Mike
On Mon, Jun 22, 2015 at 3:49 PM, Dovid Bender <dovid@telecurve.com> wrote:
Interesting that you say that about sip. We had a client that would use it for sip on ships all the time. It wasn't the best but it worked. Ping times were between 500-700ms.
Regards,
Dovid
-----Original Message----- From: Mike Lyon <mike.lyon@gmail.com> Sender: "NANOG" <nanog-bounces@nanog.org>Date: Mon, 22 Jun 2015 15:33:43 To: Nicholas Oas<nicholas.oas@gmail.com>; NANOG<nanog@nanog.org> Subject: Re: Residential VSAT experiences?
SIP will suck. VPN will suck. RDP will suck.
Have you looked to see if you have any local wireless ISPs in your area? Hit me up offlist if you want me to check for you.
-Mike
On Mon, Jun 22, 2015 at 1:39 PM, Nicholas Oas <nicholas.oas@gmail.com> wrote:
Would anyone mind sharing with me their first-hand experiences with residential satellite internet?
Right now I am evaluating HughesNet Gen4 and ViaSat Exede and I'm thinking specifically as a sysadmin who needs to use the uplink for work, not surf.
What are your experiences with the following applications? -SSH, (specifically interactive CLI shell access) -RDP -SIP over SSL -IPSec Tunneling (should be a non-starter due to latency) -GRE Tunneling
Thank you,
-Nicholas
-- Mike Lyon 408-621-4826 mike.lyon@gmail.com
-- Mike Lyon 408-621-4826 mike.lyon@gmail.com
-- 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0
Interesting that you say that about sip. We had a client that would use it for sip on ships all the time. It wasn't the best but it worked. Ping times were between 500-700ms.
It really depends on your expectations - or more to the point, your end-users' expectations. I've tested SIP in the lab up to 2000ms RTT. The protocols all hang together and keep working, but it's obviously very much in walkie-talkie mode, you can't hold a normal duplex conversation. 500ms there's more of the talking over each other / "sorry, you go" / "no, you go" dance, but it *is* workable. If your end-user is expecting land-line replacement though... Regards, Tim.
Reading about SIP made it seem like latency alone is not an issue, aside from delays which impact verbal communication as previously mentioned. What is going to be much worse is jitter and packet loss. You can eventually get used to a significant delay, but dropped calls and chopped sound renders the service useless. On Tue, Jun 23, 2015 at 3:44 AM, Tim Franklin <tim@pelican.org> wrote:
Interesting that you say that about sip. We had a client that would use it for sip on ships all the time. It wasn't the best but it worked. Ping times were between 500-700ms.
It really depends on your expectations - or more to the point, your end-users' expectations.
I've tested SIP in the lab up to 2000ms RTT. The protocols all hang together and keep working, but it's obviously very much in walkie-talkie mode, you can't hold a normal duplex conversation. 500ms there's more of the talking over each other / "sorry, you go" / "no, you go" dance, but it *is* workable. If your end-user is expecting land-line replacement though...
Regards, Tim.
On Tue, Jun 23, 2015 at 9:37 AM, Rafael Possamai <rafael@gav.ufsc.br> wrote:
Reading about SIP made it seem like latency alone is not an issue, aside from delays which impact verbal communication as previously mentioned. What is going to be much worse is jitter and packet loss. You can eventually get used to a significant delay, but dropped calls and chopped sound renders the service useless.
With modern software implementing a responsive jitter buffer, jitter shouldn't be much of a problem. Practical effect would be a longer delay as the receiver buffers enough packets to deal with the measured variance in receipt times. Perhaps a few chops early in the conversation as the software grows the buffer. Not all SIP implementations are equal. Try yours in a high-jitter environment and see what happens. High packet loss is deadly. That'll depend on the satellite vendor's network implementation, the weather, etc. But then high packet loss is deadly to essentially all IP networking activity. In situations where a high bit error rate (BER) is endemic, the layer-2 vendor is expected to redress that with forward error correction (FEC) and retransmission that trades jitter for loss. I have no idea which satellite vendors are better or worse about this. Regards, Bill Herrin -- William Herrin ................ herrin@dirtside.com bill@herrin.us Owner, Dirtside Systems ......... Web: <http://www.dirtside.com/>
On Jun 22, 2015, at 4:39 PM, Nicholas Oas <nicholas.oas@gmail.com> wrote:
Would anyone mind sharing with me their first-hand experiences with residential satellite internet?
Right now I am evaluating HughesNet Gen4 and ViaSat Exede and I'm thinking specifically as a sysadmin who needs to use the uplink for work, not surf.
My experience with geostationary was that latencies were around 720 ms in practice. Telnet was painful and it turns out my brain didn’t like typing things while I wasn’t getting instant feedback, though I understand there’s software for that problem now. Reliability was pretty good unless the satellite I was using happened to lose it’s control processors. I was using Panamsat Galaxy 4 when it failed. I don’t know how many angry phone calls we got about why we weren’t answering our pages about the entire network being down before we got into the office. In our case recovery involved getting access to another satellite and having people re-aim the dishes. My only real recollection besides that was that the signal was bad enough/long enough to induce TCP Silly Window Syndrome; but I can’t imagine anyone’s running an OS that old anymore. — Mike
On Mon, 22 Jun 2015, Michael Conlen wrote:
typing things while I wasn’t getting instant feedback, though I understand there’s software for that problem now.
Yes, https://mosh.mit.edu/ is your friend if you want to do things interactively. Still, satellite is painful, avoid if anything else decent is available. -- Mikael Abrahamsson email: swmike@swm.pp.se
On Mon, Jun 22, 2015 at 5:10 PM, Michael Conlen <mike@conlen.org> wrote:
On Jun 22, 2015, at 4:39 PM, Nicholas Oas <nicholas.oas@gmail.com> wrote:
Would anyone mind sharing with me their first-hand experiences with residential satellite internet?
Right now I am evaluating HughesNet Gen4 and ViaSat Exede and I'm thinking specifically as a sysadmin who needs to use the uplink for work, not surf.
My experience with geostationary was that latencies were around 720 ms in practice. Telnet was painful and it turns out my brain didn’t like typing things while I wasn’t getting instant feedback, though I understand there’s software for that problem now.
Mosh makes latencies like this a lot less painful. Still painful.
Reliability was pretty good unless the satellite I was using happened to lose it’s control processors. I was using Panamsat Galaxy 4 when it failed. I don’t know how many angry phone calls we got about why we weren’t answering our pages about the entire network being down before we got into the office. In our case recovery involved getting access to another satellite and having people re-aim the dishes.
My only real recollection besides that was that the signal was bad enough/long enough to induce TCP Silly Window Syndrome; but I can’t imagine anyone’s running an OS that old anymore.
— Mike
-- Dave Täht worldwide bufferbloat report: http://www.dslreports.com/speedtest/results/bufferbloat And: What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast
On Mon, Jun 22, 2015 at 10:39 PM, Nicholas Oas <nicholas.oas@gmail.com> wrote:
Would anyone mind sharing with me their first-hand experiences with residential satellite internet?
Right now I am evaluating HughesNet Gen4 and ViaSat Exede and I'm thinking specifically as a sysadmin who needs to use the uplink for work, not surf.
What are your experiences with the following applications? -SSH, (specifically interactive CLI shell access) -RDP -SIP over SSL -IPSec Tunneling (should be a non-starter due to latency) -GRE Tunneling
Working for an satellite ISP I can give you some technical background. We're only target enterprise/government/military customers with more specific use cases because offering satellite based Internet to residential customers without making them angry while being profitable is hard. So I've no experience with HughesNet Gen4 or ViaSat Exede as products in particular but I know the underling platforms. In general the systems are optimized for fast browsing and VoIP from the own operator. The modems you'll use with the mentioned services will include all kinds of acceleration features. General acceleration of TCP sessions to work around TCP implementation issues in combination the the high RTT (slow start, behavior during packet loss/high jitter, window scaling, ...) are standard for these services. The residential services usually use additional acceleration features like HTTP prefetching/pushing. That's usually done using a transparent HTTP proxy which sits at the teleport analyzing all HTTP requests/responses, download images etc. already before they are actually requested the the end users browser. They are then pushed to the modem which will delivery them as soon as the end user requests them. As a result the end user doesn't have to wait another RTT for the images etc.. Similar sniffing/spoofing acceleration options are available for other protocols. But with end to end encryption becoming more common these days all these transparent higher level acceleration features of the modems, etc. no longer work. Of course you still can do the same but you have to move the acceleration to the client device. That's not very common yet in the satellite industry. Regarding phone conversations our experience is that the high RTT is not that much of an problem in practice. People recognize the delay and wait longer before starting to talk automatically. But the experience might vary extremely depending on the operators config and end devices. You need corresponding QoS settings to keep latency/jitter low and stable. For residential services the return channel will be most likely time division multiplexed. Once the network is congested (=profitable for the operator) you'll see the latency go beyond 1 second more or less often without proper QoS settings. That of course will completely break your VoIP experience. You should expect that the operator only has corresponding QoS settings for their own VoIP service in place. Experience with third party services might suck due to that. Another issue you might run into are some VoIP phones. Some of them only support very small jitter buffers (<10ms) which might cause problems. IPsec tunneling, GRE tunneling etc. should work perfectly fine unless it's intentionally filtered. But as soon as you do tunneling/encryption you should expect that you byepass any acceleration feature or high priority QoS rule. Besides that both products you mentioned AFAIK are using Ka-Band spot beam satellites. There's probably roughly one beam per US state. Assuming 200 MHz per beam that translate to roughly to a maximum of 600-700 Mbit/s downstream capacity shared by all customers in that beam (one state). Upstream is probably designed for half of that. This grouping of customers also makes a simple experience comparison difficult as your experience will heavily depend on the congestion level in your beam. From other similar services we already know that at the launch new customers are happy (always getting the maximum speed) but as the network fills up they get angry due to bad performance during peak times. I really wouldn't recommend a sysadmin to use a geo stationary satellite based connection for your daily work unless there's no other way - simply due to the latency. You'll notice a significant productivity impact. Best Regards, Frederik Kriewitz
On 23 Jun 2015, at 3:39, Nicholas Oas wrote:
What are your experiences with the following applications? -SSH, (specifically interactive CLI shell access) -RDP -SIP over SSL -IPSec Tunneling (should be a non-starter due to latency) -GRE Tunneling
Latency, latency, latency, RTTs, RTTs, RTTs. Not an option for anything interactive, very poor for general user-type Internet access. ----------------------------------- Roland Dobbins <rdobbins@arbor.net>
Thank you all for your responses. This was exactly the kind of information and opinions I was hoping to find- way better than reading tea leaves!
participants (16)
-
Dave Taht -
Dovid Bender -
Fred Baker (fred) -
Frederik Kriewitz -
Gary Buhrmaster -
Hugo Slabbert -
Michael Conlen -
Mikael Abrahamsson -
Mike Hale -
Mike Lyon -
Nicholas Oas -
Rafael Possamai -
Roland Dobbins -
shawn wilson -
Tim Franklin -
William Herrin