Dear all, Can anyone share any thoughts or experiences for VPN links running over slow Internet connections, typically 2kB/s - 3kB/s (think 33.6k modem)? We are looking into utilising OpenVPN for out-of-office workers who would be running mobile broadband in rural areas. Typical data across the wire would be SQL queries for custom applications and not much else. Some initial thoughts include... * How well would the connection handle certificate (>= 2048 bit key) based authentication? * Is UDP or TCP better considering the speed and possibility of packet loss (no figures to hand)? * Is VPN over this type of connection simply a bad idea? Many thanks in advance. Kind regards, Ben Whorwood
If I had to guestimate, the performance would be horrible considering the VPN overhead in itself. You can't choose UDP or TCP, that is all based on the applications being used within the tunnel. So the apps will decide what protocols they will need to use, which will then be encapsulated by IPSEC. It could work, but you may not be happy and it may not provide the desired performance that you need to be productive....
Date: Thu, 21 Apr 2011 17:55:32 +0100 From: bw-ml@mube.co.uk To: nanog@nanog.org Subject: VPN over slow Internet connections
Dear all,
Can anyone share any thoughts or experiences for VPN links running over slow Internet connections, typically 2kB/s - 3kB/s (think 33.6k modem)?
We are looking into utilising OpenVPN for out-of-office workers who would be running mobile broadband in rural areas. Typical data across the wire would be SQL queries for custom applications and not much else.
Some initial thoughts include...
* How well would the connection handle certificate (>= 2048 bit key) based authentication? * Is UDP or TCP better considering the speed and possibility of packet loss (no figures to hand)? * Is VPN over this type of connection simply a bad idea?
Many thanks in advance.
Kind regards, Ben Whorwood
Ben Whorwood (bw-ml) writes:
Some initial thoughts include...
* How well would the connection handle certificate (>= 2048 bit key) based authentication? * Is UDP or TCP better considering the speed and possibility of packet loss (no figures to hand)?
I'd go for a UDP tunnel, as you wouldn't have to renegotiate a TCP session for the tunnel *and* whatever connection you've got going through that.
* Is VPN over this type of connection simply a bad idea?
I don't think it's a particularly bad idea. But why don't you make you own tests using FreeBSD/dummynet, simulating 1-2% packet loss, limit bandwidth to 33 Kbit/s, and corresponding latency (say 100ms). I'd say your biggest concern won't be the VPN (you can make it completely stateless with static keys), but whatever protocol you've got running on top of that, and how it deals with the loss. Cheers, Phil
There's not that much overhead--your certs should be ok. TCP for SQL would just make sense. I personally wouldn't want to do what you are contemplating. Here's some stuff to think about: 1. your modems will not be able to do compression. You can't easily compress random data (e.g. encrypted). 2. you won't get 33.6 unless your phone lines are pristine. You better plan on 28.8--if you are lucky. 3. I would hone my SQL sharply so it produces the smallest most relevant data sets possible. 4. you might want to give them some kind of termnial/shell access for doing their SQL remotely, instead of from home. Telnet or SSH. If you used SSH you could obviate using a separate VPN, you could use -C for compression, and you could do your SQL on the server side (or the on-site side)--all in all a speedier alternative. --Patrick Darden -----Original Message----- From: Ben Whorwood [mailto:bw-ml@mube.co.uk] Sent: Thursday, April 21, 2011 12:56 PM To: nanog@nanog.org Subject: VPN over slow Internet connections Dear all, Can anyone share any thoughts or experiences for VPN links running over slow Internet connections, typically 2kB/s - 3kB/s (think 33.6k modem)? We are looking into utilising OpenVPN for out-of-office workers who would be running mobile broadband in rural areas. Typical data across the wire would be SQL queries for custom applications and not much else. Some initial thoughts include... * How well would the connection handle certificate (>= 2048 bit key) based authentication? * Is UDP or TCP better considering the speed and possibility of packet loss (no figures to hand)? * Is VPN over this type of connection simply a bad idea? Many thanks in advance. Kind regards, Ben Whorwood
We are looking into utilising OpenVPN for out-of-office workers who would be running mobile broadband in rural areas. Typical data across the wire would be SQL queries for custom applications and not much else.
I agree with Patrick, SSH would do nicely. You could even setup a tunnel, and the sql queries could hit localhost:3306 (for mysql for example) and connect, encrypted to a remote server. At 2-3KB/sec, I wouldn't expect the ability to have too many people concurrently connecting to the database. -- Fred
I vote for Patrick's idea of allowing the end user to remote into a machine where the SQL resides. This would eliminate a lot of potential issues....wish I had thought of that first!!!
Subject: RE: VPN over slow Internet connections Date: Thu, 21 Apr 2011 13:10:09 -0400 From: darden@armc.org To: bw-ml@mube.co.uk; nanog@nanog.org
There's not that much overhead--your certs should be ok. TCP for SQL would just make sense. I personally wouldn't want to do what you are contemplating. Here's some stuff to think about:
1. your modems will not be able to do compression. You can't easily compress random data (e.g. encrypted). 2. you won't get 33.6 unless your phone lines are pristine. You better plan on 28.8--if you are lucky. 3. I would hone my SQL sharply so it produces the smallest most relevant data sets possible.
4. you might want to give them some kind of termnial/shell access for doing their SQL remotely, instead of from home. Telnet or SSH. If you used SSH you could obviate using a separate VPN, you could use -C for compression, and you could do your SQL on the server side (or the on-site side)--all in all a speedier alternative.
--Patrick Darden
-----Original Message----- From: Ben Whorwood [mailto:bw-ml@mube.co.uk] Sent: Thursday, April 21, 2011 12:56 PM To: nanog@nanog.org Subject: VPN over slow Internet connections
Dear all,
Can anyone share any thoughts or experiences for VPN links running over slow Internet connections, typically 2kB/s - 3kB/s (think 33.6k modem)?
We are looking into utilising OpenVPN for out-of-office workers who would be running mobile broadband in rural areas. Typical data across the wire would be SQL queries for custom applications and not much else.
Some initial thoughts include...
* How well would the connection handle certificate (>= 2048 bit key) based authentication? * Is UDP or TCP better considering the speed and possibility of packet loss (no figures to hand)? * Is VPN over this type of connection simply a bad idea?
Many thanks in advance.
Kind regards, Ben Whorwood
On 04/21/2011 01:32 PM, Brandon Kim wrote:
I vote for Patrick's idea of allowing the end user to remote into a machine where the SQL resides.
This would eliminate a lot of potential issues....wish I had thought of that first!!! I third this idea. Using screen would be a good idea as well.
This reminds me a project I worked on last century were we had people direct dialing our facility over modems to use a custom DB front end presented using Citrix. One of the big challenges was dropped calls. Persistence is your friend under these circumstances. At least the end users don't lose work.
Nothing like getting into the groove, then losing your connection, waiting for the modem to dial back up and then try to figure out what you were just doing!!! Again, it goes back to what I mentioned, it "could" work but how will that affect your overall productivity. Is over the air 3G or 4G not available? I'm assuming that modem is being used because broadband is not in the area....
Date: Thu, 21 Apr 2011 14:02:30 -0400 From: ryanczak@gmail.com To: nanog@nanog.org Subject: Re: VPN over slow Internet connections
On 04/21/2011 01:32 PM, Brandon Kim wrote:
I vote for Patrick's idea of allowing the end user to remote into a machine where the SQL resides.
This would eliminate a lot of potential issues....wish I had thought of that first!!! I third this idea. Using screen would be a good idea as well.
This reminds me a project I worked on last century were we had people direct dialing our facility over modems to use a custom DB front end presented using Citrix.
One of the big challenges was dropped calls. Persistence is your friend under these circumstances. At least the end users don't lose work.
On 21/04/11 11:53 AM, Brandon Kim wrote:
Nothing like getting into the groove, then losing your connection, waiting for the modem to dial back up and then try to figure out what you were just doing!!! Again, it goes back to what I mentioned, it "could" work but how will that affect your overall productivity.
Is over the air 3G or 4G not available? I'm assuming that modem is being used because broadband is not in the area....
The OP said:
Can anyone share any thoughts or experiences for VPN links running over slow Internet connections, typically 2kB/s - 3kB/s (think 33.6k modem)?
We are looking into utilising OpenVPN for out-of-office workers who would be running mobile broadband in rural areas.
It looks like they are using the fastest mobile (cellular) access solutions available in these rural areas with throughput speeds that are *about* the same as dial-up modem speeds. Mobile broadband (sic) is tuned for faster downloads, upload speeds can be very very slow, especially if you are on the edge of their service range. (I used Verizon EVDO devices for several years, have extensive first-hand experience in how slowly they work in many rural areas.) jc
On Thu, 21 Apr 2011 17:55:32 BST, Ben Whorwood said:
* How well would the connection handle certificate (>= 2048 bit key) based authentication?
It will hiccup for a moment (maybe a quarter or half second) for the data. The certificate exchange is the least of your problems.
* Is VPN over this type of connection simply a bad idea?
Well, 33.6k is a Bad Idea right there. :) But if you're stuck with that for technical reasons, but need a VPN for security reasons, it won't be all *that* much worse, unless you're doing a lot of SSH or similar short-packet single-keystroke traffic, where the VPN overhead will start being a bit painful. Shouldn't be too hard to model the traffic involved to see if it's too painful - FreeBSD has dummynet IIRC.
Valdis.Kletnieks@vt.edu wrote:
Well, 33.6k is a Bad Idea right there. :) But if you're stuck with that for technical reasons, but need a VPN for security reasons, it won't be all *that* much worse, unless you're doing a lot of SSH or similar
I would think so too. When I first moved to the States I lived in rural Oregon and used an (attempted) always on dialup for years. I could mostly get a steady 56K connection, except when there was a nest of mice in the local phone exchange/switch/box thing (whatever they're called) or a tree fell in the wrong place or it rained for weeks. Then it would be a tad slower (the fallen tree would only cause an electricity outage ;-) the phoneline often was still fine), but never go below 33.6K. Though the mice were annoying in causing a disconnect every half hour or so. Anyways, at 56K speeds 3/4 of the time and I could stream low bandwidth real audio radio and do some browsing all at the same time whilst chatting. So going from that I would say that a 33.6K connection dedicated to just VPN connectivity would work fine for things such as sql queries. Greetings, Jeroen -- http://goldmark.org/jeff/stupid-disclaimers/ http://linuxmafia.com/~rick/faq/plural-of-virus.html
On Apr 21, 2011, at 12:11 PM, Jeroen van Aart wrote:
Valdis.Kletnieks@vt.edu wrote:
Well, 33.6k is a Bad Idea right there. :) But if you're stuck with that for technical reasons, but need a VPN for security reasons, it won't be all *that* much worse, unless you're doing a lot of SSH or similar
I would think so too. When I first moved to the States I lived in rural Oregon and used an (attempted) always on dialup for years. I could mostly get a steady 56K connection, except when there was a nest of mice in the local phone exchange/switch/box thing (whatever they're called) or a tree fell in the wrong place or it rained for weeks. Then it would be a tad slower (the fallen tree would only cause an electricity outage ;-) the phoneline often was still fine), but never go below 33.6K. Though the mice were annoying in causing a disconnect every half hour or so.
Anyways, at 56K speeds 3/4 of the time and I could stream low bandwidth real audio radio and do some browsing all at the same time whilst chatting. So going from that I would say that a 33.6K connection dedicated to just VPN connectivity would work fine for things such as sql queries.
Greetings, Jeroen
-- http://goldmark.org/jeff/stupid-disclaimers/ http://linuxmafia.com/~rick/faq/plural-of-virus.html
Assuming the slow connection, I would guess there would be some packet loss to go along with it which could make not an ideal situation even worse. We've all done things we're not proud of because of business need, from the outside looking in I would definitely hammer home very low expectations before rolling it out. If they think it's going to be great they've got another thing coming, if they expect it to be barely usable they just might be pleasantly surprised. -wil
On Thu, Apr 21, 2011 at 12:55 PM, Ben Whorwood <bw-ml@mube.co.uk> wrote:
Can anyone share any thoughts or experiences for VPN links running over slow Internet connections, typically 2kB/s - 3kB/s (think 33.6k modem)?
We are looking into utilising OpenVPN for out-of-office workers who would be running mobile broadband in rural areas. Typical data across the wire would be SQL queries for custom applications and not much else.
Some initial thoughts include...
* How well would the connection handle certificate (>= 2048 bit key) based authentication?
Fine. The certificate isn't sent very often and is only 256 bytes when it is sent.
* Is UDP or TCP better considering the speed and possibility of packet loss (no figures to hand)?
TCP is more likely to pass firewalls at the user's end, especially if you put your VPN server on port 443. UDP will allow the user's various sessions to recover from packet loss independently (i.e. faster). I would pick UDP and provide an alternate TCP configuration for users who experience trouble.
* Is VPN over this type of connection simply a bad idea?
No worse than using this slow a connection in the first place. VPN overhead is 5% to 10% tops. I would use a split tunnel though; let general internet destinations go directly through the Internet connection rather than through the VPN. Regards, Bill Herrin -- William D. Herrin ................ herrin@dirtside.com bill@herrin.us 3005 Crane Dr. ...................... Web: <http://bill.herrin.us/> Falls Church, VA 22042-3004
If you haven't deployed your VPN environment yet I would seriously consider using SSL VPN instead of IPSec as your tunneling protocol. SSL VPN gives you a lot more options than IPSec. Gary -----Original Message----- From: Ben Whorwood [mailto:bw-ml@mube.co.uk] Sent: Thursday, April 21, 2011 12:56 PM To: nanog@nanog.org Subject: VPN over slow Internet connections Dear all, Can anyone share any thoughts or experiences for VPN links running over slow Internet connections, typically 2kB/s - 3kB/s (think 33.6k modem)? We are looking into utilising OpenVPN for out-of-office workers who would be running mobile broadband in rural areas. Typical data across the wire would be SQL queries for custom applications and not much else. Some initial thoughts include... * How well would the connection handle certificate (>= 2048 bit key) based authentication? * Is UDP or TCP better considering the speed and possibility of packet loss (no figures to hand)? * Is VPN over this type of connection simply a bad idea? Many thanks in advance. Kind regards, Ben Whorwood
On Thu, Apr 21, 2011 at 1:32 PM, Gary Gladney <gladney@stsci.edu> wrote:
If you haven't deployed your VPN environment yet I would seriously consider using SSL VPN instead of IPSec as your tunneling protocol. SSL VPN gives you a lot more options than IPSec.
Hi Gary, Ben was looking at OpenVPN, not IPSec.. He seems to have a field application he wants to talk directly to his SQL server. An SSL VPN that does such a good job protecting interior web services, sometimes also called a Web VPN or a clientless VPN, may not be on the table. Regards, Bill Herrin -- William D. Herrin ................ herrin@dirtside.com bill@herrin.us 3005 Crane Dr. ...................... Web: <http://bill.herrin.us/> Falls Church, VA 22042-3004
On Apr 21, 2011, at 12:55 PM, Ben Whorwood wrote:
Dear all,
Can anyone share any thoughts or experiences for VPN links running over slow Internet connections, typically 2kB/s - 3kB/s (think 33.6k modem)?
We are looking into utilising OpenVPN for out-of-office workers who would be running mobile broadband in rural areas. Typical data across the wire would be SQL queries for custom applications and not much else.
Some initial thoughts include...
* How well would the connection handle certificate (>= 2048 bit key) based authentication?
Should be fine. Might take 30 seconds to connect, but after connection it makes no difference
* Is UDP or TCP better considering the speed and possibility of packet loss (no figures to hand)?
Since you're running TCP applications (database connections), you definitely want UDP. TCP-in-UDP behaves correctly in the presence of packet loss, TCP-in-TCP behaves horribly (it causes exponential backoff on the outer VPN connection, which causes queueing of the inner packets when they should be dropped. I've seen 20-30 second latencies with TCP VPNs over slow/lossy links).
* Is VPN over this type of connection simply a bad idea?
It shouldn't be any worse than running directly over the connection. With a UDP VPN it does packet-by-packet encapsulation, so it only adds the fixed per-packet overhead.
On Apr 21, 2011, at 12:55 32PM, Ben Whorwood wrote:
Dear all,
Can anyone share any thoughts or experiences for VPN links running over slow Internet connections, typically 2kB/s - 3kB/s (think 33.6k modem)?
We are looking into utilising OpenVPN for out-of-office workers who would be running mobile broadband in rural areas. Typical data across the wire would be SQL queries for custom applications and not much else.
Some initial thoughts include...
* How well would the connection handle certificate (>= 2048 bit key) based authentication?
You're doing this rarely; it shouldn't be a problem.
* Is UDP or TCP better considering the speed and possibility of packet loss (no figures to hand)?
For your application or for the VPN? For the VPN, I *strongly* suggest you use UDP, or you're going to get dueling retransmissions and spend a lot of time sending many copies of the same thing. Consider: if a packet is dropped, either due to line noise or queuing delay for the slow link, the sending TCP will resend. If you're using TCP for OpenVPN, that session's TCP will resend. Of course, the TCP running on top of it will resend as well, so you'll get two copies of the data sent to the application's TCP, wasting precious bandwidth. If, on the other hand, OpenVPN is running UDP, it won't resend; the application's TCP will, so you'll only get one copy. I should note: IPsec, being datagram-based, will also work well. PPTP, which runs over TCP as far as I know, will suffer all of the ills I just outlined. I'm assuming that your application is using TCP. Unless the data characteristics are such that you're able to fit every query and every response into a single packet, you'll spend more effort (and probably bandwidth) doing your own retransmissions, backoffs, segementation, etc.
* Is VPN over this type of connection simply a bad idea?
If you do it correctly, a VPN is actually better: you can assign a static internal IP address to each certificate. If the modem connection drops, when you reconnect the applications will still have the same IP address, so their connections won't be interrupted. You do have to watch out for queue limits -- as Jim Gettys has reminded us, many of today's queues are far too long, so we're not getting the very beneficial effects of early drop and consequent TCP slow-down. This will require tuning your end nodes OS and/or the router at your head end. Use active queue management (e.g., RED), and consider a priority queueing scheme. Watch out for other applications -- I've had trouble with MacOS's Mail.App on slow links; it's gotten very confused and more or less forgotten about my mail folders, with the consequent need to rebuild them, reactivate my sorting rules, etc. (Note that this paragraph applies whether or not you're using a VPN; it's the effect of a slow connection, not the crypto.) The real VPN question is what the overhead is. I've never calculated it for OpenVPN; I did for IPsec some years ago (long enough ago that IPsec was still using DES or 3DES because AES and its 16-byte blocksize didn't exist); using average packet size distribution, it came to (as I recall) about 12%. That's unlikely to make or break your system. However, there's no substitute for real data -- what do your packets look like? Fairly obviously, the shorter the packet, the higher the overhead percentage. Someone suggested trying it using a FreeBSD flakeway; that's a good idea. In short -- if your VPN is set up properly, any ill effects are much more likely to be from the link speed itself, not the VPN. --Steve Bellovin, https://www.cs.columbia.edu/~smb
Steven Bellovin (smb) writes:
I should note: IPsec, being datagram-based, will also work well. PPTP, which runs over TCP as far as I know, will suffer all of the ills I just outlined.
PPTP uses 1723/tcp for control, but the tunneled traffic is GRE, so that would work fine as well.
If you do it correctly, a VPN is actually better: you can assign a static internal IP address to each certificate. If the modem connection drops, when you reconnect the applications will still have the same IP address, so their connections won't be interrupted.
Absolutely, that's the case with OpenVPN, if you assign static IPs to each profile. PPtP can do this as well, for instance using MPD. Very big advantage in fact.
Someone suggested trying it using a FreeBSD flakeway; that's a good idea.
Using a dummynet box as a router (or bridge for that matter), you have the benefit that you can run tcpdump on the trafic, and record the packet sizes with and and without VPN, then derive the actual observed overhead. Cheers, Phil
On Apr 21, 2011, at 4:31 32PM, Phil Regnauld wrote:
Steven Bellovin (smb) writes:
I should note: IPsec, being datagram-based, will also work well. PPTP, which runs over TCP as far as I know, will suffer all of the ills I just outlined.
PPTP uses 1723/tcp for control, but the tunneled traffic is GRE, so that would work fine as well.
Ah, thanks for the correction.
If you do it correctly, a VPN is actually better: you can assign a static internal IP address to each certificate. If the modem connection drops, when you reconnect the applications will still have the same IP address, so their connections won't be interrupted.
Absolutely, that's the case with OpenVPN, if you assign static IPs to each profile. PPtP can do this as well, for instance using MPD. Very big advantage in fact.
Yup, I've done this myself with OpenVPN. --Steve Bellovin, https://www.cs.columbia.edu/~smb
On Apr 21, 2011, at 4:20PM, Steven Bellovin wrote:
For your application or for the VPN? For the VPN, I *strongly* suggest you use UDP, or you're going to get dueling retransmissions and spend a lot of time sending many copies of the same thing. Consider: if a packet is dropped, either due to line noise or queuing delay for the slow link, the sending TCP will resend. If you're using TCP for OpenVPN, that session's TCP will resend. Of course, the TCP running on top of it will resend as well, so you'll get two copies of the data sent to the application's TCP, wasting precious bandwidth.
Is this actually how OpenVPN's TCP encapsulation works? I'd be curious to know. It isn't how Cisco's TCP/10000 encapsulation works, at least not with the IOS devices I have experience with. Cisco's TCP/10000 looks like TCP to a firewall, but it really isn't. There is no reliability -- no retransmits, etc. It's pretty close to UDP behavior but with a TCP header, which was confusing to troubleshoot at first but quickly made perfect sense to me for the reasons you state above. -Terry
On Apr 21, 2011, at 5:28 46PM, Terry Baranski wrote:
On Apr 21, 2011, at 4:20PM, Steven Bellovin wrote:
For your application or for the VPN? For the VPN, I *strongly* suggest you use UDP, or you're going to get dueling retransmissions and spend a lot of time sending many copies of the same thing. Consider: if a packet is dropped, either due to line noise or queuing delay for the slow link, the sending TCP will resend. If you're using TCP for OpenVPN, that session's TCP will resend. Of course, the TCP running on top of it will resend as well, so you'll get two copies of the data sent to the application's TCP, wasting precious bandwidth.
Is this actually how OpenVPN's TCP encapsulation works? I'd be curious to know. It isn't how Cisco's TCP/10000 encapsulation works, at least not with the IOS devices I have experience with.
Cisco's TCP/10000 looks like TCP to a firewall, but it really isn't. There is no reliability -- no retransmits, etc. It's pretty close to UDP behavior but with a TCP header, which was confusing to troubleshoot at first but quickly made perfect sense to me for the reasons you state above.
To the OS, OpenVPN is an application that uses the underlying TCP (or UDP)/IP stack; it can't behave any differently than any other application. Since (as far as I know) Windows, Linux, NeBSD, FreeBSD, MacOS, and all of the other platforms that OpenVPN runs on just have normal TCPs, that's what OpenVPN does. --Steve Bellovin, https://www.cs.columbia.edu/~smb
On Thu, 21 Apr 2011 17:55:32 +0100, Ben Whorwood wrote: IMHO it is not good idea to go to OpenVPN/IPSec/etc level at all (IP layer at least, and in case of Windows it is also ethernet headers). First of all OpenVPN for Windows/different OS sometimes become a headache and need admin privileges. If you have just SQL queries and one TCP port to database - just use stunnel. Maybe even easier, if it is custom application, to embed TLS directly to application, and you can still run nice certificate authorization (same TLS), and it will be much more transparent for user.
-------- Original Message -------- Subject: VPN over slow Internet connections From: Ben Whorwood <bw-ml@mube.co.uk> To: nanog@nanog.org Date: Thursday, April 21, 2011 11:55:32 AM
Dear all,
Can anyone share any thoughts or experiences for VPN links running over slow Internet connections, typically 2kB/s - 3kB/s (think 33.6k modem)?
We are looking into utilising OpenVPN for out-of-office workers who would be running mobile broadband in rural areas. Typical data across the wire would be SQL queries for custom applications and not much else.
Some initial thoughts include...
* How well would the connection handle certificate (>= 2048 bit key) based authentication? * Is UDP or TCP better considering the speed and possibility of packet loss (no figures to hand)? * Is VPN over this type of connection simply a bad idea?
Many thanks in advance.
Kind regards, Ben Whorwood
I'm not sure what type of SQL you're using, but MySQL and MS SQL both natively support (and can optionally require) SSL'd connections from clients. --Blake
participants (17)
-
Ben Jencks -
Ben Whorwood -
Blake Hudson -
Brandon Kim -
Darden, Patrick S. -
Denys Fedoryshchenko -
Fred Richards -
Gary Gladney -
JC Dill -
Jeroen van Aart -
Matt Ryanczak -
Phil Regnauld -
Steven Bellovin -
Terry Baranski -
Valdis.Kletnieks＠vt.edu -
Wil Schultz -
William Herrin