Hi folks, Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses. Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement. Thanks in advance, Bill Herrin -- William Herrin bill@herrin.us <https://bill.herrin.us/> https://bill.herrin.us/
You may try WireGuard and use ddns From: NANOG <nanog-bounces+david=xtom.com@nanog.org> On Behalf Of William Herrin Sent: Friday, February 11, 2022 2:02 AM To: nanog@nanog.org Subject: VPN recommendations? Hi folks, Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses. Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement. Thanks in advance, Bill Herrin -- William Herrin bill@herrin.us<mailto:bill@herrin.us> <https://bill.herrin.us/> https://bill.herrin.us/
How about running ZeroTier on those Linux boxes and call it a day? https://www.zerotier.com/ -Mike
On Feb 10, 2022, at 10:07, David Guo via NANOG <nanog@nanog.org> wrote:
You may try WireGuard and use ddns
From: NANOG <nanog-bounces+david=xtom.com@nanog.org> On Behalf Of William Herrin Sent: Friday, February 11, 2022 2:02 AM To: nanog@nanog.org Subject: VPN recommendations?
Hi folks,
Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses.
Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement.
Thanks in advance, Bill Herrin
-- William Herrin bill@herrin.us https://bill.herrin.us/
Hello NANOG, My name is Joy Larkin and I'm actually a long-time years-long lurker on the NANOG list (I have v odd hobbies) and I am also ZeroTier's Head of Marketing. I know I'm not supposed to be too promotional on here, but I'd love to see some of you pick up ZT. Our founder, Adam Ierymenko just did a talk at Networking Field Day 27, here are two of the recordings from that session: * ZeroTier The Planetary Data Center * https://www.youtube.com/watch?v=T2BbrqpnMAE * ZeroTier Technical Deep Dive * https://www.youtube.com/watch?v=VhQ30bVF3_s If you have questions, let me know - you can reach me at joy.larkin@zerotier.com Best, -Joy On 2022-02-10 10:12, Mike Lyon wrote:
How about running ZeroTier on those Linux boxes and call it a day?
-Mike
On Feb 10, 2022, at 10:07, David Guo via NANOG <nanog@nanog.org> wrote:
You may try WireGuard and use ddns
From: NANOG <nanog-bounces+david=xtom.com@nanog.org> On Behalf Of William Herrin Sent: Friday, February 11, 2022 2:02 AM To: nanog@nanog.org Subject: VPN recommendations?
Hi folks,
Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses.
Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement.
Thanks in advance,
Bill Herrin
--
William Herrin
bill@herrin.us
Thank you Joy for de-lurking. I actually was not familiar with ZeroTier, and this is a space that I thought I was quite familiar with, so I’m glad you brought it to everyone’s attention. I will look further at ZeroTier, it looks very interesting. I am also a very long-time lurker (although I was a NANOG list admin ~10 years ago) who is emerging to join this conversation. I have recently been doing some work to evaluate and develop VPN solutions for connecting multiple data center cloud environments, including low-power small edge sites, and I have some thoughts about the current state of the art to share. Until recently a very strong proponent of IPSEC. I liked the way IPSEC was placed within the OSI model directly at layer 3, unlike some of the VPN technologies which operate above or below layer 3. However I do not believe that IPSEC is future-proof, for the following two reasons: 1) IPSEC does not lend itself to dynamic routing or dynamic configuration. It is very much a static set-it-and-forget-it technology, but that doesn’t work in a dynamically changing environment. 2) IPSEC does not always lend itself to hardware offloading in the way some other technologies do. Some NICs do support hardware acceleration for IPSEC, but this does not always integrate well with kernel or user space when you are integrating virtual network functions (VNFs) like routers/firewalls/load-balancers. Wireguard works well in dynamic environments. TLS using something like OpenSSL does as well. Both provide key advantages, particularly on top of Linux. * Support for hardware offloads such as TCP segmentation provide vast improvements in performance on higher-end x86 hardware. Some recent testing I have been shown proves that TCP segmentation offload can provide more than a 5X speedup compared to other HW offloads without TCP segmentation (from 5Gb/s to above 25Gb/s in some tests). * With the right encryption algorithm CPU acceleration for cryptography reduces CPU load and increases performance. * Integration with kernel routing provides the ability to integrate with dynamic routing such as BGP daemons (e.g. FRRouting, etc.). * In recent Linux kernels eBPF/XDP provide a hardware interface to the kernel which accelerates network throughput to near line-rate, while minimizing CPU impact. This may not apply to William Herrin’s (OP) use case of a VPN appliance for 100mbps to 1gbps speeds, but it is something to keep in mind for building higher performance solutions or for planning for increasing bandwidth in the future. For the 100mbps+ use case I have had success building appliances using OpenVPN on top of certain ARM based platforms like Marvell Armada, or single-board computers with Intel CPUs with AES-NI acceleration. I am currently looking at implementing Wireguard on the same platforms. For a simple low-power ARM router appliance the Turris Omnia has been a great fully open platform running a custom LEDE/OpenWRT OS. The Turris Mox provides a modular hardware platform for expandability, albeit with slightly less performance. Both of these platforms are developed by the engineers at CZ.nic, the TLD registrar for the Czech Republic. https://secure.nic.cz/files/Turris-web/Omnia/Omnia2020_datasheet.pdf https://www.turris.com/en/mox/overview/ -Dan Sneddon
On Feb 10, 2022, at 10:51 AM, joy@cleverhack.com wrote:
Hello NANOG,
My name is Joy Larkin and I'm actually a long-time years-long lurker on the NANOG list (I have v odd hobbies) and I am also ZeroTier's Head of Marketing. I know I'm not supposed to be too promotional on here, but I'd love to see some of you pick up ZT.
Our founder, Adam Ierymenko just did a talk at Networking Field Day 27, here are two of the recordings from that session:
* ZeroTier The Planetary Data Center * https://www.youtube.com/watch?v=T2BbrqpnMAE
* ZeroTier Technical Deep Dive * https://www.youtube.com/watch?v=VhQ30bVF3_s
If you have questions, let me know - you can reach me at joy.larkin@zerotier.com
Best, -Joy
On 2022-02-10 10:12, Mike Lyon wrote: How about running ZeroTier on those Linux boxes and call it a day? https://www.zerotier.com/ -Mike
On Feb 10, 2022, at 10:07, David Guo via NANOG <nanog@nanog.org> wrote: You may try WireGuard and use ddns From: NANOG <nanog-bounces+david=xtom.com@nanog.org> On Behalf Of William Herrin Sent: Friday, February 11, 2022 2:02 AM To: nanog@nanog.org Subject: VPN recommendations? Hi folks, Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses. Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement. Thanks in advance, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/
Dan, One point you didn’t touch on is that IPSec is integrated into IPv6, typically hardware-accelerated on the NIC, enabling device-to-device VPNs, mitigates most of the dynamic issues associated with network-to-network IPSec over IPv4. Yes, I realize IPv4 is hanging around longer than most expect, but in some cases I think you can make a case for deploying IPv6 just on the VPN benefits alone. With no public-facing services, IPv6 is already deployed in most LANs as a direct result of its use by modern OSes for inter-LAN communication. All you typically need to do is enable IPv6 at the gateway. -mel On Feb 11, 2022, at 10:33 AM, Dan Sneddon <sneddon@gmail.com<mailto:sneddon@gmail.com>> wrote: Thank you Joy for de-lurking. I actually was not familiar with ZeroTier, and this is a space that I thought I was quite familiar with, so I’m glad you brought it to everyone’s attention. I will look further at ZeroTier, it looks very interesting. I am also a very long-time lurker (although I was a NANOG list admin ~10 years ago) who is emerging to join this conversation. I have recently been doing some work to evaluate and develop VPN solutions for connecting multiple data center cloud environments, including low-power small edge sites, and I have some thoughts about the current state of the art to share. Until recently a very strong proponent of IPSEC. I liked the way IPSEC was placed within the OSI model directly at layer 3, unlike some of the VPN technologies which operate above or below layer 3. However I do not believe that IPSEC is future-proof, for the following two reasons: 1) IPSEC does not lend itself to dynamic routing or dynamic configuration. It is very much a static set-it-and-forget-it technology, but that doesn’t work in a dynamically changing environment. 2) IPSEC does not always lend itself to hardware offloading in the way some other technologies do. Some NICs do support hardware acceleration for IPSEC, but this does not always integrate well with kernel or user space when you are integrating virtual network functions (VNFs) like routers/firewalls/load-balancers. Wireguard works well in dynamic environments. TLS using something like OpenSSL does as well. Both provide key advantages, particularly on top of Linux. * Support for hardware offloads such as TCP segmentation provide vast improvements in performance on higher-end x86 hardware. Some recent testing I have been shown proves that TCP segmentation offload can provide more than a 5X speedup compared to other HW offloads without TCP segmentation (from 5Gb/s to above 25Gb/s in some tests). * With the right encryption algorithm CPU acceleration for cryptography reduces CPU load and increases performance. * Integration with kernel routing provides the ability to integrate with dynamic routing such as BGP daemons (e.g. FRRouting, etc.). * In recent Linux kernels eBPF/XDP provide a hardware interface to the kernel which accelerates network throughput to near line-rate, while minimizing CPU impact. This may not apply to William Herrin’s (OP) use case of a VPN appliance for 100mbps to 1gbps speeds, but it is something to keep in mind for building higher performance solutions or for planning for increasing bandwidth in the future. For the 100mbps+ use case I have had success building appliances using OpenVPN on top of certain ARM based platforms like Marvell Armada, or single-board computers with Intel CPUs with AES-NI acceleration. I am currently looking at implementing Wireguard on the same platforms. For a simple low-power ARM router appliance the Turris Omnia has been a great fully open platform running a custom LEDE/OpenWRT OS. The Turris Mox provides a modular hardware platform for expandability, albeit with slightly less performance. Both of these platforms are developed by the engineers at CZ.nic, the TLD registrar for the Czech Republic. https://secure.nic.cz/files/Turris-web/Omnia/Omnia2020_datasheet.pdf https://www.turris.com/en/mox/overview/ -Dan Sneddon On Feb 10, 2022, at 10:51 AM, joy@cleverhack.com<mailto:joy@cleverhack.com> wrote: Hello NANOG, My name is Joy Larkin and I'm actually a long-time years-long lurker on the NANOG list (I have v odd hobbies) and I am also ZeroTier's Head of Marketing. I know I'm not supposed to be too promotional on here, but I'd love to see some of you pick up ZT. Our founder, Adam Ierymenko just did a talk at Networking Field Day 27, here are two of the recordings from that session: * ZeroTier The Planetary Data Center * https://www.youtube.com/watch?v=T2BbrqpnMAE * ZeroTier Technical Deep Dive * https://www.youtube.com/watch?v=VhQ30bVF3_s If you have questions, let me know - you can reach me at joy.larkin@zerotier.com<mailto:joy.larkin@zerotier.com> Best, -Joy On 2022-02-10 10:12, Mike Lyon wrote: How about running ZeroTier on those Linux boxes and call it a day? https://www.zerotier.com/ -Mike On Feb 10, 2022, at 10:07, David Guo via NANOG <nanog@nanog.org<mailto:nanog@nanog.org>> wrote: You may try WireGuard and use ddns From: NANOG <nanog-bounces+david=xtom.com@nanog.org<mailto:nanog-bounces+david=xtom.com@nanog.org>> On Behalf Of William Herrin Sent: Friday, February 11, 2022 2:02 AM To: nanog@nanog.org<mailto:nanog@nanog.org> Subject: VPN recommendations? Hi folks, Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses. Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement. Thanks in advance, Bill Herrin -- William Herrin bill@herrin.us<mailto:bill@herrin.us> https://bill.herrin.us/
On Fri, Feb 11, 2022 at 10:35 AM Dan Sneddon <sneddon@gmail.com> wrote:
1) IPSEC does not lend itself to dynamic routing or dynamic configuration. It is very much a static set-it-and-forget-it technology, but that doesn’t work in a dynamically changing environment.
Hi Dan, Depending on how you configure it, IPSEC can work fine with dynamic routing. The thing to understand is that IPSec has two modes: transport and tunnel. Transport is between exactly two IP addresses while tunnel expects a broader network to exist on at least one end. "Tunnel" mode is what everyone actually uses but you can deconstruct it: it's built up from transport mode + a tunnel protocol (gre or ipip I don't remember which) + implicit routing and firewalling which wreaks havoc on dynamic routing. Now, it turns out that you can instead configure IPSec in transport mode, configure the tunnel separately and leave out the implicit firewalling.
This may not apply to William Herrin’s (OP) use case of a VPN appliance
It's not relevant to my situation, no. I need the VPN to establish a statically addressed clean layer 3 on top of dynamically addressed and natted endpoints to support the next appliance in the chain where dynamic addressing is not possible. I don't actually care if it adds security; it just needs to establish that statically addressed layer. Oh yeah, and it has to be listed under "virtual private network" on the government NIAP list. https://www.niap-ccevs.org/product/PCL.cfm?ID624=34 Regards, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/
Intriguing. This week I started to look around for new wireguard implementation tools and appliances. I've used openvpn and ipsec in the main although last month put together a 10x and IPv6 wireguard net in my home and out to two vps hosts which is handy. For my own use this is ok -ish, but I am not so sure about keeping track of the configs, managing users and adding configs as a network grows. In other words I want help when scaling wg and handling change particularly if I am managing nets for other projects or delegating. Tailscale, ZeroTier and some others are doing a great job I feel and no doubt have a handle on that. I've not tried them as yet. Because I do like to have options that are not mediated I have kept looking as much for my own curiousity and education as for deploying a service in anger. But having a toolset that can support the latter capability has to be the aim to work towards. I've found a few potentially interesting more recent projects and am intending to start to test deploy some of these in sequence to see how I get on. I think I'll start wth https://github.com/gravitl/netmaker Please note I've only reviewed the documentation. I've not yet played with it. This seems to offer at an early stage in its development a webappliance (optionally) with CoreDNS if you want naming support and IPv6 and at least some client management features. It claims to be fast but that can be tested. It also is deployable as a docker/kubernetes k8 which is intriguing when deploying and managing containers between multiple hosts across data centres. It uses a mongodb licence which may or may not be a problem. If one plays with IPSEC then I guess one could run wg through IPSEC but is there any point unless you already have an IPSEC branch and don't want to take it down whilst adding wg for a new class of devices/userbase? I'd be interested in sharing experiences and advice (offlist) and delighted to learn from wireguard and vpn's clueful folk. thank you for an interesting discussion. Christian William Herrin <bill@herrin.us> writes:
On Fri, Feb 11, 2022 at 10:35 AM Dan Sneddon <sneddon@gmail.com> wrote:
1) IPSEC does not lend itself to dynamic routing or dynamic configuration. It is very much a static set-it-and-forget-it technology, but that doesn’t work in a dynamically changing environment.
Hi Dan,
Depending on how you configure it, IPSEC can work fine with dynamic routing. The thing to understand is that IPSec has two modes: transport and tunnel. Transport is between exactly two IP addresses while tunnel expects a broader network to exist on at least one end. "Tunnel" mode is what everyone actually uses but you can deconstruct it: it's built up from transport mode + a tunnel protocol (gre or ipip I don't remember which) + implicit routing and firewalling which wreaks havoc on dynamic routing. Now, it turns out that you can instead configure IPSec in transport mode, configure the tunnel separately and leave out the implicit firewalling.
This may not apply to William Herrin’s (OP) use case of a VPN appliance
It's not relevant to my situation, no. I need the VPN to establish a statically addressed clean layer 3 on top of dynamically addressed and natted endpoints to support the next appliance in the chain where dynamic addressing is not possible. I don't actually care if it adds security; it just needs to establish that statically addressed layer. Oh yeah, and it has to be listed under "virtual private network" on the government NIAP list. https://www.niap-ccevs.org/product/PCL.cfm?ID624=34
Regards, Bill Herrin
-- Christian de Larrinaga https://firsthand.net
On 2/11/22 12:35 PM, William Herrin wrote:
The thing to understand is that IPSec has two modes: transport and tunnel. Transport is between exactly two IP addresses while tunnel expects a broader network to exist on at least one end.
That is (syntactically) correct. However, it is possible to NAT many LAN IPs (say RFC 1918) to one single Internet IP (say from a SOHO ISP) and use IPSec /Transport/ Mode to a single remote IP. The IPSec sees exactly two IPs.
"Tunnel" mode is what everyone actually uses
I may be enough of an outlier that I'm a statistical anomaly. But I'm using IPSec /Transport/ Mode between my home router and my VPSs. I have a tiny full mesh of IPSec /Transport/ Mode connections. Using the aforementioned many-to-one NAT, my home LAN systems access the single globally routed IP of each of my VPSs without any problem. Aside: I did have to tweak MTU for LAN traffic going out to the VPS IPs. So -1 for '"Tunnel" mode is what everyone actually uses', and +1 for /Transport/ Mode
but you can deconstruct it: it's built up from transport mode + a tunnel protocol (gre or ipip I don't remember which) + implicit routing and firewalling which wreaks havoc on dynamic routing.
I question the veracity of that statement. It may be that's what many implementations / administration systems do. But I really thought that IPSec /Tunnel/ Mode was more than just IPSec /Transport/ Mode combined with some tunneling protocol.
Now, it turns out that you can instead configure IPSec in transport mode, configure the tunnel separately and leave out the implicit firewalling.
Agreed. I feel like this speaks to implementation / management systems that are built on top of IPSec.
It's not relevant to my situation, no. I need the VPN to establish a statically addressed clean layer 3 on top of dynamically addressed and natted endpoints to support the next appliance in the chain where dynamic addressing is not possible. I don't actually care if it adds security; it just needs to establish that statically addressed layer.
It sounds to me like you don't even actually need encryption of a typical VPN and might be able to use something like GRE+key or IPSec /Tunnel/ Mode with AH without ESP.
Oh yeah, and it has to be listed under "virtual private network" on the government NIAP list. https://www.niap-ccevs.org/product/PCL.cfm?ID624=34
Oh joy. Layer 8 - politics -- Grant. . . . unix || die
On Sat, 2022-02-12 at 13:24 -0700, Grant Taylor via NANOG wrote:
On 2/11/22 12:35 PM, William Herrin wrote:
The thing to understand is that IPSec has two modes: transport and tunnel. Transport is between exactly two IP addresses while tunnel expects a broader network to exist on at least one end.
That is (syntactically) correct. However, it is possible to NAT many LAN IPs (say RFC 1918) to one single Internet IP (say from a SOHO ISP) and use IPSec /Transport/ Mode to a single remote IP. The IPSec sees exactly two IPs.
"Tunnel" mode is what everyone actually uses
I may be enough of an outlier that I'm a statistical anomaly. But I'm using IPSec /Transport/ Mode between my home router and my VPSs. I have a tiny full mesh of IPSec /Transport/ Mode connections.
+1 on *cough* enterprise networks.
Using the aforementioned many-to-one NAT, my home LAN systems access the single globally routed IP of each of my VPSs without any problem.
+1
Aside: I did have to tweak MTU for LAN traffic going out to the VPS IPs.
+1
So -1 for '"Tunnel" mode is what everyone actually uses', and +1 for /Transport/ Mode
+1
On Sat, Feb 12, 2022 at 12:26 PM Grant Taylor via NANOG <nanog@nanog.org> wrote:
On 2/11/22 12:35 PM, William Herrin wrote:
The thing to understand is that IPSec has two modes: transport and but you can deconstruct it: it's built up from transport mode + a tunnel protocol (gre or ipip I don't remember which) + implicit routing and firewalling which wreaks havoc on dynamic routing.
I question the veracity of that statement. It may be that's what many implementations / administration systems do. But I really thought that IPSec /Tunnel/ Mode was more than just IPSec /Transport/ Mode combined with some tunneling protocol.
It's tunnel mode plus a tunneling protocol plus some implicit routing and firewalling which gets in the way of dynamic routing. Try it if you don't believe me. Set up tunnel mode ipsec manually on two nodes (no IKE) and get them talking to each other. Then change one to transport mode and add I think it's an IPIP tunnel but I don't remember for certain. And add the appropriate routes into the tunnel virtual device. You'll find they talk. What did you think IPSec was doing? Transport mode encrypts the layer 4 and up of the packet between two machines; it doesn't encapsulate it. When they added tunnel mode, the inner layer 3 had to go somewhere. Regards, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/
Hi Bill, On 2/12/22 8:55 PM, William Herrin wrote:
It's tunnel mode plus a tunneling protocol plus some implicit routing and firewalling which gets in the way of dynamic routing.
I assume you meant to say that it's /transport/ mode plus a tunneling protocol. I wonder if you are thinking more of an IPSec VPN management suite of sorts, e.g. wizard / helper that is included in some devices. I'm thinking at a very low (manual) level. The "implicit routing" and "firewalling" are the strongest indicators of this to me. The manual IPSec that I've done on Linux (via the `ip xfrm` command) doesn't touch firewalling and I believe that addresses inside the tunnel would be completely separate operations / commands.
Try it if you don't believe me. Set up tunnel mode ipsec manually on two nodes (no IKE) and get them talking to each other. Then change one to transport mode and add I think it's an IPIP tunnel but I don't remember for certain. And add the appropriate routes into the tunnel virtual device. You'll find they talk.
Unfortunately I don't have the leisure time to do this experimentation currently. As such I'm going to put this on my to-do pile for future investigation ~> follow up. I do not recall reading about IPSec /Tunnel/ mode re-using an existing tunneling protocol; IPIP, etc. Perhaps I'm misremembering. Perhaps it inherently does so without declaring as such.
What did you think IPSec was doing? Transport mode encrypts the layer 4 and up of the packet between two machines; it doesn't encapsulate it. When they added tunnel mode, the inner layer 3 had to go somewhere.
My understanding is that /Transport/ mode applies AH (no encryption) and / or ESP (encryption) to L4 datagrams and that /Tunnel/ mode does the same to L3 packets. P.S. I'm sending this reply to NANOG in case anyone else has any contribution / comments. I suspect any future reply will be directly to Bill as this is getting further off topic, both for NANOG in general and for this VPN recommendations thread. -- Grant. . . . unix || die
It's not like IPsec protocols (it's a suite of protocols and concepts, not one) are proprietary or something. There are pretty ASCII pictures in RFCs with all about how the packets are put together. See section 3 of RFC 4303 to see how ESP transport and tunnel mode datagrams are put together. For the tl;dr, in transport mode everything above IP header is the payload. In tunnel mode, the whole IP datagram is the payload. The contents of the payload are specified by the "Next Header" field of the ESP header. For an encapsulated IPv4 packet, it would be protocol 4 (IP-in-IP). For an IPv6 packet, it would be 41. For TCP in transport mode, it would be 6. UDP is 17. Etc. If you want to see it in action yourself, you can set the encryption algo to NULL and do a capture. On Tue, Feb 15, 2022 at 10:16 AM Grant Taylor via NANOG <nanog@nanog.org> wrote:
Hi Bill,
On 2/12/22 8:55 PM, William Herrin wrote:
It's tunnel mode plus a tunneling protocol plus some implicit routing and firewalling which gets in the way of dynamic routing.
I assume you meant to say that it's /transport/ mode plus a tunneling protocol.
I wonder if you are thinking more of an IPSec VPN management suite of sorts, e.g. wizard / helper that is included in some devices. I'm thinking at a very low (manual) level. The "implicit routing" and "firewalling" are the strongest indicators of this to me. The manual IPSec that I've done on Linux (via the `ip xfrm` command) doesn't touch firewalling and I believe that addresses inside the tunnel would be completely separate operations / commands.
Try it if you don't believe me. Set up tunnel mode ipsec manually on two nodes (no IKE) and get them talking to each other. Then change one to transport mode and add I think it's an IPIP tunnel but I don't remember for certain. And add the appropriate routes into the tunnel virtual device. You'll find they talk.
Unfortunately I don't have the leisure time to do this experimentation currently. As such I'm going to put this on my to-do pile for future investigation ~> follow up.
I do not recall reading about IPSec /Tunnel/ mode re-using an existing tunneling protocol; IPIP, etc. Perhaps I'm misremembering. Perhaps it inherently does so without declaring as such.
What did you think IPSec was doing? Transport mode encrypts the layer 4 and up of the packet between two machines; it doesn't encapsulate it. When they added tunnel mode, the inner layer 3 had to go somewhere.
My understanding is that /Transport/ mode applies AH (no encryption) and / or ESP (encryption) to L4 datagrams and that /Tunnel/ mode does the same to L3 packets.
P.S. I'm sending this reply to NANOG in case anyone else has any contribution / comments. I suspect any future reply will be directly to Bill as this is getting further off topic, both for NANOG in general and for this VPN recommendations thread.
-- Grant. . . . unix || die
Mike Lyon <mike.lyon@gmail.com> wrote:
How about running ZeroTier on those Linux boxes and call it a day? https://www.zerotier.com/
ZeroTier is not a free-as-in-freedom project. Running it in Linux boxes or network appliances to provide a VPN to paying customers may be prohibited (at least for some customers, and before 2025) by its convoluted license: https://github.com/zerotier/ZeroTierOne/blob/master/LICENSE.txt I recommend using something that doesn't have litigious companies nitpicking about what you can and can't use it for. John Gilmore
tailscale is 3-clause BSD. there is a reverse engineered version of the rendezvous protocol also. On Thu, Feb 10, 2022 at 3:41 PM John Gilmore <gnu@toad.com> wrote:
Mike Lyon <mike.lyon@gmail.com> wrote:
How about running ZeroTier on those Linux boxes and call it a day? https://www.zerotier.com/
ZeroTier is not a free-as-in-freedom project. Running it in Linux boxes or network appliances to provide a VPN to paying customers may be prohibited (at least for some customers, and before 2025) by its convoluted license:
https://github.com/zerotier/ZeroTierOne/blob/master/LICENSE.txt
I recommend using something that doesn't have litigious companies nitpicking about what you can and can't use it for.
John Gilmore
-- I tried to build a better future, a few times: https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org Dave Täht CEO, TekLibre, LLC
I work in a large oil company and we have S2S VPNs every where. Any modern Cisco or Juniper router will meet your requirements. An off the shelf security appliance will do the job to i.e ASA, Palo Alto, Fortinet or Juniper. Meraki is great if you want to manage from the cloud or vpn as a service. Good luck. Sean P Kelly
On Feb 10, 2022, at 6:51 PM, Dave Taht <dave.taht@gmail.com> wrote:
tailscale is 3-clause BSD.
there is a reverse engineered version of the rendezvous protocol also.
On Thu, Feb 10, 2022 at 3:41 PM John Gilmore <gnu@toad.com> wrote:
Mike Lyon <mike.lyon@gmail.com> wrote:
How about running ZeroTier on those Linux boxes and call it a day? https://www.zerotier.com/
ZeroTier is not a free-as-in-freedom project. Running it in Linux boxes or network appliances to provide a VPN to paying customers may be prohibited (at least for some customers, and before 2025) by its convoluted license:
https://github.com/zerotier/ZeroTierOne/blob/master/LICENSE.txt
I recommend using something that doesn't have litigious companies nitpicking about what you can and can't use it for.
John Gilmore
-- I tried to build a better future, a few times: https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
Dave Täht CEO, TekLibre, LLC
On Thu, Feb 10, 2022 at 10:04 AM David Guo <david@xtom.com> wrote:
You may try WireGuard and use ddns
Hi David, My understanding is that Wireguard is software available for general purpose operating systems. I specifically need a set of hardware network appliances. I don't overly care which protocol they're running as long as an initiator stuck behind a nat box I don't control can maintain a connection with a hub and handle speeds in the100mbps to 10gbps. On Thu, Feb 10, 2022 at 10:12 AM Mike Lyon <mike.lyon@gmail.com> wrote:
How about running ZeroTier on those Linux boxes and call it a day? https://www.zerotier.com/
I specifically cannot use general purpose Linux machines for this. I need network appliances. On Thu, Feb 10, 2022 at 10:26 AM Dave Taht <dave.taht@gmail.com> wrote:
tailscale
I specifically need an integrated network appliance, not software I add to something. I love my Linux-based VPN servers but my customer very specifically said no. I can't publicly explain why but trust me when I say it's a "hard no" and it's not a question of persuasion or education. My customer understands and likes Linux but he simply cannot use it this time. Regards, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/
On Thu, Feb 10, 2022 at 10:55 AM William Herrin <bill@herrin.us> wrote:
My understanding is that Wireguard is software available for general purpose operating systems. I specifically need a set of hardware network appliances. I don't overly care which protocol they're running as long as an initiator stuck behind a nat box I don't control can maintain a connection with a hub and handle speeds in the100mbps to 10gbps.
That was supposed to be 1gbps. I don't need over 1gbps for this use case. Regards, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/
On Thu, 10 Feb 2022 10:55:40 -0800, William Herrin wrote:
My understanding is that Wireguard is software available for general purpose operating systems. I specifically need a set of hardware network appliances.
MikroTik (hardware) RouterOS (software) version 7 has WireGuard: https://help.mikrotik.com/docs/display/ROS/WireGuard
Mikrotik with RouterOS v7 with WireGuard or ZeroTier were the first things I thought of, but it might be a a bit premature for a production environment. In a year, I'd have no problem recommending that. ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP ----- Original Message ----- From: "Ander Punnar" <ander@kvlt.ee> Cc: nanog@nanog.org Sent: Thursday, February 10, 2022 2:04:57 PM Subject: Re: VPN recommendations? On Thu, 10 Feb 2022 10:55:40 -0800, William Herrin wrote:
My understanding is that Wireguard is software available for general purpose operating systems. I specifically need a set of hardware network appliances.
MikroTik (hardware) RouterOS (software) version 7 has WireGuard: https://help.mikrotik.com/docs/display/ROS/WireGuard
On Thu, 10 Feb 2022 10:55:40 -0800, William Herrin said:
My understanding is that Wireguard is software available for general purpose operating systems. I specifically need a set of hardware network appliances.
Take a general purpose OS, strip down the userspace a bit, stick the whole thing in a box, and call it an appliance. They'll never know the difference. :)
I don't know how people around here feel about Mikrotik, but they have included Wireguard support in their latest operating system. dave -----Original Message----- From: NANOG <nanog-bounces+david=davidandrzejewski.com@nanog.org> On Behalf Of William Herrin Sent: Thursday, February 10, 2022 13:56 Cc: nanog@nanog.org Subject: Re: VPN recommendations? On Thu, Feb 10, 2022 at 10:04 AM David Guo <david@xtom.com> wrote:
You may try WireGuard and use ddns
Hi David, My understanding is that Wireguard is software available for general purpose operating systems. I specifically need a set of hardware network appliances. I don't overly care which protocol they're running as long as an initiator stuck behind a nat box I don't control can maintain a connection with a hub and handle speeds in the100mbps to 10gbps. On Thu, Feb 10, 2022 at 10:12 AM Mike Lyon <mike.lyon@gmail.com> wrote:
How about running ZeroTier on those Linux boxes and call it a day? https://www.zerotier.com/
I specifically cannot use general purpose Linux machines for this. I need network appliances. On Thu, Feb 10, 2022 at 10:26 AM Dave Taht <dave.taht@gmail.com> wrote:
tailscale
I specifically need an integrated network appliance, not software I add to something. I love my Linux-based VPN servers but my customer very specifically said no. I can't publicly explain why but trust me when I say it's a "hard no" and it's not a question of persuasion or education. My customer understands and likes Linux but he simply cannot use it this time. Regards, Bill Herrin -- William Herrin bill@herrin.us https://bill.herrin.us/
On Thu, Feb 10, 2022 at 8:51 PM David Andrzejewski <david@davidandrzejewski.com> wrote:
I don't know how people around here feel about Mikrotik, but they have included Wireguard support in their latest operating system.
They've also included fq_codel and sch_cake: https://forum.mikrotik.com/viewtopic.php?t=179307 For a site to site, kernel mode vpn such as ipsec or wireguard (but not openvpn), we successfully FQ+AQM packets entering the tunnel. If that's the bottleneck link, for a mixture of, say low rate voip and high rate file transfer traffic, the results are a pretty marvellous reduction of jitter and latency through the tunnel. Before: http://www.taht.net/~d/ipsec_fq_codel/oldqos.png After: http://www.taht.net/~d/ipsec_fq_codel/newqos.png
dave
-----Original Message----- From: NANOG <nanog-bounces+david=davidandrzejewski.com@nanog.org> On Behalf Of William Herrin Sent: Thursday, February 10, 2022 13:56 Cc: nanog@nanog.org Subject: Re: VPN recommendations?
On Thu, Feb 10, 2022 at 10:04 AM David Guo <david@xtom.com> wrote:
You may try WireGuard and use ddns
Hi David,
My understanding is that Wireguard is software available for general purpose operating systems. I specifically need a set of hardware network appliances. I don't overly care which protocol they're running as long as an initiator stuck behind a nat box I don't control can maintain a connection with a hub and handle speeds in the100mbps to 10gbps.
On Thu, Feb 10, 2022 at 10:12 AM Mike Lyon <mike.lyon@gmail.com> wrote:
How about running ZeroTier on those Linux boxes and call it a day? https://www.zerotier.com/
I specifically cannot use general purpose Linux machines for this. I need network appliances.
On Thu, Feb 10, 2022 at 10:26 AM Dave Taht <dave.taht@gmail.com> wrote:
tailscale
I specifically need an integrated network appliance, not software I add to something.
I love my Linux-based VPN servers but my customer very specifically said no. I can't publicly explain why but trust me when I say it's a "hard no" and it's not a question of persuasion or education. My customer understands and likes Linux but he simply cannot use it this time.
Regards, Bill Herrin
-- William Herrin bill@herrin.us https://bill.herrin.us/
-- I tried to build a better future, a few times: https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org Dave Täht CEO, TekLibre, LLC
On 2/11/22 06:49, David Andrzejewski wrote:
I don't know how people around here feel about Mikrotik, but they have included Wireguard support in their latest operating system.
I know some Tik heads here that are happy about this. I am running ROS 7.1.2 on my home router, but I don't use it. Mark.
Pfsense on Netgate appliances? I’ve used several of them, while not for this exact purpose they have done the roles but maybe not the amount of VPN traffic. -- Keith Stokes SalonBiz, Inc On Feb 10, 2022, at 12:02 PM, William Herrin <bill@herrin.us<mailto:bill@herrin.us>> wrote: Hi folks, Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses. Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement. Thanks in advance, Bill Herrin -- William Herrin bill@herrin.us<mailto:bill@herrin.us> <https://bill.herrin.us/> https://bill.herrin.us/
Meraki MX series? I don't like the way they do their licensing (your license runs out, the box is a paper-weight) but they do really well at establishing site-to-site VPNs in some pretty challenging scenarios. Dynamic IPs and NATs don't really cause them a problem. Some CGNats do (AT&T I'm looking at you). Shawn -----Original Message----- From: "Keith Stokes" <keiths@salonbiz.com> Sent: Thursday, February 10, 2022 1:11pm To: "William Herrin" <bill@herrin.us> Cc: "nanog@nanog.org" <nanog@nanog.org> Subject: Re: VPN recommendations? Pfsense on Netgate appliances? I’ve used several of them, while not for this exact purpose they have done the roles but maybe not the amount of VPN traffic. -- Keith Stokes SalonBiz, Inc On Feb 10, 2022, at 12:02 PM, William Herrin <[ bill@herrin.us ]( mailto:bill@herrin.us )> wrote: Hi folks, Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses. Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement. Thanks in advance, Bill Herrin -- William Herrin [ bill@herrin.us ]( mailto:bill@herrin.us )[ ]( https://bill.herrin.us/ ) [ https://bill.herrin.us/ ]( https://bill.herrin.us/ )
pfsense and opnsense both do fine with natted ipsec in the environmnets i've tested. Isn't there an openvpn appliance too? On 2/10/2022 1:17 PM, Shawn L via NANOG wrote:
Meraki MX series?
I don't like the way they do their licensing (your license runs out, the box is a paper-weight) but they do really well at establishing site-to-site VPNs in some pretty challenging scenarios. Dynamic IPs and NATs don't really cause them a problem. Some CGNats do (AT&T I'm looking at you).
Shawn
-----Original Message----- From: "Keith Stokes" <keiths@salonbiz.com> Sent: Thursday, February 10, 2022 1:11pm To: "William Herrin" <bill@herrin.us> Cc: "nanog@nanog.org" <nanog@nanog.org> Subject: Re: VPN recommendations?
Pfsense on Netgate appliances? I’ve used several of them, while not for this exact purpose they have done the roles but maybe not the amount of VPN traffic.
-- Keith Stokes SalonBiz, Inc
On Feb 10, 2022, at 12:02 PM, William Herrin <bill@herrin.us> wrote:
Hi folks, Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses. Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement. Thanks in advance, Bill Herrin
-- William Herrin bill@herrin.us <https://bill.herrin.us/> https://bill.herrin.us/
tailscale On Thu, Feb 10, 2022 at 10:24 AM Mark Wiater <mark.wiater@greybeam.com> wrote:
pfsense and opnsense both do fine with natted ipsec in the environmnets i've tested.
Isn't there an openvpn appliance too?
On 2/10/2022 1:17 PM, Shawn L via NANOG wrote:
Meraki MX series?
I don't like the way they do their licensing (your license runs out, the box is a paper-weight) but they do really well at establishing site-to-site VPNs in some pretty challenging scenarios. Dynamic IPs and NATs don't really cause them a problem. Some CGNats do (AT&T I'm looking at you).
Shawn
-----Original Message----- From: "Keith Stokes" <keiths@salonbiz.com> Sent: Thursday, February 10, 2022 1:11pm To: "William Herrin" <bill@herrin.us> Cc: "nanog@nanog.org" <nanog@nanog.org> Subject: Re: VPN recommendations?
Pfsense on Netgate appliances? I’ve used several of them, while not for this exact purpose they have done the roles but maybe not the amount of VPN traffic.
-- Keith Stokes SalonBiz, Inc
On Feb 10, 2022, at 12:02 PM, William Herrin <bill@herrin.us> wrote:
Hi folks, Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses. Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement. Thanks in advance, Bill Herrin
-- William Herrin bill@herrin.us https://bill.herrin.us/
-- I tried to build a better future, a few times: https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org Dave Täht CEO, TekLibre, LLC
Wireguard is the way to go. No platform lock-in, encrypted, extremely lightweight and an easy to configure kernel module. Only drawback being that there’s no implemented mesh topology, but that doesn’t sound like a requirement for your use case. We actively push 8Gbit through our WG tunnels with no issues. Phin On Thu, Feb 10, 2022 at 6:26 PM Dave Taht <dave.taht@gmail.com> wrote:
tailscale
On Thu, Feb 10, 2022 at 10:24 AM Mark Wiater <mark.wiater@greybeam.com> wrote:
pfsense and opnsense both do fine with natted ipsec in the environmnets
i've tested.
Isn't there an openvpn appliance too?
On 2/10/2022 1:17 PM, Shawn L via NANOG wrote:
Meraki MX series?
I don't like the way they do their licensing (your license runs out, the
box is a paper-weight) but they do really well at establishing site-to-site VPNs in some pretty challenging scenarios. Dynamic IPs and NATs don't really cause them a problem. Some CGNats do (AT&T I'm looking at you).
Shawn
-----Original Message----- From: "Keith Stokes" <keiths@salonbiz.com> Sent: Thursday, February 10, 2022 1:11pm To: "William Herrin" <bill@herrin.us> Cc: "nanog@nanog.org" <nanog@nanog.org> Subject: Re: VPN recommendations?
Pfsense on Netgate appliances? I’ve used several of them, while not for this exact purpose they have
done the roles but maybe not the amount of VPN traffic.
-- Keith Stokes SalonBiz, Inc
On Feb 10, 2022, at 12:02 PM, William Herrin <bill@herrin.us> wrote:
Hi folks, Do you have any recommendations for VPN appliances? Specifically: I need
Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but
to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses. traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement.
Thanks in advance, Bill Herrin
-- William Herrin bill@herrin.us https://bill.herrin.us/
-- I tried to build a better future, a few times: https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
Dave Täht CEO, TekLibre, LLC
On Thu, Feb 10, 2022 at 10:18 AM Shawn L <shawnl@up.net> wrote:
Meraki MX series? Dynamic IPs and NATs don't really cause them a problem. Some CGNats do (AT&T I'm looking at you).
Thanks Shawn, The documentation I found at https://documentation.meraki.com/MX/Site-to-site_VPN/Site-to-Site_VPN_Settin... suggests that the NAT firewall has to be explicitly configured to deliver UDP 500/4500 to the Meraki behind it. Are you aware of any documentation that describes: LAN - Meraki - NAT (dynaimic IP) - Internet - (static IP) Meraki - LAN Where the left-side Meraki is responsible for establishing and keeping the NAT translations alive without any special configuration on the NAT? Regards, Bill -- William Herrin bill@herrin.us https://bill.herrin.us/
(your license runs out, the box is a paper-weight)
Should be a hard no for anyone purchasing network equipment anyways, but people have reasons I guess. On Thu, Feb 10, 2022 at 1:19 PM Shawn L via NANOG <nanog@nanog.org> wrote:
Meraki MX series?
I don't like the way they do their licensing (your license runs out, the box is a paper-weight) but they do really well at establishing site-to-site VPNs in some pretty challenging scenarios. Dynamic IPs and NATs don't really cause them a problem. Some CGNats do (AT&T I'm looking at you).
Shawn
-----Original Message----- From: "Keith Stokes" <keiths@salonbiz.com> Sent: Thursday, February 10, 2022 1:11pm To: "William Herrin" <bill@herrin.us> Cc: "nanog@nanog.org" <nanog@nanog.org> Subject: Re: VPN recommendations?
Pfsense on Netgate appliances? I’ve used several of them, while not for this exact purpose they have done the roles but maybe not the amount of VPN traffic.
-- Keith Stokes SalonBiz, Inc
On Feb 10, 2022, at 12:02 PM, William Herrin <bill@herrin.us> wrote:
Hi folks, Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses. Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement. Thanks in advance, Bill Herrin
-- William Herrin bill@herrin.us <https://bill.herrin.us/> https://bill.herrin.us/
Sabri Berisha <sabri@cluecentral.net> writes:
I read on some mailing list that Meraki likes to ping 8.8.8.8 every second... :)
That's probably to be fair with the quad-x dns providers since they alrady were abusing 1.1.1.1. Makes me wonder what Meraki uses 9.9.9.9 for :-) Bjørn
If you want something gui driven I’d do something like Meraki…you can do the same with just regular old Cisco routers using DMVPN as well. It’s a pretty common use case and well established. On Thu, Feb 10, 2022 at 1:03 PM William Herrin <bill@herrin.us> wrote:
Hi folks,
Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses.
Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement.
Thanks in advance, Bill Herrin
-- William Herrin bill@herrin.us <https://bill.herrin.us/> https://bill.herrin.us/
I’ll second PFsense, done quite a bit of this in hub and spoke topologies, spokes being behind NAT (permitted the upstream fw allows udp 500,4500), on a dynamic. The hub or hubs are ideally on a static. Set the hub site up as responder only, the remotes initiate the tunnel. Peers are validated either by dynamic name or you simply allow peers sourcing from 0.0.0.0 at the hub site. This is not limited to PF, I’ve gotten this to work on Cisco firewalls, routers, and other Linux based firewalls. From: NANOG <nanog-bounces+james=digitalciti.com@nanog.org> On Behalf Of William Herrin Sent: Thursday, February 10, 2022 12:02 PM To: nanog@nanog.org Subject: VPN recommendations? Hi folks, Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses. Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement. Thanks in advance, Bill Herrin -- William Herrin bill@herrin.us<mailto:bill@herrin.us> <https://bill.herrin.us/> https://bill.herrin.us/
Matt Harris|Infrastructure Lead 816-256-5446|Direct Looking for help? Helpdesk|Email Support We build customized end-to-end technology solutions powered by NetFire Cloud. On Thu, Feb 10, 2022 at 12:03 PM William Herrin <bill@herrin.us> wrote:
Hi folks,
Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses.
Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement.
For OpenVPN, I like the Netgate boxes running pfsense. Works great, super easy integrations with stuff like AC/LDAP/radius/etc for auth, frr and others for your routing, etc. This is probably your best bet. For IPSec I tend to stick to Juniper SRX boxes. Good luck!
We use SonicWall TZ series for just this purpose. The IPSec VPN endpoints can be behind NAT, and we just use DYNDNS to map whatever is current to a FQDN. Each side thus has the public IP of the other side and can connect as long as you pass through GRE. -mel via cell On Feb 10, 2022, at 1:05 PM, Matt Harris <matt@netfire.net> wrote: [cid:image200517.png@6CD88F22.1B50C51A] Matt Harris | Infrastructure Lead 816‑256‑5446 | Direct Looking for help? Helpdesk<https://help.netfire.net/> | Email Support<mailto:help@netfire.net> [https://netfire.net/Flag-United-States-of-America.jpg] We build customized end‑to‑end technology solutions powered by NetFire Cloud. On Thu, Feb 10, 2022 at 12:03 PM William Herrin <bill@herrin.us<mailto:bill@herrin.us>> wrote: Hi folks, Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses. Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement. For OpenVPN, I like the Netgate boxes running pfsense. Works great, super easy integrations with stuff like AC/LDAP/radius/etc for auth, frr and others for your routing, etc. This is probably your best bet. For IPSec I tend to stick to Juniper SRX boxes. Good luck!
Meraki may be considered expensive, requires perpetual license to operate and is difficult to get currently (very long lead times) but is dead.stupid.simple to install and maintain. I have yet to find a business or home network that it does not work on out of the box, but if you find one it would be an issue to overcome for any solution, right? i.e. open some ports on the up stream device one time. https://documentation.meraki.com/MX/Site-to-site_VPN/Meraki_Auto_VPN_-_Confi... *Brandon Svec* On Thu, Feb 10, 2022 at 10:05 AM William Herrin <bill@herrin.us> wrote:
Hi folks,
Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses.
Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement.
Thanks in advance, Bill Herrin
-- William Herrin bill@herrin.us <https://bill.herrin.us/> https://bill.herrin.us/
I think my experience is unique, but wanted to put it out there anyway. I’ve actually had quite a few problems with Meraki equipment during the one instance I worked with them. After a few hours to days, the switches would stop functioning. You could still access them through the webgui and issue a reboot to resolve the issue, but the problem persisted even after many resets and calls with Cisco. Again, likely some bonk hardware, but in case anyone else has had a similar experience I wanted this to be known. Thank you, -- Ryland ________________________________ From: NANOG <nanog-bounces+rkremeier=barryelectric.com@nanog.org> on behalf of Brandon Svec via NANOG <nanog@nanog.org> Sent: Thursday, February 10, 2022 3:50:49 PM To: William Herrin <bill@herrin.us> Cc: nanog@nanog.org <nanog@nanog.org> Subject: Re: VPN recommendations? Meraki may be considered expensive, requires perpetual license to operate and is difficult to get currently (very long lead times) but is dead.stupid.simple to install and maintain. I have yet to find a business or home network that it does not work on out of the box, but if you find one it would be an issue to overcome for any solution, right? i.e. open some ports on the up stream device one time. https://documentation.meraki.com/MX/Site-to-site_VPN/Meraki_Auto_VPN_-_Confi... Brandon Svec On Thu, Feb 10, 2022 at 10:05 AM William Herrin <bill@herrin.us<mailto:bill@herrin.us>> wrote: Hi folks, Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses. Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement. Thanks in advance, Bill Herrin -- William Herrin bill@herrin.us<mailto:bill@herrin.us> <https://bill.herrin.us/> https://bill.herrin.us/
Howdy, I just want to say thank you to everyone who responded. It was very helpful and I now have a bunch of leads to chase. I'll let you know what I end up doing. Given the lead times on some of the equipment it may be a while... Warm regards, Bill Herrin On Thu, Feb 10, 2022 at 10:02 AM William Herrin <bill@herrin.us> wrote:
Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses.
Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement.
Thanks in advance, Bill Herrin
-- William Herrin bill@herrin.us https://bill.herrin.us/
On 2/10/22 20:02, William Herrin wrote:
Hi folks,
Do you have any recommendations for VPN appliances? Specifically: I need to build a site to site VPNs at speeds between 100mpbs and 1 gbit where all but one of the sites are behind an IPv4 NAT gateway with dynamic public IP addresses.
Normally I'd throw OpenVPN on a couple of Linux boxes and be happy but my customer insists on a network appliance. Site to site VPNs using IPSec and static IP addresses on the plaintext side are a dime a dozen but traversing NAT and dynamic IP addresses (and automatically re-establishing when the service goes out and comes back up with different addresses) is a hard requirement.
We like pfSense. I believe they sell Netgate appliances. Mark.
participants (30)
-
Ander Punnar -
Bjørn Mork -
Brandon Svec -
Christian de Larrinaga -
Crist Clark -
Dan Sneddon -
Dave Taht -
David Andrzejewski -
David Bass -
David Guo -
Grant Taylor -
James R. Price -
John Gilmore -
joy＠cleverhack.com -
Keith Stokes -
Mark Tinka -
Mark Wiater -
Matt Harris -
Mel Beckman -
Mike Hammett -
Mike Lyon -
Nathan Angelacos -
Phineas Walton -
Ryland Kremeier -
Sabri Berisha -
Sean Kelly -
Shawn L -
Tom Beecher -
Valdis Klētnieks -
William Herrin