Greetings, Dear Community! Consider the following scenario: major colo with a pair of transits, peering, and a single transport back to another colo on our backbone. Transport carries public but also several overlays (VRFs) for management and whatnot. If the transport fails, we're good on transit/peering, but we can't get back to the mothership for mgmt. We're looking at solutions (secure tunnels over transit) to bring the severed colo back to "HQ" ... looking at a hub/spoke topology with the intent of possibly doing this more than once. Requirements: * Multiple VRFs across the tunnel * OSPF - each VRF should have its own instance, so we need something that supports interface-based tunneling since IPsec doesn't handle multicast well. Open to other tunneling strategies. Wireguard? OpenVPN? * v6 a plus (OSPFv3) * 10G should suffice across the board, but it should have interfaces that are LAGable. The appliances we have considered so far do most if not all of these things, but they come with a lot of features (and cost) we simply don't need (e.g., UTM, DPI) Also open to something server (VM) based since our traffic requirements aren't that significant. Easy to support is obviously a plus. Curious if others have had similar needs and how they solved this problem. Recommendations (good or bad) greatly appreciated. Thank you! - bryan
Hi Bryan! Just wondering if you could be over complicating things. One option is to continue to manage your devices in-band with your current strategy and if that fails, switch to an out-of-band solution. This way, the box is still reachable no matter what happens short of a complete power outage or worse. One option to do this is with Opengear hardware (https://opengear.com/) and using their Lighthouse software (https://opengear.com/products/lighthouse/) to manage it back at HQ. There are other options out there as well but this could be cheaper than buying a new server for wireguard (also a viable option) or a full-on networking device. Of course, there might be other considerations given your statement "management and whatnot" but if all you are concerned about is contacting the box when a line goes down, this is might be a good option for your use case. Regards, Michael The views and opinions included in this email belong to the author and are not representative of the views and opinions of the company which employs me. If you find a spelling or grammatical error, you may keep it.
On 12/03/2026 20:25, Bryan Holloway via NANOG wrote:
* OSPF - each VRF should have its own instance, so we need something that supports interface-based tunneling since IPsec doesn't handle multicast well. Open to other tunneling strategies. Wireguard? OpenVPN?
We've built a DCN network for our optical backbone based on pfSense and FreeBSD with WireGuard, OSPF and BGP, across diverse DIA links in each data centre. Works pretty good. WireGuard is awesome! Can't imagine how we made IPSec work :-)... Mark.
Linux + WireGuard does most of what you need easily and all of what you described with some effort. I’d use a separate egg tunnel for each VRF rather than trying your mix them, but you do you. Owen
On Mar 12, 2026, at 19:58, Mark Tinka via NANOG <nanog@lists.nanog.org> wrote:
On 12/03/2026 20:25, Bryan Holloway via NANOG wrote:
* OSPF - each VRF should have its own instance, so we need something that supports interface-based tunneling since IPsec doesn't handle multicast well. Open to other tunneling strategies. Wireguard? OpenVPN?
We've built a DCN network for our optical backbone based on pfSense and FreeBSD with WireGuard, OSPF and BGP, across diverse DIA links in each data centre.
Works pretty good.
WireGuard is awesome! Can't imagine how we made IPSec work :-)...
Mark. _______________________________________________ NANOG mailing list https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/5QSGH5FP...
Mikrotik can do all of that. At 10gig speeds without issues. Contact linktechs.net if you need suggestions, etc. -----Original Message----- From: Bryan Holloway via NANOG <nanog@lists.nanog.org> Sent: Thursday, March 12, 2026 2:25 PM To: North American Network Operators Group <nanog@lists.nanog.org> Cc: Bryan Holloway <bryan@shout.net> Subject: Last-resort tunnel recommendations on WAN network ... Greetings, Dear Community! Consider the following scenario: major colo with a pair of transits, peering, and a single transport back to another colo on our backbone. Transport carries public but also several overlays (VRFs) for management and whatnot. If the transport fails, we're good on transit/peering, but we can't get back to the mothership for mgmt. We're looking at solutions (secure tunnels over transit) to bring the severed colo back to "HQ" ... looking at a hub/spoke topology with the intent of possibly doing this more than once. Requirements: * Multiple VRFs across the tunnel * OSPF - each VRF should have its own instance, so we need something that supports interface-based tunneling since IPsec doesn't handle multicast well. Open to other tunneling strategies. Wireguard? OpenVPN? * v6 a plus (OSPFv3) * 10G should suffice across the board, but it should have interfaces that are LAGable. The appliances we have considered so far do most if not all of these things, but they come with a lot of features (and cost) we simply don't need (e.g., UTM, DPI) Also open to something server (VM) based since our traffic requirements aren't that significant. Easy to support is obviously a plus. Curious if others have had similar needs and how they solved this problem. Recommendations (good or bad) greatly appreciated. Thank you! - bryan _______________________________________________ NANOG mailing list https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/77KTXLHW...
participants (5)
-
Bryan Holloway -
Dennis - LTI Support -
Mark Tinka -
Michael Greenup -
Owen DeLong