My guess is something is doing stateful filtering. If you send a SYN down one link and the SYN-ACK comes back a different link, the receiving firewall will discard it as bogus. You should be able to test this by doing pcaps to confirm the traffic is arriving (though I'm not familiar with WireGuard so maybe not), and you should be able to disable this by setting a rule or unchecking a box in your firewall.

On Wed, Jun 12, 2019, 5:47 PM Anurag Bhatia <me@anuragbhatia.com> wrote:
Hello everyone, 

Trying to get my head around a certain unexpected behaviour. 


I am running two site to site VPNs (wireguard now, OpenVPN earlier) between my home and a remote server over two different WAN links. Both WAN links are just consumer connections - one with public IP and one with CGNATed IP. 
The redundancy here is taken care of by the OSPF running via FRR on both ends. 


The unexpected behaviour I get is that if I set OSPF cost to prefer say link1 between home -> server and prefer link 2 between server -> home then connectivity completely breaks between the routed pools. The point to point IPs stay reachable (which is over expected links i.e symmetric via both ends). As long as both ends prefer link1 or link2, it works fine. At first, I thought it had to do something with NAT but still can't understand how. Since VPN tunnels have a keep-alive timer (for 10 seconds), the tunnel is always up. Any idea why asymmetric packets are being dropped here? 
This exact behaviour was in case of earlier OpenVPN + bird + iBGP and is still the same when I moved everything to Wireguard for VPN + FRR for routing + OSPF. 




Thanks. 


--


Anurag Bhatia