Instead you configure one of the available hash-based distribution functions the two endpoints have in common. [...] In my own experience, packet loss on Cisco-Juniper LACP links has arisen from inconsistent or incompatible configurations.

Mel Beckman via NANOG wrote on 26/09/2025 06:33: this analysis is straight down wrong. The hashing algos on each side of a LAG bundle are entirely independent of each other and there is no problem whatever with using different hashing algos on each side. Nick

Mel Beckman wrote on 26/09/2025 15:32:

From https://www.exam-labs.com/blog/configuring-lacp-between-cisco-ios-and-junipe...

*Understanding LACP Failures and Common Pitfalls*

Link aggregation, although a robust feature, is susceptible to various issues. To resolve these problems effectively, network administrators must first understand the potential causes of LACP failures. Here are some of the most common causes for LACP issues:

1. *Mismatched Configurations:** *Often, the primary reason for LACP failure is a misalignment in configurations between the two devices. This might include differences in LACP modes (active vs. passive), inconsistent port-channel settings, or mismatched VLANs. Proper alignment of settings across both devices is crucial to the successful negotiation of LACP.

Not sure what this has to do with your previous email, where you were specifically referring to hashing algos needing to be the same or "compatible" on each side of the link:

Instead you configure one of the available hash-based distribution functions the two endpoints have in common.

I’ve found that sometimes with LAGs between different equipment vendors one or more of these algorithms aren’t compatible, resulting in packets out of order or even dropped.

There's no such thing as the packet hashing algorithms needing to be "compatible" on each side of a LAG bundle. You can use whatever hashing algo you want on either side and it doesn't make the slightest difference at the other end. There's no concept of hashing algo compatibility, and hashing doesn't come into play with LACP or any other sort of negotiation. With the link above, you're confusing LACP negotiation issues with packet loss. If you have an LACP negotiation failure, typically the LAG interface won't come up to start with on a juniper / cisco device. Sometimes - rarely - you can have failure modes where an individual bearer circuit won't be used correctly, which would cause 1/N*100% packet loss, i.e. 50% if N=2. Also, as this is an IXP link, it probably means that it's a single untagged VLAN, i.e. no mismatched vlans. Although if there is a mismatched vlan, then that vlan will experience 100% packet loss. Nick

I think folks are mixing up concepts a bit in this thread. The *hashing algorithm* is not the same thing as the *load balancing* algorithm. I have a LACP bundle with 4 member links. The *load balancing* algorithm determines if traffic is balanced per packet or per flow across the member links. The *hashing* algorithm is what is used to decide which link to use for each traffic element, with the goal being even distribution across all possible paths. LACP establishment doesn't depend on these two things at all. There is also no requirement that both sides use the same LB or hashing algorithms. ( There are cases where you absolutely DON'T want that to happen anyways.) What can occur is that one side uses a given LB/hashing combo such that traffic hotspots to one of the member links, running it over, and some stuff gets dropped. What can also occur is that if one side does per-packet balancing, which can create all kinds of out of order packet problems. On Fri, Sep 26, 2025 at 10:33 AM Mel Beckman via NANOG < nanog@lists.nanog.org> wrote:

Nick,

From

https://www.exam-labs.com/blog/configuring-lacp-between-cisco-ios-and-junipe...

Understanding LACP Failures and Common Pitfalls

Link aggregation, although a robust feature, is susceptible to various issues. To resolve these problems effectively, network administrators must first understand the potential causes of LACP failures. Here are some of the most common causes for LACP issues:

1. Mismatched Configurations: Often, the primary reason for LACP failure is a misalignment in configurations between the two devices. This might include differences in LACP modes (active vs. passive), inconsistent port-channel settings, or mismatched VLANs. Proper alignment of settings across both devices is crucial to the successful negotiation of LACP.

-mel

On Sep 26, 2025, at 7:10 AM, Nick Hilliard <nick@foobar.org> wrote:

Mel Beckman via NANOG wrote on 26/09/2025 06:33: Instead you configure one of the available hash-based distribution functions the two endpoints have in common. [...] In my own experience, packet loss on Cisco-Juniper LACP links has arisen from inconsistent or incompatible configurations. this analysis is straight down wrong. The hashing algos on each side of a LAG bundle are entirely independent of each other and there is no problem whatever with using different hashing algos on each side.

Nick _______________________________________________ NANOG mailing list

https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/EJYFZTRG...

Hi Andy, Andy Cole via NANOG wrote on 26/09/2025 04:21:

No configuration changes to routing policy at all. After a few days we started to get customer complaints for certain sites/domains being unreachable. I worked around the issue by not announcing the customer blocks to the route servers and changed the return path to traverse transit. This solved the issue, but I'm perplexed as to what could've caused the issue, and where to look to resolve it. If you guys could provide feedback and point me in the right direction I'd appreciate it. TIA.

Besides what others have mentioned, Another thing that changed moving from the physical interface to lacp is going to be the mac used on the ixp lan. I would check connectivity to all IPs on the peering lan as they may receive your routes from the route server but not be able to contact you directly. You may have already done this but as you mentioned removing the announcements to the route server, to solve the issue it rang the route server blackhole bell for me. It wouldn't be the first time I've seen this happen. Brian Brian

HI, On Fri, 26 Sept 2025 at 17:23, Tom Beecher <beecher@beecher.cc> wrote:

...

Besides what others have mentioned, Another thing that changed moving from the physical interface to lacp is going to be the mac used on the ixp lan. I would check connectivity to all IPs on the peering lan as they may receive your routes from the route server but not be able to contact you directly. You may have already done this but as you mentioned removing the announcements to the route server, to solve the issue it rang the route server blackhole bell for me. It wouldn't be the first time I've seen this happen.

This is an excellent reminder. Many IXPs put a MAC filter on each port that you have to have them change if you change your end.

I'm guessing this isn't the case here, since the OP stated his BGP sessions came up and traffic was flowing, but it's possible and a good callout.

Yes it is not an acl on his port, as no sessions would come up, but maybe a fabric forwarding issue. I have no idea what they use if flat L2, evpn etc on the peering lan but it would be something to check. It could even be some crazy member that fixed static arp tables or L2 acls for "security purposes" ;-) Brian

IMHO, the key info here is that a known set of subnets was affected. This rules out some stuff: - LACP manages link bundling, as in “can this interface be added to the bundle?”. The effect of bundling should be to have multiple links to choose from when egressing a packet. RFC7130 is a nice addition to bundles as it uses BFD to manage each link - meaning a bad member is removed quickly (LACP timers are not that fast and LACP itself is not designed to react fast). - Hashing (which is used for load balancing traffic in hardware switches) is not managed by LACP - it’s always local to each device and as said before, each side usually has a different view of the ideal hashing. A classical example is when there are many IPs behind one firewall doing NAT - you can’t rely on diversity of IPs and MACs to select an egress link, so you usually change the hashing to be per port. - Link errors would affect random traffic to any destination / from any source So, none of the above technologies would affect traffic connectivity _selectively_. Perhaps a malformed bundle could blackhole traffic, but that wouldn’t be specific to certain subnets unless someone is *extremely* unlucky and _only_ his subnets hashed to the “bad bundle member” :) It simply looks like a routing issue through this path. Perhaps the flapping of the BGP session re-advertised this path to some place that previously wasn’t using it, and apparently can't? Pedro Martins Prado pedro.prado@gmail.com / +353 83 036 1875

On 28 Sep 2025, at 06:24, William Herrin via NANOG <nanog@lists.nanog.org> wrote:

On Sat, Sep 27, 2025 at 7:31 PM Bruce Wainer via NANOG <nanog@lists.nanog.org> wrote:

...

Excuse my ignorance about this IXP and your equipment, but is Micro-BFD (RFC 7130) supported? And if so, is it enabled or can you enable it? While configuration wise it will use the single IP addresses of the aggregate, separate BFD instances are set up for each underlying link and will confirm whether Layer 3 is working on that point-to-point connection.

Hi Bruce,

I'm also not familiar with this particular IXP but generally with IXPs we're not talking about point to point connections. The multiple participants' routers are part of a shared layer-2 fabric (a switch or switches) over which they trade layer-3 packets directly with each other. The route advertisements may transit the route servers but the routed packets do not.

You can get into some really finicky errors where both participants successfully talk to the route server and thereby exchange routes, but for one reason or another can't get packets back and forth to each other. Bonded circuits (LAGs) add complexity which makes troubleshooting that much harder.

If it were me, I would have considered building this connection differently. For speed, I'd have chosen a 100G link instead of two 10G links. Had my objective been reliability, I'd have built that at layer 3 instead of layer 2 -- two routers each with its own 10G link, and then done some balancing of the advertised routes. But in all fairness to Andy, I don't have anywhere near complete information here and the details matter a lot.

Regards, Bill Herrin

-- William Herrin bill@herrin.us https://bill.herrin.us/ _______________________________________________ NANOG mailing list https://lists.nanog.org/archives/list/nanog@lists.nanog.org/message/RXKZDTZB...