Hi Nanog-ers, Hoping someone may have come across a similar issue. Has anyone ever seen a situation where maybe like a Level3 transport system could be possibly dropping LACP frames..? End point A - tx and rx counts incrementing for LACP LACP info: Role System System Port Port Port priority identifier priority number key et-0/0/0.0 Actor 127 5c:45:27:6d:2a:c0 127 56 16 et-0/0/0.0 Partner 1 00:00:00:00:00:00 127 56 16 LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx et-0/0/0.0 6925 6922 0 0 End Point B - no RX, partner macs are 0s.. LACP info: Role System System Port Port Port priority identifier priority number key et-9/1/0.0 Actor 127 5c:45:27:77:d6:c4 127 68 16 et-9/1/0.0 Partner 1 00:00:00:00:00:00 1 68 16 LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx et-9/1/0.0 0 6752 0 0 Link works fine otherwise outside the aggregate and w/o LACP. Any inputs will be greatly appreciated. thanks, -nevin
What is performing the LACP? The Level3 transport system for the most part is purley optical, so I don't think it touches LACP. Did you check the hash values? On Sun, May 22, 2016 at 2:55 PM, Nevin Gonsalves via NANOG <nanog@nanog.org> wrote:
Hi Nanog-ers, Hoping someone may have come across a similar issue. Has anyone ever seen a situation where maybe like a Level3 transport system could be possibly dropping LACP frames..? End point A - tx and rx counts incrementing for LACP LACP info: Role System System Port Port Port priority identifier priority number key et-0/0/0.0 Actor 127 5c:45:27:6d:2a:c0 127 56 16 et-0/0/0.0 Partner 1 00:00:00:00:00:00 127 56 16 LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx et-0/0/0.0 6925 6922 0 0 End Point B - no RX, partner macs are 0s.. LACP info: Role System System Port Port Port priority identifier priority number key et-9/1/0.0 Actor 127 5c:45:27:77:d6:c4 127 68 16 et-9/1/0.0 Partner 1 00:00:00:00:00:00 1 68 16 LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx et-9/1/0.0 0 6752 0 0 Link works fine otherwise outside the aggregate and w/o LACP. Any inputs will be greatly appreciated. thanks, -nevin
Yes. Many vendors are using l2vpn/pseudo-wire services of one sort or another to provide circuits and most do not transport LACP by default. LACP uses slow-protocols address: https://wiki.wireshark.org/LinkAggregationControlProtocol If they are using ALU gear, they can enable this using the port command: configure port <port> ethernet lacp-tunnel On Tue, May 24, 2016 at 12:08 AM Colton Conor <colton.conor@gmail.com> wrote:
What is performing the LACP? The Level3 transport system for the most part is purley optical, so I don't think it touches LACP. Did you check the hash values?
On Sun, May 22, 2016 at 2:55 PM, Nevin Gonsalves via NANOG < nanog@nanog.org> wrote:
Hi Nanog-ers, Hoping someone may have come across a similar issue. Has anyone ever seen a situation where maybe like a Level3 transport system could be possibly dropping LACP frames..? End point A - tx and rx counts incrementing for LACP LACP info: Role System System Port Port Port priority identifier priority number key et-0/0/0.0 Actor 127 5c:45:27:6d:2a:c0 127 56 16 et-0/0/0.0 Partner 1 00:00:00:00:00:00 127 56 16 LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx et-0/0/0.0 6925 6922 0 0 End Point B - no RX, partner macs are 0s.. LACP info: Role System System Port Port Port priority identifier priority number key et-9/1/0.0 Actor 127 5c:45:27:77:d6:c4 127 68 16 et-9/1/0.0 Partner 1 00:00:00:00:00:00 1 68 16 LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx et-9/1/0.0 0 6752 0 0 Link works fine otherwise outside the aggregate and w/o LACP. Any inputs will be greatly appreciated. thanks, -nevin
On 24/May/16 06:29, Rob Laidlaw wrote:
Yes. Many vendors are using l2vpn/pseudo-wire services of one sort or another to provide circuits and most do not transport LACP by default.
To the OP's case, commercially, I'd find it interesting to transport a 100Gbps circuit as EoMPLS rather than EoDWDM, considering the amount of bandwidth one would need to throw at an IP/MPLS network to transport 100Gbps effectively... Mark.
Or a very reckless oversubscription ratio and misjudgment of the customer, example, if a provider had 2 x 100GbE capacity between two locations and sold a customer a 100GbE EoMPLS transport circuit from A to Z, based on the mistaken idea of "Well these guys probably aren't going to peak more than 35Gbps of traffic at any time in the near future....". Frightening. On Tue, May 24, 2016 at 2:38 PM, Mark Tinka <mark.tinka@seacom.mu> wrote:
On 24/May/16 06:29, Rob Laidlaw wrote:
Yes. Many vendors are using l2vpn/pseudo-wire services of one sort or another to provide circuits and most do not transport LACP by default.
To the OP's case, commercially, I'd find it interesting to transport a 100Gbps circuit as EoMPLS rather than EoDWDM, considering the amount of bandwidth one would need to throw at an IP/MPLS network to transport 100Gbps effectively...
Mark.
On 25/May/16 00:14, Eric Kuhnke wrote:
Or a very reckless oversubscription ratio and misjudgment of the customer, example, if a provider had 2 x 100GbE capacity between two locations and sold a customer a 100GbE EoMPLS transport circuit from A to Z, based on the mistaken idea of "Well these guys probably aren't going to peak more than 35Gbps of traffic at any time in the near future....". Frightening.
Yeah, I wouldn't do that. Easier and cheaper to deliver the circuit over EoDWDM if you can't reserve enough capacity in the backbone. You could get away with it by doing an N x 100Gbps LAG, but EoMPLS traffic may or may not load balance well, depending on platform and payload. Mark.
On May 24, 2016, at 12:06 AM, Colton Conor <colton.conor@gmail.com> wrote:
What is performing the LACP? The Level3 transport system for the most part is purley optical, so I don't think it touches LACP. Did you check the hash values?
I’ve seen optical transport gear be non-transparent in a few situations when using OTU2 vs OTU2e, but they turned out to be a bug. If a L2 platform is being used to front-end a router, you could be seeing slow protocols like CDP/LLDP/LACP being consumed by that switch. If you configure LLDP does it pass end-to-end? - Jared
On 24/May/16 08:51, Jared Mauch wrote:
I’ve seen optical transport gear be non-transparent in a few situations when using OTU2 vs OTU2e, but they turned out to be a bug.
I've seen this as well, including in an SDH transport, where OSPF packets were being eaten (something Multicast-related).
If a L2 platform is being used to front-end a router, you could be seeing slow protocols like CDP/LLDP/LACP being consumed by that switch. If you configure LLDP does it pass end-to-end?
That's what I was thinking at first when I read the OP's post. If the circuit is being transported over MPLS or being fronted by an Ethernet switch, Level(3) would have to tunnel Layer 2 protocols in order to support your LACP frames across the pw. Mark.
Nevin, good day. Sun, May 22, 2016 at 07:55:31PM +0000, Nevin Gonsalves via NANOG wrote:
Hoping someone may have come across a similar issue. Has anyone ever seen a situation where maybe like a Level3 transport system could be possibly dropping LACP frames..? End point A - tx and rx counts incrementing for LACP LACP info: Role System System Port Port Port priority identifier priority number key et-0/0/0.0 Actor 127 5c:45:27:6d:2a:c0 127 56 16 et-0/0/0.0 Partner 1 00:00:00:00:00:00 127 56 16 LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx et-0/0/0.0 6925 6922 0 0 End Point B - no RX, partner macs are 0s.. LACP info: Role System System Port Port Port priority identifier priority number key et-9/1/0.0 Actor 127 5c:45:27:77:d6:c4 127 68 16 et-9/1/0.0 Partner 1 00:00:00:00:00:00 1 68 16 LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx et-9/1/0.0 0 6752 0 0 Link works fine otherwise outside the aggregate and w/o LACP. Any inputs will be greatly appreciated.
Cisco Q-in-Q implementation in some configurations (details are blurry, since our provider turned to X-connect quite fast). Also VPLS implementation in (older) EXOS releases (Extreme Networks), https://gtacknowledge.extremenetworks.com/articles/Solution/Layer-2-Control-... -- Eygene Ryabinkin, National Research Centre "Kurchatov Institute" Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.
Thanks all..! I just had to sit and trace all the cables to make sure the tx/rx lined up for the right circuits as well as hitting the right patch panel ports. Once all that got aligned nicely things started working magically. thanks,-nevin On Tuesday, May 24, 2016 2:49 AM, Eygene Ryabinkin <rea+nanog@grid.kiae.ru> wrote: Nevin, good day. Sun, May 22, 2016 at 07:55:31PM +0000, Nevin Gonsalves via NANOG wrote:
Hoping someone may have come across a similar issue. Has anyone ever seen a situation where maybe like a Level3 transport system could be possibly dropping LACP frames..? End point A - tx and rx counts incrementing for LACP LACP info: Role System System Port Port Port priority identifier priority number key et-0/0/0.0 Actor 127 5c:45:27:6d:2a:c0 127 56 16 et-0/0/0.0 Partner 1 00:00:00:00:00:00 127 56 16 LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx et-0/0/0.0 6925 6922 0 0 End Point B - no RX, partner macs are 0s.. LACP info: Role System System Port Port Port priority identifier priority number key et-9/1/0.0 Actor 127 5c:45:27:77:d6:c4 127 68 16 et-9/1/0.0 Partner 1 00:00:00:00:00:00 1 68 16 LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx et-9/1/0.0 0 6752 0 0 Link works fine otherwise outside the aggregate and w/o LACP. Any inputs will be greatly appreciated.
Cisco Q-in-Q implementation in some configurations (details are blurry, since our provider turned to X-connect quite fast). Also VPLS implementation in (older) EXOS releases (Extreme Networks), https://gtacknowledge.extremenetworks.com/articles/Solution/Layer-2-Control-... -- Eygene Ryabinkin, National Research Centre "Kurchatov Institute" Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.
Tue, May 24, 2016 at 12:39:03PM +0000, Nevin Gonsalves wrote:
I just had to sit and trace all the cables to make sure the tx/rx lined up for the right circuits as well as hitting the right patch panel ports. Once all that got aligned nicely things started working magically.
Yep, ports in an "up" state, but LACP not working is the sign of bad cabling: had been hit by this overnight once when I was preparing to leave the facilities next day for conference, but ought to make 10G for the new servers working. Took around 1/2 hour to sense what happened at that time (tx was going to, say, port A and rx -- to port B, but overall all ports were receiving tx and rx) and 3 hours for rewiring and swearing: probably I am more skilled in the former than in the latter ;) Thought that you had checked this in the first place; my bad. Thanks for sharing! -- Eygene Ryabinkin, National Research Centre "Kurchatov Institute" Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.
participants (7)
-
Colton Conor -
Eric Kuhnke -
Eygene Ryabinkin -
Jared Mauch -
Mark Tinka -
Nevin Gonsalves -
Rob Laidlaw