When lacp_active is set to off, the bond operates in passive mode, meaning it
will only "speak when spoken to." However, the current kernel implementation
only sends an LACPDU in response when the partner's state changes.
In this situation, once LACP negotiation succeeds, the actor stops sending
LACPDUs until the partner times out and sends an "expired" LACPDU.
This leads to endless LACP state flapping.
To avoid this, we need update ntt to true once received an LACPDU from the
partner, ensuring an immediate reply. With this fix, the link becomes stable
in most cases, except for one specific scenario:
Actor: lacp_active=off, lacp_rate=slow
Partner: lacp_active=on, lacp_rate=fast
In this case, the partner expects frequent LACPDUs (every 1 second), but the
actor only responds after receiving an LACPDU, which, in this setup, the
partner sends every 30 seconds due to the actor's lacp_rate=slow. By the time
the actor replies, the partner has already timed out and sent an "expired"
LACPDU.
Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
Signed-off-by: Hangbin Liu <liuhangbin@gmail.com>
---
drivers/net/bonding/bond_3ad.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/net/bonding/bond_3ad.c b/drivers/net/bonding/bond_3ad.c
index c6807e473ab7..e001d1c8a49b 100644
--- a/drivers/net/bonding/bond_3ad.c
+++ b/drivers/net/bonding/bond_3ad.c
@@ -666,6 +666,8 @@ static void __update_default_selected(struct port *port)
*/
static void __update_ntt(struct lacpdu *lacpdu, struct port *port)
{
+ struct bonding *bond;
+
/* validate lacpdu and port */
if (lacpdu && port) {
/* check if any parameter is different then
@@ -683,6 +685,10 @@ static void __update_ntt(struct lacpdu *lacpdu, struct port *port)
) {
port->ntt = true;
}
+
+ bond = __get_bond_by_port(port);
+ if (bond && !bond->params.lacp_active)
+ port->ntt = true;
}
}
--
2.46.0
Hangbin Liu <liuhangbin@gmail.com> wrote: >When lacp_active is set to off, the bond operates in passive mode, meaning it >will only "speak when spoken to." However, the current kernel implementation >only sends an LACPDU in response when the partner's state changes. > >In this situation, once LACP negotiation succeeds, the actor stops sending >LACPDUs until the partner times out and sends an "expired" LACPDU. >This leads to endless LACP state flapping. From the above, I suspect our implementation isn't compliant to the standard. Per IEEE 802.1AX-2014 6.4.1 LACP design elements: c) Active or passive participation in LACP is controlled by LACP_Activity, an administrative control associated with each Aggregation Port, that can take the value Active LACP or Passive LACP. Passive LACP indicates the Aggregation Port’s preference for not transmitting LACPDUs unless its Partner’s control value is Active LACP (i.e., a preference not to speak unless spoken to). Active LACP indicates the Aggregation Port’s preference to participate in the protocol regardless of the Partner’s control value (i.e., a preference to speak regardless). d) Periodic transmission of LACPDUs occurs if the LACP_Activity control of either the Actor or the Partner is Active LACP. These periodic transmissions will occur at either a slow or fast transmission rate depending upon the expressed LACP_Timeout preference (Long Timeout or Short Timeout) of the Partner System. Which, in summary, means that if either end (actor or partner) has LACP_Activity set, both ends must send periodic LACPDUs at the rate specified by their respective partner's LACP_Timeout rate. >To avoid this, we need update ntt to true once received an LACPDU from the >partner, ensuring an immediate reply. With this fix, the link becomes stable >in most cases, except for one specific scenario: > >Actor: lacp_active=off, lacp_rate=slow >Partner: lacp_active=on, lacp_rate=fast > >In this case, the partner expects frequent LACPDUs (every 1 second), but the >actor only responds after receiving an LACPDU, which, in this setup, the >partner sends every 30 seconds due to the actor's lacp_rate=slow. By the time >the actor replies, the partner has already timed out and sent an "expired" >LACPDU. Presuming that I'm correct that we're not implementing 6.4.1 d), above, correctly, then I don't think this is a proper fix, as it kind of band-aids over the problem a bit. Looking at the code, I suspect the problem revolves around the "lacp_active" check in ad_periodic_machine(): static void ad_periodic_machine(struct port *port, struct bond_params *bond_params) { periodic_states_t last_state; /* keep current state machine state to compare later if it was changed */ last_state = port->sm_periodic_state; /* check if port was reinitialized */ if (((port->sm_vars & AD_PORT_BEGIN) || !(port->sm_vars & AD_PORT_LACP_ENABLED) || !port->is_enabled) || (!(port->actor_oper_port_state & LACP_STATE_LACP_ACTIVITY) && !(port->partner_oper.port_state & LACP_STATE_LACP_ACTIVITY)) || !bond_params->lacp_active) { port->sm_periodic_state = AD_NO_PERIODIC; } In the above, because all the tests are chained with ||, the lacp_active test overrides the two correct-looking LACP_STATE_LACP_ACTIVITY tests. It looks like ad_initialize_port() always sets LACP_STATE_LACP_ACTIVITY in the port->actor_oper_port_state, and nothing ever clears it. Thinking out loud, perhaps this could be fixed by a) remove the test of bond_params->lacp_active here, and, b) The lacp_active option setting controls whether LACP_ACTIVITY is set in port->actor_oper_port_state. Thoughts? -J >Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2") >Signed-off-by: Hangbin Liu <liuhangbin@gmail.com> >--- > drivers/net/bonding/bond_3ad.c | 6 ++++++ > 1 file changed, 6 insertions(+) > >diff --git a/drivers/net/bonding/bond_3ad.c b/drivers/net/bonding/bond_3ad.c >index c6807e473ab7..e001d1c8a49b 100644 >--- a/drivers/net/bonding/bond_3ad.c >+++ b/drivers/net/bonding/bond_3ad.c >@@ -666,6 +666,8 @@ static void __update_default_selected(struct port *port) > */ > static void __update_ntt(struct lacpdu *lacpdu, struct port *port) > { >+ struct bonding *bond; >+ > /* validate lacpdu and port */ > if (lacpdu && port) { > /* check if any parameter is different then >@@ -683,6 +685,10 @@ static void __update_ntt(struct lacpdu *lacpdu, struct port *port) > ) { > port->ntt = true; > } >+ >+ bond = __get_bond_by_port(port); >+ if (bond && !bond->params.lacp_active) >+ port->ntt = true; > } > } > >-- >2.46.0 > --- -Jay Vosburgh, jv@jvosburgh.net
On Tue, Jul 15, 2025 at 09:19:49PM -0700, Jay Vosburgh wrote: > Presuming that I'm correct that we're not implementing 6.4.1 d), > above, correctly, then I don't think this is a proper fix, as it kind of > band-aids over the problem a bit. > > Looking at the code, I suspect the problem revolves around the > "lacp_active" check in ad_periodic_machine(): > > static void ad_periodic_machine(struct port *port, struct bond_params *bond_params) > { > periodic_states_t last_state; > > /* keep current state machine state to compare later if it was changed */ > last_state = port->sm_periodic_state; > > /* check if port was reinitialized */ > if (((port->sm_vars & AD_PORT_BEGIN) || !(port->sm_vars & AD_PORT_LACP_ENABLED) || !port->is_enabled) || > (!(port->actor_oper_port_state & LACP_STATE_LACP_ACTIVITY) && !(port->partner_oper.port_state & LACP_STATE_LACP_ACTIVITY)) || > !bond_params->lacp_active) { > port->sm_periodic_state = AD_NO_PERIODIC; > } > > In the above, because all the tests are chained with ||, the > lacp_active test overrides the two correct-looking > LACP_STATE_LACP_ACTIVITY tests. > > It looks like ad_initialize_port() always sets > LACP_STATE_LACP_ACTIVITY in the port->actor_oper_port_state, and nothing > ever clears it. > > Thinking out loud, perhaps this could be fixed by > > a) remove the test of bond_params->lacp_active here, and, > > b) The lacp_active option setting controls whether LACP_ACTIVITY > is set in port->actor_oper_port_state. > > Thoughts? Hi Jay, I did some investigation and testing. In addition to your previous change, we also need to initialize the partner's state to 0 in ad_initialize_port_tmpl(). Otherwise, the check: ``` !(port->partner_oper.port_state & LACP_STATE_LACP_ACTIVITY) ``` in ad_periodic_machine() will fail even when the actor is in passive mode. Also, the line: ``` port->partner_oper.port_state |= LACP_STATE_LACP_ACTIVITY; ``` in ad_rx_machine() should be removed, since we can't assume the partner is in active mode. [1] With these two changes, we can ensure: 1. In passive mode, the actor will not send LACPDU before receiving any LACPDU from the partner. 2. Once it receives the partner’s LACPDU, it will start sending periodic LACPDUs as expected. Do you agree with making these changes? If so, I can post a patch for your review. [1] IEEE 8021AX-2020, Figure 6-14—LACP Receive state diagram, the AD_RX_EXPIRED statue should be ``` Partner_Oper_Port_State.Synchronization = FALSE; Partner_Oper_Port_State.Short_Timeout = TRUE; Actor_Oper_Port_State.Expired = TRUE; LACP_currentWhile = Short_Timeout_Time; ``` Thanks, Hangbin
Hangbin Liu <liuhangbin@gmail.com> wrote: >On Tue, Jul 15, 2025 at 09:19:49PM -0700, Jay Vosburgh wrote: >> Presuming that I'm correct that we're not implementing 6.4.1 d), >> above, correctly, then I don't think this is a proper fix, as it kind of >> band-aids over the problem a bit. >> >> Looking at the code, I suspect the problem revolves around the >> "lacp_active" check in ad_periodic_machine(): >> >> static void ad_periodic_machine(struct port *port, struct bond_params *bond_params) >> { >> periodic_states_t last_state; >> >> /* keep current state machine state to compare later if it was changed */ >> last_state = port->sm_periodic_state; >> >> /* check if port was reinitialized */ >> if (((port->sm_vars & AD_PORT_BEGIN) || !(port->sm_vars & AD_PORT_LACP_ENABLED) || !port->is_enabled) || >> (!(port->actor_oper_port_state & LACP_STATE_LACP_ACTIVITY) && !(port->partner_oper.port_state & LACP_STATE_LACP_ACTIVITY)) || >> !bond_params->lacp_active) { >> port->sm_periodic_state = AD_NO_PERIODIC; >> } >> >> In the above, because all the tests are chained with ||, the >> lacp_active test overrides the two correct-looking >> LACP_STATE_LACP_ACTIVITY tests. >> >> It looks like ad_initialize_port() always sets >> LACP_STATE_LACP_ACTIVITY in the port->actor_oper_port_state, and nothing >> ever clears it. >> >> Thinking out loud, perhaps this could be fixed by >> >> a) remove the test of bond_params->lacp_active here, and, >> >> b) The lacp_active option setting controls whether LACP_ACTIVITY >> is set in port->actor_oper_port_state. >> >> Thoughts? > >Hi Jay, > >I did some investigation and testing. In addition to your previous change, >we also need to initialize the partner's state to 0 in ad_initialize_port_tmpl(). >Otherwise, the check: >``` >!(port->partner_oper.port_state & LACP_STATE_LACP_ACTIVITY) >``` >in ad_periodic_machine() will fail even when the actor is in passive mode. Agreed; the .port_state in the port_params tmpl should just be zero; the magic number 1 there now, which is LACP_STATE_LACP_ACTIVITY, is just wrong. For the actor side, the lacp_active option will set it appropriately, and the partner's will be updated by any LACPDUs that arrive. >Also, the line: >``` >port->partner_oper.port_state |= LACP_STATE_LACP_ACTIVITY; >``` >in ad_rx_machine() should be removed, since we can't assume the partner is in >active mode. [1] Also agreed. >With these two changes, we can ensure: >1. In passive mode, the actor will not send LACPDU before receiving any LACPDU from the partner. >2. Once it receives the partner’s LACPDU, it will start sending periodic LACPDUs as expected. > >Do you agree with making these changes? If so, I can post a patch for your review. Yes, please post a patch. >[1] IEEE 8021AX-2020, Figure 6-14—LACP Receive state diagram, the AD_RX_EXPIRED >statue should be >``` >Partner_Oper_Port_State.Synchronization = FALSE; >Partner_Oper_Port_State.Short_Timeout = TRUE; >Actor_Oper_Port_State.Expired = TRUE; >LACP_currentWhile = Short_Timeout_Time; >``` FWIW, I usually reference the older standards 2008 or 2014, as the 2020 edition changes a lot of things and bonding isn't necessarily conformant to those changes (e.g., many of the state machines are different in large or small ways). Technically, the bonding implementation was written to the pre-802.1AX standard when it was still part of 802.3 (hence the name 802.3ad), clause 43. This particular bit (the EXPIRED state actions) is the same, but, for example, the transition test from EXPIRED to DEFAULTED is different in the 2014 vs 2020 editions, and we need to be careful not to implement the state machines piecemeal from different editions of the standard. -J --- -Jay Vosburgh, jv@jvosburgh.net
On Thu, Jul 24, 2025 at 11:57:53AM +0200, Jay Vosburgh wrote: > FWIW, I usually reference the older standards 2008 or 2014, as > the 2020 edition changes a lot of things and bonding isn't necessarily > conformant to those changes (e.g., many of the state machines are > different in large or small ways). Technically, the bonding > implementation was written to the pre-802.1AX standard when it was still > part of 802.3 (hence the name 802.3ad), clause 43. > > This particular bit (the EXPIRED state actions) is the same, > but, for example, the transition test from EXPIRED to DEFAULTED is > different in the 2014 vs 2020 editions, and we need to be careful not to > implement the state machines piecemeal from different editions of the > standard. Thanks for this info. I will download 2014 version and recheck my changes. Cheers Hangbin
On Tue, Jul 15, 2025 at 09:19:49PM -0700, Jay Vosburgh wrote: > Hangbin Liu <liuhangbin@gmail.com> wrote: > > >When lacp_active is set to off, the bond operates in passive mode, meaning it > >will only "speak when spoken to." However, the current kernel implementation > >only sends an LACPDU in response when the partner's state changes. > > > >In this situation, once LACP negotiation succeeds, the actor stops sending > >LACPDUs until the partner times out and sends an "expired" LACPDU. > >This leads to endless LACP state flapping. > > From the above, I suspect our implementation isn't compliant to > the standard. Per IEEE 802.1AX-2014 6.4.1 LACP design elements: > > c) Active or passive participation in LACP is controlled by > LACP_Activity, an administrative control associated with each > Aggregation Port, that can take the value Active LACP or Passive > LACP. Passive LACP indicates the Aggregation Port’s preference > for not transmitting LACPDUs unless its Partner’s control value > is Active LACP (i.e., a preference not to speak unless spoken > to). Active LACP indicates the Aggregation Port’s preference to OK, so this means the passive side should start sending LACPDUs when receive passive actor's LACPDUs, with the slow/fast rate based on partner's rate? Hmm, then when we should stop sending LACPDUs? After port->sm_mux_state == AD_MUX_DETACHED ? > participate in the protocol regardless of the Partner’s control > value (i.e., a preference to speak regardless). > > d) Periodic transmission of LACPDUs occurs if the LACP_Activity > control of either the Actor or the Partner is Active LACP. These > periodic transmissions will occur at either a slow or fast > transmission rate depending upon the expressed LACP_Timeout > preference (Long Timeout or Short Timeout) of the Partner > System. > > Which, in summary, means that if either end (actor or partner) > has LACP_Activity set, both ends must send periodic LACPDUs at the rate > specified by their respective partner's LACP_Timeout rate. > > >To avoid this, we need update ntt to true once received an LACPDU from the > >partner, ensuring an immediate reply. With this fix, the link becomes stable > >in most cases, except for one specific scenario: > > > >Actor: lacp_active=off, lacp_rate=slow > >Partner: lacp_active=on, lacp_rate=fast > > > >In this case, the partner expects frequent LACPDUs (every 1 second), but the > >actor only responds after receiving an LACPDU, which, in this setup, the > >partner sends every 30 seconds due to the actor's lacp_rate=slow. By the time > >the actor replies, the partner has already timed out and sent an "expired" > >LACPDU. > > Presuming that I'm correct that we're not implementing 6.4.1 d), > above, correctly, then I don't think this is a proper fix, as it kind of > band-aids over the problem a bit. > > Looking at the code, I suspect the problem revolves around the > "lacp_active" check in ad_periodic_machine(): > > static void ad_periodic_machine(struct port *port, struct bond_params *bond_params) > { > periodic_states_t last_state; > > /* keep current state machine state to compare later if it was changed */ > last_state = port->sm_periodic_state; > > /* check if port was reinitialized */ > if (((port->sm_vars & AD_PORT_BEGIN) || !(port->sm_vars & AD_PORT_LACP_ENABLED) || !port->is_enabled) || > (!(port->actor_oper_port_state & LACP_STATE_LACP_ACTIVITY) && !(port->partner_oper.port_state & LACP_STATE_LACP_ACTIVITY)) || > !bond_params->lacp_active) { > port->sm_periodic_state = AD_NO_PERIODIC; > } > > In the above, because all the tests are chained with ||, the > lacp_active test overrides the two correct-looking > LACP_STATE_LACP_ACTIVITY tests. > > It looks like ad_initialize_port() always sets > LACP_STATE_LACP_ACTIVITY in the port->actor_oper_port_state, and nothing > ever clears it. > > Thinking out loud, perhaps this could be fixed by > > a) remove the test of bond_params->lacp_active here, and, > > b) The lacp_active option setting controls whether LACP_ACTIVITY > is set in port->actor_oper_port_state. > > Thoughts? As the upper question. When should we stop sending the LACPDUs? Thanks Hangbin
Hangbin Liu <liuhangbin@gmail.com> wrote: >On Tue, Jul 15, 2025 at 09:19:49PM -0700, Jay Vosburgh wrote: >> Hangbin Liu <liuhangbin@gmail.com> wrote: >> >> >When lacp_active is set to off, the bond operates in passive mode, meaning it >> >will only "speak when spoken to." However, the current kernel implementation >> >only sends an LACPDU in response when the partner's state changes. >> > >> >In this situation, once LACP negotiation succeeds, the actor stops sending >> >LACPDUs until the partner times out and sends an "expired" LACPDU. >> >This leads to endless LACP state flapping. >> >> From the above, I suspect our implementation isn't compliant to >> the standard. Per IEEE 802.1AX-2014 6.4.1 LACP design elements: >> >> c) Active or passive participation in LACP is controlled by >> LACP_Activity, an administrative control associated with each >> Aggregation Port, that can take the value Active LACP or Passive >> LACP. Passive LACP indicates the Aggregation Port’s preference >> for not transmitting LACPDUs unless its Partner’s control value >> is Active LACP (i.e., a preference not to speak unless spoken >> to). Active LACP indicates the Aggregation Port’s preference to > >OK, so this means the passive side should start sending LACPDUs when receive >passive actor's LACPDUs, with the slow/fast rate based on partner's rate? Did you mean "receive active actor's LACPDUs"? Regardless, the standard requires both sides to initiate periodic LACPDU transmission if either or both enable LACP_Activity in their LACPDUs. So, if a received LACPDU from the partner has LACP_Activity set, then, yes, we would enable periodic LACPDU transmission, regardless of our local setting of "lacp_active" / LACP_Activity. >Hmm, then when we should stop sending LACPDUs? After >port->sm_mux_state == AD_MUX_DETACHED ? We stop sending when the criteria for NO_PERIODIC in the periodic state machine is met (IEEE 802.1AX-2014 6.4.13, Figure 6-19). Practically speaking, this happens when a BEGIN event occurs, due to a port being reinitialized. The ad_mux_machine() will set the mux state to AD_MUX_DETACHED when BEGIN occurs, so I don't think we need to test for DETACHED explicitly. The NO_PERIODIC check is the first "if" block in ad_periodic_machine() that I referenced below. The code currently tests all of the criteria from Figure 6-19, but adds a test of "!lacp_active", which is why I suspect that removing that bit and managing the lacp_active option via the LACP_Activity in the actor port state would do the right thing. -J >> participate in the protocol regardless of the Partner’s control >> value (i.e., a preference to speak regardless). >> >> d) Periodic transmission of LACPDUs occurs if the LACP_Activity >> control of either the Actor or the Partner is Active LACP. These >> periodic transmissions will occur at either a slow or fast >> transmission rate depending upon the expressed LACP_Timeout >> preference (Long Timeout or Short Timeout) of the Partner >> System. >> >> Which, in summary, means that if either end (actor or partner) >> has LACP_Activity set, both ends must send periodic LACPDUs at the rate >> specified by their respective partner's LACP_Timeout rate. >> >> >To avoid this, we need update ntt to true once received an LACPDU from the >> >partner, ensuring an immediate reply. With this fix, the link becomes stable >> >in most cases, except for one specific scenario: >> > >> >Actor: lacp_active=off, lacp_rate=slow >> >Partner: lacp_active=on, lacp_rate=fast >> > >> >In this case, the partner expects frequent LACPDUs (every 1 second), but the >> >actor only responds after receiving an LACPDU, which, in this setup, the >> >partner sends every 30 seconds due to the actor's lacp_rate=slow. By the time >> >the actor replies, the partner has already timed out and sent an "expired" >> >LACPDU. >> >> Presuming that I'm correct that we're not implementing 6.4.1 d), >> above, correctly, then I don't think this is a proper fix, as it kind of >> band-aids over the problem a bit. >> >> Looking at the code, I suspect the problem revolves around the >> "lacp_active" check in ad_periodic_machine(): >> >> static void ad_periodic_machine(struct port *port, struct bond_params *bond_params) >> { >> periodic_states_t last_state; >> >> /* keep current state machine state to compare later if it was changed */ >> last_state = port->sm_periodic_state; >> >> /* check if port was reinitialized */ >> if (((port->sm_vars & AD_PORT_BEGIN) || !(port->sm_vars & AD_PORT_LACP_ENABLED) || !port->is_enabled) || >> (!(port->actor_oper_port_state & LACP_STATE_LACP_ACTIVITY) && !(port->partner_oper.port_state & LACP_STATE_LACP_ACTIVITY)) || >> !bond_params->lacp_active) { >> port->sm_periodic_state = AD_NO_PERIODIC; >> } >> >> In the above, because all the tests are chained with ||, the >> lacp_active test overrides the two correct-looking >> LACP_STATE_LACP_ACTIVITY tests. >> >> It looks like ad_initialize_port() always sets >> LACP_STATE_LACP_ACTIVITY in the port->actor_oper_port_state, and nothing >> ever clears it. >> >> Thinking out loud, perhaps this could be fixed by >> >> a) remove the test of bond_params->lacp_active here, and, >> >> b) The lacp_active option setting controls whether LACP_ACTIVITY >> is set in port->actor_oper_port_state. >> >> Thoughts? > >As the upper question. When should we stop sending the LACPDUs? > >Thanks >Hangbin --- -Jay Vosburgh, jv@jvosburgh.net
© 2016 - 2025 Red Hat, Inc.