This patchset aims at fixing most issues observed while running the
vlan_unaware_bridge, vlan_aware_bridge and local_termination selftests.

Most tests succeed with these patches on BCM53115, connected to a
BCM6368.

It took me a while to figure out that a lot of tests will fail if all
ports have the same MAC address, as the switches drop any frames with
DA == SA. Luckily BCM63XX boards often have enough MACs allocated for
all ports, so I just needed to assign them.

The still failing tests are:

FDB learning, both vlan aware aware and unaware:

This is expected, as b53 currently does not implement changing the
ageing time, and both the bridge code and DSA ignore that, so the
learned entries don't age out as expected.

ping and ping6 in vlan unaware:

These fail because of the now fixed learning, the switch trying to
forward packet ingressing on one of the standalone ports to the learned
port of the mac address when the packets ingressed on the bridged port.

The port VLAN masks only prevent forwarding to other ports, but the ARL
lookup will still happen, and the packet gets dropped because the port
isn't allowed to forward there.

I have a fix/workaround for that, but as it is a bit more controversial
and makes use of an unrelated feature, I decided to hold off from that
and post it later.

This wasn't noticed so far, because learning was never working in VLAN
unaware mode, so the traffic was always broadcast (which sidesteps the
issue).

Finally some of the multicast tests from local_termination fail, where
the reception worked except it shouldn't. This doesn't seem to me as a
super serious issue, so I didn't attempt to debug/fix these yet.

I'm not super confident I didn't break sf2 along the way, but I did
compile test and tried to find ways it cause issues (I failed to find
any). I hope Florian will tell me.

Jonas Gorski (11):
  net: dsa: b53: allow leaky reserved multicast
  net: dsa: b53: keep CPU port always tagged again
  net: dsa: b53: fix clearing PVID of a port
  net: dsa: b53: fix flushing old pvid VLAN on pvid change
  net: dsa: b53: fix VLAN ID for untagged vlan on bridge leave
  net: dsa: b53: always rejoin default untagged VLAN on bridge leave
  net: dsa: b53: do not allow to configure VLAN 0
  net: dsa: b53: do not program vlans when vlan filtering is off
  net: dsa: b53: fix toggling vlan_filtering
  net: dsa: b53: fix learning on VLAN unaware bridges
  net: dsa: b53: do not set learning and unicast/multicast on up

 drivers/net/dsa/b53/b53_common.c | 207 ++++++++++++++++++++++---------
 drivers/net/dsa/b53/b53_priv.h   |   3 +
 drivers/net/dsa/bcm_sf2.c        |   1 +
 3 files changed, 154 insertions(+), 57 deletions(-)

-- 
2.43.0

On 4/29/2025 10:16 PM, Jonas Gorski wrote:
> This patchset aims at fixing most issues observed while running the
> vlan_unaware_bridge, vlan_aware_bridge and local_termination selftests.
> 
> Most tests succeed with these patches on BCM53115, connected to a
> BCM6368.
> 
> It took me a while to figure out that a lot of tests will fail if all
> ports have the same MAC address, as the switches drop any frames with
> DA == SA. Luckily BCM63XX boards often have enough MACs allocated for
> all ports, so I just needed to assign them.
> 
> The still failing tests are:
> 
> FDB learning, both vlan aware aware and unaware:
> 
> This is expected, as b53 currently does not implement changing the
> ageing time, and both the bridge code and DSA ignore that, so the
> learned entries don't age out as expected.
> 
> ping and ping6 in vlan unaware:
> 
> These fail because of the now fixed learning, the switch trying to
> forward packet ingressing on one of the standalone ports to the learned
> port of the mac address when the packets ingressed on the bridged port.
> 
> The port VLAN masks only prevent forwarding to other ports, but the ARL
> lookup will still happen, and the packet gets dropped because the port
> isn't allowed to forward there.
> 
> I have a fix/workaround for that, but as it is a bit more controversial
> and makes use of an unrelated feature, I decided to hold off from that
> and post it later.
> 
> This wasn't noticed so far, because learning was never working in VLAN
> unaware mode, so the traffic was always broadcast (which sidesteps the
> issue).
> 
> Finally some of the multicast tests from local_termination fail, where
> the reception worked except it shouldn't. This doesn't seem to me as a
> super serious issue, so I didn't attempt to debug/fix these yet.
> 
> I'm not super confident I didn't break sf2 along the way, but I did
> compile test and tried to find ways it cause issues (I failed to find
> any). I hope Florian will tell me.

For this series, using a combination of VLAN aware and unaware bridge, 
standalone ports with 802.1q uppers on top, thanks Jonas!

Tested-by: Florian Fainelli <florian.fainelli@broadcom.com>
Reviewed-by: Florian Fainelli <florian.fainelli@broadcom.com>
-- 
Florian

On 4/29/2025 10:16 PM, Jonas Gorski wrote:
> This patchset aims at fixing most issues observed while running the
> vlan_unaware_bridge, vlan_aware_bridge and local_termination selftests.
> 
> Most tests succeed with these patches on BCM53115, connected to a
> BCM6368.
> 
> It took me a while to figure out that a lot of tests will fail if all
> ports have the same MAC address, as the switches drop any frames with
> DA == SA. Luckily BCM63XX boards often have enough MACs allocated for
> all ports, so I just needed to assign them.
> 
> The still failing tests are:
> 
> FDB learning, both vlan aware aware and unaware:
> 
> This is expected, as b53 currently does not implement changing the
> ageing time, and both the bridge code and DSA ignore that, so the
> learned entries don't age out as expected.
> 
> ping and ping6 in vlan unaware:
> 
> These fail because of the now fixed learning, the switch trying to
> forward packet ingressing on one of the standalone ports to the learned
> port of the mac address when the packets ingressed on the bridged port.

Sorry not quite getting that part, can you expand a bit more?

> 
> The port VLAN masks only prevent forwarding to other ports, but the ARL
> lookup will still happen, and the packet gets dropped because the port
> isn't allowed to forward there.

OK.

> 
> I have a fix/workaround for that, but as it is a bit more controversial
> and makes use of an unrelated feature, I decided to hold off from that
> and post it later.

Can you expand on the fix/workaround you have?

> 
> This wasn't noticed so far, because learning was never working in VLAN
> unaware mode, so the traffic was always broadcast (which sidesteps the
> issue).
> 
> Finally some of the multicast tests from local_termination fail, where
> the reception worked except it shouldn't. This doesn't seem to me as a
> super serious issue, so I didn't attempt to debug/fix these yet.
> 
> I'm not super confident I didn't break sf2 along the way, but I did
> compile test and tried to find ways it cause issues (I failed to find
> any). I hope Florian will tell me.

I am currently out of the office but intend to test your patch series at 
some point in the next few days. Let's gather some review feedback in 
the meantime, thanks for submitting those fixes!
-- 
Florian

On Wed, Apr 30, 2025 at 10:07 AM Florian Fainelli
<florian.fainelli@broadcom.com> wrote:
>
>
>
> On 4/29/2025 10:16 PM, Jonas Gorski wrote:
> > This patchset aims at fixing most issues observed while running the
> > vlan_unaware_bridge, vlan_aware_bridge and local_termination selftests.
> >
> > Most tests succeed with these patches on BCM53115, connected to a
> > BCM6368.
> >
> > It took me a while to figure out that a lot of tests will fail if all
> > ports have the same MAC address, as the switches drop any frames with
> > DA == SA. Luckily BCM63XX boards often have enough MACs allocated for
> > all ports, so I just needed to assign them.
> >
> > The still failing tests are:
> >
> > FDB learning, both vlan aware aware and unaware:
> >
> > This is expected, as b53 currently does not implement changing the
> > ageing time, and both the bridge code and DSA ignore that, so the
> > learned entries don't age out as expected.
> >
> > ping and ping6 in vlan unaware:
> >
> > These fail because of the now fixed learning, the switch trying to
> > forward packet ingressing on one of the standalone ports to the learned
> > port of the mac address when the packets ingressed on the bridged port.
>
> Sorry not quite getting that part, can you expand a bit more?

The ping test uses four network ports, where two pairs are linked via
a network cable. So the setup is

sw1p1 <- cable -> sw1p2 <- bridge -> sw1p3 <- cable ->sw1p4

And it tries to ping from sw1p1 to sw1p4.

In the vlan_aware test, the bridge uses VLAN 1 pvid untagged, so it
learns in VLAN 1, while the standalone ports use VLAN 0. Different
FIDs, so no issue.

In the vlan_unaware test, untagged traffic uses VLAN 0 everywhere, so
the following happens:

- switch learns swp1p's MAC at sw1p2
- switch learns sw1p4's MAC at sw1p3

when sw1p1 sends a unicast to sw1p4' mac it egresses on swp1p3 and
then ingresses on sw1p4 again. The switch see's sw1p2's MAC as DA and
then ARL lookup says "sw1p3", but the port VLAN mask disallows sending
from sw1p4 to sw1p3, so it gets dropped.

Without learning, all packets are flooded, so connectivity works, as
the standalone ports are always part of the flood masks.

> > The port VLAN masks only prevent forwarding to other ports, but the ARL
> > lookup will still happen, and the packet gets dropped because the port
> > isn't allowed to forward there.
>
> OK.
>
> >
> > I have a fix/workaround for that, but as it is a bit more controversial
> > and makes use of an unrelated feature, I decided to hold off from that
> > and post it later.
>
> Can you expand on the fix/workaround you have?

It's setting EAP mode to simplified on standalone ports, where it
redirects all frames to the CPU port where there is no matching ARL
entry for that SA and port. That should work on everything semi recent
(including BCM63XX), and should work regardless of VLAN. It might
cause more traffic than expected to be sent to the switch, as I'm not
sure if multicast filtering would still work (not that I'm sure that
it currently works lol).

At first I moved standalone ports to VID 4095 for untagged traffic,
but that only fixed the issue for untagged traffic, and you would have
had the same issue again when using VLAN uppers. And VLAN uppers have
the same issue on vlan aware bridges, so the above would be a more
complete workaround.

> > This wasn't noticed so far, because learning was never working in VLAN
> > unaware mode, so the traffic was always broadcast (which sidesteps the
> > issue).
> >
> > Finally some of the multicast tests from local_termination fail, where
> > the reception worked except it shouldn't. This doesn't seem to me as a
> > super serious issue, so I didn't attempt to debug/fix these yet.
> >
> > I'm not super confident I didn't break sf2 along the way, but I did
> > compile test and tried to find ways it cause issues (I failed to find
> > any). I hope Florian will tell me.
>
> I am currently out of the office but intend to test your patch series at
> some point in the next few days. Let's gather some review feedback in
> the meantime, thanks for submitting those fixes!

If you are awake at this hour I guess your are back "home" ;-)

Sure thing, take your time! All I wanted to implement is MST support,
but then I noticed some things not working as expected, and then I
became aware of the selftests, and then I suddenly had accumulated a
lot of fixes trying to make everything say "Okay" (and sometimes
wondering why other stuff broke when I fixed things, like the learning
in unaware mode).

Best regards,
Jonas

/ unrelated to patches /

On Wed, Apr 30, 2025 at 10:43:40AM +0200, Jonas Gorski wrote:
> > > I have a fix/workaround for that, but as it is a bit more controversial
> > > and makes use of an unrelated feature, I decided to hold off from that
> > > and post it later.
> >
> > Can you expand on the fix/workaround you have?
> 
> It's setting EAP mode to simplified on standalone ports, where it
> redirects all frames to the CPU port where there is no matching ARL
> entry for that SA and port. That should work on everything semi recent
> (including BCM63XX), and should work regardless of VLAN. It might
> cause more traffic than expected to be sent to the switch, as I'm not
> sure if multicast filtering would still work (not that I'm sure that
> it currently works lol).
> 
> At first I moved standalone ports to VID 4095 for untagged traffic,
> but that only fixed the issue for untagged traffic, and you would have
> had the same issue again when using VLAN uppers. And VLAN uppers have
> the same issue on vlan aware bridges, so the above would be a more
> complete workaround.

I don't understand the logic, can you explain "you would have had the
same issue again when using VLAN uppers"? The original issue, as you
presented it, is with bridges with vlan_filtering=0, and does not exist
with vlan_filtering=1 bridges. In the problematic mode, VLAN uppers are
not committed to hardware RX filters. And bridges with mixed
vlan_filtering values are not permitted by dsa_port_can_apply_vlan_filtering().
So I don't see how making VID 4095 be the PVID of just standalone ports
(leaving VLAN-unaware bridge ports with a different VID) would not be
sufficient for the presented problem.

That being said, trapping to CPU all packets on standalone ports is not
uncontroversial, as long as it works correctly for the hardware
controlled by this driver. You seem concerned about losing RX filtering,
but if you look at dsa_switch_supports_uc_filtering() and
dsa_switch_supports_mc_filtering() you'll see b53 never had it - it
depends, among other things, on ds->fdb_isolation == true and
ds->vlan_filtering_is_global == false. Here you're working on improving
the fdb_isolation requirements, but there is still no support in the
core for devices where VLAN filtering is a global setting.

On Tue, May 6, 2025 at 3:42 PM Vladimir Oltean <olteanv@gmail.com> wrote:
>
> / unrelated to patches /
>
> On Wed, Apr 30, 2025 at 10:43:40AM +0200, Jonas Gorski wrote:
> > > > I have a fix/workaround for that, but as it is a bit more controversial
> > > > and makes use of an unrelated feature, I decided to hold off from that
> > > > and post it later.
> > >
> > > Can you expand on the fix/workaround you have?
> >
> > It's setting EAP mode to simplified on standalone ports, where it
> > redirects all frames to the CPU port where there is no matching ARL
> > entry for that SA and port. That should work on everything semi recent
> > (including BCM63XX), and should work regardless of VLAN. It might
> > cause more traffic than expected to be sent to the switch, as I'm not
> > sure if multicast filtering would still work (not that I'm sure that
> > it currently works lol).
> >
> > At first I moved standalone ports to VID 4095 for untagged traffic,
> > but that only fixed the issue for untagged traffic, and you would have
> > had the same issue again when using VLAN uppers. And VLAN uppers have
> > the same issue on vlan aware bridges, so the above would be a more
> > complete workaround.
>
> I don't understand the logic, can you explain "you would have had the
> same issue again when using VLAN uppers"? The original issue, as you
> presented it, is with bridges with vlan_filtering=0, and does not exist
> with vlan_filtering=1 bridges. In the problematic mode, VLAN uppers are
> not committed to hardware RX filters. And bridges with mixed
> vlan_filtering values are not permitted by dsa_port_can_apply_vlan_filtering().
> So I don't see how making VID 4095 be the PVID of just standalone ports
> (leaving VLAN-unaware bridge ports with a different VID) would not be
> sufficient for the presented problem.

The issue isn't the vlan filtering, it's the (missing) FDB isolation
on the ASIC.

In general if a MAC is learned on one of the bridged ports, the ASIC
will try to forward any traffic ingressing for that MAC to the port
according to the internal FDB regardless if the source port is bridged
or not. The private VLAN masks used to isolate ports is only applied
*after* the forwarding lookup was done (this is how the hardware
works), and will then cause traffic to be dropped on standalone ports.

So for vlan_aware=0, the bridge learns on the FDB for VID 0, which is
also used for untagged traffic on stand alone ports, so the issue
happens.

For vlan_aware=1, untagged traffic is assigned to a VLAN != 0 on the
bridge, so the FDBs are different. Therefore no issue.

But once using VLAN uppers, the FDB will be the same again on the
bridged ports and tagged traffic on the standalone ports, so it
happens again. And this would be true regardless of vlan_aware or not.

Therefore using VID 4095 would only avoid the issue for vlan_aware=0
and untagged, but not the VLAN upper scenario.

Unfortunately there is no bit to disable FDB lookups for standalone
ports, only learning (at least I haven't found one).

> That being said, trapping to CPU all packets on standalone ports is not
> uncontroversial, as long as it works correctly for the hardware
> controlled by this driver. You seem concerned about losing RX filtering,
> but if you look at dsa_switch_supports_uc_filtering() and
> dsa_switch_supports_mc_filtering() you'll see b53 never had it - it
> depends, among other things, on ds->fdb_isolation == true and
> ds->vlan_filtering_is_global == false. Here you're working on improving
> the fdb_isolation requirements, but there is still no support in the
> core for devices where VLAN filtering is a global setting.

Aha, so I don't need to care about this, since fdb_isolation is not
supported and filtering is always global. Good to know.

I'm mostly concerned if using the security feature for that has any
side effects I'm not aware of (e.g. dropping of some traffic), or if
it would prevent using it for its intended use, i.e. locked ports and
MAB. I don't trust it yet, feels a bit too easy of a solution ;-).
Will need to double/triple check (and read the docs).

Also it's not supported by all b53 supported ASICs, so maybe there is
a solution that works for all (but most probably there is not).

Best regards,
Jonas

On 5/6/2025 4:27 PM, Jonas Gorski wrote:
> On Tue, May 6, 2025 at 3:42 PM Vladimir Oltean <olteanv@gmail.com> wrote:
>>
>> / unrelated to patches /
>>
>> On Wed, Apr 30, 2025 at 10:43:40AM +0200, Jonas Gorski wrote:
>>>>> I have a fix/workaround for that, but as it is a bit more controversial
>>>>> and makes use of an unrelated feature, I decided to hold off from that
>>>>> and post it later.
>>>>
>>>> Can you expand on the fix/workaround you have?
>>>
>>> It's setting EAP mode to simplified on standalone ports, where it
>>> redirects all frames to the CPU port where there is no matching ARL
>>> entry for that SA and port. That should work on everything semi recent
>>> (including BCM63XX), and should work regardless of VLAN. It might
>>> cause more traffic than expected to be sent to the switch, as I'm not
>>> sure if multicast filtering would still work (not that I'm sure that
>>> it currently works lol).
>>>
>>> At first I moved standalone ports to VID 4095 for untagged traffic,
>>> but that only fixed the issue for untagged traffic, and you would have
>>> had the same issue again when using VLAN uppers. And VLAN uppers have
>>> the same issue on vlan aware bridges, so the above would be a more
>>> complete workaround.
>>
>> I don't understand the logic, can you explain "you would have had the
>> same issue again when using VLAN uppers"? The original issue, as you
>> presented it, is with bridges with vlan_filtering=0, and does not exist
>> with vlan_filtering=1 bridges. In the problematic mode, VLAN uppers are
>> not committed to hardware RX filters. And bridges with mixed
>> vlan_filtering values are not permitted by dsa_port_can_apply_vlan_filtering().
>> So I don't see how making VID 4095 be the PVID of just standalone ports
>> (leaving VLAN-unaware bridge ports with a different VID) would not be
>> sufficient for the presented problem.
> 
> The issue isn't the vlan filtering, it's the (missing) FDB isolation
> on the ASIC.

Could not we just use double tagging to overcome that limitation?
-- 
Florian

On Tue, May 6, 2025 at 9:03 PM Florian Fainelli <f.fainelli@gmail.com> wrote:
>
>
>
> On 5/6/2025 4:27 PM, Jonas Gorski wrote:
> > On Tue, May 6, 2025 at 3:42 PM Vladimir Oltean <olteanv@gmail.com> wrote:
> >>
> >> / unrelated to patches /
> >>
> >> On Wed, Apr 30, 2025 at 10:43:40AM +0200, Jonas Gorski wrote:
> >>>>> I have a fix/workaround for that, but as it is a bit more controversial
> >>>>> and makes use of an unrelated feature, I decided to hold off from that
> >>>>> and post it later.
> >>>>
> >>>> Can you expand on the fix/workaround you have?
> >>>
> >>> It's setting EAP mode to simplified on standalone ports, where it
> >>> redirects all frames to the CPU port where there is no matching ARL
> >>> entry for that SA and port. That should work on everything semi recent
> >>> (including BCM63XX), and should work regardless of VLAN. It might
> >>> cause more traffic than expected to be sent to the switch, as I'm not
> >>> sure if multicast filtering would still work (not that I'm sure that
> >>> it currently works lol).
> >>>
> >>> At first I moved standalone ports to VID 4095 for untagged traffic,
> >>> but that only fixed the issue for untagged traffic, and you would have
> >>> had the same issue again when using VLAN uppers. And VLAN uppers have
> >>> the same issue on vlan aware bridges, so the above would be a more
> >>> complete workaround.
> >>
> >> I don't understand the logic, can you explain "you would have had the
> >> same issue again when using VLAN uppers"? The original issue, as you
> >> presented it, is with bridges with vlan_filtering=0, and does not exist
> >> with vlan_filtering=1 bridges. In the problematic mode, VLAN uppers are
> >> not committed to hardware RX filters. And bridges with mixed
> >> vlan_filtering values are not permitted by dsa_port_can_apply_vlan_filtering().
> >> So I don't see how making VID 4095 be the PVID of just standalone ports
> >> (leaving VLAN-unaware bridge ports with a different VID) would not be
> >> sufficient for the presented problem.
> >
> > The issue isn't the vlan filtering, it's the (missing) FDB isolation
> > on the ASIC.
>
> Could not we just use double tagging to overcome that limitation?

Wouldn't that break VLAN filtering on a vlan aware bridge? AFAICT
double tagging mode is global, the VLAN table is then used for
customer (port) assignment, so you can't filter on the inner/802.1Q
tag anymore. Also learning would then essentially become SVL IIUCT.
Also I think there aren't switches that support double tagging, but
don't support EAP. EAP mode might be the easier way. Assuming there
isn't a gotcha I have overlooked.

Jonas

On 5/6/2025 9:48 PM, Jonas Gorski wrote:
> On Tue, May 6, 2025 at 9:03 PM Florian Fainelli <f.fainelli@gmail.com> wrote:
>>
>>
>>
>> On 5/6/2025 4:27 PM, Jonas Gorski wrote:
>>> On Tue, May 6, 2025 at 3:42 PM Vladimir Oltean <olteanv@gmail.com> wrote:
>>>>
>>>> / unrelated to patches /
>>>>
>>>> On Wed, Apr 30, 2025 at 10:43:40AM +0200, Jonas Gorski wrote:
>>>>>>> I have a fix/workaround for that, but as it is a bit more controversial
>>>>>>> and makes use of an unrelated feature, I decided to hold off from that
>>>>>>> and post it later.
>>>>>>
>>>>>> Can you expand on the fix/workaround you have?
>>>>>
>>>>> It's setting EAP mode to simplified on standalone ports, where it
>>>>> redirects all frames to the CPU port where there is no matching ARL
>>>>> entry for that SA and port. That should work on everything semi recent
>>>>> (including BCM63XX), and should work regardless of VLAN. It might
>>>>> cause more traffic than expected to be sent to the switch, as I'm not
>>>>> sure if multicast filtering would still work (not that I'm sure that
>>>>> it currently works lol).
>>>>>
>>>>> At first I moved standalone ports to VID 4095 for untagged traffic,
>>>>> but that only fixed the issue for untagged traffic, and you would have
>>>>> had the same issue again when using VLAN uppers. And VLAN uppers have
>>>>> the same issue on vlan aware bridges, so the above would be a more
>>>>> complete workaround.
>>>>
>>>> I don't understand the logic, can you explain "you would have had the
>>>> same issue again when using VLAN uppers"? The original issue, as you
>>>> presented it, is with bridges with vlan_filtering=0, and does not exist
>>>> with vlan_filtering=1 bridges. In the problematic mode, VLAN uppers are
>>>> not committed to hardware RX filters. And bridges with mixed
>>>> vlan_filtering values are not permitted by dsa_port_can_apply_vlan_filtering().
>>>> So I don't see how making VID 4095 be the PVID of just standalone ports
>>>> (leaving VLAN-unaware bridge ports with a different VID) would not be
>>>> sufficient for the presented problem.
>>>
>>> The issue isn't the vlan filtering, it's the (missing) FDB isolation
>>> on the ASIC.
>>
>> Could not we just use double tagging to overcome that limitation?
> 
> Wouldn't that break VLAN filtering on a vlan aware bridge? AFAICT
> double tagging mode is global, the VLAN table is then used for
> customer (port) assignment, so you can't filter on the inner/802.1Q
> tag anymore. Also learning would then essentially become SVL IIUCT.
> Also I think there aren't switches that support double tagging, but
> don't support EAP. EAP mode might be the easier way. Assuming there
> isn't a gotcha I have overlooked.

If EAP works, sure that seems like the way to go then.
-- 
Florian