This patch set completes the picture described by
'[RFC,devicetree] of: property: mark "interrupts" as optional for fw_devlink'
https://patchwork.kernel.org/project/netdevbpf/patch/20220513201243.2381133-1-vladimir.oltean@nxp.com/

I've CCed non-networking maintainers just in case they want to gain a
better understanding. If not, apologies and please ignore the rest.



My use case is to migrate a PHY driver from poll mode to interrupt mode
without breaking compatibility between new device trees and old kernels
which did not have a driver for that IRQ parent, and therefore (for
things to work) did not even have that interrupt listed in the "vintage
correct" DT blobs. Note that current kernels as of today are also
"old kernels" in this description.

Creating some degree of compatibility has multiple components.

1. A PHY driver must eventually give up waiting for an IRQ provider,
   since the dependency is optional and it can fall back to poll mode.
   This is currently supported thanks to commit 74befa447e68 ("net:
   mdio: don't defer probe forever if PHY IRQ provider is missing").

2. Before it finally gives up, the PHY driver has a transient phase of
   returning -EPROBE_DEFER. That transient phase causes some breakage
   which is handled by this patch set, details below.

3. PHY device probing and Ethernet controller finding it and connecting
   to it are async events. When both happen during probing, the problem
   is that finding the PHY fails if the PHY defers probe, which results
   in a missing PHY rather than waiting for it. Unfortunately there is
   no universal way to address this problem, because the majority of
   Ethernet drivers do not connect to the PHY during probe. So the
   problem is fixed only for the driver that is of interest to me in
   this context, DSA, and with special API exported by phylink
   specifically for this purpose, to limit the impact on other drivers.

Note that drivers that connect to the PHY at ndo_open are superficially
"fixed" by the patch at step 1 alone, and therefore don't need the
mechanism introduced in phylink here. This is because of the larger span
of time between PHY probe and opening the network interface (typically
initiated by user space). But this is the catch, nfsroot and other
in-kernel networking users can also open the net device, and this will
still expose the EPROBE_DEFER as a hard error for this second kind of
drivers. I don't know how to fix that. From this POV, it's better to do
what DSA does (connect to the PHY on probe).

Vladimir Oltean (2):
  net: phylink: allow PHY driver to defer probe when connecting via OF
    node
  net: dsa: wait for PHY to defer probe

 drivers/net/phy/phylink.c | 73 ++++++++++++++++++++++++++++++---------
 include/linux/phylink.h   |  2 ++
 net/dsa/dsa2.c            |  2 ++
 net/dsa/port.c            |  6 ++--
 net/dsa/slave.c           | 10 +++---
 5 files changed, 70 insertions(+), 23 deletions(-)

-- 
2.25.1

On Sat, May 14, 2022 at 02:36:38AM +0300, Vladimir Oltean wrote:
> This patch set completes the picture described by
> '[RFC,devicetree] of: property: mark "interrupts" as optional for fw_devlink'
> https://patchwork.kernel.org/project/netdevbpf/patch/20220513201243.2381133-1-vladimir.oltean@nxp.com/
> 
> I've CCed non-networking maintainers just in case they want to gain a
> better understanding. If not, apologies and please ignore the rest.
> 
> 
> 
> My use case is to migrate a PHY driver from poll mode to interrupt mode
> without breaking compatibility between new device trees and old kernels
> which did not have a driver for that IRQ parent, and therefore (for
> things to work) did not even have that interrupt listed in the "vintage
> correct" DT blobs. Note that current kernels as of today are also
> "old kernels" in this description.
> 
> Creating some degree of compatibility has multiple components.
> 
> 1. A PHY driver must eventually give up waiting for an IRQ provider,
>    since the dependency is optional and it can fall back to poll mode.
>    This is currently supported thanks to commit 74befa447e68 ("net:
>    mdio: don't defer probe forever if PHY IRQ provider is missing").
> 
> 2. Before it finally gives up, the PHY driver has a transient phase of
>    returning -EPROBE_DEFER. That transient phase causes some breakage
>    which is handled by this patch set, details below.
> 
> 3. PHY device probing and Ethernet controller finding it and connecting
>    to it are async events. When both happen during probing, the problem
>    is that finding the PHY fails if the PHY defers probe, which results
>    in a missing PHY rather than waiting for it. Unfortunately there is
>    no universal way to address this problem, because the majority of
>    Ethernet drivers do not connect to the PHY during probe. So the
>    problem is fixed only for the driver that is of interest to me in
>    this context, DSA, and with special API exported by phylink
>    specifically for this purpose, to limit the impact on other drivers.

There is a very different approach, which might be simpler.

We know polling will always work. And it should be possible to
transition between polling and interrupt at any point, so long as the
phylock is held. So if you get -EPROBE_DEFFER during probe, mark some
state in phydev that there should be an irq, but it is not around yet.
When the phy is started, and phylib starts polling, look for the state
and try getting the IRQ again. If successful, swap to interrupts, if
not, keep polling. Maybe after 60 seconds of polling and trying, give
up trying to find the irq and stick with polling.

     Andrew

Hi Andrew,

On Sat, May 14, 2022 at 02:23:51AM +0200, Andrew Lunn wrote:
> On Sat, May 14, 2022 at 02:36:38AM +0300, Vladimir Oltean wrote:
> > This patch set completes the picture described by
> > '[RFC,devicetree] of: property: mark "interrupts" as optional for fw_devlink'
> > https://patchwork.kernel.org/project/netdevbpf/patch/20220513201243.2381133-1-vladimir.oltean@nxp.com/
> > 
> > I've CCed non-networking maintainers just in case they want to gain a
> > better understanding. If not, apologies and please ignore the rest.
> > 
> > My use case is to migrate a PHY driver from poll mode to interrupt mode
> > without breaking compatibility between new device trees and old kernels
> > which did not have a driver for that IRQ parent, and therefore (for
> > things to work) did not even have that interrupt listed in the "vintage
> > correct" DT blobs. Note that current kernels as of today are also
> > "old kernels" in this description.
> > 
> > Creating some degree of compatibility has multiple components.
> > 
> > 1. A PHY driver must eventually give up waiting for an IRQ provider,
> >    since the dependency is optional and it can fall back to poll mode.
> >    This is currently supported thanks to commit 74befa447e68 ("net:
> >    mdio: don't defer probe forever if PHY IRQ provider is missing").
> > 
> > 2. Before it finally gives up, the PHY driver has a transient phase of
> >    returning -EPROBE_DEFER. That transient phase causes some breakage
> >    which is handled by this patch set, details below.
> > 
> > 3. PHY device probing and Ethernet controller finding it and connecting
> >    to it are async events. When both happen during probing, the problem
> >    is that finding the PHY fails if the PHY defers probe, which results
> >    in a missing PHY rather than waiting for it. Unfortunately there is
> >    no universal way to address this problem, because the majority of
> >    Ethernet drivers do not connect to the PHY during probe. So the
> >    problem is fixed only for the driver that is of interest to me in
> >    this context, DSA, and with special API exported by phylink
> >    specifically for this purpose, to limit the impact on other drivers.
> 
> There is a very different approach, which might be simpler.
> 
> We know polling will always work. And it should be possible to
> transition between polling and interrupt at any point, so long as the
> phylock is held. So if you get -EPROBE_DEFFER during probe, mark some
> state in phydev that there should be an irq, but it is not around yet.
> When the phy is started, and phylib starts polling, look for the state
> and try getting the IRQ again. If successful, swap to interrupts, if
> not, keep polling. Maybe after 60 seconds of polling and trying, give
> up trying to find the irq and stick with polling.

That doesn't sound like something that I'd backport to stable kernels.
Letting the PHY driver dynamically switch from poll to IRQ mode risks
racing with phylink's workqueue, and generally speaking, phylink doesn't
seem to be built around the idea that "bool poll" can change after
phylink_start().

What motivates me to make these changes in the first place is the idea
that current kernels should work with updated device trees. If I won't
be able to achieve that, I see no point in adding logic to transition
from poll to IRQ mode even in net-next, since I'd have to update the
kernel when I update the DT, and by then, I'd have a proper driver for
the IRQ parent anyway. Sorry.

> > There is a very different approach, which might be simpler.
> > 
> > We know polling will always work. And it should be possible to
> > transition between polling and interrupt at any point, so long as the
> > phylock is held. So if you get -EPROBE_DEFFER during probe, mark some
> > state in phydev that there should be an irq, but it is not around yet.
> > When the phy is started, and phylib starts polling, look for the state
> > and try getting the IRQ again. If successful, swap to interrupts, if
> > not, keep polling. Maybe after 60 seconds of polling and trying, give
> > up trying to find the irq and stick with polling.
> 
> That doesn't sound like something that I'd backport to stable kernels.

> What motivates me to make these changes in the first place is the idea
> that current kernels should work with updated device trees.

By current, you mean old kernels, LTS etc. You want an LTS kernel to
work with a new DT blob? You want forward compatibility with a DT
blob. Do the stable rules say anything about that?

      Andrew

On Thu, May 19, 2022 at 05:32:35PM +0200, Andrew Lunn wrote:
> > > There is a very different approach, which might be simpler.
> > > 
> > > We know polling will always work. And it should be possible to
> > > transition between polling and interrupt at any point, so long as the
> > > phylock is held. So if you get -EPROBE_DEFFER during probe, mark some
> > > state in phydev that there should be an irq, but it is not around yet.
> > > When the phy is started, and phylib starts polling, look for the state
> > > and try getting the IRQ again. If successful, swap to interrupts, if
> > > not, keep polling. Maybe after 60 seconds of polling and trying, give
> > > up trying to find the irq and stick with polling.
> > 
> > That doesn't sound like something that I'd backport to stable kernels.
> 
> > What motivates me to make these changes in the first place is the idea
> > that current kernels should work with updated device trees.
> 
> By current, you mean old kernels, LTS etc. You want an LTS kernel to
> work with a new DT blob? You want forward compatibility with a DT
> blob. Do the stable rules say anything about that?
> 
>       Andrew

Hmm, not sure about stable rules, but at least Marc Zyngier has
suggested in the past that this is something which should work:
https://lore.kernel.org/linux-arm-kernel/87czlzjxmz.wl-maz@kernel.org/

To quote:

| > As for compatibility between old kernel and new DT: I guess you'll hear
| > various opinions on this one.
| > https://www.spinics.net/lists/linux-mips/msg07778.html
| > 
| > | > Are we okay with the new device tree blobs breaking the old kernel?
| > |
| > | From my point of view, newer device trees are not required to work on
| > | older kernel, this would impose an unreasonable limitation and the use
| > | case is very limited.
| 
| My views are on the opposite side. DT is an ABI, full stop. If you
| change something, you *must* guarantee forward *and* backward
| compatibility. That's because:
| 
| - you don't control how updatable the firmware is
| 
| - people may need to revert to other versions of the kernel because
|   the new one is broken
| 
| - there are plenty of DT users beyond Linux, and we are not creating
|   bindings for Linux only.
| 
| You may disagree with this, but for the subsystems I maintain, this is
| the rule I intent to stick to.

On Thu, May 19, 2022 at 02:59:36PM +0000, Vladimir Oltean wrote:
> Hi Andrew,
> 
> On Sat, May 14, 2022 at 02:23:51AM +0200, Andrew Lunn wrote:
> > There is a very different approach, which might be simpler.
> > 
> > We know polling will always work. And it should be possible to
> > transition between polling and interrupt at any point, so long as the
> > phylock is held. So if you get -EPROBE_DEFFER during probe, mark some
> > state in phydev that there should be an irq, but it is not around yet.
> > When the phy is started, and phylib starts polling, look for the state
> > and try getting the IRQ again. If successful, swap to interrupts, if
> > not, keep polling. Maybe after 60 seconds of polling and trying, give
> > up trying to find the irq and stick with polling.
> 
> That doesn't sound like something that I'd backport to stable kernels.
> Letting the PHY driver dynamically switch from poll to IRQ mode risks
> racing with phylink's workqueue, and generally speaking, phylink doesn't
> seem to be built around the idea that "bool poll" can change after
> phylink_start().

I think you're confused. Andrew is merely talking about phylib's
polling, not phylink's.

Phylink's polling is only ever used in two circumstances:

1. In fixed-link mode where we have an interruptless GPIO.
2. In in-band mode when the PCS specifies it needs to be polled.

This is not used to poll ethernet PHYs - ethernet PHY polling is
handled entirely by phylib itself.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

On Thu, May 19, 2022 at 04:15:32PM +0100, Russell King (Oracle) wrote:
> On Thu, May 19, 2022 at 02:59:36PM +0000, Vladimir Oltean wrote:
> > Hi Andrew,
> > 
> > On Sat, May 14, 2022 at 02:23:51AM +0200, Andrew Lunn wrote:
> > > There is a very different approach, which might be simpler.
> > > 
> > > We know polling will always work. And it should be possible to
> > > transition between polling and interrupt at any point, so long as the
> > > phylock is held. So if you get -EPROBE_DEFFER during probe, mark some
> > > state in phydev that there should be an irq, but it is not around yet.
> > > When the phy is started, and phylib starts polling, look for the state
> > > and try getting the IRQ again. If successful, swap to interrupts, if
> > > not, keep polling. Maybe after 60 seconds of polling and trying, give
> > > up trying to find the irq and stick with polling.
> > 
> > That doesn't sound like something that I'd backport to stable kernels.
> > Letting the PHY driver dynamically switch from poll to IRQ mode risks
> > racing with phylink's workqueue, and generally speaking, phylink doesn't
> > seem to be built around the idea that "bool poll" can change after
> > phylink_start().
> 
> I think you're confused. Andrew is merely talking about phylib's
> polling, not phylink's.
> 
> Phylink's polling is only ever used in two circumstances:
> 
> 1. In fixed-link mode where we have an interruptless GPIO.
> 2. In in-band mode when the PCS specifies it needs to be polled.
> 
> This is not used to poll ethernet PHYs - ethernet PHY polling is
> handled entirely by phylib itself.

You're right, I would have probably figured that out if I actually tried
to implement what Andrew is proposing and not just superficially looking
at the code. I guess I'll try that now and see where that leads me to.
The only thing that remains in that case is fw_devlink blocking PHY
probing for lack of a supplier.

On Fri, May 13, 2022 at 4:37 PM Vladimir Oltean <vladimir.oltean@nxp.com> wrote:
>
> This patch set completes the picture described by
> '[RFC,devicetree] of: property: mark "interrupts" as optional for fw_devlink'
> https://patchwork.kernel.org/project/netdevbpf/patch/20220513201243.2381133-1-vladimir.oltean@nxp.com/

I replied to that patch. I don't think we can pull that in.

> I've CCed non-networking maintainers just in case they want to gain a
> better understanding. If not, apologies and please ignore the rest.
>
>
>
> My use case is to migrate a PHY driver from poll mode to interrupt mode
> without breaking compatibility between new device trees and old kernels
> which did not have a driver for that IRQ parent, and therefore (for
> things to work) did not even have that interrupt listed in the "vintage
> correct" DT blobs. Note that current kernels as of today are also
> "old kernels" in this description.
>
> Creating some degree of compatibility has multiple components.
>
> 1. A PHY driver must eventually give up waiting for an IRQ provider,
>    since the dependency is optional and it can fall back to poll mode.
>    This is currently supported thanks to commit 74befa447e68 ("net:
>    mdio: don't defer probe forever if PHY IRQ provider is missing").
>
> 2. Before it finally gives up, the PHY driver has a transient phase of
>    returning -EPROBE_DEFER. That transient phase causes some breakage
>    which is handled by this patch set, details below.
>
> 3. PHY device probing and Ethernet controller finding it and connecting
>    to it are async events. When both happen during probing, the problem
>    is that finding the PHY fails if the PHY defers probe, which results
>    in a missing PHY rather than waiting for it. Unfortunately there is
>    no universal way to address this problem, because the majority of
>    Ethernet drivers do not connect to the PHY during probe. So the
>    problem is fixed only for the driver that is of interest to me in
>    this context, DSA, and with special API exported by phylink
>    specifically for this purpose, to limit the impact on other drivers.

I'll take a closer look at this later this week, but once we add
phy-handle support to fw_devlink (the device_bind_driver() is making
it hard to add support), I think we can address most/all of these
problems automatically. So hopefully we can work towards that?
Actually this patch might already fix this for you:
https://lore.kernel.org/lkml/20220429220933.1350374-1-saravanak@google.com/

Before fw_devlink, we'd give up on waiting on all suppliers, whether
they had a driver (but hadn't yet probed for a multitude of reasons)
or not. fw_devlink is smart about allowing consumers to probe without
their suppliers only if the supplier has no driver or the driver fails
(I'll send a patch for this). The deferred_probe_timeout is what's
used to decide when to give up waiting for drivers.

-Saravana

>
> Note that drivers that connect to the PHY at ndo_open are superficially
> "fixed" by the patch at step 1 alone, and therefore don't need the
> mechanism introduced in phylink here. This is because of the larger span
> of time between PHY probe and opening the network interface (typically
> initiated by user space). But this is the catch, nfsroot and other
> in-kernel networking users can also open the net device, and this will
> still expose the EPROBE_DEFER as a hard error for this second kind of
> drivers. I don't know how to fix that. From this POV, it's better to do
> what DSA does (connect to the PHY on probe).
>
> Vladimir Oltean (2):
>   net: phylink: allow PHY driver to defer probe when connecting via OF
>     node
>   net: dsa: wait for PHY to defer probe
>
>  drivers/net/phy/phylink.c | 73 ++++++++++++++++++++++++++++++---------
>  include/linux/phylink.h   |  2 ++
>  net/dsa/dsa2.c            |  2 ++
>  net/dsa/port.c            |  6 ++--
>  net/dsa/slave.c           | 10 +++---
>  5 files changed, 70 insertions(+), 23 deletions(-)
>
> --
> 2.25.1
>