PCI/IOV: Add missing PCI rescan-remove locking when enabling/disabling SR-IOV

[PATCH 2/2] PCI: Add lockdep assertion in pci_stop_and_remove_bus_device()

Posted by Niklas Schnelle 5 months, 2 weeks ago

Removing a PCI devices requires holding pci_rescan_remove_lock. Prompted
by this being missed in sriov_disable() and going unnoticed since its
inception add a lockdep assert so this doesn't get missed again in the
future.

Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
---
 drivers/pci/pci.h    | 2 ++
 drivers/pci/probe.c  | 2 +-
 drivers/pci/remove.c | 1 +
 3 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 34f65d69662e9f61f0c489ec58de2ce17d21c0c6..1ad2e3ab147f3b2c42b3257e4f366fc5e424ede3 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -84,6 +84,8 @@ struct pcie_tlp_log;
 extern const unsigned char pcie_link_speed[];
 extern bool pci_early_dump;
 
+extern struct mutex pci_rescan_remove_lock;
+
 bool pcie_cap_has_lnkctl(const struct pci_dev *dev);
 bool pcie_cap_has_lnkctl2(const struct pci_dev *dev);
 bool pcie_cap_has_rtctl(const struct pci_dev *dev);
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index f41128f91ca76ab014ad669ae84a53032c7c6b6b..2b35bb39ab0366bbf86b43e721811575b9fbcefb 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -3469,7 +3469,7 @@ EXPORT_SYMBOL_GPL(pci_rescan_bus);
  * pci_rescan_bus(), pci_rescan_bus_bridge_resize() and PCI device removal
  * routines should always be executed under this mutex.
  */
-static DEFINE_MUTEX(pci_rescan_remove_lock);
+DEFINE_MUTEX(pci_rescan_remove_lock);
 
 void pci_lock_rescan_remove(void)
 {
diff --git a/drivers/pci/remove.c b/drivers/pci/remove.c
index 445afdfa6498edc88f1ef89df279af1419025495..0b9a609392cecba36a818bc496a0af64061c259a 100644
--- a/drivers/pci/remove.c
+++ b/drivers/pci/remove.c
@@ -138,6 +138,7 @@ static void pci_remove_bus_device(struct pci_dev *dev)
  */
 void pci_stop_and_remove_bus_device(struct pci_dev *dev)
 {
+	lockdep_assert_held(&pci_rescan_remove_lock);
 	pci_stop_bus_device(dev);
 	pci_remove_bus_device(dev);
 }

-- 
2.48.1

Re: [PATCH 2/2] PCI: Add lockdep assertion in pci_stop_and_remove_bus_device()

Posted by Julian Ruess 4 months, 2 weeks ago

On Tue Aug 26, 2025 at 10:52 AM CEST, Niklas Schnelle wrote:
> Removing a PCI devices requires holding pci_rescan_remove_lock. Prompted
> by this being missed in sriov_disable() and going unnoticed since its
> inception add a lockdep assert so this doesn't get missed again in the
> future.
>
> Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
> Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
> ---
>  drivers/pci/pci.h    | 2 ++
>  drivers/pci/probe.c  | 2 +-
>  drivers/pci/remove.c | 1 +
>  3 files changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
> index 34f65d69662e9f61f0c489ec58de2ce17d21c0c6..1ad2e3ab147f3b2c42b3257e4f366fc5e424ede3 100644
> --- a/drivers/pci/pci.h
> +++ b/drivers/pci/pci.h
> @@ -84,6 +84,8 @@ struct pcie_tlp_log;
>  extern const unsigned char pcie_link_speed[];
>  extern bool pci_early_dump;
>  
> +extern struct mutex pci_rescan_remove_lock;
> +
>  bool pcie_cap_has_lnkctl(const struct pci_dev *dev);
>  bool pcie_cap_has_lnkctl2(const struct pci_dev *dev);
>  bool pcie_cap_has_rtctl(const struct pci_dev *dev);
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index f41128f91ca76ab014ad669ae84a53032c7c6b6b..2b35bb39ab0366bbf86b43e721811575b9fbcefb 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -3469,7 +3469,7 @@ EXPORT_SYMBOL_GPL(pci_rescan_bus);
>   * pci_rescan_bus(), pci_rescan_bus_bridge_resize() and PCI device removal
>   * routines should always be executed under this mutex.
>   */
> -static DEFINE_MUTEX(pci_rescan_remove_lock);
> +DEFINE_MUTEX(pci_rescan_remove_lock);
>  
>  void pci_lock_rescan_remove(void)
>  {
> diff --git a/drivers/pci/remove.c b/drivers/pci/remove.c
> index 445afdfa6498edc88f1ef89df279af1419025495..0b9a609392cecba36a818bc496a0af64061c259a 100644
> --- a/drivers/pci/remove.c
> +++ b/drivers/pci/remove.c
> @@ -138,6 +138,7 @@ static void pci_remove_bus_device(struct pci_dev *dev)
>   */
>  void pci_stop_and_remove_bus_device(struct pci_dev *dev)
>  {
> +	lockdep_assert_held(&pci_rescan_remove_lock);
>  	pci_stop_bus_device(dev);
>  	pci_remove_bus_device(dev);
>  }

Feel free to add my
Reviewed-by: Julian Ruess <julianr@linux.ibm.com>

Thanks,
Julian

Re: [PATCH 2/2] PCI: Add lockdep assertion in pci_stop_and_remove_bus_device()

Posted by Gerd Bayer 5 months ago

On Tue, 2025-08-26 at 10:52 +0200, Niklas Schnelle wrote:
> Removing a PCI devices requires holding pci_rescan_remove_lock. Prompted
> by this being missed in sriov_disable() and going unnoticed since its
> inception add a lockdep assert so this doesn't get missed again in the
> future.
> 
> Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
> Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
> ---
>  drivers/pci/pci.h    | 2 ++
>  drivers/pci/probe.c  | 2 +-
>  drivers/pci/remove.c | 1 +
>  3 files changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
> index 34f65d69662e9f61f0c489ec58de2ce17d21c0c6..1ad2e3ab147f3b2c42b3257e4f366fc5e424ede3 100644
> --- a/drivers/pci/pci.h
> +++ b/drivers/pci/pci.h
> @@ -84,6 +84,8 @@ struct pcie_tlp_log;
>  extern const unsigned char pcie_link_speed[];
>  extern bool pci_early_dump;
>  
> +extern struct mutex pci_rescan_remove_lock;
> +
>  bool pcie_cap_has_lnkctl(const struct pci_dev *dev);
>  bool pcie_cap_has_lnkctl2(const struct pci_dev *dev);
>  bool pcie_cap_has_rtctl(const struct pci_dev *dev);
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index f41128f91ca76ab014ad669ae84a53032c7c6b6b..2b35bb39ab0366bbf86b43e721811575b9fbcefb 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -3469,7 +3469,7 @@ EXPORT_SYMBOL_GPL(pci_rescan_bus);
>   * pci_rescan_bus(), pci_rescan_bus_bridge_resize() and PCI device removal
>   * routines should always be executed under this mutex.
>   */
> -static DEFINE_MUTEX(pci_rescan_remove_lock);
> +DEFINE_MUTEX(pci_rescan_remove_lock);
>  
>  void pci_lock_rescan_remove(void)
>  {
> diff --git a/drivers/pci/remove.c b/drivers/pci/remove.c
> index 445afdfa6498edc88f1ef89df279af1419025495..0b9a609392cecba36a818bc496a0af64061c259a 100644
> --- a/drivers/pci/remove.c
> +++ b/drivers/pci/remove.c
> @@ -138,6 +138,7 @@ static void pci_remove_bus_device(struct pci_dev *dev)
>   */
>  void pci_stop_and_remove_bus_device(struct pci_dev *dev)
>  {
> +	lockdep_assert_held(&pci_rescan_remove_lock);
>  	pci_stop_bus_device(dev);
>  	pci_remove_bus_device(dev);
>  }

I'm totally in favor of adding this lockdep assertion, even if this
means that the mutex pci_rescan_remove_lock needs to be externalized
from drivers/pci/probe.c.

However, I was surprised that you didn't add the assertion to the
_locked() variant until I realized that here the naming of _locked vs.
not _locked variants of pci_stop_and_remove_bus_device() is just the
opposite to the naming in driver/pci/pci.c:
There _locked implies that the necessary lock is already held on
routine entry. But this change in semantics was already introduced with
commit 9d16947b7583 ("PCI: Add global pci_lock_rescan_remove()").

Looks like aligning the naming to the convention in driver/pci/pci.c
would touch quite a bit of code - but so does the introduction of this
lockdep assertion...

Sigh, Gerd