[v2] PCI: endpoint: pci-epf-test: Fix sleeping function being called from atomic context

[PATCH v2] PCI: endpoint: pci-epf-test: Fix sleeping function being called from atomic context

Posted by Bhanu Seshu Kumar Valluri 4 months, 1 week ago

When Root Complex(RC) triggers a Doorbell MSI interrupt to Endpoint(EP) it triggers a warning
in the EP. pci_endpoint kselftest target is compiled and used to run the Doorbell test in RC.

[  474.686193] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:271
[  474.710934] Call trace:
[  474.710995]  __might_resched+0x130/0x158
[  474.711011]  __might_sleep+0x70/0x88
[  474.711023]  mutex_lock+0x2c/0x80
[  474.711036]  pci_epc_get_msi+0x78/0xd8
[  474.711052]  pci_epf_test_raise_irq.isra.0+0x74/0x138
[  474.711063]  pci_epf_test_doorbell_handler+0x34/0x50

The BUG arises because the EP's pci_epf_test_doorbell_handler is making an
indirect call to pci_epc_get_msi, which uses mutex inside, from interrupt context.

To fix the issue convert hard irq handler to a threaded irq handler to allow it
to call functions that can sleep during bottom half execution. Register threaded
irq handler with IRQF_ONESHOT to keep interrupt line disabled until the threaded
irq handler completes execution.

Fixes: eff0c286aa91 ("PCI: endpoint: pci-epf-test: Add doorbell test support")
Signed-off-by: Bhanu Seshu Kumar Valluri <bhanuseshukumar@gmail.com>
---
 Note : It is compiled and tested on TI am642 board.

 Change log. V1->V2: 
  Trimmed Call trace to include only essential calls.
  Used 12 digit commit ID in fixes tag.
  Steps to reproduce the bug are removed from commit log.
  Link to V1: https://lore.kernel.org/all/20250917161817.15776-1-bhanuseshukumar@gmail.com/
 	
 Warnings can be reproduced by following steps below.
 *On EP side:
 1. Configure the pci-epf-test function using steps given below
   mount -t configfs none /sys/kernel/config
   cd /sys/kernel/config/pci_ep/
   mkdir functions/pci_epf_test/func1
   echo 0x104c > functions/pci_epf_test/func1/vendorid
   echo 0xb010 > functions/pci_epf_test/func1/deviceid
   echo 32 > functions/pci_epf_test/func1/msi_interrupts
   echo 2048 > functions/pci_epf_test/func1/msix_interrupts
   ln -s functions/pci_epf_test/func1 controllers/f102000.pcie-ep/
   echo 1 > controllers/f102000.pcie-ep/start

 *On RC side:
 1. Once EP side configuration is done do pci rescan.
   echo 1 > /sys/bus/pci/rescan
 2. Run Doorbell MSI test using pci_endpoint_test kselftest app.
  ./pci_endpoint_test -r pcie_ep_doorbell.DOORBELL_TEST
  Note: Kernel is compiled with CONFIG_DEBUG_KERNEL enabled.

 drivers/pci/endpoint/functions/pci-epf-test.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c
index e091193bd8a8..c9e2eb930ad3 100644
--- a/drivers/pci/endpoint/functions/pci-epf-test.c
+++ b/drivers/pci/endpoint/functions/pci-epf-test.c
@@ -725,8 +725,8 @@ static void pci_epf_test_enable_doorbell(struct pci_epf_test *epf_test,
 	if (bar < BAR_0)
 		goto err_doorbell_cleanup;
 
-	ret = request_irq(epf->db_msg[0].virq, pci_epf_test_doorbell_handler, 0,
-			  "pci-ep-test-doorbell", epf_test);
+	ret = request_threaded_irq(epf->db_msg[0].virq, NULL, pci_epf_test_doorbell_handler,
+				   IRQF_ONESHOT, "pci-ep-test-doorbell", epf_test);
 	if (ret) {
 		dev_err(&epf->dev,
 			"Failed to request doorbell IRQ: %d\n",
-- 
2.34.1

Re: [PATCH v2] PCI: endpoint: pci-epf-test: Fix sleeping function being called from atomic context

Posted by Bjorn Helgaas 3 months, 3 weeks ago

On Tue, Sep 30, 2025 at 08:08:09AM +0530, Bhanu Seshu Kumar Valluri wrote:
> When Root Complex(RC) triggers a Doorbell MSI interrupt to Endpoint(EP) it triggers a warning
> in the EP. pci_endpoint kselftest target is compiled and used to run the Doorbell test in RC.
> 
> [  474.686193] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:271
> [  474.710934] Call trace:
> [  474.710995]  __might_resched+0x130/0x158
> [  474.711011]  __might_sleep+0x70/0x88
> [  474.711023]  mutex_lock+0x2c/0x80
> [  474.711036]  pci_epc_get_msi+0x78/0xd8
> [  474.711052]  pci_epf_test_raise_irq.isra.0+0x74/0x138
> [  474.711063]  pci_epf_test_doorbell_handler+0x34/0x50
> 
> The BUG arises because the EP's pci_epf_test_doorbell_handler is making an
> indirect call to pci_epc_get_msi, which uses mutex inside, from interrupt context.
> 
> To fix the issue convert hard irq handler to a threaded irq handler to allow it
> to call functions that can sleep during bottom half execution. Register threaded
> irq handler with IRQF_ONESHOT to keep interrupt line disabled until the threaded
> irq handler completes execution.
> 
> Fixes: eff0c286aa91 ("PCI: endpoint: pci-epf-test: Add doorbell test support")
> Signed-off-by: Bhanu Seshu Kumar Valluri <bhanuseshukumar@gmail.com>

Thanks for the fix!  It looks like you posted it during the v6.18
merge window, so it was a little bit too late to be included in the
v6.18 changes, but it looks like good v6.19 material.

Can you please:

  - Rebase to pci/main (v6.18-rc1)
  - Add a space before each "("
  - Remove the timestamps because they're unnecessary distraction
  - Add "()" after function names in commit log
  - s/irq/IRQ/
  - Rewrap the commit log to fit in 75 columns
  - Rewrap the code below to fit in 78 columns because most of the
    rest of the file does
  - Carry Niklas' Reviewed-by when you post the v3

> ---
>  Note : It is compiled and tested on TI am642 board.
> 
>  Change log. V1->V2: 
>   Trimmed Call trace to include only essential calls.
>   Used 12 digit commit ID in fixes tag.
>   Steps to reproduce the bug are removed from commit log.
>   Link to V1: https://lore.kernel.org/all/20250917161817.15776-1-bhanuseshukumar@gmail.com/
>  	
>  Warnings can be reproduced by following steps below.
>  *On EP side:
>  1. Configure the pci-epf-test function using steps given below
>    mount -t configfs none /sys/kernel/config
>    cd /sys/kernel/config/pci_ep/
>    mkdir functions/pci_epf_test/func1
>    echo 0x104c > functions/pci_epf_test/func1/vendorid
>    echo 0xb010 > functions/pci_epf_test/func1/deviceid
>    echo 32 > functions/pci_epf_test/func1/msi_interrupts
>    echo 2048 > functions/pci_epf_test/func1/msix_interrupts
>    ln -s functions/pci_epf_test/func1 controllers/f102000.pcie-ep/
>    echo 1 > controllers/f102000.pcie-ep/start
> 
>  *On RC side:
>  1. Once EP side configuration is done do pci rescan.
>    echo 1 > /sys/bus/pci/rescan
>  2. Run Doorbell MSI test using pci_endpoint_test kselftest app.
>   ./pci_endpoint_test -r pcie_ep_doorbell.DOORBELL_TEST
>   Note: Kernel is compiled with CONFIG_DEBUG_KERNEL enabled.
> 
>  drivers/pci/endpoint/functions/pci-epf-test.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c
> index e091193bd8a8..c9e2eb930ad3 100644
> --- a/drivers/pci/endpoint/functions/pci-epf-test.c
> +++ b/drivers/pci/endpoint/functions/pci-epf-test.c
> @@ -725,8 +725,8 @@ static void pci_epf_test_enable_doorbell(struct pci_epf_test *epf_test,
>  	if (bar < BAR_0)
>  		goto err_doorbell_cleanup;
>  
> -	ret = request_irq(epf->db_msg[0].virq, pci_epf_test_doorbell_handler, 0,
> -			  "pci-ep-test-doorbell", epf_test);
> +	ret = request_threaded_irq(epf->db_msg[0].virq, NULL, pci_epf_test_doorbell_handler,
> +				   IRQF_ONESHOT, "pci-ep-test-doorbell", epf_test);
>  	if (ret) {
>  		dev_err(&epf->dev,
>  			"Failed to request doorbell IRQ: %d\n",
> -- 
> 2.34.1
>

Re: [PATCH v2] PCI: endpoint: pci-epf-test: Fix sleeping function being called from atomic context

Posted by Bhanu Seshu Kumar Valluri 3 months, 3 weeks ago

On 14/10/25 03:10, Bjorn Helgaas wrote:
> Thanks for the fix!  It looks like you posted it during the v6.18
> merge window, so it was a little bit too late to be included in the
> v6.18 changes, but it looks like good v6.19 material.
> 
> Can you please:
> 
>   - Rebase to pci/main (v6.18-rc1)
>   - Add a space before each "("
>   - Remove the timestamps because they're unnecessary distraction
>   - Add "()" after function names in commit log
>   - s/irq/IRQ/
>   - Rewrap the commit log to fit in 75 columns
>   - Rewrap the code below to fit in 78 columns because most of the
>     rest of the file does
>   - Carry Niklas' Reviewed-by when you post the v3
> 

Hi,

Thanks for the helpful review. I sent a v3 patch to address above review points.

Link to v3: https://lore.kernel.org/all/20251014024109.42287-1-bhanuseshukumar@gmail.com/

Regards,
Bhanu Seshu Kumar Valluri

Re: [PATCH v2] PCI: endpoint: pci-epf-test: Fix sleeping function being called from atomic context

Posted by Niklas Cassel 4 months ago

On Tue, Sep 30, 2025 at 08:08:09AM +0530, Bhanu Seshu Kumar Valluri wrote:
> When Root Complex(RC) triggers a Doorbell MSI interrupt to Endpoint(EP) it triggers a warning
> in the EP. pci_endpoint kselftest target is compiled and used to run the Doorbell test in RC.
> 
> [  474.686193] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:271
> [  474.710934] Call trace:
> [  474.710995]  __might_resched+0x130/0x158
> [  474.711011]  __might_sleep+0x70/0x88
> [  474.711023]  mutex_lock+0x2c/0x80
> [  474.711036]  pci_epc_get_msi+0x78/0xd8
> [  474.711052]  pci_epf_test_raise_irq.isra.0+0x74/0x138
> [  474.711063]  pci_epf_test_doorbell_handler+0x34/0x50
> 
> The BUG arises because the EP's pci_epf_test_doorbell_handler is making an
> indirect call to pci_epc_get_msi, which uses mutex inside, from interrupt context.
> 
> To fix the issue convert hard irq handler to a threaded irq handler to allow it
> to call functions that can sleep during bottom half execution. Register threaded
> irq handler with IRQF_ONESHOT to keep interrupt line disabled until the threaded
> irq handler completes execution.
> 
> Fixes: eff0c286aa91 ("PCI: endpoint: pci-epf-test: Add doorbell test support")
> Signed-off-by: Bhanu Seshu Kumar Valluri <bhanuseshukumar@gmail.com>
> ---

All other calls to pci_epf_test_raise_irq() is done from the workqueue.

While we could change pci_epf_test_doorbell_handler() to queue some work on
the workqueue (and let that work call pci_epf_test_raise_irq()), converting
pci_epf_test_doorbell_handler() to be a threaded IRQ handler seems like the
less invasive change, thus:

Reviewed-by: Niklas Cassel <cassel@kernel.org>

Re: [PATCH v2] PCI: endpoint: pci-epf-test: Fix sleeping function being called from atomic context

Posted by Bhanu Seshu Kumar Valluri 4 months ago

On 30/09/25 08:08, Bhanu Seshu Kumar Valluri wrote:
> When Root Complex(RC) triggers a Doorbell MSI interrupt to Endpoint(EP) it triggers a warning
> in the EP. pci_endpoint kselftest target is compiled and used to run the Doorbell test in RC.
> 
> [  474.686193] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:271
> [  474.710934] Call trace:
> [  474.710995]  __might_resched+0x130/0x158
> [  474.711011]  __might_sleep+0x70/0x88
> [  474.711023]  mutex_lock+0x2c/0x80
> [  474.711036]  pci_epc_get_msi+0x78/0xd8
> [  474.711052]  pci_epf_test_raise_irq.isra.0+0x74/0x138
> [  474.711063]  pci_epf_test_doorbell_handler+0x34/0x50
> 
> The BUG arises because the EP's pci_epf_test_doorbell_handler is making an
> indirect call to pci_epc_get_msi, which uses mutex inside, from interrupt context.
> 
> To fix the issue convert hard irq handler to a threaded irq handler to allow it
> to call functions that can sleep during bottom half execution. Register threaded
> irq handler with IRQF_ONESHOT to keep interrupt line disabled until the threaded
> irq handler completes execution.
> 
> Fixes: eff0c286aa91 ("PCI: endpoint: pci-epf-test: Add doorbell test support")
> Signed-off-by: Bhanu Seshu Kumar Valluri <bhanuseshukumar@gmail.com>
> ---
>  Note : It is compiled and tested on TI am642 board.
> 
>  Change log. V1->V2: 
>   Trimmed Call trace to include only essential calls.
>   Used 12 digit commit ID in fixes tag.
>   Steps to reproduce the bug are removed from commit log.
>   Link to V1: https://lore.kernel.org/all/20250917161817.15776-1-bhanuseshukumar@gmail.com/
>  	
>  Warnings can be reproduced by following steps below.
>  *On EP side:
>  1. Configure the pci-epf-test function using steps given below
>    mount -t configfs none /sys/kernel/config
>    cd /sys/kernel/config/pci_ep/
>    mkdir functions/pci_epf_test/func1
>    echo 0x104c > functions/pci_epf_test/func1/vendorid
>    echo 0xb010 > functions/pci_epf_test/func1/deviceid
>    echo 32 > functions/pci_epf_test/func1/msi_interrupts
>    echo 2048 > functions/pci_epf_test/func1/msix_interrupts
>    ln -s functions/pci_epf_test/func1 controllers/f102000.pcie-ep/
>    echo 1 > controllers/f102000.pcie-ep/start
> 
>  *On RC side:
>  1. Once EP side configuration is done do pci rescan.
>    echo 1 > /sys/bus/pci/rescan
>  2. Run Doorbell MSI test using pci_endpoint_test kselftest app.
>   ./pci_endpoint_test -r pcie_ep_doorbell.DOORBELL_TEST
>   Note: Kernel is compiled with CONFIG_DEBUG_KERNEL enabled.
> 
>  drivers/pci/endpoint/functions/pci-epf-test.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c
> index e091193bd8a8..c9e2eb930ad3 100644
> --- a/drivers/pci/endpoint/functions/pci-epf-test.c
> +++ b/drivers/pci/endpoint/functions/pci-epf-test.c
> @@ -725,8 +725,8 @@ static void pci_epf_test_enable_doorbell(struct pci_epf_test *epf_test,
>  	if (bar < BAR_0)
>  		goto err_doorbell_cleanup;
>  
> -	ret = request_irq(epf->db_msg[0].virq, pci_epf_test_doorbell_handler, 0,
> -			  "pci-ep-test-doorbell", epf_test);
> +	ret = request_threaded_irq(epf->db_msg[0].virq, NULL, pci_epf_test_doorbell_handler,
> +				   IRQF_ONESHOT, "pci-ep-test-doorbell", epf_test);
>  	if (ret) {
>  		dev_err(&epf->dev,
>  			"Failed to request doorbell IRQ: %d\n",

Hi,

I just wanted to check if you had a chance to review it or if any changes are needed from my side.

Regards,
Bhanu Seshu Kumar Valluri