From: "Jason-JH.Lin" <jason-jh.lin@mediatek.com>

When mtk-cmdq unbinds, a WARN_ON message with condition
pm_runtime_get_sync() < 0 occurs.

According to the call tracei below:
  cmdq_mbox_shutdown
  mbox_free_channel
  mbox_controller_unregister
  __devm_mbox_controller_unregister
  ...

The root cause can be deduced to be calling pm_runtime_get_sync() after
calling pm_runtime_disable() as observed below:
1. CMDQ driver uses devm_mbox_controller_register() in cmdq_probe()
   to bind the cmdq device to the mbox_controller, so
   devm_mbox_controller_unregister() will automatically unregister
   the device bound to the mailbox controller when the device-managed
   resource is removed. That means devm_mbox_controller_unregister()
   and cmdq_mbox_shoutdown() will be called after cmdq_remove().
2. CMDQ driver also uses devm_pm_runtime_enable() in cmdq_probe() after
   devm_mbox_controller_register(), so that devm_pm_runtime_disable()
   will be called after cmdq_remove(), but before
   devm_mbox_controller_unregister().

To fix this problem, cmdq_probe() needs to move
devm_mbox_controller_register() after devm_pm_runtime_enable() to make
devm_pm_runtime_disable() be called after
devm_mbox_controller_unregister().

Fixes: 623a6143a845 ("mailbox: mediatek: Add Mediatek CMDQ driver")
Signed-off-by: Jason-JH.Lin <jason-jh.lin@mediatek.com>
---
When mtk-cmdq unbinds, a WARN_ON message with condition
pm_runtime_get_sync() < 0 occurs.

According to the call tracei below:
  cmdq_mbox_shutdown
  mbox_free_channel
  mbox_controller_unregister
  __devm_mbox_controller_unregister
  ...

The root cause can be deduced to be calling pm_runtime_get_sync() after
calling pm_runtime_disable() as observed below:
1. CMDQ driver uses devm_mbox_controller_register() in cmdq_probe()
   to bind the cmdq device to the mbox_controller, so
   devm_mbox_controller_unregister() will automatically unregister
   the device bound to the mailbox controller when the device-managed
   resource is removed. That means devm_mbox_controller_unregister()
   and cmdq_mbox_shoutdown() will be called after cmdq_remove().
2. CMDQ driver also uses devm_pm_runtime_enable() in cmdq_probe() after
   devm_mbox_controller_register(), so that devm_pm_runtime_disable()
   will be called after cmdq_remove(), but before
   devm_mbox_controller_unregister().

To fix this problem, cmdq_probe() needs to move
devm_mbox_controller_register() after devm_pm_runtime_enable() to make
devm_pm_runtime_disable() be called after
devm_mbox_controller_unregister().

Change-Id: I4f0e68eac5186fe8041ea2f16c8e852d52d71cb0
---
Changes in v2:
- Change to move the calling sequence of devm_mbox_controller_register()
  and devm_pm_runtime_enable().
- Link to v1: https://lore.kernel.org/r/20240613150626.25280-1-jason-jh.lin@mediatek.com
---
 drivers/mailbox/mtk-cmdq-mailbox.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/drivers/mailbox/mtk-cmdq-mailbox.c b/drivers/mailbox/mtk-cmdq-mailbox.c
index f1dfce9e27f5..002003c3970a 100644
--- a/drivers/mailbox/mtk-cmdq-mailbox.c
+++ b/drivers/mailbox/mtk-cmdq-mailbox.c
@@ -370,6 +370,8 @@ static void cmdq_remove(struct platform_device *pdev)
 {
 	struct cmdq *cmdq = platform_get_drvdata(pdev);
 
+	devm_mbox_controller_unregister(&pdev->dev, &cmdq->mbox);
+
 	if (cmdq->pdata->sw_ddr_en)
 		cmdq_sw_ddr_enable(cmdq, false);
 
@@ -689,12 +691,6 @@ static int cmdq_probe(struct platform_device *pdev)
 		cmdq->mbox.chans[i].con_priv = (void *)&cmdq->thread[i];
 	}
 
-	err = devm_mbox_controller_register(dev, &cmdq->mbox);
-	if (err < 0) {
-		dev_err(dev, "failed to register mailbox: %d\n", err);
-		return err;
-	}
-
 	platform_set_drvdata(pdev, cmdq);
 
 	WARN_ON(clk_bulk_prepare(cmdq->pdata->gce_num, cmdq->clocks));
@@ -722,6 +718,12 @@ static int cmdq_probe(struct platform_device *pdev)
 	pm_runtime_set_autosuspend_delay(dev, CMDQ_MBOX_AUTOSUSPEND_DELAY_MS);
 	pm_runtime_use_autosuspend(dev);
 
+	err = devm_mbox_controller_register(dev, &cmdq->mbox);
+	if (err < 0) {
+		dev_err(dev, "failed to register mailbox: %d\n", err);
+		return err;
+	}
+
 	return 0;
 }
 

---
base-commit: 797012914d2d031430268fe512af0ccd7d8e46ef
change-id: 20240718-mtk-cmdq-fixup-40379650e96b

Best regards,
-- 
Jason-JH.Lin <jason-jh.lin@mediatek.com>

Il 18/07/24 10:38, Jason-JH.Lin via B4 Relay ha scritto:
> From: "Jason-JH.Lin" <jason-jh.lin@mediatek.com>
> 
> When mtk-cmdq unbinds, a WARN_ON message with condition
> pm_runtime_get_sync() < 0 occurs.
> 
> According to the call tracei below:
>    cmdq_mbox_shutdown
>    mbox_free_channel
>    mbox_controller_unregister
>    __devm_mbox_controller_unregister
>    ...
> 
> The root cause can be deduced to be calling pm_runtime_get_sync() after
> calling pm_runtime_disable() as observed below:
> 1. CMDQ driver uses devm_mbox_controller_register() in cmdq_probe()
>     to bind the cmdq device to the mbox_controller, so
>     devm_mbox_controller_unregister() will automatically unregister
>     the device bound to the mailbox controller when the device-managed
>     resource is removed. That means devm_mbox_controller_unregister()
>     and cmdq_mbox_shoutdown() will be called after cmdq_remove().
> 2. CMDQ driver also uses devm_pm_runtime_enable() in cmdq_probe() after
>     devm_mbox_controller_register(), so that devm_pm_runtime_disable()
>     will be called after cmdq_remove(), but before
>     devm_mbox_controller_unregister().
> 
> To fix this problem, cmdq_probe() needs to move
> devm_mbox_controller_register() after devm_pm_runtime_enable() to make
> devm_pm_runtime_disable() be called after
> devm_mbox_controller_unregister().
> 
> Fixes: 623a6143a845 ("mailbox: mediatek: Add Mediatek CMDQ driver")
> Signed-off-by: Jason-JH.Lin <jason-jh.lin@mediatek.com>
> ---
> When mtk-cmdq unbinds, a WARN_ON message with condition
> pm_runtime_get_sync() < 0 occurs.
> 
> According to the call tracei below:
>    cmdq_mbox_shutdown
>    mbox_free_channel
>    mbox_controller_unregister
>    __devm_mbox_controller_unregister
>    ...
> 
> The root cause can be deduced to be calling pm_runtime_get_sync() after
> calling pm_runtime_disable() as observed below:
> 1. CMDQ driver uses devm_mbox_controller_register() in cmdq_probe()
>     to bind the cmdq device to the mbox_controller, so
>     devm_mbox_controller_unregister() will automatically unregister
>     the device bound to the mailbox controller when the device-managed
>     resource is removed. That means devm_mbox_controller_unregister()
>     and cmdq_mbox_shoutdown() will be called after cmdq_remove().
> 2. CMDQ driver also uses devm_pm_runtime_enable() in cmdq_probe() after
>     devm_mbox_controller_register(), so that devm_pm_runtime_disable()
>     will be called after cmdq_remove(), but before
>     devm_mbox_controller_unregister().
> 
> To fix this problem, cmdq_probe() needs to move
> devm_mbox_controller_register() after devm_pm_runtime_enable() to make
> devm_pm_runtime_disable() be called after
> devm_mbox_controller_unregister().
> 
> Change-Id: I4f0e68eac5186fe8041ea2f16c8e852d52d71cb0
> ---
> Changes in v2:
> - Change to move the calling sequence of devm_mbox_controller_register()
>    and devm_pm_runtime_enable().
> - Link to v1: https://lore.kernel.org/r/20240613150626.25280-1-jason-jh.lin@mediatek.com
> ---
>   drivers/mailbox/mtk-cmdq-mailbox.c | 14 ++++++++------
>   1 file changed, 8 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/mailbox/mtk-cmdq-mailbox.c b/drivers/mailbox/mtk-cmdq-mailbox.c
> index f1dfce9e27f5..002003c3970a 100644
> --- a/drivers/mailbox/mtk-cmdq-mailbox.c
> +++ b/drivers/mailbox/mtk-cmdq-mailbox.c
> @@ -370,6 +370,8 @@ static void cmdq_remove(struct platform_device *pdev)
>   {
>   	struct cmdq *cmdq = platform_get_drvdata(pdev);
>   
> +	devm_mbox_controller_unregister(&pdev->dev, &cmdq->mbox);

...if you moved devm_mbox_controller_register() after devm_pm_runtime_enable(),
this means that the teardown sequence will disable pm runtime after unregistering
the mbox controller, so why do you need to call devm_mbox_controller_unregister()
manually here?

Cheers,
Angelo

> +
>   	if (cmdq->pdata->sw_ddr_en)
>   		cmdq_sw_ddr_enable(cmdq, false);
>   
> @@ -689,12 +691,6 @@ static int cmdq_probe(struct platform_device *pdev)
>   		cmdq->mbox.chans[i].con_priv = (void *)&cmdq->thread[i];
>   	}
>   
> -	err = devm_mbox_controller_register(dev, &cmdq->mbox);
> -	if (err < 0) {
> -		dev_err(dev, "failed to register mailbox: %d\n", err);
> -		return err;
> -	}
> -
>   	platform_set_drvdata(pdev, cmdq);
>   
>   	WARN_ON(clk_bulk_prepare(cmdq->pdata->gce_num, cmdq->clocks));
> @@ -722,6 +718,12 @@ static int cmdq_probe(struct platform_device *pdev)
>   	pm_runtime_set_autosuspend_delay(dev, CMDQ_MBOX_AUTOSUSPEND_DELAY_MS);
>   	pm_runtime_use_autosuspend(dev);
>   
> +	err = devm_mbox_controller_register(dev, &cmdq->mbox);
> +	if (err < 0) {
> +		dev_err(dev, "failed to register mailbox: %d\n", err);
> +		return err;
> +	}
> +
>   	return 0;
>   }
>   
> 
> ---
> base-commit: 797012914d2d031430268fe512af0ccd7d8e46ef
> change-id: 20240718-mtk-cmdq-fixup-40379650e96b
> 
> Best regards,

Hi Angelo,

Thanks for the reviews.

[snip]
> > ---
> >   drivers/mailbox/mtk-cmdq-mailbox.c | 14 ++++++++------
> >   1 file changed, 8 insertions(+), 6 deletions(-)
> > 
> > diff --git a/drivers/mailbox/mtk-cmdq-mailbox.c
> > b/drivers/mailbox/mtk-cmdq-mailbox.c
> > index f1dfce9e27f5..002003c3970a 100644
> > --- a/drivers/mailbox/mtk-cmdq-mailbox.c
> > +++ b/drivers/mailbox/mtk-cmdq-mailbox.c
> > @@ -370,6 +370,8 @@ static void cmdq_remove(struct platform_device
> > *pdev)
> >   {
> >   	struct cmdq *cmdq = platform_get_drvdata(pdev);
> >   
> > +	devm_mbox_controller_unregister(&pdev->dev, &cmdq->mbox);
> 
> ...if you moved devm_mbox_controller_register() after
> devm_pm_runtime_enable(),
> this means that the teardown sequence will disable pm runtime after
> unregistering
> the mbox controller, so why do you need to call
> devm_mbox_controller_unregister()
> manually here?
> 

Since this is the first time I use b4 --manul-rerell to the v1 path, I
forgot to remove the change in v1.

I'll remove this and send v3 soon.

Regards,
Jason-JH.Lin


> Cheers,
> Angelo