[PATCH v6 0/8] Fix RK3588 power domain problems

Sebastian Reichel posted 8 patches 9 months, 3 weeks ago
.../bindings/power/rockchip,power-controller.yaml  |   3 +
.../arm64/boot/dts/rockchip/rk3588-armsom-lm7.dtsi |   4 +
.../boot/dts/rockchip/rk3588-armsom-sige7.dts      |   4 +
arch/arm64/boot/dts/rockchip/rk3588-base.dtsi      |   2 +-
.../arm64/boot/dts/rockchip/rk3588-coolpi-cm5.dtsi |   4 +
.../dts/rockchip/rk3588-edgeble-neu6a-common.dtsi  |   4 +
arch/arm64/boot/dts/rockchip/rk3588-evb1-v10.dts   |   4 +
arch/arm64/boot/dts/rockchip/rk3588-fet3588-c.dtsi |   4 +
.../dts/rockchip/rk3588-firefly-core-3588j.dtsi    |   4 +
.../dts/rockchip/rk3588-friendlyelec-cm3588.dtsi   |   4 +
.../arm64/boot/dts/rockchip/rk3588-h96-max-v58.dts |   4 +
arch/arm64/boot/dts/rockchip/rk3588-jaguar.dts     |   4 +
arch/arm64/boot/dts/rockchip/rk3588-nanopc-t6.dtsi |   4 +
arch/arm64/boot/dts/rockchip/rk3588-ok3588-c.dts   |   4 +
.../arm64/boot/dts/rockchip/rk3588-orangepi-5.dtsi |   4 +
.../arm64/boot/dts/rockchip/rk3588-quartzpro64.dts |   4 +
arch/arm64/boot/dts/rockchip/rk3588-rock-5-itx.dts |   4 +
arch/arm64/boot/dts/rockchip/rk3588-rock-5b.dts    |   4 +
arch/arm64/boot/dts/rockchip/rk3588-tiger.dtsi     |   4 +
.../arm64/boot/dts/rockchip/rk3588-toybrick-x0.dts |   4 +
.../arm64/boot/dts/rockchip/rk3588-turing-rk1.dtsi |   4 +
arch/arm64/boot/dts/rockchip/rk3588s-coolpi-4b.dts |   4 +
arch/arm64/boot/dts/rockchip/rk3588s-evb1-v10.dts  |   4 +
.../boot/dts/rockchip/rk3588s-gameforce-ace.dts    |   4 +
.../boot/dts/rockchip/rk3588s-indiedroid-nova.dts  |   4 +
.../boot/dts/rockchip/rk3588s-khadas-edge2.dts     |   4 +
.../arm64/boot/dts/rockchip/rk3588s-nanopi-r6.dtsi |   4 +
arch/arm64/boot/dts/rockchip/rk3588s-odroid-m2.dts |   4 +
.../boot/dts/rockchip/rk3588s-orangepi-5.dtsi      |   4 +
arch/arm64/boot/dts/rockchip/rk3588s-rock-5a.dts   |   4 +
arch/arm64/boot/dts/rockchip/rk3588s-rock-5c.dts   |   4 +
drivers/pmdomain/rockchip/pm-domains.c             | 195 +++++++++++++--------
drivers/regulator/devres.c                         |  17 ++
drivers/regulator/of_regulator.c                   |  21 +++
include/linux/regulator/consumer.h                 |   6 +
35 files changed, 290 insertions(+), 70 deletions(-)
[PATCH v6 0/8] Fix RK3588 power domain problems
Posted by Sebastian Reichel 9 months, 3 weeks ago
Hi,

I got a report, that the Linux kernel crashes on Rock 5B when the panthor
driver is loaded late after booting. The crash starts with the following
shortened error print:

rockchip-pm-domain fd8d8000.power-management:power-controller: failed to set domain 'gpu', val=0
rockchip-pm-domain fd8d8000.power-management:power-controller: failed to get ack on domain 'gpu', val=0xa9fff
SError Interrupt on CPU4, code 0x00000000be000411 -- SError

This series first does some cleanups in the Rockchip power domain
driver and changes the driver, so that it no longer tries to continue
when it fails to enable a domain. This gets rid of the SError interrupt
and long backtraces. But the kernel still hangs when it fails to enable
a power domain. I have not done further analysis to check if that can
be avoided.

Last but not least this provides a fix for the GPU power domain failing
to get enabled - after some testing from my side it seems to require the
GPU voltage supply to be enabled.

This introduces devm_of_regulator_get without the _optional suffix, since
that is more sensible for the Rockchip usecase. Longer explanation can be
seen in patch 6, which adds the handling to the Rockchip driver. My merge
suggestion would be that Mark provides an immutable branch to Ulf.

The last patch, which updates the RK3588 board files should cover all RK3588
boards that are currently in Heiko's for-next branch. Any board missing the
update will behave as before, so it is perfectly fine not to update all DT
files at once (in case I missed any).

This is based on Heiko's for-next branch. Also it's probably worth mentioning,
that mesa CI is carrying this patchset for quite some time now.

Changes since PATCHv5:
 * https://lore.kernel.org/linux-rockchip/20241211143044.9550-1-sebastian.reichel@collabora.com/
 * Rebase to Heiko's for-next branch
 * Integrate the patch from Peter Geis into this series
   - https://lore.kernel.org/linux-rockchip/20241214215802.23989-1-pgwipeout@gmail.com/

Changes since PATCHv4:
 * https://lore.kernel.org/linux-rockchip/20241022154508.63563-1-sebastian.reichel@collabora.com/
 * Rebase to Heiko's for-next branch
   - update DT patch to handle new RK3588(s) boards
   - make sure to use a clean topic branch without HDMI-RX code (Heiko Stübner)
 * Add Tested-by from Heiko Stübner

Changes since PATCHv3:
 * accidently none, sorry!

Changes since PATCHv2:
 * https://lore.kernel.org/linux-rockchip/20240919091834.83572-1-sebastian.reichel@collabora.com/
 * Rebase to 6.12-rc1 + devm_of_regulator_get_optional branch (Ulf Hansson, Chen-Yu Tsai)
  - Introduce devm_of_regulator_get()
  - Add code to only request regulators for domains needing them
 * Mention other platforms in the DT binding patch (Rob Murphy)
 * Update more RK3588 DT files (Jonas Karlman)

Changes since PATCHv1:
 * https://lore.kernel.org/all/20240910180530.47194-1-sebastian.reichel@collabora.com/
 * Collect Reviewed-by/Acked-by/Tested-by
 * swap first and second patch to avoid introducing and directly removing a mutex_unlock
 * fix spelling of indentation
 * fix double empty line after rockchip_pd_regulator_disable()

Greetings,

-- Sebastian

---
Peter Geis (1):
      pmdomain: rockchip: fix rockchip_pd_power error handling

Sebastian Reichel (7):
      regulator: Add (devm_)of_regulator_get()
      pmdomain: rockchip: cleanup mutex handling in rockchip_pd_power
      pmdomain: rockchip: forward rockchip_do_pmu_set_power_domain errors
      pmdomain: rockchip: reduce indentation in rockchip_pd_power
      dt-bindings: power: rockchip: add regulator support
      pmdomain: rockchip: add regulator support
      arm64: dts: rockchip: Add GPU power domain regulator dependency for RK3588

 .../bindings/power/rockchip,power-controller.yaml  |   3 +
 .../arm64/boot/dts/rockchip/rk3588-armsom-lm7.dtsi |   4 +
 .../boot/dts/rockchip/rk3588-armsom-sige7.dts      |   4 +
 arch/arm64/boot/dts/rockchip/rk3588-base.dtsi      |   2 +-
 .../arm64/boot/dts/rockchip/rk3588-coolpi-cm5.dtsi |   4 +
 .../dts/rockchip/rk3588-edgeble-neu6a-common.dtsi  |   4 +
 arch/arm64/boot/dts/rockchip/rk3588-evb1-v10.dts   |   4 +
 arch/arm64/boot/dts/rockchip/rk3588-fet3588-c.dtsi |   4 +
 .../dts/rockchip/rk3588-firefly-core-3588j.dtsi    |   4 +
 .../dts/rockchip/rk3588-friendlyelec-cm3588.dtsi   |   4 +
 .../arm64/boot/dts/rockchip/rk3588-h96-max-v58.dts |   4 +
 arch/arm64/boot/dts/rockchip/rk3588-jaguar.dts     |   4 +
 arch/arm64/boot/dts/rockchip/rk3588-nanopc-t6.dtsi |   4 +
 arch/arm64/boot/dts/rockchip/rk3588-ok3588-c.dts   |   4 +
 .../arm64/boot/dts/rockchip/rk3588-orangepi-5.dtsi |   4 +
 .../arm64/boot/dts/rockchip/rk3588-quartzpro64.dts |   4 +
 arch/arm64/boot/dts/rockchip/rk3588-rock-5-itx.dts |   4 +
 arch/arm64/boot/dts/rockchip/rk3588-rock-5b.dts    |   4 +
 arch/arm64/boot/dts/rockchip/rk3588-tiger.dtsi     |   4 +
 .../arm64/boot/dts/rockchip/rk3588-toybrick-x0.dts |   4 +
 .../arm64/boot/dts/rockchip/rk3588-turing-rk1.dtsi |   4 +
 arch/arm64/boot/dts/rockchip/rk3588s-coolpi-4b.dts |   4 +
 arch/arm64/boot/dts/rockchip/rk3588s-evb1-v10.dts  |   4 +
 .../boot/dts/rockchip/rk3588s-gameforce-ace.dts    |   4 +
 .../boot/dts/rockchip/rk3588s-indiedroid-nova.dts  |   4 +
 .../boot/dts/rockchip/rk3588s-khadas-edge2.dts     |   4 +
 .../arm64/boot/dts/rockchip/rk3588s-nanopi-r6.dtsi |   4 +
 arch/arm64/boot/dts/rockchip/rk3588s-odroid-m2.dts |   4 +
 .../boot/dts/rockchip/rk3588s-orangepi-5.dtsi      |   4 +
 arch/arm64/boot/dts/rockchip/rk3588s-rock-5a.dts   |   4 +
 arch/arm64/boot/dts/rockchip/rk3588s-rock-5c.dts   |   4 +
 drivers/pmdomain/rockchip/pm-domains.c             | 195 +++++++++++++--------
 drivers/regulator/devres.c                         |  17 ++
 drivers/regulator/of_regulator.c                   |  21 +++
 include/linux/regulator/consumer.h                 |   6 +
 35 files changed, 290 insertions(+), 70 deletions(-)
---
base-commit: 03fa1896664722f35cdfc41a37b3ab17d60bd66e
change-id: 20250220-rk3588-gpu-pwr-domain-regulator-fe02632deb01

Best regards,
-- 
Sebastian Reichel <sre@kernel.org>

Re: [PATCH v6 0/8] Fix RK3588 power domain problems
Posted by Ulf Hansson 9 months, 2 weeks ago
On Thu, 20 Feb 2025 at 19:58, Sebastian Reichel
<sebastian.reichel@collabora.com> wrote:
>
> Hi,
>
> I got a report, that the Linux kernel crashes on Rock 5B when the panthor
> driver is loaded late after booting. The crash starts with the following
> shortened error print:
>
> rockchip-pm-domain fd8d8000.power-management:power-controller: failed to set domain 'gpu', val=0
> rockchip-pm-domain fd8d8000.power-management:power-controller: failed to get ack on domain 'gpu', val=0xa9fff
> SError Interrupt on CPU4, code 0x00000000be000411 -- SError
>
> This series first does some cleanups in the Rockchip power domain
> driver and changes the driver, so that it no longer tries to continue
> when it fails to enable a domain. This gets rid of the SError interrupt
> and long backtraces. But the kernel still hangs when it fails to enable
> a power domain. I have not done further analysis to check if that can
> be avoided.
>
> Last but not least this provides a fix for the GPU power domain failing
> to get enabled - after some testing from my side it seems to require the
> GPU voltage supply to be enabled.
>
> This introduces devm_of_regulator_get without the _optional suffix, since
> that is more sensible for the Rockchip usecase. Longer explanation can be
> seen in patch 6, which adds the handling to the Rockchip driver. My merge
> suggestion would be that Mark provides an immutable branch to Ulf.
>
> The last patch, which updates the RK3588 board files should cover all RK3588
> boards that are currently in Heiko's for-next branch. Any board missing the
> update will behave as before, so it is perfectly fine not to update all DT
> files at once (in case I missed any).
>
> This is based on Heiko's for-next branch. Also it's probably worth mentioning,
> that mesa CI is carrying this patchset for quite some time now.

[...]

I have pulled patch 1 from Mark's tree and applied patch 2 -> patch 7 for next.

Note that, patch6 (the DT patch) is also available on the immutable dt branch.

Thanks and kind regards
Uffe
Re: (subset) [PATCH v6 0/8] Fix RK3588 power domain problems
Posted by Mark Brown 9 months, 3 weeks ago
On Thu, 20 Feb 2025 19:58:03 +0100, Sebastian Reichel wrote:
> I got a report, that the Linux kernel crashes on Rock 5B when the panthor
> driver is loaded late after booting. The crash starts with the following
> shortened error print:
> 
> rockchip-pm-domain fd8d8000.power-management:power-controller: failed to set domain 'gpu', val=0
> rockchip-pm-domain fd8d8000.power-management:power-controller: failed to get ack on domain 'gpu', val=0xa9fff
> SError Interrupt on CPU4, code 0x00000000be000411 -- SError
> 
> [...]

Applied to

   https://git.kernel.org/pub/scm/linux/kernel/git/broonie/regulator.git for-next

Thanks!

[1/8] regulator: Add (devm_)of_regulator_get()
      commit: 0dffacbbf8d044456d50c893adb9499775c489f4

All being well this means that it will be integrated into the linux-next
tree (usually sometime in the next 24 hours) and sent to Linus during
the next merge window (or sooner if it is a bug fix), however if
problems are discovered then the patch may be dropped or reverted.

You may get further e-mails resulting from automated or manual testing
and review of the tree, please engage with people reporting problems and
send followup patches addressing any issues that are reported if needed.

If any updates are required or you are submitting further changes they
should be sent as incremental updates against current git, existing
patches will not be replaced.

Please add any relevant lists and maintainers to the CCs when replying
to this mail.

Thanks,
Mark
Re: (subset) [PATCH v6 0/8] Fix RK3588 power domain problems
Posted by Heiko Stuebner 9 months, 2 weeks ago
On Thu, 20 Feb 2025 19:58:03 +0100, Sebastian Reichel wrote:
> I got a report, that the Linux kernel crashes on Rock 5B when the panthor
> driver is loaded late after booting. The crash starts with the following
> shortened error print:
> 
> rockchip-pm-domain fd8d8000.power-management:power-controller: failed to set domain 'gpu', val=0
> rockchip-pm-domain fd8d8000.power-management:power-controller: failed to get ack on domain 'gpu', val=0xa9fff
> SError Interrupt on CPU4, code 0x00000000be000411 -- SError
> 
> [...]

Applied, thanks!

[8/8] arm64: dts: rockchip: Add GPU power domain regulator dependency for RK3588
      commit: f94500eb7328b35f3d0927635b1aba26c85ea4b0


Best regards,
-- 
Heiko Stuebner <heiko@sntech.de>