include/linux/iommu.h | 8 ++- include/trace/events/iommu.h | 7 -- drivers/iommu/amd/iommu.c | 26 ------- drivers/iommu/apple-dart.c | 24 ------- drivers/iommu/arm/arm-smmu/qcom_iommu.c | 23 ------ drivers/iommu/exynos-iommu.c | 1 - drivers/iommu/fsl_pamu_domain.c | 6 +- drivers/iommu/iommu-traces.c | 1 - drivers/iommu/iommu.c | 94 ++++++++++++------------- drivers/iommu/ipmmu-vmsa.c | 16 ----- drivers/iommu/msm_iommu.c | 6 +- drivers/iommu/mtk_iommu.c | 9 --- drivers/iommu/mtk_iommu_v1.c | 4 +- drivers/iommu/omap-iommu.c | 6 +- drivers/iommu/rockchip-iommu.c | 1 - drivers/iommu/s390-iommu.c | 7 +- drivers/iommu/sprd-iommu.c | 16 ----- drivers/iommu/sun50i-iommu.c | 1 - drivers/iommu/tegra-gart.c | 6 +- drivers/iommu/tegra-smmu.c | 5 +- 20 files changed, 69 insertions(+), 198 deletions(-)
Hi folks,
The iommu core calls the driver's detach_dev domain op callback only when
a device is finished assigning to user space and
iommu_group_release_dma_owner() is called to return the device to the
kernel, where iommu core wants to set the default domain to the device but
the driver didn't provide one. The code looks like:
/*
* New drivers should support default domains and so the detach_dev() op
* will never be called. Otherwise the NULL domain represents some
* platform specific behavior.
*/
if (!new_domain) {
if (WARN_ON(!group->domain->ops->detach_dev))
return -EINVAL;
__iommu_group_for_each_dev(group, group->domain,
iommu_group_do_detach_device);
group->domain = NULL;
return 0;
}
In other words, if the iommu driver provides default domains, the
.detach_dev callback will never be called; Otherwise, the .detach_dev
callback is actually called to return control back to the platform DMA
ops, other than detaching the domain from device.
This series cleanups this by:
- If the IOMMU driver provides default domains, remove .detach_dev
callback.
- Adds a new set_platform_dma iommu op. Any IOMMU driver that doesn't
provide default domain should implement set_platform_dma callback
instead.
- Retire .detach_dev callback.
This series originates from various discussion in the community. Thanks
to Jason, Robin and all others for their ideas.
The whole series is available on github:
https://github.com/LuBaolu/intel-iommu/commits/iommu-retire-detach_dev-v5
Change log:
v5:
- Merge some patches to make the series cute. No functionality changes.
- Check the op directly and WARN_ON the lack of any necessary
callbacks. Get rid of the ret and EINVAL.
v4:
- https://lore.kernel.org/linux-iommu/20230104125725.271850-1-baolu.lu@linux.intel.com/
- Drop the patch which renamed .attach_dev to .set_dev. As Robin said,
"... I also wouldn't say that "attach" loses its meaning in a context
where an equivalent "detach" operation is only ever implicit in
reattaching to something else...". If we have a better name in the
future, we can do it in other series.
- Adjust the patch of "iommu: Add set_platform_dma_ops iommu ops"
according to Jason's following suggestion " ... This is a bit ugly,
it would be better to make the previous patch call set_platform_dma
if it is set instead of detach_dev and then these patches should just
rename the driver's fsl_pamu_detach_device to
fsl_pamu_set_platform_dma ..."
- Add a new patch to remove deferred attach check from
__iommu_detach_domain() path. Make it a separate patch as the
prerequisite to remove __iommu_detach_device() helper.
- Rename set_platform_dma to set_platform_dma_ops to make it more
meaningful.
v3:
- https://lore.kernel.org/linux-iommu/20221128064648.1934720-1-baolu.lu@linux.intel.com/
- Setting blocking domain is not conceptually equal to detach_dev.
Dropped all blocking domain related staffs in the previous version.
v2:
- https://lore.kernel.org/linux-iommu/20220826123014.52709-1-baolu.lu@linux.intel.com/
- Replace .detach_dev callback with static block domain ops;
- Rename .attach_dev to .set_dev.
v1:
- https://lore.kernel.org/linux-iommu/20220516015759.2952771-1-baolu.lu@linux.intel.com/
Jason Gunthorpe (1):
iommu: Remove deferred attach check from __iommu_detach_device()
Lu Baolu (4):
iommu: Remove detach_dev callbacks
iommu: Add set_platform_dma_ops iommu ops
iommu: Add set_platform_dma_ops callbacks
iommu: Remove detach_dev callback
include/linux/iommu.h | 8 ++-
include/trace/events/iommu.h | 7 --
drivers/iommu/amd/iommu.c | 26 -------
drivers/iommu/apple-dart.c | 24 -------
drivers/iommu/arm/arm-smmu/qcom_iommu.c | 23 ------
drivers/iommu/exynos-iommu.c | 1 -
drivers/iommu/fsl_pamu_domain.c | 6 +-
drivers/iommu/iommu-traces.c | 1 -
drivers/iommu/iommu.c | 94 ++++++++++++-------------
drivers/iommu/ipmmu-vmsa.c | 16 -----
drivers/iommu/msm_iommu.c | 6 +-
drivers/iommu/mtk_iommu.c | 9 ---
drivers/iommu/mtk_iommu_v1.c | 4 +-
drivers/iommu/omap-iommu.c | 6 +-
drivers/iommu/rockchip-iommu.c | 1 -
drivers/iommu/s390-iommu.c | 7 +-
drivers/iommu/sprd-iommu.c | 16 -----
drivers/iommu/sun50i-iommu.c | 1 -
drivers/iommu/tegra-gart.c | 6 +-
drivers/iommu/tegra-smmu.c | 5 +-
20 files changed, 69 insertions(+), 198 deletions(-)
--
2.34.1
On 10/01/2023 02:54, Lu Baolu wrote:
> Hi folks,
>
> The iommu core calls the driver's detach_dev domain op callback only when
> a device is finished assigning to user space and
> iommu_group_release_dma_owner() is called to return the device to the
> kernel, where iommu core wants to set the default domain to the device but
> the driver didn't provide one. The code looks like:
>
> /*
> * New drivers should support default domains and so the detach_dev() op
> * will never be called. Otherwise the NULL domain represents some
> * platform specific behavior.
> */
> if (!new_domain) {
> if (WARN_ON(!group->domain->ops->detach_dev))
> return -EINVAL;
> __iommu_group_for_each_dev(group, group->domain,
> iommu_group_do_detach_device);
> group->domain = NULL;
> return 0;
> }
>
> In other words, if the iommu driver provides default domains, the
> .detach_dev callback will never be called; Otherwise, the .detach_dev
> callback is actually called to return control back to the platform DMA
> ops, other than detaching the domain from device.
>
> This series cleanups this by:
>
> - If the IOMMU driver provides default domains, remove .detach_dev
> callback.
> - Adds a new set_platform_dma iommu op. Any IOMMU driver that doesn't
> provide default domain should implement set_platform_dma callback
> instead.
> - Retire .detach_dev callback.
>
> This series originates from various discussion in the community. Thanks
> to Jason, Robin and all others for their ideas.
>
> The whole series is available on github:
> https://github.com/LuBaolu/intel-iommu/commits/iommu-retire-detach_dev-v5
>
> Change log:
> v5:
> - Merge some patches to make the series cute. No functionality changes.
> - Check the op directly and WARN_ON the lack of any necessary
> callbacks. Get rid of the ret and EINVAL.
>
> v4:
> - https://lore.kernel.org/linux-iommu/20230104125725.271850-1-baolu.lu@linux.intel.com/
> - Drop the patch which renamed .attach_dev to .set_dev. As Robin said,
> "... I also wouldn't say that "attach" loses its meaning in a context
> where an equivalent "detach" operation is only ever implicit in
> reattaching to something else...". If we have a better name in the
> future, we can do it in other series.
> - Adjust the patch of "iommu: Add set_platform_dma_ops iommu ops"
> according to Jason's following suggestion " ... This is a bit ugly,
> it would be better to make the previous patch call set_platform_dma
> if it is set instead of detach_dev and then these patches should just
> rename the driver's fsl_pamu_detach_device to
> fsl_pamu_set_platform_dma ..."
> - Add a new patch to remove deferred attach check from
> __iommu_detach_domain() path. Make it a separate patch as the
> prerequisite to remove __iommu_detach_device() helper.
> - Rename set_platform_dma to set_platform_dma_ops to make it more
> meaningful.
>
> v3:
> - https://lore.kernel.org/linux-iommu/20221128064648.1934720-1-baolu.lu@linux.intel.com/
> - Setting blocking domain is not conceptually equal to detach_dev.
> Dropped all blocking domain related staffs in the previous version.
>
> v2:
> - https://lore.kernel.org/linux-iommu/20220826123014.52709-1-baolu.lu@linux.intel.com/
> - Replace .detach_dev callback with static block domain ops;
> - Rename .attach_dev to .set_dev.
>
> v1:
> - https://lore.kernel.org/linux-iommu/20220516015759.2952771-1-baolu.lu@linux.intel.com/
>
> Jason Gunthorpe (1):
> iommu: Remove deferred attach check from __iommu_detach_device()
>
> Lu Baolu (4):
> iommu: Remove detach_dev callbacks
> iommu: Add set_platform_dma_ops iommu ops
> iommu: Add set_platform_dma_ops callbacks
> iommu: Remove detach_dev callback
>
> include/linux/iommu.h | 8 ++-
> include/trace/events/iommu.h | 7 --
> drivers/iommu/amd/iommu.c | 26 -------
> drivers/iommu/apple-dart.c | 24 -------
> drivers/iommu/arm/arm-smmu/qcom_iommu.c | 23 ------
> drivers/iommu/exynos-iommu.c | 1 -
> drivers/iommu/fsl_pamu_domain.c | 6 +-
> drivers/iommu/iommu-traces.c | 1 -
> drivers/iommu/iommu.c | 94 ++++++++++++-------------
> drivers/iommu/ipmmu-vmsa.c | 16 -----
> drivers/iommu/msm_iommu.c | 6 +-
> drivers/iommu/mtk_iommu.c | 9 ---
> drivers/iommu/mtk_iommu_v1.c | 4 +-
> drivers/iommu/omap-iommu.c | 6 +-
> drivers/iommu/rockchip-iommu.c | 1 -
> drivers/iommu/s390-iommu.c | 7 +-
> drivers/iommu/sprd-iommu.c | 16 -----
> drivers/iommu/sun50i-iommu.c | 1 -
> drivers/iommu/tegra-gart.c | 6 +-
> drivers/iommu/tegra-smmu.c | 5 +-
> 20 files changed, 69 insertions(+), 198 deletions(-)
>
I hit a problem with this series on a Firefly-RK3288, I bisected down to
1b932ceddd19 ("iommu: Remove detach_dev callbacks"). The first splat is:
[ 7.227853] ------------[ cut here ]------------
[ 7.227900] WARNING: CPU: 0 PID: 9 at drivers/iommu/iommu.c:2198 __iommu_group_set_domain+0xb4/0xc8
[ 7.227920] Modules linked in:
[ 7.227935] CPU: 0 PID: 9 Comm: kworker/u8:0 Not tainted 6.3.0-rc1 #1
[ 7.227942] Hardware name: Rockchip (Device Tree)
[ 7.227948] Workqueue: events_unbound deferred_probe_work_func
[ 7.227964] unwind_backtrace from show_stack+0x10/0x14
[ 7.227978] show_stack from dump_stack_lvl+0x58/0x70
[ 7.227992] dump_stack_lvl from __warn+0x7c/0x1dc
[ 7.228008] __warn from warn_slowpath_fmt+0xbc/0x1a0
[ 7.228022] warn_slowpath_fmt from __iommu_group_set_domain+0xb4/0xc8
[ 7.228035] __iommu_group_set_domain from iommu_detach_device+0x84/0xf8
[ 7.228046] iommu_detach_device from arm_iommu_detach_device+0x24/0xc4
[ 7.228057] arm_iommu_detach_device from rockchip_drm_dma_attach_device+0x30/0x74
[ 7.228073] rockchip_drm_dma_attach_device from vop_crtc_atomic_enable+0xf8/0xab0
[ 7.228085] vop_crtc_atomic_enable from drm_atomic_helper_commit_modeset_enables+0xb0/0x2a0
[ 7.228097] drm_atomic_helper_commit_modeset_enables from drm_atomic_helper_commit_tail_rpm+0x44/0x8c
[ 7.228111] drm_atomic_helper_commit_tail_rpm from commit_tail+0x9c/0x184
[ 7.228124] commit_tail from drm_atomic_helper_commit+0x164/0x18c
[ 7.228137] drm_atomic_helper_commit from drm_atomic_commit+0xb0/0xe8
[ 7.228153] drm_atomic_commit from drm_client_modeset_commit_atomic+0x240/0x288
[ 7.228166] drm_client_modeset_commit_atomic from drm_client_modeset_commit_locked+0x60/0x1cc
[ 7.228174] drm_client_modeset_commit_locked from drm_client_modeset_commit+0x24/0x40
[ 7.228183] drm_client_modeset_commit from drm_fb_helper_set_par+0xb8/0xf8
[ 7.228197] drm_fb_helper_set_par from fbcon_init+0x2a0/0x534
[ 7.228211] fbcon_init from visual_init+0xc0/0x108
[ 7.228224] visual_init from do_bind_con_driver+0x1bc/0x3a8
[ 7.228237] do_bind_con_driver from do_take_over_console+0x134/0x1d4
[ 7.228251] do_take_over_console from do_fbcon_takeover+0x6c/0xcc
[ 7.228264] do_fbcon_takeover from fbcon_fb_registered+0x198/0x1a8
[ 7.228277] fbcon_fb_registered from register_framebuffer+0x1d0/0x268
[ 7.228292] register_framebuffer from __drm_fb_helper_initial_config_and_unlock+0x358/0x578
[ 7.228308] __drm_fb_helper_initial_config_and_unlock from drm_fbdev_client_hotplug+0x6c/0xa8
[ 7.228322] drm_fbdev_client_hotplug from drm_fbdev_generic_setup+0x84/0x174
[ 7.228335] drm_fbdev_generic_setup from rockchip_drm_bind+0x1dc/0x200
[ 7.228349] rockchip_drm_bind from try_to_bring_up_aggregate_device+0x200/0x2d8
[ 7.228368] try_to_bring_up_aggregate_device from component_master_add_with_match+0xc4/0xf8
[ 7.228380] component_master_add_with_match from rockchip_drm_platform_probe+0x150/0x254
[ 7.228392] rockchip_drm_platform_probe from platform_probe+0x5c/0xb8
[ 7.228405] platform_probe from really_probe+0xe0/0x400
[ 7.228417] really_probe from __driver_probe_device+0x9c/0x1fc
[ 7.228431] __driver_probe_device from driver_probe_device+0x30/0xc0
[ 7.228445] driver_probe_device from __device_attach_driver+0xa8/0x120
[ 7.228460] __device_attach_driver from bus_for_each_drv+0x84/0xdc
[ 7.228474] bus_for_each_drv from __device_attach+0xb0/0x20c
[ 7.228488] __device_attach from bus_probe_device+0x8c/0x90
[ 7.228502] bus_probe_device from deferred_probe_work_func+0x98/0xe0
[ 7.228515] deferred_probe_work_func from process_one_work+0x290/0x740
[ 7.228528] process_one_work from worker_thread+0x54/0x518
[ 7.228536] worker_thread from kthread+0xf0/0x110
[ 7.228548] kthread from ret_from_fork+0x14/0x2c
[ 7.228561] Exception stack(0xf084dfb0 to 0xf084dff8)
[ 7.228567] dfa0: 00000000 00000000 00000000 00000000
[ 7.228573] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 7.228579] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000
[ 7.228585] irq event stamp: 138379
[ 7.228592] hardirqs last enabled at (138385): [<c03c42ac>] vprintk_emit+0x330/0x354
[ 7.228606] hardirqs last disabled at (138390): [<c03c4268>] vprintk_emit+0x2ec/0x354
[ 7.228617] softirqs last enabled at (137258): [<c03016ac>] __do_softirq+0x2f8/0x548
[ 7.228628] softirqs last disabled at (137253): [<c0350218>] __irq_exit_rcu+0x14c/0x170
[ 7.228639] ---[ end trace 0000000000000000 ]---
(complete log attached)
I'm not sure how to debug this. From the callstack I can see that
__iommu_group_set_domain is getting a NULL new_domain which triggers
a call to __iommu_group_do_set_platform_dma which is WARNing because
there is no set_platform_dma_ops callback. The NULL new_domain is
because group->default_domain is NULL in __iommu_group_set_core_domain.
So is the logic in the first patch that there is a default domain
incorrect for this rockchip driver? Or is this callpath just hitting
the function before the default domain can be created?
Help debugging this would be appreciated - I'm not very familiar
with this area of code.
Thanks,
Steve
[ 0.000000] Booting Linux on physical CPU 0x500
[ 0.000000] Linux version 6.3.0-rc1 (stepri01@e112269-lin) (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Wed Mar 15 15:23:50 GMT 2023
[ 0.000000] CPU: ARMv7 Processor [410fc0d1] revision 1 (ARMv7), cr=10c5387d
[ 0.000000] CPU: div instructions available: patching division code
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[ 0.000000] OF: fdt: Machine model: Firefly-RK3288
[ 0.000000] Memory policy: Data cache writealloc
[ 0.000000] OF: reserved mem: 0xfe000000..0xfeffffff (16384 KiB) map non-reusable dma-unusable@fe000000
[ 0.000000] cma: Reserved 64 MiB at 0x7c000000
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000000000-0x000000002fffffff]
[ 0.000000] Normal empty
[ 0.000000] HighMem [mem 0x0000000030000000-0x000000007fffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000000000-0x000000007fffffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000007fffffff]
[ 0.000000] percpu: Embedded 17 pages/cpu s38772 r8192 d22668 u69632
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 522752
[ 0.000000] Kernel command line: console=ttyS2,115200 root=/dev/nfs nfsroot=10.1.194.34:/export/root-fs,vers=4,tcp ip=dhcp rw rootwait consoleblank=0
[ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes, linear)
[ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.000000] stackdepot: allocating hash table via alloc_large_system_hash
[ 0.000000] stackdepot hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 0.000000] Memory: 1974640K/2097152K available (16384K kernel code, 2563K rwdata, 8164K rodata, 2048K init, 8838K bss, 56976K reserved, 65536K cma-reserved, 1245184K highmem)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[ 0.000000] trace event string verifier disabled
[ 0.000000] Running RCU self tests
[ 0.000000] Running RCU synchronous self tests
[ 0.000000] rcu: Hierarchical RCU implementation.
[ 0.000000] rcu: RCU event tracing is enabled.
[ 0.000000] rcu: RCU lockdep checking is enabled.
[ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=16 to nr_cpu_ids=4.
[ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[ 0.000000] Running RCU synchronous self tests
[ 0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
[ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.000000] arch_timer: cp15 timer(s) running at 24.00MHz (phys).
[ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x588fe9dc0, max_idle_ns: 440795202592 ns
[ 0.000001] sched_clock: 56 bits at 24MHz, resolution 41ns, wraps every 4398046511097ns
[ 0.000029] Switching to timer-based delay loop, resolution 41ns
[ 0.008875] Console: colour dummy device 80x30
[ 0.009012] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
[ 0.009026] ... MAX_LOCKDEP_SUBCLASSES: 8
[ 0.009039] ... MAX_LOCK_DEPTH: 48
[ 0.009051] ... MAX_LOCKDEP_KEYS: 8192
[ 0.009062] ... CLASSHASH_SIZE: 4096
[ 0.009074] ... MAX_LOCKDEP_ENTRIES: 32768
[ 0.009085] ... MAX_LOCKDEP_CHAINS: 65536
[ 0.009097] ... CHAINHASH_SIZE: 32768
[ 0.009108] memory used by lock dependency info: 4061 kB
[ 0.009120] memory used for stack traces: 2112 kB
[ 0.009131] per task-struct memory footprint: 1536 bytes
[ 0.009353] Calibrating delay loop (skipped), value calculated using timer frequency.. 48.00 BogoMIPS (lpj=240000)
[ 0.009390] pid_max: default: 32768 minimum: 301
[ 0.010959] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes, linear)
[ 0.011000] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes, linear)
[ 0.016955] CPU: Testing write buffer coherency: ok
[ 0.017251] CPU0: Spectre v2: using BPIALL workaround
[ 0.017271] Running RCU synchronous self tests
[ 0.017296] Running RCU synchronous self tests
[ 0.019224] CPU0: thread -1, cpu 0, socket 5, mpidr 80000500
[ 0.026223] Setting up static identity map for 0x300000 - 0x3000ac
[ 0.029327] rcu: Hierarchical SRCU implementation.
[ 0.029347] rcu: Max phase no-delay instances is 1000.
[ 0.049995] smp: Bringing up secondary CPUs ...
[ 0.057012] CPU1: thread -1, cpu 1, socket 5, mpidr 80000501
[ 0.057039] CPU1: Spectre v2: using BPIALL workaround
[ 0.063763] CPU2: thread -1, cpu 2, socket 5, mpidr 80000502
[ 0.063790] CPU2: Spectre v2: using BPIALL workaround
[ 0.069461] CPU3: thread -1, cpu 3, socket 5, mpidr 80000503
[ 0.069488] CPU3: Spectre v2: using BPIALL workaround
[ 0.070149] smp: Brought up 1 node, 4 CPUs
[ 0.070178] SMP: Total of 4 processors activated (192.00 BogoMIPS).
[ 0.070201] CPU: All CPU(s) started in SVC mode.
[ 0.076141] devtmpfs: initialized
[ 0.216061] VFP support v0.3: implementor 41 architecture 3 part 30 variant d rev 0
[ 0.218303] Running RCU synchronous self tests
[ 0.218498] Running RCU synchronous self tests
[ 0.219983] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[ 0.220076] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[ 0.223789] pinctrl core: initialized pinctrl subsystem
[ 0.244007] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 0.254956] DMA: preallocated 256 KiB pool for atomic coherent allocations
[ 0.276590] thermal_sys: Registered thermal governor 'step_wise'
[ 0.279749] cpuidle: using governor menu
[ 0.281984] No ATAGs?
[ 0.282669] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers.
[ 0.282726] hw-breakpoint: maximum watchpoint size is 4 bytes.
[ 0.297778] Serial: AMBA PL011 UART driver
[ 0.413760] platform ff980000.hdmi: Fixed dependency cycle(s) with /vop@ff940000/port/endpoint@0
[ 0.413997] platform ff980000.hdmi: Fixed dependency cycle(s) with /vop@ff930000/port/endpoint@0
[ 0.497815] gpio gpiochip0: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 0.501418] rockchip-gpio ff750000.gpio: probed /pinctrl/gpio@ff750000
[ 0.504556] gpio gpiochip1: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 0.505998] rockchip-gpio ff780000.gpio: probed /pinctrl/gpio@ff780000
[ 0.508938] gpio gpiochip2: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 0.510268] rockchip-gpio ff790000.gpio: probed /pinctrl/gpio@ff790000
[ 0.513244] gpio gpiochip3: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 0.515413] rockchip-gpio ff7a0000.gpio: probed /pinctrl/gpio@ff7a0000
[ 0.519859] gpio gpiochip4: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 0.522318] rockchip-gpio ff7b0000.gpio: probed /pinctrl/gpio@ff7b0000
[ 0.525524] gpio gpiochip5: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 0.526978] rockchip-gpio ff7c0000.gpio: probed /pinctrl/gpio@ff7c0000
[ 0.530174] gpio gpiochip6: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 0.531650] rockchip-gpio ff7d0000.gpio: probed /pinctrl/gpio@ff7d0000
[ 0.536246] gpio gpiochip7: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 0.537772] rockchip-gpio ff7e0000.gpio: probed /pinctrl/gpio@ff7e0000
[ 0.541421] gpio gpiochip8: Static allocation of GPIO base is deprecated, use dynamic allocation.
[ 0.542907] rockchip-gpio ff7f0000.gpio: probed /pinctrl/gpio@ff7f0000
[ 0.664233] iommu: Default domain type: Translated
[ 0.664256] iommu: DMA domain TLB invalidation policy: strict mode
[ 0.673131] SCSI subsystem initialized
[ 0.676786] usbcore: registered new interface driver usbfs
[ 0.677199] usbcore: registered new interface driver hub
[ 0.677599] usbcore: registered new device driver usb
[ 0.685798] pps_core: LinuxPPS API ver. 1 registered
[ 0.685820] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 0.685982] PTP clock support registered
[ 0.689757] scmi_core: SCMI protocol bus registered
[ 0.706081] vgaarb: loaded
[ 2.231396] clocksource: Switched to clocksource arch_sys_counter
[ 2.316703] NET: Registered PF_INET protocol family
[ 2.317523] IP idents hash table entries: 16384 (order: 5, 131072 bytes, linear)
[ 2.322008] tcp_listen_portaddr_hash hash table entries: 512 (order: 2, 20480 bytes, linear)
[ 2.322166] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 2.322242] TCP established hash table entries: 8192 (order: 3, 32768 bytes, linear)
[ 2.322723] TCP bind hash table entries: 8192 (order: 7, 655360 bytes, linear)
[ 2.326102] TCP: Hash tables configured (established 8192 bind 8192)
[ 2.326787] UDP hash table entries: 512 (order: 3, 49152 bytes, linear)
[ 2.327103] UDP-Lite hash table entries: 512 (order: 3, 49152 bytes, linear)
[ 2.328248] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 2.333607] RPC: Registered named UNIX socket transport module.
[ 2.333662] RPC: Registered udp transport module.
[ 2.333682] RPC: Registered tcp transport module.
[ 2.333701] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 2.333737] PCI: CLS 0 bytes, default 64
[ 2.907256] hw perfevents: enabled with armv7_cortex_a12 PMU driver, 7 counters available
[ 2.920408] Initialise system trusted keyrings
[ 2.921755] workingset: timestamp_bits=30 max_order=19 bucket_order=0
[ 2.926993] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 2.929892] NFS: Registering the id_resolver key type
[ 2.930113] Key type id_resolver registered
[ 2.930164] Key type id_legacy registered
[ 2.930433] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[ 2.930534] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
[ 2.932429] Key type asymmetric registered
[ 2.932545] Asymmetric key parser 'x509' registered
[ 2.933430] bounce: pool size: 64 pages
[ 2.933892] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)
[ 2.933998] io scheduler mq-deadline registered
[ 2.934026] io scheduler kyber registered
[ 2.934214] io scheduler bfq registered
[ 3.142461] dma-pl330 ff250000.dma-controller: Loaded driver for PL330 DMAC-241330
[ 3.142494] dma-pl330 ff250000.dma-controller: DBUFF-128x8bytes Num_Chans-8 Num_Peri-20 Num_Events-16
[ 3.151849] dma-pl330 ffb20000.dma-controller: Loaded driver for PL330 DMAC-241330
[ 3.151880] dma-pl330 ffb20000.dma-controller: DBUFF-64x8bytes Num_Chans-5 Num_Peri-6 Num_Events-10
[ 3.672388] Serial: 8250/16550 driver, 5 ports, IRQ sharing enabled
[ 3.701850] ff180000.serial: ttyS0 at MMIO 0xff180000 (irq = 47, base_baud = 1500000) is a 16550A
[ 3.713471] ff190000.serial: ttyS1 at MMIO 0xff190000 (irq = 48, base_baud = 1500000) is a 16550A
[ 3.724188] ff690000.serial: ttyS2 at MMIO 0xff690000 (irq = 49, base_baud = 1500000) is a 16550A
[ 3.725100] printk: console [ttyS2] enabled
[ 4.883174] ff1b0000.serial: ttyS3 at MMIO 0xff1b0000 (irq = 50, base_baud = 1500000) is a 16550A
[ 4.906393] SuperH (H)SCI(F) driver initialized
[ 4.917153] msm_serial: driver initialized
[ 4.921839] STMicroelectronics ASC driver initialized
[ 4.933244] STM32 USART driver initialized
[ 4.978503] rockchip-vop ff930000.vop: Adding to iommu group 0
[ 4.988866] rockchip-vop ff940000.vop: Adding to iommu group 1
[ 5.017116] rockchip-drm display-subsystem: bound ff930000.vop (ops vop_component_ops)
[ 5.028500] rockchip-drm display-subsystem: bound ff940000.vop (ops vop_component_ops)
[ 5.038287] dwhdmi-rockchip ff980000.hdmi: supply avdd-0v9 not found, using dummy regulator
[ 5.050040] dwhdmi-rockchip ff980000.hdmi: supply avdd-1v8 not found, using dummy regulator
[ 5.166130] brd: module loaded
[ 5.239189] loop: module loaded
[ 5.328714] CAN device driver interface
[ 5.337879] bgmac_bcma: Broadcom 47xx GBit MAC driver loaded
[ 5.350032] e1000e: Intel(R) PRO/1000 Network Driver
[ 5.355698] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[ 5.362738] igb: Intel(R) Gigabit Ethernet Network Driver
[ 5.368807] igb: Copyright (c) 2007-2014 Intel Corporation.
[ 5.399131] pegasus: Pegasus/Pegasus II USB Ethernet driver
[ 5.405747] usbcore: registered new interface driver pegasus
[ 5.412492] usbcore: registered new interface driver asix
[ 5.418856] usbcore: registered new interface driver ax88179_178a
[ 5.426070] usbcore: registered new interface driver cdc_ether
[ 5.432996] usbcore: registered new interface driver smsc75xx
[ 5.439756] usbcore: registered new interface driver smsc95xx
[ 5.446562] usbcore: registered new interface driver net1080
[ 5.453284] usbcore: registered new interface driver cdc_subset
[ 5.460232] usbcore: registered new interface driver zaurus
[ 5.466881] usbcore: registered new interface driver cdc_ncm
[ 5.473627] usbcore: registered new interface driver r8153_ecm
[ 5.492283] dwc2 ff540000.usb: supply vusb_d not found, using dummy regulator
[ 5.501797] dwc2 ff540000.usb: supply vusb_a not found, using dummy regulator
[ 5.582878] dwc2 ff540000.usb: DWC OTG Controller
[ 5.589070] dwc2 ff540000.usb: new USB bus registered, assigned bus number 1
[ 5.597414] dwc2 ff540000.usb: irq 54, io mem 0xff540000
[ 5.614258] hub 1-0:1.0: USB hub found
[ 5.619130] hub 1-0:1.0: 1 port detected
[ 5.632400] dwc2 ff580000.usb: supply vusb_d not found, using dummy regulator
[ 5.642350] dwc2 ff580000.usb: supply vusb_a not found, using dummy regulator
[ 5.791720] dwc2 ff580000.usb: EPs: 10, dedicated fifos, 972 entries in SPRAM
[ 5.802858] dwc2 ff580000.usb: DWC OTG Controller
[ 5.808320] dwc2 ff580000.usb: new USB bus registered, assigned bus number 2
[ 5.816499] dwc2 ff580000.usb: irq 55, io mem 0xff580000
[ 5.829077] hub 2-0:1.0: USB hub found
[ 5.833780] hub 2-0:1.0: 1 port detected
[ 5.859849] usbcore: registered new interface driver usb-storage
[ 5.898560] i2c_dev: i2c /dev entries driver
[ 5.935128] fan53555-regulator 0-0040: FAN53555 Option[8] Rev[1] Detected!
[ 5.953063] fan53555-regulator 0-0041: FAN53555 Option[8] Rev[1] Detected!
[ 5.969323] i2c 0-005a: Fixed dependency cycle(s) with /i2c@ff650000/act8846@5a/regulators/REG4
[ 6.006658] vcca_18: Bringing 3300000uV into 1800000-1800000uV
[ 6.114025] dw_wdt ff800000.watchdog: No valid TOPs array specified
[ 6.170777] cpufreq: cpufreq_online: CPU0: Running at unlisted initial frequency: 500000 KHz, changing to: 600000 KHz
[ 6.195226] sdhci: Secure Digital Host Controller Interface driver
[ 6.202228] sdhci: Copyright(c) Pierre Ossman
[ 6.214956] Synopsys Designware Multimedia Card Interface Driver
[ 6.226942] dwmmc_rockchip ff0d0000.mmc: IDMAC supports 32-bit address mode.
[ 6.228618] sdhci-pltfm: SDHCI platform and OF driver helper
[ 6.258238] dwmmc_rockchip ff0d0000.mmc: Using internal DMA controller.
[ 6.265719] dwmmc_rockchip ff0d0000.mmc: Version ID is 270a
[ 6.272169] dwmmc_rockchip ff0d0000.mmc: DW MMC controller at irq 63,32 bit host data width,256 deep fifo
[ 6.284539] ledtrig-cpu: registered to indicate activity on CPUs
[ 6.291687] dwmmc_rockchip ff0d0000.mmc: Failed getting OCR mask: 0
[ 6.293330] usbcore: registered new interface driver usbhid
[ 6.298836] mmc_host mmc0: card is non-removable.
[ 6.305119] usbhid: USB HID core driver
[ 6.310877] dwmmc_rockchip ff0d0000.mmc: could not set regulator OCR (-22)
[ 6.322233] dwmmc_rockchip ff0d0000.mmc: failed to enable vmmc regulator
[ 6.324503] NET: Registered PF_INET6 protocol family
[ 6.337580] Segment Routing with IPv6
[ 6.341785] In-situ OAM (IOAM) with IPv6
[ 6.346271] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[ 6.348651] mmc_host mmc0: Bus speed (slot 0) = 400000Hz (slot req 400000Hz, actual 400000HZ div = 0)
[ 6.354669] NET: Registered PF_PACKET protocol family
[ 6.369011] can: controller area network core
[ 6.374063] NET: Registered PF_CAN protocol family
[ 6.379578] can: raw protocol
[ 6.383030] can: broadcast manager protocol
[ 6.387773] can: netlink gateway - max_hops=1
[ 6.393415] Key type dns_resolver registered
[ 6.398335] ThumbEE CPU extension supported.
[ 6.403186] Registering SWP/SWPB emulation handler
[ 6.415006] dwmmc_rockchip ff0d0000.mmc: could not set regulator OCR (-22)
[ 6.422781] dwmmc_rockchip ff0d0000.mmc: failed to enable vmmc regulator
[ 6.441355] mmc_host mmc0: Bus speed (slot 0) = 300000Hz (slot req 300000Hz, actual 300000HZ div = 0)
[ 6.443478] Loading compiled-in X.509 certificates
[ 6.483046] dwmmc_rockchip ff0d0000.mmc: could not set regulator OCR (-22)
[ 6.484810] kmemleak: Kernel memory leak detector initialized (mem pool available: 15768)
[ 6.484829] kmemleak: Automatic memory scanning thread started
[ 6.490741] dwmmc_rockchip ff0d0000.mmc: failed to enable vmmc regulator
[ 6.527413] mmc_host mmc0: Bus speed (slot 0) = 200000Hz (slot req 200000Hz, actual 200000HZ div = 0)
[ 6.575931] dwmmc_rockchip ff0d0000.mmc: could not set regulator OCR (-22)
[ 6.583685] dwmmc_rockchip ff0d0000.mmc: failed to enable vmmc regulator
[ 6.604569] mmc_host mmc0: Bus speed (slot 0) = 187500Hz (slot req 187500Hz, actual 187500HZ div = 0)
[ 6.647475] rockchip-drm display-subsystem: bound ff930000.vop (ops vop_component_ops)
[ 6.658391] rockchip-drm display-subsystem: bound ff940000.vop (ops vop_component_ops)
[ 6.667869] dwhdmi-rockchip ff980000.hdmi: supply avdd-0v9 not found, using dummy regulator
[ 6.679502] dwhdmi-rockchip ff980000.hdmi: supply avdd-1v8 not found, using dummy regulator
[ 6.690798] dwhdmi-rockchip ff980000.hdmi: Detected HDMI TX controller v2.00a with HDCP (DWC MHL PHY)
[ 6.707322] rockchip-drm display-subsystem: bound ff980000.hdmi (ops dw_hdmi_rockchip_ops)
[ 6.720472] [drm] Initialized rockchip 1.0.0 20140818 for display-subsystem on minor 0
[ 7.227853] ------------[ cut here ]------------
[ 7.227900] WARNING: CPU: 0 PID: 9 at drivers/iommu/iommu.c:2198 __iommu_group_set_domain+0xb4/0xc8
[ 7.227920] Modules linked in:
[ 7.227935] CPU: 0 PID: 9 Comm: kworker/u8:0 Not tainted 6.3.0-rc1 #1
[ 7.227942] Hardware name: Rockchip (Device Tree)
[ 7.227948] Workqueue: events_unbound deferred_probe_work_func
[ 7.227964] unwind_backtrace from show_stack+0x10/0x14
[ 7.227978] show_stack from dump_stack_lvl+0x58/0x70
[ 7.227992] dump_stack_lvl from __warn+0x7c/0x1dc
[ 7.228008] __warn from warn_slowpath_fmt+0xbc/0x1a0
[ 7.228022] warn_slowpath_fmt from __iommu_group_set_domain+0xb4/0xc8
[ 7.228035] __iommu_group_set_domain from iommu_detach_device+0x84/0xf8
[ 7.228046] iommu_detach_device from arm_iommu_detach_device+0x24/0xc4
[ 7.228057] arm_iommu_detach_device from rockchip_drm_dma_attach_device+0x30/0x74
[ 7.228073] rockchip_drm_dma_attach_device from vop_crtc_atomic_enable+0xf8/0xab0
[ 7.228085] vop_crtc_atomic_enable from drm_atomic_helper_commit_modeset_enables+0xb0/0x2a0
[ 7.228097] drm_atomic_helper_commit_modeset_enables from drm_atomic_helper_commit_tail_rpm+0x44/0x8c
[ 7.228111] drm_atomic_helper_commit_tail_rpm from commit_tail+0x9c/0x184
[ 7.228124] commit_tail from drm_atomic_helper_commit+0x164/0x18c
[ 7.228137] drm_atomic_helper_commit from drm_atomic_commit+0xb0/0xe8
[ 7.228153] drm_atomic_commit from drm_client_modeset_commit_atomic+0x240/0x288
[ 7.228166] drm_client_modeset_commit_atomic from drm_client_modeset_commit_locked+0x60/0x1cc
[ 7.228174] drm_client_modeset_commit_locked from drm_client_modeset_commit+0x24/0x40
[ 7.228183] drm_client_modeset_commit from drm_fb_helper_set_par+0xb8/0xf8
[ 7.228197] drm_fb_helper_set_par from fbcon_init+0x2a0/0x534
[ 7.228211] fbcon_init from visual_init+0xc0/0x108
[ 7.228224] visual_init from do_bind_con_driver+0x1bc/0x3a8
[ 7.228237] do_bind_con_driver from do_take_over_console+0x134/0x1d4
[ 7.228251] do_take_over_console from do_fbcon_takeover+0x6c/0xcc
[ 7.228264] do_fbcon_takeover from fbcon_fb_registered+0x198/0x1a8
[ 7.228277] fbcon_fb_registered from register_framebuffer+0x1d0/0x268
[ 7.228292] register_framebuffer from __drm_fb_helper_initial_config_and_unlock+0x358/0x578
[ 7.228308] __drm_fb_helper_initial_config_and_unlock from drm_fbdev_client_hotplug+0x6c/0xa8
[ 7.228322] drm_fbdev_client_hotplug from drm_fbdev_generic_setup+0x84/0x174
[ 7.228335] drm_fbdev_generic_setup from rockchip_drm_bind+0x1dc/0x200
[ 7.228349] rockchip_drm_bind from try_to_bring_up_aggregate_device+0x200/0x2d8
[ 7.228368] try_to_bring_up_aggregate_device from component_master_add_with_match+0xc4/0xf8
[ 7.228380] component_master_add_with_match from rockchip_drm_platform_probe+0x150/0x254
[ 7.228392] rockchip_drm_platform_probe from platform_probe+0x5c/0xb8
[ 7.228405] platform_probe from really_probe+0xe0/0x400
[ 7.228417] really_probe from __driver_probe_device+0x9c/0x1fc
[ 7.228431] __driver_probe_device from driver_probe_device+0x30/0xc0
[ 7.228445] driver_probe_device from __device_attach_driver+0xa8/0x120
[ 7.228460] __device_attach_driver from bus_for_each_drv+0x84/0xdc
[ 7.228474] bus_for_each_drv from __device_attach+0xb0/0x20c
[ 7.228488] __device_attach from bus_probe_device+0x8c/0x90
[ 7.228502] bus_probe_device from deferred_probe_work_func+0x98/0xe0
[ 7.228515] deferred_probe_work_func from process_one_work+0x290/0x740
[ 7.228528] process_one_work from worker_thread+0x54/0x518
[ 7.228536] worker_thread from kthread+0xf0/0x110
[ 7.228548] kthread from ret_from_fork+0x14/0x2c
[ 7.228561] Exception stack(0xf084dfb0 to 0xf084dff8)
[ 7.228567] dfa0: 00000000 00000000 00000000 00000000
[ 7.228573] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 7.228579] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000
[ 7.228585] irq event stamp: 138379
[ 7.228592] hardirqs last enabled at (138385): [<c03c42ac>] vprintk_emit+0x330/0x354
[ 7.228606] hardirqs last disabled at (138390): [<c03c4268>] vprintk_emit+0x2ec/0x354
[ 7.228617] softirqs last enabled at (137258): [<c03016ac>] __do_softirq+0x2f8/0x548
[ 7.228628] softirqs last disabled at (137253): [<c0350218>] __irq_exit_rcu+0x14c/0x170
[ 7.228639] ---[ end trace 0000000000000000 ]---
[ 7.228648] ------------[ cut here ]------------
[ 7.228689] WARNING: CPU: 0 PID: 9 at drivers/iommu/rockchip-iommu.c:1113 rk_iommu_domain_free+0xd0/0xd4
[ 7.228704] Modules linked in:
[ 7.228716] CPU: 0 PID: 9 Comm: kworker/u8:0 Tainted: G W 6.3.0-rc1 #1
[ 7.228723] Hardware name: Rockchip (Device Tree)
[ 7.228727] Workqueue: events_unbound deferred_probe_work_func
[ 7.228741] unwind_backtrace from show_stack+0x10/0x14
[ 7.228753] show_stack from dump_stack_lvl+0x58/0x70
[ 7.228765] dump_stack_lvl from __warn+0x7c/0x1dc
[ 7.228779] __warn from warn_slowpath_fmt+0xbc/0x1a0
[ 7.228794] warn_slowpath_fmt from rk_iommu_domain_free+0xd0/0xd4
[ 7.228809] rk_iommu_domain_free from release_iommu_mapping+0x14/0x54
[ 7.228822] release_iommu_mapping from rockchip_drm_dma_attach_device+0x38/0x74
[ 7.228835] rockchip_drm_dma_attach_device from vop_crtc_atomic_enable+0xf8/0xab0
[ 7.228848] vop_crtc_atomic_enable from drm_atomic_helper_commit_modeset_enables+0xb0/0x2a0
[ 7.228859] drm_atomic_helper_commit_modeset_enables from drm_atomic_helper_commit_tail_rpm+0x44/0x8c
[ 7.228873] drm_atomic_helper_commit_tail_rpm from commit_tail+0x9c/0x184
[ 7.228886] commit_tail from drm_atomic_helper_commit+0x164/0x18c
[ 7.228899] drm_atomic_helper_commit from drm_atomic_commit+0xb0/0xe8
[ 7.228913] drm_atomic_commit from drm_client_modeset_commit_atomic+0x240/0x288
[ 7.228925] drm_client_modeset_commit_atomic from drm_client_modeset_commit_locked+0x60/0x1cc
[ 7.228934] drm_client_modeset_commit_locked from drm_client_modeset_commit+0x24/0x40
[ 7.228942] drm_client_modeset_commit from drm_fb_helper_set_par+0xb8/0xf8
[ 7.228954] drm_fb_helper_set_par from fbcon_init+0x2a0/0x534
[ 7.228967] fbcon_init from visual_init+0xc0/0x108
[ 7.228979] visual_init from do_bind_con_driver+0x1bc/0x3a8
[ 7.228992] do_bind_con_driver from do_take_over_console+0x134/0x1d4
[ 7.229006] do_take_over_console from do_fbcon_takeover+0x6c/0xcc
[ 7.229019] do_fbcon_takeover from fbcon_fb_registered+0x198/0x1a8
[ 7.229030] fbcon_fb_registered from register_framebuffer+0x1d0/0x268
[ 7.229044] register_framebuffer from __drm_fb_helper_initial_config_and_unlock+0x358/0x578
[ 7.229060] __drm_fb_helper_initial_config_and_unlock from drm_fbdev_client_hotplug+0x6c/0xa8
[ 7.229074] drm_fbdev_client_hotplug from drm_fbdev_generic_setup+0x84/0x174
[ 7.229087] drm_fbdev_generic_setup from rockchip_drm_bind+0x1dc/0x200
[ 7.229101] rockchip_drm_bind from try_to_bring_up_aggregate_device+0x200/0x2d8
[ 7.229118] try_to_bring_up_aggregate_device from component_master_add_with_match+0xc4/0xf8
[ 7.229130] component_master_add_with_match from rockchip_drm_platform_probe+0x150/0x254
[ 7.229142] rockchip_drm_platform_probe from platform_probe+0x5c/0xb8
[ 7.229154] platform_probe from really_probe+0xe0/0x400
[ 7.229166] really_probe from __driver_probe_device+0x9c/0x1fc
[ 7.229180] __driver_probe_device from driver_probe_device+0x30/0xc0
[ 7.229194] driver_probe_device from __device_attach_driver+0xa8/0x120
[ 7.229208] __device_attach_driver from bus_for_each_drv+0x84/0xdc
[ 7.229222] bus_for_each_drv from __device_attach+0xb0/0x20c
[ 7.229236] __device_attach from bus_probe_device+0x8c/0x90
[ 7.229249] bus_probe_device from deferred_probe_work_func+0x98/0xe0
[ 7.229262] deferred_probe_work_func from process_one_work+0x290/0x740
[ 7.229274] process_one_work from worker_thread+0x54/0x518
[ 7.229282] worker_thread from kthread+0xf0/0x110
[ 7.229294] kthread from ret_from_fork+0x14/0x2c
[ 7.229305] Exception stack(0xf084dfb0 to 0xf084dff8)
[ 7.229311] dfa0: 00000000 00000000 00000000 00000000
[ 7.229317] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 7.229323] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000
[ 7.229329] irq event stamp: 138441
[ 7.229335] hardirqs last enabled at (138447): [<c03c42ac>] vprintk_emit+0x330/0x354
[ 7.229347] hardirqs last disabled at (138452): [<c03c4268>] vprintk_emit+0x2ec/0x354
[ 7.229358] softirqs last enabled at (137258): [<c03016ac>] __do_softirq+0x2f8/0x548
[ 7.229369] softirqs last disabled at (137253): [<c0350218>] __irq_exit_rcu+0x14c/0x170
[ 7.229380] ---[ end trace 0000000000000000 ]---
[ 7.229479] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:580
[ 7.229487] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 9, name: kworker/u8:0
[ 7.229495] preempt_count: 2, expected: 0
[ 7.229502] 12 locks held by kworker/u8:0/9:
[ 7.229509] #0: c3006ca8 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x1ec/0x740
[ 7.229549] #1: f084df20 (deferred_probe_work){+.+.}-{0:0}, at: process_one_work+0x1ec/0x740
[ 7.229587] #2: c3278c8c (&dev->mutex){....}-{3:3}, at: __device_attach+0x30/0x20c
[ 7.229628] #3: c1fe6064 (component_mutex){+.+.}-{3:3}, at: component_master_add_with_match+0xa4/0xf8
[ 7.229665] #4: c1f5928c (registration_lock){+.+.}-{3:3}, at: register_framebuffer+0x2c/0x268
[ 7.229706] #5: c1e2dfbc (console_lock){+.+.}-{0:0}, at: fbcon_fb_registered+0x28/0x1a8
[ 7.229745] #6: c5250970 (&helper->lock){+.+.}-{3:3}, at: drm_fb_helper_set_par+0x70/0xf8
[ 7.229786] #7: c336b0ec (&dev->master_mutex){+.+.}-{3:3}, at: drm_master_internal_acquire+0x18/0x3c
[ 7.229827] #8: c5250850 (&client->modeset_mutex){+.+.}-{3:3}, at: drm_client_modeset_commit_locked+0x20/0x1cc
[ 7.229865] #9: f084da44 (crtc_ww_class_acquire){+.+.}-{0:0}, at: drm_client_modeset_commit_atomic+0x38/0x288
[ 7.229902] #10: c37854e0 (crtc_ww_class_mutex){+.+.}-{3:3}, at: drm_modeset_backoff+0xa4/0x260
[ 7.229945] #11: c44fc55c (&vop->vop_lock){+.+.}-{3:3}, at: vop_crtc_atomic_enable+0x98/0xab0
[ 7.229983] CPU: 0 PID: 9 Comm: kworker/u8:0 Tainted: G W 6.3.0-rc1 #1
[ 7.229989] Hardware name: Rockchip (Device Tree)
[ 7.229994] Workqueue: events_unbound deferred_probe_work_func
[ 7.230007] unwind_backtrace from show_stack+0x10/0x14
[ 7.230019] show_stack from dump_stack_lvl+0x58/0x70
[ 7.230031] dump_stack_lvl from __might_resched+0x158/0x2a8
[ 7.230044] __might_resched from __mutex_lock+0x40/0xa48
[ 7.230054] __mutex_lock from mutex_lock_nested+0x1c/0x24
[ 7.230062] mutex_lock_nested from iommu_attach_device+0x30/0x8c
[ 7.230072] iommu_attach_device from rockchip_drm_dma_attach_device+0x44/0x74
[ 7.230085] rockchip_drm_dma_attach_device from vop_crtc_atomic_enable+0xf8/0xab0
[ 7.230098] vop_crtc_atomic_enable from drm_atomic_helper_commit_modeset_enables+0xb0/0x2a0
[ 7.230109] drm_atomic_helper_commit_modeset_enables from drm_atomic_helper_commit_tail_rpm+0x44/0x8c
[ 7.230123] drm_atomic_helper_commit_tail_rpm from commit_tail+0x9c/0x184
[ 7.230136] commit_tail from drm_atomic_helper_commit+0x164/0x18c
[ 7.230149] drm_atomic_helper_commit from drm_atomic_commit+0xb0/0xe8
[ 7.230163] drm_atomic_commit from drm_client_modeset_commit_atomic+0x240/0x288
[ 7.230175] drm_client_modeset_commit_atomic from drm_client_modeset_commit_locked+0x60/0x1cc
[ 7.230184] drm_client_modeset_commit_locked from drm_client_modeset_commit+0x24/0x40
[ 7.230192] drm_client_modeset_commit from drm_fb_helper_set_par+0xb8/0xf8
[ 7.230204] drm_fb_helper_set_par from fbcon_init+0x2a0/0x534
[ 7.230217] fbcon_init from visual_init+0xc0/0x108
[ 7.230229] visual_init from do_bind_con_driver+0x1bc/0x3a8
[ 7.230241] do_bind_con_driver from do_take_over_console+0x134/0x1d4
[ 7.230255] do_take_over_console from do_fbcon_takeover+0x6c/0xcc
[ 7.230268] do_fbcon_takeover from fbcon_fb_registered+0x198/0x1a8
[ 7.230279] fbcon_fb_registered from register_framebuffer+0x1d0/0x268
[ 7.230293] register_framebuffer from __drm_fb_helper_initial_config_and_unlock+0x358/0x578
[ 7.230309] __drm_fb_helper_initial_config_and_unlock from drm_fbdev_client_hotplug+0x6c/0xa8
[ 7.230323] drm_fbdev_client_hotplug from drm_fbdev_generic_setup+0x84/0x174
[ 7.230336] drm_fbdev_generic_setup from rockchip_drm_bind+0x1dc/0x200
[ 7.230351] rockchip_drm_bind from try_to_bring_up_aggregate_device+0x200/0x2d8
[ 7.230367] try_to_bring_up_aggregate_device from component_master_add_with_match+0xc4/0xf8
[ 7.230380] component_master_add_with_match from rockchip_drm_platform_probe+0x150/0x254
[ 7.230392] rockchip_drm_platform_probe from platform_probe+0x5c/0xb8
[ 7.230404] platform_probe from really_probe+0xe0/0x400
[ 7.230415] really_probe from __driver_probe_device+0x9c/0x1fc
[ 7.230430] __driver_probe_device from driver_probe_device+0x30/0xc0
[ 7.230444] driver_probe_device from __device_attach_driver+0xa8/0x120
[ 7.230458] __device_attach_driver from bus_for_each_drv+0x84/0xdc
[ 7.230472] bus_for_each_drv from __device_attach+0xb0/0x20c
[ 7.230485] __device_attach from bus_probe_device+0x8c/0x90
[ 7.230499] bus_probe_device from deferred_probe_work_func+0x98/0xe0
[ 7.230512] deferred_probe_work_func from process_one_work+0x290/0x740
[ 7.230524] process_one_work from worker_thread+0x54/0x518
[ 7.230533] worker_thread from kthread+0xf0/0x110
[ 7.230544] kthread from ret_from_fork+0x14/0x2c
[ 7.230555] Exception stack(0xf084dfb0 to 0xf084dff8)
[ 7.230561] dfa0: 00000000 00000000 00000000 00000000
[ 7.230568] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 7.230573] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000
[ 7.240398] BUG: scheduling while atomic: kworker/u8:0/9/0x00000003
[ 7.240443] 12 locks held by kworker/u8:0/9:
[ 7.240450] #0: c3006ca8 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x1ec/0x740
[ 7.240491] #1: f084df20 (deferred_probe_work){+.+.}-{0:0}, at: process_one_work+0x1ec/0x740
[ 7.240529] #2: c3278c8c (&dev->mutex){....}-{3:3}, at: __device_attach+0x30/0x20c
[ 7.240570] #3: c1fe6064 (component_mutex){+.+.}-{3:3}, at: component_master_add_with_match+0xa4/0xf8
[ 7.240608] #4: c1f5928c (registration_lock){+.+.}-{3:3}, at: register_framebuffer+0x2c/0x268
[ 7.240650] #5: c1e2dfbc (console_lock){+.+.}-{0:0}, at: fbcon_fb_registered+0x28/0x1a8
[ 7.240689] #6: c5250970 (&helper->lock){+.+.}-{3:3}, at: drm_fb_helper_set_par+0x70/0xf8
[ 7.240729] #7: c336b0ec (&dev->master_mutex){+.+.}-{3:3}, at: drm_master_internal_acquire+0x18/0x3c
[ 7.240771] #8: c5250850 (&client->modeset_mutex){+.+.}-{3:3}, at: drm_client_modeset_commit_locked+0x20/0x1cc
[ 7.240809] #9: f084da44 (crtc_ww_class_acquire){+.+.}-{0:0}, at: drm_client_modeset_commit_atomic+0x38/0x288
[ 7.240846] #10: c37854e0 (crtc_ww_class_mutex){+.+.}-{3:3}, at: drm_modeset_backoff+0xa4/0x260
[ 7.240888] #11: c37858c4 (&hdmi->mutex){+.+.}-{3:3}, at: dw_hdmi_bridge_atomic_enable+0x34/0x8c
[ 7.240930] Modules linked in:
[ 7.240942] CPU: 0 PID: 9 Comm: kworker/u8:0 Tainted: G W 6.3.0-rc1 #1
[ 7.240949] Hardware name: Rockchip (Device Tree)
[ 7.240954] Workqueue: events_unbound deferred_probe_work_func
[ 7.240968] unwind_backtrace from show_stack+0x10/0x14
[ 7.240980] show_stack from dump_stack_lvl+0x58/0x70
[ 7.240992] dump_stack_lvl from __schedule_bug+0x6c/0x80
[ 7.241005] __schedule_bug from __schedule+0xa80/0xd34
[ 7.241013] __schedule from schedule+0x58/0xf8
[ 7.241020] schedule from schedule_hrtimeout_range_clock+0x138/0x288
[ 7.241030] schedule_hrtimeout_range_clock from schedule_hrtimeout_range+0x1c/0x24
[ 7.241041] schedule_hrtimeout_range from usleep_range_state+0x88/0xe4
[ 7.241052] usleep_range_state from dw_hdmi_phy_init+0x1d8/0x310
[ 7.241065] dw_hdmi_phy_init from dw_hdmi_update_power+0x5a4/0x1980
[ 7.241080] dw_hdmi_update_power from dw_hdmi_bridge_atomic_enable+0x48/0x8c
[ 7.241096] dw_hdmi_bridge_atomic_enable from drm_atomic_bridge_chain_enable+0x48/0xa4
[ 7.241112] drm_atomic_bridge_chain_enable from drm_atomic_helper_commit_modeset_enables+0x1a0/0x2a0
[ 7.241129] drm_atomic_helper_commit_modeset_enables from drm_atomic_helper_commit_tail_rpm+0x44/0x8c
[ 7.241143] drm_atomic_helper_commit_tail_rpm from commit_tail+0x9c/0x184
[ 7.241156] commit_tail from drm_atomic_helper_commit+0x164/0x18c
[ 7.241169] drm_atomic_helper_commit from drm_atomic_commit+0xb0/0xe8
[ 7.241183] drm_atomic_commit from drm_client_modeset_commit_atomic+0x240/0x288
[ 7.241195] drm_client_modeset_commit_atomic from drm_client_modeset_commit_locked+0x60/0x1cc
[ 7.241203] drm_client_modeset_commit_locked from drm_client_modeset_commit+0x24/0x40
[ 7.241212] drm_client_modeset_commit from drm_fb_helper_set_par+0xb8/0xf8
[ 7.241224] drm_fb_helper_set_par from fbcon_init+0x2a0/0x534
[ 7.241237] fbcon_init from visual_init+0xc0/0x108
[ 7.241249] visual_init from do_bind_con_driver+0x1bc/0x3a8
[ 7.241262] do_bind_con_driver from do_take_over_console+0x134/0x1d4
[ 7.241276] do_take_over_console from do_fbcon_takeover+0x6c/0xcc
[ 7.241289] do_fbcon_takeover from fbcon_fb_registered+0x198/0x1a8
[ 7.241300] fbcon_fb_registered from register_framebuffer+0x1d0/0x268
[ 7.241315] register_framebuffer from __drm_fb_helper_initial_config_and_unlock+0x358/0x578
[ 7.241330] __drm_fb_helper_initial_config_and_unlock from drm_fbdev_client_hotplug+0x6c/0xa8
[ 7.241344] drm_fbdev_client_hotplug from drm_fbdev_generic_setup+0x84/0x174
[ 7.241357] drm_fbdev_generic_setup from rockchip_drm_bind+0x1dc/0x200
[ 7.241372] rockchip_drm_bind from try_to_bring_up_aggregate_device+0x200/0x2d8
[ 7.241389] try_to_bring_up_aggregate_device from component_master_add_with_match+0xc4/0xf8
[ 7.241401] component_master_add_with_match from rockchip_drm_platform_probe+0x150/0x254
[ 7.241413] rockchip_drm_platform_probe from platform_probe+0x5c/0xb8
[ 7.241425] platform_probe from really_probe+0xe0/0x400
[ 7.241437] really_probe from __driver_probe_device+0x9c/0x1fc
[ 7.241451] __driver_probe_device from driver_probe_device+0x30/0xc0
[ 7.241465] driver_probe_device from __device_attach_driver+0xa8/0x120
[ 7.241480] __device_attach_driver from bus_for_each_drv+0x84/0xdc
[ 7.241493] bus_for_each_drv from __device_attach+0xb0/0x20c
[ 7.241507] __device_attach from bus_probe_device+0x8c/0x90
[ 7.241520] bus_probe_device from deferred_probe_work_func+0x98/0xe0
[ 7.241534] deferred_probe_work_func from process_one_work+0x290/0x740
[ 7.241545] process_one_work from worker_thread+0x54/0x518
[ 7.241553] worker_thread from kthread+0xf0/0x110
[ 7.241565] kthread from ret_from_fork+0x14/0x2c
[ 7.241576] Exception stack(0xf084dfb0 to 0xf084dff8)
[ 7.241582] dfa0: 00000000 00000000 00000000 00000000
[ 7.241588] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 7.241594] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000
[ 7.298852] Console: switching to colour frame buffer device 240x67
[ 9.285125] rockchip-drm display-subsystem: [drm] fb0: rockchipdrmfb frame buffer device
[ 9.298291] rk_gmac-dwmac ff290000.ethernet: IRQ eth_lpi not found
[ 9.305664] rk_gmac-dwmac ff290000.ethernet: PTP uses main clock
[ 9.313433] rk_gmac-dwmac ff290000.ethernet: clock input or output? (input).
[ 9.321342] rk_gmac-dwmac ff290000.ethernet: TX delay(0x30).
[ 9.327672] rk_gmac-dwmac ff290000.ethernet: RX delay(0x10).
[ 9.334045] rk_gmac-dwmac ff290000.ethernet: integrated PHY? (no).
[ 9.341175] rk_gmac-dwmac ff290000.ethernet: cannot get clock clk_mac_speed
[ 9.348982] rk_gmac-dwmac ff290000.ethernet: clock input from PHY
[ 9.360857] rk_gmac-dwmac ff290000.ethernet: init for RGMII
[ 9.368155] rk_gmac-dwmac ff290000.ethernet: User ID: 0x10, Synopsys ID: 0x35
[ 9.376213] rk_gmac-dwmac ff290000.ethernet: DWMAC1000
[ 9.382121] rk_gmac-dwmac ff290000.ethernet: DMA HW capability register supported
[ 9.390490] rk_gmac-dwmac ff290000.ethernet: RX Checksum Offload Engine supported
[ 9.398876] rk_gmac-dwmac ff290000.ethernet: COE Type 2
[ 9.404742] rk_gmac-dwmac ff290000.ethernet: TX Checksum insertion supported
[ 9.412643] rk_gmac-dwmac ff290000.ethernet: Wake-Up On Lan supported
[ 9.420105] rk_gmac-dwmac ff290000.ethernet: Normal descriptors
[ 9.426752] rk_gmac-dwmac ff290000.ethernet: Ring mode enabled
[ 9.433293] rk_gmac-dwmac ff290000.ethernet: Enable RX Mitigation via HW Watchdog Timer
[ 10.541530] RTL8211E Gigabit Ethernet stmmac-0:00: attached PHY driver (mii_bus:phy_addr=stmmac-0:00, irq=POLL)
[ 10.552993] RTL8211E Gigabit Ethernet stmmac-0:01: attached PHY driver (mii_bus:phy_addr=stmmac-0:01, irq=POLL)
[ 10.592560] dwmmc_rockchip ff0c0000.mmc: IDMAC supports 32-bit address mode.
[ 10.599818] dwmmc_rockchip ff0f0000.mmc: IDMAC supports 32-bit address mode.
[ 10.613052] dwmmc_rockchip ff0f0000.mmc: Using internal DMA controller.
[ 10.620499] dwmmc_rockchip ff0f0000.mmc: Version ID is 270a
[ 10.627565] dwmmc_rockchip ff0f0000.mmc: DW MMC controller at irq 68,32 bit host data width,256 deep fifo
[ 10.640111] dwmmc_rockchip ff0c0000.mmc: Using internal DMA controller.
[ 10.641983] mmc_host mmc1: card is non-removable.
[ 10.647556] dwmmc_rockchip ff0c0000.mmc: Version ID is 270a
[ 10.655536] mmc_host mmc1: Bus speed (slot 0) = 400000Hz (slot req 400000Hz, actual 400000HZ div = 0)
[ 10.659216] dwmmc_rockchip ff0c0000.mmc: DW MMC controller at irq 67,32 bit host data width,256 deep fifo
[ 10.728702] mmc_host mmc1: Bus speed (slot 0) = 50000000Hz (slot req 52000000Hz, actual 50000000HZ div = 0)
[ 10.740961] mmc1: new high speed MMC card at address 0001
[ 10.766773] mmcblk1: mmc1:0001 AGND3R 14.6 GiB
[ 10.819740] input: gpio-keys as /devices/platform/gpio-keys/input/input0
[ 10.825893] mmc_host mmc2: Bus speed (slot 0) = 400000Hz (slot req 400000Hz, actual 400000HZ div = 0)
[ 10.831108] mmcblk1boot0: mmc1:0001 AGND3R 4.00 MiB
[ 10.855268] mmcblk1boot1: mmc1:0001 AGND3R 4.00 MiB
[ 10.871615] mmcblk1rpmb: mmc1:0001 AGND3R 4.00 MiB, chardev (237:0)
[ 10.882753] rk_gmac-dwmac ff290000.ethernet eth0: Register MEM_TYPE_PAGE_POOL RxQ-0
[ 10.894185] rk_gmac-dwmac ff290000.ethernet eth0: PHY [stmmac-0:00] driver [RTL8211E Gigabit Ethernet] (irq=POLL)
[ 10.905772] rk_gmac-dwmac ff290000.ethernet eth0: No Safety Features support found
[ 10.914263] rk_gmac-dwmac ff290000.ethernet eth0: PTP not supported by HW
[ 10.924135] rk_gmac-dwmac ff290000.ethernet eth0: configuring for phy/rgmii link mode
[ 10.945701] mmc_host mmc2: Bus speed (slot 0) = 50000000Hz (slot req 50000000Hz, actual 50000000HZ div = 0)
[ 10.956824] mmc2: new high speed SDHC card at address 0007
[ 10.966380] mmcblk2: mmc2:0007 SD08G 7.42 GiB
[ 10.979728] mmcblk2: p1 p2
[ 13.056934] rk_gmac-dwmac ff290000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
[ 13.068055] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 13.121326] Sending DHCP requests ., OK
[ 13.186635] IP-Config: Got DHCP answer from 10.1.194.1, my address is 10.1.194.47
[ 13.195363] IP-Config: Complete:
[ 13.199023] device=eth0, hwaddr=0e:3c:67:4e:4f:6f, ipaddr=10.1.194.47, mask=255.255.254.0, gw=10.1.194.1
[ 13.210170] host=10.1.194.47, domain=cambridge.arm.com, nis-domain=(none)
[ 13.218323] bootserver=10.1.109.185, rootserver=10.1.194.34, rootpath=
[ 13.218336] nameserver0=10.1.105.170, nameserver1=10.1.105.21, nameserver2=10.121.205.16
[ 13.235734] ntpserver0=10.58.96.12, ntpserver1=10.123.17.134, ntpserver2=10.172.89.13
[ 13.252980] dw-apb-uart ff690000.serial: forbid DMA for kernel console
[ 13.312445] VFS: Mounted root (nfs4 filesystem) on device 0:17.
[ 13.320708] devtmpfs: mounted
[ 13.326976] Freeing unused kernel image (initmem) memory: 2048K
[ 13.413210] Run /sbin/init as init process
[ 14.504339] systemd[1]: System time before build time, advancing clock.
[ 14.598430] systemd[1]: systemd 247.3-7+deb11u1 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +ZSTD +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=unified)
[ 14.626347] systemd[1]: Detected architecture arm.
Welcome to Debian GNU/Linux 11 (bullseye)!
[ 14.691189] systemd[1]: Set hostname to <stepri01-debian>.
[ 16.749290] systemd[1]: Queued start job for default target Graphical Interface.
[ 16.759945] random: systemd: uninitialized urandom read (16 bytes read)
[ 16.767549] systemd[1]: system-getty.slice: unit configures an IP firewall, but the local system does not support BPF/cgroup firewalling.
[ 16.782345] systemd[1]: (This warning is only shown for the first unit using IP firewalling.)
[ 16.795994] systemd[1]: Created slice system-getty.slice.
[ OK ] Created slice system-getty.slice.
[ 16.881253] ------------[ cut here ]------------
[ 16.886415] WARNING: CPU: 0 PID: 1 at kernel/locking/lockdep.c:232 __lock_acquire+0xb58/0x2a64
[ 16.896048] DEBUG_LOCKS_WARN_ON(1)
[ 16.896051] Modules linked in:
[ 16.903250] CPU: 0 PID: 1 Comm: systemd Tainted: G W 6.3.0-rc1 #1
[ 16.911612] Hardware name: Rockchip (Device Tree)
[ 16.916864] unwind_backtrace from show_stack+0x10/0x14
[ 16.922706] show_stack from dump_stack_lvl+0x58/0x70
[ 16.928354] dump_stack_lvl from __warn+0x7c/0x1dc
[ 16.933713] __warn from warn_slowpath_fmt+0x134/0x1a0
[ 16.939459] warn_slowpath_fmt from __lock_acquire+0xb58/0x2a64
[ 16.946079] __lock_acquire from lock_acquire+0x110/0x374
[ 16.952117] lock_acquire from _raw_spin_lock+0x40/0x50
[ 16.957958] _raw_spin_lock from add_timer_on+0x94/0x1d8
[ 16.963898] add_timer_on from try_to_generate_entropy+0x1f0/0x274
[ 16.970805] try_to_generate_entropy from urandom_read_iter+0x50/0xdc
[ 16.978004] urandom_read_iter from vfs_read+0x1c8/0x290
[ 16.983942] vfs_read from ksys_read+0x60/0xe4
[ 16.988909] ksys_read from ret_fast_syscall+0x0/0x1c
[ 16.994554] Exception stack(0xf0829fa8 to 0xf0829ff0)
[ 17.000194] 9fa0: 00000010 be946970 0000001a be946970 00000010 00000000
[ 17.009332] 9fc0: 00000010 be946970 b6f500e0 00000003 00000001 b6f2c98c 00000002 00000000
[ 17.018468] 9fe0: 00000003 be9468d8 b6b8e457 b6b147e6
[ 17.024099] irq event stamp: 909082
[ 17.027981] hardirqs last enabled at (909081): [<c126adcc>] _raw_spin_unlock_irqrestore+0x44/0x48
[ 17.037994] hardirqs last disabled at (909082): [<c126aba0>] _raw_spin_lock_irqsave+0x68/0x6c
[ 17.047522] softirqs last enabled at (909078): [<c03016ac>] __do_softirq+0x2f8/0x548
[ 17.056271] softirqs last disabled at (909071): [<c0350218>] __irq_exit_rcu+0x14c/0x170
[ 17.065216] ---[ end trace 0000000000000000 ]---
[ 17.070372] 8<--- cut here ---
[ 17.073776] Unable to handle kernel NULL pointer dereference at virtual address 00000060 when read
[ 17.083777] [00000060] *pgd=00000000
[ 17.087767] Internal error: Oops: 5 [#1] SMP ARM
[ 17.092922] Modules linked in:
[ 17.096328] CPU: 0 PID: 1 Comm: systemd Tainted: G W 6.3.0-rc1 #1
[ 17.104688] Hardware name: Rockchip (Device Tree)
[ 17.109938] PC is at __lock_acquire+0x1bc/0x2a64
[ 17.115097] LR is at __lock_acquire+0xb58/0x2a64
[ 17.120255] pc : [<c03b587c>] lr : [<c03b6218>] psr: 60010093
[ 17.127253] sp : f0829d20 ip : 00000000 fp : c3238000
[ 17.133086] r10: c24e3a20 r9 : c3238758 r8 : 00000080
[ 17.138919] r7 : e1befa47 r6 : 00000000 r5 : c1d3aad1 r4 : 00000000
[ 17.146209] r3 : 00001a47 r2 : 00000000 r1 : c2083528 r0 : 00000000
[ 17.153500] Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment none
[ 17.161568] Control: 10c5387d Table: 0626c06a DAC: 00000051
[ 17.167983] Register r0 information: NULL pointer
[ 17.173237] Register r1 information: non-slab/vmalloc memory
[ 17.179557] Register r2 information: NULL pointer
[ 17.184809] Register r3 information: non-paged memory
[ 17.190449] Register r4 information: NULL pointer
[ 17.195691] Register r5 information: non-slab/vmalloc memory
[ 17.202002] Register r6 information: NULL pointer
[ 17.207244] Register r7 information: non-slab/vmalloc memory
[ 17.213555] Register r8 information: non-paged memory
[ 17.219195] Register r9 information: slab task_struct start c3238000 pointer offset 1880 size 4032
[ 17.229215] Register r10 information: non-slab/vmalloc memory
[ 17.235633] Register r11 information: slab task_struct start c3238000 pointer offset 0 size 4032
[ 17.245457] Register r12 information: NULL pointer
[ 17.250805] Process systemd (pid: 1, stack limit = 0x(ptrval))
[ 17.257319] Stack: (0xf0829d20 to 0xf082a000)
[ 17.262182] 9d20: c19236ac ffffffff 00000000 00000001 00000000 c3238758 c26917f0 c1d3a47c
[ 17.271319] 9d40: c1e095a4 00000007 c3238000 60010093 c1d3a5bc f0829d70 2d252000 00000080
[ 17.280454] 9d60: 00000000 c1d3a5bc 00000000 a0ceac6f c3238000 60010093 c1d3a5bc f0829d98
[ 17.289582] 9d80: 2d252000 00000080 00000000 c1d3a5bc 00000000 c03b8aec 00000001 00000080
[ 17.298718] 9da0: 00000000 c03fcfac 00000000 00000000 c3238000 c3238758 00000000 c03aff5c
[ 17.307855] 9dc0: c3238000 c1d3a5bc eefbfad0 60010093 c03fcfa4 a0ceac6f 00000001 c1d3aac1
[ 17.316991] 9de0: c1d3aac1 00000010 c1d3aac1 f0829e84 c2904890 c1e09e88 c1e09050 c126a9c8
[ 17.326119] 9e00: 00000001 00000000 c03fcfac c126ad0c f0829e84 c03fcfac eefbfac0 40010013
[ 17.335255] 9e20: c3238000 a0ceac6f 40010013 f0829e80 00000010 c3238000 00000004 c1252798
[ 17.344392] 9e40: c2904898 00000080 1857d94b 0000000f 00000000 c03b8aec 00000001 00000080
[ 17.353528] 9e60: 00000000 00000000 c3238758 c125ce04 c3238000 c3238758 00000000 c03aff5c
[ 17.362664] 9e80: 1857d94b 00000122 00000000 ffff9167 c125298c 09840003 c2904898 00000000
[ 17.371792] 9ea0: 00000000 c19b8cdc c3000000 00000003 00000002 c044cf74 20010013 00000000
[ 17.380928] 9ec0: 20010013 a0ceac6f c3313000 c2904890 f0829f10 c0a9f884 00000001 00000010
[ 17.390065] 9ee0: 00000000 00004004 00000000 c0a9f8d4 c656c280 f0829f80 c0a9f884 c05153b4
[ 17.399201] 9f00: c3313000 0000001a ffffff9c c03002f0 01000006 00000000 00000010 be946970
[ 17.408328] 9f20: 00000000 00000000 c656c280 00000000 00000000 00000000 00000000 00000000
[ 17.417464] 9f40: 00000000 00004004 00000000 00000000 b6ef0c4c a0ceac6f 00000142 c656c280
[ 17.426592] 9f60: c656c280 00000000 00000000 c03002f0 c3238000 00000003 00000000 c0515f30
[ 17.435728] 9f80: 00000000 00000000 00000000 a0ceac6f c3238000 00000010 be946970 b6f500e0
[ 17.444865] 9fa0: 00000003 c03000c0 00000010 be946970 0000001a be946970 00000010 00000000
[ 17.454001] 9fc0: 00000010 be946970 b6f500e0 00000003 00000001 b6f2c98c 00000002 00000000
[ 17.463128] 9fe0: 00000003 be9468d8 b6b8e457 b6b147e6 60010030 0000001a 00000000 00000000
[ 17.472263] __lock_acquire from lock_acquire+0x110/0x374
[ 17.478300] lock_acquire from _raw_spin_lock+0x40/0x50
[ 17.484141] _raw_spin_lock from add_timer_on+0x94/0x1d8
[ 17.490078] add_timer_on from try_to_generate_entropy+0x1f0/0x274
[ 17.496985] try_to_generate_entropy from urandom_read_iter+0x50/0xdc
[ 17.504183] urandom_read_iter from vfs_read+0x1c8/0x290
[ 17.510121] vfs_read from ksys_read+0x60/0xe4
[ 17.515088] ksys_read from ret_fast_syscall+0x0/0x1c
[ 17.520732] Exception stack(0xf0829fa8 to 0xf0829ff0)
[ 17.526373] 9fa0: 00000010 be946970 0000001a be946970 00000010 00000000
[ 17.535509] 9fc0: 00000010 be946970 b6f500e0 00000003 00000001 b6f2c98c 00000002 00000000
[ 17.544645] 9fe0: 00000003 be9468d8 b6b8e457 b6b147e6
[ 17.550278] Code: 13a03064 1024a493 0a0001a2 e1d931b4 (e5d44060)
[ 17.557084] ---[ end trace 0000000000000000 ]---
[ 17.562237] note: systemd[1] exited with irqs disabled
[ 17.567994] note: systemd[1] exited with preempt_count 2
[ 17.573937] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[ 17.582492] CPU1: stopping
[ 17.585512] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D W 6.3.0-rc1 #1
[ 17.594067] Hardware name: Rockchip (Device Tree)
[ 17.599318] unwind_backtrace from show_stack+0x10/0x14
[ 17.605159] show_stack from dump_stack_lvl+0x58/0x70
[ 17.610805] dump_stack_lvl from do_handle_IPI+0x360/0x394
[ 17.616936] do_handle_IPI from ipi_handler+0x18/0x20
[ 17.622581] ipi_handler from handle_percpu_devid_irq+0xc4/0x330
[ 17.629295] handle_percpu_devid_irq from generic_handle_domain_irq+0x28/0x38
[ 17.637274] generic_handle_domain_irq from gic_handle_irq+0x88/0xa8
[ 17.644377] gic_handle_irq from generic_handle_arch_irq+0x34/0x44
[ 17.651285] generic_handle_arch_irq from call_with_stack+0x18/0x20
[ 17.658292] call_with_stack from __irq_svc+0x9c/0xb8
[ 17.663935] Exception stack(0xf0861f40 to 0xf0861f88)
[ 17.669576] 1f40: c125f1ac 00000001 00000000 00000000 c324cec0 c039963c c204e8c0 c1e08f18
[ 17.678712] 1f60: c1d416a8 c204d036 00000000 00000000 05f01d04 f0861f90 c125f1ac c125f1b0
[ 17.687838] 1f80: 60010013 ffffffff
[ 17.691728] __irq_svc from default_idle_call+0x20/0x2d8
[ 17.697666] default_idle_call from do_idle+0x22c/0x2e0
[ 17.703508] do_idle from cpu_startup_entry+0x18/0x1c
[ 17.709154] cpu_startup_entry from secondary_start_kernel+0x134/0x154
[ 17.716451] secondary_start_kernel from 0x3019a0
[ 17.721706] CPU2: stopping
[ 17.724724] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G D W 6.3.0-rc1 #1
[ 17.733279] Hardware name: Rockchip (Device Tree)
[ 17.738530] unwind_backtrace from show_stack+0x10/0x14
[ 17.744371] show_stack from dump_stack_lvl+0x58/0x70
[ 17.750017] dump_stack_lvl from do_handle_IPI+0x360/0x394
[ 17.756150] do_handle_IPI from ipi_handler+0x18/0x20
[ 17.761794] ipi_handler from handle_percpu_devid_irq+0xc4/0x330
[ 17.768508] handle_percpu_devid_irq from generic_handle_domain_irq+0x28/0x38
[ 17.776485] generic_handle_domain_irq from gic_handle_irq+0x88/0xa8
[ 17.783589] gic_handle_irq from generic_handle_arch_irq+0x34/0x44
[ 17.790497] generic_handle_arch_irq from call_with_stack+0x18/0x20
[ 17.797503] call_with_stack from __irq_svc+0x9c/0xb8
[ 17.803146] Exception stack(0xf0865f40 to 0xf0865f88)
[ 17.808787] 5f40: c125f1ac 00000001 00000000 00000000 c324de80 c039963c c204e8c0 c1e08f18
[ 17.817924] 5f60: c1d416a8 c204d036 00000000 00000000 05f0567c f0865f90 c125f1ac c125f1b0
[ 17.827049] 5f80: 600b0013 ffffffff
[ 17.830939] __irq_svc from default_idle_call+0x20/0x2d8
[ 17.836877] default_idle_call from do_idle+0x22c/0x2e0
[ 17.842718] do_idle from cpu_startup_entry+0x18/0x1c
[ 17.848362] cpu_startup_entry from secondary_start_kernel+0x134/0x154
[ 17.855659] secondary_start_kernel from 0x3019a0
[ 17.860914] CPU3: stopping
[ 17.863932] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D W 6.3.0-rc1 #1
[ 17.872488] Hardware name: Rockchip (Device Tree)
[ 17.877738] unwind_backtrace from show_stack+0x10/0x14
[ 17.883579] show_stack from dump_stack_lvl+0x58/0x70
[ 17.889224] dump_stack_lvl from do_handle_IPI+0x360/0x394
[ 17.895357] do_handle_IPI from ipi_handler+0x18/0x20
[ 17.901001] ipi_handler from handle_percpu_devid_irq+0xc4/0x330
[ 17.907715] handle_percpu_devid_irq from generic_handle_domain_irq+0x28/0x38
[ 17.915692] generic_handle_domain_irq from gic_handle_irq+0x88/0xa8
[ 17.922794] gic_handle_irq from generic_handle_arch_irq+0x34/0x44
[ 17.929701] generic_handle_arch_irq from call_with_stack+0x18/0x20
[ 17.936708] call_with_stack from __irq_svc+0x9c/0xb8
[ 17.942351] Exception stack(0xf0869f40 to 0xf0869f88)
[ 17.947992] 9f40: c125f1ac 00000001 00000000 00000000 c324ee40 c039963c c204e8c0 c1e08f18
[ 17.957129] 9f60: c1d416a8 c204d036 00000000 00000000 05efe784 f0869f90 c125f1ac c125f1b0
[ 17.966264] 9f80: 60010013 ffffffff
[ 17.970155] __irq_svc from default_idle_call+0x20/0x2d8
[ 17.976091] default_idle_call from do_idle+0x22c/0x2e0
[ 17.981934] do_idle from cpu_startup_entry+0x18/0x1c
[ 17.987579] cpu_startup_entry from secondary_start_kernel+0x134/0x154
[ 17.994876] secondary_start_kernel from 0x3019a0
[ 18.000135] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]---
On 2023-03-15 15:49, Steven Price wrote:
> On 10/01/2023 02:54, Lu Baolu wrote:
>> Hi folks,
>>
>> The iommu core calls the driver's detach_dev domain op callback only when
>> a device is finished assigning to user space and
>> iommu_group_release_dma_owner() is called to return the device to the
>> kernel, where iommu core wants to set the default domain to the device but
>> the driver didn't provide one. The code looks like:
>>
>> /*
>> * New drivers should support default domains and so the detach_dev() op
>> * will never be called. Otherwise the NULL domain represents some
>> * platform specific behavior.
>> */
>> if (!new_domain) {
>> if (WARN_ON(!group->domain->ops->detach_dev))
>> return -EINVAL;
>> __iommu_group_for_each_dev(group, group->domain,
>> iommu_group_do_detach_device);
>> group->domain = NULL;
>> return 0;
>> }
>>
>> In other words, if the iommu driver provides default domains, the
>> .detach_dev callback will never be called; Otherwise, the .detach_dev
>> callback is actually called to return control back to the platform DMA
>> ops, other than detaching the domain from device.
>>
>> This series cleanups this by:
>>
>> - If the IOMMU driver provides default domains, remove .detach_dev
>> callback.
>> - Adds a new set_platform_dma iommu op. Any IOMMU driver that doesn't
>> provide default domain should implement set_platform_dma callback
>> instead.
>> - Retire .detach_dev callback.
>>
>> This series originates from various discussion in the community. Thanks
>> to Jason, Robin and all others for their ideas.
>>
>> The whole series is available on github:
>> https://github.com/LuBaolu/intel-iommu/commits/iommu-retire-detach_dev-v5
>>
>> Change log:
>> v5:
>> - Merge some patches to make the series cute. No functionality changes.
>> - Check the op directly and WARN_ON the lack of any necessary
>> callbacks. Get rid of the ret and EINVAL.
>>
>> v4:
>> - https://lore.kernel.org/linux-iommu/20230104125725.271850-1-baolu.lu@linux.intel.com/
>> - Drop the patch which renamed .attach_dev to .set_dev. As Robin said,
>> "... I also wouldn't say that "attach" loses its meaning in a context
>> where an equivalent "detach" operation is only ever implicit in
>> reattaching to something else...". If we have a better name in the
>> future, we can do it in other series.
>> - Adjust the patch of "iommu: Add set_platform_dma_ops iommu ops"
>> according to Jason's following suggestion " ... This is a bit ugly,
>> it would be better to make the previous patch call set_platform_dma
>> if it is set instead of detach_dev and then these patches should just
>> rename the driver's fsl_pamu_detach_device to
>> fsl_pamu_set_platform_dma ..."
>> - Add a new patch to remove deferred attach check from
>> __iommu_detach_domain() path. Make it a separate patch as the
>> prerequisite to remove __iommu_detach_device() helper.
>> - Rename set_platform_dma to set_platform_dma_ops to make it more
>> meaningful.
>>
>> v3:
>> - https://lore.kernel.org/linux-iommu/20221128064648.1934720-1-baolu.lu@linux.intel.com/
>> - Setting blocking domain is not conceptually equal to detach_dev.
>> Dropped all blocking domain related staffs in the previous version.
>>
>> v2:
>> - https://lore.kernel.org/linux-iommu/20220826123014.52709-1-baolu.lu@linux.intel.com/
>> - Replace .detach_dev callback with static block domain ops;
>> - Rename .attach_dev to .set_dev.
>>
>> v1:
>> - https://lore.kernel.org/linux-iommu/20220516015759.2952771-1-baolu.lu@linux.intel.com/
>>
>> Jason Gunthorpe (1):
>> iommu: Remove deferred attach check from __iommu_detach_device()
>>
>> Lu Baolu (4):
>> iommu: Remove detach_dev callbacks
>> iommu: Add set_platform_dma_ops iommu ops
>> iommu: Add set_platform_dma_ops callbacks
>> iommu: Remove detach_dev callback
>>
>> include/linux/iommu.h | 8 ++-
>> include/trace/events/iommu.h | 7 --
>> drivers/iommu/amd/iommu.c | 26 -------
>> drivers/iommu/apple-dart.c | 24 -------
>> drivers/iommu/arm/arm-smmu/qcom_iommu.c | 23 ------
>> drivers/iommu/exynos-iommu.c | 1 -
>> drivers/iommu/fsl_pamu_domain.c | 6 +-
>> drivers/iommu/iommu-traces.c | 1 -
>> drivers/iommu/iommu.c | 94 ++++++++++++-------------
>> drivers/iommu/ipmmu-vmsa.c | 16 -----
>> drivers/iommu/msm_iommu.c | 6 +-
>> drivers/iommu/mtk_iommu.c | 9 ---
>> drivers/iommu/mtk_iommu_v1.c | 4 +-
>> drivers/iommu/omap-iommu.c | 6 +-
>> drivers/iommu/rockchip-iommu.c | 1 -
>> drivers/iommu/s390-iommu.c | 7 +-
>> drivers/iommu/sprd-iommu.c | 16 -----
>> drivers/iommu/sun50i-iommu.c | 1 -
>> drivers/iommu/tegra-gart.c | 6 +-
>> drivers/iommu/tegra-smmu.c | 5 +-
>> 20 files changed, 69 insertions(+), 198 deletions(-)
>>
>
> I hit a problem with this series on a Firefly-RK3288, I bisected down to
> 1b932ceddd19 ("iommu: Remove detach_dev callbacks"). The first splat is:
>
> [ 7.227853] ------------[ cut here ]------------
> [ 7.227900] WARNING: CPU: 0 PID: 9 at drivers/iommu/iommu.c:2198 __iommu_group_set_domain+0xb4/0xc8
> [ 7.227920] Modules linked in:
> [ 7.227935] CPU: 0 PID: 9 Comm: kworker/u8:0 Not tainted 6.3.0-rc1 #1
> [ 7.227942] Hardware name: Rockchip (Device Tree)
> [ 7.227948] Workqueue: events_unbound deferred_probe_work_func
> [ 7.227964] unwind_backtrace from show_stack+0x10/0x14
> [ 7.227978] show_stack from dump_stack_lvl+0x58/0x70
> [ 7.227992] dump_stack_lvl from __warn+0x7c/0x1dc
> [ 7.228008] __warn from warn_slowpath_fmt+0xbc/0x1a0
> [ 7.228022] warn_slowpath_fmt from __iommu_group_set_domain+0xb4/0xc8
> [ 7.228035] __iommu_group_set_domain from iommu_detach_device+0x84/0xf8
> [ 7.228046] iommu_detach_device from arm_iommu_detach_device+0x24/0xc4
> [ 7.228057] arm_iommu_detach_device from rockchip_drm_dma_attach_device+0x30/0x74
> [ 7.228073] rockchip_drm_dma_attach_device from vop_crtc_atomic_enable+0xf8/0xab0
> [ 7.228085] vop_crtc_atomic_enable from drm_atomic_helper_commit_modeset_enables+0xb0/0x2a0
> [ 7.228097] drm_atomic_helper_commit_modeset_enables from drm_atomic_helper_commit_tail_rpm+0x44/0x8c
> [ 7.228111] drm_atomic_helper_commit_tail_rpm from commit_tail+0x9c/0x184
> [ 7.228124] commit_tail from drm_atomic_helper_commit+0x164/0x18c
> [ 7.228137] drm_atomic_helper_commit from drm_atomic_commit+0xb0/0xe8
> [ 7.228153] drm_atomic_commit from drm_client_modeset_commit_atomic+0x240/0x288
> [ 7.228166] drm_client_modeset_commit_atomic from drm_client_modeset_commit_locked+0x60/0x1cc
> [ 7.228174] drm_client_modeset_commit_locked from drm_client_modeset_commit+0x24/0x40
> [ 7.228183] drm_client_modeset_commit from drm_fb_helper_set_par+0xb8/0xf8
> [ 7.228197] drm_fb_helper_set_par from fbcon_init+0x2a0/0x534
> [ 7.228211] fbcon_init from visual_init+0xc0/0x108
> [ 7.228224] visual_init from do_bind_con_driver+0x1bc/0x3a8
> [ 7.228237] do_bind_con_driver from do_take_over_console+0x134/0x1d4
> [ 7.228251] do_take_over_console from do_fbcon_takeover+0x6c/0xcc
> [ 7.228264] do_fbcon_takeover from fbcon_fb_registered+0x198/0x1a8
> [ 7.228277] fbcon_fb_registered from register_framebuffer+0x1d0/0x268
> [ 7.228292] register_framebuffer from __drm_fb_helper_initial_config_and_unlock+0x358/0x578
> [ 7.228308] __drm_fb_helper_initial_config_and_unlock from drm_fbdev_client_hotplug+0x6c/0xa8
> [ 7.228322] drm_fbdev_client_hotplug from drm_fbdev_generic_setup+0x84/0x174
> [ 7.228335] drm_fbdev_generic_setup from rockchip_drm_bind+0x1dc/0x200
> [ 7.228349] rockchip_drm_bind from try_to_bring_up_aggregate_device+0x200/0x2d8
> [ 7.228368] try_to_bring_up_aggregate_device from component_master_add_with_match+0xc4/0xf8
> [ 7.228380] component_master_add_with_match from rockchip_drm_platform_probe+0x150/0x254
> [ 7.228392] rockchip_drm_platform_probe from platform_probe+0x5c/0xb8
> [ 7.228405] platform_probe from really_probe+0xe0/0x400
> [ 7.228417] really_probe from __driver_probe_device+0x9c/0x1fc
> [ 7.228431] __driver_probe_device from driver_probe_device+0x30/0xc0
> [ 7.228445] driver_probe_device from __device_attach_driver+0xa8/0x120
> [ 7.228460] __device_attach_driver from bus_for_each_drv+0x84/0xdc
> [ 7.228474] bus_for_each_drv from __device_attach+0xb0/0x20c
> [ 7.228488] __device_attach from bus_probe_device+0x8c/0x90
> [ 7.228502] bus_probe_device from deferred_probe_work_func+0x98/0xe0
> [ 7.228515] deferred_probe_work_func from process_one_work+0x290/0x740
> [ 7.228528] process_one_work from worker_thread+0x54/0x518
> [ 7.228536] worker_thread from kthread+0xf0/0x110
> [ 7.228548] kthread from ret_from_fork+0x14/0x2c
> [ 7.228561] Exception stack(0xf084dfb0 to 0xf084dff8)
> [ 7.228567] dfa0: 00000000 00000000 00000000 00000000
> [ 7.228573] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> [ 7.228579] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000
> [ 7.228585] irq event stamp: 138379
> [ 7.228592] hardirqs last enabled at (138385): [<c03c42ac>] vprintk_emit+0x330/0x354
> [ 7.228606] hardirqs last disabled at (138390): [<c03c4268>] vprintk_emit+0x2ec/0x354
> [ 7.228617] softirqs last enabled at (137258): [<c03016ac>] __do_softirq+0x2f8/0x548
> [ 7.228628] softirqs last disabled at (137253): [<c0350218>] __irq_exit_rcu+0x14c/0x170
> [ 7.228639] ---[ end trace 0000000000000000 ]---
>
> (complete log attached)
>
> I'm not sure how to debug this. From the callstack I can see that
> __iommu_group_set_domain is getting a NULL new_domain which triggers
> a call to __iommu_group_do_set_platform_dma which is WARNing because
> there is no set_platform_dma_ops callback. The NULL new_domain is
> because group->default_domain is NULL in __iommu_group_set_core_domain.
>
> So is the logic in the first patch that there is a default domain
> incorrect for this rockchip driver? Or is this callpath just hitting
> the function before the default domain can be created?
>
> Help debugging this would be appreciated - I'm not very familiar
> with this area of code.
It's the same thing Marek hit on Exynos: drivers which run on 32-bit Arm
need a .set_platform_dma op, regardless of whether they support default
domains on other platforms which use those.
Cheers,
Robin.
On 15/03/2023 15:57, Robin Murphy wrote:
> On 2023-03-15 15:49, Steven Price wrote:
>> On 10/01/2023 02:54, Lu Baolu wrote:
>>> Hi folks,
>>>
>>> The iommu core calls the driver's detach_dev domain op callback only
>>> when
>>> a device is finished assigning to user space and
>>> iommu_group_release_dma_owner() is called to return the device to the
>>> kernel, where iommu core wants to set the default domain to the
>>> device but
>>> the driver didn't provide one. The code looks like:
>>>
>>> /*
>>> * New drivers should support default domains and so the
>>> detach_dev() op
>>> * will never be called. Otherwise the NULL domain
>>> represents some
>>> * platform specific behavior.
>>> */
>>> if (!new_domain) {
>>> if (WARN_ON(!group->domain->ops->detach_dev))
>>> return -EINVAL;
>>> __iommu_group_for_each_dev(group, group->domain,
>>>
>>> iommu_group_do_detach_device);
>>> group->domain = NULL;
>>> return 0;
>>> }
>>>
>>> In other words, if the iommu driver provides default domains, the
>>> .detach_dev callback will never be called; Otherwise, the .detach_dev
>>> callback is actually called to return control back to the platform DMA
>>> ops, other than detaching the domain from device.
>>>
>>> This series cleanups this by:
>>>
>>> - If the IOMMU driver provides default domains, remove .detach_dev
>>> callback.
>>> - Adds a new set_platform_dma iommu op. Any IOMMU driver that doesn't
>>> provide default domain should implement set_platform_dma callback
>>> instead.
>>> - Retire .detach_dev callback.
>>>
>>> This series originates from various discussion in the community. Thanks
>>> to Jason, Robin and all others for their ideas.
>>>
>>> The whole series is available on github:
>>> https://github.com/LuBaolu/intel-iommu/commits/iommu-retire-detach_dev-v5
>>>
>>> Change log:
>>> v5:
>>> - Merge some patches to make the series cute. No functionality
>>> changes.
>>> - Check the op directly and WARN_ON the lack of any necessary
>>> callbacks. Get rid of the ret and EINVAL.
>>>
>>> v4:
>>> -
>>> https://lore.kernel.org/linux-iommu/20230104125725.271850-1-baolu.lu@linux.intel.com/
>>> - Drop the patch which renamed .attach_dev to .set_dev. As Robin said,
>>> "... I also wouldn't say that "attach" loses its meaning in a
>>> context
>>> where an equivalent "detach" operation is only ever implicit in
>>> reattaching to something else...". If we have a better name in the
>>> future, we can do it in other series.
>>> - Adjust the patch of "iommu: Add set_platform_dma_ops iommu ops"
>>> according to Jason's following suggestion " ... This is a bit ugly,
>>> it would be better to make the previous patch call set_platform_dma
>>> if it is set instead of detach_dev and then these patches should
>>> just
>>> rename the driver's fsl_pamu_detach_device to
>>> fsl_pamu_set_platform_dma ..."
>>> - Add a new patch to remove deferred attach check from
>>> __iommu_detach_domain() path. Make it a separate patch as the
>>> prerequisite to remove __iommu_detach_device() helper.
>>> - Rename set_platform_dma to set_platform_dma_ops to make it more
>>> meaningful.
>>>
>>> v3:
>>> -
>>> https://lore.kernel.org/linux-iommu/20221128064648.1934720-1-baolu.lu@linux.intel.com/
>>> - Setting blocking domain is not conceptually equal to detach_dev.
>>> Dropped all blocking domain related staffs in the previous version.
>>>
>>> v2:
>>> -
>>> https://lore.kernel.org/linux-iommu/20220826123014.52709-1-baolu.lu@linux.intel.com/
>>> - Replace .detach_dev callback with static block domain ops;
>>> - Rename .attach_dev to .set_dev.
>>>
>>> v1:
>>> -
>>> https://lore.kernel.org/linux-iommu/20220516015759.2952771-1-baolu.lu@linux.intel.com/
>>>
>>> Jason Gunthorpe (1):
>>> iommu: Remove deferred attach check from __iommu_detach_device()
>>>
>>> Lu Baolu (4):
>>> iommu: Remove detach_dev callbacks
>>> iommu: Add set_platform_dma_ops iommu ops
>>> iommu: Add set_platform_dma_ops callbacks
>>> iommu: Remove detach_dev callback
>>>
>>> include/linux/iommu.h | 8 ++-
>>> include/trace/events/iommu.h | 7 --
>>> drivers/iommu/amd/iommu.c | 26 -------
>>> drivers/iommu/apple-dart.c | 24 -------
>>> drivers/iommu/arm/arm-smmu/qcom_iommu.c | 23 ------
>>> drivers/iommu/exynos-iommu.c | 1 -
>>> drivers/iommu/fsl_pamu_domain.c | 6 +-
>>> drivers/iommu/iommu-traces.c | 1 -
>>> drivers/iommu/iommu.c | 94 ++++++++++++-------------
>>> drivers/iommu/ipmmu-vmsa.c | 16 -----
>>> drivers/iommu/msm_iommu.c | 6 +-
>>> drivers/iommu/mtk_iommu.c | 9 ---
>>> drivers/iommu/mtk_iommu_v1.c | 4 +-
>>> drivers/iommu/omap-iommu.c | 6 +-
>>> drivers/iommu/rockchip-iommu.c | 1 -
>>> drivers/iommu/s390-iommu.c | 7 +-
>>> drivers/iommu/sprd-iommu.c | 16 -----
>>> drivers/iommu/sun50i-iommu.c | 1 -
>>> drivers/iommu/tegra-gart.c | 6 +-
>>> drivers/iommu/tegra-smmu.c | 5 +-
>>> 20 files changed, 69 insertions(+), 198 deletions(-)
>>>
>>
>> I hit a problem with this series on a Firefly-RK3288, I bisected down to
>> 1b932ceddd19 ("iommu: Remove detach_dev callbacks"). The first splat is:
>>
>> [ 7.227853] ------------[ cut here ]------------
>> [ 7.227900] WARNING: CPU: 0 PID: 9 at drivers/iommu/iommu.c:2198
>> __iommu_group_set_domain+0xb4/0xc8
>> [ 7.227920] Modules linked in:
>> [ 7.227935] CPU: 0 PID: 9 Comm: kworker/u8:0 Not tainted 6.3.0-rc1 #1
>> [ 7.227942] Hardware name: Rockchip (Device Tree)
>> [ 7.227948] Workqueue: events_unbound deferred_probe_work_func
>> [ 7.227964] unwind_backtrace from show_stack+0x10/0x14
>> [ 7.227978] show_stack from dump_stack_lvl+0x58/0x70
>> [ 7.227992] dump_stack_lvl from __warn+0x7c/0x1dc
>> [ 7.228008] __warn from warn_slowpath_fmt+0xbc/0x1a0
>> [ 7.228022] warn_slowpath_fmt from __iommu_group_set_domain+0xb4/0xc8
>> [ 7.228035] __iommu_group_set_domain from
>> iommu_detach_device+0x84/0xf8
>> [ 7.228046] iommu_detach_device from
>> arm_iommu_detach_device+0x24/0xc4
>> [ 7.228057] arm_iommu_detach_device from
>> rockchip_drm_dma_attach_device+0x30/0x74
>> [ 7.228073] rockchip_drm_dma_attach_device from
>> vop_crtc_atomic_enable+0xf8/0xab0
>> [ 7.228085] vop_crtc_atomic_enable from
>> drm_atomic_helper_commit_modeset_enables+0xb0/0x2a0
>> [ 7.228097] drm_atomic_helper_commit_modeset_enables from
>> drm_atomic_helper_commit_tail_rpm+0x44/0x8c
>> [ 7.228111] drm_atomic_helper_commit_tail_rpm from
>> commit_tail+0x9c/0x184
>> [ 7.228124] commit_tail from drm_atomic_helper_commit+0x164/0x18c
>> [ 7.228137] drm_atomic_helper_commit from drm_atomic_commit+0xb0/0xe8
>> [ 7.228153] drm_atomic_commit from
>> drm_client_modeset_commit_atomic+0x240/0x288
>> [ 7.228166] drm_client_modeset_commit_atomic from
>> drm_client_modeset_commit_locked+0x60/0x1cc
>> [ 7.228174] drm_client_modeset_commit_locked from
>> drm_client_modeset_commit+0x24/0x40
>> [ 7.228183] drm_client_modeset_commit from
>> drm_fb_helper_set_par+0xb8/0xf8
>> [ 7.228197] drm_fb_helper_set_par from fbcon_init+0x2a0/0x534
>> [ 7.228211] fbcon_init from visual_init+0xc0/0x108
>> [ 7.228224] visual_init from do_bind_con_driver+0x1bc/0x3a8
>> [ 7.228237] do_bind_con_driver from do_take_over_console+0x134/0x1d4
>> [ 7.228251] do_take_over_console from do_fbcon_takeover+0x6c/0xcc
>> [ 7.228264] do_fbcon_takeover from fbcon_fb_registered+0x198/0x1a8
>> [ 7.228277] fbcon_fb_registered from register_framebuffer+0x1d0/0x268
>> [ 7.228292] register_framebuffer from
>> __drm_fb_helper_initial_config_and_unlock+0x358/0x578
>> [ 7.228308] __drm_fb_helper_initial_config_and_unlock from
>> drm_fbdev_client_hotplug+0x6c/0xa8
>> [ 7.228322] drm_fbdev_client_hotplug from
>> drm_fbdev_generic_setup+0x84/0x174
>> [ 7.228335] drm_fbdev_generic_setup from
>> rockchip_drm_bind+0x1dc/0x200
>> [ 7.228349] rockchip_drm_bind from
>> try_to_bring_up_aggregate_device+0x200/0x2d8
>> [ 7.228368] try_to_bring_up_aggregate_device from
>> component_master_add_with_match+0xc4/0xf8
>> [ 7.228380] component_master_add_with_match from
>> rockchip_drm_platform_probe+0x150/0x254
>> [ 7.228392] rockchip_drm_platform_probe from platform_probe+0x5c/0xb8
>> [ 7.228405] platform_probe from really_probe+0xe0/0x400
>> [ 7.228417] really_probe from __driver_probe_device+0x9c/0x1fc
>> [ 7.228431] __driver_probe_device from driver_probe_device+0x30/0xc0
>> [ 7.228445] driver_probe_device from
>> __device_attach_driver+0xa8/0x120
>> [ 7.228460] __device_attach_driver from bus_for_each_drv+0x84/0xdc
>> [ 7.228474] bus_for_each_drv from __device_attach+0xb0/0x20c
>> [ 7.228488] __device_attach from bus_probe_device+0x8c/0x90
>> [ 7.228502] bus_probe_device from deferred_probe_work_func+0x98/0xe0
>> [ 7.228515] deferred_probe_work_func from
>> process_one_work+0x290/0x740
>> [ 7.228528] process_one_work from worker_thread+0x54/0x518
>> [ 7.228536] worker_thread from kthread+0xf0/0x110
>> [ 7.228548] kthread from ret_from_fork+0x14/0x2c
>> [ 7.228561] Exception stack(0xf084dfb0 to 0xf084dff8)
>> [ 7.228567] dfa0: 00000000
>> 00000000 00000000 00000000
>> [ 7.228573] dfc0: 00000000 00000000 00000000 00000000 00000000
>> 00000000 00000000 00000000
>> [ 7.228579] dfe0: 00000000 00000000 00000000 00000000 00000013
>> 00000000
>> [ 7.228585] irq event stamp: 138379
>> [ 7.228592] hardirqs last enabled at (138385): [<c03c42ac>]
>> vprintk_emit+0x330/0x354
>> [ 7.228606] hardirqs last disabled at (138390): [<c03c4268>]
>> vprintk_emit+0x2ec/0x354
>> [ 7.228617] softirqs last enabled at (137258): [<c03016ac>]
>> __do_softirq+0x2f8/0x548
>> [ 7.228628] softirqs last disabled at (137253): [<c0350218>]
>> __irq_exit_rcu+0x14c/0x170
>> [ 7.228639] ---[ end trace 0000000000000000 ]---
>>
>> (complete log attached)
>>
>> I'm not sure how to debug this. From the callstack I can see that
>> __iommu_group_set_domain is getting a NULL new_domain which triggers
>> a call to __iommu_group_do_set_platform_dma which is WARNing because
>> there is no set_platform_dma_ops callback. The NULL new_domain is
>> because group->default_domain is NULL in __iommu_group_set_core_domain.
>>
>> So is the logic in the first patch that there is a default domain
>> incorrect for this rockchip driver? Or is this callpath just hitting
>> the function before the default domain can be created?
>>
>> Help debugging this would be appreciated - I'm not very familiar
>> with this area of code.
>
> It's the same thing Marek hit on Exynos: drivers which run on 32-bit Arm
> need a .set_platform_dma op, regardless of whether they support default
> domains on other platforms which use those.
Ah, that seems to be it. I'll post a patch.
Thanks,
Steve
Hi All,
On 10.01.2023 03:54, Lu Baolu wrote:
> Hi folks,
>
> The iommu core calls the driver's detach_dev domain op callback only when
> a device is finished assigning to user space and
> iommu_group_release_dma_owner() is called to return the device to the
> kernel, where iommu core wants to set the default domain to the device but
> the driver didn't provide one. The code looks like:
>
> /*
> * New drivers should support default domains and so the detach_dev() op
> * will never be called. Otherwise the NULL domain represents some
> * platform specific behavior.
> */
> if (!new_domain) {
> if (WARN_ON(!group->domain->ops->detach_dev))
> return -EINVAL;
> __iommu_group_for_each_dev(group, group->domain,
> iommu_group_do_detach_device);
> group->domain = NULL;
> return 0;
> }
>
> In other words, if the iommu driver provides default domains, the
> .detach_dev callback will never be called; Otherwise, the .detach_dev
> callback is actually called to return control back to the platform DMA
> ops, other than detaching the domain from device.
>
> This series cleanups this by:
>
> - If the IOMMU driver provides default domains, remove .detach_dev
> callback.
> - Adds a new set_platform_dma iommu op. Any IOMMU driver that doesn't
> provide default domain should implement set_platform_dma callback
> instead.
> - Retire .detach_dev callback.
>
> This series originates from various discussion in the community. Thanks
> to Jason, Robin and all others for their ideas.
I wonder how to handle the ARM 32bit case, which doesn't use the default
domain solution. Today, once this patchset has been merged to
linux-next, I've noticed that it broke booting of ARM 32bit Exynos based
boards.
The final solution would be to switch ARM 32bit to generic DMA-IOMMU
glue, but I'm not sure it this will happen soon. I will try to check if
any kind of quick workaround can be applied to get it working again.
> The whole series is available on github:
> https://protect2.fireeye.com/v1/url?k=ef2183b6-b0bababa-ef2008f9-000babff3563-336828b3433153d2&q=1&e=4ae5dae4-383f-4a82-9449-3f08f0422cb1&u=https%3A%2F%2Fgithub.com%2FLuBaolu%2Fintel-iommu%2Fcommits%2Fiommu-retire-detach_dev-v5
>
> Change log:
> v5:
> - Merge some patches to make the series cute. No functionality changes.
> - Check the op directly and WARN_ON the lack of any necessary
> callbacks. Get rid of the ret and EINVAL.
>
> v4:
> - https://lore.kernel.org/linux-iommu/20230104125725.271850-1-baolu.lu@linux.intel.com/
> - Drop the patch which renamed .attach_dev to .set_dev. As Robin said,
> "... I also wouldn't say that "attach" loses its meaning in a context
> where an equivalent "detach" operation is only ever implicit in
> reattaching to something else...". If we have a better name in the
> future, we can do it in other series.
> - Adjust the patch of "iommu: Add set_platform_dma_ops iommu ops"
> according to Jason's following suggestion " ... This is a bit ugly,
> it would be better to make the previous patch call set_platform_dma
> if it is set instead of detach_dev and then these patches should just
> rename the driver's fsl_pamu_detach_device to
> fsl_pamu_set_platform_dma ..."
> - Add a new patch to remove deferred attach check from
> __iommu_detach_domain() path. Make it a separate patch as the
> prerequisite to remove __iommu_detach_device() helper.
> - Rename set_platform_dma to set_platform_dma_ops to make it more
> meaningful.
>
> v3:
> - https://lore.kernel.org/linux-iommu/20221128064648.1934720-1-baolu.lu@linux.intel.com/
> - Setting blocking domain is not conceptually equal to detach_dev.
> Dropped all blocking domain related staffs in the previous version.
>
> v2:
> - https://lore.kernel.org/linux-iommu/20220826123014.52709-1-baolu.lu@linux.intel.com/
> - Replace .detach_dev callback with static block domain ops;
> - Rename .attach_dev to .set_dev.
>
> v1:
> - https://lore.kernel.org/linux-iommu/20220516015759.2952771-1-baolu.lu@linux.intel.com/
>
> Jason Gunthorpe (1):
> iommu: Remove deferred attach check from __iommu_detach_device()
>
> Lu Baolu (4):
> iommu: Remove detach_dev callbacks
> iommu: Add set_platform_dma_ops iommu ops
> iommu: Add set_platform_dma_ops callbacks
> iommu: Remove detach_dev callback
>
> include/linux/iommu.h | 8 ++-
> include/trace/events/iommu.h | 7 --
> drivers/iommu/amd/iommu.c | 26 -------
> drivers/iommu/apple-dart.c | 24 -------
> drivers/iommu/arm/arm-smmu/qcom_iommu.c | 23 ------
> drivers/iommu/exynos-iommu.c | 1 -
> drivers/iommu/fsl_pamu_domain.c | 6 +-
> drivers/iommu/iommu-traces.c | 1 -
> drivers/iommu/iommu.c | 94 ++++++++++++-------------
> drivers/iommu/ipmmu-vmsa.c | 16 -----
> drivers/iommu/msm_iommu.c | 6 +-
> drivers/iommu/mtk_iommu.c | 9 ---
> drivers/iommu/mtk_iommu_v1.c | 4 +-
> drivers/iommu/omap-iommu.c | 6 +-
> drivers/iommu/rockchip-iommu.c | 1 -
> drivers/iommu/s390-iommu.c | 7 +-
> drivers/iommu/sprd-iommu.c | 16 -----
> drivers/iommu/sun50i-iommu.c | 1 -
> drivers/iommu/tegra-gart.c | 6 +-
> drivers/iommu/tegra-smmu.c | 5 +-
> 20 files changed, 69 insertions(+), 198 deletions(-)
>
Best regards
--
Marek Szyprowski, PhD
Samsung R&D Institute Poland
On Mon, Jan 16, 2023 at 05:24:56PM +0100, Marek Szyprowski wrote:
> > The iommu core calls the driver's detach_dev domain op callback only when
> > a device is finished assigning to user space and
> > iommu_group_release_dma_owner() is called to return the device to the
> > kernel, where iommu core wants to set the default domain to the device but
> > the driver didn't provide one. The code looks like:
> >
> > /*
> > * New drivers should support default domains and so the detach_dev() op
> > * will never be called. Otherwise the NULL domain represents some
> > * platform specific behavior.
> > */
> > if (!new_domain) {
> > if (WARN_ON(!group->domain->ops->detach_dev))
> > return -EINVAL;
> > __iommu_group_for_each_dev(group, group->domain,
> > iommu_group_do_detach_device);
> > group->domain = NULL;
> > return 0;
> > }
> >
> > In other words, if the iommu driver provides default domains, the
> > .detach_dev callback will never be called; Otherwise, the .detach_dev
> > callback is actually called to return control back to the platform DMA
> > ops, other than detaching the domain from device.
> >
> > This series cleanups this by:
> >
> > - If the IOMMU driver provides default domains, remove .detach_dev
> > callback.
> > - Adds a new set_platform_dma iommu op. Any IOMMU driver that doesn't
> > provide default domain should implement set_platform_dma callback
> > instead.
> > - Retire .detach_dev callback.
> >
> > This series originates from various discussion in the community. Thanks
> > to Jason, Robin and all others for their ideas.
>
> I wonder how to handle the ARM 32bit case, which doesn't use the default
> domain solution. Today, once this patchset has been merged to
> linux-next, I've noticed that it broke booting of ARM 32bit Exynos based
> boards.
It is supposed to work, can you help debug what went wrong?
Jason
On Tue, Jan 10, 2023 at 10:54:03AM +0800, Lu Baolu wrote: > Jason Gunthorpe (1): > iommu: Remove deferred attach check from __iommu_detach_device() > > Lu Baolu (4): > iommu: Remove detach_dev callbacks > iommu: Add set_platform_dma_ops iommu ops > iommu: Add set_platform_dma_ops callbacks > iommu: Remove detach_dev callback Applied, thanks.
© 2016 - 2025 Red Hat, Inc.