.../bindings/media/qcom,sm8550-iris.yaml | 158 ++++ MAINTAINERS | 10 + drivers/media/platform/qcom/Kconfig | 1 + drivers/media/platform/qcom/Makefile | 1 + drivers/media/platform/qcom/iris/Kconfig | 13 + drivers/media/platform/qcom/iris/Makefile | 27 + drivers/media/platform/qcom/iris/iris_buffer.c | 623 ++++++++++++++ drivers/media/platform/qcom/iris/iris_buffer.h | 117 +++ drivers/media/platform/qcom/iris/iris_core.c | 96 +++ drivers/media/platform/qcom/iris/iris_core.h | 111 +++ drivers/media/platform/qcom/iris/iris_ctrls.c | 259 ++++++ drivers/media/platform/qcom/iris/iris_ctrls.h | 22 + drivers/media/platform/qcom/iris/iris_firmware.c | 116 +++ drivers/media/platform/qcom/iris/iris_firmware.h | 15 + drivers/media/platform/qcom/iris/iris_hfi_common.c | 176 ++++ drivers/media/platform/qcom/iris/iris_hfi_common.h | 155 ++++ drivers/media/platform/qcom/iris/iris_hfi_gen1.h | 16 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 826 ++++++++++++++++++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 448 ++++++++++ .../platform/qcom/iris/iris_hfi_gen1_response.c | 666 ++++++++++++++ drivers/media/platform/qcom/iris/iris_hfi_gen2.h | 41 + .../platform/qcom/iris/iris_hfi_gen2_command.c | 957 +++++++++++++++++++++ .../platform/qcom/iris/iris_hfi_gen2_defines.h | 161 ++++ .../platform/qcom/iris/iris_hfi_gen2_packet.c | 292 +++++++ .../platform/qcom/iris/iris_hfi_gen2_packet.h | 125 +++ .../platform/qcom/iris/iris_hfi_gen2_response.c | 934 ++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_hfi_queue.c | 314 +++++++ drivers/media/platform/qcom/iris/iris_hfi_queue.h | 182 ++++ drivers/media/platform/qcom/iris/iris_instance.h | 77 ++ .../platform/qcom/iris/iris_platform_common.h | 186 ++++ .../platform/qcom/iris/iris_platform_sm8250.c | 148 ++++ .../platform/qcom/iris/iris_platform_sm8550.c | 265 ++++++ drivers/media/platform/qcom/iris/iris_power.c | 140 +++ drivers/media/platform/qcom/iris/iris_power.h | 13 + drivers/media/platform/qcom/iris/iris_probe.c | 378 ++++++++ drivers/media/platform/qcom/iris/iris_resources.c | 131 +++ drivers/media/platform/qcom/iris/iris_resources.h | 18 + drivers/media/platform/qcom/iris/iris_state.c | 276 ++++++ drivers/media/platform/qcom/iris/iris_state.h | 144 ++++ drivers/media/platform/qcom/iris/iris_utils.c | 90 ++ drivers/media/platform/qcom/iris/iris_utils.h | 53 ++ drivers/media/platform/qcom/iris/iris_vb2.c | 335 ++++++++ drivers/media/platform/qcom/iris/iris_vb2.h | 19 + drivers/media/platform/qcom/iris/iris_vdec.c | 659 ++++++++++++++ drivers/media/platform/qcom/iris/iris_vdec.h | 25 + drivers/media/platform/qcom/iris/iris_vidc.c | 453 ++++++++++ drivers/media/platform/qcom/iris/iris_vidc.h | 15 + drivers/media/platform/qcom/iris/iris_vpu2.c | 38 + drivers/media/platform/qcom/iris/iris_vpu3.c | 122 +++ drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 270 ++++++ drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 91 ++ drivers/media/platform/qcom/iris/iris_vpu_common.c | 369 ++++++++ drivers/media/platform/qcom/iris/iris_vpu_common.h | 28 + .../platform/qcom/iris/iris_vpu_register_defines.h | 17 + 54 files changed, 11222 insertions(+)
Introduce support for Qualcomm new video acceleration hardware i.e.
iris, used for video stream decoding.
Iris is a multi pipe based hardware that offloads video stream decoding
from the application processor (AP). It supports H.264 decoding. The AP
communicates with hardware through a well defined protocol, called as
host firmware interface (HFI), which provides fine-grained and
asynchronous control over individual hardware features.
This driver implements upgraded HFI gen2 to communicate with firmware.
It supports SM8550 which is based out of HFI gen 2. It also supports
SM8250 which is based out of HFI gen1.
This driver comes with below capabilities:
- V4L2 complaint video driver with M2M and STREAMING capability.
- Supports H264 decoder.
This driver comes with below features:
- Centralized resource management.
- Centralized management of core and instance states.
- Defines platform specific capabilities and features. As a results, it
provides a single point of control to enable/disable a given feature
depending on specific platform capabilities.
- Handles various video recommended sequences, like DRC, Drain, Seek,
EOS.
- Implements asynchronous communication with hardware to achieve better
experience in low latency usecases.
- Output and capture planes are controlled independently. Thereby
providing a way to reconfigure individual plane.
- Native hardware support of LAST flag which is mandatory to align with
port reconfiguration and DRAIN sequence as per V4L guidelines.
Changes since v8:
- re-order supported profile for h264 (Stefan)
- move mapping of DEBLOCK cap to patch#14 (Stefan, Dmitry)
- add a way to read the firmware path from dtsi and fallback to resource
data if not available (dikshita)
- Link to v8: https://lore.kernel.org/r/20241210-qcom-video-iris-v8-0-42c5403cb1a3@quicinc.com
Changes since v7:
- Added HFI_ERROR_STREAM_UNSUPPORTED define to inform about session
error (Stefan)
- Fixed kernel doc issues
- Link to v7: https://lore.kernel.org/r/20241209-qcom-video-iris-v7-0-05c6bdead47b@quicinc.com
Changes since v6:
- Added vb2_is_busy check in s_fmt (Hans)
- Removed q->streaming checks in s_fmt (Hans)
- Removed usage of inst->subscriptions (Hans)
- Added call to v4l2_m2m_ioctl_decoder_cmd() from iris_dec_cmd() API (Hans)
- Fixed issues related to kernel docs.
- Addressed all other review comments.
- Link to v6: https://lore.kernel.org/r/20241120-qcom-video-iris-v6-0-a8cf6704e992@quicinc.com
Changes since v5:
- Fixed the memory leak in firmware load (Bryan)
- Updated all headers s/_LIKE_SO_/__LIKE_SO__ (Bryan)
- Updated value of IFACEQ_MAX_BUF_COUNT to 64 (Bryan)
- Removed actual_count from iris buffer structure and cleaned up
vb2_queue_setup (Hans)
- Used VIDEO_MAX_FRAME to set actual buffer to firmware (Hans)
- Fixed the typo in commit log and subject of patch#12 (Hans)
- Updated card field to Iris Decoder (Hans)
- Removed redundant setting of byteused (Hans)
- s/iris_driver/is_iris_driver (Jianhua)
- Addressed all other review comments.
- Link to v5: https://lore.kernel.org/r/20241105-qcom-video-iris-v5-0-a88e7c220f78@quicinc.com
Changes since v4:
- Fixed commit message length for all patches (Krzysztof)
- Migrated back to remove from remove_new (Uwe Kleine-König)
- Removed support for g_volatle_ctrl (Hans)
- Added changes to update minimum buffer count whenever the hardware
requirement changes (Hans)
- Removed state checks from try/g/s_fmt ioctls (Hans)
- Removed wait_prepare() and wait_finish callbacks (Hans)
- Added support for remove_bufs (Hans)
- Added module param based configurability to select between venus and
iris drivers for platforms supported by both drivers (Jianhua, Dmitry)
- Addressed misc other review comments from Hans.
- Fixed issues reported by kernel bot.
- Link to v4: https://lore.kernel.org/r/20241014-qcom-video-iris-v4-v4-0-c5eaa4e9ab9e@quicinc.com
Changes since v3:
- Fixed the style issues with dt binding (Krzysztof)
- Pushed the patch to add maintainers to end of the series (Krzysztof,
Dmitry)
- Moved the resource initialization next to probe in iris_probe.c and
squashed the patch of probe and resource initialization (Krzysztof)
- Removed error prints from probe function (Krzysztof)
- Defined bit fields used for register read/write with macros for better
context (Bryan)
- Converted if/else to switch/case wherever applicable (Bryan)
- Removed defensive NULL checks wherever not required (Bryan, Krzysztof)
- Removed core->state checks except for below scenarios (Bryan)
- When reverse thread (firmware) can move the state of the core to
error state.
- When client can force close the driver and there are still pending
firmware responses to be read from shared queues (msg and dbg)
- For PM operations, since its a delayed autosuspend work and sys
error handler from the reverse thread can move the state to core
deinit state anytime.
- Acquiring core->lock only for below scenarios (Bryan)
- Writing to registers.
- Reading/Writing from/to shared queues.
- Traversing the instance list of core.
- To protect the core->state when it can changed by reverse thread.
- Acquiring inst->lock only for below scenario which is needed (Bryan)
- Serializing the forward and reverse thread
- To protect the inst structure where the values can be modified by
firmware.
- Removed usage of core->power_enabled (Krzysztof, Bryan)
- Removed usage of mutex_is_locked (Krzysztof, Bryan)
- Use C structure for instance caps (Dmitry)
- Split the ctrl ops patch by keeping only the defines, struct and caps
needed to intialize the ctrl handler and to implement s/g_ctrl
(Dmitry)
- Removed the instance state checks to allow v4l2-ctl, relying on
standard vb2 checks instead. (Hans)
- Converted APIs to void wherever applicable except for below (Bryan)
- iris_hfi_gen2_handle_session_error and
iris_hfi_gen2_handle_session_property cannot be converted to void
even though they always return 0.
Because these are two of the handlers invoked from
iris_hfi_gen2_handle_session_response and are of
iris_hfi_gen2_inst_hfi_range struct type, where same prototype is
followed for all handlers and return type of all handers should be
'int'.
And we cannot have a switch case/if else to handle these
responses from firmware because we need to parse the responses in a
particular sequence.
That's why we opted for this handler based design instead of
introducing multiple for loop with code duplication.
- Fixed issues reported by kernel bot.
- Fixed v4l2 compliance issue reported with "-s" options based on the
inputs from Hans.
- Addressed all other review comments and made some code improvements.
Changes since v2:
- introduced support for HFI gen1.
- deprecated Encoder and HEVC, VP9 codecs.
- removed custom vb2 mem ops and used standard framework instead.
- added support for mmap streaming mode.
- migrated all the buffer APIs to mem2mem helper functions.
- registered iris buffer with vb2 framework.
- migrated to clk_bulk, reset_bulk and icc_bulk APIs.
- used pm_domain_attach/detach_list APIs.
- migrated to read/writel and other available helpers for register
access instead of custom wrappers.
- added documentation for various structures.
- addressed many other review comments from v2.
Note: A harmless onetime error log "Lucid PLL latch failed. Output may
be unstable!" is seen during bootup. It doesn't impact any video
usecase and is currently under discussion.
Static tools like checkpatch, smatch, dt_binding_check, sparse and
Coccinelle run successfully with this driver.
This driver is tested with v4l2-ctl[1] and Gstreamer[2].
[1]: v4l2-ctl --verbose --set-fmt-video-out=pixelformat=H264
--set-fmt-video=pixelformat=NV12 --stream-mmap --stream-out-mmap
--stream-from /media/FVDO_Freeway_720p.264 --stream-to out.NV12
[2]: gst-launch-1.0 filesrc location=/media/media/4k_decode_clip.264 !
h264parse ! v4l2h264dec capture-io-mode=dmabuf ! kmssink
The driver is tested with v4l2-compliance.
Result on SM8550:
v4l2-compliance --stream-from /media/FVDO_Freeway_720p.264 -s250
v4l2-compliance 1.29.0-5273, 64 bits, 64-bit time_t
v4l2-compliance SHA: 0ed98432fe68 2024-11-13 12:54:45
Compliance test for iris_driver device /dev/video0:
Driver Info:
Driver name : iris_driver
Card type : iris_decoder
Bus info : platform:aa00000.video-codec
Driver version : 6.12.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Detected Stateful Decoder
Required ioctls:
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 5 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK
test Composing: OK
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
test blocking wait: OK
Test input 0:
Streaming ioctls:
test read/write: OK (Not Supported)
test MMAP (select, REQBUFS): OK
test MMAP (epoll, REQBUFS): OK
test MMAP (select, CREATE_BUFS): OK
test MMAP (epoll, CREATE_BUFS): OK
test USERPTR (select): OK (Not Supported)
test DMABUF: Cannot test, specify --expbuf-device
Total for iris_driver device /dev/video0: 54, Succeeded: 54, Failed: 0,
Warnings: 0
Result on SM8250:
v4l2-compliance --stream-from /media/FVDO_Freeway_720p.264 -s250
v4l2-compliance 1.29.0-5273, 64 bits, 64-bit time_t
v4l2-compliance SHA: 0ed98432fe68 2024-11-13 12:54:45
Compliance test for iris_driver device /dev/video0:
Driver Info:
Driver name : iris_driver
Card type : iris_decoder
Bus info : platform:aa00000.video-codec
Driver version : 6.12.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Detected Stateful Decoder
Required ioctls:
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 5 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK
test Composing: OK
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
test blocking wait: OK
Test input 0:
Streaming ioctls:
test read/write: OK (Not Supported)
test MMAP (select, REQBUFS): OK
test MMAP (epoll, REQBUFS): OK
test MMAP (select, CREATE_BUFS): OK
test MMAP (epoll, CREATE_BUFS): OK
test USERPTR (select): OK (Not Supported)
test DMABUF: Cannot test, specify --expbuf-device
Total for iris_driver device /dev/video0: 54, Succeeded: 54, Failed: 0,
Warnings: 0
The result of fluster test on SM8550:
77/135 while testing JVT-AVC_V1 with
GStreamer-H.264-V4L2-Gst1.0.JVT-AVC_V1.
The failing tests are:
- 52 test vectors failed due to interlaced clips: Interlaced decoding is
not supported in iris driver.
Test Vectors:
cabac_mot_fld0_full
cabac_mot_mbaff0_full
cabac_mot_picaff0_full
CABREF3_Sand_D
CAFI1_SVA_C
CAMA1_Sony_C
CAMA1_TOSHIBA_B
cama1_vtc_c
cama2_vtc_b
CAMA3_Sand_E
cama3_vtc_b
CAMACI3_Sony_C
CAMANL1_TOSHIBA_B
CAMANL2_TOSHIBA_B
CAMANL3_Sand_E
CAMASL3_Sony_B
CAMP_MOT_MBAFF_L30
CAMP_MOT_MBAFF_L31
CANLMA2_Sony_C
CANLMA3_Sony_C
CAPA1_TOSHIBA_B
CAPAMA3_Sand_F
cavlc_mot_fld0_full_B
cavlc_mot_mbaff0_full_B
cavlc_mot_picaff0_full_B
CVCANLMA2_Sony_C
CVFI1_Sony_D
CVFI1_SVA_C
CVFI2_Sony_H
CVFI2_SVA_C
CVMA1_Sony_D
CVMA1_TOSHIBA_B
CVMANL1_TOSHIBA_B
CVMANL2_TOSHIBA_B
CVMAPAQP3_Sony_E
CVMAQP2_Sony_G
CVMAQP3_Sony_D
CVMP_MOT_FLD_L30_B
CVNLFI1_Sony_C
CVNLFI2_Sony_H
CVPA1_TOSHIBA_B
FI1_Sony_E
MR6_BT_B
MR7_BT_B
MR8_BT_B
MR9_BT_B
Sharp_MP_Field_1_B
Sharp_MP_Field_2_B
Sharp_MP_Field_3_B
Sharp_MP_PAFF_1r2
Sharp_MP_PAFF_2r
CVMP_MOT_FRM_L31_B
- 3 test vectors failed due to unsupported bitstream.
num_slice_group_minus1 greater than zero is not supported by the
hardware.
Test Vectors:
FM1_BT_B
FM1_FT_E
FM2_SVA_C
- 2 test vectors failed because SP_SLICE type is not supported by the
hardware.
Test Vectors:
SP1_BT_A
sp2_bt_b
- 1 test vector failed due to unsupported profile:
V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED is being deprecated from sm8550
onwards due to hardware issues.
Test Vectors:
BA3_SVA_C
23/69 while testing JVT-FR-EXT with
GStreamer-H.264-V4L2-Gst1.0.JVT-AVC_V1.
The failing tests are:
- 21 test vectors failed due to interlaced clips: Interlaced decoding is
not supported in iris driver.
Test Vectors:
brcm_freh4
brcm_freh5
brcm_freh6
brcm_freh10
brcm_freh11
freh7_b
FREXT01_JVC_D
FREXT02_JVC_C
FRExt2_Panasonic_C
FRExt4_Panasonic_B
HCAFF1_HHI_B
HCAMFF1_HHI_B
HCHP3_HHI_A
HPCAFL_BRCM_C
HPCAFLNL_BRCM_C
HVLCFI0_Sony_B
HVLCMFF0_Sony_B
HVLCPFF0_Sony_B
HPCAMAPALQ_BRCM_B
HPCVFL_BRCM_A
HPCVFLNL_BRCM_A
- 2 test vectors failed due to 10bit bitstream: 10bit decoding is not
supported in iris driver
Test Vectors:
FREH10-1
FREH10-2
- 21 test vectors failed due to unsupported profile:
V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_422 is not supported by the
hardware.
Test Vectors:
FREXT1_TANDBERG_A
FREXT2_TANDBERG_A
FREXT3_TANDBERG_A
Hi422FR1_SONY_A
Hi422FR2_SONY_A
Hi422FR3_SONY_A
Hi422FR4_SONY_A
Hi422FR6_SONY_A
Hi422FR7_SONY_A
Hi422FR8_SONY_A
Hi422FR9_SONY_A
Hi422FR10_SONY_A
Hi422FR11_SONY_A
Hi422FR12_SONY_A
Hi422FR13_SONY_A
Hi422FR14_SONY_A
Hi422FR15_SONY_A
Hi422FREXT16_SONY_A
Hi422FREXT17_SONY_A
Hi422FREXT18_SONY_A
Hi422FREXT19_SONY_A
- 2 test vectors failed due to unsupported bitstream.
chroma_fmt_idc is equal to 0(monochrome) in the bitstream which is not
supported by the hardware.
Test Vectors:
HPCAMOLQ_BRCM_B
HPCVMOLQ_BRCM_B
The result of fluster test on SM8250:
78/135 while testing JVT-AVC_V1 with
GStreamer-H.264-V4L2-Gst1.0.JVT-AVC_V1.
The failing tests are:
- 52 test vectors failed due to interlaced clips: Interlaced decoding is
not supported in iris driver.
Test Vectors:
cabac_mot_fld0_full
cabac_mot_mbaff0_full
cabac_mot_picaff0_full
CABREF3_Sand_D
CAFI1_SVA_C
CAMA1_Sony_C
CAMA1_TOSHIBA_B
cama1_vtc_c
cama2_vtc_b
CAMA3_Sand_E
cama3_vtc_b
CAMACI3_Sony_C
CAMANL1_TOSHIBA_B
CAMANL2_TOSHIBA_B
CAMANL3_Sand_E
CAMASL3_Sony_B
CAMP_MOT_MBAFF_L30
CAMP_MOT_MBAFF_L31
CANLMA2_Sony_C
CANLMA3_Sony_C
CAPA1_TOSHIBA_B
CAPAMA3_Sand_F
cavlc_mot_fld0_full_B
cavlc_mot_mbaff0_full_B
cavlc_mot_picaff0_full_B
CVCANLMA2_Sony_C
CVFI1_Sony_D
CVFI1_SVA_C
CVFI2_Sony_H
CVFI2_SVA_C
CVMA1_Sony_D
CVMA1_TOSHIBA_B
CVMANL1_TOSHIBA_B
CVMANL2_TOSHIBA_B
CVMAPAQP3_Sony_E
CVMAQP2_Sony_G
CVMAQP3_Sony_D
CVMP_MOT_FLD_L30_B
CVNLFI1_Sony_C
CVNLFI2_Sony_H
CVPA1_TOSHIBA_B
FI1_Sony_E
MR6_BT_B
MR7_BT_B
MR8_BT_B
MR9_BT_B
Sharp_MP_Field_1_B
Sharp_MP_Field_2_B
Sharp_MP_Field_3_B
Sharp_MP_PAFF_1r2
Sharp_MP_PAFF_2r
CVMP_MOT_FRM_L31_B
- 3 test vectors failed due to unsupported bitstream.
num_slice_group_minus1 greater than zero is not supported by the
hardware.
Test Vectors:
FM1_BT_B
FM1_FT_E
FM2_SVA_C
- 2 test vectors failed because SP_SLICE type is not supported by the
hardware.
Test Vectors:
SP1_BT_A
sp2_bt_b
23/69 while testing JVT-FR-EXT with
GStreamer-H.264-V4L2-Gst1.0.JVT-AVC_V1.
The failing tests are:
- 21 test vectors failed due to interlaced clips: Interlaced decoding is
not supported in iris driver.
Test Vectors:
brcm_freh4
brcm_freh5
brcm_freh6
brcm_freh10
brcm_freh11
freh7_b
FREXT01_JVC_D
FREXT02_JVC_C
FRExt2_Panasonic_C
FRExt4_Panasonic_B
HCAFF1_HHI_B
HCAMFF1_HHI_B
HCHP3_HHI_A
HPCAFL_BRCM_C
HPCAFLNL_BRCM_C
HVLCFI0_Sony_B
HVLCMFF0_Sony_B
HVLCPFF0_Sony_B
HPCAMAPALQ_BRCM_B
HPCVFL_BRCM_A
HPCVFLNL_BRCM_A
- 2 test vectors failed due to 10bit bitstream: 10bit decoding is not
supported in iris driver
Test Vectors:
FREH10-1
FREH10-2
- 21 test vectors failed due to unsupported profile:
V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_422 is not supported by the
hardware.
Test Vectors:
FREXT1_TANDBERG_A
FREXT2_TANDBERG_A
FREXT3_TANDBERG_A
Hi422FR1_SONY_A
Hi422FR2_SONY_A
Hi422FR3_SONY_A
Hi422FR4_SONY_A
Hi422FR6_SONY_A
Hi422FR7_SONY_A
Hi422FR8_SONY_A
Hi422FR9_SONY_A
Hi422FR10_SONY_A
Hi422FR11_SONY_A
Hi422FR12_SONY_A
Hi422FR13_SONY_A
Hi422FR14_SONY_A
Hi422FR15_SONY_A
Hi422FREXT16_SONY_A
Hi422FREXT17_SONY_A
Hi422FREXT18_SONY_A
Hi422FREXT19_SONY_A
- 2 test vectors failed due to unsupported bitstream.
chroma_fmt_idc is equal to 0(monochrome) in the bitstream which is not
supported by the hardware.
Test Vectors:
HPCAMOLQ_BRCM_B
HPCVMOLQ_BRCM_B
To: Vikash Garodia <quic_vgarodia@quicinc.com>
To: Abhinav Kumar <quic_abhinavk@quicinc.com>
To: Mauro Carvalho Chehab <mchehab@kernel.org>
To: Rob Herring <robh@kernel.org>
To: Krzysztof Kozlowski <krzk+dt@kernel.org>
To: Conor Dooley <conor+dt@kernel.org>
To: Philipp Zabel <p.zabel@pengutronix.de>
Cc: Hans Verkuil <hverkuil@xs4all.nl>
Cc: Sebastian Fricke <sebastian.fricke@collabora.com>
Cc: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Cc: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Cc: Neil Armstrong <neil.armstrong@linaro.org>
Cc: Nicolas Dufresne <nicolas@ndufresne.ca>
Cc: Uwe Kleine-König <u.kleine-koenig@baylibre.com>
Cc: Jianhua Lu <lujianhua000@gmail.com>
Cc: Stefan Schmidt <stefan.schmidt@linaro.org>
Cc: linux-media@vger.kernel.org
Cc: linux-arm-msm@vger.kernel.org
Cc: devicetree@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com>
---
Dikshita Agarwal (18):
dt-bindings: media: Add video support for QCOM SM8550 SoC
media: iris: add platform driver for iris video device
media: iris: implement iris v4l2 file ops
media: iris: introduce iris core state management with shared queues
media: iris: implement video firmware load/unload
media: iris: implement boot sequence of the firmware
media: iris: introduce host firmware interface with necessary hooks
media: iris: implement power management
media: iris: implement reqbuf ioctl with vb2_queue_setup
media: iris: implement iris v4l2_ctrl_ops
media: iris: implement vb2 streaming ops
media: iris: allocate, initialize and queue internal buffers
media: iris: implement vb2 ops for buf_queue and firmware response
media: iris: add support for dynamic resolution change
media: iris: handle streamoff/on from client in dynamic resolution change
media: iris: add support for drain sequence
media: iris: enable video driver probe of SM8250 SoC
media: MAINTAINERS: add Qualcomm iris video accelerator driver
Vedang Nagar (10):
media: iris: implement s_fmt, g_fmt and try_fmt ioctls
media: iris: implement g_selection ioctl
media: iris: implement enum_fmt and enum_framesizes ioctls
media: iris: implement subscribe_event and unsubscribe_event ioctls
media: iris: implement query_cap ioctl
media: iris: implement set properties to firmware during streamon
media: iris: subscribe parameters and properties to firmware for hfi_gen2
media: iris: add check whether the video session is supported or not
media: iris: implement power scaling for vpu2 and vpu3
media: iris: add check to allow sub states transitions
.../bindings/media/qcom,sm8550-iris.yaml | 158 ++++
MAINTAINERS | 10 +
drivers/media/platform/qcom/Kconfig | 1 +
drivers/media/platform/qcom/Makefile | 1 +
drivers/media/platform/qcom/iris/Kconfig | 13 +
drivers/media/platform/qcom/iris/Makefile | 27 +
drivers/media/platform/qcom/iris/iris_buffer.c | 623 ++++++++++++++
drivers/media/platform/qcom/iris/iris_buffer.h | 117 +++
drivers/media/platform/qcom/iris/iris_core.c | 96 +++
drivers/media/platform/qcom/iris/iris_core.h | 111 +++
drivers/media/platform/qcom/iris/iris_ctrls.c | 259 ++++++
drivers/media/platform/qcom/iris/iris_ctrls.h | 22 +
drivers/media/platform/qcom/iris/iris_firmware.c | 116 +++
drivers/media/platform/qcom/iris/iris_firmware.h | 15 +
drivers/media/platform/qcom/iris/iris_hfi_common.c | 176 ++++
drivers/media/platform/qcom/iris/iris_hfi_common.h | 155 ++++
drivers/media/platform/qcom/iris/iris_hfi_gen1.h | 16 +
.../platform/qcom/iris/iris_hfi_gen1_command.c | 826 ++++++++++++++++++
.../platform/qcom/iris/iris_hfi_gen1_defines.h | 448 ++++++++++
.../platform/qcom/iris/iris_hfi_gen1_response.c | 666 ++++++++++++++
drivers/media/platform/qcom/iris/iris_hfi_gen2.h | 41 +
.../platform/qcom/iris/iris_hfi_gen2_command.c | 957 +++++++++++++++++++++
.../platform/qcom/iris/iris_hfi_gen2_defines.h | 161 ++++
.../platform/qcom/iris/iris_hfi_gen2_packet.c | 292 +++++++
.../platform/qcom/iris/iris_hfi_gen2_packet.h | 125 +++
.../platform/qcom/iris/iris_hfi_gen2_response.c | 934 ++++++++++++++++++++
drivers/media/platform/qcom/iris/iris_hfi_queue.c | 314 +++++++
drivers/media/platform/qcom/iris/iris_hfi_queue.h | 182 ++++
drivers/media/platform/qcom/iris/iris_instance.h | 77 ++
.../platform/qcom/iris/iris_platform_common.h | 186 ++++
.../platform/qcom/iris/iris_platform_sm8250.c | 148 ++++
.../platform/qcom/iris/iris_platform_sm8550.c | 265 ++++++
drivers/media/platform/qcom/iris/iris_power.c | 140 +++
drivers/media/platform/qcom/iris/iris_power.h | 13 +
drivers/media/platform/qcom/iris/iris_probe.c | 378 ++++++++
drivers/media/platform/qcom/iris/iris_resources.c | 131 +++
drivers/media/platform/qcom/iris/iris_resources.h | 18 +
drivers/media/platform/qcom/iris/iris_state.c | 276 ++++++
drivers/media/platform/qcom/iris/iris_state.h | 144 ++++
drivers/media/platform/qcom/iris/iris_utils.c | 90 ++
drivers/media/platform/qcom/iris/iris_utils.h | 53 ++
drivers/media/platform/qcom/iris/iris_vb2.c | 335 ++++++++
drivers/media/platform/qcom/iris/iris_vb2.h | 19 +
drivers/media/platform/qcom/iris/iris_vdec.c | 659 ++++++++++++++
drivers/media/platform/qcom/iris/iris_vdec.h | 25 +
drivers/media/platform/qcom/iris/iris_vidc.c | 453 ++++++++++
drivers/media/platform/qcom/iris/iris_vidc.h | 15 +
drivers/media/platform/qcom/iris/iris_vpu2.c | 38 +
drivers/media/platform/qcom/iris/iris_vpu3.c | 122 +++
drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 270 ++++++
drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 91 ++
drivers/media/platform/qcom/iris/iris_vpu_common.c | 369 ++++++++
drivers/media/platform/qcom/iris/iris_vpu_common.h | 28 +
.../platform/qcom/iris/iris_vpu_register_defines.h | 17 +
54 files changed, 11222 insertions(+)
---
base-commit: 698b6e3163bafd61e1b7d13572e2c42974ac85ec
change-id: 20241028-qcom-video-iris-94d5a12e6d9e
Best regards,
--
Dikshita Agarwal <quic_dikshita@quicinc.com>
On Thu, 12 Dec 2024 at 22:23, Dikshita Agarwal
<quic_dikshita@quicinc.com> wrote:
>
> Introduce support for Qualcomm new video acceleration hardware i.e.
> iris, used for video stream decoding.
>
> Iris is a multi pipe based hardware that offloads video stream decoding
> from the application processor (AP). It supports H.264 decoding. The AP
> communicates with hardware through a well defined protocol, called as
> host firmware interface (HFI), which provides fine-grained and
> asynchronous control over individual hardware features.
>
> This driver implements upgraded HFI gen2 to communicate with firmware.
> It supports SM8550 which is based out of HFI gen 2. It also supports
> SM8250 which is based out of HFI gen1.
I tested this on an x1e based machine, a Surface Laptop 7. I had some
errors with loading the firmware which triggered some warnings when
trying to tear down the driver. I've pasted the WARNs at the end of
this mail.
I was using the firmware from linux-firmware, as packaged by distros:
4d2dae9a8187b728939e9c79fa68012b qcom/vpu/vpu30_p4.mbn
It appears to be signed by a test key, which I assume was the cause of
the failure. Replacing it with a properly signed version worked, and I
was able successfully decode a test video with ffplay:
$ ffplay -codec:v h264_v4l2m2m test.mp4
...
[h264_v4l2m2m @ 0xffff500054b0] Using device /dev/video0
[h264_v4l2m2m @ 0xffff500054b0] driver 'iris_driver' on card
'iris_decoder' in mplane mode
[h264_v4l2m2m @ 0xffff500054b0] requesting formats: output=H264/none
capture=NV12/yuv420p
Cheers,
Joel
---
[ 2.587909] qcom-iris aa00000.video-codec: error -22 initializing
firmware qcom/vpu/vpu30_p4.mbn
[ 2.588095] qcom-iris aa00000.video-codec: firmware download failed
[ 2.588250] ------------[ cut here ]------------
[ 2.588251] Unmap of a partial large IOPTE is not allowed
[ 2.588256] WARNING: CPU: 4 PID: 659 at
drivers/iommu/io-pgtable-arm.c:649 __arm_lpae_unmap+0x3cc/0x468
[ 2.588335] CPU: 4 UID: 0 PID: 659 Comm: v4l_id Tainted: G W
6.13.0-rc4-00092-g1bbe1a937cf6 #21
[ 2.588338] Tainted: [W]=WARN
[ 2.588339] Hardware name: Microsoft Corporation Microsoft Surface
Laptop, 7th Edition/Microsoft Surface Laptop, 7th Edition, BIOS
160.2.235 08/05/2024
[ 2.588340] pstate: 61400005 (nZCv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--)
[ 2.588342] pc : __arm_lpae_unmap+0x3cc/0x468
[ 2.588344] lr : __arm_lpae_unmap+0x3cc/0x468
[ 2.588345] sp : ffff80008259b690
[ 2.588346] x29: ffff80008259b690 x28: ffff80008259bc30 x27: ffff80008259b8f8
[ 2.588349] x26: ffffb973346978d0 x25: ffff68590225e7f0 x24: ffff80008259b8f8
[ 2.588351] x23: 00000000dfc00000 x22: 0000000000001000 x21: ffff68590396ce80
[ 2.588354] x20: ffff6859068fa4f8 x19: ffff6859068fa480 x18: fffffffffffef6a8
[ 2.588356] x17: ffff685900b8c040 x16: 0000000000000000 x15: ffff80008259b248
[ 2.588359] x14: ffffb9733457e768 x13: 6465776f6c6c6120 x12: 746f6e2073692045
[ 2.588362] x11: ffffb9733457e768 x10: 000000000000030c x9 : ffffb973345d6768
[ 2.588364] x8 : 0000000000017fe8 x7 : 00000000fffff30b x6 : ffffb973345d6768
[ 2.588367] x5 : ffff685c75f4b848 x4 : 40000000fffff30b x3 : ffffaee941a6a000
[ 2.588369] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff68590bc25640
[ 2.588372] Call trace:
[ 2.588373] __arm_lpae_unmap+0x3cc/0x468 (P)
[ 2.588376] __arm_lpae_unmap+0xf4/0x468
[ 2.588379] __arm_lpae_unmap+0xf4/0x468
[ 2.588381] arm_lpae_unmap_pages+0x70/0x84
[ 2.588383] arm_smmu_unmap_pages+0x48/0x10c
[ 2.588385] __iommu_unmap+0xf0/0x1c0
[ 2.588387] iommu_unmap_fast+0x10/0x20
[ 2.588389] __iommu_dma_unmap+0xb8/0x2c0
[ 2.588391] iommu_dma_free+0x2c/0x54
[ 2.588392] dma_free_attrs+0x9c/0xc0
[ 2.588395] iris_hfi_queues_deinit+0x70/0xa0 [iris]
[ 2.588399] iris_core_init+0xd8/0x138 [iris]
[ 2.588401] iris_open+0x3c/0x318 [iris]
[ 2.588403] v4l2_open+0xa8/0x124 [videodev]
[ 2.588406] chrdev_open+0xb0/0x21c
[ 2.588409] do_dentry_open+0x138/0x4c4
[ 2.588412] vfs_open+0x2c/0xe4
[ 2.588413] path_openat+0x6fc/0x10a0
[ 2.588415] do_filp_open+0xa8/0x170
[ 2.588417] do_sys_openat2+0xc8/0xfc
[ 2.588418] __arm64_sys_openat+0x64/0xc0
[ 2.588420] invoke_syscall+0x48/0x104
[ 2.588423] el0_svc_common.constprop.0+0xc0/0xe0
[ 2.588426] do_el0_svc+0x1c/0x28
[ 2.588428] el0_svc+0x30/0xcc
[ 2.588431] el0t_64_sync_handler+0x10c/0x138
[ 2.588433] el0t_64_sync+0x198/0x19c
[ 2.588435] ---[ end trace 0000000000000000 ]---
[ 2.588438] ------------[ cut here ]------------
[ 2.588439] WARNING: CPU: 4 PID: 659 at
drivers/iommu/dma-iommu.c:841 __iommu_dma_unmap+0x290/0x2c0
[ 2.588497] CPU: 4 UID: 0 PID: 659 Comm: v4l_id Tainted: G W
6.13.0-rc4-00092-g1bbe1a937cf6 #21
[ 2.588499] Tainted: [W]=WARN
[ 2.588500] Hardware name: Microsoft Corporation Microsoft Surface
Laptop, 7th Edition/Microsoft Surface Laptop, 7th Edition, BIOS
160.2.235 08/05/2024
[ 2.588501] pstate: 81400005 (Nzcv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--)
[ 2.588502] pc : __iommu_dma_unmap+0x290/0x2c0
[ 2.588503] lr : __iommu_dma_unmap+0xb8/0x2c0
[ 2.588505] sp : ffff80008259b8f0
[ 2.588505] x29: ffff80008259b930 x28: ffff80008259bc30 x27: 0000000000020100
[ 2.588508] x26: 0000000000020100 x25: ffff68590bf76a08 x24: ffff80008259b910
[ 2.588511] x23: ffff80008259b8f8 x22: ffff6859113cd260 x21: ffff68590bf76a00
[ 2.588513] x20: 00000000dfc00000 x19: 0000000000001000 x18: fffffffffffef6a8
[ 2.588516] x17: ffff685900b8c040 x16: 0000000000000000 x15: ffff80008259b248
[ 2.588518] x14: ffffb9733457e768 x13: 6465776f6c6c6120 x12: 746f6e2073692045
[ 2.588521] x11: ffffb9733457e768 x10: 000000000000030c x9 : ffffb973345d6768
[ 2.588523] x8 : 0000000000017fe8 x7 : 00000000fffff30b x6 : ffffb973345d6768
[ 2.588525] x5 : ffff685c75f4b848 x4 : 40000000fffff30b x3 : ffffaee941a6a000
[ 2.588528] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 0000000000000000
[ 2.588530] Call trace:
[ 2.588531] __iommu_dma_unmap+0x290/0x2c0 (P)
[ 2.588532] iommu_dma_free+0x2c/0x54
[ 2.588534] dma_free_attrs+0x9c/0xc0
[ 2.588535] iris_hfi_queues_deinit+0x70/0xa0 [iris]
[ 2.588537] iris_core_init+0xd8/0x138 [iris]
[ 2.588539] iris_open+0x3c/0x318 [iris]
[ 2.588541] v4l2_open+0xa8/0x124 [videodev]
[ 2.588543] chrdev_open+0xb0/0x21c
[ 2.588545] do_dentry_open+0x138/0x4c4
[ 2.588547] vfs_open+0x2c/0xe4
[ 2.588549] path_openat+0x6fc/0x10a0
[ 2.588550] do_filp_open+0xa8/0x170
[ 2.588551] do_sys_openat2+0xc8/0xfc
[ 2.588553] __arm64_sys_openat+0x64/0xc0
[ 2.588555] invoke_syscall+0x48/0x104
[ 2.588557] el0_svc_common.constprop.0+0xc0/0xe0
[ 2.588560] do_el0_svc+0x1c/0x28
[ 2.588562] el0_svc+0x30/0xcc
[ 2.588564] el0t_64_sync_handler+0x10c/0x138
[ 2.588566] el0t_64_sync+0x198/0x19c
[
On 1/6/2025 6:06 PM, Joel Stanley wrote: > On Thu, 12 Dec 2024 at 22:23, Dikshita Agarwal > <quic_dikshita@quicinc.com> wrote: >> >> Introduce support for Qualcomm new video acceleration hardware i.e. >> iris, used for video stream decoding. >> >> Iris is a multi pipe based hardware that offloads video stream decoding >> from the application processor (AP). It supports H.264 decoding. The AP >> communicates with hardware through a well defined protocol, called as >> host firmware interface (HFI), which provides fine-grained and >> asynchronous control over individual hardware features. >> >> This driver implements upgraded HFI gen2 to communicate with firmware. >> It supports SM8550 which is based out of HFI gen 2. It also supports >> SM8250 which is based out of HFI gen1. > > I tested this on an x1e based machine, a Surface Laptop 7. I had some > errors with loading the firmware which triggered some warnings when > trying to tear down the driver. I've pasted the WARNs at the end of > this mail. > > I was using the firmware from linux-firmware, as packaged by distros: > > 4d2dae9a8187b728939e9c79fa68012b qcom/vpu/vpu30_p4.mbn > Hi Joel, I tried again with firmware (qcom/vpu/vpu30_p4.mbn) available in linux-firmware[1] on my setup i.e SM8550 MTP and don't see any issue with firmware load. could you pls share from where you are taking the firmware? [1] git clone https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git Thanks, Dikshita > It appears to be signed by a test key, which I assume was the cause of > the failure. Replacing it with a properly signed version worked, and I > was able successfully decode a test video with ffplay: > > $ ffplay -codec:v h264_v4l2m2m test.mp4 > ... > [h264_v4l2m2m @ 0xffff500054b0] Using device /dev/video0 > [h264_v4l2m2m @ 0xffff500054b0] driver 'iris_driver' on card > 'iris_decoder' in mplane mode > [h264_v4l2m2m @ 0xffff500054b0] requesting formats: output=H264/none > capture=NV12/yuv420p > > Cheers, > > Joel > --- > [ 2.587909] qcom-iris aa00000.video-codec: error -22 initializing > firmware qcom/vpu/vpu30_p4.mbn > [ 2.588095] qcom-iris aa00000.video-codec: firmware download failed > [ 2.588250] ------------[ cut here ]------------ > [ 2.588251] Unmap of a partial large IOPTE is not allowed > [ 2.588256] WARNING: CPU: 4 PID: 659 at > drivers/iommu/io-pgtable-arm.c:649 __arm_lpae_unmap+0x3cc/0x468 > [ 2.588335] CPU: 4 UID: 0 PID: 659 Comm: v4l_id Tainted: G W > 6.13.0-rc4-00092-g1bbe1a937cf6 #21 > [ 2.588338] Tainted: [W]=WARN > [ 2.588339] Hardware name: Microsoft Corporation Microsoft Surface > Laptop, 7th Edition/Microsoft Surface Laptop, 7th Edition, BIOS > 160.2.235 08/05/2024 > [ 2.588340] pstate: 61400005 (nZCv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--) > [ 2.588342] pc : __arm_lpae_unmap+0x3cc/0x468 > [ 2.588344] lr : __arm_lpae_unmap+0x3cc/0x468 > [ 2.588345] sp : ffff80008259b690 > [ 2.588346] x29: ffff80008259b690 x28: ffff80008259bc30 x27: ffff80008259b8f8 > [ 2.588349] x26: ffffb973346978d0 x25: ffff68590225e7f0 x24: ffff80008259b8f8 > [ 2.588351] x23: 00000000dfc00000 x22: 0000000000001000 x21: ffff68590396ce80 > [ 2.588354] x20: ffff6859068fa4f8 x19: ffff6859068fa480 x18: fffffffffffef6a8 > [ 2.588356] x17: ffff685900b8c040 x16: 0000000000000000 x15: ffff80008259b248 > [ 2.588359] x14: ffffb9733457e768 x13: 6465776f6c6c6120 x12: 746f6e2073692045 > [ 2.588362] x11: ffffb9733457e768 x10: 000000000000030c x9 : ffffb973345d6768 > [ 2.588364] x8 : 0000000000017fe8 x7 : 00000000fffff30b x6 : ffffb973345d6768 > [ 2.588367] x5 : ffff685c75f4b848 x4 : 40000000fffff30b x3 : ffffaee941a6a000 > [ 2.588369] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff68590bc25640 > [ 2.588372] Call trace: > [ 2.588373] __arm_lpae_unmap+0x3cc/0x468 (P) > [ 2.588376] __arm_lpae_unmap+0xf4/0x468 > [ 2.588379] __arm_lpae_unmap+0xf4/0x468 > [ 2.588381] arm_lpae_unmap_pages+0x70/0x84 > [ 2.588383] arm_smmu_unmap_pages+0x48/0x10c > [ 2.588385] __iommu_unmap+0xf0/0x1c0 > [ 2.588387] iommu_unmap_fast+0x10/0x20 > [ 2.588389] __iommu_dma_unmap+0xb8/0x2c0 > [ 2.588391] iommu_dma_free+0x2c/0x54 > [ 2.588392] dma_free_attrs+0x9c/0xc0 > [ 2.588395] iris_hfi_queues_deinit+0x70/0xa0 [iris] > [ 2.588399] iris_core_init+0xd8/0x138 [iris] > [ 2.588401] iris_open+0x3c/0x318 [iris] > [ 2.588403] v4l2_open+0xa8/0x124 [videodev] > [ 2.588406] chrdev_open+0xb0/0x21c > [ 2.588409] do_dentry_open+0x138/0x4c4 > [ 2.588412] vfs_open+0x2c/0xe4 > [ 2.588413] path_openat+0x6fc/0x10a0 > [ 2.588415] do_filp_open+0xa8/0x170 > [ 2.588417] do_sys_openat2+0xc8/0xfc > [ 2.588418] __arm64_sys_openat+0x64/0xc0 > [ 2.588420] invoke_syscall+0x48/0x104 > [ 2.588423] el0_svc_common.constprop.0+0xc0/0xe0 > [ 2.588426] do_el0_svc+0x1c/0x28 > [ 2.588428] el0_svc+0x30/0xcc > [ 2.588431] el0t_64_sync_handler+0x10c/0x138 > [ 2.588433] el0t_64_sync+0x198/0x19c > [ 2.588435] ---[ end trace 0000000000000000 ]--- > [ 2.588438] ------------[ cut here ]------------ > [ 2.588439] WARNING: CPU: 4 PID: 659 at > drivers/iommu/dma-iommu.c:841 __iommu_dma_unmap+0x290/0x2c0 > [ 2.588497] CPU: 4 UID: 0 PID: 659 Comm: v4l_id Tainted: G W > 6.13.0-rc4-00092-g1bbe1a937cf6 #21 > [ 2.588499] Tainted: [W]=WARN > [ 2.588500] Hardware name: Microsoft Corporation Microsoft Surface > Laptop, 7th Edition/Microsoft Surface Laptop, 7th Edition, BIOS > 160.2.235 08/05/2024 > [ 2.588501] pstate: 81400005 (Nzcv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--) > [ 2.588502] pc : __iommu_dma_unmap+0x290/0x2c0 > [ 2.588503] lr : __iommu_dma_unmap+0xb8/0x2c0 > [ 2.588505] sp : ffff80008259b8f0 > [ 2.588505] x29: ffff80008259b930 x28: ffff80008259bc30 x27: 0000000000020100 > [ 2.588508] x26: 0000000000020100 x25: ffff68590bf76a08 x24: ffff80008259b910 > [ 2.588511] x23: ffff80008259b8f8 x22: ffff6859113cd260 x21: ffff68590bf76a00 > [ 2.588513] x20: 00000000dfc00000 x19: 0000000000001000 x18: fffffffffffef6a8 > [ 2.588516] x17: ffff685900b8c040 x16: 0000000000000000 x15: ffff80008259b248 > [ 2.588518] x14: ffffb9733457e768 x13: 6465776f6c6c6120 x12: 746f6e2073692045 > [ 2.588521] x11: ffffb9733457e768 x10: 000000000000030c x9 : ffffb973345d6768 > [ 2.588523] x8 : 0000000000017fe8 x7 : 00000000fffff30b x6 : ffffb973345d6768 > [ 2.588525] x5 : ffff685c75f4b848 x4 : 40000000fffff30b x3 : ffffaee941a6a000 > [ 2.588528] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 0000000000000000 > [ 2.588530] Call trace: > [ 2.588531] __iommu_dma_unmap+0x290/0x2c0 (P) > [ 2.588532] iommu_dma_free+0x2c/0x54 > [ 2.588534] dma_free_attrs+0x9c/0xc0 > [ 2.588535] iris_hfi_queues_deinit+0x70/0xa0 [iris] > [ 2.588537] iris_core_init+0xd8/0x138 [iris] > [ 2.588539] iris_open+0x3c/0x318 [iris] > [ 2.588541] v4l2_open+0xa8/0x124 [videodev] > [ 2.588543] chrdev_open+0xb0/0x21c > [ 2.588545] do_dentry_open+0x138/0x4c4 > [ 2.588547] vfs_open+0x2c/0xe4 > [ 2.588549] path_openat+0x6fc/0x10a0 > [ 2.588550] do_filp_open+0xa8/0x170 > [ 2.588551] do_sys_openat2+0xc8/0xfc > [ 2.588553] __arm64_sys_openat+0x64/0xc0 > [ 2.588555] invoke_syscall+0x48/0x104 > [ 2.588557] el0_svc_common.constprop.0+0xc0/0xe0 > [ 2.588560] do_el0_svc+0x1c/0x28 > [ 2.588562] el0_svc+0x30/0xcc > [ 2.588564] el0t_64_sync_handler+0x10c/0x138 > [ 2.588566] el0t_64_sync+0x198/0x19c > [
On Wed, Jan 08, 2025 at 03:22:49PM +0530, Dikshita Agarwal wrote: > > > On 1/6/2025 6:06 PM, Joel Stanley wrote: > > On Thu, 12 Dec 2024 at 22:23, Dikshita Agarwal > > <quic_dikshita@quicinc.com> wrote: > >> > >> Introduce support for Qualcomm new video acceleration hardware i.e. > >> iris, used for video stream decoding. > >> > >> Iris is a multi pipe based hardware that offloads video stream decoding > >> from the application processor (AP). It supports H.264 decoding. The AP > >> communicates with hardware through a well defined protocol, called as > >> host firmware interface (HFI), which provides fine-grained and > >> asynchronous control over individual hardware features. > >> > >> This driver implements upgraded HFI gen2 to communicate with firmware. > >> It supports SM8550 which is based out of HFI gen 2. It also supports > >> SM8250 which is based out of HFI gen1. > > > > I tested this on an x1e based machine, a Surface Laptop 7. I had some > > errors with loading the firmware which triggered some warnings when > > trying to tear down the driver. I've pasted the WARNs at the end of > > this mail. > > > > I was using the firmware from linux-firmware, as packaged by distros: > > > > 4d2dae9a8187b728939e9c79fa68012b qcom/vpu/vpu30_p4.mbn > > > Hi Joel, > > I tried again with firmware (qcom/vpu/vpu30_p4.mbn) available in > linux-firmware[1] on my setup i.e SM8550 MTP and don't see any issue with > firmware load. Of course. MTP is not fused to reject firmware not signed by a particular vendor key. Production devices are. Please test that the driver works correctly if the firmware gets rejected by TZ. > could you pls share from where you are taking the firmware? > > [1] git clone > https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git > > Thanks, > Dikshita > > It appears to be signed by a test key, which I assume was the cause of > > the failure. Replacing it with a properly signed version worked, and I > > was able successfully decode a test video with ffplay: > > > > $ ffplay -codec:v h264_v4l2m2m test.mp4 > > ... > > [h264_v4l2m2m @ 0xffff500054b0] Using device /dev/video0 > > [h264_v4l2m2m @ 0xffff500054b0] driver 'iris_driver' on card > > 'iris_decoder' in mplane mode > > [h264_v4l2m2m @ 0xffff500054b0] requesting formats: output=H264/none > > capture=NV12/yuv420p > > > > Cheers, > > > > Joel > > --- > > [ 2.587909] qcom-iris aa00000.video-codec: error -22 initializing > > firmware qcom/vpu/vpu30_p4.mbn > > [ 2.588095] qcom-iris aa00000.video-codec: firmware download failed > > [ 2.588250] ------------[ cut here ]------------ > > [ 2.588251] Unmap of a partial large IOPTE is not allowed > > [ 2.588256] WARNING: CPU: 4 PID: 659 at > > drivers/iommu/io-pgtable-arm.c:649 __arm_lpae_unmap+0x3cc/0x468 > > [ 2.588335] CPU: 4 UID: 0 PID: 659 Comm: v4l_id Tainted: G W > > 6.13.0-rc4-00092-g1bbe1a937cf6 #21 > > [ 2.588338] Tainted: [W]=WARN > > [ 2.588339] Hardware name: Microsoft Corporation Microsoft Surface > > Laptop, 7th Edition/Microsoft Surface Laptop, 7th Edition, BIOS > > 160.2.235 08/05/2024 > > [ 2.588340] pstate: 61400005 (nZCv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--) > > [ 2.588342] pc : __arm_lpae_unmap+0x3cc/0x468 > > [ 2.588344] lr : __arm_lpae_unmap+0x3cc/0x468 > > [ 2.588345] sp : ffff80008259b690 > > [ 2.588346] x29: ffff80008259b690 x28: ffff80008259bc30 x27: ffff80008259b8f8 > > [ 2.588349] x26: ffffb973346978d0 x25: ffff68590225e7f0 x24: ffff80008259b8f8 > > [ 2.588351] x23: 00000000dfc00000 x22: 0000000000001000 x21: ffff68590396ce80 > > [ 2.588354] x20: ffff6859068fa4f8 x19: ffff6859068fa480 x18: fffffffffffef6a8 > > [ 2.588356] x17: ffff685900b8c040 x16: 0000000000000000 x15: ffff80008259b248 > > [ 2.588359] x14: ffffb9733457e768 x13: 6465776f6c6c6120 x12: 746f6e2073692045 > > [ 2.588362] x11: ffffb9733457e768 x10: 000000000000030c x9 : ffffb973345d6768 > > [ 2.588364] x8 : 0000000000017fe8 x7 : 00000000fffff30b x6 : ffffb973345d6768 > > [ 2.588367] x5 : ffff685c75f4b848 x4 : 40000000fffff30b x3 : ffffaee941a6a000 > > [ 2.588369] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff68590bc25640 > > [ 2.588372] Call trace: > > [ 2.588373] __arm_lpae_unmap+0x3cc/0x468 (P) > > [ 2.588376] __arm_lpae_unmap+0xf4/0x468 > > [ 2.588379] __arm_lpae_unmap+0xf4/0x468 > > [ 2.588381] arm_lpae_unmap_pages+0x70/0x84 > > [ 2.588383] arm_smmu_unmap_pages+0x48/0x10c > > [ 2.588385] __iommu_unmap+0xf0/0x1c0 > > [ 2.588387] iommu_unmap_fast+0x10/0x20 > > [ 2.588389] __iommu_dma_unmap+0xb8/0x2c0 > > [ 2.588391] iommu_dma_free+0x2c/0x54 > > [ 2.588392] dma_free_attrs+0x9c/0xc0 > > [ 2.588395] iris_hfi_queues_deinit+0x70/0xa0 [iris] > > [ 2.588399] iris_core_init+0xd8/0x138 [iris] > > [ 2.588401] iris_open+0x3c/0x318 [iris] > > [ 2.588403] v4l2_open+0xa8/0x124 [videodev] > > [ 2.588406] chrdev_open+0xb0/0x21c > > [ 2.588409] do_dentry_open+0x138/0x4c4 > > [ 2.588412] vfs_open+0x2c/0xe4 > > [ 2.588413] path_openat+0x6fc/0x10a0 > > [ 2.588415] do_filp_open+0xa8/0x170 > > [ 2.588417] do_sys_openat2+0xc8/0xfc > > [ 2.588418] __arm64_sys_openat+0x64/0xc0 > > [ 2.588420] invoke_syscall+0x48/0x104 > > [ 2.588423] el0_svc_common.constprop.0+0xc0/0xe0 > > [ 2.588426] do_el0_svc+0x1c/0x28 > > [ 2.588428] el0_svc+0x30/0xcc > > [ 2.588431] el0t_64_sync_handler+0x10c/0x138 > > [ 2.588433] el0t_64_sync+0x198/0x19c > > [ 2.588435] ---[ end trace 0000000000000000 ]--- > > [ 2.588438] ------------[ cut here ]------------ > > [ 2.588439] WARNING: CPU: 4 PID: 659 at > > drivers/iommu/dma-iommu.c:841 __iommu_dma_unmap+0x290/0x2c0 > > [ 2.588497] CPU: 4 UID: 0 PID: 659 Comm: v4l_id Tainted: G W > > 6.13.0-rc4-00092-g1bbe1a937cf6 #21 > > [ 2.588499] Tainted: [W]=WARN > > [ 2.588500] Hardware name: Microsoft Corporation Microsoft Surface > > Laptop, 7th Edition/Microsoft Surface Laptop, 7th Edition, BIOS > > 160.2.235 08/05/2024 > > [ 2.588501] pstate: 81400005 (Nzcv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--) > > [ 2.588502] pc : __iommu_dma_unmap+0x290/0x2c0 > > [ 2.588503] lr : __iommu_dma_unmap+0xb8/0x2c0 > > [ 2.588505] sp : ffff80008259b8f0 > > [ 2.588505] x29: ffff80008259b930 x28: ffff80008259bc30 x27: 0000000000020100 > > [ 2.588508] x26: 0000000000020100 x25: ffff68590bf76a08 x24: ffff80008259b910 > > [ 2.588511] x23: ffff80008259b8f8 x22: ffff6859113cd260 x21: ffff68590bf76a00 > > [ 2.588513] x20: 00000000dfc00000 x19: 0000000000001000 x18: fffffffffffef6a8 > > [ 2.588516] x17: ffff685900b8c040 x16: 0000000000000000 x15: ffff80008259b248 > > [ 2.588518] x14: ffffb9733457e768 x13: 6465776f6c6c6120 x12: 746f6e2073692045 > > [ 2.588521] x11: ffffb9733457e768 x10: 000000000000030c x9 : ffffb973345d6768 > > [ 2.588523] x8 : 0000000000017fe8 x7 : 00000000fffff30b x6 : ffffb973345d6768 > > [ 2.588525] x5 : ffff685c75f4b848 x4 : 40000000fffff30b x3 : ffffaee941a6a000 > > [ 2.588528] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 0000000000000000 > > [ 2.588530] Call trace: > > [ 2.588531] __iommu_dma_unmap+0x290/0x2c0 (P) > > [ 2.588532] iommu_dma_free+0x2c/0x54 > > [ 2.588534] dma_free_attrs+0x9c/0xc0 > > [ 2.588535] iris_hfi_queues_deinit+0x70/0xa0 [iris] > > [ 2.588537] iris_core_init+0xd8/0x138 [iris] > > [ 2.588539] iris_open+0x3c/0x318 [iris] > > [ 2.588541] v4l2_open+0xa8/0x124 [videodev] > > [ 2.588543] chrdev_open+0xb0/0x21c > > [ 2.588545] do_dentry_open+0x138/0x4c4 > > [ 2.588547] vfs_open+0x2c/0xe4 > > [ 2.588549] path_openat+0x6fc/0x10a0 > > [ 2.588550] do_filp_open+0xa8/0x170 > > [ 2.588551] do_sys_openat2+0xc8/0xfc > > [ 2.588553] __arm64_sys_openat+0x64/0xc0 > > [ 2.588555] invoke_syscall+0x48/0x104 > > [ 2.588557] el0_svc_common.constprop.0+0xc0/0xe0 > > [ 2.588560] do_el0_svc+0x1c/0x28 > > [ 2.588562] el0_svc+0x30/0xcc > > [ 2.588564] el0t_64_sync_handler+0x10c/0x138 > > [ 2.588566] el0t_64_sync+0x198/0x19c > > [ -- With best wishes Dmitry
On Mon, 6 Jan 2025 at 23:06, Joel Stanley <joel@jms.id.au> wrote:
> I tested this on an x1e based machine, a Surface Laptop 7. I had some
> errors with loading the firmware which triggered some warnings when
> trying to tear down the driver. I've pasted the WARNs at the end of
> this mail.
It might be new behavior triggered by this series:
https://lore.kernel.org/all/0-v3-b3a5b5937f56+7bb-arm_no_split_jgg@nvidia.com/
Your allocation:
queue_size = ALIGN((sizeof(*q_tbl_hdr) + (IFACEQ_QUEUE_SIZE *
IFACEQ_NUMQ)), SZ_4K);
core->iface_q_table_vaddr = dma_alloc_attrs(core->dev, queue_size,
&core->iface_q_table_daddr,
GFP_KERNEL,
DMA_ATTR_WRITE_COMBINE);
and the free:
dma_free_attrs(core->dev, sizeof(struct iris_hfi_queue_table_header),
core->iface_q_table_vaddr, core->iface_q_table_daddr,
DMA_ATTR_WRITE_COMBINE);
So probably your bug, but hidden until Jason's change caused it to WARN.
> [ 2.587909] qcom-iris aa00000.video-codec: error -22 initializing
> firmware qcom/vpu/vpu30_p4.mbn
> [ 2.588095] qcom-iris aa00000.video-codec: firmware download failed
> [ 2.588250] ------------[ cut here ]------------
> [ 2.588251] Unmap of a partial large IOPTE is not allowed
> [ 2.588256] WARNING: CPU: 4 PID: 659 at
> drivers/iommu/io-pgtable-arm.c:649 __arm_lpae_unmap+0x3cc/0x468
> [ 2.588335] CPU: 4 UID: 0 PID: 659 Comm: v4l_id Tainted: G W
> 6.13.0-rc4-00092-g1bbe1a937cf6 #21
> [ 2.588338] Tainted: [W]=WARN
> [ 2.588339] Hardware name: Microsoft Corporation Microsoft Surface
> Laptop, 7th Edition/Microsoft Surface Laptop, 7th Edition, BIOS
> 160.2.235 08/05/2024
> [ 2.588340] pstate: 61400005 (nZCv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--)
> [ 2.588342] pc : __arm_lpae_unmap+0x3cc/0x468
> [ 2.588344] lr : __arm_lpae_unmap+0x3cc/0x468
> [ 2.588345] sp : ffff80008259b690
> [ 2.588346] x29: ffff80008259b690 x28: ffff80008259bc30 x27: ffff80008259b8f8
> [ 2.588349] x26: ffffb973346978d0 x25: ffff68590225e7f0 x24: ffff80008259b8f8
> [ 2.588351] x23: 00000000dfc00000 x22: 0000000000001000 x21: ffff68590396ce80
> [ 2.588354] x20: ffff6859068fa4f8 x19: ffff6859068fa480 x18: fffffffffffef6a8
> [ 2.588356] x17: ffff685900b8c040 x16: 0000000000000000 x15: ffff80008259b248
> [ 2.588359] x14: ffffb9733457e768 x13: 6465776f6c6c6120 x12: 746f6e2073692045
> [ 2.588362] x11: ffffb9733457e768 x10: 000000000000030c x9 : ffffb973345d6768
> [ 2.588364] x8 : 0000000000017fe8 x7 : 00000000fffff30b x6 : ffffb973345d6768
> [ 2.588367] x5 : ffff685c75f4b848 x4 : 40000000fffff30b x3 : ffffaee941a6a000
> [ 2.588369] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff68590bc25640
> [ 2.588372] Call trace:
> [ 2.588373] __arm_lpae_unmap+0x3cc/0x468 (P)
> [ 2.588376] __arm_lpae_unmap+0xf4/0x468
> [ 2.588379] __arm_lpae_unmap+0xf4/0x468
> [ 2.588381] arm_lpae_unmap_pages+0x70/0x84
> [ 2.588383] arm_smmu_unmap_pages+0x48/0x10c
> [ 2.588385] __iommu_unmap+0xf0/0x1c0
> [ 2.588387] iommu_unmap_fast+0x10/0x20
> [ 2.588389] __iommu_dma_unmap+0xb8/0x2c0
> [ 2.588391] iommu_dma_free+0x2c/0x54
> [ 2.588392] dma_free_attrs+0x9c/0xc0
> [ 2.588395] iris_hfi_queues_deinit+0x70/0xa0 [iris]
> [ 2.588399] iris_core_init+0xd8/0x138 [iris]
> [ 2.588401] iris_open+0x3c/0x318 [iris]
> [ 2.588403] v4l2_open+0xa8/0x124 [videodev]
> [ 2.588406] chrdev_open+0xb0/0x21c
> [ 2.588409] do_dentry_open+0x138/0x4c4
> [ 2.588412] vfs_open+0x2c/0xe4
> [ 2.588413] path_openat+0x6fc/0x10a0
> [ 2.588415] do_filp_open+0xa8/0x170
> [ 2.588417] do_sys_openat2+0xc8/0xfc
> [ 2.588418] __arm64_sys_openat+0x64/0xc0
> [ 2.588420] invoke_syscall+0x48/0x104
> [ 2.588423] el0_svc_common.constprop.0+0xc0/0xe0
> [ 2.588426] do_el0_svc+0x1c/0x28
> [ 2.588428] el0_svc+0x30/0xcc
> [ 2.588431] el0t_64_sync_handler+0x10c/0x138
> [ 2.588433] el0t_64_sync+0x198/0x19c
> [ 2.588435] ---[ end trace 0000000000000000 ]---
> [ 2.588438] ------------[ cut here ]------------
> [ 2.588439] WARNING: CPU: 4 PID: 659 at
> drivers/iommu/dma-iommu.c:841 __iommu_dma_unmap+0x290/0x2c0
> [ 2.588497] CPU: 4 UID: 0 PID: 659 Comm: v4l_id Tainted: G W
> 6.13.0-rc4-00092-g1bbe1a937cf6 #21
> [ 2.588499] Tainted: [W]=WARN
> [ 2.588500] Hardware name: Microsoft Corporation Microsoft Surface
> Laptop, 7th Edition/Microsoft Surface Laptop, 7th Edition, BIOS
> 160.2.235 08/05/2024
> [ 2.588501] pstate: 81400005 (Nzcv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--)
> [ 2.588502] pc : __iommu_dma_unmap+0x290/0x2c0
> [ 2.588503] lr : __iommu_dma_unmap+0xb8/0x2c0
> [ 2.588505] sp : ffff80008259b8f0
> [ 2.588505] x29: ffff80008259b930 x28: ffff80008259bc30 x27: 0000000000020100
> [ 2.588508] x26: 0000000000020100 x25: ffff68590bf76a08 x24: ffff80008259b910
> [ 2.588511] x23: ffff80008259b8f8 x22: ffff6859113cd260 x21: ffff68590bf76a00
> [ 2.588513] x20: 00000000dfc00000 x19: 0000000000001000 x18: fffffffffffef6a8
> [ 2.588516] x17: ffff685900b8c040 x16: 0000000000000000 x15: ffff80008259b248
> [ 2.588518] x14: ffffb9733457e768 x13: 6465776f6c6c6120 x12: 746f6e2073692045
> [ 2.588521] x11: ffffb9733457e768 x10: 000000000000030c x9 : ffffb973345d6768
> [ 2.588523] x8 : 0000000000017fe8 x7 : 00000000fffff30b x6 : ffffb973345d6768
> [ 2.588525] x5 : ffff685c75f4b848 x4 : 40000000fffff30b x3 : ffffaee941a6a000
> [ 2.588528] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 0000000000000000
> [ 2.588530] Call trace:
> [ 2.588531] __iommu_dma_unmap+0x290/0x2c0 (P)
> [ 2.588532] iommu_dma_free+0x2c/0x54
> [ 2.588534] dma_free_attrs+0x9c/0xc0
> [ 2.588535] iris_hfi_queues_deinit+0x70/0xa0 [iris]
> [ 2.588537] iris_core_init+0xd8/0x138 [iris]
> [ 2.588539] iris_open+0x3c/0x318 [iris]
> [ 2.588541] v4l2_open+0xa8/0x124 [videodev]
> [ 2.588543] chrdev_open+0xb0/0x21c
> [ 2.588545] do_dentry_open+0x138/0x4c4
> [ 2.588547] vfs_open+0x2c/0xe4
> [ 2.588549] path_openat+0x6fc/0x10a0
> [ 2.588550] do_filp_open+0xa8/0x170
> [ 2.588551] do_sys_openat2+0xc8/0xfc
> [ 2.588553] __arm64_sys_openat+0x64/0xc0
> [ 2.588555] invoke_syscall+0x48/0x104
> [ 2.588557] el0_svc_common.constprop.0+0xc0/0xe0
> [ 2.588560] do_el0_svc+0x1c/0x28
> [ 2.588562] el0_svc+0x30/0xcc
> [ 2.588564] el0t_64_sync_handler+0x10c/0x138
> [ 2.588566] el0t_64_sync+0x198/0x19c
> [
On 1/6/2025 6:16 PM, Joel Stanley wrote: > On Mon, 6 Jan 2025 at 23:06, Joel Stanley <joel@jms.id.au> wrote: >> I tested this on an x1e based machine, a Surface Laptop 7. I had some >> errors with loading the firmware which triggered some warnings when >> trying to tear down the driver. I've pasted the WARNs at the end of >> this mail. > > It might be new behavior triggered by this series: > > https://lore.kernel.org/all/0-v3-b3a5b5937f56+7bb-arm_no_split_jgg@nvidia.com/ > > Your allocation: > > queue_size = ALIGN((sizeof(*q_tbl_hdr) + (IFACEQ_QUEUE_SIZE * > IFACEQ_NUMQ)), SZ_4K); > core->iface_q_table_vaddr = dma_alloc_attrs(core->dev, queue_size, > &core->iface_q_table_daddr, > GFP_KERNEL, > DMA_ATTR_WRITE_COMBINE); > > > and the free: > > dma_free_attrs(core->dev, sizeof(struct iris_hfi_queue_table_header), > core->iface_q_table_vaddr, core->iface_q_table_daddr, > DMA_ATTR_WRITE_COMBINE); > > So probably your bug, but hidden until Jason's change caused it to WARN. > Thanks for pointing it out, its definitely a bug, will fix it. Thanks, Dikshita >> [ 2.587909] qcom-iris aa00000.video-codec: error -22 initializing >> firmware qcom/vpu/vpu30_p4.mbn >> [ 2.588095] qcom-iris aa00000.video-codec: firmware download failed >> [ 2.588250] ------------[ cut here ]------------ >> [ 2.588251] Unmap of a partial large IOPTE is not allowed >> [ 2.588256] WARNING: CPU: 4 PID: 659 at >> drivers/iommu/io-pgtable-arm.c:649 __arm_lpae_unmap+0x3cc/0x468 >> [ 2.588335] CPU: 4 UID: 0 PID: 659 Comm: v4l_id Tainted: G W >> 6.13.0-rc4-00092-g1bbe1a937cf6 #21 >> [ 2.588338] Tainted: [W]=WARN >> [ 2.588339] Hardware name: Microsoft Corporation Microsoft Surface >> Laptop, 7th Edition/Microsoft Surface Laptop, 7th Edition, BIOS >> 160.2.235 08/05/2024 >> [ 2.588340] pstate: 61400005 (nZCv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--) >> [ 2.588342] pc : __arm_lpae_unmap+0x3cc/0x468 >> [ 2.588344] lr : __arm_lpae_unmap+0x3cc/0x468 >> [ 2.588345] sp : ffff80008259b690 >> [ 2.588346] x29: ffff80008259b690 x28: ffff80008259bc30 x27: ffff80008259b8f8 >> [ 2.588349] x26: ffffb973346978d0 x25: ffff68590225e7f0 x24: ffff80008259b8f8 >> [ 2.588351] x23: 00000000dfc00000 x22: 0000000000001000 x21: ffff68590396ce80 >> [ 2.588354] x20: ffff6859068fa4f8 x19: ffff6859068fa480 x18: fffffffffffef6a8 >> [ 2.588356] x17: ffff685900b8c040 x16: 0000000000000000 x15: ffff80008259b248 >> [ 2.588359] x14: ffffb9733457e768 x13: 6465776f6c6c6120 x12: 746f6e2073692045 >> [ 2.588362] x11: ffffb9733457e768 x10: 000000000000030c x9 : ffffb973345d6768 >> [ 2.588364] x8 : 0000000000017fe8 x7 : 00000000fffff30b x6 : ffffb973345d6768 >> [ 2.588367] x5 : ffff685c75f4b848 x4 : 40000000fffff30b x3 : ffffaee941a6a000 >> [ 2.588369] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff68590bc25640 >> [ 2.588372] Call trace: >> [ 2.588373] __arm_lpae_unmap+0x3cc/0x468 (P) >> [ 2.588376] __arm_lpae_unmap+0xf4/0x468 >> [ 2.588379] __arm_lpae_unmap+0xf4/0x468 >> [ 2.588381] arm_lpae_unmap_pages+0x70/0x84 >> [ 2.588383] arm_smmu_unmap_pages+0x48/0x10c >> [ 2.588385] __iommu_unmap+0xf0/0x1c0 >> [ 2.588387] iommu_unmap_fast+0x10/0x20 >> [ 2.588389] __iommu_dma_unmap+0xb8/0x2c0 >> [ 2.588391] iommu_dma_free+0x2c/0x54 >> [ 2.588392] dma_free_attrs+0x9c/0xc0 >> [ 2.588395] iris_hfi_queues_deinit+0x70/0xa0 [iris] >> [ 2.588399] iris_core_init+0xd8/0x138 [iris] >> [ 2.588401] iris_open+0x3c/0x318 [iris] >> [ 2.588403] v4l2_open+0xa8/0x124 [videodev] >> [ 2.588406] chrdev_open+0xb0/0x21c >> [ 2.588409] do_dentry_open+0x138/0x4c4 >> [ 2.588412] vfs_open+0x2c/0xe4 >> [ 2.588413] path_openat+0x6fc/0x10a0 >> [ 2.588415] do_filp_open+0xa8/0x170 >> [ 2.588417] do_sys_openat2+0xc8/0xfc >> [ 2.588418] __arm64_sys_openat+0x64/0xc0 >> [ 2.588420] invoke_syscall+0x48/0x104 >> [ 2.588423] el0_svc_common.constprop.0+0xc0/0xe0 >> [ 2.588426] do_el0_svc+0x1c/0x28 >> [ 2.588428] el0_svc+0x30/0xcc >> [ 2.588431] el0t_64_sync_handler+0x10c/0x138 >> [ 2.588433] el0t_64_sync+0x198/0x19c >> [ 2.588435] ---[ end trace 0000000000000000 ]--- >> [ 2.588438] ------------[ cut here ]------------ >> [ 2.588439] WARNING: CPU: 4 PID: 659 at >> drivers/iommu/dma-iommu.c:841 __iommu_dma_unmap+0x290/0x2c0 >> [ 2.588497] CPU: 4 UID: 0 PID: 659 Comm: v4l_id Tainted: G W >> 6.13.0-rc4-00092-g1bbe1a937cf6 #21 >> [ 2.588499] Tainted: [W]=WARN >> [ 2.588500] Hardware name: Microsoft Corporation Microsoft Surface >> Laptop, 7th Edition/Microsoft Surface Laptop, 7th Edition, BIOS >> 160.2.235 08/05/2024 >> [ 2.588501] pstate: 81400005 (Nzcv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--) >> [ 2.588502] pc : __iommu_dma_unmap+0x290/0x2c0 >> [ 2.588503] lr : __iommu_dma_unmap+0xb8/0x2c0 >> [ 2.588505] sp : ffff80008259b8f0 >> [ 2.588505] x29: ffff80008259b930 x28: ffff80008259bc30 x27: 0000000000020100 >> [ 2.588508] x26: 0000000000020100 x25: ffff68590bf76a08 x24: ffff80008259b910 >> [ 2.588511] x23: ffff80008259b8f8 x22: ffff6859113cd260 x21: ffff68590bf76a00 >> [ 2.588513] x20: 00000000dfc00000 x19: 0000000000001000 x18: fffffffffffef6a8 >> [ 2.588516] x17: ffff685900b8c040 x16: 0000000000000000 x15: ffff80008259b248 >> [ 2.588518] x14: ffffb9733457e768 x13: 6465776f6c6c6120 x12: 746f6e2073692045 >> [ 2.588521] x11: ffffb9733457e768 x10: 000000000000030c x9 : ffffb973345d6768 >> [ 2.588523] x8 : 0000000000017fe8 x7 : 00000000fffff30b x6 : ffffb973345d6768 >> [ 2.588525] x5 : ffff685c75f4b848 x4 : 40000000fffff30b x3 : ffffaee941a6a000 >> [ 2.588528] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 0000000000000000 >> [ 2.588530] Call trace: >> [ 2.588531] __iommu_dma_unmap+0x290/0x2c0 (P) >> [ 2.588532] iommu_dma_free+0x2c/0x54 >> [ 2.588534] dma_free_attrs+0x9c/0xc0 >> [ 2.588535] iris_hfi_queues_deinit+0x70/0xa0 [iris] >> [ 2.588537] iris_core_init+0xd8/0x138 [iris] >> [ 2.588539] iris_open+0x3c/0x318 [iris] >> [ 2.588541] v4l2_open+0xa8/0x124 [videodev] >> [ 2.588543] chrdev_open+0xb0/0x21c >> [ 2.588545] do_dentry_open+0x138/0x4c4 >> [ 2.588547] vfs_open+0x2c/0xe4 >> [ 2.588549] path_openat+0x6fc/0x10a0 >> [ 2.588550] do_filp_open+0xa8/0x170 >> [ 2.588551] do_sys_openat2+0xc8/0xfc >> [ 2.588553] __arm64_sys_openat+0x64/0xc0 >> [ 2.588555] invoke_syscall+0x48/0x104 >> [ 2.588557] el0_svc_common.constprop.0+0xc0/0xe0 >> [ 2.588560] do_el0_svc+0x1c/0x28 >> [ 2.588562] el0_svc+0x30/0xcc >> [ 2.588564] el0t_64_sync_handler+0x10c/0x138 >> [ 2.588566] el0t_64_sync+0x198/0x19c >> [
[ +CC: Bjorn ] On Thu, Dec 12, 2024 at 05:21:22PM +0530, Dikshita Agarwal wrote: > Introduce support for Qualcomm new video acceleration hardware i.e. > iris, used for video stream decoding. > Note: A harmless onetime error log "Lucid PLL latch failed. Output may > be unstable!" is seen during bootup. It doesn't impact any video > usecase and is currently under discussion. This could be an indication that some resources are not described correctly and could potentially require both binding and driver changes to address. This is also something which could cause trouble later (e.g. during suspend) even if you manage to get the clock running after boot. Generally, you should not be introducing any new warnings; they are there to let you know that something is wrong. Where is this issue being discussed? I think we at least need a public analysis and understanding of the problem before merging this. > Dikshita Agarwal (18): > dt-bindings: media: Add video support for QCOM SM8550 SoC Could you please post the DT changes required for sm8550 so that people can test this series more easily? You can do it in a separate series if you prefer (with a link to this series in the cover letter) or as part of this one (at the end, with a note saying that those changes should go through the qcom SoC tree). > media: iris: add platform driver for iris video device > media: iris: implement iris v4l2 file ops > media: iris: introduce iris core state management with shared queues > media: iris: implement video firmware load/unload > media: iris: implement boot sequence of the firmware > media: iris: introduce host firmware interface with necessary hooks > media: iris: implement power management > media: iris: implement reqbuf ioctl with vb2_queue_setup > media: iris: implement iris v4l2_ctrl_ops > media: iris: implement vb2 streaming ops > media: iris: allocate, initialize and queue internal buffers > media: iris: implement vb2 ops for buf_queue and firmware response > media: iris: add support for dynamic resolution change > media: iris: handle streamoff/on from client in dynamic resolution change > media: iris: add support for drain sequence > media: iris: enable video driver probe of SM8250 SoC You should also say something in the cover letter about sm8250 now being supported by two mainline drivers and how you propose to handle that conflict. > media: MAINTAINERS: add Qualcomm iris video accelerator driver Johan
Hello Johan, On Thu, 9 Jan 2025 at 15:58, Johan Hovold <johan@kernel.org> wrote: > > [ +CC: Bjorn ] > > On Thu, Dec 12, 2024 at 05:21:22PM +0530, Dikshita Agarwal wrote: > > Introduce support for Qualcomm new video acceleration hardware i.e. > > iris, used for video stream decoding. > > > Note: A harmless onetime error log "Lucid PLL latch failed. Output may > > be unstable!" is seen during bootup. It doesn't impact any video > > usecase and is currently under discussion. > > This could be an indication that some resources are not described > correctly and could potentially require both binding and driver changes > to address. > > This is also something which could cause trouble later (e.g. during > suspend) even if you manage to get the clock running after boot. > > Generally, you should not be introducing any new warnings; they are > there to let you know that something is wrong. > > Where is this issue being discussed? > > I think we at least need a public analysis and understanding of the > problem before merging this. Taniya Das proposed a patchset to reconfigure PLL in the clk-alpha-pll which allows the videocc-sm8550 driver to configure it correctly. https://lore.kernel.org/linux-arm-msm/20250113-support-pll-reconfigure-v1-0-1fae6bc1062d@quicinc.com/T/ I tested the Iris driver with this patchset and I am no longer seeing the Lucid PLL latch failed warning. regards Stefan Schmidt
On Wed, Jan 22, 2025 at 04:34:51PM +0100, Stefan Schmidt wrote: > On Thu, 9 Jan 2025 at 15:58, Johan Hovold <johan@kernel.org> wrote: > > On Thu, Dec 12, 2024 at 05:21:22PM +0530, Dikshita Agarwal wrote: > > > Introduce support for Qualcomm new video acceleration hardware i.e. > > > iris, used for video stream decoding. > > > > > Note: A harmless onetime error log "Lucid PLL latch failed. Output may > > > be unstable!" is seen during bootup. It doesn't impact any video > > > usecase and is currently under discussion. > > > > This could be an indication that some resources are not described > > correctly and could potentially require both binding and driver changes > > to address. > > > > This is also something which could cause trouble later (e.g. during > > suspend) even if you manage to get the clock running after boot. > > > > Generally, you should not be introducing any new warnings; they are > > there to let you know that something is wrong. > > > > Where is this issue being discussed? > > > > I think we at least need a public analysis and understanding of the > > problem before merging this. > > Taniya Das proposed a patchset to reconfigure PLL in the clk-alpha-pll > which allows the videocc-sm8550 driver to configure it correctly. > https://lore.kernel.org/linux-arm-msm/20250113-support-pll-reconfigure-v1-0-1fae6bc1062d@quicinc.com/T/ > > I tested the Iris driver with this patchset and I am no longer seeing > the Lucid PLL latch failed warning. Thanks for the pointer. Please make sure to reference this series (and summarise the underlying issue) when resending this series. Judging from a quick look the approach taken there seems like a bit of a hack so it may not get merged in its current form. IIUC fixing the PLL issue properly may depend on adding support for multiple power domains to the clock drivers. Johan
On 2/3/2025 2:13 PM, Johan Hovold wrote: > On Wed, Jan 22, 2025 at 04:34:51PM +0100, Stefan Schmidt wrote: >> On Thu, 9 Jan 2025 at 15:58, Johan Hovold <johan@kernel.org> wrote: >>> On Thu, Dec 12, 2024 at 05:21:22PM +0530, Dikshita Agarwal wrote: > >>>> Introduce support for Qualcomm new video acceleration hardware i.e. >>>> iris, used for video stream decoding. >>> >>>> Note: A harmless onetime error log "Lucid PLL latch failed. Output may >>>> be unstable!" is seen during bootup. It doesn't impact any video >>>> usecase and is currently under discussion. >>> >>> This could be an indication that some resources are not described >>> correctly and could potentially require both binding and driver changes >>> to address. >>> >>> This is also something which could cause trouble later (e.g. during >>> suspend) even if you manage to get the clock running after boot. >>> >>> Generally, you should not be introducing any new warnings; they are >>> there to let you know that something is wrong. >>> >>> Where is this issue being discussed? >>> >>> I think we at least need a public analysis and understanding of the >>> problem before merging this. >> >> Taniya Das proposed a patchset to reconfigure PLL in the clk-alpha-pll >> which allows the videocc-sm8550 driver to configure it correctly. >> https://lore.kernel.org/linux-arm-msm/20250113-support-pll-reconfigure-v1-0-1fae6bc1062d@quicinc.com/T/ >> >> I tested the Iris driver with this patchset and I am no longer seeing >> the Lucid PLL latch failed warning. > > Thanks for the pointer. Please make sure to reference this series (and > summarise the underlying issue) when resending this series. > > Judging from a quick look the approach taken there seems like a bit of a > hack so it may not get merged in its current form. IIUC fixing the PLL > issue properly may depend on adding support for multiple power domains > to the clock drivers. It would be good if you can add your comment to the PLL fix series [1] so that the patch owner can discuss on your suggestion. Regards, Vikash [1] https://lore.kernel.org/linux-arm-msm/20250113-support-pll-reconfigure-v1-0-1fae6bc1062d@quicinc.com/T/
On 12/12/2024 11:51, Dikshita Agarwal wrote:
> Introduce support for Qualcomm new video acceleration hardware i.e.
> iris, used for video stream decoding.
I'm ~happy enough~ to give a Tested-by for this series bar the 8250
probe selection story which I think still needs work.
commit 0d78bb6e4a9190b005b3fe5746be2f42e27f6949 (HEAD ->
linux-media-next-25-01-15-sm8550-x1e)
Author: Dikshita Agarwal <quic_dikshita@quicinc.com>
Date: Thu Dec 12 17:21:50 2024 +0530
uname -a
Linux linaro-gnome 6.13.0-rc1-00197-g0d78bb6e4a91-dirty #226 SMP PREEMPT
Wed Jan 15 21:45:36 GMT 2025 aarch64 GNU/Linux
gst-launch-1.0 -vvv -e filesrc location=sample-30s.mp4 ! qtdemux !
parsebin ! v4l2h264dec ! autovideosink
cat /proc/interrupts | grep iris
287: 415 0 0 0 0 0
0 0 GICv3 206 Level iris
cat /proc/interrupts | grep iris
287: 458 0 0 0 0 0
0 0 GICv3 206 Level iris
I get a display of busses driving around a park and can verify the
decoder IRQ bumps as we do it.
Please apply my:
Tested-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Except to the probe 8250 patch which I'm still not happy about.
---
bod
On 12/12/2024 12:51, Dikshita Agarwal wrote: > Introduce support for Qualcomm new video acceleration hardware i.e. > iris, used for video stream decoding. > > Iris is a multi pipe based hardware that offloads video stream decoding > from the application processor (AP). It supports H.264 decoding. The AP > communicates with hardware through a well defined protocol, called as > host firmware interface (HFI), which provides fine-grained and > asynchronous control over individual hardware features. > > This driver implements upgraded HFI gen2 to communicate with firmware. > It supports SM8550 which is based out of HFI gen 2. It also supports > SM8250 which is based out of HFI gen1. > > This driver comes with below capabilities: > - V4L2 complaint video driver with M2M and STREAMING capability. > - Supports H264 decoder. > > This driver comes with below features: > - Centralized resource management. > - Centralized management of core and instance states. > - Defines platform specific capabilities and features. As a results, it > provides a single point of control to enable/disable a given feature > depending on specific platform capabilities. > - Handles various video recommended sequences, like DRC, Drain, Seek, > EOS. > - Implements asynchronous communication with hardware to achieve better > experience in low latency usecases. > - Output and capture planes are controlled independently. Thereby > providing a way to reconfigure individual plane. > - Native hardware support of LAST flag which is mandatory to align with > port reconfiguration and DRAIN sequence as per V4L guidelines. > > Changes since v8: > - re-order supported profile for h264 (Stefan) > - move mapping of DEBLOCK cap to patch#14 (Stefan, Dmitry) > - add a way to read the firmware path from dtsi and fallback to resource > data if not available (dikshita) > - Link to v8: https://lore.kernel.org/r/20241210-qcom-video-iris-v8-0-42c5403cb1a3@quicinc.com > > Changes since v7: > - Added HFI_ERROR_STREAM_UNSUPPORTED define to inform about session > error (Stefan) > - Fixed kernel doc issues > - Link to v7: https://lore.kernel.org/r/20241209-qcom-video-iris-v7-0-05c6bdead47b@quicinc.com > > Changes since v6: > - Added vb2_is_busy check in s_fmt (Hans) > - Removed q->streaming checks in s_fmt (Hans) > - Removed usage of inst->subscriptions (Hans) > - Added call to v4l2_m2m_ioctl_decoder_cmd() from iris_dec_cmd() API (Hans) > - Fixed issues related to kernel docs. > - Addressed all other review comments. > - Link to v6: https://lore.kernel.org/r/20241120-qcom-video-iris-v6-0-a8cf6704e992@quicinc.com > > Changes since v5: > - Fixed the memory leak in firmware load (Bryan) > - Updated all headers s/_LIKE_SO_/__LIKE_SO__ (Bryan) > - Updated value of IFACEQ_MAX_BUF_COUNT to 64 (Bryan) > - Removed actual_count from iris buffer structure and cleaned up > vb2_queue_setup (Hans) > - Used VIDEO_MAX_FRAME to set actual buffer to firmware (Hans) > - Fixed the typo in commit log and subject of patch#12 (Hans) > - Updated card field to Iris Decoder (Hans) > - Removed redundant setting of byteused (Hans) > - s/iris_driver/is_iris_driver (Jianhua) > - Addressed all other review comments. > - Link to v5: https://lore.kernel.org/r/20241105-qcom-video-iris-v5-0-a88e7c220f78@quicinc.com > > Changes since v4: > - Fixed commit message length for all patches (Krzysztof) > - Migrated back to remove from remove_new (Uwe Kleine-König) > - Removed support for g_volatle_ctrl (Hans) > - Added changes to update minimum buffer count whenever the hardware > requirement changes (Hans) > - Removed state checks from try/g/s_fmt ioctls (Hans) > - Removed wait_prepare() and wait_finish callbacks (Hans) > - Added support for remove_bufs (Hans) > - Added module param based configurability to select between venus and > iris drivers for platforms supported by both drivers (Jianhua, Dmitry) > - Addressed misc other review comments from Hans. > - Fixed issues reported by kernel bot. > - Link to v4: https://lore.kernel.org/r/20241014-qcom-video-iris-v4-v4-0-c5eaa4e9ab9e@quicinc.com > > Changes since v3: > - Fixed the style issues with dt binding (Krzysztof) > - Pushed the patch to add maintainers to end of the series (Krzysztof, > Dmitry) > - Moved the resource initialization next to probe in iris_probe.c and > squashed the patch of probe and resource initialization (Krzysztof) > - Removed error prints from probe function (Krzysztof) > - Defined bit fields used for register read/write with macros for better > context (Bryan) > - Converted if/else to switch/case wherever applicable (Bryan) > - Removed defensive NULL checks wherever not required (Bryan, Krzysztof) > - Removed core->state checks except for below scenarios (Bryan) > - When reverse thread (firmware) can move the state of the core to > error state. > - When client can force close the driver and there are still pending > firmware responses to be read from shared queues (msg and dbg) > - For PM operations, since its a delayed autosuspend work and sys > error handler from the reverse thread can move the state to core > deinit state anytime. > - Acquiring core->lock only for below scenarios (Bryan) > - Writing to registers. > - Reading/Writing from/to shared queues. > - Traversing the instance list of core. > - To protect the core->state when it can changed by reverse thread. > - Acquiring inst->lock only for below scenario which is needed (Bryan) > - Serializing the forward and reverse thread > - To protect the inst structure where the values can be modified by > firmware. > - Removed usage of core->power_enabled (Krzysztof, Bryan) > - Removed usage of mutex_is_locked (Krzysztof, Bryan) > - Use C structure for instance caps (Dmitry) > - Split the ctrl ops patch by keeping only the defines, struct and caps > needed to intialize the ctrl handler and to implement s/g_ctrl > (Dmitry) > - Removed the instance state checks to allow v4l2-ctl, relying on > standard vb2 checks instead. (Hans) > - Converted APIs to void wherever applicable except for below (Bryan) > - iris_hfi_gen2_handle_session_error and > iris_hfi_gen2_handle_session_property cannot be converted to void > even though they always return 0. > Because these are two of the handlers invoked from > iris_hfi_gen2_handle_session_response and are of > iris_hfi_gen2_inst_hfi_range struct type, where same prototype is > followed for all handlers and return type of all handers should be > 'int'. > And we cannot have a switch case/if else to handle these > responses from firmware because we need to parse the responses in a > particular sequence. > That's why we opted for this handler based design instead of > introducing multiple for loop with code duplication. > - Fixed issues reported by kernel bot. > - Fixed v4l2 compliance issue reported with "-s" options based on the > inputs from Hans. > - Addressed all other review comments and made some code improvements. > > Changes since v2: > - introduced support for HFI gen1. > - deprecated Encoder and HEVC, VP9 codecs. > - removed custom vb2 mem ops and used standard framework instead. > - added support for mmap streaming mode. > - migrated all the buffer APIs to mem2mem helper functions. > - registered iris buffer with vb2 framework. > - migrated to clk_bulk, reset_bulk and icc_bulk APIs. > - used pm_domain_attach/detach_list APIs. > - migrated to read/writel and other available helpers for register > access instead of custom wrappers. > - added documentation for various structures. > - addressed many other review comments from v2. > > Note: A harmless onetime error log "Lucid PLL latch failed. Output may > be unstable!" is seen during bootup. It doesn't impact any video > usecase and is currently under discussion. > > Static tools like checkpatch, smatch, dt_binding_check, sparse and > Coccinelle run successfully with this driver. > > This driver is tested with v4l2-ctl[1] and Gstreamer[2]. > > [1]: v4l2-ctl --verbose --set-fmt-video-out=pixelformat=H264 > --set-fmt-video=pixelformat=NV12 --stream-mmap --stream-out-mmap > --stream-from /media/FVDO_Freeway_720p.264 --stream-to out.NV12 > > [2]: gst-launch-1.0 filesrc location=/media/media/4k_decode_clip.264 ! > h264parse ! v4l2h264dec capture-io-mode=dmabuf ! kmssink > > The driver is tested with v4l2-compliance. > > Result on SM8550: > > v4l2-compliance --stream-from /media/FVDO_Freeway_720p.264 -s250 > > v4l2-compliance 1.29.0-5273, 64 bits, 64-bit time_t > v4l2-compliance SHA: 0ed98432fe68 2024-11-13 12:54:45 > > Compliance test for iris_driver device /dev/video0: > > Driver Info: > Driver name : iris_driver > Card type : iris_decoder > Bus info : platform:aa00000.video-codec > Driver version : 6.12.0 > Capabilities : 0x84204000 > Video Memory-to-Memory Multiplanar > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x04204000 > Video Memory-to-Memory Multiplanar > Streaming > Extended Pix Format > Detected Stateful Decoder > > Required ioctls: > test VIDIOC_QUERYCAP: OK > test invalid ioctls: OK > > Allow for multiple opens: > test second /dev/video0 open: OK > test VIDIOC_QUERYCAP: OK > test VIDIOC_G/S_PRIORITY: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 0 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Control ioctls: > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK > test VIDIOC_QUERYCTRL: OK > test VIDIOC_G/S_CTRL: OK > test VIDIOC_G/S/TRY_EXT_CTRLS: OK > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 5 Private Controls: 0 > > Format ioctls: > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK > test VIDIOC_TRY_FMT: OK > test VIDIOC_S_FMT: OK > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK > test Composing: OK > test Scaling: OK (Not Supported) > > Codec ioctls: > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK > > Buffer ioctls: > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > test CREATE_BUFS maximum buffers: OK > test VIDIOC_REMOVE_BUFS: OK > test VIDIOC_EXPBUF: OK > test Requests: OK (Not Supported) > test blocking wait: OK > > Test input 0: > > Streaming ioctls: > test read/write: OK (Not Supported) > test MMAP (select, REQBUFS): OK > test MMAP (epoll, REQBUFS): OK > test MMAP (select, CREATE_BUFS): OK > test MMAP (epoll, CREATE_BUFS): OK > test USERPTR (select): OK (Not Supported) > test DMABUF: Cannot test, specify --expbuf-device > > Total for iris_driver device /dev/video0: 54, Succeeded: 54, Failed: 0, > Warnings: 0 > > Result on SM8250: > > v4l2-compliance --stream-from /media/FVDO_Freeway_720p.264 -s250 > > v4l2-compliance 1.29.0-5273, 64 bits, 64-bit time_t > v4l2-compliance SHA: 0ed98432fe68 2024-11-13 12:54:45 > > Compliance test for iris_driver device /dev/video0: > > Driver Info: > Driver name : iris_driver > Card type : iris_decoder > Bus info : platform:aa00000.video-codec > Driver version : 6.12.0 > Capabilities : 0x84204000 > Video Memory-to-Memory Multiplanar > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x04204000 > Video Memory-to-Memory Multiplanar > Streaming > Extended Pix Format > Detected Stateful Decoder > > Required ioctls: > test VIDIOC_QUERYCAP: OK > test invalid ioctls: OK > > Allow for multiple opens: > test second /dev/video0 open: OK > test VIDIOC_QUERYCAP: OK > test VIDIOC_G/S_PRIORITY: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 0 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Control ioctls: > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK > test VIDIOC_QUERYCTRL: OK > test VIDIOC_G/S_CTRL: OK > test VIDIOC_G/S/TRY_EXT_CTRLS: OK > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 5 Private Controls: 0 > > Format ioctls: > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK > test VIDIOC_TRY_FMT: OK > test VIDIOC_S_FMT: OK > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK > test Composing: OK > test Scaling: OK (Not Supported) > > Codec ioctls: > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK > > Buffer ioctls: > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > test CREATE_BUFS maximum buffers: OK > test VIDIOC_REMOVE_BUFS: OK > test VIDIOC_EXPBUF: OK > test Requests: OK (Not Supported) > test blocking wait: OK > > Test input 0: > > Streaming ioctls: > test read/write: OK (Not Supported) > test MMAP (select, REQBUFS): OK > test MMAP (epoll, REQBUFS): OK > test MMAP (select, CREATE_BUFS): OK > test MMAP (epoll, CREATE_BUFS): OK > test USERPTR (select): OK (Not Supported) > test DMABUF: Cannot test, specify --expbuf-device > > Total for iris_driver device /dev/video0: 54, Succeeded: 54, Failed: 0, > Warnings: 0 > > The result of fluster test on SM8550: > > 77/135 while testing JVT-AVC_V1 with > GStreamer-H.264-V4L2-Gst1.0.JVT-AVC_V1. > The failing tests are: > - 52 test vectors failed due to interlaced clips: Interlaced decoding is > not supported in iris driver. > Test Vectors: > cabac_mot_fld0_full > cabac_mot_mbaff0_full > cabac_mot_picaff0_full > CABREF3_Sand_D > CAFI1_SVA_C > CAMA1_Sony_C > CAMA1_TOSHIBA_B > cama1_vtc_c > cama2_vtc_b > CAMA3_Sand_E > cama3_vtc_b > CAMACI3_Sony_C > CAMANL1_TOSHIBA_B > CAMANL2_TOSHIBA_B > CAMANL3_Sand_E > CAMASL3_Sony_B > CAMP_MOT_MBAFF_L30 > CAMP_MOT_MBAFF_L31 > CANLMA2_Sony_C > CANLMA3_Sony_C > CAPA1_TOSHIBA_B > CAPAMA3_Sand_F > cavlc_mot_fld0_full_B > cavlc_mot_mbaff0_full_B > cavlc_mot_picaff0_full_B > CVCANLMA2_Sony_C > CVFI1_Sony_D > CVFI1_SVA_C > CVFI2_Sony_H > CVFI2_SVA_C > CVMA1_Sony_D > CVMA1_TOSHIBA_B > CVMANL1_TOSHIBA_B > CVMANL2_TOSHIBA_B > CVMAPAQP3_Sony_E > CVMAQP2_Sony_G > CVMAQP3_Sony_D > CVMP_MOT_FLD_L30_B > CVNLFI1_Sony_C > CVNLFI2_Sony_H > CVPA1_TOSHIBA_B > FI1_Sony_E > MR6_BT_B > MR7_BT_B > MR8_BT_B > MR9_BT_B > Sharp_MP_Field_1_B > Sharp_MP_Field_2_B > Sharp_MP_Field_3_B > Sharp_MP_PAFF_1r2 > Sharp_MP_PAFF_2r > CVMP_MOT_FRM_L31_B > - 3 test vectors failed due to unsupported bitstream. > num_slice_group_minus1 greater than zero is not supported by the > hardware. > Test Vectors: > FM1_BT_B > FM1_FT_E > FM2_SVA_C > - 2 test vectors failed because SP_SLICE type is not supported by the > hardware. > Test Vectors: > SP1_BT_A > sp2_bt_b > - 1 test vector failed due to unsupported profile: > V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED is being deprecated from sm8550 > onwards due to hardware issues. > Test Vectors: > BA3_SVA_C > > 23/69 while testing JVT-FR-EXT with > GStreamer-H.264-V4L2-Gst1.0.JVT-AVC_V1. > The failing tests are: > - 21 test vectors failed due to interlaced clips: Interlaced decoding is > not supported in iris driver. > Test Vectors: > brcm_freh4 > brcm_freh5 > brcm_freh6 > brcm_freh10 > brcm_freh11 > freh7_b > FREXT01_JVC_D > FREXT02_JVC_C > FRExt2_Panasonic_C > FRExt4_Panasonic_B > HCAFF1_HHI_B > HCAMFF1_HHI_B > HCHP3_HHI_A > HPCAFL_BRCM_C > HPCAFLNL_BRCM_C > HVLCFI0_Sony_B > HVLCMFF0_Sony_B > HVLCPFF0_Sony_B > HPCAMAPALQ_BRCM_B > HPCVFL_BRCM_A > HPCVFLNL_BRCM_A > - 2 test vectors failed due to 10bit bitstream: 10bit decoding is not > supported in iris driver > Test Vectors: > FREH10-1 > FREH10-2 > - 21 test vectors failed due to unsupported profile: > V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_422 is not supported by the > hardware. > Test Vectors: > FREXT1_TANDBERG_A > FREXT2_TANDBERG_A > FREXT3_TANDBERG_A > Hi422FR1_SONY_A > Hi422FR2_SONY_A > Hi422FR3_SONY_A > Hi422FR4_SONY_A > Hi422FR6_SONY_A > Hi422FR7_SONY_A > Hi422FR8_SONY_A > Hi422FR9_SONY_A > Hi422FR10_SONY_A > Hi422FR11_SONY_A > Hi422FR12_SONY_A > Hi422FR13_SONY_A > Hi422FR14_SONY_A > Hi422FR15_SONY_A > Hi422FREXT16_SONY_A > Hi422FREXT17_SONY_A > Hi422FREXT18_SONY_A > Hi422FREXT19_SONY_A > - 2 test vectors failed due to unsupported bitstream. > chroma_fmt_idc is equal to 0(monochrome) in the bitstream which is not > supported by the hardware. > Test Vectors: > HPCAMOLQ_BRCM_B > HPCVMOLQ_BRCM_B > > The result of fluster test on SM8250: > > 78/135 while testing JVT-AVC_V1 with > GStreamer-H.264-V4L2-Gst1.0.JVT-AVC_V1. > The failing tests are: > - 52 test vectors failed due to interlaced clips: Interlaced decoding is > not supported in iris driver. > Test Vectors: > cabac_mot_fld0_full > cabac_mot_mbaff0_full > cabac_mot_picaff0_full > CABREF3_Sand_D > CAFI1_SVA_C > CAMA1_Sony_C > CAMA1_TOSHIBA_B > cama1_vtc_c > cama2_vtc_b > CAMA3_Sand_E > cama3_vtc_b > CAMACI3_Sony_C > CAMANL1_TOSHIBA_B > CAMANL2_TOSHIBA_B > CAMANL3_Sand_E > CAMASL3_Sony_B > CAMP_MOT_MBAFF_L30 > CAMP_MOT_MBAFF_L31 > CANLMA2_Sony_C > CANLMA3_Sony_C > CAPA1_TOSHIBA_B > CAPAMA3_Sand_F > cavlc_mot_fld0_full_B > cavlc_mot_mbaff0_full_B > cavlc_mot_picaff0_full_B > CVCANLMA2_Sony_C > CVFI1_Sony_D > CVFI1_SVA_C > CVFI2_Sony_H > CVFI2_SVA_C > CVMA1_Sony_D > CVMA1_TOSHIBA_B > CVMANL1_TOSHIBA_B > CVMANL2_TOSHIBA_B > CVMAPAQP3_Sony_E > CVMAQP2_Sony_G > CVMAQP3_Sony_D > CVMP_MOT_FLD_L30_B > CVNLFI1_Sony_C > CVNLFI2_Sony_H > CVPA1_TOSHIBA_B > FI1_Sony_E > MR6_BT_B > MR7_BT_B > MR8_BT_B > MR9_BT_B > Sharp_MP_Field_1_B > Sharp_MP_Field_2_B > Sharp_MP_Field_3_B > Sharp_MP_PAFF_1r2 > Sharp_MP_PAFF_2r > CVMP_MOT_FRM_L31_B > - 3 test vectors failed due to unsupported bitstream. > num_slice_group_minus1 greater than zero is not supported by the > hardware. > Test Vectors: > FM1_BT_B > FM1_FT_E > FM2_SVA_C > - 2 test vectors failed because SP_SLICE type is not supported by the > hardware. > Test Vectors: > SP1_BT_A > sp2_bt_b > > 23/69 while testing JVT-FR-EXT with > GStreamer-H.264-V4L2-Gst1.0.JVT-AVC_V1. > The failing tests are: > - 21 test vectors failed due to interlaced clips: Interlaced decoding is > not supported in iris driver. > Test Vectors: > brcm_freh4 > brcm_freh5 > brcm_freh6 > brcm_freh10 > brcm_freh11 > freh7_b > FREXT01_JVC_D > FREXT02_JVC_C > FRExt2_Panasonic_C > FRExt4_Panasonic_B > HCAFF1_HHI_B > HCAMFF1_HHI_B > HCHP3_HHI_A > HPCAFL_BRCM_C > HPCAFLNL_BRCM_C > HVLCFI0_Sony_B > HVLCMFF0_Sony_B > HVLCPFF0_Sony_B > HPCAMAPALQ_BRCM_B > HPCVFL_BRCM_A > HPCVFLNL_BRCM_A > - 2 test vectors failed due to 10bit bitstream: 10bit decoding is not > supported in iris driver > Test Vectors: > FREH10-1 > FREH10-2 > - 21 test vectors failed due to unsupported profile: > V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_422 is not supported by the > hardware. > Test Vectors: > FREXT1_TANDBERG_A > FREXT2_TANDBERG_A > FREXT3_TANDBERG_A > Hi422FR1_SONY_A > Hi422FR2_SONY_A > Hi422FR3_SONY_A > Hi422FR4_SONY_A > Hi422FR6_SONY_A > Hi422FR7_SONY_A > Hi422FR8_SONY_A > Hi422FR9_SONY_A > Hi422FR10_SONY_A > Hi422FR11_SONY_A > Hi422FR12_SONY_A > Hi422FR13_SONY_A > Hi422FR14_SONY_A > Hi422FR15_SONY_A > Hi422FREXT16_SONY_A > Hi422FREXT17_SONY_A > Hi422FREXT18_SONY_A > Hi422FREXT19_SONY_A > - 2 test vectors failed due to unsupported bitstream. > chroma_fmt_idc is equal to 0(monochrome) in the bitstream which is not > supported by the hardware. > Test Vectors: > HPCAMOLQ_BRCM_B > HPCVMOLQ_BRCM_B > > To: Vikash Garodia <quic_vgarodia@quicinc.com> > To: Abhinav Kumar <quic_abhinavk@quicinc.com> > To: Mauro Carvalho Chehab <mchehab@kernel.org> > To: Rob Herring <robh@kernel.org> > To: Krzysztof Kozlowski <krzk+dt@kernel.org> > To: Conor Dooley <conor+dt@kernel.org> > To: Philipp Zabel <p.zabel@pengutronix.de> > Cc: Hans Verkuil <hverkuil@xs4all.nl> > Cc: Sebastian Fricke <sebastian.fricke@collabora.com> > Cc: Bryan O'Donoghue <bryan.odonoghue@linaro.org> > Cc: Dmitry Baryshkov <dmitry.baryshkov@linaro.org> > Cc: Neil Armstrong <neil.armstrong@linaro.org> > Cc: Nicolas Dufresne <nicolas@ndufresne.ca> > Cc: Uwe Kleine-König <u.kleine-koenig@baylibre.com> > Cc: Jianhua Lu <lujianhua000@gmail.com> > Cc: Stefan Schmidt <stefan.schmidt@linaro.org> > Cc: linux-media@vger.kernel.org > Cc: linux-arm-msm@vger.kernel.org > Cc: devicetree@vger.kernel.org > Cc: linux-kernel@vger.kernel.org > > Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com> > --- > Dikshita Agarwal (18): > dt-bindings: media: Add video support for QCOM SM8550 SoC > media: iris: add platform driver for iris video device > media: iris: implement iris v4l2 file ops > media: iris: introduce iris core state management with shared queues > media: iris: implement video firmware load/unload > media: iris: implement boot sequence of the firmware > media: iris: introduce host firmware interface with necessary hooks > media: iris: implement power management > media: iris: implement reqbuf ioctl with vb2_queue_setup > media: iris: implement iris v4l2_ctrl_ops > media: iris: implement vb2 streaming ops > media: iris: allocate, initialize and queue internal buffers > media: iris: implement vb2 ops for buf_queue and firmware response > media: iris: add support for dynamic resolution change > media: iris: handle streamoff/on from client in dynamic resolution change > media: iris: add support for drain sequence > media: iris: enable video driver probe of SM8250 SoC > media: MAINTAINERS: add Qualcomm iris video accelerator driver > > Vedang Nagar (10): > media: iris: implement s_fmt, g_fmt and try_fmt ioctls > media: iris: implement g_selection ioctl > media: iris: implement enum_fmt and enum_framesizes ioctls > media: iris: implement subscribe_event and unsubscribe_event ioctls > media: iris: implement query_cap ioctl > media: iris: implement set properties to firmware during streamon > media: iris: subscribe parameters and properties to firmware for hfi_gen2 > media: iris: add check whether the video session is supported or not > media: iris: implement power scaling for vpu2 and vpu3 > media: iris: add check to allow sub states transitions > > .../bindings/media/qcom,sm8550-iris.yaml | 158 ++++ > MAINTAINERS | 10 + > drivers/media/platform/qcom/Kconfig | 1 + > drivers/media/platform/qcom/Makefile | 1 + > drivers/media/platform/qcom/iris/Kconfig | 13 + > drivers/media/platform/qcom/iris/Makefile | 27 + > drivers/media/platform/qcom/iris/iris_buffer.c | 623 ++++++++++++++ > drivers/media/platform/qcom/iris/iris_buffer.h | 117 +++ > drivers/media/platform/qcom/iris/iris_core.c | 96 +++ > drivers/media/platform/qcom/iris/iris_core.h | 111 +++ > drivers/media/platform/qcom/iris/iris_ctrls.c | 259 ++++++ > drivers/media/platform/qcom/iris/iris_ctrls.h | 22 + > drivers/media/platform/qcom/iris/iris_firmware.c | 116 +++ > drivers/media/platform/qcom/iris/iris_firmware.h | 15 + > drivers/media/platform/qcom/iris/iris_hfi_common.c | 176 ++++ > drivers/media/platform/qcom/iris/iris_hfi_common.h | 155 ++++ > drivers/media/platform/qcom/iris/iris_hfi_gen1.h | 16 + > .../platform/qcom/iris/iris_hfi_gen1_command.c | 826 ++++++++++++++++++ > .../platform/qcom/iris/iris_hfi_gen1_defines.h | 448 ++++++++++ > .../platform/qcom/iris/iris_hfi_gen1_response.c | 666 ++++++++++++++ > drivers/media/platform/qcom/iris/iris_hfi_gen2.h | 41 + > .../platform/qcom/iris/iris_hfi_gen2_command.c | 957 +++++++++++++++++++++ > .../platform/qcom/iris/iris_hfi_gen2_defines.h | 161 ++++ > .../platform/qcom/iris/iris_hfi_gen2_packet.c | 292 +++++++ > .../platform/qcom/iris/iris_hfi_gen2_packet.h | 125 +++ > .../platform/qcom/iris/iris_hfi_gen2_response.c | 934 ++++++++++++++++++++ > drivers/media/platform/qcom/iris/iris_hfi_queue.c | 314 +++++++ > drivers/media/platform/qcom/iris/iris_hfi_queue.h | 182 ++++ > drivers/media/platform/qcom/iris/iris_instance.h | 77 ++ > .../platform/qcom/iris/iris_platform_common.h | 186 ++++ > .../platform/qcom/iris/iris_platform_sm8250.c | 148 ++++ > .../platform/qcom/iris/iris_platform_sm8550.c | 265 ++++++ > drivers/media/platform/qcom/iris/iris_power.c | 140 +++ > drivers/media/platform/qcom/iris/iris_power.h | 13 + > drivers/media/platform/qcom/iris/iris_probe.c | 378 ++++++++ > drivers/media/platform/qcom/iris/iris_resources.c | 131 +++ > drivers/media/platform/qcom/iris/iris_resources.h | 18 + > drivers/media/platform/qcom/iris/iris_state.c | 276 ++++++ > drivers/media/platform/qcom/iris/iris_state.h | 144 ++++ > drivers/media/platform/qcom/iris/iris_utils.c | 90 ++ > drivers/media/platform/qcom/iris/iris_utils.h | 53 ++ > drivers/media/platform/qcom/iris/iris_vb2.c | 335 ++++++++ > drivers/media/platform/qcom/iris/iris_vb2.h | 19 + > drivers/media/platform/qcom/iris/iris_vdec.c | 659 ++++++++++++++ > drivers/media/platform/qcom/iris/iris_vdec.h | 25 + > drivers/media/platform/qcom/iris/iris_vidc.c | 453 ++++++++++ > drivers/media/platform/qcom/iris/iris_vidc.h | 15 + > drivers/media/platform/qcom/iris/iris_vpu2.c | 38 + > drivers/media/platform/qcom/iris/iris_vpu3.c | 122 +++ > drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 270 ++++++ > drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 91 ++ > drivers/media/platform/qcom/iris/iris_vpu_common.c | 369 ++++++++ > drivers/media/platform/qcom/iris/iris_vpu_common.h | 28 + > .../platform/qcom/iris/iris_vpu_register_defines.h | 17 + > 54 files changed, 11222 insertions(+) > --- > base-commit: 698b6e3163bafd61e1b7d13572e2c42974ac85ec > change-id: 20241028-qcom-video-iris-94d5a12e6d9e > > Best regards, Tested-by: Neil Armstrong <neil.armstrong@linaro.org> # on SM8550-QRD Tested-by: Neil Armstrong <neil.armstrong@linaro.org> # on SM8550-HDK Thanks, Neil
On 12/12/2024 12:51, Dikshita Agarwal wrote: > Introduce support for Qualcomm new video acceleration hardware i.e. > iris, used for video stream decoding. > > Iris is a multi pipe based hardware that offloads video stream decoding > from the application processor (AP). It supports H.264 decoding. The AP > communicates with hardware through a well defined protocol, called as > host firmware interface (HFI), which provides fine-grained and > asynchronous control over individual hardware features. > > This driver implements upgraded HFI gen2 to communicate with firmware. > It supports SM8550 which is based out of HFI gen 2. It also supports > SM8250 which is based out of HFI gen1. > Thank you for your contributions. This patchset is relatively big, so please slow down with sending new versions to one per few days, not one per 24h. This gives people more chances to review. Best regards, Krzysztof
© 2016 - 2025 Red Hat, Inc.