drivers/media/platform/qcom/iris/iris_buffer.h | 2 + drivers/media/platform/qcom/iris/iris_ctrls.c | 8 + drivers/media/platform/qcom/iris/iris_hfi_common.h | 3 + .../platform/qcom/iris/iris_hfi_gen2_command.c | 109 ++++++++- .../platform/qcom/iris/iris_hfi_gen2_defines.h | 10 + .../platform/qcom/iris/iris_hfi_gen2_response.c | 22 ++ drivers/media/platform/qcom/iris/iris_instance.h | 1 + .../platform/qcom/iris/iris_platform_common.h | 15 ++ .../media/platform/qcom/iris/iris_platform_gen2.c | 156 ++++++++++++- .../platform/qcom/iris/iris_platform_sm8250.c | 17 ++ drivers/media/platform/qcom/iris/iris_vdec.c | 25 +- drivers/media/platform/qcom/iris/iris_vidc.c | 1 + drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 255 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 105 +++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/uapi/linux/videodev2.h | 1 + 16 files changed, 703 insertions(+), 28 deletions(-)
Hi all,
This patch series adds initial support for the AV1 stateful
decoder codecs in iris decoder. Also it adds support for AV1
stateful decoder in V4l2. The objective of this work is to
extend the Iris decoder's capabilities to handle AV1 format
codec streams, including necessary format handling and buffer
management. I'm sharing this series as an RFC because conformance
testing and gstreamer testing are still in progress. While initial
functional tests show positive results, I would appreciate early
feedback on the design, implementation, and fixes before moving to
a formal submission. I plan to submit a formal patch series after
completing all the compliance checks. Meanwhile, any feedback or
suggestion to improve this work are very welcome and will be of
great help.
Gstreamer testing:
Gstreamer MR for enabling AV1 stateful decoder:
https://gitlab.freedesktop.org/dmadival/gstreamer/-/merge_requests/1
Thanks to Nicolas Dufresne for proving the MR
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9469
However, Gst testing with parsebin is not resolving to av1parser as
below:
Ex: With the following command parsebin is unable to resolve to
av1parser.
GST_DEBUG=*:2,parsebin:6 gst-launch-1.0 --no-fault
filesrc
location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/
argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu
! parsebin ! v4l2av1dec ! video/x-raw ! videoconvert dither=none !
video/x-raw,format=I420
! filesink location=gst_decoder_output.yuv
0:00:00.051674896 400 0xffff8c000b90 DEBUG
parsebin gstparsebin.c:2439:type_found:<parsebin0> typefind found caps
video/x-h263, variant=(string)itu
The same test with the av1parse command parses correctly:
GST_DEBUG=*:2,av1parse:6 gst-launch-1.0 --no-fault
filesrc
location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/
argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu
! av1parse ! v4l2av1dec ! video/x-raw ! videoconvert dither=none !
video/x-raw,format=I420
! filesink location=/tmp/gst_decoder_output.yuv
Got EOS from element "pipeline0".
Execution ended after 0:00:01.599088176
Setting pipeline to NULL ...
0:00:03.580831249 1075 0x3354f960 DEBUG
av1parse gstav1parse.c:435:gst_av1_parse_stop:<av1parse0> stop
Fluster testing:
As fluster.py is using parsebin for gstreamer, seeing the same issue as
described above for the following testsuites.
AV1-ARGON-PROFILE0-CORE-ANNEX-B
AV1-ARGON-PROFILE0-NON-ANNEX-B
AV1-ARGON-PROFILE0-NON-ANNEX-B
Test suite: AV1-TEST-VECTORS
The result of fluster test on SM8550:
134/242 testcases passed while testing AV1-TEST-VECTORS with
GStreamer-AV1-V4L2-Gst1.0
unsupported content, bit depth: a000a (66 tests)
Iris hardware decoder supports only 8bit NV12
av1-1-b10-00-quantizer-*
Unsupported resolution (36 tests).
Iris hardware decoder supports min resolution of 96x96
av1-1-b8-01-size-*
Unsupported colorformat (1 test)
av1-1-b8-24-monochrome
Crc mismatch: debug in progress (5tests)
av1-1-b8-03-sizeup
av1-1-b8-03-sizedown
av1-1-b8-16-intra_only-intrabc-extreme-dv
av1-1-b8-22-svc-L2T1
av1-1-b8-22-svc-L2T2
Testsuite: CHROMIUM-8bit-AV1-TEST-VECTORS
12/13 testcases passed while testing CHROMIUM-8bit-AV1-TEST-VECTORS with
GStreamer-AV1-V4L2-Gst1.0
Crc mismatch: debug in progress
av1-1-b8-03-sizeup.ivf
Unsupported test suites:
Iris Hardware Decoder supports only
PROFILE0/V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN
and 8 bit, 420 only
AV1-ARGON-PROFILE1-CORE-ANNEX-B
AV1-ARGON-PROFILE1-NON-ANNEX-B
AV1-ARGON-PROFILE1-STRESS-ANNEX-B
AV1-ARGON-PROFILE2-CORE-ANNEX-B
AV1-ARGON-PROFILE2-NON-ANNEX-B
AV1-ARGON-PROFILE2-STRESS-ANNEX-B
CHROMIUM-10bit-AV1-TEST-VECTORS
Compliance test for iris_driver device /dev/video0:
Driver Info:
Driver name : iris_driver
Card type : Iris Decoder
Bus info : platform:aa00000.video-codec
Driver version : 6.16.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Detected Stateful Decoder
Required ioctls:
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 12 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK
test Composing: OK
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
[65391.311675] qcom-iris aa00000.video-codec: invalid plane
[65395.340586] qcom-iris aa00000.video-codec: invalid plane
test blocking wait: OK
Total for iris_driver device /dev/video0: 48, Succeeded: 48, Failed: 0,
Warnings: 0
V4l2-ctl Test verified for 2 streams as well.
Thanks,
Deepa
Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
---
DEEPA GUTHYAPPA MADIVALARA (5):
media: uapi: videodev2: Add support for AV1 stateful decoder
media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc()
media: iris: Add support for AV1 format in iris decoder
media: iris: Add internal buffer calculation for AV1 decoder
media: iris: Define AV1-specific platform capabilities and properties
drivers/media/platform/qcom/iris/iris_buffer.h | 2 +
drivers/media/platform/qcom/iris/iris_ctrls.c | 8 +
drivers/media/platform/qcom/iris/iris_hfi_common.h | 3 +
.../platform/qcom/iris/iris_hfi_gen2_command.c | 109 ++++++++-
.../platform/qcom/iris/iris_hfi_gen2_defines.h | 10 +
.../platform/qcom/iris/iris_hfi_gen2_response.c | 22 ++
drivers/media/platform/qcom/iris/iris_instance.h | 1 +
.../platform/qcom/iris/iris_platform_common.h | 15 ++
.../media/platform/qcom/iris/iris_platform_gen2.c | 156 ++++++++++++-
.../platform/qcom/iris/iris_platform_sm8250.c | 17 ++
drivers/media/platform/qcom/iris/iris_vdec.c | 25 +-
drivers/media/platform/qcom/iris/iris_vidc.c | 1 +
drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 255 ++++++++++++++++++++-
drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 105 +++++++++
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/videodev2.h | 1 +
16 files changed, 703 insertions(+), 28 deletions(-)
---
base-commit: 88a6b4187eacb700a678296afb0c610eb3781e2f
change-id: 20250821-rfc_split-c3ff834bb2c9
prerequisite-change-id: 20250704-iris-video-encoder-b193350b487a:v3
prerequisite-patch-id: 8a566690da276da34430c10dbc2fe64c1d623a9c
prerequisite-patch-id: 1430a33603b425d0b142aab98befcda771fb885e
prerequisite-patch-id: 32024cd49d2445ff396e31f40739b32597be59a4
prerequisite-patch-id: 65b569952650647174e8221dc7adde9b000a7ae3
prerequisite-patch-id: da128980fab8538bf668f19016c5121fb03759c2
prerequisite-patch-id: 079823dffbe8b89990797bf7f7640b754382d8ce
prerequisite-patch-id: 6ce10e03d7b3b96b2391e26cda703b650bde7cd0
prerequisite-patch-id: b5950670ac5068a0c5b26651ebd433f7d3bbe6ca
prerequisite-patch-id: 4de7a934f6bdfe28c84e461f70495925aa98365e
prerequisite-patch-id: 07682a6d2530b5796122bf8763f94b5bc92949ec
prerequisite-patch-id: 72b7eba20f1a222908d41323f28be3ba84106759
prerequisite-patch-id: fd9e2e1b157112c39c69486799493ee99e6033a7
prerequisite-patch-id: ae0ad8a04a04dd3434a092d4c2bb3f493417c6e1
prerequisite-patch-id: 52631eec348735d1dc5f5804b573e3cf942550a0
prerequisite-patch-id: 4109c59edb1b757162db46297914c8f7c14408dc
prerequisite-patch-id: fc0b713eb4822047e8172d11fd4cd5a097ef23a5
prerequisite-patch-id: 20ac8e7307f1f852b2a43268b2474178fbc0b94c
prerequisite-patch-id: e8419d716573beb64ad89968f0074d6bddfa86d3
prerequisite-patch-id: bdc72f5876ceb2e981d594c86a45cb21a6264af3
prerequisite-patch-id: fc5d26d01cab94d229a00eab819ae80196f3f5d5
prerequisite-patch-id: 32a9fe1371fffc9abd9a862b2814050a144d1968
prerequisite-patch-id: 25184583b5de886f78ee0444a4a59d5f3c271ce5
prerequisite-patch-id: df3376b9de27b23ae81a4c7b7a8fe4b429c32423
prerequisite-patch-id: e68fbf7c82567d2e9f3fdd0fdf2e2911329d5ccd
prerequisite-patch-id: 580fa40de01a81a8685e56420f562d299bfc60fa
prerequisite-patch-id: 89548da6690681854ee1de992a491bed73202b83
Best regards,
--
Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
Le mardi 02 septembre 2025 à 16:00 -0700, DEEPA GUTHYAPPA MADIVALARA a écrit : > Hi all, > > This patch series adds initial support for the AV1 stateful > decoder codecs in iris decoder. Also it adds support for AV1 > stateful decoder in V4l2. The objective of this work is to > extend the Iris decoder's capabilities to handle AV1 format > codec streams, including necessary format handling and buffer > management. I'm sharing this series as an RFC because conformance > testing and gstreamer testing are still in progress. While initial > functional tests show positive results, I would appreciate early > feedback on the design, implementation, and fixes before moving to > a formal submission. I plan to submit a formal patch series after > completing all the compliance checks. Meanwhile, any feedback or > suggestion to improve this work are very welcome and will be of > great help. > > Gstreamer testing: > Gstreamer MR for enabling AV1 stateful decoder: > https://gitlab.freedesktop.org/dmadival/gstreamer/-/merge_requests/1 > > Thanks to Nicolas Dufresne for proving the MR > https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9469 > > However, Gst testing with parsebin is not resolving to av1parser as > below: > Ex: With the following command parsebin is unable to resolve to > av1parser. > GST_DEBUG=*:2,parsebin:6 gst-launch-1.0 --no-fault > filesrc > location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/ > argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu > ! parsebin ! v4l2av1dec ! video/x-raw ! videoconvert dither=none ! > video/x-raw,format=I420 > ! filesink location=gst_decoder_output.yuv Be aware that Argon integration with fluster/GStreamer still needs some work, see issue tracker: https://github.com/fluendo/fluster/issues/222 It is fine to proceed with testing the other (IVF or MKV based) test suites. Feel free to report all wrongly identified files onto that issue, this will be addressed in GStreamer (and eventually in FFMPEG too if need be). > > 0:00:00.051674896 400 0xffff8c000b90 DEBUG > parsebin gstparsebin.c:2439:type_found:<parsebin0> typefind found caps > video/x-h263, variant=(string)itu > > The same test with the av1parse command parses correctly: > GST_DEBUG=*:2,av1parse:6 gst-launch-1.0 --no-fault > filesrc > location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/ > argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu > ! av1parse ! v4l2av1dec ! video/x-raw ! videoconvert dither=none ! > video/x-raw,format=I420 > ! filesink location=/tmp/gst_decoder_output.yuv > > Got EOS from element "pipeline0". > Execution ended after 0:00:01.599088176 > Setting pipeline to NULL ... > 0:00:03.580831249 1075 0x3354f960 DEBUG > av1parse gstav1parse.c:435:gst_av1_parse_stop:<av1parse0> stop > > Fluster testing: > As fluster.py is using parsebin for gstreamer, seeing the same issue as > described above for the following testsuites. > AV1-ARGON-PROFILE0-CORE-ANNEX-B > AV1-ARGON-PROFILE0-NON-ANNEX-B > AV1-ARGON-PROFILE0-NON-ANNEX-B > > Test suite: AV1-TEST-VECTORS > The result of fluster test on SM8550: > 134/242 testcases passed while testing AV1-TEST-VECTORS with > GStreamer-AV1-V4L2-Gst1.0 > unsupported content, bit depth: a000a (66 tests) > Iris hardware decoder supports only 8bit NV12 > av1-1-b10-00-quantizer-* That is interesting, I believe there was no profile without 10bit, which would mean this is non-compliant hardware. Not a blocker, just a remark, and me being surprised 10bit isn't supported on modern codec in 2025. > > Unsupported resolution (36 tests). > Iris hardware decoder supports min resolution of 96x96 > av1-1-b8-01-size-* > > Unsupported colorformat (1 test) > av1-1-b8-24-monochrome Fair enough. > > Crc mismatch: debug in progress (5tests) > av1-1-b8-03-sizeup > av1-1-b8-03-sizedown These should be basic dynamic resolution changes cases, let me know what you find. > av1-1-b8-16-intra_only-intrabc-extreme-dv No idea about this one. > av1-1-b8-22-svc-L2T1 > av1-1-b8-22-svc-L2T2 These two are spatial SVC. That means that some decode only frames are going to be smaller in dimensions. Only the last frame of a TU is displayed. Both do have the optional sequence header announcing the maximum resolution though. On stateless side, these usually works by simply decoding the hidden frames into the much larger buffers. Dealing with firmware can be harder, since firmware may silently drop the decode only frames, which leads to wrong timestamp matching, which could have side effect in frameworks. On stateful decoding, SVC have never been done, some documentation will likely be needed. Strictly unsupported is the case the sequence header is not there, which results in resolution change on non-keyframe. We now have the ability to allocate frames of different dimension at run-time, and free old unused frames, but in stateful, decoders streamoff the capture queue, which currently results in lost of references. Let us know your progress, inter-frame resolution change is not a strict requirement, it is quite rare in real life. looking forward, there is nothing particularly alarming in this report. Thanks for the transparency. Nicolas > > Testsuite: CHROMIUM-8bit-AV1-TEST-VECTORS > 12/13 testcases passed while testing CHROMIUM-8bit-AV1-TEST-VECTORS with > GStreamer-AV1-V4L2-Gst1.0 > Crc mismatch: debug in progress > av1-1-b8-03-sizeup.ivf > > Unsupported test suites: > Iris Hardware Decoder supports only > PROFILE0/V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN > and 8 bit, 420 only > AV1-ARGON-PROFILE1-CORE-ANNEX-B > AV1-ARGON-PROFILE1-NON-ANNEX-B > AV1-ARGON-PROFILE1-STRESS-ANNEX-B > AV1-ARGON-PROFILE2-CORE-ANNEX-B > AV1-ARGON-PROFILE2-NON-ANNEX-B > AV1-ARGON-PROFILE2-STRESS-ANNEX-B > CHROMIUM-10bit-AV1-TEST-VECTORS > > Compliance test for iris_driver device /dev/video0: > > Driver Info: > Driver name : iris_driver > Card type : Iris Decoder > Bus info : platform:aa00000.video-codec > Driver version : 6.16.0 > Capabilities : 0x84204000 > Video Memory-to-Memory Multiplanar > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x04204000 > Video Memory-to-Memory Multiplanar > Streaming > Extended Pix Format > Detected Stateful Decoder > > Required ioctls: > test VIDIOC_QUERYCAP: OK > test invalid ioctls: OK > > Allow for multiple opens: > test second /dev/video0 open: OK > test VIDIOC_QUERYCAP: OK > test VIDIOC_G/S_PRIORITY: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 0 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Control ioctls: > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK > test VIDIOC_QUERYCTRL: OK > test VIDIOC_G/S_CTRL: OK > test VIDIOC_G/S/TRY_EXT_CTRLS: OK > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 12 Private Controls: 0 > > Format ioctls: > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK > test VIDIOC_TRY_FMT: OK > test VIDIOC_S_FMT: OK > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK > test Composing: OK > test Scaling: OK (Not Supported) > > Codec ioctls: > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK > > Buffer ioctls: > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > test CREATE_BUFS maximum buffers: OK > test VIDIOC_REMOVE_BUFS: OK > test VIDIOC_EXPBUF: OK > test Requests: OK (Not Supported) > [65391.311675] qcom-iris aa00000.video-codec: invalid plane > [65395.340586] qcom-iris aa00000.video-codec: invalid plane > test blocking wait: OK > > Total for iris_driver device /dev/video0: 48, Succeeded: 48, Failed: 0, > Warnings: 0 > > V4l2-ctl Test verified for 2 streams as well. > > Thanks, > Deepa > > Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com> > --- > DEEPA GUTHYAPPA MADIVALARA (5): > media: uapi: videodev2: Add support for AV1 stateful decoder > media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc() > media: iris: Add support for AV1 format in iris decoder > media: iris: Add internal buffer calculation for AV1 decoder > media: iris: Define AV1-specific platform capabilities and properties > > drivers/media/platform/qcom/iris/iris_buffer.h | 2 + > drivers/media/platform/qcom/iris/iris_ctrls.c | 8 + > drivers/media/platform/qcom/iris/iris_hfi_common.h | 3 + > .../platform/qcom/iris/iris_hfi_gen2_command.c | 109 ++++++++- > .../platform/qcom/iris/iris_hfi_gen2_defines.h | 10 + > .../platform/qcom/iris/iris_hfi_gen2_response.c | 22 ++ > drivers/media/platform/qcom/iris/iris_instance.h | 1 + > .../platform/qcom/iris/iris_platform_common.h | 15 ++ > .../media/platform/qcom/iris/iris_platform_gen2.c | 156 ++++++++++++- > .../platform/qcom/iris/iris_platform_sm8250.c | 17 ++ > drivers/media/platform/qcom/iris/iris_vdec.c | 25 +- > drivers/media/platform/qcom/iris/iris_vidc.c | 1 + > drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 255 ++++++++++++++++++++- > drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 105 +++++++++ > drivers/media/v4l2-core/v4l2-ioctl.c | 1 + > include/uapi/linux/videodev2.h | 1 + > 16 files changed, 703 insertions(+), 28 deletions(-) > --- > base-commit: 88a6b4187eacb700a678296afb0c610eb3781e2f > change-id: 20250821-rfc_split-c3ff834bb2c9 > prerequisite-change-id: 20250704-iris-video-encoder-b193350b487a:v3 > prerequisite-patch-id: 8a566690da276da34430c10dbc2fe64c1d623a9c > prerequisite-patch-id: 1430a33603b425d0b142aab98befcda771fb885e > prerequisite-patch-id: 32024cd49d2445ff396e31f40739b32597be59a4 > prerequisite-patch-id: 65b569952650647174e8221dc7adde9b000a7ae3 > prerequisite-patch-id: da128980fab8538bf668f19016c5121fb03759c2 > prerequisite-patch-id: 079823dffbe8b89990797bf7f7640b754382d8ce > prerequisite-patch-id: 6ce10e03d7b3b96b2391e26cda703b650bde7cd0 > prerequisite-patch-id: b5950670ac5068a0c5b26651ebd433f7d3bbe6ca > prerequisite-patch-id: 4de7a934f6bdfe28c84e461f70495925aa98365e > prerequisite-patch-id: 07682a6d2530b5796122bf8763f94b5bc92949ec > prerequisite-patch-id: 72b7eba20f1a222908d41323f28be3ba84106759 > prerequisite-patch-id: fd9e2e1b157112c39c69486799493ee99e6033a7 > prerequisite-patch-id: ae0ad8a04a04dd3434a092d4c2bb3f493417c6e1 > prerequisite-patch-id: 52631eec348735d1dc5f5804b573e3cf942550a0 > prerequisite-patch-id: 4109c59edb1b757162db46297914c8f7c14408dc > prerequisite-patch-id: fc0b713eb4822047e8172d11fd4cd5a097ef23a5 > prerequisite-patch-id: 20ac8e7307f1f852b2a43268b2474178fbc0b94c > prerequisite-patch-id: e8419d716573beb64ad89968f0074d6bddfa86d3 > prerequisite-patch-id: bdc72f5876ceb2e981d594c86a45cb21a6264af3 > prerequisite-patch-id: fc5d26d01cab94d229a00eab819ae80196f3f5d5 > prerequisite-patch-id: 32a9fe1371fffc9abd9a862b2814050a144d1968 > prerequisite-patch-id: 25184583b5de886f78ee0444a4a59d5f3c271ce5 > prerequisite-patch-id: df3376b9de27b23ae81a4c7b7a8fe4b429c32423 > prerequisite-patch-id: e68fbf7c82567d2e9f3fdd0fdf2e2911329d5ccd > prerequisite-patch-id: 580fa40de01a81a8685e56420f562d299bfc60fa > prerequisite-patch-id: 89548da6690681854ee1de992a491bed73202b83 > > Best regards,
On 9/3/2025 6:07 AM, Nicolas Dufresne wrote: > Le mardi 02 septembre 2025 à 16:00 -0700, DEEPA GUTHYAPPA MADIVALARA a écrit : >> Hi all, >> >> This patch series adds initial support for the AV1 stateful >> decoder codecs in iris decoder. Also it adds support for AV1 >> stateful decoder in V4l2. The objective of this work is to >> extend the Iris decoder's capabilities to handle AV1 format >> codec streams, including necessary format handling and buffer >> management. I'm sharing this series as an RFC because conformance >> testing and gstreamer testing are still in progress. While initial >> functional tests show positive results, I would appreciate early >> feedback on the design, implementation, and fixes before moving to >> a formal submission. I plan to submit a formal patch series after >> completing all the compliance checks. Meanwhile, any feedback or >> suggestion to improve this work are very welcome and will be of >> great help. >> >> Gstreamer testing: >> Gstreamer MR for enabling AV1 stateful decoder: >> https://gitlab.freedesktop.org/dmadival/gstreamer/-/merge_requests/1 Hi Nicholas, Thank you so much for a quick response. Can you please provide feedback for the AV1 stateful decoder plugin - GST MR ? Thanks to Nicolas Dufresne for proving the MR https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9469 However, Gst testing with parsebin is not resolving to av1parser as below: Ex: With the following command parsebin is unable to resolve to av1parser. GST_DEBUG=*:2,parsebin:6 gst-launch-1.0 --no-fault filesrc location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/ argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu ! parsebin ! v4l2av1dec ! video/x-raw ! videoconvert dither=none ! video/x-raw,format=I420 ! filesink location=gst_decoder_output.yuv > Be aware that Argon integration with fluster/GStreamer still needs some work, > see issue tracker: > > https://github.com/fluendo/fluster/issues/222 > > It is fine to proceed with testing the other (IVF or MKV based) test suites. > Feel free to report all wrongly identified files onto that issue, this will be > addressed in GStreamer (and eventually in FFMPEG too if need be). Thanks for the above information. I will continue to work on ivf and mkv based test-suites. Is it okay to post the AV1 patch series even though Argon test-suite has failures? >> 0:00:00.051674896 400 0xffff8c000b90 DEBUG >> parsebin gstparsebin.c:2439:type_found:<parsebin0> typefind found caps >> video/x-h263, variant=(string)itu >> >> The same test with the av1parse command parses correctly: >> GST_DEBUG=*:2,av1parse:6 gst-launch-1.0 --no-fault >> filesrc >> location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/ >> argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu >> ! av1parse ! v4l2av1dec ! video/x-raw ! videoconvert dither=none ! >> video/x-raw,format=I420 >> ! filesink location=/tmp/gst_decoder_output.yuv >> >> Got EOS from element "pipeline0". >> Execution ended after 0:00:01.599088176 >> Setting pipeline to NULL ... >> 0:00:03.580831249 1075 0x3354f960 DEBUG >> av1parse gstav1parse.c:435:gst_av1_parse_stop:<av1parse0> stop >> >> Fluster testing: >> As fluster.py is using parsebin for gstreamer, seeing the same issue as >> described above for the following testsuites. >> AV1-ARGON-PROFILE0-CORE-ANNEX-B >> AV1-ARGON-PROFILE0-NON-ANNEX-B >> AV1-ARGON-PROFILE0-NON-ANNEX-B >> >> Test suite: AV1-TEST-VECTORS >> The result of fluster test on SM8550: >> 134/242 testcases passed while testing AV1-TEST-VECTORS with >> GStreamer-AV1-V4L2-Gst1.0 >> unsupported content, bit depth: a000a (66 tests) >> Iris hardware decoder supports only 8bit NV12 >> av1-1-b10-00-quantizer-* > That is interesting, I believe there was no profile without 10bit, which would > mean this is non-compliant hardware. Not a blocker, just a remark, and me being > surprised 10bit isn't supported on modern codec in 2025. Sorry, hardware does support 10bit. As we continue to add more features, we plan to include support for additional color formats as well. >> Unsupported resolution (36 tests). >> Iris hardware decoder supports min resolution of 96x96 >> av1-1-b8-01-size-* >> >> Unsupported colorformat (1 test) >> av1-1-b8-24-monochrome > Fair enough. > >> Crc mismatch: debug in progress (5tests) >> av1-1-b8-03-sizeup >> av1-1-b8-03-sizedown > These should be basic dynamic resolution changes cases, let me know what you > find. I have a corruption after second sequence change, which propagates further leading to hardware hang. I am working on fixing this. >> av1-1-b8-16-intra_only-intrabc-extreme-dv > No idea about this one. This is passing actually. It is just an all intra bitstream. >> av1-1-b8-22-svc-L2T1 >> av1-1-b8-22-svc-L2T2 > These two are spatial SVC. That means that some decode only frames are going to > be smaller in dimensions. Only the last frame of a TU is displayed. Both do have > the optional sequence header announcing the maximum resolution though. On > stateless side, these usually works by simply decoding the hidden frames into > the much larger buffers. Dealing with firmware can be harder, since firmware may > silently drop the decode only frames, which leads to wrong timestamp matching, > which could have side effect in frameworks. > > On stateful decoding, SVC have never been done, some documentation will likely > be needed. Strictly unsupported is the case the sequence header is not there, > which results in resolution change on non-keyframe. We now have the ability to > allocate frames of different dimension at run-time, and free old unused frames, > but in stateful, decoders streamoff the capture queue, which currently results > in lost of references. > > Let us know your progress, inter-frame resolution change is not a strict > requirement, it is quite rare in real life. > > looking forward, there is nothing particularly alarming in this report. Thanks > for the transparency. > > Nicolas I did not have a chance into debug this yet. But these streams do have a resolution change on non-keyframe. >> Testsuite: CHROMIUM-8bit-AV1-TEST-VECTORS >> 12/13 testcases passed while testing CHROMIUM-8bit-AV1-TEST-VECTORS with >> GStreamer-AV1-V4L2-Gst1.0 >> Crc mismatch: debug in progress >> av1-1-b8-03-sizeup.ivf Same corruption after second sequence change, which propagates further leading to hardware hang. >> Unsupported test suites: >> Iris Hardware Decoder supports only >> PROFILE0/V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN >> and 8 bit, 420 only >> AV1-ARGON-PROFILE1-CORE-ANNEX-B >> AV1-ARGON-PROFILE1-NON-ANNEX-B >> AV1-ARGON-PROFILE1-STRESS-ANNEX-B >> AV1-ARGON-PROFILE2-CORE-ANNEX-B >> AV1-ARGON-PROFILE2-NON-ANNEX-B >> AV1-ARGON-PROFILE2-STRESS-ANNEX-B >> CHROMIUM-10bit-AV1-TEST-VECTORS >> >> Compliance test for iris_driver device /dev/video0: >> >> Driver Info: >> Driver name : iris_driver >> Card type : Iris Decoder >> Bus info : platform:aa00000.video-codec >> Driver version : 6.16.0 >> Capabilities : 0x84204000 >> Video Memory-to-Memory Multiplanar >> Streaming >> Extended Pix Format >> Device Capabilities >> Device Caps : 0x04204000 >> Video Memory-to-Memory Multiplanar >> Streaming >> Extended Pix Format >> Detected Stateful Decoder >> >> Required ioctls: >> test VIDIOC_QUERYCAP: OK >> test invalid ioctls: OK >> >> Allow for multiple opens: >> test second /dev/video0 open: OK >> test VIDIOC_QUERYCAP: OK >> test VIDIOC_G/S_PRIORITY: OK >> test for unlimited opens: OK >> >> Debug ioctls: >> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) >> test VIDIOC_LOG_STATUS: OK (Not Supported) >> >> Input ioctls: >> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) >> test VIDIOC_G/S_FREQUENCY: OK (Not Supported) >> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) >> test VIDIOC_ENUMAUDIO: OK (Not Supported) >> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) >> test VIDIOC_G/S_AUDIO: OK (Not Supported) >> Inputs: 0 Audio Inputs: 0 Tuners: 0 >> >> Output ioctls: >> test VIDIOC_G/S_MODULATOR: OK (Not Supported) >> test VIDIOC_G/S_FREQUENCY: OK (Not Supported) >> test VIDIOC_ENUMAUDOUT: OK (Not Supported) >> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) >> test VIDIOC_G/S_AUDOUT: OK (Not Supported) >> Outputs: 0 Audio Outputs: 0 Modulators: 0 >> >> Input/Output configuration ioctls: >> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) >> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) >> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) >> test VIDIOC_G/S_EDID: OK (Not Supported) >> >> Control ioctls: >> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK >> test VIDIOC_QUERYCTRL: OK >> test VIDIOC_G/S_CTRL: OK >> test VIDIOC_G/S/TRY_EXT_CTRLS: OK >> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK >> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) >> Standard Controls: 12 Private Controls: 0 >> >> Format ioctls: >> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK >> test VIDIOC_G/S_PARM: OK (Not Supported) >> test VIDIOC_G_FBUF: OK (Not Supported) >> test VIDIOC_G_FMT: OK >> test VIDIOC_TRY_FMT: OK >> test VIDIOC_S_FMT: OK >> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) >> test Cropping: OK >> test Composing: OK >> test Scaling: OK (Not Supported) >> >> Codec ioctls: >> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) >> test VIDIOC_G_ENC_INDEX: OK (Not Supported) >> test VIDIOC_(TRY_)DECODER_CMD: OK >> >> Buffer ioctls: >> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK >> test CREATE_BUFS maximum buffers: OK >> test VIDIOC_REMOVE_BUFS: OK >> test VIDIOC_EXPBUF: OK >> test Requests: OK (Not Supported) >> [65391.311675] qcom-iris aa00000.video-codec: invalid plane >> [65395.340586] qcom-iris aa00000.video-codec: invalid plane >> test blocking wait: OK >> >> Total for iris_driver device /dev/video0: 48, Succeeded: 48, Failed: 0, >> Warnings: 0 >> >> V4l2-ctl Test verified for 2 streams as well. >> >> Thanks, >> Deepa >> >> Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com> >> --- >> DEEPA GUTHYAPPA MADIVALARA (5): >> media: uapi: videodev2: Add support for AV1 stateful decoder >> media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc() >> media: iris: Add support for AV1 format in iris decoder >> media: iris: Add internal buffer calculation for AV1 decoder >> media: iris: Define AV1-specific platform capabilities and properties >> >> drivers/media/platform/qcom/iris/iris_buffer.h | 2 + >> drivers/media/platform/qcom/iris/iris_ctrls.c | 8 + >> drivers/media/platform/qcom/iris/iris_hfi_common.h | 3 + >> .../platform/qcom/iris/iris_hfi_gen2_command.c | 109 ++++++++- >> .../platform/qcom/iris/iris_hfi_gen2_defines.h | 10 + >> .../platform/qcom/iris/iris_hfi_gen2_response.c | 22 ++ >> drivers/media/platform/qcom/iris/iris_instance.h | 1 + >> .../platform/qcom/iris/iris_platform_common.h | 15 ++ >> .../media/platform/qcom/iris/iris_platform_gen2.c | 156 ++++++++++++- >> .../platform/qcom/iris/iris_platform_sm8250.c | 17 ++ >> drivers/media/platform/qcom/iris/iris_vdec.c | 25 +- >> drivers/media/platform/qcom/iris/iris_vidc.c | 1 + >> drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 255 ++++++++++++++++++++- >> drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 105 +++++++++ >> drivers/media/v4l2-core/v4l2-ioctl.c | 1 + >> include/uapi/linux/videodev2.h | 1 + >> 16 files changed, 703 insertions(+), 28 deletions(-) >> --- >> base-commit: 88a6b4187eacb700a678296afb0c610eb3781e2f >> change-id: 20250821-rfc_split-c3ff834bb2c9 >> prerequisite-change-id: 20250704-iris-video-encoder-b193350b487a:v3 >> prerequisite-patch-id: 8a566690da276da34430c10dbc2fe64c1d623a9c >> prerequisite-patch-id: 1430a33603b425d0b142aab98befcda771fb885e >> prerequisite-patch-id: 32024cd49d2445ff396e31f40739b32597be59a4 >> prerequisite-patch-id: 65b569952650647174e8221dc7adde9b000a7ae3 >> prerequisite-patch-id: da128980fab8538bf668f19016c5121fb03759c2 >> prerequisite-patch-id: 079823dffbe8b89990797bf7f7640b754382d8ce >> prerequisite-patch-id: 6ce10e03d7b3b96b2391e26cda703b650bde7cd0 >> prerequisite-patch-id: b5950670ac5068a0c5b26651ebd433f7d3bbe6ca >> prerequisite-patch-id: 4de7a934f6bdfe28c84e461f70495925aa98365e >> prerequisite-patch-id: 07682a6d2530b5796122bf8763f94b5bc92949ec >> prerequisite-patch-id: 72b7eba20f1a222908d41323f28be3ba84106759 >> prerequisite-patch-id: fd9e2e1b157112c39c69486799493ee99e6033a7 >> prerequisite-patch-id: ae0ad8a04a04dd3434a092d4c2bb3f493417c6e1 >> prerequisite-patch-id: 52631eec348735d1dc5f5804b573e3cf942550a0 >> prerequisite-patch-id: 4109c59edb1b757162db46297914c8f7c14408dc >> prerequisite-patch-id: fc0b713eb4822047e8172d11fd4cd5a097ef23a5 >> prerequisite-patch-id: 20ac8e7307f1f852b2a43268b2474178fbc0b94c >> prerequisite-patch-id: e8419d716573beb64ad89968f0074d6bddfa86d3 >> prerequisite-patch-id: bdc72f5876ceb2e981d594c86a45cb21a6264af3 >> prerequisite-patch-id: fc5d26d01cab94d229a00eab819ae80196f3f5d5 >> prerequisite-patch-id: 32a9fe1371fffc9abd9a862b2814050a144d1968 >> prerequisite-patch-id: 25184583b5de886f78ee0444a4a59d5f3c271ce5 >> prerequisite-patch-id: df3376b9de27b23ae81a4c7b7a8fe4b429c32423 >> prerequisite-patch-id: e68fbf7c82567d2e9f3fdd0fdf2e2911329d5ccd >> prerequisite-patch-id: 580fa40de01a81a8685e56420f562d299bfc60fa >> prerequisite-patch-id: 89548da6690681854ee1de992a491bed73202b83 >> >> Best regards,
© 2016 - 2025 Red Hat, Inc.