[PATCH v7 00/17] media: rkvdec: Add support for VDPU381 and VDPU383

Detlev Casanova posted 17 patches 1 month, 2 weeks ago
There is a newer version of this series
.../media/v4l/ext-ctrls-codec-stateless.rst   | 120 +++
.../media/v4l/videodev2.h.rst.exceptions      |   2 +
.../media/v4l/vidioc-queryctrl.rst            |  12 +
.../media/platform/rockchip/rkvdec/Makefile   |  14 +-
.../{rkvdec-hevc-data.c => rkvdec-cabac.c}    | 506 +++++++++-
.../rockchip/rkvdec/rkvdec-h264-common.c      | 258 +++++
.../rockchip/rkvdec/rkvdec-h264-common.h      |  89 ++
.../platform/rockchip/rkvdec/rkvdec-h264.c    | 892 ++----------------
.../rockchip/rkvdec/rkvdec-hevc-common.c      | 512 ++++++++++
.../rockchip/rkvdec/rkvdec-hevc-common.h      | 109 +++
.../platform/rockchip/rkvdec/rkvdec-hevc.c    | 281 +-----
.../platform/rockchip/rkvdec/rkvdec-rcb.c     | 179 ++++
.../platform/rockchip/rkvdec/rkvdec-rcb.h     |  29 +
.../platform/rockchip/rkvdec/rkvdec-regs.h    | 586 ++++++++----
.../rockchip/rkvdec/rkvdec-vdpu381-h264.c     | 465 +++++++++
.../rockchip/rkvdec/rkvdec-vdpu381-hevc.c     | 621 ++++++++++++
.../rockchip/rkvdec/rkvdec-vdpu381-regs.h     | 424 +++++++++
.../rockchip/rkvdec/rkvdec-vdpu383-h264.c     | 578 ++++++++++++
.../rockchip/rkvdec/rkvdec-vdpu383-hevc.c     | 720 ++++++++++++++
.../rockchip/rkvdec/rkvdec-vdpu383-regs.h     | 281 ++++++
.../platform/rockchip/rkvdec/rkvdec-vp9.c     | 232 ++---
.../media/platform/rockchip/rkvdec/rkvdec.c   | 572 +++++++++--
.../media/platform/rockchip/rkvdec/rkvdec.h   |  49 +-
drivers/media/test-drivers/visl/visl-dec.c    |   7 +
drivers/media/test-drivers/visl/visl-dec.h    |   3 +
.../media/test-drivers/visl/visl-trace-hevc.h |  59 ++
drivers/media/v4l2-core/v4l2-ctrls-core.c     |  28 +
drivers/media/v4l2-core/v4l2-ctrls-defs.c     |  10 +
include/uapi/linux/v4l2-controls.h            |  61 ++
include/uapi/linux/videodev2.h                |   2 +
30 files changed, 6238 insertions(+), 1463 deletions(-)
rename drivers/media/platform/rockchip/rkvdec/{rkvdec-hevc-data.c => rkvdec-cabac.c} (86%)
create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-h264-common.c
create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-h264-common.h
create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.h
create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-rcb.c
create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-rcb.h
create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-h264.c
create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c
create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-regs.h
create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu383-h264.c
create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu383-hevc.c
create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu383-regs.h
[PATCH v7 00/17] media: rkvdec: Add support for VDPU381 and VDPU383
Posted by Detlev Casanova 1 month, 2 weeks ago
These variants are found respectively in the RK3588 and RK3576 SoCs.
This patch only adds support for H264 and H265 in both variants.

As there is a considerable part of the code that can be shared with the
already supported rkvdec decoder driver, the support for these variants
is added here rather than writing a new driver.

This patch set uses the newly introduced hevc_ext_sps_[ls]t_rps v4l2
controls for HEVC.
Therefore, a patched version of userpace tools is needed for HEVC
support (added for GStreamer[1] and in an early stage for FFmpeg[2]).

The DTS changes can be found in another patch set: [3]

[1]: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9355
[2]: https://gitlab.collabora.com/detlev/ffmpeg
[3]: https://lore.kernel.org/all/20251020212009.8852-1-detlev.casanova@collabora.com/

Changes since v6:
 - Remove coded_fmts capabilities
 - Remove inexistant V4L2_HEVC kconfig dependency
 - Add missing rkvdec registers
 - Fix memcpy sizes
 - Remove non-necessary name and style changes
 - Use common get_image_fmt
 - Do not validate sps in start()
 - Remove unused variable
 - Change named_regs to has_single_reg_region
 - Compared to v6, I didn't use a union for the ref addr and flags
   registers, as I find the flat struct more readable.

Changes since v5:
- Extract multicore management in separate commit
- Move variant specific functions to an ops struct
- Fix a typo and "h264" -> "hevc" copy/paste mistakes
- Move RKVDEC_*_PIXELS to rkvdec.h
- Drop VDPU383_*_PIXELS, use the the RKVDEC ones instead
- Change RKVDEC_1080P_PIXELS value to 1920 * 1088
- Long and short term ref sets extend controls:
  - Add them in visl traces
  - Reword documentation to specify SPS extension and use
  - Mention Vulkan Video API similarity
  - Specify behavior for empty array values
  - Reorder struct fields to ensure fixed compiled size
  - Remove the mention of V4L2_CTRL_FLAG_DYNAMIC_ARRAY flag

Changes since v4:
 - Add basic ext_sps_[ls]t_rps validation (flags only)
 - Add missing copyright in new common files
 - Rename num_coded_fmts, rcb_num, clk_count
 - Avoid allocating RCB config when not needed
 - Merge rkvdec_config into rkvdec_variant
 - Keep rkvdec_hevc_get_image_fmt per variant

Changes since v3:
 - Rebased on latest next (ea299a216426) + HEVC support for rk3399

Changes since v2:
 - Rebased on Kwiboo's HEVC support for rk3399
 - Include hevc_ext_sps_[ls]t_rps v4l2 controls commits
 - Remove empty lines at end of files
 - Rename RKVDEC2_ALLOC_* to RKVDEC_ALLOC_*
 - Set min_with and min_height to 64
 - Remove vdpu38x_fill_pixfmt_mp() function
 - Handle userspace not supporting hevc_ext_sps_[ls]t_rps
 - Make all config structs static const

Changes since v1:
 - Add parsing of the short and long term ref frame sets from the new v4l2
   controls
 - Add RPS cache to avoid parsing the same data again
 - Fix HEVC pixel formats selection
 - Fix multiple indentation errors

Detlev Casanova (17):
  media: uapi: HEVC: Add v4l2_ctrl_hevc_ext_sps_[ls]t_rps controls
  media: v4l2-ctrls: Add hevc_ext_sps_[ls]t_rps controls
  media: visl: Add HEVC short and long term RPS sets
  media: rkvdec: Switch to using structs instead of writel
  media: rkvdec: Move cabac tables to their own source file
  media: rkvdec: Use structs to represent the HW RPS
  media: rkvdec: Move h264 functions to common file
  media: rkvdec: Move hevc functions to common file
  media: rkvdec: Add variant specific coded formats list
  media: rkvdec: Add RCB and SRAM support
  media: rkvdec: Support per-variant interrupt handler
  media: rkvdec: Enable all clocks without naming them
  media: rkvdec: Disable multicore support
  media: rkvdec: Add H264 support for the VDPU381 variant
  media: rkvdec: Add H264 support for the VDPU383 variant
  media: rkvdec: Add HEVC support for the VDPU381 variant
  media: rkvdec: Add HEVC support for the VDPU383 variant

 .../media/v4l/ext-ctrls-codec-stateless.rst   | 120 +++
 .../media/v4l/videodev2.h.rst.exceptions      |   2 +
 .../media/v4l/vidioc-queryctrl.rst            |  12 +
 .../media/platform/rockchip/rkvdec/Makefile   |  14 +-
 .../{rkvdec-hevc-data.c => rkvdec-cabac.c}    | 506 +++++++++-
 .../rockchip/rkvdec/rkvdec-h264-common.c      | 258 +++++
 .../rockchip/rkvdec/rkvdec-h264-common.h      |  89 ++
 .../platform/rockchip/rkvdec/rkvdec-h264.c    | 892 ++----------------
 .../rockchip/rkvdec/rkvdec-hevc-common.c      | 512 ++++++++++
 .../rockchip/rkvdec/rkvdec-hevc-common.h      | 109 +++
 .../platform/rockchip/rkvdec/rkvdec-hevc.c    | 281 +-----
 .../platform/rockchip/rkvdec/rkvdec-rcb.c     | 179 ++++
 .../platform/rockchip/rkvdec/rkvdec-rcb.h     |  29 +
 .../platform/rockchip/rkvdec/rkvdec-regs.h    | 586 ++++++++----
 .../rockchip/rkvdec/rkvdec-vdpu381-h264.c     | 465 +++++++++
 .../rockchip/rkvdec/rkvdec-vdpu381-hevc.c     | 621 ++++++++++++
 .../rockchip/rkvdec/rkvdec-vdpu381-regs.h     | 424 +++++++++
 .../rockchip/rkvdec/rkvdec-vdpu383-h264.c     | 578 ++++++++++++
 .../rockchip/rkvdec/rkvdec-vdpu383-hevc.c     | 720 ++++++++++++++
 .../rockchip/rkvdec/rkvdec-vdpu383-regs.h     | 281 ++++++
 .../platform/rockchip/rkvdec/rkvdec-vp9.c     | 232 ++---
 .../media/platform/rockchip/rkvdec/rkvdec.c   | 572 +++++++++--
 .../media/platform/rockchip/rkvdec/rkvdec.h   |  49 +-
 drivers/media/test-drivers/visl/visl-dec.c    |   7 +
 drivers/media/test-drivers/visl/visl-dec.h    |   3 +
 .../media/test-drivers/visl/visl-trace-hevc.h |  59 ++
 drivers/media/v4l2-core/v4l2-ctrls-core.c     |  28 +
 drivers/media/v4l2-core/v4l2-ctrls-defs.c     |  10 +
 include/uapi/linux/v4l2-controls.h            |  61 ++
 include/uapi/linux/videodev2.h                |   2 +
 30 files changed, 6238 insertions(+), 1463 deletions(-)
 rename drivers/media/platform/rockchip/rkvdec/{rkvdec-hevc-data.c => rkvdec-cabac.c} (86%)
 create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-h264-common.c
 create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-h264-common.h
 create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
 create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.h
 create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-rcb.c
 create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-rcb.h
 create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-h264.c
 create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c
 create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-regs.h
 create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu383-h264.c
 create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu383-hevc.c
 create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu383-regs.h

-- 
2.52.0
Re: [PATCH v7 00/17] media: rkvdec: Add support for VDPU381 and VDPU383
Posted by Piotr Oniszczuk 1 month, 2 weeks ago

> Wiadomość napisana przez Detlev Casanova <detlev.casanova@collabora.com> w dniu 19 gru 2025, o godz. 00:28:
> 
> These variants are found respectively in the RK3588 and RK3576 SoCs.
> This patch only adds support for H264 and H265 in both variants.
> 
> As there is a considerable part of the code that can be shared with the
> already supported rkvdec decoder driver, the support for these variants
> is added here rather than writing a new driver.
> 
> This patch set uses the newly introduced hevc_ext_sps_[ls]t_rps v4l2
> controls for HEVC.
> Therefore, a patched version of userpace tools is needed for HEVC
> support (added for GStreamer[1] and in an early stage for FFmpeg[2]).
> 
> The DTS changes can be found in another patch set: [3]
> 
> [1]: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9355
> [2]: https://gitlab.collabora.com/detlev/ffmpeg
> [3]: https://lore.kernel.org/all/20251020212009.8852-1-detlev.casanova@collabora.com/
> 

Detlev,

FYI:

In my testbed with 

i. v7 of [1] on mainline 6.18.2
ii ffmpeg8 v4l2_request [2]
iii. mythv as player [3]
iv. minimyth2 as distro [4]
v. NO applied yours ffmpeg sps/rps hinting [5] (as i'm on ffmpeg8)

I run set of my mpeg2/h264/hevc/vp8/vp9 samples on: rk3328. rk3399, rk3566, rk3576, rk3588 and h616 (just for brief test for potential cross-regressions)

Results:

a. rk3328: no any regressions; all samples are decoded ok

b. rk3399: no any regressions; all samples are decoded ok

c. rk3566: no any regressions (e.g. in vp9. v6 and older were regressing; v7 is ok)  
c.1 h264: all ok
c.2 hevc: samples without long-term sps/rps: all ok.
c.3 hevc: samples with long-tern sps/rps: v7 is a bit regressing here: it gives me sometimes kernel oops requiring hw. reset to bring decoder back to live. 
Exemplary oops: https://termbin.com/upju
Older vers of your code were handling such cases without needs of hw. reset

d. rk3576:
d.1: h264: works ok - but randomly i'm getting playback with distortions like this: https://postimg.cc/Yh8Djc3L. it is avd. 1 per few playbacks.
d.2: hevc: the same like c.1 and c.2

e. rk3588
e.1: h264: all ok
e.2: hevc: the same like c.1 and c.2
   
g. h616 with cedrus: all ok.

btw: 
Adding to [1] vp9 android code [6] offers me nice working vp9 hwdec accell on rk3566/3588.  

Great work!

[1] https://gitlab.collabora.com/detlev/linux/-/commits/add-vdpu381-and-383-to-rkvdec-v7
[2] https://code.ffmpeg.org/Kwiboo/FFmpeg/src/branch/v4l2-request-n8.0.1
[3] https://github.com/MythTV/mythtv
[4] https://github.com/warpme/minimyth2
[5] https://gitlab.collabora.com/detlev/ffmpeg
[6] https://github.com/dvab-sarma/android_kernel_rk_opi/commit/fd0b1b17cbdfcf0c15bc9f411cc7127105157e57
   
 
Re: [PATCH v7 00/17] media: rkvdec: Add support for VDPU381 and VDPU383
Posted by Detlev Casanova 1 month, 2 weeks ago
Hi Piotr,

On 12/19/25 08:50, Piotr Oniszczuk wrote:
>
>> Wiadomość napisana przez Detlev Casanova <detlev.casanova@collabora.com> w dniu 19 gru 2025, o godz. 00:28:
>>
>> These variants are found respectively in the RK3588 and RK3576 SoCs.
>> This patch only adds support for H264 and H265 in both variants.
>>
>> As there is a considerable part of the code that can be shared with the
>> already supported rkvdec decoder driver, the support for these variants
>> is added here rather than writing a new driver.
>>
>> This patch set uses the newly introduced hevc_ext_sps_[ls]t_rps v4l2
>> controls for HEVC.
>> Therefore, a patched version of userpace tools is needed for HEVC
>> support (added for GStreamer[1] and in an early stage for FFmpeg[2]).
>>
>> The DTS changes can be found in another patch set: [3]
>>
>> [1]: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9355
>> [2]: https://gitlab.collabora.com/detlev/ffmpeg
>> [3]: https://lore.kernel.org/all/20251020212009.8852-1-detlev.casanova@collabora.com/
>>
> Detlev,
>
> FYI:
>
> In my testbed with
>
> i. v7 of [1] on mainline 6.18.2
> ii ffmpeg8 v4l2_request [2]
> iii. mythv as player [3]
> iv. minimyth2 as distro [4]
> v. NO applied yours ffmpeg sps/rps hinting [5] (as i'm on ffmpeg8)
And [5] won't be compatible anymore actually, I'm waiting for the final 
version of the uapi to tackle that.
> I run set of my mpeg2/h264/hevc/vp8/vp9 samples on: rk3328. rk3399, rk3566, rk3576, rk3588 and h616 (just for brief test for potential cross-regressions)
>
> Results:
>
> a. rk3328: no any regressions; all samples are decoded ok
>
> b. rk3399: no any regressions; all samples are decoded ok
>
> c. rk3566: no any regressions (e.g. in vp9. v6 and older were regressing; v7 is ok)
VP9 improvement is expected as I added a missing reg_num multiplier in 
VP9 as spotted by Jonas.
> c.1 h264: all ok
> c.2 hevc: samples without long-term sps/rps: all ok.
> c.3 hevc: samples with long-tern sps/rps: v7 is a bit regressing here: it gives me sometimes kernel oops requiring hw. reset to bring decoder back to live.
> Exemplary oops: https://termbin.com/upju
> Older vers of your code were handling such cases without needs of hw. reset

The 3566 support doesn't need any driver changes, only device tree nodes 
using the "rockchip,rk3588-vdec" compatible ?

I'll check that oops, I haven't seen it yet.

>
> d. rk3576:
> d.1: h264: works ok - but randomly i'm getting playback with distortions like this: https://postimg.cc/Yh8Djc3L. it is avd. 1 per few playbacks.
> d.2: hevc: the same like c.1 and c.2
>
> e. rk3588
> e.1: h264: all ok
> e.2: hevc: the same like c.1 and c.2
>     
> g. h616 with cedrus: all ok.
>
> btw:
> Adding to [1] vp9 android code [6] offers me nice working vp9 hwdec accell on rk3566/3588.
>
> Great work!
Thank you, and thanks for your valuable testing !
>
> [1] https://gitlab.collabora.com/detlev/linux/-/commits/add-vdpu381-and-383-to-rkvdec-v7
> [2] https://code.ffmpeg.org/Kwiboo/FFmpeg/src/branch/v4l2-request-n8.0.1
> [3] https://github.com/MythTV/mythtv
> [4] https://github.com/warpme/minimyth2
> [5] https://gitlab.collabora.com/detlev/ffmpeg
> [6] https://github.com/dvab-sarma/android_kernel_rk_opi/commit/fd0b1b17cbdfcf0c15bc9f411cc7127105157e57
>     
>   
>
>
> _______________________________________________
> Linux-rockchip mailing list
> Linux-rockchip@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-rockchip