This series add a HEVC backend to the Rockchip Video Decoder driver.

With the dependent H.264 High 10 and 4:2:2 profile support series
finally merged there is finally time to send a v2 with minor changes and
a suggested code style fix of this series. v1 of this series has been
fully functional up until recent unstaging of the rkvdec driver.

A version of this HEVC backend has been in use by the LibreELEC distro
for the past 5+ years [1]. It was initially created based on a copy of
the H264 backend, unstable HEVC uAPI controls and a cabac table + scaling
matrix functions shamelessly copied 1:1 from the Rockchip mpp library.

It has since then been extended to use the stable HEVC uAPI controls and
improved opon e.g. to include support for rk3288 and fix decoding issues
by Alex Bee and Nicolas Dufresne.

The version submitted in this series is based on the code currently used
by the LibreELEC distro, excluding hard/soft reset, and with cabac table
and scaling matrix functions picked from Sebastian Fricke prior series
to add a HEVC backend [2].

Big thanks to Alex Bee, Nicolas Dufresne and Sebastian Fricke for making
this series possible!

Patch 1 add the new HEVC backend.
Patch 2-3 add variants support to the driver.
Patch 4 add support for a rk3288 variant.
Patch 5 add a rk3328 variant to work around hw quirks.
Patch 6-7 add device tree node for rk3288.

This was tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328):

  v4l2-compliance 1.30.1, 64 bits, 64-bit time_t
  ...
  Total for rkvdec device /dev/video1: 49, Succeeded: 49, Failed: 0, Warnings: 0

  Running test suite JCT-VC-HEVC_V1 with decoder FFmpeg-H.265-v4l2request
  ...
  Ran 137/147 tests successfully

  Running test suite JCT-VC-MV-HEVC with decoder FFmpeg-H.265-v4l2request
  ...
  Ran 9/9 tests successfully

And on a TinkerBoard (RK3288):

  v4l2-compliance 1.30.1, 32 bits, 32-bit time_t
  ...
  Total for rkvdec device /dev/video3: 49, Succeeded: 49, Failed: 0, Warnings: 0

  Running test suite JCT-VC-HEVC_V1 with decoder FFmpeg-H.265-v4l2request
  ...
  Ran 137/147 tests successfully

  Running test suite JCT-VC-MV-HEVC with decoder FFmpeg-H.265-v4l2request
  ...
  Ran 9/9 tests successfully

The WPP_x_ericsson tests from test suite JCT-VC-HEVC_V1 has been showing
a mix of both Success and/or Fail result for FFmpeg-H.265-v4l2request.

Full summary of fluster run can be found at [3].

Please note that there is a known issue with concurrent decoding,
decoding errors in one decode session may affect a separate session.
The only known mitigation to this is to pause decoding for some time
and/or do a full HW reset, something to handle in future series.

Changes in v3:
- Change to use file_to_rkvdec_ctx()
- Rename assemble_hw_rps to assemble_sw_rps
- Use a reference to rkvdec_variant instead of copying capabilities and
  quirks to rkvdec_dev
- Add num_regs field to rkvdec_variant, currently not used for anything
- Add and use rkvdec_quirks_disable_qos() helper to apply qos quirk
- Collect t-b and r-b tags
Link to v2: https://lore.kernel.org/linux-media/20250810212454.3237486-1-jonas@kwiboo.se

Changes in v2:
- Rabase after h264 high10/422 merge and unstaging of rkvdec driver
- Use new_value in transpose_and_flatten_matrices()
- Add NULL check for ctrl->new_elems in rkvdec_hevc_run_preamble()
- Set RKVDEC_WR_DDR_ALIGN_EN for RK3328
- Adjust code style in rkvdec_enum_coded_fmt_desc()
- Collect a-b tag
- Drop merged vdec node reg size patches
Link to v1: https://lore.kernel.org/linux-media/20231105233630.3927502-1-jonas@kwiboo.se

[1] https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Rockchip/patches/linux/default/linux-2000-v4l2-wip-rkvdec-hevc.patch
[2] https://lore.kernel.org/linux-media/20230101-patch-series-v2-6-2-rc1-v2-0-fa1897efac14@collabora.com/
[3] https://gist.github.com/Kwiboo/0ea22df1c9c3f3a48479d3f7ec28169d

Alex Bee (4):
  media: rkvdec: Add variants support
  media: rkvdec: Add RK3288 variant
  media: rkvdec: Disable QoS for HEVC and VP9 on RK3328
  ARM: dts: rockchip: Add vdec node for RK3288

Jonas Karlman (3):
  media: rkvdec: Add HEVC backend
  media: rkvdec: Implement capability filtering
  media: dt-bindings: rockchip,vdec: Add RK3288 compatible

 .../bindings/media/rockchip,vdec.yaml         |    1 +
 arch/arm/boot/dts/rockchip/rk3288.dtsi        |   17 +-
 .../media/platform/rockchip/rkvdec/Makefile   |    2 +-
 .../rockchip/rkvdec/rkvdec-hevc-data.c        | 1848 +++++++++++++++++
 .../platform/rockchip/rkvdec/rkvdec-hevc.c    |  820 ++++++++
 .../platform/rockchip/rkvdec/rkvdec-regs.h    |    4 +
 .../platform/rockchip/rkvdec/rkvdec-vp9.c     |    4 +
 .../media/platform/rockchip/rkvdec/rkvdec.c   |  198 +-
 .../media/platform/rockchip/rkvdec/rkvdec.h   |   17 +
 9 files changed, 2890 insertions(+), 21 deletions(-)
 create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-data.c
 create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-hevc.c

-- 
2.51.0

Hi Jonas,

On 9/5/25 12:19, Jonas Karlman wrote:
> This series add a HEVC backend to the Rockchip Video Decoder driver.
>
> With the dependent H.264 High 10 and 4:2:2 profile support series
> finally merged there is finally time to send a v2 with minor changes and
> a suggested code style fix of this series. v1 of this series has been
> fully functional up until recent unstaging of the rkvdec driver.
>
> A version of this HEVC backend has been in use by the LibreELEC distro
> for the past 5+ years [1]. It was initially created based on a copy of
> the H264 backend, unstable HEVC uAPI controls and a cabac table + scaling
> matrix functions shamelessly copied 1:1 from the Rockchip mpp library.
>
> It has since then been extended to use the stable HEVC uAPI controls and
> improved opon e.g. to include support for rk3288 and fix decoding issues
> by Alex Bee and Nicolas Dufresne.
>
> The version submitted in this series is based on the code currently used
> by the LibreELEC distro, excluding hard/soft reset, and with cabac table
> and scaling matrix functions picked from Sebastian Fricke prior series
> to add a HEVC backend [2].
>
> Big thanks to Alex Bee, Nicolas Dufresne and Sebastian Fricke for making
> this series possible!
>
> Patch 1 add the new HEVC backend.
> Patch 2-3 add variants support to the driver.
> Patch 4 add support for a rk3288 variant.
> Patch 5 add a rk3328 variant to work around hw quirks.
> Patch 6-7 add device tree node for rk3288.
>
> This was tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328):
>
>    v4l2-compliance 1.30.1, 64 bits, 64-bit time_t
>    ...
>    Total for rkvdec device /dev/video1: 49, Succeeded: 49, Failed: 0, Warnings: 0
>
>    Running test suite JCT-VC-HEVC_V1 with decoder FFmpeg-H.265-v4l2request
>    ...
>    Ran 137/147 tests successfully
>
>    Running test suite JCT-VC-MV-HEVC with decoder FFmpeg-H.265-v4l2request
>    ...
>    Ran 9/9 tests successfully
>
> And on a TinkerBoard (RK3288):
>
>    v4l2-compliance 1.30.1, 32 bits, 32-bit time_t
>    ...
>    Total for rkvdec device /dev/video3: 49, Succeeded: 49, Failed: 0, Warnings: 0
>
>    Running test suite JCT-VC-HEVC_V1 with decoder FFmpeg-H.265-v4l2request
>    ...
>    Ran 137/147 tests successfully
>
>    Running test suite JCT-VC-MV-HEVC with decoder FFmpeg-H.265-v4l2request
>    ...
>    Ran 9/9 tests successfully
>
> The WPP_x_ericsson tests from test suite JCT-VC-HEVC_V1 has been showing
> a mix of both Success and/or Fail result for FFmpeg-H.265-v4l2request.
>
> Full summary of fluster run can be found at [3].
>
> Please note that there is a known issue with concurrent decoding,
> decoding errors in one decode session may affect a separate session.
> The only known mitigation to this is to pause decoding for some time
> and/or do a full HW reset, something to handle in future series.
I also tested on Rock Pi 4 SE (but with Gstreamer instead of FFmpeg) and 
can
confirm the same results, so, for the whole series:

Tested-by: Detlev Casanova <detlev.casanova@collabora.com> # RK3399

Best regards,

Detlev

> Changes in v3:
> - Change to use file_to_rkvdec_ctx()
> - Rename assemble_hw_rps to assemble_sw_rps
> - Use a reference to rkvdec_variant instead of copying capabilities and
>    quirks to rkvdec_dev
> - Add num_regs field to rkvdec_variant, currently not used for anything
> - Add and use rkvdec_quirks_disable_qos() helper to apply qos quirk
> - Collect t-b and r-b tags
> Link to v2: https://lore.kernel.org/linux-media/20250810212454.3237486-1-jonas@kwiboo.se
>
> Changes in v2:
> - Rabase after h264 high10/422 merge and unstaging of rkvdec driver
> - Use new_value in transpose_and_flatten_matrices()
> - Add NULL check for ctrl->new_elems in rkvdec_hevc_run_preamble()
> - Set RKVDEC_WR_DDR_ALIGN_EN for RK3328
> - Adjust code style in rkvdec_enum_coded_fmt_desc()
> - Collect a-b tag
> - Drop merged vdec node reg size patches
> Link to v1: https://lore.kernel.org/linux-media/20231105233630.3927502-1-jonas@kwiboo.se
>
> [1] https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Rockchip/patches/linux/default/linux-2000-v4l2-wip-rkvdec-hevc.patch
> [2] https://lore.kernel.org/linux-media/20230101-patch-series-v2-6-2-rc1-v2-0-fa1897efac14@collabora.com/
> [3] https://gist.github.com/Kwiboo/0ea22df1c9c3f3a48479d3f7ec28169d
>
> Alex Bee (4):
>    media: rkvdec: Add variants support
>    media: rkvdec: Add RK3288 variant
>    media: rkvdec: Disable QoS for HEVC and VP9 on RK3328
>    ARM: dts: rockchip: Add vdec node for RK3288
>
> Jonas Karlman (3):
>    media: rkvdec: Add HEVC backend
>    media: rkvdec: Implement capability filtering
>    media: dt-bindings: rockchip,vdec: Add RK3288 compatible
>
>   .../bindings/media/rockchip,vdec.yaml         |    1 +
>   arch/arm/boot/dts/rockchip/rk3288.dtsi        |   17 +-
>   .../media/platform/rockchip/rkvdec/Makefile   |    2 +-
>   .../rockchip/rkvdec/rkvdec-hevc-data.c        | 1848 +++++++++++++++++
>   .../platform/rockchip/rkvdec/rkvdec-hevc.c    |  820 ++++++++
>   .../platform/rockchip/rkvdec/rkvdec-regs.h    |    4 +
>   .../platform/rockchip/rkvdec/rkvdec-vp9.c     |    4 +
>   .../media/platform/rockchip/rkvdec/rkvdec.c   |  198 +-
>   .../media/platform/rockchip/rkvdec/rkvdec.h   |   17 +
>   9 files changed, 2890 insertions(+), 21 deletions(-)
>   create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-data.c
>   create mode 100644 drivers/media/platform/rockchip/rkvdec/rkvdec-hevc.c
>

On Fri Sep 5, 2025 at 6:19 PM CEST, Jonas Karlman wrote:
> This series add a HEVC backend to the Rockchip Video Decoder driver.

I did some testing and the TL;DR version is:

Tested-by: Diederik de Haas <didi.debian@cknow.org>  # Rock64, RockPro64, Quartz64-B, NanoPi R5S

Now for the long version ;-P

I built a 5.17-rc5 kernel with this patch set [1], which was based upon
linuxtv's next branch, so I just took all their commits on 2025-09-07.
Then rebased Detlev's rkvdec patch set on top of that.
As I have quite a few rk356x based devices and there wasn't a DT patch
to enable rkvdec2, I 'assembled' my own one. (And some more patches)

I have a patched ffmpeg [2] and with its -dev libraries built a patched
mpv [3]. Installed that onto (mostly) Debian Forky systems with mesa
25.2.2 and sway 1.11. I have a number of test files which are provided
via NFS so storage device wouldn't matter and it's also what I use with
LibreELEC (both 12.2 as 12.90.1 provided by 'chewitt').
And then I went on to perform the same tests on these devices:

1) Pine64 Rock64 (rk3328)
2) Pine64 RockPro64 (rk3399)
3) Pine64 Quartz64 Model B (rk3566) *
4) FriendlyELEC NanoPi R5S (rk3568) *

*) I blacklisted the hantro driver

My testing showed that the 'classical' Big Buck Bunny video worked the
best for testing ... meaning it caused the most 'problems'.
I used the 'Standard 2D' 'Full HD (1920x1080)' '60 fps' version which
you can download from [4]. That'll give you an 8-bit encoded x264
version of that video. I have converted that video to x265 with a script
I wrote (quite) a while ago [5]; instructions near the end of that
script (no hdr no resize 'branch' about $TENBIT_PARAMS).
That resulted in 2 test files:
a) bbb_sunflower_1080p_60fps_normal.x265.cf24.slow.8bit.mp4
b) bbb_sunflower_1080p_60fps_normal.x265.cf24.medium.10bit.mp4

Testing went as follows:
I logged into the device, started sway and opened a 'foot' terminal and
navigated to my NFS share with the test files. Then I used

  mpv --hwdec=v4l2request <video-file> [--fullscreen=yes]

Without the 'fullscreen' param it shows the video in the right part of
the screen (1080p monitor), while the left part shows mpv's output.
This made it easy to see 'dropped frames' and any audio delay.
I played each file twice, one using half the screen and one fullscreen.

This had less of an effect then I expected; only in a few cases it
'pushed it over the edge' and the only real effect was with Rock64 ...
which being the least powerful is (kinda) expected.

The results of the 10-bit x265 files was uniform: only a blue screen was
visible and OSD (with 'I' or 'O' in mpv) didn't work. Interestingly it
also had no frame drops ... except when doing 'I' (with no visible
effect), then it dropped a couple of frames (one time quite a few).
Mpv showed this error message each time:
[vo/gpu] Initializing texture for hardware decoding failed

For the 8-bit x265 file, the results were more varied:
- On Rock64 HW accel worked, but the videos had a red 'glare' over them.
  None of the other devices had that, so maybe related to 'lima'?
  On LE it did NOT have the red glare. It did seem to have quite a bit
  of trouble starting it
- On Rock64 it dropped 970 frames in 60 secs and 2480 in fullscreen.
  This resulted in quite a shockery display and noticeable artifacts.
  That was not (or at least a whole lot less) the case with LE.
- On the other devices there was not much difference in dropped frames
  when it came to windowed vs full-screen, but the frame drops were
  quite high ~2000 when HW accelerated and up to ~3000 when not
  I got the impression that ~3Mbps bitrate was a tipping point.
- The original BBB file (thus x264) had a similar high frame drop rate,
  so the problem seems unrelated to this patch set
- Audio delay was sometimes huge (>60 secs after 60 secs :-O), and HW
  acceleration fixed that (due to free 'CPU' capacity?)

The 'scores' for my other test files were actually quite good \o/
All my video were HW accelerated; 8-bit with nv12 worked good while
10-bit with nv15 was very blue. But all were corrected detected.

So IMO this is a massive improvement! Thanks a LOT :-D

Cheers,
  Diederik

[1] https://salsa.debian.org/diederik/linux/-/tree/cknow/media-next
(Salsa CI fails as it can't deal with the ~ in 6.17~rc5)
[2] https://salsa.debian.org/diederik/ffmpeg/-/tree/v4l2request-2025-v3-rkvdec-n7.1
[3] https://salsa.debian.org/diederik/mpv/-/tree/v4l2request-support
[4] http://bbb3d.renderfarming.net/download.html
[5] https://paste.sr.ht/~diederik/52b81ebc4c14b5146eb9b687bb1e8c1d62787991

On Fri, 05 Sep 2025 16:19:18 +0000, Jonas Karlman wrote:
> This series add a HEVC backend to the Rockchip Video Decoder driver.
> 
> With the dependent H.264 High 10 and 4:2:2 profile support series
> finally merged there is finally time to send a v2 with minor changes and
> a suggested code style fix of this series. v1 of this series has been
> fully functional up until recent unstaging of the rkvdec driver.
> 
> [...]

Applied, thanks!

[7/7] ARM: dts: rockchip: Add vdec node for RK3288
      commit: e74470cf3101da79666f20186c9406192223e9a8

Best regards,
-- 
Heiko Stuebner <heiko@sntech.de>