[PATCH 00/14] media: Add V4L2 H.264 stateless encode and VC8000E support

Paul Kocialkowski posted 14 patches 2 days, 7 hours ago
arch/arm64/boot/dts/freescale/imx8mp.dtsi     |   11 +
drivers/media/platform/verisilicon/Kconfig    |    1 +
drivers/media/platform/verisilicon/Makefile   |    2 +
drivers/media/platform/verisilicon/hantro.h   |   17 +
.../media/platform/verisilicon/hantro_drv.c   |  180 +-
.../media/platform/verisilicon/hantro_h264.c  |    6 +-
.../media/platform/verisilicon/hantro_hw.h    |   28 +
.../media/platform/verisilicon/hantro_v4l2.c  |  123 +-
.../platform/verisilicon/hantro_vc8000e.c     |   68 +
.../verisilicon/hantro_vc8000e_h264_enc.c     |  883 +++++++
.../verisilicon/hantro_vc8000e_regs.h         | 2129 +++++++++++++++++
.../media/platform/verisilicon/imx8m_vpu_hw.c |  113 +
drivers/media/v4l2-core/Kconfig               |    4 +
drivers/media/v4l2-core/Makefile              |    2 +
drivers/media/v4l2-core/v4l2-ctrls-core.c     |   62 +
drivers/media/v4l2-core/v4l2-ctrls-defs.c     |    4 +
drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c  | 1173 +++++++++
drivers/media/v4l2-core/v4l2-h264-enc-rc.c    |  558 +++++
drivers/media/v4l2-core/v4l2-h264-enc.c       | 1322 ++++++++++
drivers/media/v4l2-core/v4l2-h264.c           |   69 +
include/media/v4l2-ctrls.h                    |    2 +
include/media/v4l2-h264-enc-rbsp.h            |   72 +
include/media/v4l2-h264-enc-rc.h              |  108 +
include/media/v4l2-h264-enc.h                 |  135 ++
include/media/v4l2-h264.h                     |  146 ++
include/uapi/linux/v4l2-controls.h            |   33 +
include/uapi/linux/videodev2.h                |    1 +
27 files changed, 7231 insertions(+), 21 deletions(-)
create mode 100644 drivers/media/platform/verisilicon/hantro_vc8000e.c
create mode 100644 drivers/media/platform/verisilicon/hantro_vc8000e_h264_enc.c
create mode 100644 drivers/media/platform/verisilicon/hantro_vc8000e_regs.h
create mode 100644 drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c
create mode 100644 drivers/media/v4l2-core/v4l2-h264-enc-rc.c
create mode 100644 drivers/media/v4l2-core/v4l2-h264-enc.c
create mode 100644 include/media/v4l2-h264-enc-rbsp.h
create mode 100644 include/media/v4l2-h264-enc-rc.h
create mode 100644 include/media/v4l2-h264-enc.h
[PATCH 00/14] media: Add V4L2 H.264 stateless encode and VC8000E support
Posted by Paul Kocialkowski 2 days, 7 hours ago
This series introduces support for the V4L2 H.264 stateless encode uAPI,
core and support in the hantro driver for the Verisilicon VC8000E.

While this is a first version that will likely need some level of rework,
it is already usable for most common use-cases and supports constant
bitrate rate-control.

A GStreamer tree can be used to test the series at:
https://github.com/paulkocialkowski/gstreamer/tree/v4l2codecs/h264enc
And an example pipeline would look like:
gst-launch-1.0 videotestsrc pattern=smpte num-buffers=25 ! video/x-raw,width=640,height=480 ! v4l2slh264enc rate-control=cbr bitrate=8000000 qp-min=8 qp-max=42 ! h264parse ! matroskamux ! filesink location=encode.mkv

Note that documentation for the new uAPI is intentionally left out of
this series since it has not yet received approval.

Marco Felsch (2):
  media: hantro: use hantro_decoded_buffer only for dst_vq
  arm64: dts: imx8mp: add VC8000E encoder node

Paul Kocialkowski (12):
  media: h264: Add a more generic reflist builder init
  media: uapi: Add H.264 stateless encode support
  media: h264: Add SPS video definitions
  media: h264: Add stateless encode core
  media: h264: Add stateless encode rbsp
  media: h264: Add stateless encode reference management
  media: h264: Add stateless encode rate control
  media: verisilicon: Report default pixel coding for non-JPEG and fix
    JPEG case
  media: verisilicon: Cancel job with runtime pm put/clk disable on
    failure
  media: verisilicon: Add common encoder parm and frameintervals ioctls
  media: verisilicon: Add support for the VC8000E H.264 encoder
  media: verilisicon: imx8m: Add support for the VC8000E on i.MX8MP

 arch/arm64/boot/dts/freescale/imx8mp.dtsi     |   11 +
 drivers/media/platform/verisilicon/Kconfig    |    1 +
 drivers/media/platform/verisilicon/Makefile   |    2 +
 drivers/media/platform/verisilicon/hantro.h   |   17 +
 .../media/platform/verisilicon/hantro_drv.c   |  180 +-
 .../media/platform/verisilicon/hantro_h264.c  |    6 +-
 .../media/platform/verisilicon/hantro_hw.h    |   28 +
 .../media/platform/verisilicon/hantro_v4l2.c  |  123 +-
 .../platform/verisilicon/hantro_vc8000e.c     |   68 +
 .../verisilicon/hantro_vc8000e_h264_enc.c     |  883 +++++++
 .../verisilicon/hantro_vc8000e_regs.h         | 2129 +++++++++++++++++
 .../media/platform/verisilicon/imx8m_vpu_hw.c |  113 +
 drivers/media/v4l2-core/Kconfig               |    4 +
 drivers/media/v4l2-core/Makefile              |    2 +
 drivers/media/v4l2-core/v4l2-ctrls-core.c     |   62 +
 drivers/media/v4l2-core/v4l2-ctrls-defs.c     |    4 +
 drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c  | 1173 +++++++++
 drivers/media/v4l2-core/v4l2-h264-enc-rc.c    |  558 +++++
 drivers/media/v4l2-core/v4l2-h264-enc.c       | 1322 ++++++++++
 drivers/media/v4l2-core/v4l2-h264.c           |   69 +
 include/media/v4l2-ctrls.h                    |    2 +
 include/media/v4l2-h264-enc-rbsp.h            |   72 +
 include/media/v4l2-h264-enc-rc.h              |  108 +
 include/media/v4l2-h264-enc.h                 |  135 ++
 include/media/v4l2-h264.h                     |  146 ++
 include/uapi/linux/v4l2-controls.h            |   33 +
 include/uapi/linux/videodev2.h                |    1 +
 27 files changed, 7231 insertions(+), 21 deletions(-)
 create mode 100644 drivers/media/platform/verisilicon/hantro_vc8000e.c
 create mode 100644 drivers/media/platform/verisilicon/hantro_vc8000e_h264_enc.c
 create mode 100644 drivers/media/platform/verisilicon/hantro_vc8000e_regs.h
 create mode 100644 drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c
 create mode 100644 drivers/media/v4l2-core/v4l2-h264-enc-rc.c
 create mode 100644 drivers/media/v4l2-core/v4l2-h264-enc.c
 create mode 100644 include/media/v4l2-h264-enc-rbsp.h
 create mode 100644 include/media/v4l2-h264-enc-rc.h
 create mode 100644 include/media/v4l2-h264-enc.h

-- 
2.53.0
Re: [PATCH 00/14] media: Add V4L2 H.264 stateless encode and VC8000E support
Posted by Nicolas Dufresne 2 days, 3 hours ago
Hi Paul,

Le vendredi 22 mai 2026 à 12:16 +0200, Paul Kocialkowski a écrit :
> This series introduces support for the V4L2 H.264 stateless encode uAPI,
> core and support in the hantro driver for the Verisilicon VC8000E.
> 
> While this is a first version that will likely need some level of rework,
> it is already usable for most common use-cases and supports constant
> bitrate rate-control.
> 
> A GStreamer tree can be used to test the series at:
> https://github.com/paulkocialkowski/gstreamer/tree/v4l2codecs/h264enc
> And an example pipeline would look like:
> gst-launch-1.0 videotestsrc pattern=smpte num-buffers=25 ! video/x-raw,width=640,height=480 ! v4l2slh264enc rate-control=cbr bitrate=8000000 qp-min=8 qp-max=42 ! h264parse ! matroskamux ! filesink location=encode.mkv
> 
> Note that documentation for the new uAPI is intentionally left out of
> this series since it has not yet received approval.

The unfortunate part is that the spec is important for the actual review and
interops analyses. That omission qualify this submission as RFC from my point of
view. Any chances you can reply to this cover filling the blank for the few
important bit (something similar but non spec style of:

https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/dev-stateless-decoder.html#initialization

- Querying capabilities
- Initialization
- Encoding
- Dynamic Resolution Changes**

The other sections are just going to be the same imho. The ** for the DRC is
because H.264 is not that complicated, its intra only, so you can reset the
encoder (use the drain flow). So that's probably something for future us
(VP9/AV1). Adding your view on these topics (or just matching what you have
implemented) will make review more constructive.

cheers,
Nicolas

> 
> Marco Felsch (2):
>   media: hantro: use hantro_decoded_buffer only for dst_vq
>   arm64: dts: imx8mp: add VC8000E encoder node
> 
> Paul Kocialkowski (12):
>   media: h264: Add a more generic reflist builder init
>   media: uapi: Add H.264 stateless encode support
>   media: h264: Add SPS video definitions
>   media: h264: Add stateless encode core
>   media: h264: Add stateless encode rbsp
>   media: h264: Add stateless encode reference management
>   media: h264: Add stateless encode rate control
>   media: verisilicon: Report default pixel coding for non-JPEG and fix JPEG case
>   media: verisilicon: Cancel job with runtime pm put/clk disable on failure
>   media: verisilicon: Add common encoder parm and frameintervals ioctls
>   media: verisilicon: Add support for the VC8000E H.264 encoder
>   media: verilisicon: imx8m: Add support for the VC8000E on i.MX8MP
> 
>  arch/arm64/boot/dts/freescale/imx8mp.dtsi     |   11 +
>  drivers/media/platform/verisilicon/Kconfig    |    1 +
>  drivers/media/platform/verisilicon/Makefile   |    2 +
>  drivers/media/platform/verisilicon/hantro.h   |   17 +
>  .../media/platform/verisilicon/hantro_drv.c   |  180 +-
>  .../media/platform/verisilicon/hantro_h264.c  |    6 +-
>  .../media/platform/verisilicon/hantro_hw.h    |   28 +
>  .../media/platform/verisilicon/hantro_v4l2.c  |  123 +-
>  .../platform/verisilicon/hantro_vc8000e.c     |   68 +
>  .../verisilicon/hantro_vc8000e_h264_enc.c     |  883 +++++++
>  .../verisilicon/hantro_vc8000e_regs.h         | 2129 +++++++++++++++++
>  .../media/platform/verisilicon/imx8m_vpu_hw.c |  113 +
>  drivers/media/v4l2-core/Kconfig               |    4 +
>  drivers/media/v4l2-core/Makefile              |    2 +
>  drivers/media/v4l2-core/v4l2-ctrls-core.c     |   62 +
>  drivers/media/v4l2-core/v4l2-ctrls-defs.c     |    4 +
>  drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c  | 1173 +++++++++
>  drivers/media/v4l2-core/v4l2-h264-enc-rc.c    |  558 +++++
>  drivers/media/v4l2-core/v4l2-h264-enc.c       | 1322 ++++++++++
>  drivers/media/v4l2-core/v4l2-h264.c           |   69 +
>  include/media/v4l2-ctrls.h                    |    2 +
>  include/media/v4l2-h264-enc-rbsp.h            |   72 +
>  include/media/v4l2-h264-enc-rc.h              |  108 +
>  include/media/v4l2-h264-enc.h                 |  135 ++
>  include/media/v4l2-h264.h                     |  146 ++
>  include/uapi/linux/v4l2-controls.h            |   33 +
>  include/uapi/linux/videodev2.h                |    1 +
>  27 files changed, 7231 insertions(+), 21 deletions(-)
>  create mode 100644 drivers/media/platform/verisilicon/hantro_vc8000e.c
>  create mode 100644 drivers/media/platform/verisilicon/hantro_vc8000e_h264_enc.c
>  create mode 100644 drivers/media/platform/verisilicon/hantro_vc8000e_regs.h
>  create mode 100644 drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c
>  create mode 100644 drivers/media/v4l2-core/v4l2-h264-enc-rc.c
>  create mode 100644 drivers/media/v4l2-core/v4l2-h264-enc.c
>  create mode 100644 include/media/v4l2-h264-enc-rbsp.h
>  create mode 100644 include/media/v4l2-h264-enc-rc.h
>  create mode 100644 include/media/v4l2-h264-enc.h