[RFC v3 0/4] Add video encoder ROI ctrls

Ming Qian posted 4 patches 1 month ago
There is a newer version of this series
.../media/v4l/ext-ctrls-codec.rst             |  95 ++++++++++
.../media/v4l/vidioc-g-ext-ctrls.rst          |  26 ++-
.../media/v4l/vidioc-queryctrl.rst            |  14 ++
.../media/videodev2.h.rst.exceptions          |   4 +
drivers/media/i2c/imx214.c                    |   4 +-
.../media/platform/qcom/venus/venc_ctrls.c    |   9 +-
.../media/test-drivers/vivid/vivid-ctrls.c    |  34 ++++
drivers/media/v4l2-core/v4l2-ctrls-api.c      |  54 ++++--
drivers/media/v4l2-core/v4l2-ctrls-core.c     | 169 +++++++++++++++---
drivers/media/v4l2-core/v4l2-ctrls-defs.c     |  46 +++++
drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +-
include/media/v4l2-ctrls.h                    |  62 ++++++-
include/uapi/linux/v4l2-controls.h            |  16 ++
include/uapi/linux/videodev2.h                |   5 +
14 files changed, 493 insertions(+), 49 deletions(-)
[RFC v3 0/4] Add video encoder ROI ctrls
Posted by Ming Qian 1 month ago
Hi,

This patch set implements region of interest (ROI) ctrls for video
encoder.

One video encoder IP may support the following two ROI configurations or
one of them:
    1. configure ROI as a rectangular region, and set a delta QP parameter.
    2. configure ROI as a rectangular region, and set a priority parameter.
    3. configure ROI as a QP map as an array. Each value represents the delta QP
of a block in raster scan order. The block size is determined by
the specific IP.
    4. configure ROI as a QP map as an array. Each value represents the absolute QP
of a block in raster scan order. The block size is determined by
the specific IP.

To achieve this, I made the following change:
    1. I reuse the type V4L2_CTRL_TYPE_RECT that is defined in the UVC ROI patchset
    2. Define a ctrl V4L2_CID_MPEG_VIDEO_ROI_MODE to choose ROI configuration
    3. Define some ctrl to configure ROI
    4. Define a ctrl V4L2_CID_MPEG_VIDEO_ROI_BLOCK_SIZE to query block size

I referred the patchset "Implement UVC v1.5 ROI" (https://lwn.net/Articles/953532/)
and pick some patches from it.

changelog:

v3
- Drop the type V4L2_CTRL_TYPE_REGION
- Split the compound control into 2 ctrls
- Define 4 ROI mode

v2
- export symbol of v4l2_ctrl_type_op_minimum
- export symbol of v4l2_ctrl_type_op_maximum

Hans Verkuil (1):
  media: v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL

Ming Qian (1):
  media: v4l2-ctrls: Add video encoder ROI ctrls

Yunke Cao (2):
  media: v4l2_ctrl: Add V4L2_CTRL_TYPE_RECT
  media: vivid: Add an rectangle control

 .../media/v4l/ext-ctrls-codec.rst             |  95 ++++++++++
 .../media/v4l/vidioc-g-ext-ctrls.rst          |  26 ++-
 .../media/v4l/vidioc-queryctrl.rst            |  14 ++
 .../media/videodev2.h.rst.exceptions          |   4 +
 drivers/media/i2c/imx214.c                    |   4 +-
 .../media/platform/qcom/venus/venc_ctrls.c    |   9 +-
 .../media/test-drivers/vivid/vivid-ctrls.c    |  34 ++++
 drivers/media/v4l2-core/v4l2-ctrls-api.c      |  54 ++++--
 drivers/media/v4l2-core/v4l2-ctrls-core.c     | 169 +++++++++++++++---
 drivers/media/v4l2-core/v4l2-ctrls-defs.c     |  46 +++++
 drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +-
 include/media/v4l2-ctrls.h                    |  62 ++++++-
 include/uapi/linux/v4l2-controls.h            |  16 ++
 include/uapi/linux/videodev2.h                |   5 +
 14 files changed, 493 insertions(+), 49 deletions(-)

-- 
2.43.0-rc1
Re: [RFC v3 0/4] Add video encoder ROI ctrls
Posted by Hans Verkuil 4 weeks ago
On 22/10/2024 10:40, Ming Qian wrote:
> Hi,
> 
> This patch set implements region of interest (ROI) ctrls for video
> encoder.
> 
> One video encoder IP may support the following two ROI configurations or
> one of them:
>     1. configure ROI as a rectangular region, and set a delta QP parameter.
>     2. configure ROI as a rectangular region, and set a priority parameter.
>     3. configure ROI as a QP map as an array. Each value represents the delta QP
> of a block in raster scan order. The block size is determined by
> the specific IP.
>     4. configure ROI as a QP map as an array. Each value represents the absolute QP
> of a block in raster scan order. The block size is determined by
> the specific IP.
> 
> To achieve this, I made the following change:
>     1. I reuse the type V4L2_CTRL_TYPE_RECT that is defined in the UVC ROI patchset
>     2. Define a ctrl V4L2_CID_MPEG_VIDEO_ROI_MODE to choose ROI configuration
>     3. Define some ctrl to configure ROI
>     4. Define a ctrl V4L2_CID_MPEG_VIDEO_ROI_BLOCK_SIZE to query block size
> 
> I referred the patchset "Implement UVC v1.5 ROI" (https://lwn.net/Articles/953532/)
> and pick some patches from it.

Please note that since this introduces the new TYPE_RECT and WHICH_MIN/MAX flags,
you also need to update v4l-utils.

This was done last year:

https://lore.kernel.org/linux-media/20230817072537.2837504-1-yunkec@google.com/

Can you pick this up and post a v2 for these v4l-utils changes? Note that I had some
comments at the time, so please fix those before posting v2.

Regards,

	Hans

> 
> changelog:
> 
> v3
> - Drop the type V4L2_CTRL_TYPE_REGION
> - Split the compound control into 2 ctrls
> - Define 4 ROI mode
> 
> v2
> - export symbol of v4l2_ctrl_type_op_minimum
> - export symbol of v4l2_ctrl_type_op_maximum
> 
> Hans Verkuil (1):
>   media: v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL
> 
> Ming Qian (1):
>   media: v4l2-ctrls: Add video encoder ROI ctrls
> 
> Yunke Cao (2):
>   media: v4l2_ctrl: Add V4L2_CTRL_TYPE_RECT
>   media: vivid: Add an rectangle control
> 
>  .../media/v4l/ext-ctrls-codec.rst             |  95 ++++++++++
>  .../media/v4l/vidioc-g-ext-ctrls.rst          |  26 ++-
>  .../media/v4l/vidioc-queryctrl.rst            |  14 ++
>  .../media/videodev2.h.rst.exceptions          |   4 +
>  drivers/media/i2c/imx214.c                    |   4 +-
>  .../media/platform/qcom/venus/venc_ctrls.c    |   9 +-
>  .../media/test-drivers/vivid/vivid-ctrls.c    |  34 ++++
>  drivers/media/v4l2-core/v4l2-ctrls-api.c      |  54 ++++--
>  drivers/media/v4l2-core/v4l2-ctrls-core.c     | 169 +++++++++++++++---
>  drivers/media/v4l2-core/v4l2-ctrls-defs.c     |  46 +++++
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +-
>  include/media/v4l2-ctrls.h                    |  62 ++++++-
>  include/uapi/linux/v4l2-controls.h            |  16 ++
>  include/uapi/linux/videodev2.h                |   5 +
>  14 files changed, 493 insertions(+), 49 deletions(-)
>
Re: [RFC v3 0/4] Add video encoder ROI ctrls
Posted by Ming Qian(OSS) 3 weeks, 6 days ago
Hi Hans,


On 2024/10/28 15:57, Hans Verkuil wrote:
> On 22/10/2024 10:40, Ming Qian wrote:
>> Hi,
>>
>> This patch set implements region of interest (ROI) ctrls for video
>> encoder.
>>
>> One video encoder IP may support the following two ROI configurations or
>> one of them:
>>      1. configure ROI as a rectangular region, and set a delta QP parameter.
>>      2. configure ROI as a rectangular region, and set a priority parameter.
>>      3. configure ROI as a QP map as an array. Each value represents the delta QP
>> of a block in raster scan order. The block size is determined by
>> the specific IP.
>>      4. configure ROI as a QP map as an array. Each value represents the absolute QP
>> of a block in raster scan order. The block size is determined by
>> the specific IP.
>>
>> To achieve this, I made the following change:
>>      1. I reuse the type V4L2_CTRL_TYPE_RECT that is defined in the UVC ROI patchset
>>      2. Define a ctrl V4L2_CID_MPEG_VIDEO_ROI_MODE to choose ROI configuration
>>      3. Define some ctrl to configure ROI
>>      4. Define a ctrl V4L2_CID_MPEG_VIDEO_ROI_BLOCK_SIZE to query block size
>>
>> I referred the patchset "Implement UVC v1.5 ROI" (https://lwn.net/Articles/953532/)
>> and pick some patches from it.
> 
> Please note that since this introduces the new TYPE_RECT and WHICH_MIN/MAX flags,
> you also need to update v4l-utils.
> 
> This was done last year:
> 
> https://lore.kernel.org/linux-media/20230817072537.2837504-1-yunkec@google.com/
> 
> Can you pick this up and post a v2 for these v4l-utils changes? Note that I had some
> comments at the time, so please fix those before posting v2.

I'll check the patch first, and pick it.

> 
> Regards,
> 
> 	Hans
> 
>>
>> changelog:
>>
>> v3
>> - Drop the type V4L2_CTRL_TYPE_REGION
>> - Split the compound control into 2 ctrls
>> - Define 4 ROI mode
>>
>> v2
>> - export symbol of v4l2_ctrl_type_op_minimum
>> - export symbol of v4l2_ctrl_type_op_maximum
>>
>> Hans Verkuil (1):
>>    media: v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL
>>
>> Ming Qian (1):
>>    media: v4l2-ctrls: Add video encoder ROI ctrls
>>
>> Yunke Cao (2):
>>    media: v4l2_ctrl: Add V4L2_CTRL_TYPE_RECT
>>    media: vivid: Add an rectangle control
>>
>>   .../media/v4l/ext-ctrls-codec.rst             |  95 ++++++++++
>>   .../media/v4l/vidioc-g-ext-ctrls.rst          |  26 ++-
>>   .../media/v4l/vidioc-queryctrl.rst            |  14 ++
>>   .../media/videodev2.h.rst.exceptions          |   4 +
>>   drivers/media/i2c/imx214.c                    |   4 +-
>>   .../media/platform/qcom/venus/venc_ctrls.c    |   9 +-
>>   .../media/test-drivers/vivid/vivid-ctrls.c    |  34 ++++
>>   drivers/media/v4l2-core/v4l2-ctrls-api.c      |  54 ++++--
>>   drivers/media/v4l2-core/v4l2-ctrls-core.c     | 169 +++++++++++++++---
>>   drivers/media/v4l2-core/v4l2-ctrls-defs.c     |  46 +++++
>>   drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +-
>>   include/media/v4l2-ctrls.h                    |  62 ++++++-
>>   include/uapi/linux/v4l2-controls.h            |  16 ++
>>   include/uapi/linux/videodev2.h                |   5 +
>>   14 files changed, 493 insertions(+), 49 deletions(-)
>>
>