Add userspace documentation for V4L2 ISP generic parameters and
statistics formats.
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
.../userspace-api/media/v4l/meta-formats.rst | 1 +
Documentation/userspace-api/media/v4l/v4l2-isp.rst | 121 +++++++++++++++++++++
MAINTAINERS | 1 +
3 files changed, 123 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst
index 0de80328c36bf148051a19abe9e5241234ddfe5c..261483f8e4d832d3d0ce8aa11df4b4eb1645f22f 100644
--- a/Documentation/userspace-api/media/v4l/meta-formats.rst
+++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
@@ -24,3 +24,4 @@ These formats are used for the :ref:`metadata` interface only.
metafmt-vivid
metafmt-vsp1-hgo
metafmt-vsp1-hgt
+ v4l2-isp
diff --git a/Documentation/userspace-api/media/v4l/v4l2-isp.rst b/Documentation/userspace-api/media/v4l/v4l2-isp.rst
new file mode 100644
index 0000000000000000000000000000000000000000..41078558ba5cc1faf922a9b9112e64c99ff37080
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/v4l2-isp.rst
@@ -0,0 +1,121 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _v4l2-isp:
+
+************************
+Generic V4L2 ISP formats
+************************
+
+ISP configuration and statistics: theory of operations
+======================================================
+
+ISP configuration parameters are computed by userspace and programmed into a
+*parameters buffer* which is queued to the ISP driver on a per-frame basis.
+
+ISP statistics are collected by the driver at a specific time point and drivers
+use them to populate a *statistics buffer* which is then returned to userspace.
+
+The parameters and statistics buffers are organized in a driver-specific
+way, and their data layout differs between one driver and another.
+
+ISP drivers generally exchange parameters and statistics with userspace through
+a metadata capture and output node respectively, implementing the
+:c:type:`v4l2_meta_format` interface. Each ISP driver defines a metadata capture
+and output format to be used on those video nodes, and the buffer layout and
+organization is fixed by the format definition.
+
+The uAPI/ABI problem
+--------------------
+
+By upstreaming the metadata formats that describe the parameters and statistics
+buffers layout, driver developers make them part of the Linux kernel ABI. As it
+sometimes happens for most peripherals in Linux, ISP drivers development is
+often an iterative process, where sometimes not all the hardware features are
+supported in the first version that lands in the kernel, and some parts of the
+interface have to later be modified for bug-fixes or improvements.
+
+If any later bug-fix/improvement requires changes to the metadata formats,
+this is considered an ABI-breakage that is strictly forbidden by the Linux
+kernel policies. For this reason, any change in the ISP parameters and
+statistics buffer layout would require defining a new metadata format.
+
+For these reasons Video4Linux2 has introduced support for generic ISP parameters
+and statistics data types, designed with the goal of being:
+
+- Extensible: new features can be added later on without breaking the existing
+ interface
+- Versioned: different versions of the format can be defined without
+ breaking the existing interface
+
+ISP configuration
+=================
+
+Before the introduction of generic formats
+------------------------------------------
+
+Metadata cature formats that describe ISP configuration parameters were most
+the time realized by defining C structures that reflect the ISP registers layout
+and gets populated by userspace before queueing the buffer to the ISP. Each
+C structure usually corresponds to one ISP *processing block*, with each block
+implementing one of the ISP supported features.
+
+The number of supported ISP blocks, the layout of their configuration data are
+fixed by the format definition, incurring the in the above described uAPI/uABI
+problems.
+
+Generic ISP parameters
+----------------------
+
+The generic ISP configuration parameters format is realized by a defining a
+single C structure that contains an header, followed by a binary buffer where
+userspace programs a variable number of ISP configuration data block, one for
+each supported ISP feature.
+
+The :c:type:`v4l2_isp_params_buffer` structure defines the parameters buffer
+header which is followed by a binary buffer of ISP configuration parameters.
+Userspace shall correctly populate the buffer header with the versioning
+information and with the size (in bytes) of the binary data buffer where it will
+store the ISP blocks configuration.
+
+Each *ISP configuration block* is preceded by an header implemented by the
+:c:type:`v4l2_isp_params_block_header` structure, followed by the configuration
+parameters for that specific block, defined by the ISP driver specific data
+types.
+
+Userspace applications are responsible for correctly populating each block's
+header fields (type, flags and size) and the block-specific parameters.
+
+ISP Block enabling, disabling and configuration
+-----------------------------------------------
+
+When userspace wants to configure and enable an ISP block it shall fully
+populate the block configuration and set the V4L2_ISP_PARAMS_FL_BLOCK_ENABLE
+bit in the block header's `flags` field.
+
+When userspace simply wants to disable an ISP block the
+V4L2_ISP_PARAMS_FL_BLOCK_DISABLE bit should be set in block header's `flags`
+field. Drivers accept a configuration parameters block with no additional
+data after the header in this case.
+
+If the configuration of an already active ISP block has to be updated,
+userspace shall fully populate the ISP block parameters and omit setting the
+V4L2_ISP_PARAMS_FL_BLOCK_ENABLE and V4L2_ISP_PARAMS_FL_BLOCK_DISABLE bits in the
+header's `flags` field.
+
+Setting both the V4L2_ISP_PARAMS_FL_BLOCK_ENABLE and
+V4L2_ISP_PARAMS_FL_BLOCK_DISABLE bits in the flags field is not allowed and not
+accepted.
+
+Any further extension to the parameters layout that happens after the ISP driver
+has been merged in Linux can be implemented by adding new blocks definition
+without invalidating the existing ones.
+
+ISP statistics
+==============
+
+Support for generic statistics format is not yet implemented in Video4Linux2.
+
+V4L2 ISP uAPI data types
+========================
+
+.. kernel-doc:: include/uapi/linux/media/v4l2-isp.h
diff --git a/MAINTAINERS b/MAINTAINERS
index e9ac834d212f88222437e8d806800b2516d44f01..340353334299cd5eebf1f72132b7e91b6f5fdbfe 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -26857,6 +26857,7 @@ V4L2 GENERIC ISP PARAMETERS AND STATISTIC FORMATS
M: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
L: linux-media@vger.kernel.org
S: Maintained
+F: Documentation/userspace-api/media/v4l/v4l2-isp.rst
F: include/uapi/linux/media/v4l2-isp.h
VF610 NAND DRIVER
--
2.51.0
Hi Jacopo, Thanks for your efforts! On 10/14/25 10:00, Jacopo Mondi wrote: > [...] > + > +The uAPI/ABI problem > +-------------------- > + > +By upstreaming the metadata formats that describe the parameters and statistics > +buffers layout, driver developers make them part of the Linux kernel ABI. As it > +sometimes happens for most peripherals in Linux, ISP drivers development is > +often an iterative process, where sometimes not all the hardware features are > +supported in the first version that lands in the kernel, and some parts of the > +interface have to later be modified for bug-fixes or improvements. Suggestion: As for most peripherals, ISP driver development in Linux is often an iterative process, in which not all of the hardware features are supported in the first version. The support for them and/or bug fixes may land in the kernel at a later stage. > + > +If any later bug-fix/improvement requires changes to the metadata formats, s/bug-fix/bug fix > +this is considered an ABI-breakage that is strictly forbidden by the Linux s/ABI-breakage/ABI breakage > +kernel policies. For this reason, any change in the ISP parameters and > +statistics buffer layout would require defining a new metadata format. > + > +For these reasons Video4Linux2 has introduced support for generic ISP parameters > +and statistics data types, designed with the goal of being: > + > +- Extensible: new features can be added later on without breaking the existing > + interface > +- Versioned: different versions of the format can be defined without > + breaking the existing interface > + > +ISP configuration > +================= > + > +Before the introduction of generic formats > +------------------------------------------ > + > +Metadata cature formats that describe ISP configuration parameters were most s/cature/capture s/most the time/"most of the time" or "typically" or "usually" or "normally"? > +the time realized by defining C structures that reflect the ISP registers layout > +and gets populated by userspace before queueing the buffer to the ISP. Each s/gets/get > +C structure usually corresponds to one ISP *processing block*, with each block > +implementing one of the ISP supported features. > + > +The number of supported ISP blocks, the layout of their configuration data are > +fixed by the format definition, incurring the in the above described uAPI/uABI > +problems. incurring the described uAPI/ABI problems described above. > + > +Generic ISP parameters > +---------------------- > + > +The generic ISP configuration parameters format is realized by a defining a > +single C structure that contains an header, followed by a binary buffer where s/an header/a header > +userspace programs a variable number of ISP configuration data block, one for > +each supported ISP feature. > + > +The :c:type:`v4l2_isp_params_buffer` structure defines the parameters buffer > +header which is followed by a binary buffer of ISP configuration parameters. > +Userspace shall correctly populate the buffer header with the versioning > +information and with the size (in bytes) of the binary data buffer where it will > +store the ISP blocks configuration. > + > +Each *ISP configuration block* is preceded by an header implemented by the > +:c:type:`v4l2_isp_params_block_header` structure, followed by the configuration > +parameters for that specific block, defined by the ISP driver specific data > +types. > + > +Userspace applications are responsible for correctly populating each block's > +header fields (type, flags and size) and the block-specific parameters. > + > +ISP Block enabling, disabling and configuration > +----------------------------------------------- > + > +When userspace wants to configure and enable an ISP block it shall fully > +populate the block configuration and set the V4L2_ISP_PARAMS_FL_BLOCK_ENABLE > +bit in the block header's `flags` field. > + > +When userspace simply wants to disable an ISP block the > +V4L2_ISP_PARAMS_FL_BLOCK_DISABLE bit should be set in block header's `flags` > +field. Drivers accept a configuration parameters block with no additional > +data after the header in this case. > + > +If the configuration of an already active ISP block has to be updated, > +userspace shall fully populate the ISP block parameters and omit setting the > +V4L2_ISP_PARAMS_FL_BLOCK_ENABLE and V4L2_ISP_PARAMS_FL_BLOCK_DISABLE bits in the > +header's `flags` field. > + > +Setting both the V4L2_ISP_PARAMS_FL_BLOCK_ENABLE and > +V4L2_ISP_PARAMS_FL_BLOCK_DISABLE bits in the flags field is not allowed and not > +accepted. > + > +Any further extension to the parameters layout that happens after the ISP driver > +has been merged in Linux can be implemented by adding new blocks definition > +without invalidating the existing ones. > + > +ISP statistics > +============== > + > +Support for generic statistics format is not yet implemented in Video4Linux2. > + > +V4L2 ISP uAPI data types > +======================== > + > +.. kernel-doc:: include/uapi/linux/media/v4l2-isp.h > diff --git a/MAINTAINERS b/MAINTAINERS > index e9ac834d212f88222437e8d806800b2516d44f01..340353334299cd5eebf1f72132b7e91b6f5fdbfe 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -26857,6 +26857,7 @@ V4L2 GENERIC ISP PARAMETERS AND STATISTIC FORMATS > M: Jacopo Mondi <jacopo.mondi@ideasonboard.com> > L: linux-media@vger.kernel.org > S: Maintained > +F: Documentation/userspace-api/media/v4l/v4l2-isp.rst > F: include/uapi/linux/media/v4l2-isp.h > > VF610 NAND DRIVER > With the comments above addressed, Reviewed-by: Michael Riesch <michael.riesch@collabora.com> Thanks and best regards, Michael
Hi Michael thanks for review. I took all comments in but.. On Tue, Oct 14, 2025 at 11:23:29AM +0200, Michael Riesch wrote: > Hi Jacopo, > > Thanks for your efforts! > > On 10/14/25 10:00, Jacopo Mondi wrote: > > [...] > > + > > +The uAPI/ABI problem > > +-------------------- > > + > > +By upstreaming the metadata formats that describe the parameters and statistics > > +buffers layout, driver developers make them part of the Linux kernel ABI. As it > > +sometimes happens for most peripherals in Linux, ISP drivers development is > > +often an iterative process, where sometimes not all the hardware features are > > +supported in the first version that lands in the kernel, and some parts of the > > +interface have to later be modified for bug-fixes or improvements. > > Suggestion: > > As for most peripherals, ISP driver development in Linux is often an > iterative process, in which not all of the hardware features are > supported in the first version. The support for them and/or bug fixes > may land in the kernel at a later stage. > > > + > > +If any later bug-fix/improvement requires changes to the metadata formats, > > s/bug-fix/bug fix > > > +this is considered an ABI-breakage that is strictly forbidden by the Linux > > s/ABI-breakage/ABI breakage > > > +kernel policies. For this reason, any change in the ISP parameters and > > +statistics buffer layout would require defining a new metadata format. > > + > > +For these reasons Video4Linux2 has introduced support for generic ISP parameters > > +and statistics data types, designed with the goal of being: > > + > > +- Extensible: new features can be added later on without breaking the existing > > + interface > > +- Versioned: different versions of the format can be defined without > > + breaking the existing interface > > + > > +ISP configuration > > +================= > > + > > +Before the introduction of generic formats > > +------------------------------------------ > > + > > +Metadata cature formats that describe ISP configuration parameters were most > > s/cature/capture > > s/most the time/"most of the time" or "typically" or "usually" or > "normally"? > > > +the time realized by defining C structures that reflect the ISP registers layout > > +and gets populated by userspace before queueing the buffer to the ISP. Each > > s/gets/get > > > +C structure usually corresponds to one ISP *processing block*, with each block > > +implementing one of the ISP supported features. > > + > > +The number of supported ISP blocks, the layout of their configuration data are > > +fixed by the format definition, incurring the in the above described uAPI/uABI > > +problems. > > incurring the described uAPI/ABI problems described above. > .. this one, for which I think the correct form is > +The number of supported ISP blocks, the layout of their configuration data are > +fixed by the format definition, incurring in the above described uAPI/uABI > +problem. Thanks j > > + > > +Generic ISP parameters > > +---------------------- > > + > > +The generic ISP configuration parameters format is realized by a defining a > > +single C structure that contains an header, followed by a binary buffer where > > s/an header/a header > > > +userspace programs a variable number of ISP configuration data block, one for > > +each supported ISP feature. > > + > > +The :c:type:`v4l2_isp_params_buffer` structure defines the parameters buffer > > +header which is followed by a binary buffer of ISP configuration parameters. > > +Userspace shall correctly populate the buffer header with the versioning > > +information and with the size (in bytes) of the binary data buffer where it will > > +store the ISP blocks configuration. > > + > > +Each *ISP configuration block* is preceded by an header implemented by the > > +:c:type:`v4l2_isp_params_block_header` structure, followed by the configuration > > +parameters for that specific block, defined by the ISP driver specific data > > +types. > > + > > +Userspace applications are responsible for correctly populating each block's > > +header fields (type, flags and size) and the block-specific parameters. > > + > > +ISP Block enabling, disabling and configuration > > +----------------------------------------------- > > + > > +When userspace wants to configure and enable an ISP block it shall fully > > +populate the block configuration and set the V4L2_ISP_PARAMS_FL_BLOCK_ENABLE > > +bit in the block header's `flags` field. > > + > > +When userspace simply wants to disable an ISP block the > > +V4L2_ISP_PARAMS_FL_BLOCK_DISABLE bit should be set in block header's `flags` > > +field. Drivers accept a configuration parameters block with no additional > > +data after the header in this case. > > + > > +If the configuration of an already active ISP block has to be updated, > > +userspace shall fully populate the ISP block parameters and omit setting the > > +V4L2_ISP_PARAMS_FL_BLOCK_ENABLE and V4L2_ISP_PARAMS_FL_BLOCK_DISABLE bits in the > > +header's `flags` field. > > + > > +Setting both the V4L2_ISP_PARAMS_FL_BLOCK_ENABLE and > > +V4L2_ISP_PARAMS_FL_BLOCK_DISABLE bits in the flags field is not allowed and not > > +accepted. > > + > > +Any further extension to the parameters layout that happens after the ISP driver > > +has been merged in Linux can be implemented by adding new blocks definition > > +without invalidating the existing ones. > > + > > +ISP statistics > > +============== > > + > > +Support for generic statistics format is not yet implemented in Video4Linux2. > > + > > +V4L2 ISP uAPI data types > > +======================== > > + > > +.. kernel-doc:: include/uapi/linux/media/v4l2-isp.h > > diff --git a/MAINTAINERS b/MAINTAINERS > > index e9ac834d212f88222437e8d806800b2516d44f01..340353334299cd5eebf1f72132b7e91b6f5fdbfe 100644 > > --- a/MAINTAINERS > > +++ b/MAINTAINERS > > @@ -26857,6 +26857,7 @@ V4L2 GENERIC ISP PARAMETERS AND STATISTIC FORMATS > > M: Jacopo Mondi <jacopo.mondi@ideasonboard.com> > > L: linux-media@vger.kernel.org > > S: Maintained > > +F: Documentation/userspace-api/media/v4l/v4l2-isp.rst > > F: include/uapi/linux/media/v4l2-isp.h > > > > VF610 NAND DRIVER > > > > > With the comments above addressed, > > Reviewed-by: Michael Riesch <michael.riesch@collabora.com> > > Thanks and best regards, > Michael > >
Hi Jacopo, On 10/20/25 10:09, Jacopo Mondi wrote: > Hi Michael > > thanks for review. I took all comments in but.. Cool, thanks! > > > On Tue, Oct 14, 2025 at 11:23:29AM +0200, Michael Riesch wrote: >> Hi Jacopo, >> >> Thanks for your efforts! >> >> On 10/14/25 10:00, Jacopo Mondi wrote: >>> [...] >>> + >>> +The uAPI/ABI problem >>> +-------------------- >>> + >>> +By upstreaming the metadata formats that describe the parameters and statistics >>> +buffers layout, driver developers make them part of the Linux kernel ABI. As it >>> +sometimes happens for most peripherals in Linux, ISP drivers development is >>> +often an iterative process, where sometimes not all the hardware features are >>> +supported in the first version that lands in the kernel, and some parts of the >>> +interface have to later be modified for bug-fixes or improvements. >> >> Suggestion: >> >> As for most peripherals, ISP driver development in Linux is often an >> iterative process, in which not all of the hardware features are >> supported in the first version. The support for them and/or bug fixes >> may land in the kernel at a later stage. >> >>> + >>> +If any later bug-fix/improvement requires changes to the metadata formats, >> >> s/bug-fix/bug fix >> >>> +this is considered an ABI-breakage that is strictly forbidden by the Linux >> >> s/ABI-breakage/ABI breakage >> >>> +kernel policies. For this reason, any change in the ISP parameters and >>> +statistics buffer layout would require defining a new metadata format. >>> + >>> +For these reasons Video4Linux2 has introduced support for generic ISP parameters >>> +and statistics data types, designed with the goal of being: >>> + >>> +- Extensible: new features can be added later on without breaking the existing >>> + interface >>> +- Versioned: different versions of the format can be defined without >>> + breaking the existing interface >>> + >>> +ISP configuration >>> +================= >>> + >>> +Before the introduction of generic formats >>> +------------------------------------------ >>> + >>> +Metadata cature formats that describe ISP configuration parameters were most >> >> s/cature/capture >> >> s/most the time/"most of the time" or "typically" or "usually" or >> "normally"? >> >>> +the time realized by defining C structures that reflect the ISP registers layout >>> +and gets populated by userspace before queueing the buffer to the ISP. Each >> >> s/gets/get >> >>> +C structure usually corresponds to one ISP *processing block*, with each block >>> +implementing one of the ISP supported features. >>> + >>> +The number of supported ISP blocks, the layout of their configuration data are >>> +fixed by the format definition, incurring the in the above described uAPI/uABI >>> +problems. >> >> incurring the described uAPI/ABI problems described above. >> > > .. this one, for which I think the correct form is > > > +The number of supported ISP blocks, the layout of their configuration data are > > +fixed by the format definition, incurring in the above described uAPI/uABI > > +problem. Maybe it's just me, but the sentence still does not sound correct to my ears. First, you enumerate two items, so they should be joined with "and", right? And then I understand that the two items have the uAPI/uABI problem as consequence, correct? "to result in" or "to lead to" seem to be better choices. That said, don't hesitate to point out things that I misunderstood. Best regards, Michael > > Thanks > j > >>> + >>> +Generic ISP parameters >>> +---------------------- >>> + >>> +The generic ISP configuration parameters format is realized by a defining a >>> +single C structure that contains an header, followed by a binary buffer where >> >> s/an header/a header >> >>> +userspace programs a variable number of ISP configuration data block, one for >>> +each supported ISP feature. >>> + >>> +The :c:type:`v4l2_isp_params_buffer` structure defines the parameters buffer >>> +header which is followed by a binary buffer of ISP configuration parameters. >>> +Userspace shall correctly populate the buffer header with the versioning >>> +information and with the size (in bytes) of the binary data buffer where it will >>> +store the ISP blocks configuration. >>> + >>> +Each *ISP configuration block* is preceded by an header implemented by the >>> +:c:type:`v4l2_isp_params_block_header` structure, followed by the configuration >>> +parameters for that specific block, defined by the ISP driver specific data >>> +types. >>> + >>> +Userspace applications are responsible for correctly populating each block's >>> +header fields (type, flags and size) and the block-specific parameters. >>> + >>> +ISP Block enabling, disabling and configuration >>> +----------------------------------------------- >>> + >>> +When userspace wants to configure and enable an ISP block it shall fully >>> +populate the block configuration and set the V4L2_ISP_PARAMS_FL_BLOCK_ENABLE >>> +bit in the block header's `flags` field. >>> + >>> +When userspace simply wants to disable an ISP block the >>> +V4L2_ISP_PARAMS_FL_BLOCK_DISABLE bit should be set in block header's `flags` >>> +field. Drivers accept a configuration parameters block with no additional >>> +data after the header in this case. >>> + >>> +If the configuration of an already active ISP block has to be updated, >>> +userspace shall fully populate the ISP block parameters and omit setting the >>> +V4L2_ISP_PARAMS_FL_BLOCK_ENABLE and V4L2_ISP_PARAMS_FL_BLOCK_DISABLE bits in the >>> +header's `flags` field. >>> + >>> +Setting both the V4L2_ISP_PARAMS_FL_BLOCK_ENABLE and >>> +V4L2_ISP_PARAMS_FL_BLOCK_DISABLE bits in the flags field is not allowed and not >>> +accepted. >>> + >>> +Any further extension to the parameters layout that happens after the ISP driver >>> +has been merged in Linux can be implemented by adding new blocks definition >>> +without invalidating the existing ones. >>> + >>> +ISP statistics >>> +============== >>> + >>> +Support for generic statistics format is not yet implemented in Video4Linux2. >>> + >>> +V4L2 ISP uAPI data types >>> +======================== >>> + >>> +.. kernel-doc:: include/uapi/linux/media/v4l2-isp.h >>> diff --git a/MAINTAINERS b/MAINTAINERS >>> index e9ac834d212f88222437e8d806800b2516d44f01..340353334299cd5eebf1f72132b7e91b6f5fdbfe 100644 >>> --- a/MAINTAINERS >>> +++ b/MAINTAINERS >>> @@ -26857,6 +26857,7 @@ V4L2 GENERIC ISP PARAMETERS AND STATISTIC FORMATS >>> M: Jacopo Mondi <jacopo.mondi@ideasonboard.com> >>> L: linux-media@vger.kernel.org >>> S: Maintained >>> +F: Documentation/userspace-api/media/v4l/v4l2-isp.rst >>> F: include/uapi/linux/media/v4l2-isp.h >>> >>> VF610 NAND DRIVER >>> >> >> >> With the comments above addressed, >> >> Reviewed-by: Michael Riesch <michael.riesch@collabora.com> >> >> Thanks and best regards, >> Michael >> >>
© 2016 - 2025 Red Hat, Inc.