Hello
This patch series introduces the NXP Neo Image Signal Processor (ISP)
driver, used in the NXP i.MX95 SoC and future devices in the i.MX9 family.
---
Dependencies
------------
This series is based on v1 of:
https://lore.kernel.org/linux-media/20260505-extensible-stats-v1-0-e16f326b8dad@ideasonboard.com
The reference tag I rebased all on is:
next-20260505
---
Description
-----------
The Neo ISP processes one or more camera streams, converting RAW formats
into YUV or RGB outputs. Its architecture is largely influenced by the
PISP driver. To limit the number of v4l2 devices, the driver supports only
one context, with three sink pads (main input, second input for HDR, and
parameter buffers) and three source pads (RGB output, IR output, and
statistics metadata).
The driver uses the generic extensible v4l2-isp framework for parameters,
similar to rkisp1 and mali-c55. The same mechanism is reused for
statistics buffers by relying on the extensible stats introduced in
Jacopo's series.
The driver currently supports M2M operation; inline mode (CSI-to-ISP
streaming) is still under evaluation.
A few checkpatch warnings in v4l2-ioctl.c are intentionally kept to match
the existing coding style in that file. Another checkpatch warning
related to job pointer initialization is also kept, as it matches the
approach used in the pispbe driver.
Testing was performed on the i.MX95 EVK with neoisp configured in
standalone mode, with a vivid (Virtual Video Device) instance and a
libcamera neo pipeline handler. An engineering version of the libcamera
pipeline handler is under preparation, and can be shared as dedicated
branch if needed.
End-to-end camera-to-ISP capture, using a single media graph, has been
validated using the downstream NXP kernel.
Thanks,
Antoine
---
link to v1: https://lore.kernel.org/linux-media/20260413160331.2611829-1-antoine.bouyer@nxp.com/
Diff compared to v1:
* Fix dt_binding_check errors reported by Rob's bot.
* Remove extensible stats introduction, and use v4l2_isp patches
from Jacopo's series instead.
* Use the common v4l2_isp definitions for stats and params in neoisp.
Replace all occurences of v4l2_isp_params_* and v4l2_isp_stats_* by
the common structs and helpers.
* Use the new v4l2_isp helpers for statistics buffer handling.
* Apply comment from Geert in Kconfig.
* Fix some typo in the neoisp documentation.
* Remove `neoisp_feat_ctrl_s` from uapi (used only in legacy format
which is not supported anymore).
link to RFC: https://lore.kernel.org/linux-media/20260123080938.3367348-1-antoine.bouyer@nxp.com/
Diff compared to RFC:
* Integrate Krzysztof's comments in neoisp bindings document: use
maxItems for clocks, remove unused configs, and rename filename to
match compatible name used in imx95 SoC.
* Provide a `neoisp_core_media_register` API to let neoisp register
itself into an existing media graph, instead of creating its own. The
goal is to prepare for supporting inline mode alongside M2M mode, and
to allow userspace to select between these modes at runtime (the 2
modes cannot run together because of hardware constraints).
* Use only one Neo ISP context, whereas the RFC version prepared 8
contexts.
* Add a module parameter to support a standalone mode. When enabled,
neoisp registers its own media graph. The goal is to allow testing the
Neo ISP IP without a camera or other subdevice drivers, such as ISI,
pixel formatter, etc.
* Remove support of the legacy mode using fixed-size buffers for parameters
and statistics; only the generic extensible framework is supported.
* Remove support of the hardware version 1.
* Use job scheduling like pispbe driver, to limit impact on interrupt
handler.
---
Antoine Bouyer (8):
dt-bindings: media: Add nxp neoisp support
media: v4l2-ctrls: Add user control base for NXP neoisp controls
media: Add meta formats supported by NXP neoisp driver
media: uapi: Add NXP NEOISP user interface header file
media: Documentation: Add NXP neoisp driver documentation
media: platform: Add NXP Neoisp Image Signal Processor
media: platform: neoisp: Add debugfs support
arm64: dts: freescale: imx95: Add NXP neoisp device tree node
.../admin-guide/media/nxp-neoisp-diagram.dot | 24 +
.../admin-guide/media/nxp-neoisp.dot | 18 +
.../admin-guide/media/nxp-neoisp.rst | 179 ++
.../admin-guide/media/v4l-drivers.rst | 1 +
.../bindings/media/nxp,imx95-neoisp.yaml | 62 +
.../userspace-api/media/v4l/meta-formats.rst | 1 +
.../media/v4l/metafmt-nxp-neoisp.rst | 70 +
MAINTAINERS | 9 +
.../boot/dts/freescale/imx95-19x19-evk.dts | 4 +
arch/arm64/boot/dts/freescale/imx95.dtsi | 11 +
drivers/media/platform/nxp/Kconfig | 1 +
drivers/media/platform/nxp/Makefile | 1 +
drivers/media/platform/nxp/neoisp/Kconfig | 16 +
drivers/media/platform/nxp/neoisp/Makefile | 8 +
drivers/media/platform/nxp/neoisp/neoisp.h | 263 ++
.../media/platform/nxp/neoisp/neoisp_core.h | 30 +
.../media/platform/nxp/neoisp/neoisp_ctx.c | 2657 +++++++++++++++++
.../media/platform/nxp/neoisp/neoisp_ctx.h | 78 +
.../platform/nxp/neoisp/neoisp_debugfs.c | 494 +++
.../media/platform/nxp/neoisp/neoisp_fmt.h | 495 +++
drivers/media/platform/nxp/neoisp/neoisp_hw.h | 557 ++++
.../media/platform/nxp/neoisp/neoisp_main.c | 1916 ++++++++++++
.../media/platform/nxp/neoisp/neoisp_nodes.h | 54 +
.../media/platform/nxp/neoisp/neoisp_regs.h | 2498 ++++++++++++++++
drivers/media/v4l2-core/v4l2-ioctl.c | 2 +
include/uapi/linux/media/nxp/nxp_neoisp.h | 1694 +++++++++++
include/uapi/linux/v4l2-controls.h | 6 +
include/uapi/linux/videodev2.h | 4 +
28 files changed, 11153 insertions(+)
create mode 100644 Documentation/admin-guide/media/nxp-neoisp-diagram.dot
create mode 100644 Documentation/admin-guide/media/nxp-neoisp.dot
create mode 100644 Documentation/admin-guide/media/nxp-neoisp.rst
create mode 100644 Documentation/devicetree/bindings/media/nxp,imx95-neoisp.yaml
create mode 100644 Documentation/userspace-api/media/v4l/metafmt-nxp-neoisp.rst
create mode 100644 drivers/media/platform/nxp/neoisp/Kconfig
create mode 100644 drivers/media/platform/nxp/neoisp/Makefile
create mode 100644 drivers/media/platform/nxp/neoisp/neoisp.h
create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_core.h
create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_ctx.c
create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_ctx.h
create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_debugfs.c
create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_fmt.h
create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_hw.h
create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_main.c
create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_nodes.h
create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_regs.h
create mode 100644 include/uapi/linux/media/nxp/nxp_neoisp.h
---
base-commit: 4cd074ae20bbcc293bbbce9163abe99d68ae6ae0
prerequisite-patch-id: 2939dff7f477209138725aa8d86318d6580f4ea3
prerequisite-patch-id: f77370d8fc480b2e972a773f7d56f33ff1995eef
prerequisite-patch-id: bd3b231bc86129d25fca4ae34408a61db43bf883
prerequisite-patch-id: 691322bcfe69b3c51a886441fb241067355b9bdc
prerequisite-patch-id: a5f27213eba078f7ed5420d90df3ecd45151219e
prerequisite-patch-id: 9b2fbe610fb17689030c6da6c529f7db91b86d9c
--
2.51.0