arch/arm64/boot/dts/freescale/Makefile | 4 + .../imx8mm-tqma8mqml-mba8mx-imx327.dts | 95 +++++++++++++ drivers/staging/media/imx/imx-ic-prpencvf.c | 3 +- drivers/staging/media/imx/imx-media-capture.c | 20 ++- drivers/staging/media/imx/imx-media-csi.c | 3 +- drivers/staging/media/imx/imx-media-utils.c | 130 +++++++++++++++++- drivers/staging/media/imx/imx-media.h | 6 +- drivers/staging/media/imx/imx7-media-csi.c | 15 +- drivers/staging/media/imx/imx7-mipi-csis.c | 1 + 9 files changed, 262 insertions(+), 15 deletions(-) create mode 100644 arch/arm64/boot/dts/freescale/imx8mm-tqma8mqml-mba8mx-imx327.dts
Hey everyone,
this is a set of patch for imx[7] media drivers based on next-20220203. With
this set I was able to capture video frames from a MIPI CSI-2 camera in my
TQMa8MQML + MBA8MX hardware. The actual camera used is a Vision Components
'VC MIPI IMX327 C' camera. IMX327 is compatible to IMX290. Patch 8 shows the
DT overlay I'm using, not suitable for merging.
Please ignore the FPGA part, this is mainly for power supply and GPIO reset
line. This is currently a custom driver I'm working on, but I do not want to
focus on in this series.
Please note I tested this only on this imx8 platform.
First thanks to Dorota for the patchset at [1] (patches 1-4) which is necessary
to capture correct images. I integrated Hans' review into it. I hope the
I didn't make a mistake and the original author is kept along. I used v4 for that
patchset, it is v1 again in this set. I hope this is not confusing.
Starting from patch 5 there are small fixes which allows me to configure my
media device.
Device configuration:
```
media-ctl -l "'imx290 2-001a':0->'csis-32e30000.mipi-csi':0 [1]"
media-ctl -V "'imx290 2-001a':0 [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw]"
media-ctl -V "'csi':0 [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw]"
v4l2-ctl -d0 --set-fmt-video width=1920,height=1080,pixelformat='RG10',field=none
media-ctl -p
```
The media-ctl topology is:
```
# media-ctl -p
Media controller API version 5.17.0
Media device information
------------------------
driver imx7-csi
model imx-media
serial
bus info platform:32e20000.csi
hw revision 0x0
driver version 5.17.0
Device topology
- entity 1: csi (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range]
<- "csis-32e30000.mipi-csi":1 [ENABLED,IMMUTABLE]
pad1: Source
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range]
-> "csi capture":0 [ENABLED,IMMUTABLE]
- entity 4: csi capture (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "csi":1 [ENABLED,IMMUTABLE]
- entity 10: csis-32e30000.mipi-csi (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:709 ycbcr:601 quantization:lim-range]
<- "imx290 2-001a":0 [ENABLED]
pad1: Source
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:709 ycbcr:601 quantization:lim-range]
-> "csi":0 [ENABLED,IMMUTABLE]
- entity 15: imx290 2-001a (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw]
-> "csis-32e30000.mipi-csi":0 [ENABLED]
```
Note: MIPI CSI settle times are not calculated correctly right now and need a
manual overwrite:
echo 13 > /sys/kernel/debug/32e30000.mipi-csi/ths_settle
echo 2 > /sys/kernel/debug/32e30000.mipi-csi/tclk_settle
I ignored the settings for xfer, ycbcr and quantization for now. I do neither
know how they will affect me nor what it should be.
Also I did not focus on v4l2-compliance tool, this is a further task as well.
IMHO imx7-mipi-csis.c should also create an immutable link to the camera
sensor.
Regards,
Alexander
[1] https://patchwork.linuxtv.org/project/linux-media/patch/20211104113631.206899-2-dorota.czaplejewicz@puri.sm/
Alexander Stein (4):
media: imx: imx7_mipi_csis: store colorspace in set_fmt as well
media: imx: imx7_media-csi: Add support for additional Bayer patterns
media: imx: utils: Add more Bayer formats
[DNI] arm64: dts: tqma8mqml: add IMX327 MIPI-CSI overlay
Dorota Czaplejewicz (4):
media: imx: Store the type of hardware implementation
media: imx: Forward type of hardware implementation
media: imx: Use dedicated format handler for i.MX7/8
media: imx: Fail conversion if pixel format not supported
arch/arm64/boot/dts/freescale/Makefile | 4 +
.../imx8mm-tqma8mqml-mba8mx-imx327.dts | 95 +++++++++++++
drivers/staging/media/imx/imx-ic-prpencvf.c | 3 +-
drivers/staging/media/imx/imx-media-capture.c | 20 ++-
drivers/staging/media/imx/imx-media-csi.c | 3 +-
drivers/staging/media/imx/imx-media-utils.c | 130 +++++++++++++++++-
drivers/staging/media/imx/imx-media.h | 6 +-
drivers/staging/media/imx/imx7-media-csi.c | 15 +-
drivers/staging/media/imx/imx7-mipi-csis.c | 1 +
9 files changed, 262 insertions(+), 15 deletions(-)
create mode 100644 arch/arm64/boot/dts/freescale/imx8mm-tqma8mqml-mba8mx-imx327.dts
--
2.25.1
Hi Alexander, On Fri, Feb 04, 2022 at 01:15:06PM +0100, Alexander Stein wrote: > Hey everyone, > > this is a set of patch for imx[7] media drivers based on next-20220203. With > this set I was able to capture video frames from a MIPI CSI-2 camera in my > TQMa8MQML + MBA8MX hardware. The actual camera used is a Vision Components > 'VC MIPI IMX327 C' camera. IMX327 is compatible to IMX290. Patch 8 shows the > DT overlay I'm using, not suitable for merging. You may be interested in https://gitlab.com/ideasonboard/nxp/linux/-/commits/pinchartl/v5.17/sensors/ > Please ignore the FPGA part, this is mainly for power supply and GPIO reset > line. This is currently a custom driver I'm working on, but I do not want to > focus on in this series. > > Please note I tested this only on this imx8 platform. > > First thanks to Dorota for the patchset at [1] (patches 1-4) which is necessary > to capture correct images. I integrated Hans' review into it. I hope the > I didn't make a mistake and the original author is kept along. I used v4 for that > patchset, it is v1 again in this set. I hope this is not confusing. > > Starting from patch 5 there are small fixes which allows me to configure my > media device. > > Device configuration: > ``` > media-ctl -l "'imx290 2-001a':0->'csis-32e30000.mipi-csi':0 [1]" > media-ctl -V "'imx290 2-001a':0 [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw]" > media-ctl -V "'csi':0 [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw]" > v4l2-ctl -d0 --set-fmt-video width=1920,height=1080,pixelformat='RG10',field=none > media-ctl -p > ``` > > The media-ctl topology is: > ``` > # media-ctl -p > Media controller API version 5.17.0 > > Media device information > ------------------------ > driver imx7-csi > model imx-media > serial > bus info platform:32e20000.csi > hw revision 0x0 > driver version 5.17.0 > > Device topology > - entity 1: csi (2 pads, 2 links) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev0 > pad0: Sink > [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range] > <- "csis-32e30000.mipi-csi":1 [ENABLED,IMMUTABLE] > pad1: Source > [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range] > -> "csi capture":0 [ENABLED,IMMUTABLE] > > - entity 4: csi capture (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video0 > pad0: Sink > <- "csi":1 [ENABLED,IMMUTABLE] > > - entity 10: csis-32e30000.mipi-csi (2 pads, 2 links) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev1 > pad0: Sink > [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:709 ycbcr:601 quantization:lim-range] > <- "imx290 2-001a":0 [ENABLED] > pad1: Source > [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:709 ycbcr:601 quantization:lim-range] > -> "csi":0 [ENABLED,IMMUTABLE] > > - entity 15: imx290 2-001a (1 pad, 1 link) > type V4L2 subdev subtype Sensor flags 0 > device node name /dev/v4l-subdev2 > pad0: Source > [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw] > -> "csis-32e30000.mipi-csi":0 [ENABLED] > ``` > > Note: MIPI CSI settle times are not calculated correctly right now and need a > manual overwrite: > echo 13 > /sys/kernel/debug/32e30000.mipi-csi/ths_settle > echo 2 > /sys/kernel/debug/32e30000.mipi-csi/tclk_settle The reference manual isn't very prolix on tclk_settle :-S That's something I'd love to figure out one day. For ths_settle, is the issue on the CSIS driver side, or the sensor driver side ? > I ignored the settings for xfer, ycbcr and quantization for now. I do neither > know how they will affect me nor what it should be. > Also I did not focus on v4l2-compliance tool, this is a further task as well. > IMHO imx7-mipi-csis.c should also create an immutable link to the camera > sensor. That makes sense, but note that, while CSI-2 is meant to be a point-to-point bus, there are boards designed with multiple sensors connected to the same CSI-2 RX without any hardware multiplexer. They keep one of the two sensors in reset at all times and are lucky that the signal reflections don't mess things up. > [1] https://patchwork.linuxtv.org/project/linux-media/patch/20211104113631.206899-2-dorota.czaplejewicz@puri.sm/ > > Alexander Stein (4): > media: imx: imx7_mipi_csis: store colorspace in set_fmt as well > media: imx: imx7_media-csi: Add support for additional Bayer patterns > media: imx: utils: Add more Bayer formats > [DNI] arm64: dts: tqma8mqml: add IMX327 MIPI-CSI overlay > > Dorota Czaplejewicz (4): > media: imx: Store the type of hardware implementation > media: imx: Forward type of hardware implementation > media: imx: Use dedicated format handler for i.MX7/8 > media: imx: Fail conversion if pixel format not supported > > arch/arm64/boot/dts/freescale/Makefile | 4 + > .../imx8mm-tqma8mqml-mba8mx-imx327.dts | 95 +++++++++++++ > drivers/staging/media/imx/imx-ic-prpencvf.c | 3 +- > drivers/staging/media/imx/imx-media-capture.c | 20 ++- > drivers/staging/media/imx/imx-media-csi.c | 3 +- > drivers/staging/media/imx/imx-media-utils.c | 130 +++++++++++++++++- > drivers/staging/media/imx/imx-media.h | 6 +- > drivers/staging/media/imx/imx7-media-csi.c | 15 +- > drivers/staging/media/imx/imx7-mipi-csis.c | 1 + > 9 files changed, 262 insertions(+), 15 deletions(-) > create mode 100644 arch/arm64/boot/dts/freescale/imx8mm-tqma8mqml-mba8mx-imx327.dts -- Regards, Laurent Pinchart
Hi Laurent, Am Samstag, 5. Februar 2022, 04:16:54 CET schrieb Laurent Pinchart: > On Fri, Feb 04, 2022 at 01:15:06PM +0100, Alexander Stein wrote: > > Hey everyone, > > > > this is a set of patch for imx[7] media drivers based on next-20220203. > > With this set I was able to capture video frames from a MIPI CSI-2 camera > > in my TQMa8MQML + MBA8MX hardware. The actual camera used is a Vision > > Components 'VC MIPI IMX327 C' camera. IMX327 is compatible to IMX290. > > Patch 8 shows the DT overlay I'm using, not suitable for merging. > > You may be interested in > https://gitlab.com/ideasonboard/nxp/linux/-/commits/pinchartl/v5.17/sensors > / Thanks for your feedback. Working on imx290 driver for proper imx327 support is on my todo. For the records, there are also patches at [1]. > > Please ignore the FPGA part, this is mainly for power supply and GPIO > > reset > > line. This is currently a custom driver I'm working on, but I do not want > > to focus on in this series. > > > > Please note I tested this only on this imx8 platform. > > > > First thanks to Dorota for the patchset at [1] (patches 1-4) which is > > necessary to capture correct images. I integrated Hans' review into it. I > > hope the I didn't make a mistake and the original author is kept along. I > > used v4 for that patchset, it is v1 again in this set. I hope this is not > > confusing. > > > > Starting from patch 5 there are small fixes which allows me to configure > > my > > media device. > > > > Device configuration: > > ``` > > media-ctl -l "'imx290 2-001a':0->'csis-32e30000.mipi-csi':0 [1]" > > media-ctl -V "'imx290 2-001a':0 [fmt:SRGGB10_1X10/1920x1080 field:none > > colorspace:raw]" media-ctl -V "'csi':0 [fmt:SRGGB10_1X10/1920x1080 > > field:none colorspace:raw]" v4l2-ctl -d0 --set-fmt-video > > width=1920,height=1080,pixelformat='RG10',field=none media-ctl -p > > ``` > > > > The media-ctl topology is: > > ``` > > # media-ctl -p > > Media controller API version 5.17.0 > > > > Media device information > > ------------------------ > > driver imx7-csi > > model imx-media > > serial > > bus info platform:32e20000.csi > > hw revision 0x0 > > driver version 5.17.0 > > > > Device topology > > - entity 1: csi (2 pads, 2 links) > > > > type V4L2 subdev subtype Unknown flags 0 > > device node name /dev/v4l-subdev0 > > > > pad0: Sink > > > > [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw > > xfer:none ycbcr:601 quantization:full-range] <- > > "csis-32e30000.mipi-csi":1 [ENABLED,IMMUTABLE] > > > > pad1: Source > > > > [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw > > xfer:none ycbcr:601 quantization:full-range] -> "csi > > capture":0 [ENABLED,IMMUTABLE] > > > > - entity 4: csi capture (1 pad, 1 link) > > > > type Node subtype V4L flags 0 > > device node name /dev/video0 > > > > pad0: Sink > > > > <- "csi":1 [ENABLED,IMMUTABLE] > > > > - entity 10: csis-32e30000.mipi-csi (2 pads, 2 links) > > > > type V4L2 subdev subtype Unknown flags 0 > > device node name /dev/v4l-subdev1 > > > > pad0: Sink > > > > [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw > > xfer:709 ycbcr:601 quantization:lim-range] <- "imx290 > > 2-001a":0 [ENABLED] > > > > pad1: Source > > > > [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw > > xfer:709 ycbcr:601 quantization:lim-range] -> "csi":0 > > [ENABLED,IMMUTABLE] > > > > - entity 15: imx290 2-001a (1 pad, 1 link) > > > > type V4L2 subdev subtype Sensor flags 0 > > device node name /dev/v4l-subdev2 > > > > pad0: Source > > > > [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw] > > -> "csis-32e30000.mipi-csi":0 [ENABLED] > > > > ``` > > > > Note: MIPI CSI settle times are not calculated correctly right now and > > need a manual overwrite: > > echo 13 > /sys/kernel/debug/32e30000.mipi-csi/ths_settle > > echo 2 > /sys/kernel/debug/32e30000.mipi-csi/tclk_settle > > The reference manual isn't very prolix on tclk_settle :-S That's > something I'd love to figure out one day. > > For ths_settle, is the issue on the CSIS driver side, or the sensor > driver side ? I can't answer that yet, but during my work on a fslc linux-5.10 I noticed that the link frequency and/or some other clock was wrong, so the calculated value didn't match. Anyway I get the impression that the final values might also be affected by the actual hardware. But I do not know any details regarding this. > > I ignored the settings for xfer, ycbcr and quantization for now. I do > > neither know how they will affect me nor what it should be. > > Also I did not focus on v4l2-compliance tool, this is a further task as > > well. IMHO imx7-mipi-csis.c should also create an immutable link to the > > camera sensor. > > That makes sense, but note that, while CSI-2 is meant to be a > point-to-point bus, there are boards designed with multiple sensors > connected to the same CSI-2 RX without any hardware multiplexer. They > keep one of the two sensors in reset at all times and are lucky that the > signal reflections don't mess things up. Still, even if only one device is actually powered, it is a immutable link from v4l2 perspective. You don't switch the sensors on the fly, or do you really do that? In the end immutable links are only a minor issue (to me). Regards, Alexander [1] https://github.com/raspberrypi/linux/commits/rpi-5.10.y/drivers/media/i2c/ imx290.c
© 2016 - 2026 Red Hat, Inc.