Add starfive_camss.rst file that documents the Starfive Camera
Subsystem driver which is used for handing image sensor data.
Signed-off-by: Jack Zhu <jack.zhu@starfivetech.com>
---
.../admin-guide/media/starfive_camss.rst | 57 +++++++++++++++++++
.../media/starfive_camss_graph.dot | 16 ++++++
.../admin-guide/media/v4l-drivers.rst | 1 +
MAINTAINERS | 1 +
4 files changed, 75 insertions(+)
create mode 100644 Documentation/admin-guide/media/starfive_camss.rst
create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot
diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst
new file mode 100644
index 000000000000..a6378849384f
--- /dev/null
+++ b/Documentation/admin-guide/media/starfive_camss.rst
@@ -0,0 +1,57 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. include:: <isonum.txt>
+
+================================
+Starfive Camera Subsystem driver
+================================
+
+Introduction
+------------
+
+This file documents the driver for the Starfive Camera Subsystem found on
+Starfive JH7110 SoC. The driver is located under drivers/media/platform/
+starfive.
+
+The driver implements V4L2, Media controller and v4l2_subdev interfaces.
+Camera sensor using V4L2 subdev interface in the kernel is supported.
+
+The driver has been successfully used on the Gstreamer 1.18.5 with
+v4l2src plugin.
+
+
+Starfive Camera Subsystem hardware
+----------------------------------
+
+The Starfive Camera Subsystem hardware consists of:
+
+- MIPI DPHY Receiver: receives mipi data from a MIPI camera sensor.
+- MIPI CSIRx Controller: is responsible for handling and decoding CSI2 protocol
+ based camera sensor data stream.
+- ISP: handles the image data streams from the MIPI CSIRx Controller.
+- VIN(Video In): a top-level module, is responsible for controlling power
+ and clocks to other modules, dumps the input data to memory or transfers the
+ input data to ISP.
+
+
+Topology
+--------
+
+The media controller pipeline graph is as follows:
+
+.. _starfive_camss_graph:
+
+.. kernel-figure:: starfive_camss_graph.dot
+ :alt: starfive_camss_graph.dot
+ :align: center
+
+The driver has 2 video devices:
+
+- stf_vin0_wr_video0: capture device for images directly from the VIN module.
+- stf_vin0_isp0_video1: capture device for images without scaling.
+
+The driver has 3 subdevices:
+
+- stf_isp0: is responsible for all the isp operations.
+- stf_vin0_wr: used to dump RAW images to memory.
+- stf_vin0_isp0: used to capture images for the stf_vin0_isp0_video1 device.
diff --git a/Documentation/admin-guide/media/starfive_camss_graph.dot b/Documentation/admin-guide/media/starfive_camss_graph.dot
new file mode 100644
index 000000000000..275661b720dc
--- /dev/null
+++ b/Documentation/admin-guide/media/starfive_camss_graph.dot
@@ -0,0 +1,16 @@
+digraph board {
+ rankdir=TB
+ n00000001 [label="{{<port0> 0} | stf_isp0\n/dev/v4l-subdev0 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000001:port1 -> n0000000d:port0 [style=dashed]
+ n00000004 [label="{{<port0> 0} | stf_vin0_wr\n/dev/v4l-subdev1 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000004:port1 -> n00000007 [style=bold]
+ n00000007 [label="stf_vin0_wr_video0\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
+ n0000000d [label="{{<port0> 0} | stf_vin0_isp0\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
+ n0000000d:port1 -> n00000010 [style=bold]
+ n00000010 [label="stf_vin0_isp0_video1\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
+ n00000018 [label="{{<port0> 0} | cdns_csi2rx.19800000.csi-bridge\n | {<port1> 1 | <port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000018:port1 -> n00000004:port0 [style=dashed]
+ n00000018:port1 -> n00000001:port0 [style=dashed]
+ n00000022 [label="{{} | imx219 6-0010\n/dev/v4l-subdev3 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000022:port0 -> n00000018:port0 [style=bold]
+}
diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst
index 1c41f87c3917..2274fd29c1d7 100644
--- a/Documentation/admin-guide/media/v4l-drivers.rst
+++ b/Documentation/admin-guide/media/v4l-drivers.rst
@@ -27,6 +27,7 @@ Video4Linux (V4L) driver-specific documentation
si470x
si4713
si476x
+ starfive_camss
vimc
visl
vivid
diff --git a/MAINTAINERS b/MAINTAINERS
index 9ef5d0e5dff0..71291dc58671 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -20075,6 +20075,7 @@ M: Jack Zhu <jack.zhu@starfivetech.com>
M: Changhuang Liang <changhuang.liang@starfivetech.com>
L: linux-media@vger.kernel.org
S: Maintained
+F: Documentation/admin-guide/media/starfive_camss.rst
F: Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
STARFIVE DEVICETREES
--
2.34.1On 19/06/2023 12:28, Jack Zhu wrote: > Add starfive_camss.rst file that documents the Starfive Camera > Subsystem driver which is used for handing image sensor data. > > Signed-off-by: Jack Zhu <jack.zhu@starfivetech.com> > --- > .../admin-guide/media/starfive_camss.rst | 57 +++++++++++++++++++ > .../media/starfive_camss_graph.dot | 16 ++++++ > .../admin-guide/media/v4l-drivers.rst | 1 + > MAINTAINERS | 1 + > 4 files changed, 75 insertions(+) > create mode 100644 Documentation/admin-guide/media/starfive_camss.rst > create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot > > diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst > new file mode 100644 > index 000000000000..a6378849384f > --- /dev/null > +++ b/Documentation/admin-guide/media/starfive_camss.rst > @@ -0,0 +1,57 @@ > +.. SPDX-License-Identifier: GPL-2.0 > + > +.. include:: <isonum.txt> > + > +================================ > +Starfive Camera Subsystem driver > +================================ > + > +Introduction > +------------ > + > +This file documents the driver for the Starfive Camera Subsystem found on > +Starfive JH7110 SoC. The driver is located under drivers/media/platform/ > +starfive. > + > +The driver implements V4L2, Media controller and v4l2_subdev interfaces. > +Camera sensor using V4L2 subdev interface in the kernel is supported. > + > +The driver has been successfully used on the Gstreamer 1.18.5 with > +v4l2src plugin. > + > + > +Starfive Camera Subsystem hardware > +---------------------------------- > + > +The Starfive Camera Subsystem hardware consists of: > + > +- MIPI DPHY Receiver: receives mipi data from a MIPI camera sensor. Feels like a terribe nit-pick but you have "mipi" and "MIPI" here. I'd be consistent with one - recommend MIPI throughout your documentation. > +- MIPI CSIRx Controller: is responsible for handling and decoding CSI2 protocol > + based camera sensor data stream. > +- ISP: handles the image data streams from the MIPI CSIRx Controller. Maybe you've done this elsewhere but, it would be worthwhile describing what the ISP does. Debayering ? 3As ? Just raw dumps ? > +- VIN(Video In): a top-level module, is responsible for controlling power > + and clocks to other modules, dumps the input data to memory or transfers the > + input data to ISP. > + > + > +Topology > +-------- > + > +The media controller pipeline graph is as follows: > + > +.. _starfive_camss_graph: > + > +.. kernel-figure:: starfive_camss_graph.dot > + :alt: starfive_camss_graph.dot > + :align: center > + > +The driver has 2 video devices: > + > +- stf_vin0_wr_video0: capture device for images directly from the VIN module. OK so you can get the raw images. > +- stf_vin0_isp0_video1: capture device for images without scaling. > + > +The driver has 3 subdevices: > + > +- stf_isp0: is responsible for all the isp operations. > +- stf_vin0_wr: used to dump RAW images to memory. > +- stf_vin0_isp0: used to capture images for the stf_vin0_isp0_video1 device. But what is being output here ? RGB, jpeg, YUV ? It would be worth adding a few bits of text to describe that so you don't have to dive into code to understand it. --- bod
On Wed, Jul 26, 2023 at 12:26:18PM +0100, Bryan O'Donoghue wrote:
> On 19/06/2023 12:28, Jack Zhu wrote:
> > Add starfive_camss.rst file that documents the Starfive Camera
> > Subsystem driver which is used for handing image sensor data.
> >
> > Signed-off-by: Jack Zhu <jack.zhu@starfivetech.com>
> > ---
> > .../admin-guide/media/starfive_camss.rst | 57 +++++++++++++++++++
> > .../media/starfive_camss_graph.dot | 16 ++++++
> > .../admin-guide/media/v4l-drivers.rst | 1 +
> > MAINTAINERS | 1 +
> > 4 files changed, 75 insertions(+)
> > create mode 100644 Documentation/admin-guide/media/starfive_camss.rst
> > create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot
> >
> > diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst
> > new file mode 100644
> > index 000000000000..a6378849384f
> > --- /dev/null
> > +++ b/Documentation/admin-guide/media/starfive_camss.rst
> > @@ -0,0 +1,57 @@
> > +.. SPDX-License-Identifier: GPL-2.0
> > +
> > +.. include:: <isonum.txt>
> > +
> > +================================
> > +Starfive Camera Subsystem driver
> > +================================
> > +
> > +Introduction
> > +------------
> > +
> > +This file documents the driver for the Starfive Camera Subsystem found on
> > +Starfive JH7110 SoC. The driver is located under drivers/media/platform/
> > +starfive.
> > +
> > +The driver implements V4L2, Media controller and v4l2_subdev interfaces.
> > +Camera sensor using V4L2 subdev interface in the kernel is supported.
> > +
> > +The driver has been successfully used on the Gstreamer 1.18.5 with
> > +v4l2src plugin.
> > +
> > +
> > +Starfive Camera Subsystem hardware
> > +----------------------------------
> > +
> > +The Starfive Camera Subsystem hardware consists of:
> > +
> > +- MIPI DPHY Receiver: receives mipi data from a MIPI camera sensor.
>
> Feels like a terribe nit-pick but you have "mipi" and "MIPI" here. I'd
> be consistent with one - recommend MIPI throughout your documentation.
I'd drop the "mipi" and just write "data", not "MIPI data".
> > +- MIPI CSIRx Controller: is responsible for handling and decoding CSI2 protocol
It's "CSI-2", not "CSI2".
> > + based camera sensor data stream.
> > +- ISP: handles the image data streams from the MIPI CSIRx Controller.
>
> Maybe you've done this elsewhere but, it would be worthwhile describing
> what the ISP does. Debayering ? 3As ? Just raw dumps ?
>
> > +- VIN(Video In): a top-level module, is responsible for controlling power
> > + and clocks to other modules, dumps the input data to memory or transfers the
> > + input data to ISP.
A block diagram could be nice. Let me give it a try, based on my
understanding:
Starfive Camera Subsystem hardware
----------------------------------
The Starfive Camera Subsystem hardware consists of::
+---------------------------------------------+
| VIN |
+--------+ | +-------+ +-------+ +-----+ |
| Camera |----->| D-PHY |-->| CSI-2 |-+->| DMA | |
| Sensor | | | RX | | RX | | +-----+ |
+--------+ | +-------+ +-------+ | |
| | +-----+ +-----+ |
| \->| ISP |-->| DMA | |
| +-----+ +-----+ |
+---------------------------------------------+
- VIN (Video IN): The top-level module, responsible for controlling
power and clocks, and routing data between modules.
- D-PHY RX: The MIPI D-PHY receiver, receiving data from a MIPI CSI-2
camera sensor.
- CSI-2 RX: The MIPI CSI-2 receiver controller, responsible for decoding
the CSI-2 protocol.
- ISP: The ISP, processing raw Bayer data from the CSI-2 receiver and
producing YUV frames.
I haven't tried compiling this, so please fix formatting if anything is
wrong (and of course feel free to adapt the diagram and text).
> > +
> > +
> > +Topology
> > +--------
> > +
> > +The media controller pipeline graph is as follows:
> > +
> > +.. _starfive_camss_graph:
> > +
> > +.. kernel-figure:: starfive_camss_graph.dot
> > + :alt: starfive_camss_graph.dot
> > + :align: center
> > +
> > +The driver has 2 video devices:
> > +
> > +- stf_vin0_wr_video0: capture device for images directly from the VIN module.
>
> OK so you can get the raw images.
>
> > +- stf_vin0_isp0_video1: capture device for images without scaling.
> > +
> > +The driver has 3 subdevices:
> > +
> > +- stf_isp0: is responsible for all the isp operations.
> > +- stf_vin0_wr: used to dump RAW images to memory.
> > +- stf_vin0_isp0: used to capture images for the stf_vin0_isp0_video1 device.
>
> But what is being output here ? RGB, jpeg, YUV ?
> It would be worth adding a few bits of text to describe that so you
> don't have to dive into code to understand it.
--
Regards,
Laurent Pinchart
Hi Laurent, Thank you for your comments. On 2023/7/27 18:23, Laurent Pinchart wrote: > On Wed, Jul 26, 2023 at 12:26:18PM +0100, Bryan O'Donoghue wrote: >> On 19/06/2023 12:28, Jack Zhu wrote: >> > Add starfive_camss.rst file that documents the Starfive Camera >> > Subsystem driver which is used for handing image sensor data. >> > >> > Signed-off-by: Jack Zhu <jack.zhu@starfivetech.com> >> > --- >> > .../admin-guide/media/starfive_camss.rst | 57 +++++++++++++++++++ >> > .../media/starfive_camss_graph.dot | 16 ++++++ >> > .../admin-guide/media/v4l-drivers.rst | 1 + >> > MAINTAINERS | 1 + >> > 4 files changed, 75 insertions(+) >> > create mode 100644 Documentation/admin-guide/media/starfive_camss.rst >> > create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot >> > >> > diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst >> > new file mode 100644 >> > index 000000000000..a6378849384f >> > --- /dev/null >> > +++ b/Documentation/admin-guide/media/starfive_camss.rst >> > @@ -0,0 +1,57 @@ >> > +.. SPDX-License-Identifier: GPL-2.0 >> > + >> > +.. include:: <isonum.txt> >> > + >> > +================================ >> > +Starfive Camera Subsystem driver >> > +================================ >> > + >> > +Introduction >> > +------------ >> > + >> > +This file documents the driver for the Starfive Camera Subsystem found on >> > +Starfive JH7110 SoC. The driver is located under drivers/media/platform/ >> > +starfive. >> > + >> > +The driver implements V4L2, Media controller and v4l2_subdev interfaces. >> > +Camera sensor using V4L2 subdev interface in the kernel is supported. >> > + >> > +The driver has been successfully used on the Gstreamer 1.18.5 with >> > +v4l2src plugin. >> > + >> > + >> > +Starfive Camera Subsystem hardware >> > +---------------------------------- >> > + >> > +The Starfive Camera Subsystem hardware consists of: >> > + >> > +- MIPI DPHY Receiver: receives mipi data from a MIPI camera sensor. >> >> Feels like a terribe nit-pick but you have "mipi" and "MIPI" here. I'd >> be consistent with one - recommend MIPI throughout your documentation. > > I'd drop the "mipi" and just write "data", not "MIPI data". > Ok, I will modify it. >> > +- MIPI CSIRx Controller: is responsible for handling and decoding CSI2 protocol > > It's "CSI-2", not "CSI2". Ok, I will modify it. > >> > + based camera sensor data stream. >> > +- ISP: handles the image data streams from the MIPI CSIRx Controller. >> >> Maybe you've done this elsewhere but, it would be worthwhile describing >> what the ISP does. Debayering ? 3As ? Just raw dumps ? >> >> > +- VIN(Video In): a top-level module, is responsible for controlling power >> > + and clocks to other modules, dumps the input data to memory or transfers the >> > + input data to ISP. > > A block diagram could be nice. Let me give it a try, based on my > understanding: > > Starfive Camera Subsystem hardware > ---------------------------------- > > The Starfive Camera Subsystem hardware consists of:: > > +---------------------------------------------+ > | VIN | > +--------+ | +-------+ +-------+ +-----+ | > | Camera |----->| D-PHY |-->| CSI-2 |-+->| DMA | | > | Sensor | | | RX | | RX | | +-----+ | > +--------+ | +-------+ +-------+ | | > | | +-----+ +-----+ | > | \->| ISP |-->| DMA | | > | +-----+ +-----+ | > +---------------------------------------------+ > > - VIN (Video IN): The top-level module, responsible for controlling > power and clocks, and routing data between modules. > > - D-PHY RX: The MIPI D-PHY receiver, receiving data from a MIPI CSI-2 > camera sensor. > > - CSI-2 RX: The MIPI CSI-2 receiver controller, responsible for decoding > the CSI-2 protocol. > > - ISP: The ISP, processing raw Bayer data from the CSI-2 receiver and > producing YUV frames. > > > I haven't tried compiling this, so please fix formatting if anything is > wrong (and of course feel free to adapt the diagram and text). > Ok, I will add a block diagram. >> > + >> > + >> > +Topology >> > +-------- >> > + >> > +The media controller pipeline graph is as follows: >> > + >> > +.. _starfive_camss_graph: >> > + >> > +.. kernel-figure:: starfive_camss_graph.dot >> > + :alt: starfive_camss_graph.dot >> > + :align: center >> > + >> > +The driver has 2 video devices: >> > + >> > +- stf_vin0_wr_video0: capture device for images directly from the VIN module. >> >> OK so you can get the raw images. >> >> > +- stf_vin0_isp0_video1: capture device for images without scaling. >> > + >> > +The driver has 3 subdevices: >> > + >> > +- stf_isp0: is responsible for all the isp operations. >> > +- stf_vin0_wr: used to dump RAW images to memory. >> > +- stf_vin0_isp0: used to capture images for the stf_vin0_isp0_video1 device. >> >> But what is being output here ? RGB, jpeg, YUV ? >> It would be worth adding a few bits of text to describe that so you >> don't have to dive into code to understand it. > -- Regards, Jack Zhu
Hi Bryan, Thank you for your comments. On 2023/7/26 19:26, Bryan O'Donoghue wrote: > On 19/06/2023 12:28, Jack Zhu wrote: >> Add starfive_camss.rst file that documents the Starfive Camera >> Subsystem driver which is used for handing image sensor data. >> >> Signed-off-by: Jack Zhu <jack.zhu@starfivetech.com> >> --- >> .../admin-guide/media/starfive_camss.rst | 57 +++++++++++++++++++ >> .../media/starfive_camss_graph.dot | 16 ++++++ >> .../admin-guide/media/v4l-drivers.rst | 1 + >> MAINTAINERS | 1 + >> 4 files changed, 75 insertions(+) >> create mode 100644 Documentation/admin-guide/media/starfive_camss.rst >> create mode 100644 Documentation/admin-guide/media/starfive_camss_graph.dot >> >> diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst >> new file mode 100644 >> index 000000000000..a6378849384f >> --- /dev/null >> +++ b/Documentation/admin-guide/media/starfive_camss.rst >> @@ -0,0 +1,57 @@ >> +.. SPDX-License-Identifier: GPL-2.0 >> + >> +.. include:: <isonum.txt> >> + >> +================================ >> +Starfive Camera Subsystem driver >> +================================ >> + >> +Introduction >> +------------ >> + >> +This file documents the driver for the Starfive Camera Subsystem found on >> +Starfive JH7110 SoC. The driver is located under drivers/media/platform/ >> +starfive. >> + >> +The driver implements V4L2, Media controller and v4l2_subdev interfaces. >> +Camera sensor using V4L2 subdev interface in the kernel is supported. >> + >> +The driver has been successfully used on the Gstreamer 1.18.5 with >> +v4l2src plugin. >> + >> + >> +Starfive Camera Subsystem hardware >> +---------------------------------- >> + >> +The Starfive Camera Subsystem hardware consists of: >> + >> +- MIPI DPHY Receiver: receives mipi data from a MIPI camera sensor. > > Feels like a terribe nit-pick but you have "mipi" and "MIPI" here. I'd be consistent with one - recommend MIPI throughout your documentation. > Okay, I will keep a consistent description. >> +- MIPI CSIRx Controller: is responsible for handling and decoding CSI2 protocol >> + based camera sensor data stream. >> +- ISP: handles the image data streams from the MIPI CSIRx Controller. > > Maybe you've done this elsewhere but, it would be worthwhile describing what the ISP does. Debayering ? 3As ? Just raw dumps ? > Ok, I will add more descriptions in the next version. >> +- VIN(Video In): a top-level module, is responsible for controlling power >> + and clocks to other modules, dumps the input data to memory or transfers the >> + input data to ISP. >> + >> + >> +Topology >> +-------- >> + >> +The media controller pipeline graph is as follows: >> + >> +.. _starfive_camss_graph: >> + >> +.. kernel-figure:: starfive_camss_graph.dot >> + :alt: starfive_camss_graph.dot >> + :align: center >> + >> +The driver has 2 video devices: >> + >> +- stf_vin0_wr_video0: capture device for images directly from the VIN module. > > OK so you can get the raw images. > >> +- stf_vin0_isp0_video1: capture device for images without scaling. >> + >> +The driver has 3 subdevices: >> + >> +- stf_isp0: is responsible for all the isp operations. >> +- stf_vin0_wr: used to dump RAW images to memory. >> +- stf_vin0_isp0: used to capture images for the stf_vin0_isp0_video1 device. > > But what is being output here ? RGB, jpeg, YUV ? > It would be worth adding a few bits of text to describe that so you don't have to dive into code to understand it. > Ok, I will add more descriptions. > --- > bod -- Regards, Jack Zhu
© 2016 - 2026 Red Hat, Inc.