[v3] media: imx8: add camera support

[PATCH v3 04/12] media: nxp: imx8-isi: Allow num_sources to be greater than num_sink

Posted by Frank Li 12 months ago

Allow num_sources (drvdata: num_channels) to be greater than num_sink
(drvdata: num_ports + 1).

ISI support stream multiplexing, such as differentiates multiple cameras
from a single 2-lane MIPI input, or duplicates input stream into multiple
outputs. So num_channels may be greater than num_ports at some platform.

Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
change from v1 to v3
- none
---
 drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
index 93a55c97cd173..ba5b2d3617a23 100644
--- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
+++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
@@ -188,11 +188,11 @@ static int mxc_isi_crossbar_init_state(struct v4l2_subdev *sd,
 	 * Create a 1:1 mapping between pixel link inputs and outputs to
 	 * pipelines by default.
 	 */
-	routes = kcalloc(xbar->num_sources, sizeof(*routes), GFP_KERNEL);
+	routes = kcalloc(xbar->num_sinks - 1, sizeof(*routes), GFP_KERNEL);
 	if (!routes)
 		return -ENOMEM;
 
-	for (i = 0; i < xbar->num_sources; ++i) {
+	for (i = 0; i < xbar->num_sinks - 1; ++i) {
 		struct v4l2_subdev_route *route = &routes[i];
 
 		route->sink_pad = i;
@@ -200,7 +200,7 @@ static int mxc_isi_crossbar_init_state(struct v4l2_subdev *sd,
 		route->flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE;
 	}
 
-	routing.num_routes = xbar->num_sources;
+	routing.num_routes = xbar->num_sinks - 1;
 	routing.routes = routes;
 
 	ret = __mxc_isi_crossbar_set_routing(sd, state, &routing);
@@ -453,7 +453,7 @@ int mxc_isi_crossbar_init(struct mxc_isi_dev *isi)
 	 * the memory input.
 	 */
 	xbar->num_sinks = isi->pdata->num_ports + 1;
-	xbar->num_sources = isi->pdata->num_ports;
+	xbar->num_sources = isi->pdata->num_channels;
 	num_pads = xbar->num_sinks + xbar->num_sources;
 
 	xbar->pads = kcalloc(num_pads, sizeof(*xbar->pads), GFP_KERNEL);

-- 
2.34.1

Re: [PATCH v3 04/12] media: nxp: imx8-isi: Allow num_sources to be greater than num_sink

Posted by Adam Ford 10 months, 2 weeks ago

On Mon, Feb 10, 2025 at 3:01 PM Frank Li <Frank.Li@nxp.com> wrote:
>
> Allow num_sources (drvdata: num_channels) to be greater than num_sink
> (drvdata: num_ports + 1).
>
> ISI support stream multiplexing, such as differentiates multiple cameras
> from a single 2-lane MIPI input, or duplicates input stream into multiple
> outputs. So num_channels may be greater than num_ports at some platform.

Can you tell me which platforms support this?   Is this through
virtual channels, or do you physically connect lanes 1 and 2 to one
camera and lanes 3 and 4 to another?

adam
>
> Signed-off-by: Frank Li <Frank.Li@nxp.com>
> ---
> change from v1 to v3
> - none
> ---
>  drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> index 93a55c97cd173..ba5b2d3617a23 100644
> --- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> +++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> @@ -188,11 +188,11 @@ static int mxc_isi_crossbar_init_state(struct v4l2_subdev *sd,
>          * Create a 1:1 mapping between pixel link inputs and outputs to
>          * pipelines by default.
>          */
> -       routes = kcalloc(xbar->num_sources, sizeof(*routes), GFP_KERNEL);
> +       routes = kcalloc(xbar->num_sinks - 1, sizeof(*routes), GFP_KERNEL);
>         if (!routes)
>                 return -ENOMEM;
>
> -       for (i = 0; i < xbar->num_sources; ++i) {
> +       for (i = 0; i < xbar->num_sinks - 1; ++i) {
>                 struct v4l2_subdev_route *route = &routes[i];
>
>                 route->sink_pad = i;
> @@ -200,7 +200,7 @@ static int mxc_isi_crossbar_init_state(struct v4l2_subdev *sd,
>                 route->flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE;
>         }
>
> -       routing.num_routes = xbar->num_sources;
> +       routing.num_routes = xbar->num_sinks - 1;
>         routing.routes = routes;
>
>         ret = __mxc_isi_crossbar_set_routing(sd, state, &routing);
> @@ -453,7 +453,7 @@ int mxc_isi_crossbar_init(struct mxc_isi_dev *isi)
>          * the memory input.
>          */
>         xbar->num_sinks = isi->pdata->num_ports + 1;
> -       xbar->num_sources = isi->pdata->num_ports;
> +       xbar->num_sources = isi->pdata->num_channels;
>         num_pads = xbar->num_sinks + xbar->num_sources;
>
>         xbar->pads = kcalloc(num_pads, sizeof(*xbar->pads), GFP_KERNEL);
>
> --
> 2.34.1
>
>

Re: [PATCH v3 04/12] media: nxp: imx8-isi: Allow num_sources to be greater than num_sink

Posted by Frank Li 10 months, 2 weeks ago

On Thu, Mar 27, 2025 at 03:02:29PM -0500, Adam Ford wrote:
> On Mon, Feb 10, 2025 at 3:01 PM Frank Li <Frank.Li@nxp.com> wrote:
> >
> > Allow num_sources (drvdata: num_channels) to be greater than num_sink
> > (drvdata: num_ports + 1).
> >
> > ISI support stream multiplexing, such as differentiates multiple cameras
> > from a single 2-lane MIPI input, or duplicates input stream into multiple
> > outputs. So num_channels may be greater than num_ports at some platform.
>
> Can you tell me which platforms support this?

iMX8QM.

>  Is this through
> virtual channels, or do you physically connect lanes 1 and 2 to one
> camera and lanes 3 and 4 to another?

Not like that.  for example, camera 0 as input0

channels 0 can covert input0 to RGB format to memory 1
channels 1 can scale down input0 to to memory 2

I think it most likely use for channel0 to preview image, channel1 to save
input to file on storage.

Frank
>
> adam
> >
> > Signed-off-by: Frank Li <Frank.Li@nxp.com>
> > ---
> > change from v1 to v3
> > - none
> > ---
> >  drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c | 8 ++++----
> >  1 file changed, 4 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> > index 93a55c97cd173..ba5b2d3617a23 100644
> > --- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> > +++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> > @@ -188,11 +188,11 @@ static int mxc_isi_crossbar_init_state(struct v4l2_subdev *sd,
> >          * Create a 1:1 mapping between pixel link inputs and outputs to
> >          * pipelines by default.
> >          */
> > -       routes = kcalloc(xbar->num_sources, sizeof(*routes), GFP_KERNEL);
> > +       routes = kcalloc(xbar->num_sinks - 1, sizeof(*routes), GFP_KERNEL);
> >         if (!routes)
> >                 return -ENOMEM;
> >
> > -       for (i = 0; i < xbar->num_sources; ++i) {
> > +       for (i = 0; i < xbar->num_sinks - 1; ++i) {
> >                 struct v4l2_subdev_route *route = &routes[i];
> >
> >                 route->sink_pad = i;
> > @@ -200,7 +200,7 @@ static int mxc_isi_crossbar_init_state(struct v4l2_subdev *sd,
> >                 route->flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE;
> >         }
> >
> > -       routing.num_routes = xbar->num_sources;
> > +       routing.num_routes = xbar->num_sinks - 1;
> >         routing.routes = routes;
> >
> >         ret = __mxc_isi_crossbar_set_routing(sd, state, &routing);
> > @@ -453,7 +453,7 @@ int mxc_isi_crossbar_init(struct mxc_isi_dev *isi)
> >          * the memory input.
> >          */
> >         xbar->num_sinks = isi->pdata->num_ports + 1;
> > -       xbar->num_sources = isi->pdata->num_ports;
> > +       xbar->num_sources = isi->pdata->num_channels;
> >         num_pads = xbar->num_sinks + xbar->num_sources;
> >
> >         xbar->pads = kcalloc(num_pads, sizeof(*xbar->pads), GFP_KERNEL);
> >
> > --
> > 2.34.1
> >
> >

Re: [PATCH v3 04/12] media: nxp: imx8-isi: Allow num_sources to be greater than num_sink

Posted by Laurent Pinchart 10 months, 2 weeks ago

Hi Frank,

Thank you for the patch.

On Mon, Feb 10, 2025 at 03:59:23PM -0500, Frank Li wrote:
> Allow num_sources (drvdata: num_channels) to be greater than num_sink
> (drvdata: num_ports + 1).
> 
> ISI support stream multiplexing, such as differentiates multiple cameras
> from a single 2-lane MIPI input, or duplicates input stream into multiple
> outputs. So num_channels may be greater than num_ports at some platform.
> 
> Signed-off-by: Frank Li <Frank.Li@nxp.com>
> ---
> change from v1 to v3
> - none
> ---
>  drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> index 93a55c97cd173..ba5b2d3617a23 100644
> --- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> +++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> @@ -188,11 +188,11 @@ static int mxc_isi_crossbar_init_state(struct v4l2_subdev *sd,
>  	 * Create a 1:1 mapping between pixel link inputs and outputs to
>  	 * pipelines by default.
>  	 */

Move

	routing.num_routes = xbar->num_sinks - 1;

here, and use routing.num_routes below to replace xbar->num_sinks - 1.

This assumes that there will always be as many or more pipelines than
pixel links. It's true for all the currently supported SoCs and the ones
you add in this series, but maybe we could make it safer with

	routing.num_routes = min(xbar->num_sinks - 1, xbar->num_sources);

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> -	routes = kcalloc(xbar->num_sources, sizeof(*routes), GFP_KERNEL);
> +	routes = kcalloc(xbar->num_sinks - 1, sizeof(*routes), GFP_KERNEL);
>  	if (!routes)
>  		return -ENOMEM;
>  
> -	for (i = 0; i < xbar->num_sources; ++i) {
> +	for (i = 0; i < xbar->num_sinks - 1; ++i) {
>  		struct v4l2_subdev_route *route = &routes[i];
>  
>  		route->sink_pad = i;
> @@ -200,7 +200,7 @@ static int mxc_isi_crossbar_init_state(struct v4l2_subdev *sd,
>  		route->flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE;
>  	}
>  
> -	routing.num_routes = xbar->num_sources;
> +	routing.num_routes = xbar->num_sinks - 1;
>  	routing.routes = routes;
>  
>  	ret = __mxc_isi_crossbar_set_routing(sd, state, &routing);
> @@ -453,7 +453,7 @@ int mxc_isi_crossbar_init(struct mxc_isi_dev *isi)
>  	 * the memory input.
>  	 */
>  	xbar->num_sinks = isi->pdata->num_ports + 1;
> -	xbar->num_sources = isi->pdata->num_ports;
> +	xbar->num_sources = isi->pdata->num_channels;
>  	num_pads = xbar->num_sinks + xbar->num_sources;
>  
>  	xbar->pads = kcalloc(num_pads, sizeof(*xbar->pads), GFP_KERNEL);

-- 
Regards,

Laurent Pinchart