[PATCH v3 00/15] media: rcar: Streams support

Tomi Valkeinen posted 15 patches 6 months, 2 weeks ago
drivers/media/platform/renesas/rcar-csi2.c      | 426 +++++++++++++++++-------
drivers/media/platform/renesas/rcar-isp/csisp.c | 228 ++++++++++---
2 files changed, 479 insertions(+), 175 deletions(-)
[PATCH v3 00/15] media: rcar: Streams support
Posted by Tomi Valkeinen 6 months, 2 weeks ago
Add streams support to Renesas rcar platform driver.

The series attempts to keep compatibility with the current upstream.
However, in upstream there's some kind of custom multi-stream support
implemented to the rcar driver, which breaks at patch "media: rcar-csi2:
Simplify rcsi2_calc_mbps()".

The behavior should not change when using a single stream.

Testing is problematic, as the only way currently for me to get multiple
streams is by using the GMSL2 deserializer add-on board with GMSL2
serializers. These are not supported in upstream. If someone has the
hardware and wants to test, I can share the very-WIP branch that
contains the missing pieces.

 Tomi

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
Changes in v3:
- Rebased on top of latest linux-media
- Dropped dependencies which are already in linux-media (only remaining
  dependency is v4l2_subdev_get_frame_desc_passthrough)
- Tested on white-hawk board, using the staging deser TPG
- Also tested in a WIP branch for GMSL2 (two video streams)
- Link to v2: https://lore.kernel.org/r/20250326-rcar-streams-v2-0-d0d7002c641f@ideasonboard.com

Changes in v2:
- Rebased on top of latest upstream, and updated the dependencies to
  match the latest serieses sent.
- Add new patch "media: rcar-csi2: Use the pad version of v4l2_get_link_freq()"
- Drop "media: rcar-csi2: Fix typo" (it was not a typo)
- Update the code in calc_mbps(). The previous method relied on
  V4L2_CID_LINK_FREQ, but that's not available if the link-freq is
  provided via get_mbus_config().
- Dropped dependencies to Niklas' old series which doesn't apply
  cleanly. It's needed for multi-stream, but not for the current
  upstream which only has a single stream use case.
- Link to v1: https://lore.kernel.org/r/20250219-rcar-streams-v1-0-f1b93e370aab@ideasonboard.com

---
Tomi Valkeinen (15):
      media: rcar-csi2: Use the pad version of v4l2_get_link_freq()
      media: rcar-isp: Improve ISPPROCMODE_DT_PROC_MODE_VC
      media: rcar-isp: Move {enable|disable}_streams() calls
      media: rcar-csi2: Move {enable|disable}_streams() calls
      media: rcar-csi2: Move rcar2_calc_mbps()
      media: rcar-csi2: Simplify rcsi2_calc_mbps()
      media: rcar-csi2: Optimize rcsi2_calc_mbps()
      media: rcar-csi2: Switch to Streams API
      media: rcar-isp: Switch to Streams API
      media: rcar-csi2: Add .get_frame_desc op
      media: rcar-isp: Call get_frame_desc to find out VC & DT
      media: rcar-csi2: Add more stream support to rcsi2_calc_mbps()
      media: rcar-csi2: Call get_frame_desc to find out VC & DT (Gen3)
      media: rcar-csi2: Add full streams support
      media: rcar-isp: Add full streams support

 drivers/media/platform/renesas/rcar-csi2.c      | 426 +++++++++++++++++-------
 drivers/media/platform/renesas/rcar-isp/csisp.c | 228 ++++++++++---
 2 files changed, 479 insertions(+), 175 deletions(-)
---
base-commit: 5e1ff2314797bf53636468a97719a8222deca9ae
change-id: 20250219-rcar-streams-1fdea8860e5e
prerequisite-change-id: 20250218-frame-desc-passthrough-66805e413974:v4
prerequisite-patch-id: bce4a915a29a64f88ed1bb600c08df37d2ba20c6
prerequisite-patch-id: 69b75e7dad9ced905cb39a72f18bebbf3e8f998a
prerequisite-patch-id: 58463f6944c76acd6cf203b14a2836cdb0db2461

Best regards,
-- 
Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
Re: [PATCH v3 00/15] media: rcar: Streams support
Posted by Niklas Söderlund 6 months, 1 week ago
Hi Tomi,

On 2025-05-30 16:50:29 +0300, Tomi Valkeinen wrote:
> Add streams support to Renesas rcar platform driver.
> 
> The series attempts to keep compatibility with the current upstream.
> However, in upstream there's some kind of custom multi-stream support
> implemented to the rcar driver, which breaks at patch "media: rcar-csi2:
> Simplify rcsi2_calc_mbps()".
> 
> The behavior should not change when using a single stream.
> 
> Testing is problematic, as the only way currently for me to get multiple
> streams is by using the GMSL2 deserializer add-on board with GMSL2
> serializers. These are not supported in upstream. If someone has the
> hardware and wants to test, I can share the very-WIP branch that
> contains the missing pieces.

I'm happy to see this new version of this work, it looks so clean! I'm 
equally happy to see all the hard-coded assumptions we needed in the 
pipeline to emulate streams before being replaced with core 
functionality!

I have tested this with the single stream use-cases I have had before on 
Gen2, Gen3 and Gen4 and they all seem to function as before, nice work!

Tested-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

As this is a rather large series do you think it would make sens to try 
and get some of the preparation/clean up patches merged before the new 
streams support?

> 
>  Tomi
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
> ---
> Changes in v3:
> - Rebased on top of latest linux-media
> - Dropped dependencies which are already in linux-media (only remaining
>   dependency is v4l2_subdev_get_frame_desc_passthrough)
> - Tested on white-hawk board, using the staging deser TPG
> - Also tested in a WIP branch for GMSL2 (two video streams)
> - Link to v2: https://lore.kernel.org/r/20250326-rcar-streams-v2-0-d0d7002c641f@ideasonboard.com
> 
> Changes in v2:
> - Rebased on top of latest upstream, and updated the dependencies to
>   match the latest serieses sent.
> - Add new patch "media: rcar-csi2: Use the pad version of v4l2_get_link_freq()"
> - Drop "media: rcar-csi2: Fix typo" (it was not a typo)
> - Update the code in calc_mbps(). The previous method relied on
>   V4L2_CID_LINK_FREQ, but that's not available if the link-freq is
>   provided via get_mbus_config().
> - Dropped dependencies to Niklas' old series which doesn't apply
>   cleanly. It's needed for multi-stream, but not for the current
>   upstream which only has a single stream use case.
> - Link to v1: https://lore.kernel.org/r/20250219-rcar-streams-v1-0-f1b93e370aab@ideasonboard.com
> 
> ---
> Tomi Valkeinen (15):
>       media: rcar-csi2: Use the pad version of v4l2_get_link_freq()
>       media: rcar-isp: Improve ISPPROCMODE_DT_PROC_MODE_VC
>       media: rcar-isp: Move {enable|disable}_streams() calls
>       media: rcar-csi2: Move {enable|disable}_streams() calls
>       media: rcar-csi2: Move rcar2_calc_mbps()
>       media: rcar-csi2: Simplify rcsi2_calc_mbps()
>       media: rcar-csi2: Optimize rcsi2_calc_mbps()
>       media: rcar-csi2: Switch to Streams API
>       media: rcar-isp: Switch to Streams API
>       media: rcar-csi2: Add .get_frame_desc op
>       media: rcar-isp: Call get_frame_desc to find out VC & DT
>       media: rcar-csi2: Add more stream support to rcsi2_calc_mbps()
>       media: rcar-csi2: Call get_frame_desc to find out VC & DT (Gen3)
>       media: rcar-csi2: Add full streams support
>       media: rcar-isp: Add full streams support
> 
>  drivers/media/platform/renesas/rcar-csi2.c      | 426 +++++++++++++++++-------
>  drivers/media/platform/renesas/rcar-isp/csisp.c | 228 ++++++++++---
>  2 files changed, 479 insertions(+), 175 deletions(-)
> ---
> base-commit: 5e1ff2314797bf53636468a97719a8222deca9ae
> change-id: 20250219-rcar-streams-1fdea8860e5e
> prerequisite-change-id: 20250218-frame-desc-passthrough-66805e413974:v4
> prerequisite-patch-id: bce4a915a29a64f88ed1bb600c08df37d2ba20c6
> prerequisite-patch-id: 69b75e7dad9ced905cb39a72f18bebbf3e8f998a
> prerequisite-patch-id: 58463f6944c76acd6cf203b14a2836cdb0db2461
> 
> Best regards,
> -- 
> Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
> 

-- 
Kind Regards,
Niklas Söderlund