[v2] media: uvcvideo: Add support for orientation and rotation.

[PATCH v2 12/12] media: uvcvideo: Do not create MC entities for virtual entities

Posted by Ricardo Ribalda 6 months, 2 weeks ago

Neither the GPIO nor the SWENTITY entities form part of the device
pipeline. We just create them to hold emulated uvc controls.

When the device initializes, a warning is thrown by the v4l2 core:
uvcvideo 1-1:1.0: Entity type for entity SWENTITY was not initialized!

There are no entity function that matches what we are doing here, and
it does not make to much sense to create a function for entities that
do not really exist.

Do not create MC entities for them and pretend nothing happened here.

Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
---
 drivers/media/usb/uvc/uvc_entity.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/media/usb/uvc/uvc_entity.c b/drivers/media/usb/uvc/uvc_entity.c
index d1a652ef35ec34801bd39a5124b834edf838a79e..2dbeb4ab0c4c8cc763ff2dcd2d836a50f3c6a040 100644
--- a/drivers/media/usb/uvc/uvc_entity.c
+++ b/drivers/media/usb/uvc/uvc_entity.c
@@ -72,6 +72,16 @@ static int uvc_mc_init_entity(struct uvc_video_chain *chain,
 {
 	int ret;
 
+	/*
+	 * Do not initialize virtual entities, they do not really exist
+	 * and are not connected to any other entities.
+	 */
+	switch (UVC_ENTITY_TYPE(entity)) {
+	case UVC_EXT_GPIO_UNIT:
+	case UVC_SWENTITY_UNIT:
+		return 0;
+	}
+
 	if (UVC_ENTITY_TYPE(entity) != UVC_TT_STREAMING) {
 		u32 function;
 

-- 
2.50.0.rc0.642.g800a2b2222-goog

Re: [PATCH v2 12/12] media: uvcvideo: Do not create MC entities for virtual entities

Posted by Laurent Pinchart 5 months, 3 weeks ago

Hi Ricardo,

Thank you for the patch.

I would use "software entities" and not "virtual entities" in the
subject line and everywhere else, as those entities are not virtual.

On Thu, Jun 05, 2025 at 05:53:05PM +0000, Ricardo Ribalda wrote:
> Neither the GPIO nor the SWENTITY entities form part of the device
> pipeline. We just create them to hold emulated uvc controls.
> 
> When the device initializes, a warning is thrown by the v4l2 core:
> uvcvideo 1-1:1.0: Entity type for entity SWENTITY was not initialized!
> 
> There are no entity function that matches what we are doing here, and
> it does not make to much sense to create a function for entities that
> do not really exist.

I don't agree with this. The purpose of reporting entities to userspace
through the MC API is to let application enumerate what entities a
device contains. Being able to enumerate software entities seems as
useful as being able to enumerate hardware entities.

> Do not create MC entities for them and pretend nothing happened here.
> 
> Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_entity.c | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/drivers/media/usb/uvc/uvc_entity.c b/drivers/media/usb/uvc/uvc_entity.c
> index d1a652ef35ec34801bd39a5124b834edf838a79e..2dbeb4ab0c4c8cc763ff2dcd2d836a50f3c6a040 100644
> --- a/drivers/media/usb/uvc/uvc_entity.c
> +++ b/drivers/media/usb/uvc/uvc_entity.c
> @@ -72,6 +72,16 @@ static int uvc_mc_init_entity(struct uvc_video_chain *chain,
>  {
>  	int ret;
>  
> +	/*
> +	 * Do not initialize virtual entities, they do not really exist
> +	 * and are not connected to any other entities.
> +	 */
> +	switch (UVC_ENTITY_TYPE(entity)) {
> +	case UVC_EXT_GPIO_UNIT:
> +	case UVC_SWENTITY_UNIT:
> +		return 0;
> +	}
> +
>  	if (UVC_ENTITY_TYPE(entity) != UVC_TT_STREAMING) {
>  		u32 function;
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH v2 12/12] media: uvcvideo: Do not create MC entities for virtual entities

Posted by Ricardo Ribalda 5 months, 2 weeks ago

Hi Laurent and Hans


On Sun, 29 Jun 2025 at 20:06, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi Ricardo,
>
> Thank you for the patch.
>
> I would use "software entities" and not "virtual entities" in the
> subject line and everywhere else, as those entities are not virtual.
>
> On Thu, Jun 05, 2025 at 05:53:05PM +0000, Ricardo Ribalda wrote:
> > Neither the GPIO nor the SWENTITY entities form part of the device
> > pipeline. We just create them to hold emulated uvc controls.
> >
> > When the device initializes, a warning is thrown by the v4l2 core:
> > uvcvideo 1-1:1.0: Entity type for entity SWENTITY was not initialized!
> >
> > There are no entity function that matches what we are doing here, and
> > it does not make to much sense to create a function for entities that
> > do not really exist.
>
> I don't agree with this. The purpose of reporting entities to userspace
> through the MC API is to let application enumerate what entities a
> device contains. Being able to enumerate software entities seems as
> useful as being able to enumerate hardware entities.

What function shall we use in this case? Nothing here seems to match a
software entity
https://www.kernel.org/doc/html/latest/userspace-api/media/mediactl/media-types.html

Any suggestion for name?
Shall we just live with the warning in dmesg?

>
> > Do not create MC entities for them and pretend nothing happened here.
> >
> > Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
> > ---
> >  drivers/media/usb/uvc/uvc_entity.c | 10 ++++++++++
> >  1 file changed, 10 insertions(+)
> >
> > diff --git a/drivers/media/usb/uvc/uvc_entity.c b/drivers/media/usb/uvc/uvc_entity.c
> > index d1a652ef35ec34801bd39a5124b834edf838a79e..2dbeb4ab0c4c8cc763ff2dcd2d836a50f3c6a040 100644
> > --- a/drivers/media/usb/uvc/uvc_entity.c
> > +++ b/drivers/media/usb/uvc/uvc_entity.c
> > @@ -72,6 +72,16 @@ static int uvc_mc_init_entity(struct uvc_video_chain *chain,
> >  {
> >       int ret;
> >
> > +     /*
> > +      * Do not initialize virtual entities, they do not really exist
> > +      * and are not connected to any other entities.
> > +      */
> > +     switch (UVC_ENTITY_TYPE(entity)) {
> > +     case UVC_EXT_GPIO_UNIT:
> > +     case UVC_SWENTITY_UNIT:
> > +             return 0;
> > +     }
> > +
> >       if (UVC_ENTITY_TYPE(entity) != UVC_TT_STREAMING) {
> >               u32 function;
> >
>
> --
> Regards,
>
> Laurent Pinchart



-- 
Ricardo Ribalda

Re: [PATCH v2 12/12] media: uvcvideo: Do not create MC entities for virtual entities

Posted by Ricardo Ribalda 5 months, 1 week ago

Hi Laurent

On Tue, 1 Jul 2025 at 13:20, Ricardo Ribalda <ribalda@chromium.org> wrote:
>
> Hi Laurent and Hans
>
>
> On Sun, 29 Jun 2025 at 20:06, Laurent Pinchart
> <laurent.pinchart@ideasonboard.com> wrote:
> >
> > Hi Ricardo,
> >
> > Thank you for the patch.
> >
> > I would use "software entities" and not "virtual entities" in the
> > subject line and everywhere else, as those entities are not virtual.
> >
> > On Thu, Jun 05, 2025 at 05:53:05PM +0000, Ricardo Ribalda wrote:
> > > Neither the GPIO nor the SWENTITY entities form part of the device
> > > pipeline. We just create them to hold emulated uvc controls.
> > >
> > > When the device initializes, a warning is thrown by the v4l2 core:
> > > uvcvideo 1-1:1.0: Entity type for entity SWENTITY was not initialized!
> > >
> > > There are no entity function that matches what we are doing here, and
> > > it does not make to much sense to create a function for entities that
> > > do not really exist.
> >
> > I don't agree with this. The purpose of reporting entities to userspace
> > through the MC API is to let application enumerate what entities a
> > device contains. Being able to enumerate software entities seems as
> > useful as being able to enumerate hardware entities.
>
> What function shall we use in this case? Nothing here seems to match a
> software entity
> https://www.kernel.org/doc/html/latest/userspace-api/media/mediactl/media-types.html
>
> Any suggestion for name?
> Shall we just live with the warning in dmesg?

 I just realised that if/when we move to the control framework, the
software entity will be gone.... So to avoid introducing a uAPI change
that will be reverted later I think that we should keep this patch.

Regards
>
> >
> > > Do not create MC entities for them and pretend nothing happened here.
> > >
> > > Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
> > > ---
> > >  drivers/media/usb/uvc/uvc_entity.c | 10 ++++++++++
> > >  1 file changed, 10 insertions(+)
> > >
> > > diff --git a/drivers/media/usb/uvc/uvc_entity.c b/drivers/media/usb/uvc/uvc_entity.c
> > > index d1a652ef35ec34801bd39a5124b834edf838a79e..2dbeb4ab0c4c8cc763ff2dcd2d836a50f3c6a040 100644
> > > --- a/drivers/media/usb/uvc/uvc_entity.c
> > > +++ b/drivers/media/usb/uvc/uvc_entity.c
> > > @@ -72,6 +72,16 @@ static int uvc_mc_init_entity(struct uvc_video_chain *chain,
> > >  {
> > >       int ret;
> > >
> > > +     /*
> > > +      * Do not initialize virtual entities, they do not really exist
> > > +      * and are not connected to any other entities.
> > > +      */
> > > +     switch (UVC_ENTITY_TYPE(entity)) {
> > > +     case UVC_EXT_GPIO_UNIT:
> > > +     case UVC_SWENTITY_UNIT:
> > > +             return 0;
> > > +     }
> > > +
> > >       if (UVC_ENTITY_TYPE(entity) != UVC_TT_STREAMING) {
> > >               u32 function;
> > >
> >
> > --
> > Regards,
> >
> > Laurent Pinchart
>
>
>
> --
> Ricardo Ribalda



-- 
Ricardo Ribalda

Re: [PATCH v2 12/12] media: uvcvideo: Do not create MC entities for virtual entities

Posted by Laurent Pinchart 5 months, 1 week ago

On Tue, Jul 08, 2025 at 08:28:21AM +0200, Ricardo Ribalda wrote:
> On Tue, 1 Jul 2025 at 13:20, Ricardo Ribalda wrote:
> > On Sun, 29 Jun 2025 at 20:06, Laurent Pinchart wrote:
> > > Hi Ricardo,
> > >
> > > Thank you for the patch.
> > >
> > > I would use "software entities" and not "virtual entities" in the
> > > subject line and everywhere else, as those entities are not virtual.
> > >
> > > On Thu, Jun 05, 2025 at 05:53:05PM +0000, Ricardo Ribalda wrote:
> > > > Neither the GPIO nor the SWENTITY entities form part of the device
> > > > pipeline. We just create them to hold emulated uvc controls.
> > > >
> > > > When the device initializes, a warning is thrown by the v4l2 core:
> > > > uvcvideo 1-1:1.0: Entity type for entity SWENTITY was not initialized!
> > > >
> > > > There are no entity function that matches what we are doing here, and
> > > > it does not make to much sense to create a function for entities that
> > > > do not really exist.
> > >
> > > I don't agree with this. The purpose of reporting entities to userspace
> > > through the MC API is to let application enumerate what entities a
> > > device contains. Being able to enumerate software entities seems as
> > > useful as being able to enumerate hardware entities.
> >
> > What function shall we use in this case? Nothing here seems to match a
> > software entity
> > https://www.kernel.org/doc/html/latest/userspace-api/media/mediactl/media-types.html
> >
> > Any suggestion for name?
> > Shall we just live with the warning in dmesg?
> 
>  I just realised that if/when we move to the control framework, the
> software entity will be gone.... So to avoid introducing a uAPI change
> that will be reverted later I think that we should keep this patch.

You know my opinion about moving to the control framework, so that's not
a very compelling argument :-)

We could use MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER, as that's the
function already used by XUs, and the SWENTITY fulfills the same role as
XUs in some devices.

> > > > Do not create MC entities for them and pretend nothing happened here.
> > > >
> > > > Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
> > > > ---
> > > >  drivers/media/usb/uvc/uvc_entity.c | 10 ++++++++++
> > > >  1 file changed, 10 insertions(+)
> > > >
> > > > diff --git a/drivers/media/usb/uvc/uvc_entity.c b/drivers/media/usb/uvc/uvc_entity.c
> > > > index d1a652ef35ec34801bd39a5124b834edf838a79e..2dbeb4ab0c4c8cc763ff2dcd2d836a50f3c6a040 100644
> > > > --- a/drivers/media/usb/uvc/uvc_entity.c
> > > > +++ b/drivers/media/usb/uvc/uvc_entity.c
> > > > @@ -72,6 +72,16 @@ static int uvc_mc_init_entity(struct uvc_video_chain *chain,
> > > >  {
> > > >       int ret;
> > > >
> > > > +     /*
> > > > +      * Do not initialize virtual entities, they do not really exist
> > > > +      * and are not connected to any other entities.
> > > > +      */
> > > > +     switch (UVC_ENTITY_TYPE(entity)) {
> > > > +     case UVC_EXT_GPIO_UNIT:
> > > > +     case UVC_SWENTITY_UNIT:
> > > > +             return 0;
> > > > +     }
> > > > +
> > > >       if (UVC_ENTITY_TYPE(entity) != UVC_TT_STREAMING) {
> > > >               u32 function;
> > > >

-- 
Regards,

Laurent Pinchart

Re: [PATCH v2 12/12] media: uvcvideo: Do not create MC entities for virtual entities

Posted by Ricardo Ribalda 5 months, 1 week ago

On Mon, 14 Jul 2025 at 16:36, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> On Tue, Jul 08, 2025 at 08:28:21AM +0200, Ricardo Ribalda wrote:
> > On Tue, 1 Jul 2025 at 13:20, Ricardo Ribalda wrote:
> > > On Sun, 29 Jun 2025 at 20:06, Laurent Pinchart wrote:
> > > > Hi Ricardo,
> > > >
> > > > Thank you for the patch.
> > > >
> > > > I would use "software entities" and not "virtual entities" in the
> > > > subject line and everywhere else, as those entities are not virtual.
> > > >
> > > > On Thu, Jun 05, 2025 at 05:53:05PM +0000, Ricardo Ribalda wrote:
> > > > > Neither the GPIO nor the SWENTITY entities form part of the device
> > > > > pipeline. We just create them to hold emulated uvc controls.
> > > > >
> > > > > When the device initializes, a warning is thrown by the v4l2 core:
> > > > > uvcvideo 1-1:1.0: Entity type for entity SWENTITY was not initialized!
> > > > >
> > > > > There are no entity function that matches what we are doing here, and
> > > > > it does not make to much sense to create a function for entities that
> > > > > do not really exist.
> > > >
> > > > I don't agree with this. The purpose of reporting entities to userspace
> > > > through the MC API is to let application enumerate what entities a
> > > > device contains. Being able to enumerate software entities seems as
> > > > useful as being able to enumerate hardware entities.
> > >
> > > What function shall we use in this case? Nothing here seems to match a
> > > software entity
> > > https://www.kernel.org/doc/html/latest/userspace-api/media/mediactl/media-types.html
> > >
> > > Any suggestion for name?
> > > Shall we just live with the warning in dmesg?
> >
> >  I just realised that if/when we move to the control framework, the
> > software entity will be gone.... So to avoid introducing a uAPI change
> > that will be reverted later I think that we should keep this patch.
>
> You know my opinion about moving to the control framework, so that's not
> a very compelling argument :-)

Correct me if I am wrong, your opinion is that it will take too much
work, not that it can't be done or that it is a bad idea.

Will send a patch using MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER, but
when/if we use the control framework, please let me drop the swentity.

Thanks!


>
> We could use MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER, as that's the
> function already used by XUs, and the SWENTITY fulfills the same role as
> XUs in some devices.
>
> > > > > Do not create MC entities for them and pretend nothing happened here.
> > > > >
> > > > > Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
> > > > > ---
> > > > >  drivers/media/usb/uvc/uvc_entity.c | 10 ++++++++++
> > > > >  1 file changed, 10 insertions(+)
> > > > >
> > > > > diff --git a/drivers/media/usb/uvc/uvc_entity.c b/drivers/media/usb/uvc/uvc_entity.c
> > > > > index d1a652ef35ec34801bd39a5124b834edf838a79e..2dbeb4ab0c4c8cc763ff2dcd2d836a50f3c6a040 100644
> > > > > --- a/drivers/media/usb/uvc/uvc_entity.c
> > > > > +++ b/drivers/media/usb/uvc/uvc_entity.c
> > > > > @@ -72,6 +72,16 @@ static int uvc_mc_init_entity(struct uvc_video_chain *chain,
> > > > >  {
> > > > >       int ret;
> > > > >
> > > > > +     /*
> > > > > +      * Do not initialize virtual entities, they do not really exist
> > > > > +      * and are not connected to any other entities.
> > > > > +      */
> > > > > +     switch (UVC_ENTITY_TYPE(entity)) {
> > > > > +     case UVC_EXT_GPIO_UNIT:
> > > > > +     case UVC_SWENTITY_UNIT:
> > > > > +             return 0;
> > > > > +     }
> > > > > +
> > > > >       if (UVC_ENTITY_TYPE(entity) != UVC_TT_STREAMING) {
> > > > >               u32 function;
> > > > >
>
> --
> Regards,
>
> Laurent Pinchart



-- 
Ricardo Ribalda

Re: [PATCH v2 12/12] media: uvcvideo: Do not create MC entities for virtual entities

Posted by Laurent Pinchart 5 months, 1 week ago

On Mon, Jul 14, 2025 at 06:04:50PM +0200, Ricardo Ribalda wrote:
> On Mon, 14 Jul 2025 at 16:36, Laurent Pinchart wrote:
> > On Tue, Jul 08, 2025 at 08:28:21AM +0200, Ricardo Ribalda wrote:
> > > On Tue, 1 Jul 2025 at 13:20, Ricardo Ribalda wrote:
> > > > On Sun, 29 Jun 2025 at 20:06, Laurent Pinchart wrote:
> > > > > Hi Ricardo,
> > > > >
> > > > > Thank you for the patch.
> > > > >
> > > > > I would use "software entities" and not "virtual entities" in the
> > > > > subject line and everywhere else, as those entities are not virtual.
> > > > >
> > > > > On Thu, Jun 05, 2025 at 05:53:05PM +0000, Ricardo Ribalda wrote:
> > > > > > Neither the GPIO nor the SWENTITY entities form part of the device
> > > > > > pipeline. We just create them to hold emulated uvc controls.
> > > > > >
> > > > > > When the device initializes, a warning is thrown by the v4l2 core:
> > > > > > uvcvideo 1-1:1.0: Entity type for entity SWENTITY was not initialized!
> > > > > >
> > > > > > There are no entity function that matches what we are doing here, and
> > > > > > it does not make to much sense to create a function for entities that
> > > > > > do not really exist.
> > > > >
> > > > > I don't agree with this. The purpose of reporting entities to userspace
> > > > > through the MC API is to let application enumerate what entities a
> > > > > device contains. Being able to enumerate software entities seems as
> > > > > useful as being able to enumerate hardware entities.
> > > >
> > > > What function shall we use in this case? Nothing here seems to match a
> > > > software entity
> > > > https://www.kernel.org/doc/html/latest/userspace-api/media/mediactl/media-types.html
> > > >
> > > > Any suggestion for name?
> > > > Shall we just live with the warning in dmesg?
> > >
> > >  I just realised that if/when we move to the control framework, the
> > > software entity will be gone.... So to avoid introducing a uAPI change
> > > that will be reverted later I think that we should keep this patch.
> >
> > You know my opinion about moving to the control framework, so that's not
> > a very compelling argument :-)
> 
> Correct me if I am wrong, your opinion is that it will take too much
> work, not that it can't be done or that it is a bad idea.

My opinion is that it's not worth the time needed to write patches, nor
the time needed to review them. It would be a big change, with risks of
regressions, for minimal gain, if any. I don't know if it can be done,
but I think it's a bad idea. Don't fix something that isn't broken, the
patches backlog is big enough, let's focus on useful changes.

> Will send a patch using MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER, but
> when/if we use the control framework, please let me drop the swentity.
> 
> Thanks!
> 
> > We could use MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER, as that's the
> > function already used by XUs, and the SWENTITY fulfills the same role as
> > XUs in some devices.
> >
> > > > > > Do not create MC entities for them and pretend nothing happened here.
> > > > > >
> > > > > > Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
> > > > > > ---
> > > > > >  drivers/media/usb/uvc/uvc_entity.c | 10 ++++++++++
> > > > > >  1 file changed, 10 insertions(+)
> > > > > >
> > > > > > diff --git a/drivers/media/usb/uvc/uvc_entity.c b/drivers/media/usb/uvc/uvc_entity.c
> > > > > > index d1a652ef35ec34801bd39a5124b834edf838a79e..2dbeb4ab0c4c8cc763ff2dcd2d836a50f3c6a040 100644
> > > > > > --- a/drivers/media/usb/uvc/uvc_entity.c
> > > > > > +++ b/drivers/media/usb/uvc/uvc_entity.c
> > > > > > @@ -72,6 +72,16 @@ static int uvc_mc_init_entity(struct uvc_video_chain *chain,
> > > > > >  {
> > > > > >       int ret;
> > > > > >
> > > > > > +     /*
> > > > > > +      * Do not initialize virtual entities, they do not really exist
> > > > > > +      * and are not connected to any other entities.
> > > > > > +      */
> > > > > > +     switch (UVC_ENTITY_TYPE(entity)) {
> > > > > > +     case UVC_EXT_GPIO_UNIT:
> > > > > > +     case UVC_SWENTITY_UNIT:
> > > > > > +             return 0;
> > > > > > +     }
> > > > > > +
> > > > > >       if (UVC_ENTITY_TYPE(entity) != UVC_TT_STREAMING) {
> > > > > >               u32 function;
> > > > > >

-- 
Regards,

Laurent Pinchart