remoteproc: core: Add support for predefined notifyids

[RFC PATCH] remoteproc: core: Add support for predefined notifyids

Posted by Daniel Baluta 1 month, 1 week ago

Currently we generate notifyids in the linux kernel and override
those found in rsc_table.

This doesn't play well with users expecting to use the exact ids
from rsc_table.

So, use predefined notifyids found in rsc_table if any. Otherwise,
let Linux generate the ids as before.

Keypoint is we also define an invalid notifid as 0xFFFFFFFFU. This
should be placed as notifids if users want Linux to generate the ids.

Signed-off-by: Alexandru Lastur <alexandru.lastur@nxp.com>
Signed-off-by: Daniel Baluta <daniel.baluta@nxp.com>
---
 drivers/remoteproc/remoteproc_core.c | 14 ++++++++++++--
 include/linux/remoteproc.h           |  1 +
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index f276956f2c5c..9f00fe16da38 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -332,6 +332,7 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)
 	int ret, notifyid;
 	struct rproc_mem_entry *mem;
 	size_t size;
+	int start, end;
 
 	/* actual size of vring (in bytes) */
 	size = PAGE_ALIGN(vring_size(rvring->num, rvring->align));
@@ -363,9 +364,18 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)
 	/*
 	 * Assign an rproc-wide unique index for this vring
 	 * TODO: assign a notifyid for rvdev updates as well
-	 * TODO: support predefined notifyids (via resource table)
 	 */
-	ret = idr_alloc(&rproc->notifyids, rvring, 0, 0, GFP_KERNEL);
+
+	start = 0;
+	end = 0;
+
+	/* use id if specified in rsc table */
+	if (rsc->vring[i].notifyid != RSC_INVALID_NOTIFYID) {
+		start = rsc->vring[i].notifyid;
+		end = start + 1;
+	}
+
+	ret = idr_alloc(&rproc->notifyids, rvring, start, end, GFP_KERNEL);
 	if (ret < 0) {
 		dev_err(dev, "idr_alloc failed: %d\n", ret);
 		return ret;
diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
index b4795698d8c2..98c3e181086e 100644
--- a/include/linux/remoteproc.h
+++ b/include/linux/remoteproc.h
@@ -238,6 +238,7 @@ struct fw_rsc_trace {
 	u8 name[32];
 } __packed;
 
+#define RSC_INVALID_NOTIFYID 0xFFFFFFFFU
 /**
  * struct fw_rsc_vdev_vring - vring descriptor entry
  * @da: device address
-- 
2.43.0

Re: [RFC PATCH] remoteproc: core: Add support for predefined notifyids

Posted by Mathieu Poirier 1 month ago

Hello Daniel,

On Fri, Oct 18, 2024 at 02:09:29PM +0300, Daniel Baluta wrote:
> Currently we generate notifyids in the linux kernel and override
> those found in rsc_table.
> 
> This doesn't play well with users expecting to use the exact ids
> from rsc_table.
> 
> So, use predefined notifyids found in rsc_table if any. Otherwise,
> let Linux generate the ids as before.
> 
> Keypoint is we also define an invalid notifid as 0xFFFFFFFFU. This
> should be placed as notifids if users want Linux to generate the ids.
> 
> Signed-off-by: Alexandru Lastur <alexandru.lastur@nxp.com>
> Signed-off-by: Daniel Baluta <daniel.baluta@nxp.com>
> ---
>  drivers/remoteproc/remoteproc_core.c | 14 ++++++++++++--
>  include/linux/remoteproc.h           |  1 +
>  2 files changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
> index f276956f2c5c..9f00fe16da38 100644
> --- a/drivers/remoteproc/remoteproc_core.c
> +++ b/drivers/remoteproc/remoteproc_core.c
> @@ -332,6 +332,7 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)
>  	int ret, notifyid;
>  	struct rproc_mem_entry *mem;
>  	size_t size;
> +	int start, end;
>  
>  	/* actual size of vring (in bytes) */
>  	size = PAGE_ALIGN(vring_size(rvring->num, rvring->align));
> @@ -363,9 +364,18 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)
>  	/*
>  	 * Assign an rproc-wide unique index for this vring
>  	 * TODO: assign a notifyid for rvdev updates as well
> -	 * TODO: support predefined notifyids (via resource table)
>  	 */
> -	ret = idr_alloc(&rproc->notifyids, rvring, 0, 0, GFP_KERNEL);
> +
> +	start = 0;
> +	end = 0;
> +
> +	/* use id if specified in rsc table */
> +	if (rsc->vring[i].notifyid != RSC_INVALID_NOTIFYID) {
> +		start = rsc->vring[i].notifyid;
> +		end = start + 1;
> +	}

This will likely introduce a backward compatibility issue where anyone that
has more than one vring and set their notifyids to anything else than 0xFFFFFFFF
in the resource table will see a boot failure.

A while back the openAMP group started discussions on using the configuration
space of a virtio device to enhance device discovery, with exactly this kind of
use case in mind.  I think it is the only way to move forward with this
feature, though it is a big job that requires a lot of community interactions.

Regards,
Mathieu

> +
> +	ret = idr_alloc(&rproc->notifyids, rvring, start, end, GFP_KERNEL);
>  	if (ret < 0) {
>  		dev_err(dev, "idr_alloc failed: %d\n", ret);
>  		return ret;
> diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
> index b4795698d8c2..98c3e181086e 100644
> --- a/include/linux/remoteproc.h
> +++ b/include/linux/remoteproc.h
> @@ -238,6 +238,7 @@ struct fw_rsc_trace {
>  	u8 name[32];
>  } __packed;
>  
> +#define RSC_INVALID_NOTIFYID 0xFFFFFFFFU
>  /**
>   * struct fw_rsc_vdev_vring - vring descriptor entry
>   * @da: device address
> -- 
> 2.43.0
>

Re: [RFC PATCH] remoteproc: core: Add support for predefined notifyids

Posted by Daniel Baluta 3 weeks, 5 days ago

On Wed, Oct 23, 2024 at 6:32 PM Mathieu Poirier
<mathieu.poirier@linaro.org> wrote:
>
> Hello Daniel,
>
> On Fri, Oct 18, 2024 at 02:09:29PM +0300, Daniel Baluta wrote:
> > Currently we generate notifyids in the linux kernel and override
> > those found in rsc_table.
> >
> > This doesn't play well with users expecting to use the exact ids
> > from rsc_table.
> >
> > So, use predefined notifyids found in rsc_table if any. Otherwise,
> > let Linux generate the ids as before.
> >
> > Keypoint is we also define an invalid notifid as 0xFFFFFFFFU. This
> > should be placed as notifids if users want Linux to generate the ids.
> >
> > Signed-off-by: Alexandru Lastur <alexandru.lastur@nxp.com>
> > Signed-off-by: Daniel Baluta <daniel.baluta@nxp.com>
> > ---
> >  drivers/remoteproc/remoteproc_core.c | 14 ++++++++++++--
> >  include/linux/remoteproc.h           |  1 +
> >  2 files changed, 13 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
> > index f276956f2c5c..9f00fe16da38 100644
> > --- a/drivers/remoteproc/remoteproc_core.c
> > +++ b/drivers/remoteproc/remoteproc_core.c
> > @@ -332,6 +332,7 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)
> >       int ret, notifyid;
> >       struct rproc_mem_entry *mem;
> >       size_t size;
> > +     int start, end;
> >
> >       /* actual size of vring (in bytes) */
> >       size = PAGE_ALIGN(vring_size(rvring->num, rvring->align));
> > @@ -363,9 +364,18 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)
> >       /*
> >        * Assign an rproc-wide unique index for this vring
> >        * TODO: assign a notifyid for rvdev updates as well
> > -      * TODO: support predefined notifyids (via resource table)
> >        */
> > -     ret = idr_alloc(&rproc->notifyids, rvring, 0, 0, GFP_KERNEL);
> > +
> > +     start = 0;
> > +     end = 0;
> > +
> > +     /* use id if specified in rsc table */
> > +     if (rsc->vring[i].notifyid != RSC_INVALID_NOTIFYID) {
> > +             start = rsc->vring[i].notifyid;
> > +             end = start + 1;
> > +     }
>
> This will likely introduce a backward compatibility issue where anyone that
> has more than one vring and set their notifyids to anything else than 0xFFFFFFFF
> in the resource table will see a boot failure.
>
> A while back the openAMP group started discussions on using the configuration
> space of a virtio device to enhance device discovery, with exactly this kind of
> use case in mind.  I think it is the only way to move forward with this
> feature, though it is a big job that requires a lot of community interactions.

I also found this attempt few years ago:

https://lkml.iu.edu/hypermail/linux/kernel/2001.2/05248.html

I think the best way to go is to bump the resource table version and support
the old behavior for v1 and the new behavior for the new version
(which should be v2).

We will be back next week with a patch to express this.

thanks,
Daniel.

Re: [RFC PATCH] remoteproc: core: Add support for predefined notifyids

Posted by Arnaud POULIQUEN 1 month ago

hello

On 10/23/24 17:32, Mathieu Poirier wrote:
> Hello Daniel,
> 
> On Fri, Oct 18, 2024 at 02:09:29PM +0300, Daniel Baluta wrote:
>> Currently we generate notifyids in the linux kernel and override
>> those found in rsc_table.
>>
>> This doesn't play well with users expecting to use the exact ids
>> from rsc_table.
>>
>> So, use predefined notifyids found in rsc_table if any. Otherwise,
>> let Linux generate the ids as before.
>>
>> Keypoint is we also define an invalid notifid as 0xFFFFFFFFU. This
>> should be placed as notifids if users want Linux to generate the ids.
>>
>> Signed-off-by: Alexandru Lastur <alexandru.lastur@nxp.com>
>> Signed-off-by: Daniel Baluta <daniel.baluta@nxp.com>
>> ---
>>  drivers/remoteproc/remoteproc_core.c | 14 ++++++++++++--
>>  include/linux/remoteproc.h           |  1 +
>>  2 files changed, 13 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
>> index f276956f2c5c..9f00fe16da38 100644
>> --- a/drivers/remoteproc/remoteproc_core.c
>> +++ b/drivers/remoteproc/remoteproc_core.c
>> @@ -332,6 +332,7 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)
>>  	int ret, notifyid;
>>  	struct rproc_mem_entry *mem;
>>  	size_t size;
>> +	int start, end;
>>  
>>  	/* actual size of vring (in bytes) */
>>  	size = PAGE_ALIGN(vring_size(rvring->num, rvring->align));
>> @@ -363,9 +364,18 @@ int rproc_alloc_vring(struct rproc_vdev *rvdev, int i)
>>  	/*
>>  	 * Assign an rproc-wide unique index for this vring
>>  	 * TODO: assign a notifyid for rvdev updates as well
>> -	 * TODO: support predefined notifyids (via resource table)
>>  	 */
>> -	ret = idr_alloc(&rproc->notifyids, rvring, 0, 0, GFP_KERNEL);
>> +
>> +	start = 0;
>> +	end = 0;
>> +
>> +	/* use id if specified in rsc table */
>> +	if (rsc->vring[i].notifyid != RSC_INVALID_NOTIFYID) {
>> +		start = rsc->vring[i].notifyid;
>> +		end = start + 1;
>> +	}
> 
> This will likely introduce a backward compatibility issue where anyone that
> has more than one vring and set their notifyids to anything else than 0xFFFFFFFF
> in the resource table will see a boot failure.
> 
> A while back the openAMP group started discussions on using the configuration
> space of a virtio device to enhance device discovery, with exactly this kind of
> use case in mind.  I think it is the only way to move forward with this
> feature, though it is a big job that requires a lot of community interactions.
> 
FYI a PR exists in OpenAMP to use this configuration space to customize the
size of the buffers [2].

This probably means we need something scalable for the configuration space.

I guess your objective is to match the vring ID with the mailbox channel ID, right?
We have a similar issue with the STM32MP1/2. It is hard-coded in the stm32_rproc
driver[1], which is far from perfect because it is not simple to add a new vdev
instance.

I would have two other alternatives to highlight

- An alternative could be to have a vendor resource in the resource table and
use rproc_handle_rsc operations. This resource would define the match between the
vrings IDs and the mailbox IDs.
However, if several vendors are facing the same issue, perhaps better to find a
generic solution.

- another alternative already discussed several times in openAMP group would be
to define a new version of the resource table.

[1]https://elixir.bootlin.com/linux/v6.11/source/drivers/remoteproc/stm32_rproc.c#L328
[2] https://github.com/OpenAMP/open-amp/pull/155

Regards
Arnaud

> Regards,
> Mathieu
> 
>> +
>> +	ret = idr_alloc(&rproc->notifyids, rvring, start, end, GFP_KERNEL);
>>  	if (ret < 0) {
>>  		dev_err(dev, "idr_alloc failed: %d\n", ret);
>>  		return ret;
>> diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
>> index b4795698d8c2..98c3e181086e 100644
>> --- a/include/linux/remoteproc.h
>> +++ b/include/linux/remoteproc.h
>> @@ -238,6 +238,7 @@ struct fw_rsc_trace {
>>  	u8 name[32];
>>  } __packed;
>>  
>> +#define RSC_INVALID_NOTIFYID 0xFFFFFFFFU
>>  /**
>>   * struct fw_rsc_vdev_vring - vring descriptor entry
>>   * @da: device address
>> -- 
>> 2.43.0
>>
>