This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform.

Ethos-U Linux driver is to provide an example of how a rich operating
system like Linux can dispatch inferences to an Arm Cortex-M subsystem,
consisting of an Arm Cortex-M and an Arm Ethos-U NPU.

----------------------------------------------------------------
Alison Wang (8):
      ethosu: Add Arm Ethos-U driver
      ethosu: Use RPMsg messaging protocol based on i.MX Rpmsg implementation
      ethosu: Add inference type option for model and op
      ethosu: Add suspend/resume power management
      ethosu: Use ids for identifying messages sent to Ethos-U firmware
      ethosu: Add core message about network info
      ethosu: Add core message about inference cancellation
      ethosu: Add rwlock when alloc and remove msg id

 drivers/firmware/Kconfig                          |   1 +
 drivers/firmware/Makefile                         |   1 +
 drivers/firmware/ethosu/Kconfig                   |  24 ++++
 drivers/firmware/ethosu/Makefile                  |  31 +++++
 drivers/firmware/ethosu/ethosu_buffer.c           | 319 +++++++++++++++++++++++++++++++++++++++++++
 drivers/firmware/ethosu/ethosu_buffer.h           | 106 +++++++++++++++
 drivers/firmware/ethosu/ethosu_cancel_inference.c | 185 +++++++++++++++++++++++++
 drivers/firmware/ethosu/ethosu_cancel_inference.h |  55 ++++++++
 drivers/firmware/ethosu/ethosu_capabilities.c     | 157 ++++++++++++++++++++++
 drivers/firmware/ethosu/ethosu_capabilities.h     |  47 +++++++
 drivers/firmware/ethosu/ethosu_core_interface.h   | 276 ++++++++++++++++++++++++++++++++++++++
 drivers/firmware/ethosu/ethosu_device.c           | 404 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 drivers/firmware/ethosu/ethosu_device.h           |  81 +++++++++++
 drivers/firmware/ethosu/ethosu_driver.c           | 201 +++++++++++++++++++++++++++
 drivers/firmware/ethosu/ethosu_inference.c        | 529 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 drivers/firmware/ethosu/ethosu_inference.h        | 125 +++++++++++++++++
 drivers/firmware/ethosu/ethosu_network.c          | 229 +++++++++++++++++++++++++++++++
 drivers/firmware/ethosu/ethosu_network.h          |  84 ++++++++++++
 drivers/firmware/ethosu/ethosu_network_info.c     | 184 +++++++++++++++++++++++++
 drivers/firmware/ethosu/ethosu_network_info.h     |  56 ++++++++
 drivers/firmware/ethosu/ethosu_rpmsg.c            | 414 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 drivers/firmware/ethosu/ethosu_rpmsg.h            | 155 +++++++++++++++++++++
 drivers/firmware/ethosu/uapi/ethosu.h             | 285 +++++++++++++++++++++++++++++++++++++++
 23 files changed, 3949 insertions(+)
 create mode 100644 drivers/firmware/ethosu/Kconfig
 create mode 100644 drivers/firmware/ethosu/Makefile
 create mode 100644 drivers/firmware/ethosu/ethosu_buffer.c
 create mode 100644 drivers/firmware/ethosu/ethosu_buffer.h
 create mode 100644 drivers/firmware/ethosu/ethosu_cancel_inference.c
 create mode 100644 drivers/firmware/ethosu/ethosu_cancel_inference.h
 create mode 100644 drivers/firmware/ethosu/ethosu_capabilities.c
 create mode 100644 drivers/firmware/ethosu/ethosu_capabilities.h
 create mode 100644 drivers/firmware/ethosu/ethosu_core_interface.h
 create mode 100644 drivers/firmware/ethosu/ethosu_device.c
 create mode 100644 drivers/firmware/ethosu/ethosu_device.h
 create mode 100644 drivers/firmware/ethosu/ethosu_driver.c
 create mode 100644 drivers/firmware/ethosu/ethosu_inference.c
 create mode 100644 drivers/firmware/ethosu/ethosu_inference.h
 create mode 100644 drivers/firmware/ethosu/ethosu_network.c
 create mode 100644 drivers/firmware/ethosu/ethosu_network.h
 create mode 100644 drivers/firmware/ethosu/ethosu_network_info.c
 create mode 100644 drivers/firmware/ethosu/ethosu_network_info.h
 create mode 100644 drivers/firmware/ethosu/ethosu_rpmsg.c
 create mode 100644 drivers/firmware/ethosu/ethosu_rpmsg.h
 create mode 100644 drivers/firmware/ethosu/uapi/ethosu.h

On Fri, Jun 16, 2023 at 01:59:05PM +0800, Alison Wang wrote:
> This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform.
> 
> Ethos-U Linux driver is to provide an example of how a rich operating
> system like Linux can dispatch inferences to an Arm Cortex-M subsystem,
> consisting of an Arm Cortex-M and an Arm Ethos-U NPU.

What is an "inference"?

Where are the userspace tools (if any) that are needed for this driver
to work properly?

thanks,

greg k-h

On Fri, Jun 16, 2023 at 01:59:05PM +0800, Alison Wang wrote:
> This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform.
>
> Ethos-U Linux driver is to provide an example of how a rich operating 
> system like Linux can dispatch inferences to an Arm Cortex-M 
> subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU.

What is an "inference"?
[Alison Wang] The inference here means machine learning inference. It is the process of
running data points into a machine learning model to calculate an output such as a single 
numerical score. For example, we use this driver, user space application and Cortex-M firmware
to complete an inference about image classification.

Where are the userspace tools (if any) that are needed for this driver to work properly?
[Alison Wang] You could refer to https://github.com/nxp-imx/ethos-u-driver-stack-imx.git .


thanks,

greg k-h

On Fri, Jun 16, 2023 at 08:26:28AM +0000, Alison Wang wrote:
> 
> On Fri, Jun 16, 2023 at 01:59:05PM +0800, Alison Wang wrote:
> > This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform.
> >
> > Ethos-U Linux driver is to provide an example of how a rich operating 
> > system like Linux can dispatch inferences to an Arm Cortex-M 
> > subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU.
> 
> What is an "inference"?
> [Alison Wang] The inference here means machine learning inference. It is the process of
> running data points into a machine learning model to calculate an output such as a single 
> numerical score. For example, we use this driver, user space application and Cortex-M firmware
> to complete an inference about image classification.

Nice, so why isn't this under drivers/accel/ then?  Does it not need
that framework?

> Where are the userspace tools (if any) that are needed for this driver to work properly?
> [Alison Wang] You could refer to https://github.com/nxp-imx/ethos-u-driver-stack-imx.git .

Please put that in the documentation for the commit.

thanks,

greg k-h

On Fri, Jun 16, 2023 at 08:26:28AM +0000, Alison Wang wrote:
>
> On Fri, Jun 16, 2023 at 01:59:05PM +0800, Alison Wang wrote:
> > This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform.
> >
> > Ethos-U Linux driver is to provide an example of how a rich
> > operating system like Linux can dispatch inferences to an Arm
> > Cortex-M subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU.
>
> What is an "inference"?
> [Alison Wang] The inference here means machine learning inference. It
> is the process of running data points into a machine learning model to
> calculate an output such as a single numerical score. For example, we
> use this driver, user space application and Cortex-M firmware to complete an inference about image classification.

Nice, so why isn't this under drivers/accel/ then?  Does it not need that framework?
[Alison Wang] This driver doesn't need that framework.

> Where are the userspace tools (if any) that are needed for this driver to work properly?
> [Alison Wang] You could refer to https://github.com/nxp-imx/ethos-u-driver-stack-imx.git .

Please put that in the documentation for the commit.
[Alison Wang] Sure, I will.

thanks,

greg k-h

On Fri, Jun 16, 2023 at 10:47:56AM +0000, Alison Wang wrote:
> On Fri, Jun 16, 2023 at 08:26:28AM +0000, Alison Wang wrote:
> >
> > On Fri, Jun 16, 2023 at 01:59:05PM +0800, Alison Wang wrote:
> > > This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform.
> > >
> > > Ethos-U Linux driver is to provide an example of how a rich
> > > operating system like Linux can dispatch inferences to an Arm
> > > Cortex-M subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU.
> >
> > What is an "inference"?
> > [Alison Wang] The inference here means machine learning inference. It
> > is the process of running data points into a machine learning model to
> > calculate an output such as a single numerical score. For example, we
> > use this driver, user space application and Cortex-M firmware to complete an inference about image classification.
> 
> Nice, so why isn't this under drivers/accel/ then?  Does it not need that framework?
> [Alison Wang] This driver doesn't need that framework.

Odd quoting style...

Anyway, why not use that famework?  That's what that subsystem is for,
so you are going to have to document why this does not fit into that
model at all and you need your own interface instead.

thanks,

greg k-h