Hi All,

This RFC introduces a driver for the Arm Core Local Accelerator (CLA), a
CPU-local interface for programming attached accelerators. While the interface
is agnostic to the accelerator type, the initial (and currently only) target is
a compute engine.

The RFC aims to engage the community and get feedback on some key aspects. This
will help plan our approach for eventual upstreaming. The patches implement a
bare bones driver to aid discussion. Arm plans to publish the CLA spec in
future, but for now I hope the documentation included with patch 1 suffices.
Note that the CLA is not part of the Arm Architecture.

Patch 1 documents the hardware and driver design, and adds a driver skeleton.
Patches 2-4 initialize and probe the device. Patches 5-8 add context management
for switching the device between different users.

Aspects I'm seeking feedback for:

* The general structure of the driver: The current shape addresses the
  performance requirements we have for the use cases, and is therefore our
  preferred approach. But there are some unusual aspects due to the HW design.

* Driver interaction with architectural support: I've opted to treat the CLA as
  a device rather than a CPU extension, so it's implemented as a (mostly)
  self-contained driver. It has some unavoidable coupling to the arch code since
  it needs to share page tables and ASIDs (arm64_mm_context_[put|get]()).

* The location of the driver: although most accelerators will likely be compute
  ones, CLA is a generic MMIO interface that could handle any kind of
  accelerator. It's currently implemented as a misc driver. accel is another
  potential option, but given the current SVA approach, we would not use any of
  the services (memory-management or otherwise) that accel provides.

* User space availability: The kernel driver exposes the capabilities of the
  hardware to user space. Arm plans to open source a user space driver, but does
  not yet have any committed date. I'd like to understand if the availability of
  this component will be a prerequisite for upstream acceptance of the kernel
  driver; either way, I'm hoping we can at least progress with some discussion
  in its absence.

I'm deliberately constraining the scope to bare-metal support for now.
Virtualization is something we are considering (and have prototyped), but plan
to post a separate RFC for that as follow-up, once we have agreement on
direction for the bare-metal driver.

Source, along with some tests, is available at [1]. Patches based on v7.2-rc3.

[1] https://gitlab.arm.com/linux-arm/linux-rr/-/tree/features/cla-driver-v1-rfc-tests

Thanks,
Ryan


Jean-Philippe Brucker (5):
  misc/arm-cla: Add driver skeleton and documentation
  misc/arm-cla: Add launch operation helpers
  misc/arm-cla: Probe firmware-described devices
  misc/arm-cla: Initialize devices on CPU bringup
  misc/arm-cla: Accelerator context save and restore

Ryan Roberts (3):
  misc/arm-cla: Set up memory translation context
  misc/arm-cla: Manage domain contexts
  misc/arm-cla: Add userspace interface

 Documentation/misc-devices/arm-cla.rst | 206 ++++++++++
 drivers/misc/Kconfig                   |   1 +
 drivers/misc/Makefile                  |   1 +
 drivers/misc/arm-cla/Kconfig           |  10 +
 drivers/misc/arm-cla/Makefile          |  13 +
 drivers/misc/arm-cla/arm-cla-regs.h    | 179 +++++++++
 drivers/misc/arm-cla/arm-cla.h         | 296 +++++++++++++++
 drivers/misc/arm-cla/cla-ctx.c         | 142 +++++++
 drivers/misc/arm-cla/cla-init.c        | 503 +++++++++++++++++++++++++
 drivers/misc/arm-cla/cla-mtc.c         | 139 +++++++
 drivers/misc/arm-cla/cla-ops.c         | 342 +++++++++++++++++
 drivers/misc/arm-cla/cla-regs.c        | 263 +++++++++++++
 drivers/misc/arm-cla/cla-sched.c       | 474 +++++++++++++++++++++++
 drivers/misc/arm-cla/cla-topology.c    | 187 +++++++++
 drivers/misc/arm-cla/cla-user.c        | 351 +++++++++++++++++
 include/uapi/linux/arm-cla.h           | 207 ++++++++++
 16 files changed, 3314 insertions(+)
 create mode 100644 Documentation/misc-devices/arm-cla.rst
 create mode 100644 drivers/misc/arm-cla/Kconfig
 create mode 100644 drivers/misc/arm-cla/Makefile
 create mode 100644 drivers/misc/arm-cla/arm-cla-regs.h
 create mode 100644 drivers/misc/arm-cla/arm-cla.h
 create mode 100644 drivers/misc/arm-cla/cla-ctx.c
 create mode 100644 drivers/misc/arm-cla/cla-init.c
 create mode 100644 drivers/misc/arm-cla/cla-mtc.c
 create mode 100644 drivers/misc/arm-cla/cla-ops.c
 create mode 100644 drivers/misc/arm-cla/cla-regs.c
 create mode 100644 drivers/misc/arm-cla/cla-sched.c
 create mode 100644 drivers/misc/arm-cla/cla-topology.c
 create mode 100644 drivers/misc/arm-cla/cla-user.c
 create mode 100644 include/uapi/linux/arm-cla.h

--
2.43.0

Hi Ryan,

I haven't bothered to look at the code (looks like Sashiko is having fun
with that), but I'm going to jump at this bit:

On Fri, Jul 17, 2026 at 11:47:44AM +0100, Ryan Roberts wrote:
> * User space availability: The kernel driver exposes the capabilities of the
>   hardware to user space. Arm plans to open source a user space driver, but does
>   not yet have any committed date. I'd like to understand if the availability of
>   this component will be a prerequisite for upstream acceptance of the kernel
>   driver; either way, I'm hoping we can at least progress with some discussion
>   in its absence.

From my perspective, I'm not particularly interested in having code in
the upstream kernel tree that we can't meaningfully exercise or benefit
from. I also think that the incentive for Arm to open source the
user-space driver practically disappears if we merge the kernel part
first. So, at the moment, this just looks like a burden to me, especially
as it appears to create a brand new, device-specific UAPI for what is
ostensibly a form of SVA - something which the community is actively
working on already.

Relatedly, is there a spec and/or fastmodel/qemu (sorry...) support for
this?

> I'm deliberately constraining the scope to bare-metal support for now.
> Virtualization is something we are considering (and have prototyped), but plan
> to post a separate RFC for that as follow-up, once we have agreement on
> direction for the bare-metal driver.

I'd actually like to see what the virtualisation part looks like first
because doing it as a bolt-on later feels like the wrong approach. The
structure you have at the moment is remarkably clean, given the
architectural/CPU interactions (this thing even apparently builds as a
module, nice!), but I'm unsure how far you can push the separation once
you need to start hacking at KVM. Maybe the MMU notifiers are enough,
but I can't tell.

Will

On Fri, Jul 17, 2026 at 12:33:17PM +0100, Will Deacon wrote:

> first. So, at the moment, this just looks like a burden to me, especially
> as it appears to create a brand new, device-specific UAPI for what is
> ostensibly a form of SVA - something which the community is actively
> working on already.

Yeah, I think it was a mistake to hardwire the invalidation to the
CPU. It would fit much better into Linux if the invalidation was
separate and independently controllable with an option to follow the
CPU.

Then you could use a normal S2 page table through iommufd and not mess
with kvm. (though the VMID sharing with KVM for a BTM-like scheme was
never solved upstream)

I also wouldn't expect you to use vfio-mdev, if the devices are
discovered and known at boot then it should be a normal vfio driver.

Jason

On 17/07/2026 14:32, Jason Gunthorpe wrote:
> On Fri, Jul 17, 2026 at 12:33:17PM +0100, Will Deacon wrote:
> 
>> first. So, at the moment, this just looks like a burden to me, especially
>> as it appears to create a brand new, device-specific UAPI for what is
>> ostensibly a form of SVA - something which the community is actively
>> working on already.
> 
> Yeah, I think it was a mistake to hardwire the invalidation to the
> CPU. It would fit much better into Linux if the invalidation was
> separate and independently controllable with an option to follow the
> CPU.

Well I'm certainly not going to disagree, but the HW is somewhat fixed at this
point :-|

> 
> Then you could use a normal S2 page table through iommufd and not mess
> with kvm. (though the VMID sharing with KVM for a BTM-like scheme was
> never solved upstream)
> 
> I also wouldn't expect you to use vfio-mdev, if the devices are
> discovered and known at boot then it should be a normal vfio driver.

OK good feedback - I'm not going to claim to be a VFIO expert (especially not in
front of an actual VFIO expert) - We'll look closer at this and come back if we
have questions (although unlikely over the summer holiday).

Thanks,
Ryan


> 
> Jason

Hi Will,

On 17/07/2026 12:33, Will Deacon wrote:
> Hi Ryan,
> 
> I haven't bothered to look at the code (looks like Sashiko is having fun
> with that), but I'm going to jump at this bit:

Thanks for the quick reply!

JP and I had a quick glance at the Sashiko feedback; looks like some useful
stuff, and a few false alarms. Nothing cricial though. Not that it matters too
much for this stage of discussion.

> 
> On Fri, Jul 17, 2026 at 11:47:44AM +0100, Ryan Roberts wrote:
>> * User space availability: The kernel driver exposes the capabilities of the
>>   hardware to user space. Arm plans to open source a user space driver, but does
>>   not yet have any committed date. I'd like to understand if the availability of
>>   this component will be a prerequisite for upstream acceptance of the kernel
>>   driver; either way, I'm hoping we can at least progress with some discussion
>>   in its absence.
> 
> From my perspective, I'm not particularly interested in having code in
> the upstream kernel tree that we can't meaningfully exercise or benefit
> from. I also think that the incentive for Arm to open source the
> user-space driver practically disappears if we merge the kernel part
> first. So, at the moment, this just looks like a burden to me, especially

Understood. Now that I have such a clear statement, hopefully I can use that to
work internally to get commitments and dates to build a proper plan. What I'm
trying to get out of this RFC though, is "is having an open source user space
the main blocker, or are there other significant challenges here too?".

> as it appears to create a brand new, device-specific UAPI for what is
> ostensibly a form of SVA - something which the community is actively
> working on already.
> 
> Relatedly, is there a spec and/or fastmodel/qemu (sorry...) support for
> this?

They both exist internally of course. There is a plan to publish the spec, but I
don't think we will converge on a timeframe until after the summer now. I'll
follow up regarding fastmodel.

> 
>> I'm deliberately constraining the scope to bare-metal support for now.
>> Virtualization is something we are considering (and have prototyped), but plan
>> to post a separate RFC for that as follow-up, once we have agreement on
>> direction for the bare-metal driver.
> 
> I'd actually like to see what the virtualisation part looks like first
> because doing it as a bolt-on later feels like the wrong approach. The
> structure you have at the moment is remarkably clean, given the
> architectural/CPU interactions (this thing even apparently builds as a
> module, nice!), but I'm unsure how far you can push the separation once
> you need to start hacking at KVM. Maybe the MMU notifiers are enough,
> but I can't tell.

The virtualization implementation is not as advanced as bare-metal and JP can
probably comment on it in more detail, but the intention is to expose the HW
using VFIO-MDEV (mediated device framework). We will need exported functions
from (the arm64 part of) KVM to grab the S2 pgdir and to get/put the VMID - we
would not be using MMU notifiers but instead sharing the pgtable and VMID and
piggybacking the CPU's TBLI operations. My current expectation is that it can be
made to work without deep KVM integration and still as a module, but let's see
where we get to with the code. (I have spoken with Marc about this briefly - I
would say he doesn't love it, but agreed to look at the code when we have it).

But I think I'm hearing that to get into a deeper discussion we will need to
post virt support? In which case, I'll prioritise getting something together.
I'm about to be on sabbatical for 4 weeks though, so probably won't be until
~early Sept.

Thanks,
Ryan


> 
> Will

Thanks Will for roping me in.

On Fri, 17 Jul 2026 12:33:17 +0100,
Will Deacon <will@kernel.org> wrote:
> 
> > I'm deliberately constraining the scope to bare-metal support for now.
> > Virtualization is something we are considering (and have prototyped), but plan
> > to post a separate RFC for that as follow-up, once we have agreement on
> > direction for the bare-metal driver.
> 
> I'd actually like to see what the virtualisation part looks like first
> because doing it as a bolt-on later feels like the wrong approach. The
> structure you have at the moment is remarkably clean, given the
> architectural/CPU interactions (this thing even apparently builds as a
> module, nice!), but I'm unsure how far you can push the separation once
> you need to start hacking at KVM. Maybe the MMU notifiers are enough,
> but I can't tell.

+1.

Virtualisation cannot be a "bolt on the side" exercise. It is an
integral part of the arm64 tree, particularly for memory management
and scheduling, all of which have a direct impact on KVM.

I don't think we can really evaluate anything here without looking at
the full picture.

Thanks,

	M.

-- 
Jazz isn't dead. It just smells funny.

On 17/07/2026 13:09, Marc Zyngier wrote:
> Thanks Will for roping me in.

Sorry - I intentionally didn't include you because last time we spoke you said
you were only really interested in discussions on the virt side of things and I
didn't want to spam your inbox. Perhaps the wrong decision...

> 
> On Fri, 17 Jul 2026 12:33:17 +0100,
> Will Deacon <will@kernel.org> wrote:
>>
>>> I'm deliberately constraining the scope to bare-metal support for now.
>>> Virtualization is something we are considering (and have prototyped), but plan
>>> to post a separate RFC for that as follow-up, once we have agreement on
>>> direction for the bare-metal driver.
>>
>> I'd actually like to see what the virtualisation part looks like first
>> because doing it as a bolt-on later feels like the wrong approach. The
>> structure you have at the moment is remarkably clean, given the
>> architectural/CPU interactions (this thing even apparently builds as a
>> module, nice!), but I'm unsure how far you can push the separation once
>> you need to start hacking at KVM. Maybe the MMU notifiers are enough,
>> but I can't tell.
> 
> +1.
> 
> Virtualisation cannot be a "bolt on the side" exercise. It is an
> integral part of the arm64 tree, particularly for memory management
> and scheduling, all of which have a direct impact on KVM.
> 
> I don't think we can really evaluate anything here without looking at
> the full picture.

OK understood - as per reply to Will, we'll prioritise doing a version with virt
support.

Thanks,
Ryan


> 
> Thanks,
> 
> 	M.
>

On Fri, 17 Jul 2026 13:33:07 +0100,
Ryan Roberts <ryan.roberts@arm.com> wrote:
> 
> On 17/07/2026 13:09, Marc Zyngier wrote:
> > Thanks Will for roping me in.
> 
> Sorry - I intentionally didn't include you because last time we spoke you said
> you were only really interested in discussions on the virt side of things and I
> didn't want to spam your inbox. Perhaps the wrong decision...

I'm indeed mostly interested in the impacts on the KVM side. However,
a lot of the decisions that you make on the non-virt side of the
kernel have a two-pronged effect:

- both memory management and scheduling changes have a direct impact
  on KVM because of the intricate weaving of KVM's own MM with the
  kernel's.

- most of the requirements that you define for bare-metal will have to
  be enforced by KVM, because it is not conceivable that you'd have a
  different behaviour between host and guest.

For these reasons, I think it is necessary to involve KVM reviewers
from the beginning, rather than after everything has already been set
in stone. Unless you say upfront that virtualisation of CLA will never
be supported. And even then, you'd have to look into the effects of a
CLA user also being a VMM...

And please don't worry about my Inbox ;-).

Thanks,

	M.

-- 
Jazz isn't dead. It just smells funny.