In the NVIDIA vGPU RFC [1], the vGPU type blobs must be provided to the GSP before userspace can enumerate available vGPU types and create vGPU instances. The original design relied on the firmware loading interface, but fwctl is a more natural fit for this use case, as it is designed for uploading configuration or firmware data required before the device becomes operational. This series introduces a Rust abstraction over the fwctl subsystem, providing safe and idiomatic bindings. The series is now a single Rust patch. The fwctl core release hook from v4 is no longer needed. Instead, `Registration` owns the callback-visible driver data and ties it to the parent device binding lifetime through Danilo's higher-ranked lifetime model. `Device<T>` remains the refcounted fwctl object, while callbacks borrow registration data for the callback lifetime. The Rust fwctl module allows Rust drivers to integrate with the existing C-side fwctl core through a typed trait interface. It provides: - `Operations` trait: defines driver-specific callbacks: `open()`, `close()`, `info()`, and `fw_rpc()`. The implementing type itself serves as the per-FD user context, one instance per open(). - `RegistrationData`: a `ForLt` associated type owned by `Registration` and borrowed by callbacks under the fwctl registration lock. - `Device<T>`: wraps `struct fwctl_device` and stores the pointer to registration-owned driver data used during callbacks. - `Registration<T>`: registration and automatic unregistration of `struct fwctl_device` objects. - `RpcScope` / `FwRpcResponse`: type-safe enums for RPC scope and response handling, keeping unsafe pointer manipulation confined to the abstraction layer. `rust/kernel/lib.rs` is updated to conditionally include this module under `CONFIG_RUST_FWCTL_ABSTRACTIONS`. v5: - Rebase on top of the current drm-rust-next. - Adopt Danilo's higher-ranked lifetime model for callback-visible registration data. - Drop the fwctl core release hook patch. - Reject `FwRpcResponse::InPlace(len)` when `len` is larger than the input buffer length. - Add the missing BNXT fwctl device type to the Rust `DeviceType` enum. - Document that `Registration::new()` requires an unregistered `Device` with no live `Registration`, and no concurrent registration attempt for that device. Link to v4: [2] v4: - Rebase on top of the current drm-rust-next. - Split out the fwctl core release hook before the Rust abstraction. - Drop the fwctl init-ordering change from this series; it is already in drm-rust-next as commit a55f80233f38 ("fwctl: Fix class init ordering to avoid NULL pointer dereference on device removal"). - Add compile-time layout checks for the embedded `struct fwctl_device` and `struct fwctl_uctx` offset assumptions. (Jason) - Use `const_assert!()` for generic layout assertions. (Zhi) - Require `Operations` and its `DeviceData` to be `Send + Sync`. (Danilo) - Pass pinned shared references to `info()` and `fw_rpc()`. (Danilo) - Make `Operations::open()` return an initializer directly and report open failures through the initializer error path. (Danilo) - Drop `DeviceData` from the fwctl device release hook. - Fix clippy warnings for raw pointer casts and unsafe blocks. (Danilo) - Fix the rustdoc broken link warning. (Danilo) Link to v3: [3] v3: Quite some updates in this version. Here you can find the example nova-core fwctl driver [4]. The interface is still WIP so it is just to demonstrate the use of the rust fwctl abstractions. Comments from folks: - Use an enum for the return of fw_rpc. (Joel) - Remove FWCTL_DEVICE_TYPE_RUST_FWCTL_TEST together with the sample driver. (Jason) - Remove DeviceType:Error. (Gary) - Add __rust_helper for fwctl_get/fwctl_put. (Gary) - Refine the design of the device private data. Now it has a similar device private data structure as DRM. (Danilo) - Separate fwctl alloc and register in the abstractions. (Jason) - Registration::new() now takes &fwctl::Device<T> and the parent &Device<Bound> to align with other class device abstractions. (Danilo) - Update the Registration SAFETY comments. (Danilo & Jason) - Take self as per-FD user context in callbacks. (Danilo) - {open, close}_uctx -> {open, close}(). open() now takes &Device<Self>. (Danilo) Updates from me: - Introduce enums for fwctl RPC scope. - Introduce AlwaysRefCounted to avoid hacks after introducing the refined flow of device private data. - Introduce default implementation of close()/info(). - Fix a leak: Drop T::UserCtx in the close_uctx_callback explicitly. v2: - Don't open fwctl_put(). Add a rust helper. (Jason/Danilo) - Wrap Registration with Devres to guarantee proper lifetime management. (Jason/Danilo) - Rename FwctlOps to Operations, FwctlUserCtx to UserCtx, FwctlDevice to Device. (Danilo) - Use fwctl::DeviceType enum instead of raw u32 for DEVICE_TYPE. (Danilo) - Change fwctl_uctx field in UserCtx to Opaque<bindings::fwctl_uctx> and make it private. (Danilo) - Provide Deref and DerefMut implementations for UserCtx::uctx. (Danilo) - Add UserCtx::parent_device_from_raw() to simplify parent device access. - Use cast() and cast_mut() instead of manual pointer casts. (Danilo) - Implement AlwaysRefCounted for Device and use ARef<Device> in Registration. (Danilo) - Add rust_helper_fwctl_get() for reference counting. - Improve safety comments for slice::from_raw_parts_mut() in fw_rpc_callback. (Danilo) - Convert imports to vertical style. - Fix all clippy warnings. v1: - Initial submission introducing fwctl Rust abstractions. [1] https://lore.kernel.org/all/20250903221111.3866249-1-zhiw@nvidia.com/ [2] https://lore.kernel.org/all/20260624091758.1678092-1-zhiw@nvidia.com/ [3] https://lore.kernel.org/all/20260217204909.211793-1-zhiw@nvidia.com/ [4] https://github.com/zhiwang-nvidia/nova-core/commit/2068da7e8caf58da9584b0aa6c81fed8f547d59f Zhi Wang (1): rust: introduce abstractions for fwctl drivers/fwctl/Kconfig | 12 + rust/bindings/bindings_helper.h | 1 + rust/helpers/fwctl.c | 17 + rust/helpers/helpers.c | 3 +- rust/kernel/fwctl.rs | 540 ++++++++++++++++++++++++++++++++ rust/kernel/lib.rs | 2 + 6 files changed, 574 insertions(+), 1 deletion(-) create mode 100644 rust/helpers/fwctl.c create mode 100644 rust/kernel/fwctl.rs -- 2.51.0 Introduce safe Rust wrappers around struct fwctl_device and struct fwctl_uctx. This lets Rust drivers register fwctl devices and implement firmware RPC callbacks through a typed trait interface. The abstraction keeps lifetime and reference-count handling inside the wrapper, exposes pinned per-FD user contexts to drivers, and validates the layout assumptions required by the C fwctl allocation model. Registration owns driver private data with a lifetime tied to the bound parent device. fwctl callbacks access that data through the Registration lifetime, while Device remains only the refcounted fwctl object. This avoids requiring Rust drop glue from the fwctl_device release path after unregister or module teardown. Suggested-by: Danilo Krummrich <dakr@kernel.org> Link: https://lore.kernel.org/r/DJHGQN654CJR.281FZSV2S5AI8@kernel.org Signed-off-by: Zhi Wang <zhiw@nvidia.com> --- drivers/fwctl/Kconfig | 12 + rust/bindings/bindings_helper.h | 1 + rust/helpers/fwctl.c | 17 + rust/helpers/helpers.c | 3 +- rust/kernel/fwctl.rs | 540 ++++++++++++++++++++++++++++++++ rust/kernel/lib.rs | 2 + 6 files changed, 574 insertions(+), 1 deletion(-) create mode 100644 rust/helpers/fwctl.c create mode 100644 rust/kernel/fwctl.rs diff --git a/drivers/fwctl/Kconfig b/drivers/fwctl/Kconfig index XXXXXXX..XXXXXXX 100644 --- a/drivers/fwctl/Kconfig +++ b/drivers/fwctl/Kconfig @@ -XXX,XX +XXX,XX @@ menuconfig FWCTL fit neatly into an existing subsystem. if FWCTL + +config RUST_FWCTL_ABSTRACTIONS + bool "Rust fwctl abstractions" + depends on RUST + help + This enables the Rust abstractions for the fwctl device firmware + access framework. It provides safe wrappers around struct fwctl_device + and struct fwctl_uctx, allowing Rust drivers to register fwctl devices + and implement their control and RPC logic in safe Rust. + + If unsure, say N. + config FWCTL_BNXT tristate "bnxt control fwctl driver" depends on BNXT diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h index XXXXXXX..XXXXXXX 100644 --- a/rust/bindings/bindings_helper.h +++ b/rust/bindings/bindings_helper.h @@ -XXX,XX +XXX,XX @@ #include <linux/fdtable.h> #include <linux/file.h> #include <linux/firmware.h> +#include <linux/fwctl.h> #include <linux/fs.h> #include <linux/i2c.h> #include <linux/interrupt.h> diff --git a/rust/helpers/fwctl.c b/rust/helpers/fwctl.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/rust/helpers/fwctl.c @@ -XXX,XX +XXX,XX @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/fwctl.h> + +#if IS_ENABLED(CONFIG_RUST_FWCTL_ABSTRACTIONS) + +__rust_helper struct fwctl_device *rust_helper_fwctl_get(struct fwctl_device *fwctl) +{ + return fwctl_get(fwctl); +} + +__rust_helper void rust_helper_fwctl_put(struct fwctl_device *fwctl) +{ + fwctl_put(fwctl); +} + +#endif diff --git a/rust/helpers/helpers.c b/rust/helpers/helpers.c index XXXXXXX..XXXXXXX 100644 --- a/rust/helpers/helpers.c +++ b/rust/helpers/helpers.c @@ -XXX,XX +XXX,XX @@ #include "drm.c" #include "drm_gpuvm.c" #include "err.c" -#include "irq.c" #include "fs.c" +#include "fwctl.c" #include "gpu.c" #include "io.c" +#include "irq.c" #include "jump_label.c" #include "kunit.c" #include "list.c" diff --git a/rust/kernel/fwctl.rs b/rust/kernel/fwctl.rs new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/rust/kernel/fwctl.rs @@ -XXX,XX +XXX,XX @@ +// SPDX-License-Identifier: GPL-2.0-only + +//! Abstractions for the fwctl subsystem. +//! +//! C header: `include/linux/fwctl.h` + +use crate::{ + bindings, + container_of, + device, + prelude::*, + sync::aref::{ + ARef, + AlwaysRefCounted, // + }, + types::{ + ForLt, + Opaque, // + }, // +}; +use core::{ + cell::UnsafeCell, + marker::PhantomData, + mem, + ptr::NonNull, + slice, // +}; + +/// Represents a fwctl device type. +/// +/// Corresponds to the C `enum fwctl_device_type`. +#[repr(u32)] +#[derive(Copy, Clone, Debug, Eq, PartialEq)] +pub enum DeviceType { + /// Mellanox ConnectX (mlx5) device. + Mlx5 = bindings::fwctl_device_type_FWCTL_DEVICE_TYPE_MLX5, + /// CXL (Compute Express Link) device. + Cxl = bindings::fwctl_device_type_FWCTL_DEVICE_TYPE_CXL, + /// AMD/Pensando PDS device. + Pds = bindings::fwctl_device_type_FWCTL_DEVICE_TYPE_PDS, + /// Broadcom NetXtreme (bnxt) device. + Bnxt = bindings::fwctl_device_type_FWCTL_DEVICE_TYPE_BNXT, +} + +impl From<DeviceType> for u32 { + fn from(device_type: DeviceType) -> Self { + device_type as u32 + } +} + +/// Scope of access for an RPC request. +/// +/// Corresponds to the C `enum fwctl_rpc_scope`. +#[repr(u32)] +#[derive(Copy, Clone, Debug, Eq, PartialEq)] +pub enum RpcScope { + /// Read/write access to device configuration. + Configuration = bindings::fwctl_rpc_scope_FWCTL_RPC_CONFIGURATION, + /// Read-only access to debug information. + DebugReadOnly = bindings::fwctl_rpc_scope_FWCTL_RPC_DEBUG_READ_ONLY, + /// Write access to lockdown-compatible debug information. + DebugWrite = bindings::fwctl_rpc_scope_FWCTL_RPC_DEBUG_WRITE, + /// Full read/write access to all debug information (requires `CAP_SYS_RAWIO`). + DebugWriteFull = bindings::fwctl_rpc_scope_FWCTL_RPC_DEBUG_WRITE_FULL, +} + +impl TryFrom<u32> for RpcScope { + type Error = Error; + + fn try_from(value: u32) -> Result<Self, Error> { + match value { + v if v == Self::Configuration as u32 => Ok(Self::Configuration), + v if v == Self::DebugReadOnly as u32 => Ok(Self::DebugReadOnly), + v if v == Self::DebugWrite as u32 => Ok(Self::DebugWrite), + v if v == Self::DebugWriteFull as u32 => Ok(Self::DebugWriteFull), + _ => Err(ENOTSUPP), + } + } +} + +/// Response from a [`Operations::fw_rpc`] call. +pub enum FwRpcResponse { + /// Reuse the input buffer as the output, with the given output length. + InPlace(usize), + /// Return a newly allocated buffer as the output. + NewBuffer(KVec<u8>), +} + +/// Trait implemented by each Rust driver that integrates with the fwctl subsystem. +/// +/// The implementing type **is** the per-FD user context: one instance is +/// created for each `open()` call and dropped when the FD is closed. +/// +/// Each implementation corresponds to a specific device type and provides the +/// vtable used by the core `fwctl` layer to manage per-FD user contexts and +/// handle RPC requests. +pub trait Operations: Sized + Send + Sync { + /// Data owned by the [`Registration`] and accessible during callbacks. + /// + /// This [`ForLt`](trait@ForLt) type is tied to the parent bus device binding scope. Drivers + /// use it to store references to resources bound to this scope, such as PCI BARs or typed bus + /// device references. + type RegistrationData: ForLt; + + /// fwctl device type identifier. + const DEVICE_TYPE: DeviceType; + + /// Called when a new user context is opened. + /// + /// Returns a [`PinInit`] initializer for `Self`. The instance is dropped + /// automatically when the FD is closed (after [`close`](Self::close)). + fn open<'a>( + device: &'a Device<Self>, + reg_data: &'a <Self::RegistrationData as ForLt>::Of<'a>, + ) -> impl PinInit<Self, Error>; + + /// Called when the user context is closed. + /// + /// The driver may perform additional cleanup here that requires access + /// to the owning [`Device`]. `Self` is dropped automatically after this + /// returns. + fn close<'a>( + _this: Pin<&mut Self>, + _device: &'a Device<Self>, + _reg_data: &'a <Self::RegistrationData as ForLt>::Of<'a>, + ) { + } + + /// Return device information to userspace. + /// + /// The default implementation returns no device-specific data. + fn info<'a>( + _this: Pin<&Self>, + _device: &'a Device<Self>, + _reg_data: &'a <Self::RegistrationData as ForLt>::Of<'a>, + ) -> Result<KVec<u8>, Error> { + Ok(KVec::new()) + } + + /// Handle a userspace RPC request. + fn fw_rpc<'a>( + this: Pin<&Self>, + device: &'a Device<Self>, + reg_data: &'a <Self::RegistrationData as ForLt>::Of<'a>, + scope: RpcScope, + rpc_in: &mut [u8], + ) -> Result<FwRpcResponse, Error>; +} + +/// A fwctl device. +/// +/// `#[repr(C)]` with the `fwctl_device` at offset 0, matching the C `fwctl_alloc_device()` layout +/// convention. Contains a pointer to the [`Registration`]'s data, set at registration time and +/// cleared on unregistration. +/// +/// # Invariants +/// +/// - `dev` is embedded at offset 0 and is initialised by fwctl. +/// - The fwctl refcount owns the allocation lifetime. +/// - `registration_data` is either `NonNull::dangling()` (before registration / after +/// unregistration) or points to valid data owned by the [`Registration`]. +#[repr(C)] +pub struct Device<T: Operations> { + dev: Opaque<bindings::fwctl_device>, + registration_data: UnsafeCell<NonNull<<T::RegistrationData as ForLt>::Of<'static>>>, +} + +impl<T: Operations> Device<T> { + /// Allocate a new fwctl device. + /// + /// Returns an [`ARef`] that can be passed to [`Registration::new()`] + /// to make the device visible to userspace. + pub fn new(parent: &device::Device<device::Bound>) -> Result<ARef<Self>> { + const_assert!( + core::mem::offset_of!(Self, dev) == 0, + "struct fwctl_device must be at offset 0" + ); + + let ops = core::ptr::from_ref::<bindings::fwctl_ops>(&VTable::<T>::VTABLE).cast_mut(); + + // SAFETY: `ops` is static, `parent` is bound, and `size` covers the full `Device<T>`. + let raw = unsafe { + bindings::_fwctl_alloc_device(parent.as_raw(), ops, core::mem::size_of::<Self>()) + }; + + if raw.is_null() { + return Err(ENOMEM); + } + + let this = raw.cast::<Self>(); + + // INVARIANT: Set `registration_data` to dangling (no registration yet). + // SAFETY: `this` points to the allocation just returned by fwctl. + unsafe { + core::ptr::addr_of_mut!((*this).registration_data) + .write(UnsafeCell::new(NonNull::dangling())); + }; + + // SAFETY: `raw` owns the initial reference. + Ok(unsafe { ARef::from_raw(NonNull::new_unchecked(this)) }) + } + + fn as_raw(&self) -> *mut bindings::fwctl_device { + self.dev.get() + } + + /// # Safety + /// + /// `ptr` must point to a valid `fwctl_device` embedded in a `Device<T>`. + unsafe fn from_raw<'a>(ptr: *mut bindings::fwctl_device) -> &'a Self { + // SAFETY: The caller upholds the offset-0 `Device<T>` invariant. + unsafe { &*ptr.cast() } + } + + /// Returns a reference to the registration data. + /// + /// # Safety + /// + /// The caller must ensure the device is registered, i.e. that this is called within a fwctl + /// callback protected by `registration_lock`. The pointer cast from `Of<'static>` to `Of<'_>` + /// is sound because [`ForLt`] guarantees covariance. + unsafe fn registration_data_unchecked(&self) -> &<T::RegistrationData as ForLt>::Of<'_> { + // SAFETY: Caller guarantees the device is registered, so the pointer is valid. + // Lifetimes do not affect layout, so the cast is sound. + unsafe { (*self.registration_data.get()).cast::<_>().as_ref() } + } +} + +impl<T: Operations> AsRef<device::Device> for Device<T> { + fn as_ref(&self) -> &device::Device { + // SAFETY: `self` contains a live fwctl_device. + let dev = unsafe { core::ptr::addr_of_mut!((*self.as_raw()).dev) }; + // SAFETY: The embedded device is initialised by fwctl. + unsafe { device::Device::from_raw(dev) } + } +} + +// SAFETY: `fwctl_get` increments the refcount of a valid fwctl_device. +// `fwctl_put` decrements it and frees the device when it reaches zero. +unsafe impl<T: Operations> AlwaysRefCounted for Device<T> { + fn inc_ref(&self) { + // SAFETY: `self` holds a live reference. + unsafe { bindings::fwctl_get(self.as_raw()) }; + } + + unsafe fn dec_ref(obj: NonNull<Self>) { + // SAFETY: The caller owns a live reference. + unsafe { bindings::fwctl_put(obj.cast().as_ptr()) }; + } +} + +// SAFETY: `Device<T>` is refcounted by the fwctl core and may be released from any thread. +unsafe impl<T: Operations> Send for Device<T> {} + +// SAFETY: Shared access to the embedded `fwctl_device` is protected by the fwctl core. The +// `registration_data` field is only mutated before registration and after unregistration (both +// single-threaded with respect to callbacks). +unsafe impl<T: Operations> Sync for Device<T> {} + +/// A registered fwctl device. +/// +/// Carries the lifetime `'a` of the parent device to ensure that [`fwctl_unregister`] runs (via +/// [`Drop`]) before the parent driver unbinds. Owns the +/// [`RegistrationData`](Operations::RegistrationData) which is accessible during callbacks via the +/// pointer stored in [`Device`]. +/// +/// On drop the device is unregistered (all user contexts are closed and `ops` is set to `NULL`) +/// and the registration data is dropped. +/// +/// [`fwctl_unregister`]: srctree/drivers/fwctl/main.c +pub struct Registration<'a, T: Operations> { + dev: ARef<Device<T>>, + _reg_data: Pin<KBox<<T::RegistrationData as ForLt>::Of<'a>>>, +} + +impl<'a, T: Operations> Registration<'a, T> +where + for<'b> <T::RegistrationData as ForLt>::Of<'b>: Send + Sync, +{ + /// Register a previously allocated fwctl device with the given registration data. + /// + /// The `reg_data` is owned by the registration and accessible during callbacks via + /// `Device::registration_data_unchecked()`. + /// + /// # Safety + /// + /// Callers must not `mem::forget()` the returned [`Registration`] or otherwise prevent its + /// [`Drop`] implementation from running, since `fwctl_unregister` must be called before the + /// parent device is unbound. + /// + /// `dev` must be an unregistered [`Device`] that is not associated with any live + /// [`Registration`], and no other thread may attempt to register the same device concurrently. + pub unsafe fn new( + _parent: &'a device::Device<device::Bound>, + dev: &Device<T>, + reg_data: impl PinInit<<T::RegistrationData as ForLt>::Of<'a>, Error>, + ) -> Result<Self> { + let reg_data: Pin<KBox<<T::RegistrationData as ForLt>::Of<'a>>> = + KBox::pin_init(reg_data, GFP_KERNEL)?; + + // Store the registration data pointer in the device before registration, so that it is + // visible once callbacks can be invoked. + // + // SAFETY: Lifetimes do not affect layout, so the pointer cast is sound. + let ptr: NonNull<<T::RegistrationData as ForLt>::Of<'static>> = + unsafe { mem::transmute(NonNull::from(Pin::get_ref(reg_data.as_ref()))) }; + + // SAFETY: No concurrent access; the device is not yet registered. + unsafe { *dev.registration_data.get() = ptr }; + + // SAFETY: `dev` is a valid fwctl_device backed by an ARef. + let ret = unsafe { bindings::fwctl_register(dev.as_raw()) }; + if ret != 0 { + // SAFETY: No concurrent readers; registration failed. + unsafe { *dev.registration_data.get() = NonNull::dangling() }; + return Err(Error::from_errno(ret)); + } + + Ok(Self { + dev: dev.into(), + _reg_data: reg_data, + }) + } +} + +impl<T: Operations> Drop for Registration<'_, T> { + fn drop(&mut self) { + // SAFETY: The Registration lifetime guarantees that the parent device is still bound. + // `fwctl_unregister` takes the write lock, closes all user contexts, and sets ops=NULL. + // After it returns, no callbacks can be running or will run. + unsafe { bindings::fwctl_unregister(self.dev.as_raw()) }; + + // SAFETY: `fwctl_unregister` guarantees no concurrent readers. + unsafe { *self.dev.registration_data.get() = NonNull::dangling() }; + + // `self._reg_data` is dropped here, after callbacks have stopped. + } +} + +// SAFETY: `Registration` can be sent between threads; the underlying fwctl_device uses internal +// locking. +unsafe impl<T: Operations> Send for Registration<'_, T> {} + +// SAFETY: `Registration` provides no mutable access; the underlying fwctl_device is protected by +// internal locking. +unsafe impl<T: Operations> Sync for Registration<'_, T> {} + +/// Internal per-FD user context wrapping `struct fwctl_uctx` and `T`. +/// +/// Not exposed to drivers; they work with `&T` / `Pin<&mut T>` directly. +#[repr(C)] +#[pin_data] +struct UserCtx<T: Operations> { + #[pin] + fwctl_uctx: Opaque<bindings::fwctl_uctx>, + #[pin] + uctx: T, +} + +impl<T: Operations> UserCtx<T> { + /// # Safety + /// + /// `ptr` must point to a `fwctl_uctx` embedded in a live `UserCtx<T>`. + unsafe fn from_raw<'a>(ptr: *mut bindings::fwctl_uctx) -> &'a Self { + // SAFETY: The caller upholds the `UserCtx<T>` embedding invariant. + unsafe { &*container_of!(Opaque::cast_from(ptr), Self, fwctl_uctx) } + } + + /// # Safety + /// + /// `ptr` must point to a `fwctl_uctx` embedded in a live `UserCtx<T>`. + /// The caller must ensure exclusive access to the `UserCtx<T>`. + unsafe fn from_raw_mut<'a>(ptr: *mut bindings::fwctl_uctx) -> &'a mut Self { + // SAFETY: The caller upholds the embedding and exclusivity invariants. + unsafe { &mut *container_of!(Opaque::cast_from(ptr), Self, fwctl_uctx).cast_mut() } + } + + /// Returns a reference to the fwctl [`Device`] that owns this context. + fn device(&self) -> &Device<T> { + // SAFETY: fwctl initialises this pointer before any driver callback. + let raw_fwctl = unsafe { (*self.fwctl_uctx.get()).fwctl }; + // SAFETY: Rust fwctl devices use the offset-0 `Device<T>` layout. + unsafe { Device::from_raw(raw_fwctl) } + } +} + +/// Static vtable mapping Rust trait methods to C callbacks. +pub struct VTable<T: Operations>(PhantomData<T>); + +impl<T: Operations> VTable<T> { + /// The fwctl operations vtable for this driver type. + pub const VTABLE: bindings::fwctl_ops = bindings::fwctl_ops { + device_type: T::DEVICE_TYPE as u32, + uctx_size: core::mem::size_of::<UserCtx<T>>(), + open_uctx: Some(Self::open_uctx_callback), + close_uctx: Some(Self::close_uctx_callback), + info: Some(Self::info_callback), + fw_rpc: Some(Self::fw_rpc_callback), + }; + + /// # Safety + /// + /// `uctx` must be a valid `fwctl_uctx` embedded in a `UserCtx<T>` with + /// sufficient allocated space for the uctx field. + unsafe extern "C" fn open_uctx_callback(uctx: *mut bindings::fwctl_uctx) -> ffi::c_int { + const_assert!( + core::mem::offset_of!(UserCtx<T>, fwctl_uctx) == 0, + "struct fwctl_uctx must be at offset 0" + ); + + // SAFETY: fwctl sets this pointer before calling `open_uctx`. + let raw_fwctl = unsafe { (*uctx).fwctl }; + // SAFETY: Rust fwctl devices use the offset-0 `Device<T>` layout. + let device = unsafe { Device::<T>::from_raw(raw_fwctl) }; + + // SAFETY: open_uctx is called under registration_lock read; the device is registered. + let reg_data = unsafe { device.registration_data_unchecked() }; + + let initializer = T::open(device, reg_data); + + let uctx_offset = core::mem::offset_of!(UserCtx<T>, uctx); + // SAFETY: `uctx_size` reserves space for the full `UserCtx<T>`. + let uctx_ptr: *mut T = unsafe { uctx.cast::<u8>().add(uctx_offset).cast() }; + + // SAFETY: `uctx_ptr` addresses the uninitialised pinned context. + match unsafe { initializer.__pinned_init(uctx_ptr.cast()) } { + Ok(()) => 0, + Err(e) => e.to_errno(), + } + } + + /// # Safety + /// + /// `uctx` must point to a fully initialised `UserCtx<T>`. + unsafe extern "C" fn close_uctx_callback(uctx: *mut bindings::fwctl_uctx) { + // SAFETY: fwctl keeps the owning device live for this callback. + let device = unsafe { Device::<T>::from_raw((*uctx).fwctl) }; + + // SAFETY: close_uctx is called under registration_lock write (from fwctl_unregister) or + // under registration_lock read (from fwctl_fops_release); the device is registered. + let reg_data = unsafe { device.registration_data_unchecked() }; + + // SAFETY: close is called for an opened Rust user context. + let ctx = unsafe { UserCtx::<T>::from_raw_mut(uctx) }; + + { + // SAFETY: fwctl never moves an opened user context. + let pinned = unsafe { Pin::new_unchecked(&mut ctx.uctx) }; + T::close(pinned, device, reg_data); + } + + // SAFETY: close is the last callback before fwctl frees the allocation. + unsafe { core::ptr::drop_in_place(&mut ctx.uctx) }; + } + + /// # Safety + /// + /// `uctx` must point to a fully initialised `UserCtx<T>`. + /// `length` must be a valid pointer. + unsafe extern "C" fn info_callback( + uctx: *mut bindings::fwctl_uctx, + length: *mut usize, + ) -> *mut ffi::c_void { + // SAFETY: info is called for an opened Rust user context. + let ctx = unsafe { UserCtx::<T>::from_raw(uctx) }; + let device = ctx.device(); + + // SAFETY: info is called under registration_lock read; the device is registered. + let reg_data = unsafe { device.registration_data_unchecked() }; + + // SAFETY: fwctl never moves an opened user context. + let pinned = unsafe { Pin::new_unchecked(&ctx.uctx) }; + + match T::info(pinned, device, reg_data) { + Ok(kvec) if kvec.is_empty() => { + // SAFETY: `length` is a valid out-parameter. + unsafe { *length = 0 }; + // Return NULL for empty data; kfree(NULL) is safe. + core::ptr::null_mut() + } + Ok(kvec) => { + let (ptr, len, _cap) = kvec.into_raw_parts(); + // SAFETY: `length` is a valid out-parameter. + unsafe { *length = len }; + ptr.cast::<ffi::c_void>() + } + Err(e) => Error::to_ptr(e), + } + } + + /// # Safety + /// + /// `uctx` must point to a fully initialised `UserCtx<T>`. + /// `rpc_in` must be valid for `in_len` bytes. `out_len` must be valid. + unsafe extern "C" fn fw_rpc_callback( + uctx: *mut bindings::fwctl_uctx, + scope: u32, + rpc_in: *mut ffi::c_void, + in_len: usize, + out_len: *mut usize, + ) -> *mut ffi::c_void { + let scope = match RpcScope::try_from(scope) { + Ok(s) => s, + Err(e) => return Error::to_ptr(e), + }; + + // SAFETY: RPC is called for an opened Rust user context. + let ctx = unsafe { UserCtx::<T>::from_raw(uctx) }; + let device = ctx.device(); + + // SAFETY: fw_rpc is called under registration_lock read; the device is registered. + let reg_data = unsafe { device.registration_data_unchecked() }; + + // SAFETY: fwctl passes a valid in/out buffer for this callback. + let rpc_in_slice: &mut [u8] = + unsafe { slice::from_raw_parts_mut(rpc_in.cast::<u8>(), in_len) }; + + // SAFETY: fwctl never moves an opened user context. + let pinned = unsafe { Pin::new_unchecked(&ctx.uctx) }; + + match T::fw_rpc(pinned, device, reg_data, scope, rpc_in_slice) { + Ok(FwRpcResponse::InPlace(len)) => { + if len > in_len { + return Error::to_ptr(EINVAL); + } + + // SAFETY: `out_len` is valid. + unsafe { *out_len = len }; + rpc_in + } + Ok(FwRpcResponse::NewBuffer(kvec)) => { + let (ptr, len, _cap) = kvec.into_raw_parts(); + // SAFETY: `out_len` is valid. + unsafe { *out_len = len }; + ptr.cast::<ffi::c_void>() + } + Err(e) => Error::to_ptr(e), + } + } +} diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index XXXXXXX..XXXXXXX 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -XXX,XX +XXX,XX @@ pub mod firmware; pub mod fmt; pub mod fs; +#[cfg(CONFIG_RUST_FWCTL_ABSTRACTIONS)] +pub mod fwctl; #[cfg(CONFIG_GPU_BUDDY = "y")] pub mod gpu; #[cfg(CONFIG_I2C = "y")] -- 2.51.0 In the NVIDIA vGPU RFC [1], the vGPU type blobs must be provided to the GSP before userspace can enumerate available vGPU types and create vGPU instances. The original design relied on the firmware loading interface, but fwctl is a more natural fit for this use case, as it is designed for uploading configuration or firmware data required before the device becomes operational. This series introduces a Rust abstraction over the fwctl subsystem, providing safe and idiomatic bindings. The series is a single patch. `Registration` owns the callback-visible driver data and ties it to the parent device binding lifetime using a GAT on `Operations`. `Device<T>` remains the refcounted fwctl object, while callbacks borrow registration data for the callback lifetime. The Rust fwctl module provides: - `Operations`: driver-specific `open()`, `close()`, `info()`, and `fw_rpc()` callbacks. The implementing type is the per-FD user context. - `RegistrationData<'a>`: a GAT associated type owned by `Registration` and borrowed by callbacks under the fwctl registration lock. - `Device<T>`: a refcounted wrapper around `struct fwctl_device`. - `Registration<T>`: registration and automatic unregistration, including verification that the supplied bound parent is the device's actual parent. - `RpcScope` and `FwRpcResponse`: type-safe RPC scope and response handling. The safe callback receives the userspace output-buffer size as a numeric value, while raw output-length pointer handling remains confined to the abstraction. `DeviceType` intentionally models every valid, non-error `fwctl_device_type` UAPI value rather than the set of existing Rust drivers. This preserves a typed interface while making `FWCTL_DEVICE_TYPE_ERROR` and arbitrary integers unrepresentable. Device-specific Rust consumers can add new variants together with their UAPI allocation. `rust/kernel/lib.rs` conditionally includes the module under `CONFIG_RUST_FWCTL_ABSTRACTIONS`. This series is based on drm-rust-next commit d85845b64c00 ("gpu: nova-core: build SetRegistry entries dynamically"), so Nova can build on it directly. Danilo suggested taking it through the DRM tree; Jason, please let us know if you would prefer it to go through the fwctl tree. v7: - Rebase onto the latest drm-rust-next. - Pad fwctl allocation sizes for Rust alignment. (Danilo, Alexandre) - Validate the actual parent in `Registration::new()`. (Alexandre, Danilo) - Add an HRTB helper and remove the lifetime transmute. (Alexandre) - Pass `max_output_len` to safe RPC callbacks. (Alexandre, Joel) - Write `out_len` once. (Alexandre) - Return `NULL` for empty RPC replies. (Danilo) - Convert allocation results to `NonNull` immediately. (Alexandre, Gary) - Simplify callback lifetimes and raw pointer use. (Danilo, Alexandre) - Make `VTable` and `VTABLE` private. (Danilo) - Remove manual `Registration` `Send`/`Sync` impls. (Alexandre) - Tighten the rustdoc and safety comments. (Alexandre) - Document `DeviceType` as the non-error UAPI domain. (Danilo) - Update the FWCTL `MAINTAINERS` entry. (Miguel, Danilo) - Add Danilo's `Signed-off-by` trailer. (Danilo) Link to v6: [2] Links to the v6 reviews: [8] [9] [10] v6: - Use a GAT lifetime for `Operations::RegistrationData<'a>` instead of requiring `ForLt`. (Danilo) - Require `FWCTL=y` for `RUST_FWCTL_ABSTRACTIONS`. - Add `#[inline]` to small forwarding helpers. - Return `EINVAL` for an unrecognized RPC scope. Link to Danilo's GAT diff: [3] Link to v5: [4] v5: - Rebase on top of drm-rust-next. - Adopt Danilo's higher-ranked lifetime model for callback-visible registration data. - Drop the fwctl core release hook patch. - Reject `FwRpcResponse::InPlace(len)` when `len` exceeds the input buffer. - Add the missing BNXT fwctl device type. - Document the registration exclusivity requirements. Link to v4: [5] v4: - Rebase on top of drm-rust-next. - Split out the fwctl core release hook. - Drop the already-merged fwctl init-ordering change. - Add compile-time layout checks. (Jason) - Use `const_assert!()` for generic layout assertions. - Require `Operations` and its data to be `Send + Sync`. (Danilo) - Pass pinned shared references to `info()` and `fw_rpc()`. (Danilo) - Return an initializer directly from `open()`. (Danilo) - Fix clippy and rustdoc warnings. (Danilo) Link to v3: [6] v3: - Use an enum for the RPC response. (Joel) - Remove the test UAPI and sample driver. (Jason, John) - Remove `DeviceType::Error` and add fwctl reference helpers. (Gary) - Refine device-private data ownership. (Danilo) - Separate allocation and registration. (Jason) - Tie registration to a bound parent device. (Danilo) - Make the implementing type the per-FD context. (Danilo) An example nova-core fwctl consumer is available at [7]. v2: - Use a Rust helper for `fwctl_put()`. (Jason, Danilo) - Guarantee registration lifetime management. (Jason, Danilo) - Rename the main abstraction types and use a typed device identifier. (Danilo) - Improve pointer casts, reference counting, safety comments, and clippy cleanliness. v1: - Initial submission introducing fwctl Rust abstractions. [1] https://lore.kernel.org/all/20250903221111.3866249-1-zhiw@nvidia.com/ [2] https://lore.kernel.org/r/20260629150156.3169384-1-zhiw@nvidia.com/ [3] https://lore.kernel.org/r/DJJW7X4ESDSM.QCVYK2FC7ZR3@kernel.org/ [4] https://lore.kernel.org/r/20260629103946.3117152-1-zhiw@nvidia.com/ [5] https://lore.kernel.org/r/20260624091758.1678092-1-zhiw@nvidia.com/ [6] https://lore.kernel.org/r/20260217204909.211793-1-zhiw@nvidia.com/ [7] https://lore.kernel.org/r/20260305190936.398590-1-zhiw@nvidia.com/ [8] https://lore.kernel.org/r/DJQ1KX0O60J7.Q3RUZ46V153O@kernel.org/ [9] https://lore.kernel.org/r/DJR76RV8NEKN.2MP5M9Z66U4TY@nvidia.com/ [10] https://lore.kernel.org/r/DJRH022OWTTK.1KPQDQHKG7CTF@garyguo.net/ Zhi Wang (1): rust: introduce abstractions for fwctl MAINTAINERS | 2 + drivers/fwctl/Kconfig | 12 + rust/bindings/bindings_helper.h | 1 + rust/helpers/fwctl.c | 17 + rust/helpers/helpers.c | 3 +- rust/kernel/fwctl.rs | 578 ++++++++++++++++++++++++++++++++ rust/kernel/lib.rs | 2 + 7 files changed, 614 insertions(+), 1 deletion(-) create mode 100644 rust/helpers/fwctl.c create mode 100644 rust/kernel/fwctl.rs base-commit: d85845b64c0020b2812243a22fa79d57cc1c1e38 -- 2.51.0 Introduce safe Rust wrappers around struct fwctl_device and struct fwctl_uctx. This lets Rust drivers register fwctl devices and implement firmware RPC callbacks through a typed trait interface. The abstraction keeps lifetime and reference-count handling inside the wrapper, exposes pinned per-FD user contexts to drivers, and validates the layout assumptions required by the C fwctl allocation model. Allocation sizes are padded so the kmalloc-backed C allocations also satisfy Rust alignment requirements. Registration owns driver private data with a lifetime tied to the bound parent device and verifies the parent identity before registration. Callbacks access that data through a higher-ranked closure, preventing its erased lifetime from escaping, while Device remains only the refcounted fwctl object. This avoids requiring Rust drop glue from the fwctl_device release path after unregister or module teardown. RPC callbacks receive typed scope information, a mutable request/response buffer, and the userspace output-buffer size. Response pointer conversion, length validation, and raw output-length handling remain inside the abstraction. Add the Rust sources to the FWCTL MAINTAINERS entry. Co-developed-by: Danilo Krummrich <dakr@kernel.org> Signed-off-by: Danilo Krummrich <dakr@kernel.org> Link: https://lore.kernel.org/r/DJJW7X4ESDSM.QCVYK2FC7ZR3@kernel.org Link: https://lore.kernel.org/r/20260629150156.3169384-2-zhiw@nvidia.com Signed-off-by: Zhi Wang <zhiw@nvidia.com> --- MAINTAINERS | 2 + drivers/fwctl/Kconfig | 12 + rust/bindings/bindings_helper.h | 1 + rust/helpers/fwctl.c | 17 + rust/helpers/helpers.c | 3 +- rust/kernel/fwctl.rs | 578 ++++++++++++++++++++++++++++++++ rust/kernel/lib.rs | 2 + 7 files changed, 614 insertions(+), 1 deletion(-) create mode 100644 rust/helpers/fwctl.c create mode 100644 rust/kernel/fwctl.rs diff --git a/MAINTAINERS b/MAINTAINERS index XXXXXXX..XXXXXXX 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -XXX,XX +XXX,XX @@ F: Documentation/userspace-api/fwctl/ F: drivers/fwctl/ F: include/linux/fwctl.h F: include/uapi/fwctl/ +F: rust/helpers/fwctl.c +F: rust/kernel/fwctl.rs FWCTL BNXT DRIVER M: Pavan Chebbi <pavan.chebbi@broadcom.com> diff --git a/drivers/fwctl/Kconfig b/drivers/fwctl/Kconfig index XXXXXXX..XXXXXXX 100644 --- a/drivers/fwctl/Kconfig +++ b/drivers/fwctl/Kconfig @@ -XXX,XX +XXX,XX @@ menuconfig FWCTL fit neatly into an existing subsystem. if FWCTL + +config RUST_FWCTL_ABSTRACTIONS + bool "Rust fwctl abstractions" + depends on RUST && FWCTL=y + help + This enables the Rust abstractions for the fwctl device firmware + access framework. It provides safe wrappers around struct fwctl_device + and struct fwctl_uctx, allowing Rust drivers to register fwctl devices + and implement their control and RPC logic in safe Rust. + + If unsure, say N. + config FWCTL_BNXT tristate "bnxt control fwctl driver" depends on BNXT diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h index XXXXXXX..XXXXXXX 100644 --- a/rust/bindings/bindings_helper.h +++ b/rust/bindings/bindings_helper.h @@ -XXX,XX +XXX,XX @@ #include <linux/fdtable.h> #include <linux/file.h> #include <linux/firmware.h> +#include <linux/fwctl.h> #include <linux/fs.h> #include <linux/i2c.h> #include <linux/interrupt.h> diff --git a/rust/helpers/fwctl.c b/rust/helpers/fwctl.c new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/rust/helpers/fwctl.c @@ -XXX,XX +XXX,XX @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/fwctl.h> + +#if IS_ENABLED(CONFIG_RUST_FWCTL_ABSTRACTIONS) + +__rust_helper struct fwctl_device *rust_helper_fwctl_get(struct fwctl_device *fwctl) +{ + return fwctl_get(fwctl); +} + +__rust_helper void rust_helper_fwctl_put(struct fwctl_device *fwctl) +{ + fwctl_put(fwctl); +} + +#endif diff --git a/rust/helpers/helpers.c b/rust/helpers/helpers.c index XXXXXXX..XXXXXXX 100644 --- a/rust/helpers/helpers.c +++ b/rust/helpers/helpers.c @@ -XXX,XX +XXX,XX @@ #include "drm.c" #include "drm_gpuvm.c" #include "err.c" -#include "irq.c" #include "fs.c" +#include "fwctl.c" #include "gpu.c" #include "io.c" +#include "irq.c" #include "jump_label.c" #include "kunit.c" #include "list.c" diff --git a/rust/kernel/fwctl.rs b/rust/kernel/fwctl.rs new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/rust/kernel/fwctl.rs @@ -XXX,XX +XXX,XX @@ +// SPDX-License-Identifier: GPL-2.0-only + +//! Abstractions for the fwctl subsystem. +//! +//! C header: `include/linux/fwctl.h` + +use crate::{ + bindings, + container_of, + device, + prelude::*, + sync::aref::{ + ARef, + AlwaysRefCounted, // + }, + types::Opaque, // +}; +use core::{ + alloc::Layout, + cell::UnsafeCell, + marker::PhantomData, + ptr::NonNull, + slice, // +}; + +/// Returns a kmalloc-compatible allocation size for `T`. +const fn kmalloc_aligned_size<T>() -> usize { + Layout::new::<T>().pad_to_align().size() +} + +/// Represents a fwctl device type. +/// +/// Corresponds to the C `enum fwctl_device_type`. All non-error UAPI values are represented so +/// Rust drivers can select a device type without passing an untyped integer, while +/// `FWCTL_DEVICE_TYPE_ERROR` remains unrepresentable. +#[repr(u32)] +#[derive(Copy, Clone, Debug, Eq, PartialEq)] +pub enum DeviceType { + /// Mellanox ConnectX (mlx5) device. + Mlx5 = bindings::fwctl_device_type_FWCTL_DEVICE_TYPE_MLX5, + /// CXL (Compute Express Link) device. + Cxl = bindings::fwctl_device_type_FWCTL_DEVICE_TYPE_CXL, + /// AMD/Pensando PDS device. + Pds = bindings::fwctl_device_type_FWCTL_DEVICE_TYPE_PDS, + /// Broadcom NetXtreme (bnxt) device. + Bnxt = bindings::fwctl_device_type_FWCTL_DEVICE_TYPE_BNXT, +} + +impl From<DeviceType> for u32 { + fn from(device_type: DeviceType) -> Self { + device_type as u32 + } +} + +/// Scope of access for an RPC request. +/// +/// Corresponds to the C `enum fwctl_rpc_scope`. +#[repr(u32)] +#[derive(Copy, Clone, Debug, Eq, PartialEq)] +pub enum RpcScope { + /// Read/write access to device configuration. + Configuration = bindings::fwctl_rpc_scope_FWCTL_RPC_CONFIGURATION, + /// Read-only access to debug information. + DebugReadOnly = bindings::fwctl_rpc_scope_FWCTL_RPC_DEBUG_READ_ONLY, + /// Write access to lockdown-compatible debug information. + DebugWrite = bindings::fwctl_rpc_scope_FWCTL_RPC_DEBUG_WRITE, + /// Full read/write access to all debug information (requires `CAP_SYS_RAWIO`). + DebugWriteFull = bindings::fwctl_rpc_scope_FWCTL_RPC_DEBUG_WRITE_FULL, +} + +impl TryFrom<u32> for RpcScope { + type Error = Error; + + #[inline] + fn try_from(value: u32) -> Result<Self, Error> { + match value { + v if v == Self::Configuration as u32 => Ok(Self::Configuration), + v if v == Self::DebugReadOnly as u32 => Ok(Self::DebugReadOnly), + v if v == Self::DebugWrite as u32 => Ok(Self::DebugWrite), + v if v == Self::DebugWriteFull as u32 => Ok(Self::DebugWriteFull), + _ => Err(EINVAL), + } + } +} + +/// Response from a [`Operations::fw_rpc`] call. +pub enum FwRpcResponse { + /// Reuse the input buffer as the output, with the given output length. + InPlace(usize), + /// Return a newly allocated buffer as the output. + NewBuffer(KVec<u8>), +} + +/// Trait implemented by each Rust driver that integrates with the fwctl subsystem. +/// +/// The implementing type **is** the per-FD user context: one instance is +/// created for each `open()` call and dropped when the FD is closed. +/// +/// Each implementation corresponds to a specific device type and provides the +/// vtable used by the core `fwctl` layer to manage per-FD user contexts and +/// handle RPC requests. +pub trait Operations: Sized + Send + Sync + 'static { + /// Data owned by the [`Registration`] and accessible during callbacks. + /// + /// The lifetime `'a` is tied to the [`Registration`] scope (which lives within the parent bus + /// device binding scope). Drivers use it to store references to resources bound to this scope, + /// such as PCI BARs or typed bus device references. + type RegistrationData<'a>: Send + Sync + 'a + where + Self: 'a; + + /// fwctl device type identifier. + const DEVICE_TYPE: DeviceType; + + /// Called when a new user context is opened. + /// + /// Returns a [`PinInit`] initializer for `Self`. The instance is dropped + /// automatically when the FD is closed (after [`close`](Self::close)). + fn open<'a>( + device: &Device<Self>, + reg_data: &Self::RegistrationData<'a>, + ) -> impl PinInit<Self, Error>; + + /// Called when the user context is closed. + /// + /// The driver may perform additional cleanup here that requires access + /// to the owning [`Device`]. `Self` is dropped automatically after this + /// returns. + fn close<'a>( + _this: Pin<&mut Self>, + _device: &Device<Self>, + _reg_data: &Self::RegistrationData<'a>, + ) { + } + + /// Return device information to userspace. + /// + /// The default implementation returns no device-specific data. + fn info<'a>( + _this: Pin<&Self>, + _device: &Device<Self>, + _reg_data: &Self::RegistrationData<'a>, + ) -> Result<KVec<u8>, Error> { + Ok(KVec::new()) + } + + /// Handle a userspace RPC request. + /// + /// `max_output_len` is the size of the userspace output buffer. A driver may return a larger + /// response to report the required size; the fwctl core copies only the bytes that fit and + /// reports the full response length to userspace. + fn fw_rpc<'a>( + this: Pin<&Self>, + device: &Device<Self>, + reg_data: &Self::RegistrationData<'a>, + scope: RpcScope, + rpc_buf: &mut [u8], + max_output_len: usize, + ) -> Result<FwRpcResponse, Error>; +} + +/// A fwctl device. +/// +/// `#[repr(C)]` with the `fwctl_device` at offset 0, matching the C `fwctl_alloc_device()` layout +/// convention. Contains a pointer to the [`Registration`]'s data, set at registration time and +/// cleared on unregistration. +/// +/// # Invariants +/// +/// - `dev` is embedded at offset 0 and is initialised by fwctl. +/// - The fwctl refcount owns the allocation lifetime. +/// - `registration_data` is either `NonNull::dangling()` (before registration / after +/// unregistration) or points to valid data owned by the [`Registration`]. +#[repr(C)] +pub struct Device<T: Operations> { + dev: Opaque<bindings::fwctl_device>, + registration_data: UnsafeCell<NonNull<T::RegistrationData<'static>>>, +} + +impl<T: Operations> Device<T> { + /// Allocate a new fwctl device. + /// + /// Returns an [`ARef`] that can be passed to [`Registration::new()`] + /// to make the device visible to userspace. + pub fn new(parent: &device::Device<device::Bound>) -> Result<ARef<Self>> { + const_assert!( + core::mem::offset_of!(Self, dev) == 0, + "struct fwctl_device must be at offset 0" + ); + + let size = kmalloc_aligned_size::<Self>(); + let ops = core::ptr::from_ref::<bindings::fwctl_ops>(&VTable::<T>::VTABLE).cast_mut(); + + // SAFETY: `ops` is static, `parent` is bound, and `size` is padded so the allocation made + // by `_fwctl_alloc_device` satisfies the size and alignment required by `Device<T>`. + let raw = unsafe { bindings::_fwctl_alloc_device(parent.as_raw(), ops, size) }; + let this = NonNull::new(raw.cast::<Self>()).ok_or(ENOMEM)?; + + // INVARIANT: Set `registration_data` to dangling (no registration yet). + // SAFETY: `this` points to the allocation just returned by fwctl. + unsafe { + (&raw mut (*this.as_ptr()).registration_data) + .write(UnsafeCell::new(NonNull::dangling())); + }; + + // SAFETY: `this` owns the initial reference. + Ok(unsafe { ARef::from_raw(this) }) + } + + #[inline] + fn as_raw(&self) -> *mut bindings::fwctl_device { + self.dev.get() + } + + /// # Safety + /// + /// `ptr` must point to a valid `fwctl_device` embedded in a [`Device<T>`]. + #[inline] + unsafe fn from_raw<'a>(ptr: *mut bindings::fwctl_device) -> &'a Self { + // SAFETY: The caller upholds the offset-0 `Device<T>` invariant. + unsafe { &*ptr.cast() } + } + + /// Invokes `f` with the registration data. + /// + /// The higher-ranked callback prevents the erased registration lifetime from escaping and + /// permits registration data that is invariant over its lifetime parameter. + /// + /// # Safety + /// + /// The caller must ensure that the device is registered and that this is called from a fwctl + /// callback protected by `registration_lock`. + #[inline] + unsafe fn with_registration_data<R>( + &self, + f: impl for<'a> FnOnce(&Device<T>, &'a T::RegistrationData<'a>) -> R, + ) -> R { + // SAFETY: Caller guarantees the device is registered, so the pointer is valid. + // Lifetimes do not affect layout. The higher-ranked callback prevents the shortened + // lifetime from escaping or being selected by the caller. + let reg_data = unsafe { + (*self.registration_data.get()) + .cast::<T::RegistrationData<'_>>() + .as_ref() + }; + + f(self, reg_data) + } +} + +impl<T: Operations> AsRef<device::Device> for Device<T> { + #[inline] + fn as_ref(&self) -> &device::Device { + // SAFETY: `self` contains a live fwctl_device. + let dev = unsafe { &raw mut (*self.as_raw()).dev }; + // SAFETY: The embedded device is initialised by fwctl. + unsafe { device::Device::from_raw(dev) } + } +} + +// SAFETY: `fwctl_get` increments the refcount of a valid fwctl_device. +// `fwctl_put` decrements it and frees the device when it reaches zero. +unsafe impl<T: Operations> AlwaysRefCounted for Device<T> { + #[inline] + fn inc_ref(&self) { + // SAFETY: `self` holds a live reference. + unsafe { bindings::fwctl_get(self.as_raw()) }; + } + + #[inline] + unsafe fn dec_ref(obj: NonNull<Self>) { + // SAFETY: The caller owns a live reference. + unsafe { bindings::fwctl_put(obj.cast().as_ptr()) }; + } +} + +// SAFETY: `Device<T>` is refcounted by the fwctl core and may be released from any thread. +unsafe impl<T: Operations> Send for Device<T> {} + +// SAFETY: Shared access to the embedded `fwctl_device` is protected by the fwctl core. The +// `registration_data` field is only mutated before registration and after unregistration (both +// single-threaded with respect to callbacks). +unsafe impl<T: Operations> Sync for Device<T> {} + +/// A registered fwctl device. +/// +/// Owns the [`RegistrationData`](Operations::RegistrationData) made available to driver callbacks. +/// The parent device lifetime ensures that [`fwctl_unregister`] runs before the parent driver +/// unbinds. +/// +/// On drop the device is unregistered (all user contexts are closed and `ops` is set to `NULL`) +/// and the registration data is dropped. +/// +/// [`fwctl_unregister`]: srctree/drivers/fwctl/main.c +pub struct Registration<'a, T: Operations> { + dev: ARef<Device<T>>, + _reg_data: Pin<KBox<T::RegistrationData<'a>>>, +} + +impl<'a, T: Operations> Registration<'a, T> { + /// Register a previously allocated fwctl device with the given registration data. + /// + /// The `reg_data` is owned by the registration and accessible during callbacks. + /// + /// # Safety + /// + /// Callers must not `mem::forget()` the returned [`Registration`] or otherwise prevent its + /// [`Drop`] implementation from running, since `fwctl_unregister` must be called before the + /// parent device is unbound. + /// + /// `dev` must be an unregistered [`Device`] that is not associated with any live + /// [`Registration`], and no other thread may attempt to register the same device concurrently. + pub unsafe fn new( + parent: &'a device::Device<device::Bound>, + dev: &Device<T>, + reg_data: impl PinInit<T::RegistrationData<'a>, Error>, + ) -> Result<Self> { + let actual_parent = dev.as_ref().parent().ok_or(EINVAL)?; + let parent_device: &device::Device = parent; + if !core::ptr::eq(actual_parent, parent_device) { + return Err(EINVAL); + } + + let reg_data: Pin<KBox<T::RegistrationData<'a>>> = KBox::pin_init(reg_data, GFP_KERNEL)?; + + // Store the registration data pointer in the device before registration, so that it is + // visible once callbacks can be invoked. The `'static` type is only an erased storage + // handle; callbacks access the pointer through a higher-ranked closure. + let ptr: NonNull<T::RegistrationData<'static>> = + NonNull::from(Pin::get_ref(reg_data.as_ref())).cast(); + + // SAFETY: No concurrent access; the device is not yet registered. + unsafe { *dev.registration_data.get() = ptr }; + + // SAFETY: `dev` is a valid fwctl_device backed by an ARef. + let ret = unsafe { bindings::fwctl_register(dev.as_raw()) }; + if ret != 0 { + // SAFETY: No concurrent readers; registration failed. + unsafe { *dev.registration_data.get() = NonNull::dangling() }; + return Err(Error::from_errno(ret)); + } + + Ok(Self { + dev: dev.into(), + _reg_data: reg_data, + }) + } +} + +impl<T: Operations> Drop for Registration<'_, T> { + fn drop(&mut self) { + // SAFETY: The Registration lifetime guarantees that the parent device is still bound. + // `fwctl_unregister` takes the write lock, closes all user contexts, and sets ops=NULL. + // After it returns, no callbacks can be running or will run. + unsafe { bindings::fwctl_unregister(self.dev.as_raw()) }; + + // SAFETY: `fwctl_unregister` guarantees no concurrent readers. + unsafe { *self.dev.registration_data.get() = NonNull::dangling() }; + + // `self._reg_data` is dropped here, after callbacks have stopped. + } +} + +/// Internal per-FD user context wrapping `struct fwctl_uctx` and `T`. +/// +/// Not exposed to drivers; they work with `&T` / `Pin<&mut T>` directly. +#[repr(C)] +#[pin_data] +struct UserCtx<T: Operations> { + #[pin] + fwctl_uctx: Opaque<bindings::fwctl_uctx>, + #[pin] + uctx: T, +} + +impl<T: Operations> UserCtx<T> { + /// # Safety + /// + /// `ptr` must point to a `fwctl_uctx` embedded in a live `UserCtx<T>`. + #[inline] + unsafe fn from_raw<'a>(ptr: *mut bindings::fwctl_uctx) -> &'a Self { + // SAFETY: The caller upholds the `UserCtx<T>` embedding invariant. + unsafe { &*container_of!(Opaque::cast_from(ptr), Self, fwctl_uctx) } + } + + /// # Safety + /// + /// `ptr` must point to a `fwctl_uctx` embedded in a live `UserCtx<T>`. + /// The caller must ensure exclusive access to the `UserCtx<T>`. + #[inline] + unsafe fn from_raw_mut<'a>(ptr: *mut bindings::fwctl_uctx) -> &'a mut Self { + // SAFETY: The caller upholds the embedding and exclusivity invariants. + unsafe { &mut *container_of!(Opaque::cast_from(ptr), Self, fwctl_uctx).cast_mut() } + } + + /// Returns a reference to the fwctl [`Device`] that owns this context. + #[inline] + fn device(&self) -> &Device<T> { + // SAFETY: fwctl initialises this pointer before any driver callback. + let raw_fwctl = unsafe { (*self.fwctl_uctx.get()).fwctl }; + // SAFETY: Rust fwctl devices use the offset-0 `Device<T>` layout. + unsafe { Device::from_raw(raw_fwctl) } + } +} + +/// Static vtable mapping Rust trait methods to C callbacks. +struct VTable<T: Operations>(PhantomData<T>); + +impl<T: Operations> VTable<T> { + /// The fwctl operations vtable for this driver type. + const VTABLE: bindings::fwctl_ops = bindings::fwctl_ops { + device_type: T::DEVICE_TYPE as u32, + uctx_size: kmalloc_aligned_size::<UserCtx<T>>(), + open_uctx: Some(Self::open_uctx_callback), + close_uctx: Some(Self::close_uctx_callback), + info: Some(Self::info_callback), + fw_rpc: Some(Self::fw_rpc_callback), + }; + + /// # Safety + /// + /// `uctx` must be a valid `fwctl_uctx` embedded in a `UserCtx<T>` with + /// sufficient allocated space for the uctx field. + unsafe extern "C" fn open_uctx_callback(uctx: *mut bindings::fwctl_uctx) -> ffi::c_int { + const_assert!( + core::mem::offset_of!(UserCtx<T>, fwctl_uctx) == 0, + "struct fwctl_uctx must be at offset 0" + ); + + // SAFETY: fwctl sets this pointer before calling `open_uctx`. + let raw_fwctl = unsafe { (*uctx).fwctl }; + // SAFETY: Rust fwctl devices use the offset-0 `Device<T>` layout. + let device = unsafe { Device::<T>::from_raw(raw_fwctl) }; + + let uctx_offset = core::mem::offset_of!(UserCtx<T>, uctx); + // SAFETY: `uctx_size` reserves space for the full `UserCtx<T>`. + let uctx_ptr: *mut T = unsafe { uctx.byte_add(uctx_offset).cast() }; + + // SAFETY: `open_uctx` is called under `registration_lock` read, so the device is + // registered. `uctx_ptr` addresses the uninitialised pinned context reserved by + // `uctx_size`. + unsafe { + device.with_registration_data(|device, reg_data| { + match T::open(device, reg_data).__pinned_init(uctx_ptr) { + Ok(()) => 0, + Err(e) => e.to_errno(), + } + }) + } + } + + /// # Safety + /// + /// `uctx` must point to a fully initialised `UserCtx<T>`. + unsafe extern "C" fn close_uctx_callback(uctx: *mut bindings::fwctl_uctx) { + // SAFETY: fwctl keeps the owning device live for this callback. + let device = unsafe { Device::<T>::from_raw((*uctx).fwctl) }; + + // SAFETY: close is called for an opened Rust user context. + let ctx = unsafe { UserCtx::<T>::from_raw_mut(uctx) }; + + // SAFETY: `close_uctx` is called under `registration_lock` write (from + // `fwctl_unregister`) or read (from `fwctl_fops_release`), so the device is registered. + // fwctl never moves an opened user context. + unsafe { + device.with_registration_data(|device, reg_data| { + T::close(Pin::new_unchecked(&mut ctx.uctx), device, reg_data); + }); + } + + // SAFETY: close is the last callback before fwctl frees the allocation. + unsafe { core::ptr::drop_in_place(&mut ctx.uctx) }; + } + + /// # Safety + /// + /// `uctx` must point to a fully initialised `UserCtx<T>`. + /// `length` must be a valid pointer. + unsafe extern "C" fn info_callback( + uctx: *mut bindings::fwctl_uctx, + length: *mut usize, + ) -> *mut ffi::c_void { + // SAFETY: info is called for an opened Rust user context. + let ctx = unsafe { UserCtx::<T>::from_raw(uctx) }; + let device = ctx.device(); + + // SAFETY: `info` is called under `registration_lock` read, so the device is registered. + // fwctl never moves an opened user context. + let result = unsafe { + device.with_registration_data(|device, reg_data| { + T::info(Pin::new_unchecked(&ctx.uctx), device, reg_data) + }) + }; + + match result { + Ok(kvec) if kvec.is_empty() => { + // SAFETY: `length` is a valid out-parameter. + unsafe { *length = 0 }; + // Return NULL for empty data; kfree(NULL) is safe. + core::ptr::null_mut() + } + Ok(kvec) => { + let (ptr, len, _cap) = kvec.into_raw_parts(); + // SAFETY: `length` is a valid out-parameter. + unsafe { *length = len }; + ptr.cast::<ffi::c_void>() + } + Err(e) => Error::to_ptr(e), + } + } + + /// # Safety + /// + /// `uctx` must point to a fully initialised `UserCtx<T>`. + /// `rpc_in` must be valid, initialised, and exclusively accessible for `in_len` bytes. + /// `out_len` must be valid for reading and writing an initialised `usize`. + unsafe extern "C" fn fw_rpc_callback( + uctx: *mut bindings::fwctl_uctx, + scope: u32, + rpc_in: *mut ffi::c_void, + in_len: usize, + out_len: *mut usize, + ) -> *mut ffi::c_void { + let scope = match RpcScope::try_from(scope) { + Ok(s) => s, + Err(e) => return Error::to_ptr(e), + }; + + // SAFETY: `out_len` points to the userspace output buffer length supplied by fwctl. + let max_output_len = unsafe { *out_len }; + + // SAFETY: RPC is called for an opened Rust user context. + let ctx = unsafe { UserCtx::<T>::from_raw(uctx) }; + let device = ctx.device(); + + // SAFETY: fwctl passes an exclusively owned buffer that is valid and initialised for + // `in_len` bytes. It remains live for the duration of this callback. + let rpc_buf: &mut [u8] = unsafe { slice::from_raw_parts_mut(rpc_in.cast::<u8>(), in_len) }; + + // SAFETY: `fw_rpc` is called under `registration_lock` read, so the device is registered. + // fwctl never moves an opened user context. + let result = unsafe { + device.with_registration_data(|device, reg_data| { + T::fw_rpc( + Pin::new_unchecked(&ctx.uctx), + device, + reg_data, + scope, + rpc_buf, + max_output_len, + ) + }) + }; + + let (response, response_len) = match result { + Ok(FwRpcResponse::InPlace(len)) => { + if len > in_len { + return Error::to_ptr(EINVAL); + } + + (rpc_in, len) + } + Ok(FwRpcResponse::NewBuffer(kvec)) if kvec.is_empty() => { + // Return NULL for empty data; kvfree(NULL) is safe. + (core::ptr::null_mut(), 0) + } + Ok(FwRpcResponse::NewBuffer(kvec)) => { + let (ptr, len, _cap) = kvec.into_raw_parts(); + (ptr.cast::<ffi::c_void>(), len) + } + Err(e) => return Error::to_ptr(e), + }; + + // SAFETY: `out_len` is a valid out-parameter. + unsafe { *out_len = response_len }; + response + } +} diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index XXXXXXX..XXXXXXX 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -XXX,XX +XXX,XX @@ pub mod firmware; pub mod fmt; pub mod fs; +#[cfg(CONFIG_RUST_FWCTL_ABSTRACTIONS)] +pub mod fwctl; #[cfg(CONFIG_GPU_BUDDY = "y")] pub mod gpu; #[cfg(CONFIG_I2C = "y")] -- 2.51.0