From nobody Tue Mar 3 05:22:16 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E646D33D6CA for ; Mon, 2 Mar 2026 23:22:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772493746; cv=none; b=lnLSVCvnFI71q4Y0t0kGV2tsBPVBP+OCVLizTU0lv1Eoxhiz3N4O8f8EvmJsY84KBqahluRwe7aDwr/hE1t6kwFmudjlT3j1deGMJ2iPdOmE/4TJ+4y+r4lHlTng4o0JaoAvrFS6dcw3hs2ptkultO0PE+Dr6LtWheEEog+sIXo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772493746; c=relaxed/simple; bh=p0VtqUDl335E7qpNFvsHIaCkonrdV9cwdRZnWfsKY7U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Ol1Wt0HXo+yme3YU6E0yIKLqtEDRth37g0iQXGzuAaZJJOONGibGaVE4N+GgvFEF+3DBtPdNmOLc7/ci/qf9ksr8GEvtM7bmpg+ZDCQDNVnCYJUNPWB7t+Xkx6eRH65D3SZscyIUSySSCR7IE0VkfdyCGWtRc/XPfDJir1munsg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=DLE7cpY0; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DLE7cpY0" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1772493744; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kMEkQtPXSQuxirFnSbNpA1+f6ekDbkk478oBEEpolBc=; b=DLE7cpY0ln1XpR8gvWqbvLJYPWRTngovRdB/8BpVA9NMgk2cPgynCEtmf5yE7Go29fJihz F31iFetmsl2NRjDux2iUjvMTZ8JUue28kqK+Wjv9WlY1e3Vr1rkDduj2B4cDhnFMSA8vcE 67zUm1pJnemFs7uSI3RvDgEqbKAY6rA= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-480-h3PA2ApRMcG7hjzC9Hw72w-1; Mon, 02 Mar 2026 18:22:20 -0500 X-MC-Unique: h3PA2ApRMcG7hjzC9Hw72w-1 X-Mimecast-MFC-AGG-ID: h3PA2ApRMcG7hjzC9Hw72w_1772493738 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 779C619560A7; Mon, 2 Mar 2026 23:22:18 +0000 (UTC) Received: from GoldenWind.lan (unknown [10.22.90.7]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 901091800576; Mon, 2 Mar 2026 23:22:15 +0000 (UTC) From: Lyude Paul To: rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, Thomas Gleixner Cc: Boqun Feng , Daniel Almeida , Miguel Ojeda , Alex Gaynor , Gary Guo , =?UTF-8?q?Bj=C3=B6rn=20Roy=20Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Andrew Morton , Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long Subject: [PATCH v19 5/5] rust: sync: Introduce SpinLockIrq::lock_with() and friends Date: Mon, 2 Mar 2026 18:16:48 -0500 Message-ID: <20260302232154.861916-6-lyude@redhat.com> In-Reply-To: <20260302232154.861916-1-lyude@redhat.com> References: <20260302232154.861916-1-lyude@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 `SpinLockIrq` and `SpinLock` use the exact same underlying C structure, with the only real difference being that the former uses the irq_disable() and irq_enable() variants for locking/unlocking. These variants can introduce some minor overhead in contexts where we already know that local processor interrupts are disabled, and as such we want a way to be able to skip modifying processor interrupt state in said contexts in order to avoid some overhead - just like the current C API allows us to do. In order to do this, we add some special functions for SpinLockIrq: lock_with() and try_lock_with(), which allow acquiring the lock without changing the interrupt state - as long as the caller can provide a LocalInterruptDisabled reference to prove that local processor interrupts have been disabled. In some hacked-together benchmarks we ran, most of the time this did actually seem to lead to a noticeable difference in overhead: From an aarch64 VM running on a MacBook M4: lock() when irq is disabled, 100 times cost Delta { nanos: 500 } lock_with() when irq is disabled, 100 times cost Delta { nanos: 292 } lock() when irq is enabled, 100 times cost Delta { nanos: 834 } lock() when irq is disabled, 100 times cost Delta { nanos: 459 } lock_with() when irq is disabled, 100 times cost Delta { nanos: 291 } lock() when irq is enabled, 100 times cost Delta { nanos: 709 } From an x86_64 VM (qemu/kvm) running on a i7-13700H lock() when irq is disabled, 100 times cost Delta { nanos: 1002 } lock_with() when irq is disabled, 100 times cost Delta { nanos: 729 } lock() when irq is enabled, 100 times cost Delta { nanos: 1516 } lock() when irq is disabled, 100 times cost Delta { nanos: 754 } lock_with() when irq is disabled, 100 times cost Delta { nanos: 966 } lock() when irq is enabled, 100 times cost Delta { nanos: 1227 } (note that there were some runs on x86_64 where lock() on irq disabled vs. lock_with() on irq disabled had equivalent benchmarks, but it very much appeared to be a minority of test runs. While it's not clear how this affects real-world workloads yet, let's add this for the time being so we can find out. This makes it so that a `SpinLockIrq` will work like a `SpinLock` if interrupts are disabled. So a function: (&'a SpinLockIrq, &'a InterruptDisabled) -> Guard<'a, .., SpinLockB= ackend> makes sense. Note that due to `Guard` and `InterruptDisabled` having the same lifetime, interrupts cannot be enabled while the Guard exists. Signed-off-by: Lyude Paul Co-developed-by: Boqun Feng Signed-off-by: Boqun Feng Reviewed-by: Gary Guo --- This was originally two patches, but keeping them split didn't make sense after going from BackendInContext to BackendWithContext. V10: * Fix typos - Dirk/Lyude * Since we're adding support for context locks to GlobalLock as well, let's also make sure to cover try_lock while we're at it and add try_lock_with * Add a private function as_lock_in_context() for handling casting from a Lock to Lock so we don't have to duplicate safety comments V11: * Fix clippy::ref_as_ptr error in Lock::as_lock_in_context() V14: * Add benchmark results, rewrite commit message V17: * Introduce `BackendWithContext`, move context-related bits into there and out of `Backend`. * Add missing #[must_use =3D =E2=80=A6] for try_lock_with() * Remove all unsafe code from lock_with() and try_lock_with(): Somehow I never noticed that literally none of the unsafe code in these two functions is needed with as_lock_in_context()... V18: * Get rid of BackendWithContext * Just use transmute in as_lock_in_context() * Now that we're only supporting IRQ spinlocks and not using traits, use the type aliases for SpinLock and SpinLockGuard * Improve the docs now that we're not using traits. V19: * #[inline] #[inline] #[inline] rust/kernel/sync/lock/spinlock.rs | 81 ++++++++++++++++++++++++++++++- 1 file changed, 80 insertions(+), 1 deletion(-) diff --git a/rust/kernel/sync/lock/spinlock.rs b/rust/kernel/sync/lock/spin= lock.rs index 6b8f92d5b0467..069fcdb58735a 100644 --- a/rust/kernel/sync/lock/spinlock.rs +++ b/rust/kernel/sync/lock/spinlock.rs @@ -4,7 +4,10 @@ //! //! This module allows Rust code to use the kernel's `spinlock_t`. use super::*; -use crate::prelude::*; +use crate::{ + interrupt::LocalInterruptDisabled, + prelude::*, // +}; =20 /// Creates a [`SpinLock`] initialiser with the given name and a newly-cre= ated lock class. /// @@ -224,6 +227,45 @@ macro_rules! new_spinlock_irq { /// # Ok::<(), Error>(()) /// ``` /// +/// The next example demonstrates locking a [`SpinLockIrq`] using [`lock_w= ith()`] in a function +/// which can only be called when local processor interrupts are already d= isabled. +/// +/// ``` +/// use kernel::sync::{new_spinlock_irq, SpinLockIrq}; +/// use kernel::interrupt::*; +/// +/// struct Inner { +/// a: u32, +/// } +/// +/// #[pin_data] +/// struct Example { +/// #[pin] +/// inner: SpinLockIrq, +/// } +/// +/// impl Example { +/// fn new() -> impl PinInit { +/// pin_init!(Self { +/// inner <- new_spinlock_irq!(Inner { a: 20 }), +/// }) +/// } +/// } +/// +/// // Accessing an `Example` from a function that can only be called in n= o-interrupt contexts. +/// fn noirq_work(e: &Example, interrupt_disabled: &LocalInterruptDisabled= ) { +/// // Because we know interrupts are disabled from interrupt_disable,= we can skip toggling +/// // interrupt state using lock_with() and the provided token +/// assert_eq!(e.inner.lock_with(interrupt_disabled).a, 20); +/// } +/// +/// # let e =3D KBox::pin_init(Example::new(), GFP_KERNEL)?; +/// # let interrupt_guard =3D local_interrupt_disable(); +/// # noirq_work(&e, &interrupt_guard); +/// # +/// # Ok::<(), Error>(()) +/// ``` +/// /// [`lock()`]: SpinLockIrq::lock /// [`lock_with()`]: SpinLockIrq::lock_with pub type SpinLockIrq =3D super::Lock; @@ -291,6 +333,43 @@ unsafe fn assert_is_held(ptr: *mut Self::State) { } } =20 +impl Lock { + /// Casts the lock as a `Lock`. + #[inline] + fn as_lock_in_interrupt<'a>(&'a self, _context: &'a LocalInterruptDisa= bled) -> &'a SpinLock { + // SAFETY: + // - `Lock` and `Lock` = both have identical data + // layouts. + // - As long as local interrupts are disabled (which is proven to = be true by _context), it + // is safe to treat a lock with SpinLockIrqBackend as a SpinLock= Backend lock. + unsafe { core::mem::transmute(self) } + } + + /// Acquires the lock without modifying local interrupt state. + /// + /// This function should be used in place of the more expensive [`Lock= ::lock()`] function when + /// possible for [`SpinLockIrq`] locks. + #[inline] + pub fn lock_with<'a>(&'a self, context: &'a LocalInterruptDisabled) ->= SpinLockGuard<'a, T> { + self.as_lock_in_interrupt(context).lock() + } + + /// Tries to acquire the lock without modifying local interrupt state. + /// + /// This function should be used in place of the more expensive [`Lock= ::try_lock()`] function + /// when possible for [`SpinLockIrq`] locks. + /// + /// Returns a guard that can be used to access the data protected by t= he lock if successful. + #[must_use =3D "if unused, the lock will be immediately unlocked"] + #[inline] + pub fn try_lock_with<'a>( + &'a self, + context: &'a LocalInterruptDisabled, + ) -> Option> { + self.as_lock_in_interrupt(context).try_lock() + } +} + #[kunit_tests(rust_spinlock_irq_condvar)] mod tests { use super::*; --=20 2.53.0