This introduces a module for dealing with interrupt-disabled contexts,
including the ability to enable and disable interrupts
(with_irqs_disabled()) - along with the ability to annotate functions as
expecting that IRQs are already disabled on the local CPU.
Signed-off-by: Lyude Paul <lyude@redhat.com>
---
V2:
* Actually make it so that we check whether or not we have interrupts
disabled with debug assertions
* Fix issues in the documentation (added suggestions, missing periods, made
sure that all rustdoc examples compile properly)
* Pass IrqDisabled by value, not reference
* Ensure that IrqDisabled is !Send and !Sync using
PhantomData<(&'a (), *mut ())>
* Add all of the suggested derives from Benno Lossin
V3:
* Use `impl` for FnOnce bounds in with_irqs_disabled()
* Use higher-ranked trait bounds for the lifetime of with_irqs_disabled()
* Wording changes in the documentation for the module itself
V4:
* Use the actual unsafe constructor for IrqDisabled in
with_irqs_disabled()
* Fix comment style in with_irqs_disabled example
* Check before calling local_irq_restore() in with_irqs_disabled that
interrupts are still disabled. It would have been nice to do this from a
Drop implementation like I hoped, but I realized rust doesn't allow that
for types that implement Copy.
* Document that interrupts can't be re-enabled within the `cb` provided to
`with_irqs_disabled`, and link to the github issue I just filed about
this that describes the solution for this.
V5:
* Rebase against rust-next for the helpers split
* Fix typo (enabled -> disabled) - Dirk
Signed-off-by: Lyude Paul <lyude@redhat.com>
---
rust/helpers/helpers.c | 1 +
rust/helpers/irq.c | 22 +++++++++++
rust/kernel/irq.rs | 90 ++++++++++++++++++++++++++++++++++++++++++
rust/kernel/lib.rs | 1 +
4 files changed, 114 insertions(+)
create mode 100644 rust/helpers/irq.c
create mode 100644 rust/kernel/irq.rs
diff --git a/rust/helpers/helpers.c b/rust/helpers/helpers.c
index 30f40149f3a96..0bb48df454bd8 100644
--- a/rust/helpers/helpers.c
+++ b/rust/helpers/helpers.c
@@ -12,6 +12,7 @@
#include "build_assert.c"
#include "build_bug.c"
#include "err.c"
+#include "irq.c"
#include "kunit.c"
#include "mutex.c"
#include "page.c"
diff --git a/rust/helpers/irq.c b/rust/helpers/irq.c
new file mode 100644
index 0000000000000..ec1e8d700a220
--- /dev/null
+++ b/rust/helpers/irq.c
@@ -0,0 +1,22 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/irqflags.h>
+
+unsigned long rust_helper_local_irq_save(void)
+{
+ unsigned long flags;
+
+ local_irq_save(flags);
+
+ return flags;
+}
+
+void rust_helper_local_irq_restore(unsigned long flags)
+{
+ local_irq_restore(flags);
+}
+
+bool rust_helper_irqs_disabled(void)
+{
+ return irqs_disabled();
+}
diff --git a/rust/kernel/irq.rs b/rust/kernel/irq.rs
new file mode 100644
index 0000000000000..0673087161f08
--- /dev/null
+++ b/rust/kernel/irq.rs
@@ -0,0 +1,90 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Interrupt controls
+//!
+//! This module allows Rust code to control processor interrupts. [`with_irqs_disabled()`] may be
+//! used for nested disables of interrupts, whereas [`IrqDisabled`] can be used for annotating code
+//! that requires interrupts to be disabled.
+
+use bindings;
+use core::marker::*;
+
+/// A token that is only available in contexts where IRQs are disabled.
+///
+/// [`IrqDisabled`] is marker made available when interrupts are not active. Certain functions take
+/// an [`IrqDisabled`] in order to indicate that they may only be run in IRQ-free contexts.
+///
+/// This is a marker type; it has no size, and is simply used as a compile-time guarantee that
+/// interrupts are disabled where required.
+///
+/// This token can be created by [`with_irqs_disabled`]. See [`with_irqs_disabled`] for examples and
+/// further information.
+#[derive(Copy, Clone, Debug, Ord, Eq, PartialOrd, PartialEq, Hash)]
+pub struct IrqDisabled<'a>(PhantomData<(&'a (), *mut ())>);
+
+impl IrqDisabled<'_> {
+ /// Create a new [`IrqDisabled`] without disabling interrupts.
+ ///
+ /// This creates an [`IrqDisabled`] token, which can be passed to functions that must be run
+ /// without interrupts. If debug assertions are enabled, this function will assert that
+ /// interrupts are disabled upon creation. Otherwise, it has no size or cost at runtime.
+ ///
+ /// # Panics
+ ///
+ /// If debug assertions are enabled, this function will panic if interrupts are not disabled
+ /// upon creation.
+ ///
+ /// # Safety
+ ///
+ /// This function must only be called in contexts where it is already known that interrupts have
+ /// been disabled for the current CPU, as the user is making a promise that they will remain
+ /// disabled at least until this [`IrqDisabled`] is dropped.
+ pub unsafe fn new() -> Self {
+ // SAFETY: FFI call with no special requirements
+ debug_assert!(unsafe { bindings::irqs_disabled() });
+
+ Self(PhantomData)
+ }
+}
+
+/// Run the closure `cb` with interrupts disabled on the local CPU.
+///
+/// This creates an [`IrqDisabled`] token, which can be passed to functions that must be run
+/// without interrupts. Note that interrupts must be disabled for the entire duration of `cb`, they
+/// cannot be re-enabled. In the future, this may be expanded on
+/// [as documented here](https://github.com/Rust-for-Linux/linux/issues/1115).
+///
+/// # Examples
+///
+/// Using [`with_irqs_disabled`] to call a function that can only be called with interrupts
+/// disabled:
+///
+/// ```
+/// use kernel::irq::{IrqDisabled, with_irqs_disabled};
+///
+/// // Requiring interrupts be disabled to call a function
+/// fn dont_interrupt_me(_irq: IrqDisabled<'_>) {
+/// // When this token is available, IRQs are known to be disabled. Actions that rely on this
+/// // can be safely performed
+/// }
+///
+/// // Disabling interrupts. They'll be re-enabled once this closure completes.
+/// with_irqs_disabled(|irq| dont_interrupt_me(irq));
+/// ```
+#[inline]
+pub fn with_irqs_disabled<T>(cb: impl for<'a> FnOnce(IrqDisabled<'a>) -> T) -> T {
+ // SAFETY: FFI call with no special requirements
+ let flags = unsafe { bindings::local_irq_save() };
+
+ // SAFETY: We just disabled IRQs using `local_irq_save()`
+ let ret = cb(unsafe { IrqDisabled::new() });
+
+ // Confirm that IRQs are still disabled now that the callback has finished
+ // SAFETY: FFI call with no special requirements
+ debug_assert!(unsafe { bindings::irqs_disabled() });
+
+ // SAFETY: `flags` comes from our previous call to local_irq_save
+ unsafe { bindings::local_irq_restore(flags) };
+
+ ret
+}
diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
index f10b06a78b9d5..df10c58e95c19 100644
--- a/rust/kernel/lib.rs
+++ b/rust/kernel/lib.rs
@@ -36,6 +36,7 @@
pub mod firmware;
pub mod init;
pub mod ioctl;
+pub mod irq;
#[cfg(CONFIG_KUNIT)]
pub mod kunit;
pub mod list;
--
2.46.0On Thu, 12 Sep 2024 15:04:16 -0400
Lyude Paul <lyude@redhat.com> wrote:
> This introduces a module for dealing with interrupt-disabled contexts,
> including the ability to enable and disable interrupts
> (with_irqs_disabled()) - along with the ability to annotate functions as
> expecting that IRQs are already disabled on the local CPU.
>
> Signed-off-by: Lyude Paul <lyude@redhat.com>
>
> ---
>
> V2:
> * Actually make it so that we check whether or not we have interrupts
> disabled with debug assertions
> * Fix issues in the documentation (added suggestions, missing periods, made
> sure that all rustdoc examples compile properly)
> * Pass IrqDisabled by value, not reference
> * Ensure that IrqDisabled is !Send and !Sync using
> PhantomData<(&'a (), *mut ())>
> * Add all of the suggested derives from Benno Lossin
>
> V3:
> * Use `impl` for FnOnce bounds in with_irqs_disabled()
> * Use higher-ranked trait bounds for the lifetime of with_irqs_disabled()
> * Wording changes in the documentation for the module itself
>
> V4:
> * Use the actual unsafe constructor for IrqDisabled in
> with_irqs_disabled()
> * Fix comment style in with_irqs_disabled example
> * Check before calling local_irq_restore() in with_irqs_disabled that
> interrupts are still disabled. It would have been nice to do this from a
> Drop implementation like I hoped, but I realized rust doesn't allow that
> for types that implement Copy.
> * Document that interrupts can't be re-enabled within the `cb` provided to
> `with_irqs_disabled`, and link to the github issue I just filed about
> this that describes the solution for this.
>
> V5:
> * Rebase against rust-next for the helpers split
> * Fix typo (enabled -> disabled) - Dirk
>
> Signed-off-by: Lyude Paul <lyude@redhat.com>
> ---
> rust/helpers/helpers.c | 1 +
> rust/helpers/irq.c | 22 +++++++++++
> rust/kernel/irq.rs | 90 ++++++++++++++++++++++++++++++++++++++++++
> rust/kernel/lib.rs | 1 +
> 4 files changed, 114 insertions(+)
> create mode 100644 rust/helpers/irq.c
> create mode 100644 rust/kernel/irq.rs
>
> diff --git a/rust/helpers/helpers.c b/rust/helpers/helpers.c
> index 30f40149f3a96..0bb48df454bd8 100644
> --- a/rust/helpers/helpers.c
> +++ b/rust/helpers/helpers.c
> @@ -12,6 +12,7 @@
> #include "build_assert.c"
> #include "build_bug.c"
> #include "err.c"
> +#include "irq.c"
> #include "kunit.c"
> #include "mutex.c"
> #include "page.c"
> diff --git a/rust/helpers/irq.c b/rust/helpers/irq.c
> new file mode 100644
> index 0000000000000..ec1e8d700a220
> --- /dev/null
> +++ b/rust/helpers/irq.c
> @@ -0,0 +1,22 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#include <linux/irqflags.h>
> +
> +unsigned long rust_helper_local_irq_save(void)
> +{
> + unsigned long flags;
> +
> + local_irq_save(flags);
> +
> + return flags;
> +}
> +
> +void rust_helper_local_irq_restore(unsigned long flags)
> +{
> + local_irq_restore(flags);
> +}
> +
> +bool rust_helper_irqs_disabled(void)
> +{
> + return irqs_disabled();
> +}
> diff --git a/rust/kernel/irq.rs b/rust/kernel/irq.rs
> new file mode 100644
> index 0000000000000..0673087161f08
> --- /dev/null
> +++ b/rust/kernel/irq.rs
> @@ -0,0 +1,90 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +//! Interrupt controls
> +//!
> +//! This module allows Rust code to control processor interrupts. [`with_irqs_disabled()`] may be
> +//! used for nested disables of interrupts, whereas [`IrqDisabled`] can be used for annotating code
> +//! that requires interrupts to be disabled.
> +
> +use bindings;
> +use core::marker::*;
> +
> +/// A token that is only available in contexts where IRQs are disabled.
> +///
> +/// [`IrqDisabled`] is marker made available when interrupts are not active. Certain functions take
> +/// an [`IrqDisabled`] in order to indicate that they may only be run in IRQ-free contexts.
I think this should be encoded as a type invariant?
> +///
> +/// This is a marker type; it has no size, and is simply used as a compile-time guarantee that
> +/// interrupts are disabled where required.
> +///
> +/// This token can be created by [`with_irqs_disabled`]. See [`with_irqs_disabled`] for examples and
> +/// further information.
> +#[derive(Copy, Clone, Debug, Ord, Eq, PartialOrd, PartialEq, Hash)]
> +pub struct IrqDisabled<'a>(PhantomData<(&'a (), *mut ())>);
> +
> +impl IrqDisabled<'_> {
> + /// Create a new [`IrqDisabled`] without disabling interrupts.
> + ///
> + /// This creates an [`IrqDisabled`] token, which can be passed to functions that must be run
> + /// without interrupts. If debug assertions are enabled, this function will assert that
> + /// interrupts are disabled upon creation. Otherwise, it has no size or cost at runtime.
> + ///
> + /// # Panics
> + ///
> + /// If debug assertions are enabled, this function will panic if interrupts are not disabled
> + /// upon creation.
> + ///
> + /// # Safety
> + ///
> + /// This function must only be called in contexts where it is already known that interrupts have
> + /// been disabled for the current CPU, as the user is making a promise that they will remain
> + /// disabled at least until this [`IrqDisabled`] is dropped.
> + pub unsafe fn new() -> Self {
> + // SAFETY: FFI call with no special requirements
> + debug_assert!(unsafe { bindings::irqs_disabled() });
> +
> + Self(PhantomData)
This needs to assert INVARIANT.
> + }
> +}
> +
> +/// Run the closure `cb` with interrupts disabled on the local CPU.
> +///
> +/// This creates an [`IrqDisabled`] token, which can be passed to functions that must be run
> +/// without interrupts. Note that interrupts must be disabled for the entire duration of `cb`, they
> +/// cannot be re-enabled. In the future, this may be expanded on
> +/// [as documented here](https://github.com/Rust-for-Linux/linux/issues/1115).
> +///
> +/// # Examples
> +///
> +/// Using [`with_irqs_disabled`] to call a function that can only be called with interrupts
> +/// disabled:
> +///
> +/// ```
> +/// use kernel::irq::{IrqDisabled, with_irqs_disabled};
> +///
> +/// // Requiring interrupts be disabled to call a function
> +/// fn dont_interrupt_me(_irq: IrqDisabled<'_>) {
> +/// // When this token is available, IRQs are known to be disabled. Actions that rely on this
> +/// // can be safely performed
> +/// }
> +///
> +/// // Disabling interrupts. They'll be re-enabled once this closure completes.
> +/// with_irqs_disabled(|irq| dont_interrupt_me(irq));
> +/// ```
> +#[inline]
> +pub fn with_irqs_disabled<T>(cb: impl for<'a> FnOnce(IrqDisabled<'a>) -> T) -> T {
> + // SAFETY: FFI call with no special requirements
> + let flags = unsafe { bindings::local_irq_save() };
> +
> + // SAFETY: We just disabled IRQs using `local_irq_save()`
> + let ret = cb(unsafe { IrqDisabled::new() });
> +
> + // Confirm that IRQs are still disabled now that the callback has finished
> + // SAFETY: FFI call with no special requirements
> + debug_assert!(unsafe { bindings::irqs_disabled() });
> +
> + // SAFETY: `flags` comes from our previous call to local_irq_save
> + unsafe { bindings::local_irq_restore(flags) };
> +
> + ret
> +}
> diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
> index f10b06a78b9d5..df10c58e95c19 100644
> --- a/rust/kernel/lib.rs
> +++ b/rust/kernel/lib.rs
> @@ -36,6 +36,7 @@
> pub mod firmware;
> pub mod init;
> pub mod ioctl;
> +pub mod irq;
> #[cfg(CONFIG_KUNIT)]
> pub mod kunit;
> pub mod list;
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
On 12.09.24 21:04, Lyude Paul wrote:
> This introduces a module for dealing with interrupt-disabled contexts,
> including the ability to enable and disable interrupts
> (with_irqs_disabled()) - along with the ability to annotate functions as
> expecting that IRQs are already disabled on the local CPU.
>
> Signed-off-by: Lyude Paul <lyude@redhat.com>
>
> ---
>
> V2:
> * Actually make it so that we check whether or not we have interrupts
> disabled with debug assertions
> * Fix issues in the documentation (added suggestions, missing periods, made
> sure that all rustdoc examples compile properly)
> * Pass IrqDisabled by value, not reference
> * Ensure that IrqDisabled is !Send and !Sync using
> PhantomData<(&'a (), *mut ())>
> * Add all of the suggested derives from Benno Lossin
>
> V3:
> * Use `impl` for FnOnce bounds in with_irqs_disabled()
> * Use higher-ranked trait bounds for the lifetime of with_irqs_disabled()
> * Wording changes in the documentation for the module itself
>
> V4:
> * Use the actual unsafe constructor for IrqDisabled in
> with_irqs_disabled()
> * Fix comment style in with_irqs_disabled example
> * Check before calling local_irq_restore() in with_irqs_disabled that
> interrupts are still disabled. It would have been nice to do this from a
> Drop implementation like I hoped, but I realized rust doesn't allow that
> for types that implement Copy.
> * Document that interrupts can't be re-enabled within the `cb` provided to
> `with_irqs_disabled`, and link to the github issue I just filed about
> this that describes the solution for this.
>
> V5:
> * Rebase against rust-next for the helpers split
> * Fix typo (enabled -> disabled) - Dirk
>
> Signed-off-by: Lyude Paul <lyude@redhat.com>
> ---
> rust/helpers/helpers.c | 1 +
> rust/helpers/irq.c | 22 +++++++++++
> rust/kernel/irq.rs | 90 ++++++++++++++++++++++++++++++++++++++++++
> rust/kernel/lib.rs | 1 +
> 4 files changed, 114 insertions(+)
> create mode 100644 rust/helpers/irq.c
> create mode 100644 rust/kernel/irq.rs
This looks pretty good, I have two documentation improvements below,
with those fixed:
Reviewed-by: Benno Lossin <benno.lossin@proton.me>
> diff --git a/rust/kernel/irq.rs b/rust/kernel/irq.rs
> new file mode 100644
> index 0000000000000..0673087161f08
> --- /dev/null
> +++ b/rust/kernel/irq.rs
> @@ -0,0 +1,90 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +//! Interrupt controls
> +//!
> +//! This module allows Rust code to control processor interrupts. [`with_irqs_disabled()`] may be
> +//! used for nested disables of interrupts, whereas [`IrqDisabled`] can be used for annotating code
> +//! that requires interrupts to be disabled.
> +
> +use bindings;
> +use core::marker::*;
> +
> +/// A token that is only available in contexts where IRQs are disabled.
> +///
> +/// [`IrqDisabled`] is marker made available when interrupts are not active. Certain functions take
> +/// an [`IrqDisabled`] in order to indicate that they may only be run in IRQ-free contexts.
I feel like "indicate" is not strong enough, how about using "require"?
> +///
> +/// This is a marker type; it has no size, and is simply used as a compile-time guarantee that
> +/// interrupts are disabled where required.
> +///
> +/// This token can be created by [`with_irqs_disabled`]. See [`with_irqs_disabled`] for examples and
> +/// further information.
> +#[derive(Copy, Clone, Debug, Ord, Eq, PartialOrd, PartialEq, Hash)]
> +pub struct IrqDisabled<'a>(PhantomData<(&'a (), *mut ())>);
> +/// Run the closure `cb` with interrupts disabled on the local CPU.
> +///
> +/// This creates an [`IrqDisabled`] token, which can be passed to functions that must be run
> +/// without interrupts. Note that interrupts must be disabled for the entire duration of `cb`, they
> +/// cannot be re-enabled. In the future, this may be expanded on
> +/// [as documented here](https://github.com/Rust-for-Linux/linux/issues/1115).
> +///
> +/// # Examples
> +///
> +/// Using [`with_irqs_disabled`] to call a function that can only be called with interrupts
> +/// disabled:
> +///
> +/// ```
> +/// use kernel::irq::{IrqDisabled, with_irqs_disabled};
> +///
> +/// // Requiring interrupts be disabled to call a function
> +/// fn dont_interrupt_me(_irq: IrqDisabled<'_>) {
> +/// // When this token is available, IRQs are known to be disabled. Actions that rely on this
> +/// // can be safely performed
> +/// }
> +///
> +/// // Disabling interrupts. They'll be re-enabled once this closure completes.
Please change this to be "Disables interrupts, their previous state will
be restored once the closure completes.".
ie it is important to stress that this can be nested. Please also add
this in a paragraph above the examples.
---
Cheers,
Benno
> +/// with_irqs_disabled(|irq| dont_interrupt_me(irq));
> +/// ```
> +#[inline]
> +pub fn with_irqs_disabled<T>(cb: impl for<'a> FnOnce(IrqDisabled<'a>) -> T) -> T {
> + // SAFETY: FFI call with no special requirements
> + let flags = unsafe { bindings::local_irq_save() };
> +
> + // SAFETY: We just disabled IRQs using `local_irq_save()`
> + let ret = cb(unsafe { IrqDisabled::new() });
> +
> + // Confirm that IRQs are still disabled now that the callback has finished
> + // SAFETY: FFI call with no special requirements
> + debug_assert!(unsafe { bindings::irqs_disabled() });
> +
> + // SAFETY: `flags` comes from our previous call to local_irq_save
> + unsafe { bindings::local_irq_restore(flags) };
> +
> + ret
> +}
> diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
> index f10b06a78b9d5..df10c58e95c19 100644
> --- a/rust/kernel/lib.rs
> +++ b/rust/kernel/lib.rs
> @@ -36,6 +36,7 @@
> pub mod firmware;
> pub mod init;
> pub mod ioctl;
> +pub mod irq;
> #[cfg(CONFIG_KUNIT)]
> pub mod kunit;
> pub mod list;
> --
> 2.46.0
>
© 2016 - 2024 Red Hat, Inc.