Provides an abstraction for C bitmap API and bitops operations.
We follow the C Bitmap API closely in naming and semantics, with
a few differences that take advantage of Rust language facilities
and idioms:
* We leverage Rust type system guarantees as follows:
* Ownership transfer of a Bitmap is restricted so that it cannot
be passed between threads (does not implement `Send`).
* all (non-atomic) mutating operations require a &mut reference which
amounts to exclusive access.
* It is permissible to pass shared references &Bitmap between
threads, and we expose atomic operations through interior mutability.
Since these atomic operations do not provide any ordering guarantees,
we mark these as `unsafe`. Anyone who calls the atomic methods needs
to document that the lack of ordering is safe.
* The Rust API offers `{set,clear}_bit` vs `{set,clear}_bit_atomic`,
which is different from the C naming convention (set_bit vs __set_bit).
* we include enough operations for the API to be useful, but not all
operations are exposed yet in order to avoid dead code. This commit
includes enough to enable a Rust implementation of an Android Binder
data structure that was introduced in commit 15d9da3f818c ("binder:
use bitmap for faster descriptor lookup"), which can be found in
drivers/android/dbitmap.h. We need this in order to upstream the Rust
port of Android Binder driver.
* We follow the C API closely and fine-grained approach to safety:
* Low-level bit-ops methods get a safe API with bounds checks.
* methods correspond to find_* C methods tolerate out-of-bounds
arguments. Since these are already safe we the same behavior, and
return results using `Option` types to represent "not found".
* the Rust API is optimized to represent the bitmap inline if it would
take the space of a pointer. This saves allocations which is
relevant in the Binder use case.
* Bounds checks where out-of-bounds values would not result in
immediate access faults are configured via a RUST_BITMAP_HARDENED
config.
The underlying C bitmap is *not* exposed, and must never be exposed
(except in tests). Exposing the representation would lose all static
guarantees, and moreover would prevent the optimized representation of
short bitmaps.
An alternative route of vendoring an existing Rust bitmap package was
considered but suboptimal overall. Reusing the C implementation is
preferable for a basic data structure like bitmaps. It enables Rust
code to be a lot more similar and predictable with respect to C code
that uses the same data structures and enables the use of code that
has been tried-and-tested in the kernel, with the same performance
characteristics whenever possible.
We use the `usize` type for sizes and indices into the bitmap,
because Rust generally always uses that type for indices and lengths
and it will be more convenient if the API accepts that type. This means
that we need to perform some casts to/from u32 and usize, since the C
headers use unsigned int instead of size_t/unsigned long for these
numbers in some places.
Adds new MAINTAINERS section BITMAP API [RUST].
Suggested-by: Alice Ryhl <aliceryhl@google.com>
Suggested-by: Yury Norov <yury.norov@gmail.com>
Signed-off-by: Burak Emir <bqe@google.com>
---
MAINTAINERS | 7 +
rust/kernel/bitmap.rs | 415 +++++++++++++++++++++++++++++++++++++
rust/kernel/lib.rs | 1 +
security/Kconfig.hardening | 9 +
4 files changed, 432 insertions(+)
create mode 100644 rust/kernel/bitmap.rs
diff --git a/MAINTAINERS b/MAINTAINERS
index 04d6727e944c..565eaa015d9e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4127,6 +4127,13 @@ S: Maintained
F: rust/helpers/bitmap.c
F: rust/helpers/cpumask.c
+BITMAP API [RUST]
+M: Alice Ryhl <aliceryhl@google.com>
+M: Burak Emir <bqe@google.com>
+R: Yury Norov <yury.norov@gmail.com>
+S: Maintained
+F: rust/kernel/bitmap.rs
+
BITOPS API
M: Yury Norov <yury.norov@gmail.com>
R: Rasmus Villemoes <linux@rasmusvillemoes.dk>
diff --git a/rust/kernel/bitmap.rs b/rust/kernel/bitmap.rs
new file mode 100644
index 000000000000..943dbef7948b
--- /dev/null
+++ b/rust/kernel/bitmap.rs
@@ -0,0 +1,415 @@
+// SPDX-License-Identifier: GPL-2.0
+
+// Copyright (C) 2025 Google LLC.
+
+//! Rust API for bitmap.
+//!
+//! C headers: [`include/linux/bitmap.h`](srctree/include/linux/bitmap.h).
+
+use crate::alloc::{AllocError, Flags};
+use crate::bindings;
+use core::ptr::NonNull;
+
+/// Holds either a pointer to array of `unsigned long` or a small bitmap.
+#[repr(C)]
+union BitmapRepr {
+ bitmap: usize,
+ ptr: NonNull<usize>,
+}
+
+macro_rules! bitmap_hardening_assert {
+ ($cond:expr, $($arg:tt)+) => {
+ #[cfg(RUST_BITMAP_HARDENED)]
+ assert!($e, $($arg)*);
+ }
+}
+
+/// Represents a bitmap.
+///
+/// Wraps underlying C bitmap API.
+///
+/// # Examples
+///
+/// Basic usage
+///
+/// ```
+/// use kernel::alloc::flags::GFP_KERNEL;
+/// use kernel::bitmap::Bitmap;
+///
+/// let mut b = Bitmap::new(16, GFP_KERNEL)?;
+///
+/// assert_eq!(16, b.len());
+/// for i in 0..16 {
+/// if i % 4 == 0 {
+/// b.set_bit(i);
+/// }
+/// }
+/// assert_eq!(Some(0), b.next_bit(0));
+/// assert_eq!(Some(1), b.next_zero_bit(0));
+/// assert_eq!(Some(4), b.next_bit(1));
+/// assert_eq!(Some(5), b.next_zero_bit(4));
+/// assert_eq!(Some(12), b.last_bit());
+/// # Ok::<(), Error>(())
+/// ```
+///
+/// # Invariants
+///
+/// * `nbits` is `<= i32::MAX` and never changes.
+/// * if `nbits <= bindings::BITS_PER_LONG`, then `repr` is a bitmap.
+/// * otherwise, `repr` holds a non-null pointer that was obtained from a
+/// successful call to `bitmap_zalloc` and holds the address of an initialized
+/// array of `unsigned long` that is large enough to hold `nbits` bits.
+pub struct Bitmap {
+ /// Representation of bitmap.
+ repr: BitmapRepr,
+ /// Length of this bitmap. Must be `<= i32::MAX`.
+ nbits: usize,
+}
+
+/// Enable unsynchronized concurrent access to [`Bitmap`] through shared references.
+///
+/// # Safety
+///
+/// * When no thread performs any mutations, concurrent access is safe.
+/// * Mutations are permitted through atomic operations and interior mutability.
+/// All such methods are marked unsafe, to account for the lack of ordering
+/// guarantees. Callers must acknowledge that updates may be observed in any
+/// order.
+unsafe impl Sync for Bitmap {}
+
+impl Drop for Bitmap {
+ fn drop(&mut self) {
+ if self.nbits <= bindings::BITS_PER_LONG as _ {
+ return;
+ }
+ // SAFETY: `self.ptr` was returned by the C `bitmap_zalloc`.
+ //
+ // INVARIANT: there is no other use of the `self.ptr` after this
+ // call and the value is being dropped so the broken invariant is
+ // not observable on function exit.
+ unsafe { bindings::bitmap_free(self.as_mut_ptr()) };
+ }
+}
+
+impl Bitmap {
+ /// Constructs a new [`Bitmap`].
+ ///
+ /// Fails with [`AllocError`] when the [`Bitmap`] could not be allocated. This
+ /// includes the case when `nbits` is greater than `i32::MAX`.
+ #[inline]
+ pub fn new(nbits: usize, flags: Flags) -> Result<Self, AllocError> {
+ if nbits <= bindings::BITS_PER_LONG as _ {
+ return Ok(Bitmap {
+ repr: BitmapRepr { bitmap: 0 },
+ nbits,
+ });
+ }
+ if nbits > i32::MAX.try_into().unwrap() {
+ return Err(AllocError);
+ }
+ let nbits_u32 = u32::try_from(nbits).unwrap();
+ // SAFETY: `bindings::BITS_PER_LONG < nbits` and `nbits <= i32::MAX`.
+ let ptr = unsafe { bindings::bitmap_zalloc(nbits_u32, flags.as_raw()) };
+ let ptr = NonNull::new(ptr).ok_or(AllocError)?;
+ // INVARIANT: `ptr` returned by C `bitmap_zalloc` and `nbits` checked.
+ return Ok(Bitmap {
+ repr: BitmapRepr { ptr },
+ nbits,
+ });
+ }
+
+ /// Returns length of this [`Bitmap`].
+ #[inline]
+ pub fn len(&self) -> usize {
+ self.nbits
+ }
+
+ /// Returns a mutable raw pointer to the backing [`Bitmap`].
+ #[inline]
+ fn as_mut_ptr(&mut self) -> *mut usize {
+ if self.nbits <= bindings::BITS_PER_LONG as _ {
+ // SAFETY: Bitmap is represented inline.
+ unsafe { core::ptr::addr_of_mut!(self.repr.bitmap) }
+ } else {
+ // SAFETY: Bitmap is represented as array of `unsigned long`.
+ unsafe { self.repr.ptr.as_mut() }
+ }
+ }
+
+ /// Returns a raw pointer to the backing [`Bitmap`].
+ #[inline]
+ fn as_ptr(&self) -> *const usize {
+ if self.nbits <= bindings::BITS_PER_LONG as _ {
+ // SAFETY: Bitmap is represented inline.
+ unsafe { core::ptr::addr_of!(self.repr.bitmap) }
+ } else {
+ // SAFETY: Bitmap is represented as array of `unsigned long`.
+ unsafe { self.repr.ptr.as_ptr() }
+ }
+ }
+
+ /// Set bit with index `index`.
+ ///
+ /// ATTENTION: `set_bit` is non-atomic, which differs from the naming
+ /// convention in C code. The corresponding C function is `__set_bit`.
+ ///
+ /// # Panics
+ ///
+ /// Panics if `index` is greater than or equal to `self.nbits`.
+ #[inline]
+ pub fn set_bit(&mut self, index: usize) {
+ assert!(
+ index < self.nbits,
+ "Bit `index` must be < {}, was {}",
+ self.nbits,
+ index
+ );
+ // SAFETY: Bit `index` is within bounds.
+ unsafe { bindings::__set_bit(index as u32, self.as_mut_ptr()) };
+ }
+
+ /// Set bit with index `index`, atomically.
+ ///
+ /// ATTENTION: The naming convention differs from C, where the corresponding
+ /// function is called `set_bit`.
+ ///
+ /// # Safety
+ ///
+ /// This is a relaxed atomic operation (no implied memory barriers, no
+ /// ordering guarantees). The caller must ensure that this is safe, as
+ /// the compiler cannot prevent code with an exclusive reference from
+ /// calling atomic operations.
+ ///
+ /// # Panics
+ ///
+ /// Panics if `index` is greater than or equal to `self.nbits`.
+ #[inline]
+ pub unsafe fn set_bit_atomic(&self, index: usize) {
+ assert!(
+ index < self.nbits,
+ "Bit `index` must be < {}, was {}",
+ self.nbits,
+ index
+ );
+ // SAFETY: `index` is within bounds and the caller has ensured that
+ // there is no mix of non-atomic and atomic operations.
+ unsafe { bindings::set_bit(index as u32, self.as_ptr() as *mut usize) };
+ }
+
+ /// Clear `index` bit.
+ ///
+ /// ATTENTION: `clear_bit` is non-atomic, which differs from the naming
+ /// convention in C code. The corresponding C function is `__clear_bit`.
+ /// # Panics
+ ///
+ /// Panics if `index` is greater than or equal to `self.nbits`.
+ #[inline]
+ pub fn clear_bit(&mut self, index: usize) {
+ assert!(
+ index < self.nbits,
+ "Bit `index` must be < {}, was {}",
+ self.nbits,
+ index
+ );
+ // SAFETY: `index` is within bounds.
+ unsafe { bindings::__clear_bit(index as u32, self.as_mut_ptr()) };
+ }
+
+ /// Clear `index` bit, atomically.
+ ///
+ /// ATTENTION: The naming convention differs from C, where the corresponding
+ /// function is called `clear_bit`.
+ ///
+ /// # Safety
+ ///
+ /// This is a relaxed atomic operation (no implied memory barriers, no
+ /// ordering guarantees). The caller must ensure that this is safe, as
+ /// the compiler cannot prevent code with an exclusive reference from
+ /// calling atomic operations.
+ ///
+ /// # Panics
+ ///
+ /// Panics if `index` is greater than or equal to `self.nbits`.
+ #[inline]
+ pub unsafe fn clear_bit_atomic(&self, index: usize) {
+ assert!(
+ index < self.nbits,
+ "Bit `index` must be < {}, was {}",
+ self.nbits,
+ index
+ );
+ // SAFETY: `index` is within bounds and the caller has ensured that
+ // there is no mix of non-atomic and atomic operations.
+ unsafe { bindings::clear_bit(index as u32, self.as_ptr() as *mut usize) };
+ }
+
+ /// Copy `src` into this [`Bitmap`] and set any remaining bits to zero.
+ ///
+ /// # Examples
+ ///
+ /// ```
+ /// use kernel::alloc::{AllocError, flags::GFP_KERNEL};
+ /// use kernel::bitmap::Bitmap;
+ ///
+ /// let mut long_bitmap = Bitmap::new(256, GFP_KERNEL)?;
+ //
+ /// assert_eq!(None, long_bitmap.last_bit());
+ //
+ /// let mut short_bitmap = Bitmap::new(16, GFP_KERNEL)?;
+ //
+ /// short_bitmap.set_bit(7);
+ /// long_bitmap.copy_and_extend(&short_bitmap);
+ /// assert_eq!(Some(7), long_bitmap.last_bit());
+ ///
+ /// # Ok::<(), AllocError>(())
+ /// ```
+ #[inline]
+ pub fn copy_and_extend(&mut self, src: &Bitmap) {
+ let len = core::cmp::min(src.nbits, self.nbits);
+ // SAFETY: access to `self` and `src` is within bounds.
+ unsafe {
+ bindings::bitmap_copy_and_extend(
+ self.as_mut_ptr(),
+ src.as_ptr(),
+ len as u32,
+ self.nbits as u32,
+ )
+ };
+ }
+
+ /// Finds last set bit.
+ ///
+ /// # Examples
+ ///
+ /// ```
+ /// use kernel::alloc::{AllocError, flags::GFP_KERNEL};
+ /// use kernel::bitmap::Bitmap;
+ ///
+ /// let bitmap = Bitmap::new(64, GFP_KERNEL)?;
+ ///
+ /// match bitmap.last_bit() {
+ /// Some(idx) => {
+ /// pr_info!("The last bit has index {idx}.\n");
+ /// }
+ /// None => {
+ /// pr_info!("All bits in this bitmap are 0.\n");
+ /// }
+ /// }
+ /// # Ok::<(), AllocError>(())
+ /// ```
+ #[inline]
+ pub fn last_bit(&self) -> Option<usize> {
+ // SAFETY: `_find_next_bit` access is within bounds due to invariant.
+ let index = unsafe { bindings::_find_last_bit(self.as_ptr(), self.nbits) };
+ if index >= self.nbits {
+ None
+ } else {
+ Some(index)
+ }
+ }
+
+ /// Finds next set bit, starting from `start`.
+ /// Returns `None` if `start` is greater of equal than `self.nbits`.
+ #[inline]
+ pub fn next_bit(&self, start: usize) -> Option<usize> {
+ bitmap_hardening_assert!(start < self.nbits, "`start` must be < {} was {}", self.nbits, start);
+ // SAFETY: `_find_next_bit` tolerates out-of-bounds arguments and returns a
+ // value larger than or equal to `self.nbits` in that case.
+ let index = unsafe { bindings::_find_next_bit(self.as_ptr(), self.nbits, start) };
+ if index >= self.nbits {
+ None
+ } else {
+ Some(index)
+ }
+ }
+
+ /// Finds next zero bit, starting from `start`.
+ /// Returns `None` if `start` is greater than or equal to `self.nbits`.
+ #[inline]
+ pub fn next_zero_bit(&self, start: usize) -> Option<usize> {
+ bitmap_hardening_assert!(start < self.nbits, "`start` must be < {} was {}", self.nbits, start);
+ // SAFETY: `_find_next_zero_bit` tolerates out-of-bounds arguments and returns a
+ // value larger than or equal to `self.nbits` in that case.
+ let index = unsafe { bindings::_find_next_zero_bit(self.as_ptr(), self.nbits, start) };
+ if index >= self.nbits {
+ None
+ } else {
+ Some(index)
+ }
+ }
+}
+
+use macros::kunit_tests;
+
+#[kunit_tests(rust_kernel_bitmap)]
+mod tests {
+ use super::*;
+ use kernel::alloc::flags::GFP_KERNEL;
+
+ #[test]
+ fn bitmap_new() {
+ let b = Bitmap::new(0, GFP_KERNEL).unwrap();
+ assert_eq!(0, b.len());
+
+ let b = Bitmap::new(3, GFP_KERNEL).unwrap();
+ assert_eq!(3, b.len());
+
+ let b = Bitmap::new(1024, GFP_KERNEL).unwrap();
+ assert_eq!(1024, b.len());
+
+ // Requesting too large values results in [`AllocError`].
+ let b = Bitmap::new(1 << 31, GFP_KERNEL);
+ assert!(b.is_err());
+ }
+
+ #[test]
+ fn bitmap_set_clear_find() {
+ let mut b = Bitmap::new(128, GFP_KERNEL).unwrap();
+
+ // Zero-initialized
+ assert_eq!(None, b.last_bit());
+
+ b.set_bit(17);
+
+ assert_eq!(Some(17), b.next_bit(0));
+ assert_eq!(Some(17), b.last_bit());
+
+ b.set_bit(107);
+
+ assert_eq!(Some(17), b.next_bit(0));
+ assert_eq!(Some(17), b.next_bit(17));
+ assert_eq!(Some(107), b.next_bit(18));
+ assert_eq!(Some(107), b.last_bit());
+
+ b.clear_bit(17);
+
+ assert_eq!(Some(107), b.next_bit(0));
+ assert_eq!(Some(107), b.last_bit());
+ }
+
+ #[test]
+ fn bitmap_out_of_bounds() {
+ let mut b = Bitmap::new(128, GFP_KERNEL).unwrap();
+
+ assert_eq!(None, b.next_bit(2048));
+ assert_eq!(None, b.next_zero_bit(2048));
+ assert_eq!(None, b.last_bit(2048));
+ }
+
+ #[test]
+ fn bitmap_copy_and_extend() {
+ let mut long_bitmap = Bitmap::new(256, GFP_KERNEL).unwrap();
+
+ long_bitmap.set_bit(3);
+ long_bitmap.set_bit(200);
+
+ let mut short_bitmap = Bitmap::new(32, GFP_KERNEL).unwrap();
+
+ short_bitmap.set_bit(17);
+
+ long_bitmap.copy_and_extend(&short_bitmap);
+ // The larger bits have been cleared.
+ assert_eq!(Some(17), long_bitmap.next_bit(0));
+ assert_eq!(Some(17), long_bitmap.last_bit());
+ }
+}
diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
index de07aadd1ff5..8c4161cd82ac 100644
--- a/rust/kernel/lib.rs
+++ b/rust/kernel/lib.rs
@@ -38,6 +38,7 @@
pub use ffi;
pub mod alloc;
+pub mod bitmap;
#[cfg(CONFIG_BLOCK)]
pub mod block;
#[doc(hidden)]
diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening
index 3fe9d7b945c4..926665bbc8f2 100644
--- a/security/Kconfig.hardening
+++ b/security/Kconfig.hardening
@@ -324,6 +324,15 @@ config LIST_HARDENED
If unsure, say N.
+config RUST_BITMAP_HARDENED
+ bool "Check integrity of linked list manipulation"
+ help
+ Enables additional assertions in the Rust Bitmap API to catch
+ arguments that are not guaranteed to result in an immediate access
+ fault.
+
+ If unsure, say N.
+
config BUG_ON_DATA_CORRUPTION
bool "Trigger a BUG when data corruption is detected"
select LIST_HARDENED
--
2.49.0.1101.gccaa498523-googHi Burak,
kernel test robot noticed the following build errors:
[auto build test ERROR on rust/rust-next]
[also build test ERROR on akpm-mm/mm-nonmm-unstable kees/for-next/hardening kees/for-next/pstore kees/for-next/kspp linus/master v6.15-rc7]
[cannot apply to next-20250516]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Burak-Emir/rust-add-bindings-for-bitmap-h/20250520-002216
base: https://github.com/Rust-for-Linux/linux rust-next
patch link: https://lore.kernel.org/r/20250519161712.2609395-4-bqe%40google.com
patch subject: [PATCH v8 3/5] rust: add bitmap API.
config: x86_64-rhel-9.4-rust (https://download.01.org/0day-ci/archive/20250520/202505201339.bUfYKFye-lkp@intel.com/config)
compiler: clang version 18.1.8 (https://github.com/llvm/llvm-project 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff)
rustc: rustc 1.78.0 (9b00956e5 2024-04-29)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250520/202505201339.bUfYKFye-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202505201339.bUfYKFye-lkp@intel.com/
All errors (new ones prefixed by >>):
>> error[E0061]: this method takes 0 arguments but 1 argument was supplied
--> rust/kernel/bitmap.rs:396:28
|
396 | assert_eq!(None, b.last_bit(2048));
| ^^^^^^^^ ----
| |
| unexpected argument of type `{integer}`
| help: remove the extra argument
|
note: method defined here
--> rust/kernel/bitmap.rs:301:12
|
301 | pub fn last_bit(&self) -> Option<usize> {
| ^^^^^^^^
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
On Mon, May 19, 2025 at 6:24 PM Burak Emir <bqe@google.com> wrote:
> + /// Set bit with index `index`, atomically.
> + ///
> + /// ATTENTION: The naming convention differs from C, where the corresponding
> + /// function is called `set_bit`.
> + ///
> + /// # Safety
> + ///
> + /// This is a relaxed atomic operation (no implied memory barriers, no
> + /// ordering guarantees). The caller must ensure that this is safe, as
> + /// the compiler cannot prevent code with an exclusive reference from
> + /// calling atomic operations.
How can atomic operations through an exclusive reference be unsafe?
You can't have a data race between two atomic operations, and an
exclusive reference should anyway prevent any concurrency, right?
> + ///
> + /// # Panics
> + ///
> + /// Panics if `index` is greater than or equal to `self.nbits`.
> + #[inline]
> + pub unsafe fn set_bit_atomic(&self, index: usize) {
> + assert!(
> + index < self.nbits,
> + "Bit `index` must be < {}, was {}",
> + self.nbits,
> + index
> + );
> + // SAFETY: `index` is within bounds and the caller has ensured that
> + // there is no mix of non-atomic and atomic operations.
Aren't non-atomic operations only possible through a mutable
reference? And doesn't the rust compiler enforce that, if someone
holds a mutable reference, nobody else can hold any reference at all?
You wrote yourself above: "all (non-atomic) mutating operations
require a &mut reference which amounts to exclusive access."
I don't understand what's going on here, unless you're saying that
Rust does not enforce that an object ownership transfer between
threads has proper RELEASE/ACQUIRE (or RELEASE/CONSUME) memory
ordering or something like that?
> + unsafe { bindings::set_bit(index as u32, self.as_ptr() as *mut usize) };
> + }
On Mon, May 19, 2025 at 9:01 PM Jann Horn <jannh@google.com> wrote:
>
> On Mon, May 19, 2025 at 6:24 PM Burak Emir <bqe@google.com> wrote:
> > + /// Set bit with index `index`, atomically.
> > + ///
> > + /// ATTENTION: The naming convention differs from C, where the corresponding
> > + /// function is called `set_bit`.
> > + ///
> > + /// # Safety
> > + ///
> > + /// This is a relaxed atomic operation (no implied memory barriers, no
> > + /// ordering guarantees). The caller must ensure that this is safe, as
> > + /// the compiler cannot prevent code with an exclusive reference from
> > + /// calling atomic operations.
>
> How can atomic operations through an exclusive reference be unsafe?
> You can't have a data race between two atomic operations, and an
> exclusive reference should anyway prevent any concurrency, right?
The atomic operations take a &self (shared reference).
The patch is missing the implementation of Sync for now. With that,
one would get concurrent write access through shared references.
The "unsafe" here should serve as reminder to argue why it is ok to
not have any ordering guarantees.
The last sentence is supposed to say: when you have a &mut bitmap, you
can reborrow it as &bitmap, and then happily call this atomic op.
Even though it is unnecessary.
It is unsafe because if one has shared refs on multiple threads, one
needs to be ready to observe updates in any order.
Does it make more sense now? I am open to ideas how to describe this better.
> > + ///
> > + /// # Panics
> > + ///
> > + /// Panics if `index` is greater than or equal to `self.nbits`.
> > + #[inline]
> > + pub unsafe fn set_bit_atomic(&self, index: usize) {
> > + assert!(
> > + index < self.nbits,
> > + "Bit `index` must be < {}, was {}",
> > + self.nbits,
> > + index
> > + );
> > + // SAFETY: `index` is within bounds and the caller has ensured that
> > + // there is no mix of non-atomic and atomic operations.
>
> Aren't non-atomic operations only possible through a mutable
> reference? And doesn't the rust compiler enforce that, if someone
> holds a mutable reference, nobody else can hold any reference at all?
>
> You wrote yourself above: "all (non-atomic) mutating operations
> require a &mut reference which amounts to exclusive access."
As noted above, an exclusive ref can always be reborrowed as a shared ref.
> I don't understand what's going on here, unless you're saying that
> Rust does not enforce that an object ownership transfer between
> threads has proper RELEASE/ACQUIRE (or RELEASE/CONSUME) memory
> ordering or something like that?
Indeed without the Sync implementation, it does not make sense to have
atomic ops that take &self.
Sorry for the confusion, I should have added the Sync implementation.
The normal scenario is that concurrent access would be accompanied by
synchronization through a lock.
For things where the order does not matter, something like counters,
using the atomic ops would be ok.
Ownership transfer between threads (sending) is not possible right
now, unless we implement also `Send`.
> > + unsafe { bindings::set_bit(index as u32, self.as_ptr() as *mut usize) };
> > + }
On Mon, May 19, 2025 at 10:08 PM Burak Emir <bqe@google.com> wrote: > > The "unsafe" here should serve as reminder to argue why it is ok to > not have any ordering guarantees. `unsafe` should be used for unsafe functions, not as a general "danger" or "advanced" marker. (Having such a marker could be useful, but `unsafe fn` is not it) > The last sentence is supposed to say: when you have a &mut bitmap, you > can reborrow it as &bitmap, and then happily call this atomic op. > Even though it is unnecessary. I don't think that is related to safety preconditions. A "# Safety" section is intended to explain what the preconditions are. So, for instance, stating "The caller must ensure that this is safe" does not add much. Cheers, Miguel
On Mon, May 19, 2025 at 11:42 PM Miguel Ojeda <miguel.ojeda.sandonis@gmail.com> wrote: > > On Mon, May 19, 2025 at 10:08 PM Burak Emir <bqe@google.com> wrote: > > > > The "unsafe" here should serve as reminder to argue why it is ok to > > not have any ordering guarantees. > > `unsafe` should be used for unsafe functions, not as a general > "danger" or "advanced" marker. > > (Having such a marker could be useful, but `unsafe fn` is not it) > I can see the appeal of having a strict definition "safe = no UB". > > The last sentence is supposed to say: when you have a &mut bitmap, you > > can reborrow it as &bitmap, and then happily call this atomic op. > > Even though it is unnecessary. > > I don't think that is related to safety preconditions. A "# Safety" > section is intended to explain what the preconditions are. > > So, for instance, stating "The caller must ensure that this is safe" > does not add much. I see what you are saying. Not being sensitive to order is a precondition to a property. There are many different kinds of (colloquial) safety e.g. crash safety or data integrity. Sticking to a technical definition of safety has the advantage that one can be consistent. So I'll remove the unsafe marker then. Thanks, - Burak
On Mon, May 19, 2025 at 10:08 PM Burak Emir <bqe@google.com> wrote: > On Mon, May 19, 2025 at 9:01 PM Jann Horn <jannh@google.com> wrote: > > On Mon, May 19, 2025 at 6:24 PM Burak Emir <bqe@google.com> wrote: > > > + /// Set bit with index `index`, atomically. > > > + /// > > > + /// ATTENTION: The naming convention differs from C, where the corresponding > > > + /// function is called `set_bit`. > > > + /// > > > + /// # Safety > > > + /// > > > + /// This is a relaxed atomic operation (no implied memory barriers, no > > > + /// ordering guarantees). The caller must ensure that this is safe, as > > > + /// the compiler cannot prevent code with an exclusive reference from > > > + /// calling atomic operations. > > > > How can atomic operations through an exclusive reference be unsafe? > > You can't have a data race between two atomic operations, and an > > exclusive reference should anyway prevent any concurrency, right? > > The atomic operations take a &self (shared reference). > > The patch is missing the implementation of Sync for now. With that, > one would get concurrent write access through shared references. > > The "unsafe" here should serve as reminder to argue why it is ok to > not have any ordering guarantees. > > The last sentence is supposed to say: when you have a &mut bitmap, you > can reborrow it as &bitmap, and then happily call this atomic op. > Even though it is unnecessary. But using an atomic op when you have a &mut reference is not a safety issue, right? You wrote a comment about behavior with exclusive references in the "# Safety" comment block. If that's not supposed to be a safety problem, this should probably not be in the "# Safety" section?
On Mon, May 19, 2025 at 10:36:52PM +0200, Jann Horn wrote: > On Mon, May 19, 2025 at 10:08 PM Burak Emir <bqe@google.com> wrote: > > On Mon, May 19, 2025 at 9:01 PM Jann Horn <jannh@google.com> wrote: > > > On Mon, May 19, 2025 at 6:24 PM Burak Emir <bqe@google.com> wrote: > > > > + /// Set bit with index `index`, atomically. > > > > + /// > > > > + /// ATTENTION: The naming convention differs from C, where the corresponding > > > > + /// function is called `set_bit`. > > > > + /// > > > > + /// # Safety > > > > + /// > > > > + /// This is a relaxed atomic operation (no implied memory barriers, no > > > > + /// ordering guarantees). The caller must ensure that this is safe, as > > > > + /// the compiler cannot prevent code with an exclusive reference from > > > > + /// calling atomic operations. > > > > > > How can atomic operations through an exclusive reference be unsafe? > > > You can't have a data race between two atomic operations, and an > > > exclusive reference should anyway prevent any concurrency, right? > > > > The atomic operations take a &self (shared reference). > > > > The patch is missing the implementation of Sync for now. With that, > > one would get concurrent write access through shared references. > > > > The "unsafe" here should serve as reminder to argue why it is ok to > > not have any ordering guarantees. I don't think ordering is safety related. For example, relaxed atomics are still safe function. I think it's wrong to mark this as unsafe, do you have an example that you can construct an UB with pure safe code? Regards, Boqun > > > > The last sentence is supposed to say: when you have a &mut bitmap, you > > can reborrow it as &bitmap, and then happily call this atomic op. > > Even though it is unnecessary. > > But using an atomic op when you have a &mut reference is not a safety > issue, right? You wrote a comment about behavior with exclusive > references in the "# Safety" comment block. If that's not supposed to > be a safety problem, this should probably not be in the "# Safety" > section?
On Mon, May 19, 2025 at 10:07 PM Burak Emir <bqe@google.com> wrote: > > On Mon, May 19, 2025 at 9:01 PM Jann Horn <jannh@google.com> wrote: > > > > I don't understand what's going on here, unless you're saying that > > Rust does not enforce that an object ownership transfer between > > threads has proper RELEASE/ACQUIRE (or RELEASE/CONSUME) memory > > ordering or something like that? > > Indeed without the Sync implementation, it does not make sense to have > atomic ops that take &self. > Sorry for the confusion, I should have added the Sync implementation. Hang on, the Sync implementation is actually there in this patch! It was missing previously. Does that clarify things? cheers, Burak
On Mon, May 19, 2025 at 04:17:03PM +0000, Burak Emir wrote:
> Provides an abstraction for C bitmap API and bitops operations.
> We follow the C Bitmap API closely in naming and semantics, with
> a few differences that take advantage of Rust language facilities
> and idioms:
>
> * We leverage Rust type system guarantees as follows:
>
> * Ownership transfer of a Bitmap is restricted so that it cannot
> be passed between threads (does not implement `Send`).
Can you explain why you decided to not implement it? You can 'share' a
reference, but prohibit 'sending' it...
> * all (non-atomic) mutating operations require a &mut reference which
> amounts to exclusive access.
>
> * It is permissible to pass shared references &Bitmap between
> threads, and we expose atomic operations through interior mutability.
> Since these atomic operations do not provide any ordering guarantees,
> we mark these as `unsafe`. Anyone who calls the atomic methods needs
> to document that the lack of ordering is safe.
>
> * The Rust API offers `{set,clear}_bit` vs `{set,clear}_bit_atomic`,
> which is different from the C naming convention (set_bit vs __set_bit).
>
> * we include enough operations for the API to be useful, but not all
> operations are exposed yet in order to avoid dead code. This commit
This sentence and the following one say the same thing. Can you please
rephrase?
> includes enough to enable a Rust implementation of an Android Binder
> data structure that was introduced in commit 15d9da3f818c ("binder:
> use bitmap for faster descriptor lookup"), which can be found in
> drivers/android/dbitmap.h. We need this in order to upstream the Rust
> port of Android Binder driver.
>
> * We follow the C API closely and fine-grained approach to safety:
>
> * Low-level bit-ops methods get a safe API with bounds checks.
>
> * methods correspond to find_* C methods tolerate out-of-bounds
> arguments. Since these are already safe we the same behavior, and
> return results using `Option` types to represent "not found".
Nit: the above 2 lines look misaligned. Everywhere else you align
items such that new lines under asterisk align with the first
character, not the asterisk itself.
>
> * the Rust API is optimized to represent the bitmap inline if it would
> take the space of a pointer. This saves allocations which is
s/take the space of/fits into/
> relevant in the Binder use case.
>
> * Bounds checks where out-of-bounds values would not result in
> immediate access faults are configured via a RUST_BITMAP_HARDENED
> config.
>
> The underlying C bitmap is *not* exposed, and must never be exposed
> (except in tests). Exposing the representation would lose all static
> guarantees, and moreover would prevent the optimized representation of
> short bitmaps.
Does it mean that all existing kernel bitmaps declared in C are not
accessible in Rust as well?
> An alternative route of vendoring an existing Rust bitmap package was
> considered but suboptimal overall. Reusing the C implementation is
> preferable for a basic data structure like bitmaps. It enables Rust
> code to be a lot more similar and predictable with respect to C code
> that uses the same data structures and enables the use of code that
> has been tried-and-tested in the kernel, with the same performance
> characteristics whenever possible.
This should go in cover letter as well. Did you run any performance
tests against the native bitmaps?
> We use the `usize` type for sizes and indices into the bitmap,
> because Rust generally always uses that type for indices and lengths
> and it will be more convenient if the API accepts that type. This means
> that we need to perform some casts to/from u32 and usize, since the C
> headers use unsigned int instead of size_t/unsigned long for these
> numbers in some places.
>
> Adds new MAINTAINERS section BITMAP API [RUST].
>
> Suggested-by: Alice Ryhl <aliceryhl@google.com>
> Suggested-by: Yury Norov <yury.norov@gmail.com>
> Signed-off-by: Burak Emir <bqe@google.com>
> ---
> MAINTAINERS | 7 +
> rust/kernel/bitmap.rs | 415 +++++++++++++++++++++++++++++++++++++
> rust/kernel/lib.rs | 1 +
> security/Kconfig.hardening | 9 +
> 4 files changed, 432 insertions(+)
> create mode 100644 rust/kernel/bitmap.rs
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 04d6727e944c..565eaa015d9e 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -4127,6 +4127,13 @@ S: Maintained
> F: rust/helpers/bitmap.c
> F: rust/helpers/cpumask.c
>
> +BITMAP API [RUST]
> +M: Alice Ryhl <aliceryhl@google.com>
> +M: Burak Emir <bqe@google.com>
> +R: Yury Norov <yury.norov@gmail.com>
> +S: Maintained
> +F: rust/kernel/bitmap.rs
> +
> BITOPS API
> M: Yury Norov <yury.norov@gmail.com>
> R: Rasmus Villemoes <linux@rasmusvillemoes.dk>
> diff --git a/rust/kernel/bitmap.rs b/rust/kernel/bitmap.rs
> new file mode 100644
> index 000000000000..943dbef7948b
> --- /dev/null
> +++ b/rust/kernel/bitmap.rs
> @@ -0,0 +1,415 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +// Copyright (C) 2025 Google LLC.
> +
> +//! Rust API for bitmap.
> +//!
> +//! C headers: [`include/linux/bitmap.h`](srctree/include/linux/bitmap.h).
> +
> +use crate::alloc::{AllocError, Flags};
> +use crate::bindings;
> +use core::ptr::NonNull;
> +
> +/// Holds either a pointer to array of `unsigned long` or a small bitmap.
> +#[repr(C)]
> +union BitmapRepr {
> + bitmap: usize,
> + ptr: NonNull<usize>,
> +}
> +
> +macro_rules! bitmap_hardening_assert {
> + ($cond:expr, $($arg:tt)+) => {
> + #[cfg(RUST_BITMAP_HARDENED)]
> + assert!($e, $($arg)*);
> + }
> +}
> +
> +/// Represents a bitmap.
> +///
> +/// Wraps underlying C bitmap API.
> +///
> +/// # Examples
> +///
> +/// Basic usage
> +///
> +/// ```
> +/// use kernel::alloc::flags::GFP_KERNEL;
> +/// use kernel::bitmap::Bitmap;
> +///
> +/// let mut b = Bitmap::new(16, GFP_KERNEL)?;
> +///
> +/// assert_eq!(16, b.len());
> +/// for i in 0..16 {
> +/// if i % 4 == 0 {
> +/// b.set_bit(i);
> +/// }
> +/// }
> +/// assert_eq!(Some(0), b.next_bit(0));
> +/// assert_eq!(Some(1), b.next_zero_bit(0));
> +/// assert_eq!(Some(4), b.next_bit(1));
> +/// assert_eq!(Some(5), b.next_zero_bit(4));
> +/// assert_eq!(Some(12), b.last_bit());
> +/// # Ok::<(), Error>(())
> +/// ```
> +///
> +/// # Invariants
> +///
> +/// * `nbits` is `<= i32::MAX` and never changes.
> +/// * if `nbits <= bindings::BITS_PER_LONG`, then `repr` is a bitmap.
> +/// * otherwise, `repr` holds a non-null pointer that was obtained from a
> +/// successful call to `bitmap_zalloc` and holds the address of an initialized
> +/// array of `unsigned long` that is large enough to hold `nbits` bits.
> +pub struct Bitmap {
> + /// Representation of bitmap.
> + repr: BitmapRepr,
> + /// Length of this bitmap. Must be `<= i32::MAX`.
> + nbits: usize,
> +}
> +
> +/// Enable unsynchronized concurrent access to [`Bitmap`] through shared references.
> +///
> +/// # Safety
> +///
> +/// * When no thread performs any mutations, concurrent access is safe.
> +/// * Mutations are permitted through atomic operations and interior mutability.
> +/// All such methods are marked unsafe, to account for the lack of ordering
> +/// guarantees. Callers must acknowledge that updates may be observed in any
> +/// order.
> +unsafe impl Sync for Bitmap {}
> +
> +impl Drop for Bitmap {
> + fn drop(&mut self) {
> + if self.nbits <= bindings::BITS_PER_LONG as _ {
> + return;
> + }
> + // SAFETY: `self.ptr` was returned by the C `bitmap_zalloc`.
> + //
> + // INVARIANT: there is no other use of the `self.ptr` after this
> + // call and the value is being dropped so the broken invariant is
> + // not observable on function exit.
> + unsafe { bindings::bitmap_free(self.as_mut_ptr()) };
> + }
> +}
> +
> +impl Bitmap {
> + /// Constructs a new [`Bitmap`].
> + ///
> + /// Fails with [`AllocError`] when the [`Bitmap`] could not be allocated. This
> + /// includes the case when `nbits` is greater than `i32::MAX`.
> + #[inline]
> + pub fn new(nbits: usize, flags: Flags) -> Result<Self, AllocError> {
> + if nbits <= bindings::BITS_PER_LONG as _ {
> + return Ok(Bitmap {
> + repr: BitmapRepr { bitmap: 0 },
> + nbits,
> + });
> + }
> + if nbits > i32::MAX.try_into().unwrap() {
> + return Err(AllocError);
> + }
> + let nbits_u32 = u32::try_from(nbits).unwrap();
> + // SAFETY: `bindings::BITS_PER_LONG < nbits` and `nbits <= i32::MAX`.
> + let ptr = unsafe { bindings::bitmap_zalloc(nbits_u32, flags.as_raw()) };
> + let ptr = NonNull::new(ptr).ok_or(AllocError)?;
> + // INVARIANT: `ptr` returned by C `bitmap_zalloc` and `nbits` checked.
> + return Ok(Bitmap {
> + repr: BitmapRepr { ptr },
> + nbits,
> + });
> + }
> +
> + /// Returns length of this [`Bitmap`].
> + #[inline]
> + pub fn len(&self) -> usize {
> + self.nbits
> + }
> +
> + /// Returns a mutable raw pointer to the backing [`Bitmap`].
> + #[inline]
> + fn as_mut_ptr(&mut self) -> *mut usize {
> + if self.nbits <= bindings::BITS_PER_LONG as _ {
> + // SAFETY: Bitmap is represented inline.
> + unsafe { core::ptr::addr_of_mut!(self.repr.bitmap) }
> + } else {
> + // SAFETY: Bitmap is represented as array of `unsigned long`.
> + unsafe { self.repr.ptr.as_mut() }
> + }
> + }
> +
> + /// Returns a raw pointer to the backing [`Bitmap`].
> + #[inline]
> + fn as_ptr(&self) -> *const usize {
> + if self.nbits <= bindings::BITS_PER_LONG as _ {
> + // SAFETY: Bitmap is represented inline.
> + unsafe { core::ptr::addr_of!(self.repr.bitmap) }
> + } else {
> + // SAFETY: Bitmap is represented as array of `unsigned long`.
> + unsafe { self.repr.ptr.as_ptr() }
> + }
> + }
> +
> + /// Set bit with index `index`.
> + ///
> + /// ATTENTION: `set_bit` is non-atomic, which differs from the naming
> + /// convention in C code. The corresponding C function is `__set_bit`.
> + ///
> + /// # Panics
> + ///
> + /// Panics if `index` is greater than or equal to `self.nbits`.
> + #[inline]
> + pub fn set_bit(&mut self, index: usize) {
> + assert!(
> + index < self.nbits,
> + "Bit `index` must be < {}, was {}",
> + self.nbits,
> + index
> + );
Shouldn't this assertion be protected with hardening too? I already
said that: panicking on out-of-boundary access with hardening
disabled is a wrong way to go.
Can you turn your bitmap_hardening_assert() to just bitmap_assert(),
which panics only if hardening is enabled, and otherwise just prints
error with pr_err()?
Did you measure performance impact of hardening? Are those numbers in
cover letter collected with hardening enabled or disabled? If
performance impact is measurable, it should be mentioned in config
description.
> + // SAFETY: Bit `index` is within bounds.
> + unsafe { bindings::__set_bit(index as u32, self.as_mut_ptr()) };
> + }
> +
> + /// Set bit with index `index`, atomically.
> + ///
> + /// ATTENTION: The naming convention differs from C, where the corresponding
> + /// function is called `set_bit`.
> + ///
> + /// # Safety
> + ///
> + /// This is a relaxed atomic operation (no implied memory barriers, no
> + /// ordering guarantees). The caller must ensure that this is safe, as
> + /// the compiler cannot prevent code with an exclusive reference from
> + /// calling atomic operations.
> + ///
> + /// # Panics
> + ///
> + /// Panics if `index` is greater than or equal to `self.nbits`.
> + #[inline]
> + pub unsafe fn set_bit_atomic(&self, index: usize) {
> + assert!(
> + index < self.nbits,
> + "Bit `index` must be < {}, was {}",
> + self.nbits,
> + index
> + );
> + // SAFETY: `index` is within bounds and the caller has ensured that
> + // there is no mix of non-atomic and atomic operations.
> + unsafe { bindings::set_bit(index as u32, self.as_ptr() as *mut usize) };
> + }
> +
> + /// Clear `index` bit.
> + ///
> + /// ATTENTION: `clear_bit` is non-atomic, which differs from the naming
> + /// convention in C code. The corresponding C function is `__clear_bit`.
> + /// # Panics
> + ///
> + /// Panics if `index` is greater than or equal to `self.nbits`.
> + #[inline]
> + pub fn clear_bit(&mut self, index: usize) {
> + assert!(
> + index < self.nbits,
> + "Bit `index` must be < {}, was {}",
> + self.nbits,
> + index
> + );
> + // SAFETY: `index` is within bounds.
> + unsafe { bindings::__clear_bit(index as u32, self.as_mut_ptr()) };
> + }
> +
> + /// Clear `index` bit, atomically.
> + ///
> + /// ATTENTION: The naming convention differs from C, where the corresponding
> + /// function is called `clear_bit`.
> + ///
> + /// # Safety
> + ///
> + /// This is a relaxed atomic operation (no implied memory barriers, no
> + /// ordering guarantees). The caller must ensure that this is safe, as
Memory barriers is an implementation of 'ordering guarantees', so all
this sounds tautology.
> + /// the compiler cannot prevent code with an exclusive reference from
> + /// calling atomic operations.
> + ///
> + /// # Panics
> + ///
> + /// Panics if `index` is greater than or equal to `self.nbits`.
> + #[inline]
> + pub unsafe fn clear_bit_atomic(&self, index: usize) {
> + assert!(
> + index < self.nbits,
> + "Bit `index` must be < {}, was {}",
> + self.nbits,
> + index
> + );
> + // SAFETY: `index` is within bounds and the caller has ensured that
> + // there is no mix of non-atomic and atomic operations.
> + unsafe { bindings::clear_bit(index as u32, self.as_ptr() as *mut usize) };
> + }
> +
> + /// Copy `src` into this [`Bitmap`] and set any remaining bits to zero.
> + ///
> + /// # Examples
> + ///
> + /// ```
> + /// use kernel::alloc::{AllocError, flags::GFP_KERNEL};
> + /// use kernel::bitmap::Bitmap;
> + ///
> + /// let mut long_bitmap = Bitmap::new(256, GFP_KERNEL)?;
> + //
> + /// assert_eq!(None, long_bitmap.last_bit());
> + //
> + /// let mut short_bitmap = Bitmap::new(16, GFP_KERNEL)?;
> + //
> + /// short_bitmap.set_bit(7);
> + /// long_bitmap.copy_and_extend(&short_bitmap);
> + /// assert_eq!(Some(7), long_bitmap.last_bit());
> + ///
> + /// # Ok::<(), AllocError>(())
> + /// ```
> + #[inline]
> + pub fn copy_and_extend(&mut self, src: &Bitmap) {
> + let len = core::cmp::min(src.nbits, self.nbits);
> + // SAFETY: access to `self` and `src` is within bounds.
> + unsafe {
> + bindings::bitmap_copy_and_extend(
> + self.as_mut_ptr(),
> + src.as_ptr(),
> + len as u32,
> + self.nbits as u32,
> + )
> + };
> + }
> +
> + /// Finds last set bit.
> + ///
> + /// # Examples
> + ///
> + /// ```
> + /// use kernel::alloc::{AllocError, flags::GFP_KERNEL};
> + /// use kernel::bitmap::Bitmap;
> + ///
> + /// let bitmap = Bitmap::new(64, GFP_KERNEL)?;
> + ///
> + /// match bitmap.last_bit() {
> + /// Some(idx) => {
> + /// pr_info!("The last bit has index {idx}.\n");
> + /// }
> + /// None => {
> + /// pr_info!("All bits in this bitmap are 0.\n");
> + /// }
> + /// }
> + /// # Ok::<(), AllocError>(())
> + /// ```
> + #[inline]
> + pub fn last_bit(&self) -> Option<usize> {
> + // SAFETY: `_find_next_bit` access is within bounds due to invariant.
> + let index = unsafe { bindings::_find_last_bit(self.as_ptr(), self.nbits) };
> + if index >= self.nbits {
> + None
> + } else {
> + Some(index)
> + }
> + }
> +
> + /// Finds next set bit, starting from `start`.
> + /// Returns `None` if `start` is greater of equal than `self.nbits`.
> + #[inline]
> + pub fn next_bit(&self, start: usize) -> Option<usize> {
> + bitmap_hardening_assert!(start < self.nbits, "`start` must be < {} was {}", self.nbits, start);
> + // SAFETY: `_find_next_bit` tolerates out-of-bounds arguments and returns a
> + // value larger than or equal to `self.nbits` in that case.
> + let index = unsafe { bindings::_find_next_bit(self.as_ptr(), self.nbits, start) };
> + if index >= self.nbits {
> + None
> + } else {
> + Some(index)
> + }
> + }
> +
> + /// Finds next zero bit, starting from `start`.
> + /// Returns `None` if `start` is greater than or equal to `self.nbits`.
> + #[inline]
> + pub fn next_zero_bit(&self, start: usize) -> Option<usize> {
> + bitmap_hardening_assert!(start < self.nbits, "`start` must be < {} was {}", self.nbits, start);
> + // SAFETY: `_find_next_zero_bit` tolerates out-of-bounds arguments and returns a
> + // value larger than or equal to `self.nbits` in that case.
> + let index = unsafe { bindings::_find_next_zero_bit(self.as_ptr(), self.nbits, start) };
> + if index >= self.nbits {
> + None
> + } else {
> + Some(index)
> + }
> + }
> +}
> +
> +use macros::kunit_tests;
> +
> +#[kunit_tests(rust_kernel_bitmap)]
> +mod tests {
> + use super::*;
> + use kernel::alloc::flags::GFP_KERNEL;
> +
> + #[test]
> + fn bitmap_new() {
> + let b = Bitmap::new(0, GFP_KERNEL).unwrap();
> + assert_eq!(0, b.len());
> +
> + let b = Bitmap::new(3, GFP_KERNEL).unwrap();
> + assert_eq!(3, b.len());
> +
> + let b = Bitmap::new(1024, GFP_KERNEL).unwrap();
> + assert_eq!(1024, b.len());
> +
> + // Requesting too large values results in [`AllocError`].
> + let b = Bitmap::new(1 << 31, GFP_KERNEL);
> + assert!(b.is_err());
> + }
> +
> + #[test]
> + fn bitmap_set_clear_find() {
> + let mut b = Bitmap::new(128, GFP_KERNEL).unwrap();
> +
> + // Zero-initialized
> + assert_eq!(None, b.last_bit());
> +
> + b.set_bit(17);
> +
> + assert_eq!(Some(17), b.next_bit(0));
> + assert_eq!(Some(17), b.last_bit());
> +
> + b.set_bit(107);
> +
> + assert_eq!(Some(17), b.next_bit(0));
> + assert_eq!(Some(17), b.next_bit(17));
> + assert_eq!(Some(107), b.next_bit(18));
> + assert_eq!(Some(107), b.last_bit());
> +
> + b.clear_bit(17);
> +
> + assert_eq!(Some(107), b.next_bit(0));
> + assert_eq!(Some(107), b.last_bit());
> + }
> +
> + #[test]
> + fn bitmap_out_of_bounds() {
> + let mut b = Bitmap::new(128, GFP_KERNEL).unwrap();
> +
> + assert_eq!(None, b.next_bit(2048));
> + assert_eq!(None, b.next_zero_bit(2048));
> + assert_eq!(None, b.last_bit(2048));
> + }
> +
> + #[test]
> + fn bitmap_copy_and_extend() {
> + let mut long_bitmap = Bitmap::new(256, GFP_KERNEL).unwrap();
> +
> + long_bitmap.set_bit(3);
> + long_bitmap.set_bit(200);
> +
> + let mut short_bitmap = Bitmap::new(32, GFP_KERNEL).unwrap();
> +
> + short_bitmap.set_bit(17);
> +
> + long_bitmap.copy_and_extend(&short_bitmap);
> + // The larger bits have been cleared.
> + assert_eq!(Some(17), long_bitmap.next_bit(0));
> + assert_eq!(Some(17), long_bitmap.last_bit());
> + }
> +}
> diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
> index de07aadd1ff5..8c4161cd82ac 100644
> --- a/rust/kernel/lib.rs
> +++ b/rust/kernel/lib.rs
> @@ -38,6 +38,7 @@
> pub use ffi;
>
> pub mod alloc;
> +pub mod bitmap;
> #[cfg(CONFIG_BLOCK)]
> pub mod block;
> #[doc(hidden)]
> diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening
> index 3fe9d7b945c4..926665bbc8f2 100644
> --- a/security/Kconfig.hardening
> +++ b/security/Kconfig.hardening
> @@ -324,6 +324,15 @@ config LIST_HARDENED
>
> If unsure, say N.
>
> +config RUST_BITMAP_HARDENED
> + bool "Check integrity of linked list manipulation"
Wah?
> + help
> + Enables additional assertions in the Rust Bitmap API to catch
> + arguments that are not guaranteed to result in an immediate access
> + fault.
> +
> + If unsure, say N.
> +
> config BUG_ON_DATA_CORRUPTION
> bool "Trigger a BUG when data corruption is detected"
> select LIST_HARDENED
> --
> 2.49.0.1101.gccaa498523-goog
On Mon, May 19, 2025 at 8:22 PM Yury Norov <yury.norov@gmail.com> wrote:
>
> On Mon, May 19, 2025 at 04:17:03PM +0000, Burak Emir wrote:
> > Provides an abstraction for C bitmap API and bitops operations.
> > We follow the C Bitmap API closely in naming and semantics, with
> > a few differences that take advantage of Rust language facilities
> > and idioms:
> >
> > * We leverage Rust type system guarantees as follows:
> >
> > * Ownership transfer of a Bitmap is restricted so that it cannot
> > be passed between threads (does not implement `Send`).
>
> Can you explain why you decided to not implement it? You can 'share' a
> reference, but prohibit 'sending' it...
>
My intention here was to be conservative. It seems safe to send a
Bitmap to a different thread,
in the sense that it would not cause data races.
Maybe it would be better to commit now to keep Bitmap "send"able - for all time.
It would however constrain the API quite a bit. One could never use this API
with a thread-local bitmap.
> > * all (non-atomic) mutating operations require a &mut reference which
> > amounts to exclusive access.
> >
> > * It is permissible to pass shared references &Bitmap between
> > threads, and we expose atomic operations through interior mutability.
> > Since these atomic operations do not provide any ordering guarantees,
> > we mark these as `unsafe`. Anyone who calls the atomic methods needs
> > to document that the lack of ordering is safe.
> >
> > * The Rust API offers `{set,clear}_bit` vs `{set,clear}_bit_atomic`,
> > which is different from the C naming convention (set_bit vs __set_bit).
> >
> > * we include enough operations for the API to be useful, but not all
> > operations are exposed yet in order to avoid dead code. This commit
>
> This sentence and the following one say the same thing. Can you please
> rephrase?
Thanks for catching that, will do.
> > includes enough to enable a Rust implementation of an Android Binder
> > data structure that was introduced in commit 15d9da3f818c ("binder:
> > use bitmap for faster descriptor lookup"), which can be found in
> > drivers/android/dbitmap.h. We need this in order to upstream the Rust
> > port of Android Binder driver.
> >
> > * We follow the C API closely and fine-grained approach to safety:
> >
> > * Low-level bit-ops methods get a safe API with bounds checks.
> >
> > * methods correspond to find_* C methods tolerate out-of-bounds
> > arguments. Since these are already safe we the same behavior, and
> > return results using `Option` types to represent "not found".
>
> Nit: the above 2 lines look misaligned. Everywhere else you align
> items such that new lines under asterisk align with the first
> character, not the asterisk itself.
Yes, will fix.
> >
> > * the Rust API is optimized to represent the bitmap inline if it would
> > take the space of a pointer. This saves allocations which is
>
> s/take the space of/fits into/
>
> > relevant in the Binder use case.
> >
> > * Bounds checks where out-of-bounds values would not result in
> > immediate access faults are configured via a RUST_BITMAP_HARDENED
> > config.
> >
> > The underlying C bitmap is *not* exposed, and must never be exposed
> > (except in tests). Exposing the representation would lose all static
> > guarantees, and moreover would prevent the optimized representation of
> > short bitmaps.
>
> Does it mean that all existing kernel bitmaps declared in C are not
> accessible in Rust as well?
At the moment, we do not permit construction of a Rust Bitmap from an
existing C bitmap.
The point is more about the other direction though, not being able to
pass the Rust-owned bitmap to C code.
One could think of an API that requires an exclusively owned (no one
else has access) pointer to a C bitmap to Rust.
Though there is no general way to ensure that property, there are
situations where it would make sense (e.g. newly created).
> > An alternative route of vendoring an existing Rust bitmap package was
> > considered but suboptimal overall. Reusing the C implementation is
> > preferable for a basic data structure like bitmaps. It enables Rust
> > code to be a lot more similar and predictable with respect to C code
> > that uses the same data structures and enables the use of code that
> > has been tried-and-tested in the kernel, with the same performance
> > characteristics whenever possible.
>
> This should go in cover letter as well. Did you run any performance
> tests against the native bitmaps?
ok, I will mention it there. I have not run this against the Rust native bitmap.
I'd need to find out how to get a Rust native bitmap into kernel Rust code.
[...]
> > + /// Set bit with index `index`.
> > + ///
> > + /// ATTENTION: `set_bit` is non-atomic, which differs from the naming
> > + /// convention in C code. The corresponding C function is `__set_bit`.
> > + ///
> > + /// # Panics
> > + ///
> > + /// Panics if `index` is greater than or equal to `self.nbits`.
> > + #[inline]
> > + pub fn set_bit(&mut self, index: usize) {
> > + assert!(
> > + index < self.nbits,
> > + "Bit `index` must be < {}, was {}",
> > + self.nbits,
> > + index
> > + );
>
> Shouldn't this assertion be protected with hardening too? I already
> said that: panicking on out-of-boundary access with hardening
> disabled is a wrong way to go.
I considered it, but could not convince myself that __set_bit etc are
actually always safe.
For the methods that have the hardening assert, I was sure, but for
this one, not.
Are all bit ops guaranteed to handle out-of-bounds gracefully?
> Can you turn your bitmap_hardening_assert() to just bitmap_assert(),
> which panics only if hardening is enabled, and otherwise just prints
> error with pr_err()?
If there is no risk of undefined behavior, then I agree that checking
bounds is hardening.
If a missing bounds check loses safety, we then we should not skip it.
> Did you measure performance impact of hardening? Are those numbers in
> cover letter collected with hardening enabled or disabled? If
> performance impact is measurable, it should be mentioned in config
> description.
The hardening was enabled and it crossed my mind to mention it.
Given that not all methods have hardening, I though it might be misleading.
I'll have a more complete comparision and description in the next version.
[...]
> > + /// Clear `index` bit, atomically.
> > + ///
> > + /// ATTENTION: The naming convention differs from C, where the corresponding
> > + /// function is called `clear_bit`.
> > + ///
> > + /// # Safety
> > + ///
> > + /// This is a relaxed atomic operation (no implied memory barriers, no
> > + /// ordering guarantees). The caller must ensure that this is safe, as
>
> Memory barriers is an implementation of 'ordering guarantees', so all
> this sounds tautology.
>
ok, i will remove one of them.
[...]
> > diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening
> > index 3fe9d7b945c4..926665bbc8f2 100644
> > --- a/security/Kconfig.hardening
> > +++ b/security/Kconfig.hardening
> > @@ -324,6 +324,15 @@ config LIST_HARDENED
> >
> > If unsure, say N.
> >
> > +config RUST_BITMAP_HARDENED
> > + bool "Check integrity of linked list manipulation"
>
> Wah?
oh, thanks for catching that.
Thanks,
Burak
On Mon, May 19, 2025 at 10:41:58PM +0200, Burak Emir wrote:
> On Mon, May 19, 2025 at 8:22 PM Yury Norov <yury.norov@gmail.com> wrote:
> >
> > On Mon, May 19, 2025 at 04:17:03PM +0000, Burak Emir wrote:
> > > Provides an abstraction for C bitmap API and bitops operations.
> > > We follow the C Bitmap API closely in naming and semantics, with
> > > a few differences that take advantage of Rust language facilities
> > > and idioms:
> > >
> > > * We leverage Rust type system guarantees as follows:
> > >
> > > * Ownership transfer of a Bitmap is restricted so that it cannot
> > > be passed between threads (does not implement `Send`).
> >
> > Can you explain why you decided to not implement it? You can 'share' a
> > reference, but prohibit 'sending' it...
> >
>
> My intention here was to be conservative. It seems safe to send a
> Bitmap to a different thread,
> in the sense that it would not cause data races.
>
> Maybe it would be better to commit now to keep Bitmap "send"able - for all time.
> It would however constrain the API quite a bit. One could never use this API
> with a thread-local bitmap.
I'd implement the Send method, or just say that the method is to be
implemented in future when it will be needed.
> > > * all (non-atomic) mutating operations require a &mut reference which
> > > amounts to exclusive access.
> > >
> > > * It is permissible to pass shared references &Bitmap between
> > > threads, and we expose atomic operations through interior mutability.
> > > Since these atomic operations do not provide any ordering guarantees,
> > > we mark these as `unsafe`. Anyone who calls the atomic methods needs
> > > to document that the lack of ordering is safe.
> > >
> > > * The Rust API offers `{set,clear}_bit` vs `{set,clear}_bit_atomic`,
> > > which is different from the C naming convention (set_bit vs __set_bit).
> > >
> > > * we include enough operations for the API to be useful, but not all
> > > operations are exposed yet in order to avoid dead code. This commit
> >
> > This sentence and the following one say the same thing. Can you please
> > rephrase?
>
> Thanks for catching that, will do.
>
> > > includes enough to enable a Rust implementation of an Android Binder
> > > data structure that was introduced in commit 15d9da3f818c ("binder:
> > > use bitmap for faster descriptor lookup"), which can be found in
> > > drivers/android/dbitmap.h. We need this in order to upstream the Rust
> > > port of Android Binder driver.
> > >
> > > * We follow the C API closely and fine-grained approach to safety:
> > >
> > > * Low-level bit-ops methods get a safe API with bounds checks.
> > >
> > > * methods correspond to find_* C methods tolerate out-of-bounds
> > > arguments. Since these are already safe we the same behavior, and
> > > return results using `Option` types to represent "not found".
> >
> > Nit: the above 2 lines look misaligned. Everywhere else you align
> > items such that new lines under asterisk align with the first
> > character, not the asterisk itself.
>
> Yes, will fix.
>
> > >
> > > * the Rust API is optimized to represent the bitmap inline if it would
> > > take the space of a pointer. This saves allocations which is
> >
> > s/take the space of/fits into/
> >
> > > relevant in the Binder use case.
> > >
> > > * Bounds checks where out-of-bounds values would not result in
> > > immediate access faults are configured via a RUST_BITMAP_HARDENED
> > > config.
> > >
> > > The underlying C bitmap is *not* exposed, and must never be exposed
> > > (except in tests). Exposing the representation would lose all static
> > > guarantees, and moreover would prevent the optimized representation of
> > > short bitmaps.
> >
> > Does it mean that all existing kernel bitmaps declared in C are not
> > accessible in Rust as well?
>
> At the moment, we do not permit construction of a Rust Bitmap from an
> existing C bitmap.
> The point is more about the other direction though, not being able to
> pass the Rust-owned bitmap to C code.
>
> One could think of an API that requires an exclusively owned (no one
> else has access) pointer to a C bitmap to Rust.
> Though there is no general way to ensure that property, there are
> situations where it would make sense (e.g. newly created).
OK. Can you also mention it? And if we'll need to share bitmaps
between C and Rust modules, are you going to create a new class, or
extent the existing one?
> > > An alternative route of vendoring an existing Rust bitmap package was
> > > considered but suboptimal overall. Reusing the C implementation is
> > > preferable for a basic data structure like bitmaps. It enables Rust
> > > code to be a lot more similar and predictable with respect to C code
> > > that uses the same data structures and enables the use of code that
> > > has been tried-and-tested in the kernel, with the same performance
> > > characteristics whenever possible.
> >
> > This should go in cover letter as well. Did you run any performance
> > tests against the native bitmaps?
>
> ok, I will mention it there. I have not run this against the Rust native bitmap.
> I'd need to find out how to get a Rust native bitmap into kernel Rust code.
>
> [...]
>
> > > + /// Set bit with index `index`.
> > > + ///
> > > + /// ATTENTION: `set_bit` is non-atomic, which differs from the naming
> > > + /// convention in C code. The corresponding C function is `__set_bit`.
> > > + ///
> > > + /// # Panics
> > > + ///
> > > + /// Panics if `index` is greater than or equal to `self.nbits`.
> > > + #[inline]
> > > + pub fn set_bit(&mut self, index: usize) {
> > > + assert!(
> > > + index < self.nbits,
> > > + "Bit `index` must be < {}, was {}",
> > > + self.nbits,
> > > + index
> > > + );
> >
> > Shouldn't this assertion be protected with hardening too? I already
> > said that: panicking on out-of-boundary access with hardening
> > disabled is a wrong way to go.
>
> I considered it, but could not convince myself that __set_bit etc are
> actually always safe.
> For the methods that have the hardening assert, I was sure, but for
> this one, not.
>
> Are all bit ops guaranteed to handle out-of-bounds gracefully?
No. set_bit(), clear_bit() and test_bit() don't know the bitmap size
and can't check out-of-boundary.
But your Rust set_bit() does! You can check boundaries unconditionally,
and throw errors depending on the hardening config. If user wants to set
an out-of-boundary bit, you should just do nothing,
> > Can you turn your bitmap_hardening_assert() to just bitmap_assert(),
> > which panics only if hardening is enabled, and otherwise just prints
> > error with pr_err()?
>
> If there is no risk of undefined behavior, then I agree that checking
> bounds is hardening.
> If a missing bounds check loses safety, we then we should not skip it.
>
> > Did you measure performance impact of hardening? Are those numbers in
> > cover letter collected with hardening enabled or disabled? If
> > performance impact is measurable, it should be mentioned in config
> > description.
>
> The hardening was enabled and it crossed my mind to mention it.
> Given that not all methods have hardening, I though it might be misleading.
>
> I'll have a more complete comparision and description in the next version.
The hardening should be disabled for benchmarking reasons, isn't?
On Mon, May 19, 2025 at 10:42 PM Burak Emir <bqe@google.com> wrote:
> On Mon, May 19, 2025 at 8:22 PM Yury Norov <yury.norov@gmail.com> wrote:
> > On Mon, May 19, 2025 at 04:17:03PM +0000, Burak Emir wrote:
> > > + /// Set bit with index `index`.
> > > + ///
> > > + /// ATTENTION: `set_bit` is non-atomic, which differs from the naming
> > > + /// convention in C code. The corresponding C function is `__set_bit`.
> > > + ///
> > > + /// # Panics
> > > + ///
> > > + /// Panics if `index` is greater than or equal to `self.nbits`.
> > > + #[inline]
> > > + pub fn set_bit(&mut self, index: usize) {
> > > + assert!(
> > > + index < self.nbits,
> > > + "Bit `index` must be < {}, was {}",
> > > + self.nbits,
> > > + index
> > > + );
> >
> > Shouldn't this assertion be protected with hardening too? I already
> > said that: panicking on out-of-boundary access with hardening
> > disabled is a wrong way to go.
>
> I considered it, but could not convince myself that __set_bit etc are
> actually always safe.
> For the methods that have the hardening assert, I was sure, but for
> this one, not.
>
> Are all bit ops guaranteed to handle out-of-bounds gracefully?
>
> > Can you turn your bitmap_hardening_assert() to just bitmap_assert(),
> > which panics only if hardening is enabled, and otherwise just prints
> > error with pr_err()?
>
> If there is no risk of undefined behavior, then I agree that checking
> bounds is hardening.
> If a missing bounds check loses safety, we then we should not skip it.
There are no bounds checks in these C APIs, and there can't be,
because the C side does not store a length. bitmap_zalloc() just gives
you a raw array of bits (represented in C as an array of unsigned
longs), it's a very lightweight wrapper around kmalloc_array().
And if you expand __set_bit(nr, addr), you'll see that it turns into:
bitop(___set_bit, nr, addr)
which turns into:
((__builtin_constant_p(nr) &&
__builtin_constant_p((uintptr_t)(addr) != (uintptr_t)NULL) &&
(uintptr_t)(addr) != (uintptr_t)NULL &&
__builtin_constant_p(*(const unsigned long *)(addr))) ?
const___set_bit(nr, addr) : ___set_bit(nr, addr))
which (assuming a non-constant index) is:
___set_bit(nr, addr)
which is a debug-instrumented wrapper around
arch___set_bit(nr, addr)
which just leads to a raw assembly instruction (example from x86):
static __always_inline void
arch___set_bit(unsigned long nr, volatile unsigned long *addr)
{
asm volatile(__ASM_SIZE(bts) " %1,%0" : : ADDR, "Ir" (nr) : "memory");
}
© 2016 - 2025 Red Hat, Inc.