From nobody Mon Jun 8 06:36:46 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9FFD23911CA; Fri, 5 Jun 2026 10:34:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780655672; cv=none; b=i9stI0jTUZ2FQ+XXYPw9OkjmNkWB/3AVpvWqmOGbHfLQpSybzqb0kXcM7cfO+bCQtgp884kskrEgiZ4Hs5WXSDIXs9dq0lubI2IvVuZ7Un0uOPYpVCV+Lsvq1zwOJ/TZvi03vuWITy2dxi0jcizLaIoVNXG9CeBhm81R2sQ/BRg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780655672; c=relaxed/simple; bh=eRfxmUkQqrmh+7ydAl1CyAaP+7fYfnTrTU8habC1k08=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dhL1i0cNfFF/NCibC7IxHAhCwvfmzMwgAHwVbpiSWDk6u9s5tzT5i3N+izm8nMx+ZfF7A0T6dWfBruI2BekDqCGgOmBSrD4OdBpHbN+/y4ypUg/nGfOKVtPvxrPLgpnZWPcM3zmbP6NO83NUL7iD+qs6HXhNFlFFYn3PnkatIYk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=iGC2KChr; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="iGC2KChr" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5AA091F00893; Fri, 5 Jun 2026 10:34:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1780655671; bh=dFrQhcTjYjYlQDboe4vfQTAW28MKKhrj+2TnMlIO5uI=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=iGC2KChr+cM2ShXwX3tEWOKqXt5ArIT5thwI9IvtSsOdSElCRqDB5wZzh4JJuHkPm k0hqAYp/ZaOPUPhjIrphb10bvG3JyQcC/q1vkXtIN55OJMZS4QqO/agY3zwS9yOFYu i1JJGQR2W25GI8iWhaSaR8/YGXqWzYH4FKEQmHa6C5NAOOdaorhiYzPU/FczqiFVvy 5c7x6vV+sSk9dx26IDFXsQ2l1a67/eSfiJioaiqEbNSauQSPHBvKsOKlnuJvGr52u1 Z61ZJZIS99KE/OB1HRZi3lRAwKJlrizHbL1CnaKH3/MIKcu3sNbKZP8rfTmVpwVTB6 NfED15UIHCS2g== From: Andreas Hindborg Date: Fri, 05 Jun 2026 12:34:13 +0200 Subject: [PATCH v4 1/2] rust: sync: atomic: add atomic_per_byte_memcpy Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260605-page-volatile-io-v4-1-0e217d9bf11a@kernel.org> References: <20260605-page-volatile-io-v4-0-0e217d9bf11a@kernel.org> In-Reply-To: <20260605-page-volatile-io-v4-0-0e217d9bf11a@kernel.org> To: Alice Ryhl , Miguel Ojeda , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Trevor Gross , Danilo Krummrich , Will Deacon , Peter Zijlstra , Mark Rutland , Boqun Feng , Lorenzo Stoakes , "Liam R. Howlett" , Lorenzo Stoakes , "Liam R. Howlett" , Boqun Feng Cc: linux-mm@kvack.org, rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, Andreas Hindborg X-Mailer: b4 0.16-dev X-Developer-Signature: v=1; a=openpgp-sha256; l=3097; i=a.hindborg@kernel.org; h=from:subject:message-id; bh=eRfxmUkQqrmh+7ydAl1CyAaP+7fYfnTrTU8habC1k08=; b=owEBbQKS/ZANAwAKAfpQKQiqxb3QAcsmYgBqIqYpP6qiupR/U70KNHOs89WvnUNI/gtu/Zo+7 I2fFFhQiB2JAjMEAAEKAB0WIQRXitnI2WZ2JirAaob6UCkIqsW90AUCaiKmKQAKCRD6UCkIqsW9 0KeKEACD8pnjBqC3fn4EZ4nMld6CxkNc5dIvf62nnvTRQ9TfcKZ55qD3cn26nWR3IgsvXuOQ7FS sZMCsWA5S93ZlCU2egckrQByPuy6y02y1FFoFDWon2RzwGnk5rRSB5iwIweKS2UxmUXRI8slSxD A4+Fj6P9iBMXDSlOsVbSFVqgKBjU8i0WYExpQYvaLldVS3BgU/QB0ox4Xo0C5ac+0YiDYnc8WnY lUJ3cqJ7L7GOG/jj9IzFDy3HQvzvmLIZ7LGNaw1kzHNf6PxitsWdeg7GZZJ1GU4CiEd9soq4trD dhYxD2Z6pYiONUDXkrAukSUBGPhO/6Wggoo+nZJ/gG2t+4G5DlZsPs+KHMu97myYs70ZizEuzmh LbYRRyMNDK3dES8QEgRZTThIAD+MKhlREdR/FDBHi7leiFRzGBT2RM5i1NpL2c3xbDQHZOEKk79 +zu4SAMu4Ybii5E6FMJxIq6duOOVwNzCealyZ7QgmaP0YCF+lZBWlzO3Ode17DSPSDn1H+2YgV6 KYn6cGor9f91TO30z+Pes0qekQyGSA9adg/ick7oJqHC47OyZj6vvB7mINgIyagCCxNYmZAlVff 1SbIVuRcSPEqMNPrIIiVhut7pxJPJIYi6L+4pfGpvO3M4wZgSt2KL5FnnHeu7sp5GYOlFRx+5VI cHUC+hGGeYphYbA== X-Developer-Key: i=a.hindborg@kernel.org; a=openpgp; fpr=3108C10F46872E248D1FB221376EB100563EF7A7 Add a helper to copy `len` bytes from `src` to `dst` using byte-wise atomic memory operations. This is the concurrent-safe counterpart of `core::ptr::copy()` (the equivalent of standard C's `memcpy()`). Because of the atomicity at byte level, when the copy races with another concurrent atomic access (or when a normal read races with an atomic read), or with an external access (from DMA or userspace), the behavior of this function is defined: the memory is copied at (at least) byte granularity. The helper is the building block for higher-level byte-wise atomic copy methods (e.g., on `Page`) that need to remain defined under racing accesses, such as when copying to or from a buffer that may be concurrently accessed by userspace or DMA. The implementation forwards to the kernel's `memcpy()`, which is implemented in a way that byte-wise atomic memory load/store instructions are used. Signed-off-by: Andreas Hindborg --- rust/kernel/sync/atomic.rs | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/rust/kernel/sync/atomic.rs b/rust/kernel/sync/atomic.rs index 9cd009d57e35..3c76f1a14b53 100644 --- a/rust/kernel/sync/atomic.rs +++ b/rust/kernel/sync/atomic.rs @@ -848,3 +848,38 @@ pub unsafe fn cmpxchg( // per LKMM. unsafe { Atomic::from_ptr(ptr) }.cmpxchg(old, new, o) } + +/// Copy `len` bytes from `src` to `dst` using byte-wise atomic operations. +/// +/// This is the concurrent-safe counterpart of `core::ptr::copy()` (the eq= uivalent of standard +/// C's `memcpy()`). Because of the atomicity at byte level, when racing w= ith another concurrent +/// atomic access (or when a normal read races with an atomic read), or wi= th an external access +/// (from DMA or userspace), the behavior of this function is defined: the= memory is copied at +/// (at least) byte granularity. +/// +/// Implementation note: this is currently implemented by the kernel's `me= mcpy()`, which is +/// implemented in a way such that byte-wise atomic memory load/store inst= ructions are used. +/// +/// This copy operation is volatile. +/// +/// # Safety +/// +/// Callers must ensure that: +/// +/// - `src` is valid for atomic reads for `len` bytes for the duration of = the call. +/// - `dst` is valid for atomic writes for `len` bytes for the duration of= the call. +pub unsafe fn atomic_per_byte_memcpy(src: *const u8, dst: *mut u8, len: us= ize) { + // SAFETY: By the safety requirements of this function, the following = operation will not: + // - Trap. + // - Invalidate any reference invariants. + // - Race with any operation by the Rust AM, as `bindings::memcpy` is= a byte-wise atomic + // operation and all operations by the Rust AM to the involved memo= ry areas use byte-wise + // atomic semantics. + unsafe { + bindings::memcpy( + dst.cast::(), + src.cast::(), + len, + ) + }; +} --=20 2.51.2 From nobody Mon Jun 8 06:36:46 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25E323911CA; Fri, 5 Jun 2026 10:34:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780655677; cv=none; b=hnN6ZBbUf9pIvt+66tt6+nBTlLhY9XPcSJCW2zznh9UfFoPZPHwI857WkaiZ+PgWSS+ZInqqV9UUPytBWdPOQkzvtnObdtDOPKv1nIZXW1fCvTf5ESEuCfRPtvUDD270AWcALbqYFSuzJx1hqqB6DJ3cqSkXWoFW5l5IOZwZe3w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780655677; c=relaxed/simple; bh=DX8++9L27SP2fzIpUixQUyM6WiazGqLQWDakH66n1oU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=AnHUc52hROKrKF2hxo5g/z3qfEies6ix8443PnjSHppsad54QWZLYxQG4mZ28XoR64KnpUKzO+FdpjGlK+QidcABQ/v5iBVm/459jI4oOKwW8TNKoHdb8OAC+2X3ghakxtFaQ+WK+yNm9sJ2Md5EUceVIINPINnFQQW8mj7ZhFM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=IvGlW61y; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="IvGlW61y" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E5CFE1F00898; Fri, 5 Jun 2026 10:34:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1780655675; bh=+7JQf8IUtRGP37hIV4LYjpS7Iz51JvjpcMlTL4+STF0=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=IvGlW61yjtcZvNvfpsz8yYcP/YVEMZHbe342EhS45RnmQwwoDZs+vt62/lmCuslC7 O5TlQ7DD5SZqj3LobjPRTEJe717h6ycoIFa1S65p4beAOngF4wngZqGA67ARGikh5X KljHOYUEIm1PzUhCpRdAdjIX0U4ZHiajCpWu/vSiLHgMXRWUsNFp304/0oFs/cTaS9 jfP8RkvLLbVjhGKfi2hI1VPyuMw2vKVtIqz2y+QpWcsTadrr2COguCRyAWzEOVTytk 6voLYSlN0m5a6ryoKi4kgtbMbqSXcbE6bpjWW1JNrZGB4KGH6T9J8ieFdOxBzBlOMT zM3daR2QHGDTA== From: Andreas Hindborg Date: Fri, 05 Jun 2026 12:34:14 +0200 Subject: [PATCH v4 2/2] rust: page: add byte-wise atomic memory copy methods Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260605-page-volatile-io-v4-2-0e217d9bf11a@kernel.org> References: <20260605-page-volatile-io-v4-0-0e217d9bf11a@kernel.org> In-Reply-To: <20260605-page-volatile-io-v4-0-0e217d9bf11a@kernel.org> To: Alice Ryhl , Miguel Ojeda , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Trevor Gross , Danilo Krummrich , Will Deacon , Peter Zijlstra , Mark Rutland , Boqun Feng , Lorenzo Stoakes , "Liam R. Howlett" , Lorenzo Stoakes , "Liam R. Howlett" , Boqun Feng Cc: linux-mm@kvack.org, rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, Andreas Hindborg X-Mailer: b4 0.16-dev X-Developer-Signature: v=1; a=openpgp-sha256; l=5384; i=a.hindborg@kernel.org; h=from:subject:message-id; bh=DX8++9L27SP2fzIpUixQUyM6WiazGqLQWDakH66n1oU=; b=owEBbQKS/ZANAwAKAfpQKQiqxb3QAcsmYgBqIqYqBtBaekmbf4rV8ceuwFuR1DUVbhBXdQxvs 6JRnWUKVbyJAjMEAAEKAB0WIQRXitnI2WZ2JirAaob6UCkIqsW90AUCaiKmKgAKCRD6UCkIqsW9 0Dt4EACDRgLGFbSSmO6aZi4d1MvDVtc6UJr49O9Mj/u9i9d6Z8lMYIlaXwuVqPrSOJSzQ9EOPv4 QEb8KWZvGyZ1ZFBZsAHk85tnp0/vHY3d0RKRraMJv1gHnUd+a/bYu9+Dur9T0T34CaVCCXYR6Ox HNg1BUkyztNBystgcH8amPU3O42Oz/7ZSrueGexznPiGb0FaGN68v6WycMhZDDWZqWOm76J07zr 0GMRNY5QX86FBsfpSLYXd4wZ+/tRACuEjHRozKWsxdbsnm2jGuyZISaHSoiqg6rDSL3Sk8N//r2 JGWPNL/spaaC9tH7HjHuU4LxW2/aluG14bZUtoSIqZaTuI2l6rlDvPnJbbQ6ttXmogkJrrCQN9x uj4hZKNFURbI0jtLaUdLPjci+uS1Uxrw1eOq/YbhxhC0eHCbOPyszWvZS5tavW+TNvDAkQtEC3V ywp9MRaT/G5hSIYruo5U3YWFOR2b9I1vKOFtiT+ex+EkwV0arqPNHi7uzXC56fM9VJ9v9JkmrK0 hOv0tocKxONG951TG+S8k7r8Vbxww4fiMLrO3oDJZfZx628DOUpqzUBJRrWAT0Adu1Y3+R3yTh8 E5q8qwZX+yZLgr3qw9ui+6qY7rk8UDnE7dKYdgQLeRu1WTaGypp/mNSH9v0ihZDVBIQHNlDDpu0 6XihUunKp3R/1Gg== X-Developer-Key: i=a.hindborg@kernel.org; a=openpgp; fpr=3108C10F46872E248D1FB221376EB100563EF7A7 When copying data from buffers that are mapped to user space, it is impossible to guarantee absence of concurrent memory operations on those buffers. Copying data to/from `Page` from/to these buffers would be undefined behavior if no special considerations are made. Add `Page::{read,write}_bytewise_atomic` to read from / write to a page using byte-wise atomic operations layered on `atomic_per_byte_memcpy`. The methods are asymmetric: the parameter buffer must support byte-wise atomic accesses, while the page side held through `&self` only requires the absence of concurrent writes (or of concurrent reads or writes for the write variant). This follows the intended usage where the page is private to the caller and the parameter buffer may be shared (e.g., with userspace). Signed-off-by: Andreas Hindborg --- rust/kernel/page.rs | 70 +++++++++++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 70 insertions(+) diff --git a/rust/kernel/page.rs b/rust/kernel/page.rs index adecb200c654..7bb201442679 100644 --- a/rust/kernel/page.rs +++ b/rust/kernel/page.rs @@ -296,6 +296,38 @@ pub unsafe fn read_raw(&self, dst: *mut u8, offset: us= ize, len: usize) -> Result }) } =20 + /// Maps the page and reads from it into the given memory region using= byte-wise atomic memory + /// operations. + /// + /// This method will perform bounds checks on the page offset. If `off= set .. offset+len` goes + /// outside of the page, then this call returns [`EINVAL`]. + /// + /// This function is guaranteed to perform byte-wise atomic memory wri= tes to `dst`, but it may + /// perform only normal (non-atomic) memory reads from the [`Page`] `s= elf`. Accordingly, the + /// safety requirements below ask for byte-wise atomic discipline on `= dst` and only for absence + /// of concurrent writes on the source page. + /// + /// # Safety + /// + /// Callers must ensure that: + /// + /// - `dst` is valid for atomic writes for `len` bytes for the duratio= n of the call. + /// - This call does not race with a write to the source page that ove= rlaps with this read. + pub unsafe fn read_bytewise_atomic(&self, dst: *mut u8, offset: usize,= len: usize) -> Result { + self.with_pointer_into_page(offset, len, move |src| { + // SAFETY: + // - If `with_pointer_into_page` calls into this closure, then= it has performed a + // bounds check and guarantees that `src` is valid for `len`= bytes. + // - By function safety requirements `dst` is valid for writes= for `len` bytes. + // - By function safety requirements there are no other writes= to `src` during this + // call. + // - By function safety requirements all other access to `dst`= during this call are + // atomic. + unsafe { kernel::sync::atomic::atomic_per_byte_memcpy(src, dst= , len) }; + Ok(()) + }) + } + /// Maps the page and writes into it from the given buffer. /// /// This method will perform bounds checks on the page offset. If `off= set .. offset+len` goes @@ -317,6 +349,44 @@ pub unsafe fn write_raw(&self, src: *const u8, offset:= usize, len: usize) -> Res }) } =20 + /// Maps the page and writes into it from the given memory region usin= g byte-wise atomic memory + /// operations. + /// + /// This method will perform bounds checks on the page offset. If `off= set .. offset+len` goes + /// outside of the page, then this call returns [`EINVAL`]. + /// + /// This function is guaranteed to perform byte-wise atomic memory rea= ds from `src`, but it may + /// perform only normal (non-atomic) memory writes to the [`Page`] `se= lf`. Accordingly, the + /// safety requirements below ask for byte-wise atomic discipline on `= src` and only for absence + /// of concurrent reads or writes on the destination page. + /// + /// # Safety + /// + /// Callers must ensure that: + /// + /// - `src` is valid for atomic reads for `len` bytes for the duration= of the call. + /// - This call does not race with a read or write to the destination = page that overlaps with + /// this write. + pub unsafe fn write_bytewise_atomic( + &self, + src: *const u8, + offset: usize, + len: usize, + ) -> Result { + self.with_pointer_into_page(offset, len, move |dst| { + // SAFETY: + // - By function safety requirements `src` is valid for writes= for `len` bytes. + // - If `with_pointer_into_page` calls into this closure, then= it has performed a + // bounds check and guarantees that `dst` is valid for `len`= bytes. + // - By function safety requirements there are no other writes= to `dst` during this + // call. + // - By function safety requirements all other access to `src`= during this call are + // atomic. + unsafe { kernel::sync::atomic::atomic_per_byte_memcpy(src, dst= , len) }; + Ok(()) + }) + } + /// Maps the page and zeroes the given slice. /// /// This method will perform bounds checks on the page offset. If `off= set .. offset+len` goes --=20 2.51.2