From nobody Thu Dec 18 07:11:25 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC33E168AE6; Wed, 1 May 2024 23:01:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714604493; cv=none; b=PcdmXomb7tV+QV9oJxlSO1wdcxFr0OqIOM0Ns1zaVUrRfZ5HEeJQEy0uae5K8nCIgZr8Y0Y95zD1SN1FR7ixMsuYz1sTqBzmn7uIufYIRDcorTp9PogcwpqlcdnbF5GxNtdsNRe7qkhjcIFOLBI8DAi2Dzgm0gC9nZVDe3SnRQ4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714604493; c=relaxed/simple; bh=Zs6ak+wPmG4tPQkLrLcc6JxAJjrKyJZjKo6tat/+S/g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ubU8WkWbUWzO2Lw01U4Y122Alo4yFplFsIR8mVpiijMhKlEbDTdAoUNHSMdF7F+nY2tFSIBA4xNli4O2oAcxS0y9zg8jIaHVHX+o3o6vrZndsl5Leigcw2x1MLP7BF3wpRXaeKbiPR7J4T9TLXM8vcPj3C1G4lyDbHgHUbNToXY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XH9vIivI; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XH9vIivI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 95074C4AF51; Wed, 1 May 2024 23:01:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1714604492; bh=Zs6ak+wPmG4tPQkLrLcc6JxAJjrKyJZjKo6tat/+S/g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XH9vIivIX47KRQExC5xnU+LZC/uG5D0FeEfAoXkVPwo/PTEd9yqIYpCPKiZcS7ifl 3ERfMzuFeuTI57s+bH5QGzcUtcsD0t7XyR0NyJeMNKEesobUphmk37JASgd+wTWv9h jeCYJ+rXWPe8/H/pmlpj8BziFD6iXQYJ34DCpQAvMFw8tl4VZnG/kWqSOGfqulujH4 VrvpnKeTVU8fHuwPcCU/cFmzzdvGWyfOUKt+GjhBXzpXhXHgpVtwfnork9aIxwRNhk wfYkihPp4voUhbeAJqC5JMczxccduyhU+c+6FuCRt/x2j5E/AR8jAI+ntK/qSgNUx5 oXHnotmaZeiMQ== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id D3895CE2281; Wed, 1 May 2024 16:01:31 -0700 (PDT) From: "Paul E. McKenney" To: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org Cc: elver@google.com, akpm@linux-foundation.org, tglx@linutronix.de, peterz@infradead.org, dianders@chromium.org, pmladek@suse.com, arnd@arndb.de, torvalds@linux-foundation.org, kernel-team@meta.com, "Paul E. McKenney" Subject: [PATCH v2 cmpxchg 09/13] lib: Add one-byte emulation function Date: Wed, 1 May 2024 16:01:26 -0700 Message-Id: <20240501230130.1111603-9-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Architectures are required to provide four-byte cmpxchg() and 64-bit architectures are additionally required to provide eight-byte cmpxchg(). However, there are cases where one-byte cmpxchg() would be extremely useful. Therefore, provide cmpxchg_emu_u8() that emulates one-byte cmpxchg() in terms of four-byte cmpxchg(). Note that this emulations is fully ordered, and can (for example) cause one-byte cmpxchg_relaxed() to incur the overhead of full ordering. If this causes problems for a given architecture, that architecture is free to provide its own lighter-weight primitives. [ paulmck: Apply Marco Elver feedback. ] [ paulmck: Apply kernel test robot feedback. ] [ paulmck: Drop two-byte support per Arnd Bergmann feedback. ] Link: https://lore.kernel.org/all/0733eb10-5e7a-4450-9b8a-527b97c842ff@paul= mck-laptop/ Signed-off-by: Paul E. McKenney Acked-by: Marco Elver Cc: Andrew Morton Cc: Thomas Gleixner Cc: "Peter Zijlstra (Intel)" Cc: Douglas Anderson Cc: Petr Mladek Cc: Arnd Bergmann Cc: --- arch/Kconfig | 3 +++ include/linux/cmpxchg-emu.h | 15 +++++++++++++ lib/Makefile | 1 + lib/cmpxchg-emu.c | 45 +++++++++++++++++++++++++++++++++++++ 4 files changed, 64 insertions(+) create mode 100644 include/linux/cmpxchg-emu.h create mode 100644 lib/cmpxchg-emu.c diff --git a/arch/Kconfig b/arch/Kconfig index 9f066785bb71d..284663392eef8 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -1609,4 +1609,7 @@ config CC_HAS_SANE_FUNCTION_ALIGNMENT # strict alignment always, even with -falign-functions. def_bool CC_HAS_MIN_FUNCTION_ALIGNMENT || CC_IS_CLANG =20 +config ARCH_NEED_CMPXCHG_1_EMU + bool + endmenu diff --git a/include/linux/cmpxchg-emu.h b/include/linux/cmpxchg-emu.h new file mode 100644 index 0000000000000..998deec67740a --- /dev/null +++ b/include/linux/cmpxchg-emu.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * Emulated 1-byte and 2-byte cmpxchg operations for architectures + * lacking direct support for these sizes. These are implemented in terms + * of 4-byte cmpxchg operations. + * + * Copyright (C) 2024 Paul E. McKenney. + */ + +#ifndef __LINUX_CMPXCHG_EMU_H +#define __LINUX_CMPXCHG_EMU_H + +uintptr_t cmpxchg_emu_u8(volatile u8 *p, uintptr_t old, uintptr_t new); + +#endif /* __LINUX_CMPXCHG_EMU_H */ diff --git a/lib/Makefile b/lib/Makefile index ffc6b2341b45a..cc3d52fdb477d 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -236,6 +236,7 @@ obj-$(CONFIG_FUNCTION_ERROR_INJECTION) +=3D error-injec= t.o lib-$(CONFIG_GENERIC_BUG) +=3D bug.o =20 obj-$(CONFIG_HAVE_ARCH_TRACEHOOK) +=3D syscall.o +obj-$(CONFIG_ARCH_NEED_CMPXCHG_1_EMU) +=3D cmpxchg-emu.o =20 obj-$(CONFIG_DYNAMIC_DEBUG_CORE) +=3D dynamic_debug.o #ensure exported functions have prototypes diff --git a/lib/cmpxchg-emu.c b/lib/cmpxchg-emu.c new file mode 100644 index 0000000000000..27f6f97cb60dd --- /dev/null +++ b/lib/cmpxchg-emu.c @@ -0,0 +1,45 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Emulated 1-byte cmpxchg operation for architectures lacking direct + * support for this size. This is implemented in terms of 4-byte cmpxchg + * operations. + * + * Copyright (C) 2024 Paul E. McKenney. + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +union u8_32 { + u8 b[4]; + u32 w; +}; + +/* Emulate one-byte cmpxchg() in terms of 4-byte cmpxchg. */ +uintptr_t cmpxchg_emu_u8(volatile u8 *p, uintptr_t old, uintptr_t new) +{ + u32 *p32 =3D (u32 *)(((uintptr_t)p) & ~0x3); + int i =3D ((uintptr_t)p) & 0x3; + union u8_32 old32; + union u8_32 new32; + u32 ret; + + ret =3D READ_ONCE(*p32); + do { + old32.w =3D ret; + if (old32.b[i] !=3D old) + return old32.b[i]; + new32.w =3D old32.w; + new32.b[i] =3D new; + instrument_atomic_read_write(p, 1); + ret =3D data_race(cmpxchg(p32, old32.w, new32.w)); // Overridden above. + } while (ret !=3D old32.w); + return old; +} +EXPORT_SYMBOL_GPL(cmpxchg_emu_u8); --=20 2.40.1