From nobody Sat Feb 7 18:34:17 2026 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A10AD3346B9 for ; Tue, 3 Feb 2026 11:34:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770118468; cv=none; b=nPzMJrKDBKX+hTdaPnL24WhwHfTH3ljUtvnStspquMczalzZ3fd4nJrQ0/HP9VNfUBZK/6ntpz3u0Td61A105oWoXjqI106rd7He4LVuIIWHPOPlUqwht+K/LoLLY+R2koPeedpRCSVRg1RArIHZkv5prxLhjlKOx4kCeY6TTpU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770118468; c=relaxed/simple; bh=LTE4AJa8TogtsWMlXBs5xjtqn4b+TaIqmp+N7wyBtJM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=VI1vWVJ3ZwymvggBwAYl8+CImn+9+meMRhgB0KD9pjHu7IZJrVGF90Y8kNmbwARrBm/SbWTT4FcqDbIBqgF/MkziZRAltfiwp9IDoaskDSLGGsaeL4T2sjKPCCnDaV5dXF52rZLUgaKBEY5w+u4VkKHwKE13Q72QJJ3w9/4phSg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=xGq+DbK9; arc=none smtp.client-ip=209.85.208.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="xGq+DbK9" Received: by mail-ed1-f73.google.com with SMTP id 4fb4d7f45d1cf-65821c9e5b7so5461610a12.0 for ; Tue, 03 Feb 2026 03:34:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1770118464; x=1770723264; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=FR/i+eARikPdIZxYew6is+8VLxNQJTGYF2f9lrQ3EtM=; b=xGq+DbK9HZrDPyigyy81KPztLvXQ9tnhx+8J3W7HWY0ID0rS9wRWmfSNknN3lDtE4R ZuziL9KD+C4SjG8TfAcXpppQtVO/IyM6rKrVfhHhMqh3XwdDX3jbnNGw6Ni3yaR+aApv sRnioUQYUXcq/Z/l8LpZ/Vfi4mGjf6dT6T/BDnK52Lo6dwf63+0x47Ogc9YEychjZ0Lu ztQMGSZbdxyaR223DEg2225q/E710iRMwuz3Wcu8R4YHl1R5CXTIQu3AD78frLyHCypk 5vrsyxmBehFT5NSSGncWnvgNxXlYWVZB6Z2Ax9CuvsxtsM6Kq/AH+eE0ALWkwXCnR7KW jxhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770118464; x=1770723264; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FR/i+eARikPdIZxYew6is+8VLxNQJTGYF2f9lrQ3EtM=; b=DTPpQlLM324YBil4KCcz+gtnrj1VHHf15Nat0Jl5mP45gZr3n8eo/D+GtMhUBVAcJX B9m4SNmVRCTsd2wD8L/Y7TSQd+yZibhJVTz/5YGwYwHndM9MNS5SnfzhLO5652gbTYgh ZILI9SzFaf5pp3VjuAyIL33mfD42ArpSPFuU4DUBc2p3xae+VdC/jjJvkqcWPmM+ouii 7giStQHKavThQ2ywnG5QZESBTGuZeDxmNSAlwrUOHQoqe5c9fQ7ZJCHqfBZOEvEVfVYO A9ejKczC43m/DYlpVss4H4OuCzHPj1sZqrL2aVGxa1BvF5W6XBz1PvyE4gDtGTqys1Fo mJww== X-Forwarded-Encrypted: i=1; AJvYcCX0gyoJdnNf+nPoMg+MEvdLsRwR4GAbiNUC4K+zbgPanktNE9ANflmKCMyqB2OstbMxl2rgwQiKZYUGfK8=@vger.kernel.org X-Gm-Message-State: AOJu0YyVIQ+ho00P+rn+8E1w0ZWLEMFLak82cDOl8lJCHWQkCMYk37oB zJcY//waBvzxle1Dte28bUGPKII8Ls1pXa7RuzGMBbH6sCanOWgQcAWcW26tkKxrAGGlbGJ4zw/ RSzALpd5B/XImKX+EDA== X-Received: from edck20.prod.google.com ([2002:a05:6402:3414:b0:659:31dd:52cb]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:2695:b0:659:4383:c491 with SMTP id 4fb4d7f45d1cf-6594383c6bfmr200660a12.33.1770118463947; Tue, 03 Feb 2026 03:34:23 -0800 (PST) Date: Tue, 03 Feb 2026 11:34:08 +0000 In-Reply-To: <20260203-inline-helpers-v2-0-beb8547a03c9@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260203-inline-helpers-v2-0-beb8547a03c9@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=1712; i=aliceryhl@google.com; h=from:subject:message-id; bh=FzzjhSQNQHIbmcapK6yaktEju2IYPpFJD84GeqRslPI=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBpgd09+3ffdigiyO2/yaYtXtLiJKV4JefwxHsD6 g8QYpJOBz+JAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCaYHdPQAKCRAEWL7uWMY5 Rn3ZEACPLsut+e2SxZtW2kE96JBHj9IsiPLxVorIorli9qVvROiAeojumIiArRAp1voBxa9BiHF LWm7/7tkVv4Pb5qS9HYATKtorNFgunu5fSYL+Ybvytw3bTpze4jLe5qV3lgnOIEGHkjYr35i9b9 se6vm0FtXtYcFLV3cIz+TnswQOEUo9RnwLzPLlY6U3mzKQk+DIQ7J3c8p/9Z5SCvSy6dJUCSJMd TjqQZ780e7J87CiLHFqHLabrjyMgdzBQjumAEdUJtXT1lSjJyky5JjOA8lAH/Fc3zie3LoPMXCy 0OZfvtxwd4uET0cX1OszPfbBV0wMNlL3RejRdb0SZoXDS+2e8nPT9EFeY7wry6jXsEYIj0Ars+o XnYdBzBqXepyP0wVxUX15DwlpGtldajTPlVNcgP+tZotK+vqXYRRN2p9QjrMZiENvc1Q6bLmVyF D+KGm9tuJgKaCdLi887JUpYshggFLO0lrvJGGZb5eCFtIAdAR7T/LrbRGxBomcERILGtXbQHeuR 1EyOuFeN9V2sWEPS35jlR41VCeY3DIU5+G3zg9eoSguH2wzcQ6jYTyVy75DhBv8loGVQYV9f6+M JDN4EOaLwhbxIro5SFcjPzVDMPxxA48Q4QhyWXri+NX4Dm6NHKDX42CavrWJTc/eX59WGZDKuOS 8wKBxTR4AClAIeQ== X-Mailer: b4 0.14.2 Message-ID: <20260203-inline-helpers-v2-1-beb8547a03c9@google.com> Subject: [PATCH v2 1/3] kbuild: rust: add `CONFIG_RUSTC_CLANG_LLVM_COMPATIBLE` From: Alice Ryhl To: Miguel Ojeda Cc: Boqun Feng , Gary Guo , "=?utf-8?q?Bj=C3=B6rn_Roy_Baron?=" , Benno Lossin , Andreas Hindborg , Trevor Gross , Danilo Krummrich , Alexandre Courbot , Will Deacon , Peter Zijlstra , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Nicolas Schier , Andrew Morton , Uladzislau Rezki , rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-kbuild@vger.kernel.org, linux-mm@kvack.org, Alice Ryhl , Matthew Maurer Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Gary Guo This config detects if Rust and Clang have matching LLVM major version. All IR or bitcode operations (e.g. LTO) rely on LLVM major version to be matching, otherwise it may generate errors, or worse, miscompile silently due to change of IR semantics. It's usually suggested to use the exact same LLVM version, but this can be difficult to guarantee. Rust's suggestion [1] is also major-version only, so I think this check is sufficient for the kernel. Link: https://doc.rust-lang.org/rustc/linker-plugin-lto.html [1] Reviewed-by: Andreas Hindborg Signed-off-by: Gary Guo Signed-off-by: Matthew Maurer Signed-off-by: Alice Ryhl --- init/Kconfig | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/init/Kconfig b/init/Kconfig index e95d43457851862afc8313389777e4dd9348c178..0e900d3d8be7874a33e0f44754a= 8d038e68d7e65 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -82,6 +82,21 @@ config RUSTC_LLVM_VERSION int default $(rustc-llvm-version) =20 +config RUSTC_LLVM_MAJOR_VERSION + int + default $(shell,expr $(rustc-llvm-version) / 10000) + +config RUSTC_CLANG_LLVM_COMPATIBLE + bool + default y if CC_IS_CLANG && RUSTC_LLVM_MAJOR_VERSION =3D $(shell,expr $(c= c-version) / 10000) + help + This indicates whether Rust and Clang use LLVM of the same major + version. + + Operations involving handling LLVM IR or bitcode (e.g. cross-language + LTO) requires the same LLVM major version to work properly. For best + compatibility it is recommended that the exact same LLVM is used. + config ARCH_HAS_CC_CAN_LINK bool =20 --=20 2.53.0.rc1.225.gd81095ad13-goog From nobody Sat Feb 7 18:34:17 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E19593358B7 for ; Tue, 3 Feb 2026 11:34:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770118469; cv=none; b=bwBGRJAWiQsu1OvWBtTELRQn4HlXbwrw/9mMaturyAtqi0DCsvr7lQP7MxlHQgwbdS0CyFG4Th0Gj9raB+7BfW6SX0p7G7p8XtUW9+hsIayi0gWxJ59s9tw+wkOq1UnMwl2D7y4qv09MM7mhtvi/9xeTZuZ8lAugnFk3G2HFbFo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770118469; c=relaxed/simple; bh=ArCa741Tr0ZegW0iFyhR+FgMhtwZFXdT5kD1PX345c8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hzUte9uc3vxYjmw/nMa1uBHDr280ZdY+YfSYGuxm4h+MDMdaOmNiLPYar6XC7sp8JUqGZ8lNCvzCH4goPPyEuRKRejLnK4oheD/A8aogOhxXz2nroEvrvteHViYbup+kqllW0wdkdtACoFh6jD91r8u0WHmjEcklqUC2Y4e5QXg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=SJwkGlHJ; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SJwkGlHJ" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-47ee056e5cfso50549215e9.1 for ; Tue, 03 Feb 2026 03:34:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1770118465; x=1770723265; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=44BM+t61qGHb8bSQxUBenNw4K81sfYTOIHgLXsB72RY=; b=SJwkGlHJR7WuQTdqa23A0LFh3/dpTZGUcmsI2fctEataXU0USY4amki1iUupkLZ4sx Q5UeVmYjqLS5WaNiGfnbaeTqn2ZVnwW+6iTHynLChFs5WX/g+yd20b3HMCJh+/aeNrFJ hiQ7dWhjqMpr312fNIPr701WpxvNb3bmuQcdMAU4+Edj3PkVgehzFrX+MES0jmi00CNE t5lu0e+IzumtGHnq8imINbfG5gbU86bG2fr/VOJP3iHHEgdHHfIb5QgRifN8DbNkWF6u yGZTsJXIxOsmZXataB4MOR6Lr7+ba0m0EHyOmTRCKQk1wETpR7LXHirA4euZITehMpS+ Fd5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770118465; x=1770723265; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=44BM+t61qGHb8bSQxUBenNw4K81sfYTOIHgLXsB72RY=; b=Sf2+AZiwr+FRFYRfl19x7j9ycqKIG55LowCQrsvYscdpVTN161UCY9Wm0VNZwm4Dn6 a7V2s26D4SdOpvqaIeWjMUkaSrbMw/OGhq6bgmc1IThFilQfGKLVPfmtlgphmELaZMwy mh4RSUNZWAtVlhhTQbi2S+eG532D0nL6xGc0SptlPLHe0ct+QWv8B9tCeMKoAoaKRFQ4 6sU+JEuEfxV+TOOBSDCV5Ipx2+NRYrYELaQAQGD47wP7x+LnvxmXLjASeOe98OJdgxTv +IuqgSOzBMRuZj8UrZK3aQO6eAMC5DDS3GEtXRXJIiMotY6llBNsnUHa05lpKnVQU48D ZVRg== X-Forwarded-Encrypted: i=1; AJvYcCWjb2U9ZZRI//0VqTK2YPiRBB2bs0Pr9msYsiXxwjtmCXRXOENmdzSzJyLHEA3RmlPvAF4SYTPMmZEmSBI=@vger.kernel.org X-Gm-Message-State: AOJu0YyIz0Ue8ceEWlzwDf/WB0pepKiD2KJd1HpdwU84VVMufZbCpPX8 mpXh9kyrM9ke79J8MqvhKjNiw9KtGNLgYQ/nY/hltyL2UbNpKA+xs+Jz6azTQTRQ9fzByD6qrMy Yx9sk48kfyStAHVsh6Q== X-Received: from wmoo1.prod.google.com ([2002:a05:600d:101:b0:480:6bd0:3fcd]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:8115:b0:47e:e2b0:15b8 with SMTP id 5b1f17b1804b1-482db447c40mr231437485e9.4.1770118465517; Tue, 03 Feb 2026 03:34:25 -0800 (PST) Date: Tue, 03 Feb 2026 11:34:09 +0000 In-Reply-To: <20260203-inline-helpers-v2-0-beb8547a03c9@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260203-inline-helpers-v2-0-beb8547a03c9@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=3615; i=aliceryhl@google.com; h=from:subject:message-id; bh=c8KaIUkKoLbnzKPHztCk8a85+2NFtPPqK1S79zJ3Ew4=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBpgd09MDQTJ/taK3Lf49afHdmnGD/aHx2441aqI WiSFym9hA2JAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCaYHdPQAKCRAEWL7uWMY5 Rmo1EACd7QneqQSnMGd/PSyupu+WC8Qo0Dt5lLlYv+YuBAghZSMC40dNomgCxRecczFi73AtCF1 xQWsJUbfpi0qSHg5D7b2yhKZgm2z8o97sciV2yYOLmzty/siHbIAK4DzUqZrht3LqWAF1oUQHZh DZQWpVnLt372AzmSfnAlngaFeryk+5Kgkrgj8VkmOsZb0f/xK52bAKdefYYqtY0NyMo+C0RtjGU R0X1L2Xne/PyIsIUPVchOAKrL8I3Bmwfq81tUDW2u84eHRkQXBYRNFsWIfaOqq+5mddEBNm7Wkm 6kacZLAWeLreMRtqKFla1B9Y31g119GAB3R9NU8CDupVLiipPz+1zSJ/nYpoZ1T+G5fgYIbmfme isbeGXOU8NQF0ehV9rIBnZLk6SuqqYCFi0JuBAcTohsSvO2R7yEZK8mrzo8lyYW8f/WkZyokuXL AmxVqO2vl+gLIVAcRgScCC7Ty3xbpRkArdS/tDt+UfL5G36cKcwWR+AloosJ2L0C0AdHIGi+wwe T47u6O6NNstj2VJOHi1udvRWBa+KzCZjguMjfUkDTni9AoNss0l8UpkxyJVL6z02jTc8lMszCBx 1tnhbxOr9prmPAO93SH8qQaU/DGebUpg9wIhTl0mErECRp1F0zHk5b9nINJPIm52x/kIJotfQZa GXngUhbk/tn3KQA== X-Mailer: b4 0.14.2 Message-ID: <20260203-inline-helpers-v2-2-beb8547a03c9@google.com> Subject: [PATCH v2 2/3] rust: helpers: #define __rust_helper From: Alice Ryhl To: Miguel Ojeda Cc: Boqun Feng , Gary Guo , "=?utf-8?q?Bj=C3=B6rn_Roy_Baron?=" , Benno Lossin , Andreas Hindborg , Trevor Gross , Danilo Krummrich , Alexandre Courbot , Will Deacon , Peter Zijlstra , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Nicolas Schier , Andrew Morton , Uladzislau Rezki , rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-kbuild@vger.kernel.org, linux-mm@kvack.org, Alice Ryhl Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Gary Guo Because of LLVM inling checks, it's generally not possible to inline a C helper into Rust code, even with LTO: * LLVM doesn't want to inline functions compiled with `-fno-delete-null-pointer-checks` with code compiled without. The C CGUs all have this enabled and Rust CGUs don't. Inlining is okay since this is one of the hardening features that does not change the ABI, and we shouldn't have null pointer dereferences in these helpers. * LLVM doesn't want to inline functions with different list of builtins. C side has `-fno-builtin-wcslen`; `wcslen` is not a Rust builtin, so they should be compatible, but LLVM does not perform inlining due to attributes mismatch. * clang and Rust doesn't have the exact target string. Clang generates `+cmov,+cx8,+fxsr` but Rust doesn't enable them (in fact, Rust will complain if `-Ctarget-feature=3D+cmov,+cx8,+fxsr` is used). x86-64 always enable these features, so they are in fact the same target string, but LLVM doesn't understand this and so inlining is inhibited. This can be bypassed with `--ignore-tti-inline-compatible`, but this is a hidden option. To fix this, we can add __always_inline on every helper, which skips these LLVM inlining checks. For this purpose, introduce a new __rust_helper macro that needs to be added to every helper. Most helpers already have __rust_helper specified, but there are a few missing. The only consequence of this is that those specific helpers do not get inlined. Signed-off-by: Gary Guo Signed-off-by: Alice Ryhl --- rust/helpers/helpers.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/rust/helpers/helpers.c b/rust/helpers/helpers.c index a3c42e51f00a0990bea81ebce6e99bb397ce7533..e05c6e7e4abb7e6a4c4a3a417e3= 5022dac1d9c58 100644 --- a/rust/helpers/helpers.c +++ b/rust/helpers/helpers.c @@ -7,7 +7,36 @@ * Sorted alphabetically. */ =20 +#include + +#ifdef __BINDGEN__ +// Omit `inline` for bindgen as it ignores inline functions. #define __rust_helper +#else +// The helper functions are all inline functions. +// +// We use `__always_inline` here to bypass LLVM inlining checks, in case t= he +// helpers are inlined directly into Rust CGUs. +// +// The LLVM inlining checks are false positives: +// * LLVM doesn't want to inline functions compiled with +// `-fno-delete-null-pointer-checks` with code compiled without. +// The C CGUs all have this enabled and Rust CGUs don't. Inlining is okay +// since this is one of the hardening features that does not change the = ABI, +// and we shouldn't have null pointer dereferences in these helpers. +// * LLVM doesn't want to inline functions with different list of builtins= . C +// side has `-fno-builtin-wcslen`; `wcslen` is not a Rust builtin, so th= ey +// should be compatible, but LLVM does not perform inlining due to attri= butes +// mismatch. +// * clang and Rust doesn't have the exact target string. Clang generates +// `+cmov,+cx8,+fxsr` but Rust doesn't enable them (in fact, Rust will +// complain if `-Ctarget-feature=3D+cmov,+cx8,+fxsr` is used). x86-64 al= ways +// enable these features, so they are in fact the same target string, but +// LLVM doesn't understand this and so inlining is inhibited. This can be +// bypassed with `--ignore-tti-inline-compatible`, but this is a hidden +// option. +#define __rust_helper __always_inline +#endif =20 #include "atomic.c" #include "atomic_ext.c" --=20 2.53.0.rc1.225.gd81095ad13-goog From nobody Sat Feb 7 18:34:17 2026 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C34D33986C for ; Tue, 3 Feb 2026 11:34:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770118470; cv=none; b=dAIxU1+1JQTaJvWg5EQQfP7gD3YmnuWJCHF9IIkUS4tUoYIy5WRpWCv//3i1RcQR57VimZJCRIsDlyYlQCjkn+LxGDLwO+galjXNaySTMXQey9+Doo14DNnKttSSOFgYaEj7/uXY+M9mYrsopuJJh9dUMh0L/4YAyz9F2rbEyck= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770118470; c=relaxed/simple; bh=//P1rk5eFGynhrO1/9spa/pn1mv9PbMqJLjl3bUpxoU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Y3XD0Wh8dO0utqNSMQBxFtkyKD7Z6jKB8E7GYdpJjnBe08Uu9st6GdxSD8wz1qp7VeAtLyOceqiMdVaA9D7CX0nCjfgPYeyzUOznK5KIXY4uBIoUCPu6wYMMmdp/wLB5+Znlc8SRbl72uu/psO45qKKgW3Y6Kh8gZyNY/UDw3zg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=RgsvswK7; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="RgsvswK7" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-430ffc4dc83so5617028f8f.3 for ; Tue, 03 Feb 2026 03:34:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1770118467; x=1770723267; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=aE0I4qCeFvAhyrBrsfU7k4O+fmQQ+xM6mESOeLTMWYY=; b=RgsvswK7bV7MH2UzZsRE3lIJfuNh4EG6g99y3V89dlVOBojTMjuwUM0jf1FGpCT8E1 7wIzkSUeFzH+rZOXOaDyHhm+6B5TB1slc7Z0eoAPxN3jbiv4mocdugNIxOdeO3FBZpK+ jPVt9esr1mDlpp7s/s1j2XAr0AOjxIygUJSaBI/xzvMIS9FG8fCTdyDvGksZg70NZw9W w1Wk0LdRbwPkKpEOf1DPz0hg4glka+JQr/ohd1+O503NwuZj8sx7l1YBSAFbaQbMrb+u Z4lXxYSibXYXwUFTAYQfh3z92DlQ5EQbX3vOdw9FnPpMlhvrlPwdTapPv/zYkwRtfM2n A2Gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770118467; x=1770723267; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=aE0I4qCeFvAhyrBrsfU7k4O+fmQQ+xM6mESOeLTMWYY=; b=qT0tmJ8s/4wtJ1CniEIuoz81Tr1pTIUx240JI+OBkJxxm05G6/thmyOqWIEEoU3+uU Xg07JdeGa6QuR2Q1J2G2462cHRuUVxWrrW+ncPE/O1l4yZNMohtMwjN5QuVXbsUmFHEg NKoW022HVvTapgsUOgMpepTXw/AWcWkXUHUXXCKtGUlJd8a02styTVj5ALnztcV9OcoM HDs45laIwzEHvt/2YphyMcawEaeU1aL6foaT9gvmUkDplO7tGbSsffkngkrTZH0tyJjF bondfqBDQvfM3cNkXVB3KsqAwo9gdjCK70iJnMyjCYyJH2N5/o2PhRiFhJVEufL04oEM WMeg== X-Forwarded-Encrypted: i=1; AJvYcCXPLUqeexIQ8bVVtfQGrbr24BPzzD8fd5+cg/pol0mGk5AXBeBlROsW0TfE8NpSmeIupxOivXC1gxb9m4g=@vger.kernel.org X-Gm-Message-State: AOJu0YwfpKfgtx3YcX/pOQ1k46GsDmRETSN4smnH/hccqrY2/lxNFBfx AX7yxT4mdVysHKg0plOK4cWgi3SelFNATXR/1itDcYec1Ty/JQ/Z5FzP11JaSuCH56IPW2CkIF1 vcryvOUOFulvuF2VHbg== X-Received: from wrfu10.prod.google.com ([2002:a05:6000:38a:b0:436:1564:8b38]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:26c6:b0:435:a815:dd81 with SMTP id ffacd0b85a97d-435f3a6a9e4mr23351106f8f.11.1770118466723; Tue, 03 Feb 2026 03:34:26 -0800 (PST) Date: Tue, 03 Feb 2026 11:34:10 +0000 In-Reply-To: <20260203-inline-helpers-v2-0-beb8547a03c9@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260203-inline-helpers-v2-0-beb8547a03c9@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=9996; i=aliceryhl@google.com; h=from:subject:message-id; bh=ccYqFunpYUJ3E9tFyGaM8o4GOz90mNx5YokAm40heqc=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBpgd09hi110q+7UOnBQB1eL+5EOe+WuCfuQr49U unbqQxlHpmJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCaYHdPQAKCRAEWL7uWMY5 RmdyEAC8Gb4J7AKABnGI1ESN5mKs8wfRAVbeEptQbCRE5TG56C8/CwBu3BxngSIJaR/jsj4cg+v 7E0sKl/e97o+ZFQ1e9+FgXjFE7mVT12T4WUYBVqLBjTCamAok2TTrixp+zYddUq+ccwGolxU2MY wBGy5AIj3hlrdI1XTXv0NYJa4I4Glhfer2cK5LuX/6pkbjv7AQywMBrJtEapUtDvygG+cs0Uz4n zHRznz9Xh6RSnlT17RKc8EABqA3R7Z/Y1TfSPuiYNT+i1vFhkHNGluu0cxtkBL/PEZd15dIpN9B CXvT+lZRPcrEVVL2fuqYe+gQgP56/sgeJvjD7jcfyJI3zXMfQpMyFC8pB+rbFKJIVG7GNB7L0Rt 4WqgYZBp0kM9WfQ+cmN7CaAFLaSsq6PDDnQFm61oEIBCoup6Q5tW2ceX6RNTUETpSUxy80qbzu8 Rhm7tuiSWxxpbMg+aRVGw5nFrxnliAzu/PFfSdq7Ro9SOo045XuS6+UKMlWE9o9WbFN/DB0TD9L w8uffrD51wqql0njRGx4d6VDYm6dS3Zi9JOQVznM/Z2AIqWquCTTwpocObwK8qDoX6PwKhb9ix/ H5C5Hb9A4mUHv16LwURptoWOxXnWZP3I9pTQEulcrf6CnuKSZ9eQszFn5hXt60Q4x6yPGd9DXBd EsK/t8LslrkpMww== X-Mailer: b4 0.14.2 Message-ID: <20260203-inline-helpers-v2-3-beb8547a03c9@google.com> Subject: [PATCH v2 3/3] build: rust: provide an option to inline C helpers into Rust From: Alice Ryhl To: Miguel Ojeda Cc: Boqun Feng , Gary Guo , "=?utf-8?q?Bj=C3=B6rn_Roy_Baron?=" , Benno Lossin , Andreas Hindborg , Trevor Gross , Danilo Krummrich , Alexandre Courbot , Will Deacon , Peter Zijlstra , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Nicolas Schier , Andrew Morton , Uladzislau Rezki , rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-kbuild@vger.kernel.org, linux-mm@kvack.org, Alice Ryhl , Matthew Maurer Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Gary Guo A new experimental Kconfig option, `RUST_INLINE_HELPERS` is added to allow C helpers (which were created to allow Rust to call into inline/macro C functions without having to re-implement the logic in Rust) to be inlined into Rust crates without performing global LTO. If the option is enabled, the following is performed: * For helpers, instead of compiling them to an object file to be linked into vmlinux, they're compiled to LLVM IR bitcode. Two versions are generated: one for built-in code (`helpers.bc`) and one for modules (`helpers_module.bc`, with -DMODULE defined). This ensures that C macros/inlines that behave differently for modules (e.g. static calls) function correctly when inlined. * When a Rust crate or object is compiled, instead of generating an object file, LLVM bitcode is generated. * llvm-link is invoked with --internalize to combine the helper bitcode with the crate bitcode. This step is similar to LTO, but this is much faster since it only needs to inline the helpers. * clang is invoked to turn the combined bitcode into a final object file. * Since clang may produce LLVM bitcode when LTO is enabled, and objtool requires ELF input, $(cmd_ld_single) is invoked to ensure the object is converted to ELF before objtool runs. The --internalize flag tells llvm-link to treat all symbols in helpers.bc using `internal` linkage [1]. This matches the behavior of `clang` on `static inline` functions, and avoids exporting the symbol from the object file. To ensure that RUST_INLINE_HELPERS is not incompatible with BTF, we pass the -g0 flag when building helpers. See commit 5daa0c35a1f0 ("rust: Disallow BTF generation with Rust + LTO") for details. We have an intended triple mismatch of `aarch64-unknown-none` vs `aarch64-unknown-linux-gnu`, so we pass --suppress-warnings to llvm-link to suppress it. We add $(cmd_ld_single) before $(cmd_objtool). Otherwise objtool fails to parse the resulting files as Elf when CONFIG_LTO is enabled. I considered adding some sort of check that KBUILD_MODNAME is not present in helpers_module.bc, but this is actually not so easy to carry out because .bc files store strings in a weird binary format, so you cannot just grep it for a string to check whether it ended up using KBUILD_MODNAME anywhere. Link: https://github.com/llvm/llvm-project/pull/170397 [1] Co-developed-by: Boqun Feng Signed-off-by: Boqun Feng Co-developed-by: Matthew Maurer Signed-off-by: Matthew Maurer Signed-off-by: Gary Guo Co-developed-by: Alice Ryhl Signed-off-by: Alice Ryhl --- Makefile | 3 ++- lib/Kconfig.debug | 15 +++++++++++++++ rust/Makefile | 29 +++++++++++++++++++++++++---- rust/exports.c | 5 ++++- scripts/Makefile.build | 7 ++++++- 5 files changed, 52 insertions(+), 7 deletions(-) diff --git a/Makefile b/Makefile index c97f06ee0dda1c922aa23bd5249052591d528eb6..155db03cf489d931eefdb4bd7e3= b93e5aa3ea2d6 100644 --- a/Makefile +++ b/Makefile @@ -519,6 +519,7 @@ OBJCOPY =3D $(LLVM_PREFIX)llvm-objcopy$(LLVM_SUFFIX) OBJDUMP =3D $(LLVM_PREFIX)llvm-objdump$(LLVM_SUFFIX) READELF =3D $(LLVM_PREFIX)llvm-readelf$(LLVM_SUFFIX) STRIP =3D $(LLVM_PREFIX)llvm-strip$(LLVM_SUFFIX) +LLVM_LINK =3D $(LLVM_PREFIX)llvm-link$(LLVM_SUFFIX) else CC =3D $(CROSS_COMPILE)gcc LD =3D $(CROSS_COMPILE)ld @@ -627,7 +628,7 @@ export RUSTC_BOOTSTRAP :=3D 1 export CLIPPY_CONF_DIR :=3D $(srctree) =20 export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPI= LE LD CC HOSTPKG_CONFIG -export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN +export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN= LLVM_LINK export HOSTRUSTC KBUILD_HOSTRUSTFLAGS export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX Y= ACC AWK INSTALLKERNEL export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 7570a694d54e6170ae6738634218b37e38a4b76a..c44c08b49f3a3c2e06fca42adeb= 9e04304e7a869 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -3551,6 +3551,21 @@ config RUST_KERNEL_DOCTESTS =20 If unsure, say N. =20 +config RUST_INLINE_HELPERS + bool "Inline C helpers into Rust code" + depends on RUST && RUSTC_CLANG_LLVM_COMPATIBLE + depends on EXPERT + help + Inlines C helpers into Rust code using Link Time Optimization. + + If this option is enabled, C helper functions declared in + rust/helpers/ are inlined into Rust code, which is helpful for + performance of Rust code. This requires a matching LLVM version for + Clang and rustc. + + If you are sure that you're using Clang and rustc with matching LLVM + versions, say Y. Otherwise say N. + endmenu # "Rust" =20 endmenu # Kernel hacking diff --git a/rust/Makefile b/rust/Makefile index 63464bd2c1e9734cd6e659f7ee3db58bf995d6dd..a6839f2d39feefcfea497384ae2= 02a7c6b475942 100644 --- a/rust/Makefile +++ b/rust/Makefile @@ -6,15 +6,19 @@ rustdoc_output :=3D $(objtree)/Documentation/output/rust/= rustdoc obj-$(CONFIG_RUST) +=3D core.o compiler_builtins.o ffi.o always-$(CONFIG_RUST) +=3D exports_core_generated.h =20 +ifdef CONFIG_RUST_INLINE_HELPERS +always-$(CONFIG_RUST) +=3D helpers/helpers.bc helpers/helpers_module.bc +else +obj-$(CONFIG_RUST) +=3D helpers/helpers.o +always-$(CONFIG_RUST) +=3D exports_helpers_generated.h +endif # Missing prototypes are expected in the helpers since these are exported # for Rust only, thus there is no header nor prototypes. -obj-$(CONFIG_RUST) +=3D helpers/helpers.o CFLAGS_REMOVE_helpers/helpers.o =3D -Wmissing-prototypes -Wmissing-declara= tions =20 always-$(CONFIG_RUST) +=3D bindings/bindings_generated.rs bindings/binding= s_helpers_generated.rs obj-$(CONFIG_RUST) +=3D bindings.o pin_init.o kernel.o -always-$(CONFIG_RUST) +=3D exports_helpers_generated.h \ - exports_bindings_generated.h exports_kernel_generated.h +always-$(CONFIG_RUST) +=3D exports_bindings_generated.h exports_kernel_gen= erated.h =20 always-$(CONFIG_RUST) +=3D uapi/uapi_generated.rs obj-$(CONFIG_RUST) +=3D uapi.o @@ -480,6 +484,14 @@ $(obj)/bindings/bindings_helpers_generated.rs: private= bindgen_target_extra =3D ; $(obj)/bindings/bindings_helpers_generated.rs: $(src)/helpers/helpers.c FO= RCE $(call if_changed_dep,bindgen) =20 +quiet_cmd_rust_helper =3D HELPER $@ + cmd_rust_helper =3D \ + $(CC) $(filter-out $(CFLAGS_REMOVE_helpers/helpers.o), $(c_flags)) \ + -c -g0 $< $(if $(filter %_module.bc,$@),-DMODULE) -emit-llvm -o $@ + +$(obj)/helpers/helpers.bc $(obj)/helpers/helpers_module.bc: $(src)/helpers= /helpers.c FORCE + +$(call if_changed_dep,rust_helper) + rust_exports =3D $(NM) -p --defined-only $(1) | awk '$$2~/(T|R|D|B)/ && $$= 3!~/__(pfx|cfi|odr_asan)/ { printf $(2),$$3 }' =20 quiet_cmd_exports =3D EXPORTS $@ @@ -561,12 +573,16 @@ quiet_cmd_rustc_library =3D $(if $(skip_clippy),RUSTC= ,$(RUSTC_OR_CLIPPY_QUIET)) L OBJTREE=3D$(abspath $(objtree)) \ $(if $(skip_clippy),$(RUSTC),$(RUSTC_OR_CLIPPY)) \ $(filter-out $(skip_flags),$(rust_flags)) $(rustc_target_flags) \ - --emit=3Ddep-info=3D$(depfile) --emit=3Dobj=3D$@ \ + --emit=3Ddep-info=3D$(depfile) --emit=3D$(if $(link_helper),llvm-bc=3D$(= patsubst %.o,%.bc,$@),obj=3D$@) \ --emit=3Dmetadata=3D$(dir $@)$(patsubst %.o,lib%.rmeta,$(notdir $@)) \ --crate-type rlib -L$(objtree)/$(obj) \ --crate-name $(patsubst %.o,%,$(notdir $@)) $< \ --sysroot=3D/dev/null \ + $(if $(link_helper),;$(LLVM_LINK) --internalize --suppress-warnings $(pat= subst %.o,%.bc,$@) \ + $(obj)/helpers/helpers$(if $(part-of-module),_module).bc -o $(patsubst %= .o,%.m.bc,$@); \ + $(CC) $(CLANG_FLAGS) $(KBUILD_CFLAGS) -Wno-override-module -c $(patsubst= %.o,%.m.bc,$@) -o $@) \ $(if $(rustc_objcopy),;$(OBJCOPY) $(rustc_objcopy) $@) \ + $(cmd_ld_single) \ $(cmd_objtool) =20 rust-analyzer: @@ -696,4 +712,9 @@ $(obj)/kernel.o: $(obj)/kernel/generated_arch_warn_asm.= rs $(obj)/kernel/generate endif endif =20 +ifdef CONFIG_RUST_INLINE_HELPERS +$(obj)/kernel.o: private link_helper =3D 1 +$(obj)/kernel.o: $(obj)/helpers/helpers.bc +endif + endif # CONFIG_RUST diff --git a/rust/exports.c b/rust/exports.c index 587f0e776aba52854080f15aa91094b55996c072..1b52460b0f4eeef6df9081abb9b= 7e054a28c3c21 100644 --- a/rust/exports.c +++ b/rust/exports.c @@ -16,10 +16,13 @@ #define EXPORT_SYMBOL_RUST_GPL(sym) extern int sym; EXPORT_SYMBOL_GPL(sym) =20 #include "exports_core_generated.h" -#include "exports_helpers_generated.h" #include "exports_bindings_generated.h" #include "exports_kernel_generated.h" =20 +#ifndef CONFIG_RUST_INLINE_HELPERS +#include "exports_helpers_generated.h" +#endif + // For modules using `rust/build_error.rs`. #ifdef CONFIG_RUST_BUILD_ASSERT_ALLOW EXPORT_SYMBOL_RUST_GPL(rust_build_error); diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 0c838c467c764e14a51ad132444544373e90a84c..7176d997ecab7c3ffd1e00400a8= a79a8309485e1 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -343,7 +343,12 @@ rust_common_cmd =3D \ # would not match each other. =20 quiet_cmd_rustc_o_rs =3D $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@ - cmd_rustc_o_rs =3D $(rust_common_cmd) --emit=3Dobj=3D$@ $< $(cmd_obj= tool) + cmd_rustc_o_rs =3D $(rust_common_cmd) --emit=3D$(if $(CONFIG_RUST_IN= LINE_HELPERS),llvm-bc=3D$(patsubst %.o,%.bc,$@),obj=3D$@) $< \ + $(if $(CONFIG_RUST_INLINE_HELPERS),;$(LLVM_LINK) --internalize --suppress= -warnings $(patsubst %.o,%.bc,$@) \ + $(objtree)/rust/helpers/helpers$(if $(part-of-module),_module).bc -o $(p= atsubst %.o,%.m.bc,$@); \ + $(CC) $(CLANG_FLAGS) $(KBUILD_CFLAGS) -Wno-override-module -c $(patsubst= %.o,%.m.bc,$@) -o $@) \ + $(cmd_ld_single) \ + $(cmd_objtool) =20 define rule_rustc_o_rs $(call cmd_and_fixdep,rustc_o_rs) --=20 2.53.0.rc1.225.gd81095ad13-goog