From nobody Sun Feb 8 00:38:34 2026 Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 56CFB2EA493 for ; Tue, 2 Dec 2025 20:28:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764707294; cv=none; b=ZFRT6TKYtpfQtIBdgV2nXkfjgqex8HwSjzOwdfr8DenZaRrtxjMlBSXPxR4mPgdhlgFFmC5m+HvUn3iukMqNXDwLOKPNh7wVmqW4NBYreQF3k+QUTNFkVI1nGzSIleMgAZtAawNGBCpymfKXkFxkzN7hXhS4FkbKevZNb0fyz3o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764707294; c=relaxed/simple; bh=o237WR5t4MMOHA/6xwLqKKnBZN3/rDT9uVvqOcTiQhs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=JngMLvZQReT6jeWobE9wteyDf0LdUh8/X0KgBXOuhURoJsry8t4ZaxFqOkLI/UQ07DrzeGerh9PZKJggEQK0PkE3aqJQn3r0FwhUJBddnHktAxtHnhtC31OmV9Vwk+ClIFsR17znFRD38RcPfNl/+DDFKTkHbUqJxel7tJmavYM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=RvOx4AtQ; arc=none smtp.client-ip=209.85.218.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="RvOx4AtQ" Received: by mail-ej1-f73.google.com with SMTP id a640c23a62f3a-b73533b84eeso516139766b.3 for ; Tue, 02 Dec 2025 12:28:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764707291; x=1765312091; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=0yaLbdbXJAqPOG8MVf6C4XR3DKZYvDAAnU5HF8zvPbo=; b=RvOx4AtQHBpN6vRQmlqdo3GHoSePk/yMwKE1iFz79u/gbST6sQqKbv7j4u13qxNFN1 oUTo9wxfPfx+HxrWAcCbLvxi9mbqbFLKFMCzhE6U4rXPz6xb3R8TVcc7Z50n26C0VVFy tRKYfebbyaDmFK36TTuFADVaqDZZ7fidQdxNtiwQ5//tnnXReEr5uN3Iti9Zgfpw9m70 oxCi9Acp8GlkbhAFRo8rcOopUh0FtCpMD6G7NNVRTOb6OzqLEkgTgEYPBdMwb3hO/oNQ lz+ZN17j7SX/c0zr3ckfWAz08NqYwfUiAfbyiMj6L7R+KABatidh/KvkQeVWFMPOkbLG 2HKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764707291; x=1765312091; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=0yaLbdbXJAqPOG8MVf6C4XR3DKZYvDAAnU5HF8zvPbo=; b=CvzNB7jvTSMP7rGq4Cq7JT7gn1LlaJj3TmRSBRdUOBnGgLiWK6dximZ65e52WEBV7r GpXPSwuXSXXH8oBP/eCAhgqejzq9IvBmx5PnZMyMgyTyMkLyAyvMmgMybjX+5fs65ewr oVeZmuGwOQ8219rDauuWV+KgCDDNSEy1MA5eYJ+M6nY5CECgDkVkyfhqDs+7X+NjWKfT Hl4nbvNFRFC+TyVhhyX+/wdPrrQeU/L52tukZdpx5LSRySIapHwfWrmOmzcbZ7AUNec9 +EFRLJmhZwVlRAXJBKa6aS8C2zMgXTunP5MuLq3p9ddbOJJUWTotUgQ0PSUcrARDArTP Qndw== X-Forwarded-Encrypted: i=1; AJvYcCVYsdamWES7khp9in62ecisK+bzx50nqzkM688qIYNwETFVKEgGUcwnY9VnhRRNs+5zCfsakL6vpXt1GUM=@vger.kernel.org X-Gm-Message-State: AOJu0YzO2e5QPsS00le5cn7X4j+RmdD1t0ongjQ92DOupF8TmDxhKR6I KKtpa5x1YPxPE68Yp4S1py8ETUEE8Nw3dP1MCpPdD6XVho1oUlNqZWnncPUCrw7cqr4YJHp34GA iT6bjsK9zQTVxOq0Z1A== X-Google-Smtp-Source: AGHT+IFxbJTUi3LG/UxAzVg0DHJfs/QK8ZkbgJFUFzeew/1o2JPNxmwoD7ry+SYsd199ZxrJFbneEJ7rLxcSHeI= X-Received: from ejckp26.prod.google.com ([2002:a17:907:999a:b0:b72:6a6d:5018]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a17:907:9289:b0:b73:76c5:8f7c with SMTP id a640c23a62f3a-b767184bc4cmr5031432266b.43.1764707290453; Tue, 02 Dec 2025 12:28:10 -0800 (PST) Date: Tue, 02 Dec 2025 20:27:56 +0000 In-Reply-To: <20251202-inline-helpers-v1-0-879dae33a66a@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251202-inline-helpers-v1-0-879dae33a66a@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=993; i=aliceryhl@google.com; h=from:subject:message-id; bh=o237WR5t4MMOHA/6xwLqKKnBZN3/rDT9uVvqOcTiQhs=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBpL0vXS2KTuDwIHjFPV9SPMyVgAzUi/MU12KYc+ WMaROi5+LCJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCaS9L1wAKCRAEWL7uWMY5 RjdSD/41DCALGMN7nyQxyeW2hZ9gETkyG8/iqH2wlmXrShi1r1+WB5woa9Xyur1kDN4fny2B3pO Y/5ZTPeLOcZ/aZiVYBNJA/D9HwDtronU7j5mUuxB0yWDU86evl6AxsKiEvJuzy96b8AtZ0ARx94 IgGN41NqoWqbtVWQg6QlCkP56YIrgkpzGAS8atcgpDUrXNb+PE5wxdosva9Kh9do0WUVeLVeSKd BdEA9iG36VbrEo4YdJv4GRojxiLeD9CDzvYaGx6ijS8AsM1Osv5V66FEhywlu5eg8vNaBbPlcj2 xI8IBYCVYdeDDvb1eKyZRLyLRwSvHfbhkMIgkLYycP51rfgkb9kJ8xzMWQGx4zynmLXzAXH0J0m 5EB4i8kqfcM55bSpXOcCJzmldnW5zNj6/a6PmCC3hINHPjiHjIKkZmyQFZ+xyfRWH5U5ZZnvqIq RTDSAE/Vjex1zfELbKMrNyp4+efGryIeLo2hKskD8xXiqzcEDhKiNZY7C3d1jHSgp0uyZLRoe7e N/0L4pucmpbvlsisOuuXdvhokGnGK0s/K4V9WjZ94AoP9QZDTbOpFHNW6iUVflzEdZFfDfqHEs/ ToADXA5Wr2TwTuT9flTY9YEdDg46zEMD7PVRz2llCMVxG6AYZdE+1gKeJLV9PHtLVfRe6h8qxrt EQLssKb+G2MtrPw== X-Mailer: b4 0.14.2 Message-ID: <20251202-inline-helpers-v1-1-879dae33a66a@google.com> Subject: [PATCH 1/4] vmalloc: export vrealloc_node_align_noprof From: Alice Ryhl To: Miguel Ojeda Cc: Boqun Feng , Gary Guo , "=?utf-8?q?Bj=C3=B6rn_Roy_Baron?=" , Benno Lossin , Andreas Hindborg , Trevor Gross , Danilo Krummrich , Alexandre Courbot , Will Deacon , Peter Zijlstra , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Nicolas Schier , Andrew Morton , Uladzislau Rezki , rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-kbuild@vger.kernel.org, linux-mm@kvack.org, nouveau@lists.freedesktop.org, Alice Ryhl Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable This symbol is used from the Nova driver, so it needs to be exported to avoid a build failure when building Nova as a module. ERROR: modpost: "vrealloc_node_align_noprof" [drivers/gpu/nova-core/nova_co= re.ko] undefined! ERROR: modpost: "vrealloc_node_align_noprof" [samples/rust/rust_dma.ko] und= efined! This error is only triggered if inlining of helpers into Rust is enabled. Signed-off-by: Alice Ryhl --- mm/vmalloc.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 798b2ed21e46059f341ed0d46c7fe56bbe357b22..e086d00f04d61ebd481cb84e3df= ea51d8a2ffc57 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -4200,6 +4200,7 @@ void *vrealloc_node_align_noprof(const void *p, size_= t size, unsigned long align =20 return n; } +EXPORT_SYMBOL(vrealloc_node_align_noprof); =20 #if defined(CONFIG_64BIT) && defined(CONFIG_ZONE_DMA32) #define GFP_VMALLOC32 (GFP_DMA32 | GFP_KERNEL) --=20 2.52.0.158.g65b55ccf14-goog From nobody Sun Feb 8 00:38:34 2026 Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 022992F260C for ; Tue, 2 Dec 2025 20:28:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764707296; cv=none; b=pGswvzopWBwyml1KieL6Rtk4zg6pe4e6dY/GXLSIXUqZO6zniTgQ9Hxf350gzHQyAGNdiNOVituLppkbMDoad0nP01ml6VhGfmi7IqUSG4SyXshOulSd3h5Q4zH0qJYhjwZD56cB1Uf5q5R+0/mydNEPbGKjjRdi9yfWEGW6VQ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764707296; c=relaxed/simple; bh=yeBX0eL+YEEo9zWpSpzNVC/hIVvpLSYTUpSpVAG9gsI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=b6w1JaeNxnDdMPjA7jhSD7kSf6fP7ZfkAMYlcg2ctn1y/yy8Rs4YBuIOZEmgIaIhCi6owvOk0wwH9wWuHVAjLXZdLtSmQGoNYem82sFjjNxJ0UWsAiGHBL+jz2qfxTysm0zLs65/GzC65Ay0xv8P+0Up8hIHXsI53IMHkWY3duU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=kHaC/jli; arc=none smtp.client-ip=209.85.208.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="kHaC/jli" Received: by mail-ed1-f74.google.com with SMTP id 4fb4d7f45d1cf-64097bef0e2so8012418a12.3 for ; Tue, 02 Dec 2025 12:28:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764707292; x=1765312092; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=6p+COANrRJg4iyKTRaWVzi/wMI4z3c+bFvZRqbOPrm4=; b=kHaC/jliomvb9Ib1C68yoUQ+HOhllmssGoXgSbmm+P+h/FUkApnWoB7J/Q8eAvnYps l++0yClw9aw+80DGeEiDeNAMHR/rhPdVlojmEayNEOsiV7r/gfzxThVKuDld5eCX3aoT J9oA+dJGN7cGRIRLj05FO6SzfjvUZ7E6rUVVDjHQ1ie24NoTEct2iWQ4PmaBkcIFPIh5 rZJ9N/b++4KMgPxP9y9Hrwf6V44dtKE0qBADv9A0hS9CtOJJnC36qHSyPn25BL9MwUDN 2//wqblOmNilS5XFyF7v7Xr9EEfwD4q8Aw2skCRb1XvprtYKkSHu0h7658nE8fMj4gmg WlwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764707292; x=1765312092; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6p+COANrRJg4iyKTRaWVzi/wMI4z3c+bFvZRqbOPrm4=; b=fokWidjSzQrfTQuzrKaT6NdiAwD8lftnlw1HRMgl734EaCMTxC/xuYJ2j0lgFFSpH5 747FDpAitKI/O/bZhMlB6Bw4Fl3ehBjDqeD1ejgU5Oy1idadMAryNw1D7RCcAfsk5v3/ +VD9bheFWLM9nI+WFNeB2HZtPbjzDDQyKyRwdTNQUhmqWdGNiutGEGvk0x7T+O+JFuB2 h90xbFfGnyahUHHsPN1nF8Ys9mGIoxg2vknFXe8kx/5O3rnhUNQQ4V5ZPTK9IP+XZzuN erjJ+ZB+nyjpzoCFKg3gUqV7FsVoJnlz9vSoNIh8bq1QJs7+c8U6voUldyg0We7buHRF zSCA== X-Forwarded-Encrypted: i=1; AJvYcCXuY/MxawLYVIQHIICd7z5Kdvw+qDRTjjajEUV4z0TGXqX4UlfwGCeOmak8gCiQvtNxJPKvWthsLvR8Tw0=@vger.kernel.org X-Gm-Message-State: AOJu0YyNYI9dNyVicGbALbm005ciB7ZqBq+81w1ffuzAv+6/fJzH8SvI ogJJLWm6pKxwcPubClzbn1lMAUyGnVjHn6pdGGlRkJBrHzEif5sRTkDNWH+6TbLYCAuXLzbRHgz 4NX+3EHwnIAlaczPbSw== X-Google-Smtp-Source: AGHT+IHlno9/sG7/bayEdrUxcRJ+XAK+NEe9iVymOZCA9KTFp8CJ1IAcUSelacyyX3nFi5Yad+XDFFP4AwI+IlQ= X-Received: from edaa21.prod.google.com ([2002:a05:6402:24d5:b0:640:92f5:e6fc]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:3584:b0:640:74f5:d9f6 with SMTP id 4fb4d7f45d1cf-64794f74e19mr816955a12.25.1764707292013; Tue, 02 Dec 2025 12:28:12 -0800 (PST) Date: Tue, 02 Dec 2025 20:27:57 +0000 In-Reply-To: <20251202-inline-helpers-v1-0-879dae33a66a@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251202-inline-helpers-v1-0-879dae33a66a@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=4811; i=aliceryhl@google.com; h=from:subject:message-id; bh=ILovg7w2jH5gFOEQybF86Y3dXSsF4YeG9sZZSLyxF8g=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBpL0vXWcW4aVlMjlcTWMGU9dXcoeSC3ebU+h+yW gJ6gVmPh1OJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCaS9L1wAKCRAEWL7uWMY5 RngFEACMVsbQT0Zwv8K0Ot/Yv1D0f0w7dRnqAHbcNhDHMtJMF0eKhiccoIyiC0h/BYKvRp8B5Pu KSeabwFIVcOwoJsWy+5R1rqIx2eJqi4leqQWKrIzXrso29P9gXpzBWdBOHwwr0PRYGZZs9eqSEp Ni8/A1g20ZtVifJDwYSoZtF/5PminXSIGBdm5krya3CD3v+xoirkqH8c7cd2uifxi5KrJD6qek3 5UiQj/e4qgQTTynWcQS9YiJt+sMktrku/XQEmJXyVaDIHrxSMZoWHIYLpdeIwuiWJpPkFPmM8Ni UzcSlSLe2Bxh/onkCZs/OKqLm0ynAVl4fGqZIA6f3VhqaSBHVnokKZ+zij8Jdz14c35nUQN/dlr fvOd1oiCesg5Ish6+R3Bi9z1VNW+dWygGww6u5+rCxA0+GEIXwc1qoQir54sFgOIf+hunXFo4Ps uLzgkzsB6nmOXPv2xxyib9QijAdpCBZ86gSnclbCWDhA/RolFCvtuAfNdVfh8WJ1Iy0/4U/1DaE rITItP7ShM8YN+VXCGtruti4f6K9bW3e4z5o9gr5h2C2ODxhxNafTH018EFzRJ9GMSoZZiqSNG+ wV0PayaQaYSr4TIzmDNriukMQuoKo6fI+++0wRf8nzwUNjnaAZu/qQppZTmL/cbdyjVfrZ8J0ZC bpyEsLN+f3zJ2gA== X-Mailer: b4 0.14.2 Message-ID: <20251202-inline-helpers-v1-2-879dae33a66a@google.com> Subject: [PATCH 2/4] rust: helpers: #define __rust_helper From: Alice Ryhl To: Miguel Ojeda Cc: Boqun Feng , Gary Guo , "=?utf-8?q?Bj=C3=B6rn_Roy_Baron?=" , Benno Lossin , Andreas Hindborg , Trevor Gross , Danilo Krummrich , Alexandre Courbot , Will Deacon , Peter Zijlstra , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Nicolas Schier , Andrew Morton , Uladzislau Rezki , rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-kbuild@vger.kernel.org, linux-mm@kvack.org, nouveau@lists.freedesktop.org, Alice Ryhl Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Gary Guo Because of LLVM inling checks, it's generally not possible to inline a C helper into Rust code, even with LTO: * LLVM doesn't want to inline functions compiled with `-fno-delete-null-pointer-checks` with code compiled without. The C CGUs all have this enabled and Rust CGUs don't. Inlining is okay since this is one of the hardening features that does not change the ABI, and we shouldn't have null pointer dereferences in these helpers. * LLVM doesn't want to inline functions with different list of builtins. C side has `-fno-builtin-wcslen`; `wcslen` is not a Rust builtin, so they should be compatible, but LLVM does not perform inlining due to attributes mismatch. * clang and Rust doesn't have the exact target string. Clang generates `+cmov,+cx8,+fxsr` but Rust doesn't enable them (in fact, Rust will complain if `-Ctarget-feature=3D+cmov,+cx8,+fxsr` is used). x86-64 always enable these features, so they are in fact the same target string, but LLVM doesn't understand this and so inlining is inhibited. This can be bypassed with `--ignore-tti-inline-compatible`, but this is a hidden option. To fix this, we can add __always_inline on every helper, which skips these LLVM inlining checks. For this purpose, introduce a new __rust_helper macro that needs to be added to every helper. The actual additions of __rust_helper can happen in separate patches. A "flag day" change is not required since missing annotations do not lead to anything worse than missing inlining. Signed-off-by: Gary Guo Signed-off-by: Alice Ryhl Reviewed-by: Andreas Hindborg --- rust/helpers/atomic.c | 5 ----- rust/helpers/helpers.c | 31 +++++++++++++++++++++++++++= ++++ scripts/atomic/gen-rust-atomic-helpers.sh | 5 ----- 3 files changed, 31 insertions(+), 10 deletions(-) diff --git a/rust/helpers/atomic.c b/rust/helpers/atomic.c index cf06b7ef9a1c559e8d7bdfc2bcd2aeb8951c29d1..a48605628ed73ac32aae2e62804= 81407a670e88f 100644 --- a/rust/helpers/atomic.c +++ b/rust/helpers/atomic.c @@ -11,11 +11,6 @@ =20 #include =20 -// TODO: Remove this after INLINE_HELPERS support is added. -#ifndef __rust_helper -#define __rust_helper -#endif - __rust_helper int rust_helper_atomic_read(const atomic_t *v) { diff --git a/rust/helpers/helpers.c b/rust/helpers/helpers.c index 551da6c9b5064c324d6f62bafcec672c6c6f5bee..d0e2f1f9b449b2248cfbddcee1e= 8bf9becc8a2f9 100644 --- a/rust/helpers/helpers.c +++ b/rust/helpers/helpers.c @@ -7,6 +7,37 @@ * Sorted alphabetically. */ =20 +#include + +#ifdef __BINDGEN__ +// Omit `inline` for bindgen as it ignores inline functions. +#define __rust_helper +#else +// The helper functions are all inline functions. +// +// We use `__always_inline` here to bypass LLVM inlining checks, in case t= he +// helpers are inlined directly into Rust CGUs. +// +// The LLVM inlining checks are false positives: +// * LLVM doesn't want to inline functions compiled with +// `-fno-delete-null-pointer-checks` with code compiled without. +// The C CGUs all have this enabled and Rust CGUs don't. Inlining is okay +// since this is one of the hardening features that does not change the = ABI, +// and we shouldn't have null pointer dereferences in these helpers. +// * LLVM doesn't want to inline functions with different list of builtins= . C +// side has `-fno-builtin-wcslen`; `wcslen` is not a Rust builtin, so th= ey +// should be compatible, but LLVM does not perform inlining due to attri= butes +// mismatch. +// * clang and Rust doesn't have the exact target string. Clang generates +// `+cmov,+cx8,+fxsr` but Rust doesn't enable them (in fact, Rust will +// complain if `-Ctarget-feature=3D+cmov,+cx8,+fxsr` is used). x86-64 al= ways +// enable these features, so they are in fact the same target string, but +// LLVM doesn't understand this and so inlining is inhibited. This can be +// bypassed with `--ignore-tti-inline-compatible`, but this is a hidden +// option. +#define __rust_helper __always_inline +#endif + #include "atomic.c" #include "auxiliary.c" #include "barrier.c" diff --git a/scripts/atomic/gen-rust-atomic-helpers.sh b/scripts/atomic/gen= -rust-atomic-helpers.sh index 45b1e100ed7c63108ee6cb07e48a17668f860d47..a3732153af29f415e397e17cab6= e75cb5d7efafc 100755 --- a/scripts/atomic/gen-rust-atomic-helpers.sh +++ b/scripts/atomic/gen-rust-atomic-helpers.sh @@ -47,11 +47,6 @@ cat << EOF =20 #include =20 -// TODO: Remove this after INLINE_HELPERS support is added. -#ifndef __rust_helper -#define __rust_helper -#endif - EOF =20 grep '^[a-z]' "$1" | while read name meta args; do --=20 2.52.0.158.g65b55ccf14-goog From nobody Sun Feb 8 00:38:34 2026 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C78E02F291B for ; Tue, 2 Dec 2025 20:28:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764707298; cv=none; b=CwGjXOPPXCcd5Ifta3hb0F4JG0cXM4qMJtcNbOhyv/VXgRLauZRaZ3Z+7qYAa6TXVXiHbCHg2LmnzHepej8PSrzPHrXUEVpea1DCYMOtDO5HyX+ay8cAAZ0bs4cjGiYvf802MpdN8y5tvLg/Wido+zpil+1Arc4SKDp13/YihfQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764707298; c=relaxed/simple; bh=lnkOuIsj111D22tJJih/W9w2NYP5jJU1fAP23hUjMUo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GKNFBNnUevlRD8PbbnAhGon/+bQA12lmgzP6HENx/4ZqAYIWCzx381ZLgYWnwaUTsO1irRfgpU644dP+vXCfKMBkllrbt51zU0e9RLNMxK09R6CnAx2QGcWC8r+2C/wdxj3Exzxf2bqstuLrsDL85u8lfyeP20J6zkp1VL2enP4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Qx3MkD1P; arc=none smtp.client-ip=209.85.221.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Qx3MkD1P" Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-42e2e3c3e1aso1830707f8f.1 for ; Tue, 02 Dec 2025 12:28:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764707294; x=1765312094; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=gzxkQHE3Gn13GhwfvC/ki50S4oawaHmvikFNCkwElMo=; b=Qx3MkD1PM2sgoei91jvo+u+nbrh1Qmd0Ob4l9erI4WFACOuqjSBFwAKDUnm/t7ersE s1XA/7WIG7kYkAcOlIuoA/cQXD7DS/zS4JRBod3jUgMlztaXTCRfRnKYFoCv9i9BLHxR x+dl1J3OQjS9yTIzQlfQsR388Dry/B0K+FXMeRr6qCRHGPfRdbaTDVHJjINl00Rfzahy QVB+WkopfNVcQSBeK7ic4ll62k+yM8Bbe5jr2cfsdIDxm7aXt22Ng4ryHf5yd8fbKyhf 2yJKSmsBqpJypsJyNv9PYv18v+SBbBdhxsTZNxgJDKHZj7nYhB9Uci7atP8PtfmM26u/ N/HA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764707294; x=1765312094; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=gzxkQHE3Gn13GhwfvC/ki50S4oawaHmvikFNCkwElMo=; b=tkPdbdGDV71E+bt2r3/PTYqdTJqqnpJOwX+o5EvwUSYRXxrBXdrb/GdYLkP5Yl+q9Z IZ96dbo6i07rEApQBnYIuSW7lbxaZPq/n5AoyFbM9cvsp8N26EExcupTbMTrM9GHgNvf ub6shrWekzyIMYmR6FhgWcCKBMC+lDdLnz+SAQX6bVUaun+ScFd68Tp1Udd700SlUnTf cz+wsPdScOx7jku/KRoIomyOpKP473K2ksKz/W9Nx6mZzUVEFb2OSNrPg0frttlQ46qR mlJ4BYhjVJ8udY1dBlRu/70s2vr/d03fWsFpYuqpjXHnuuw+2k0JF/vGlLwLFr1Bk+kQ 0Zwg== X-Forwarded-Encrypted: i=1; AJvYcCVk0mMX5pjPsZ0CbOzEC4Eh7PwhHuNjgKjafziHok5rhriaGO2U8b/oyYslla3WX7B0yVpraRcRRdgHKB0=@vger.kernel.org X-Gm-Message-State: AOJu0Yz1TLYQ0v3IN3/32Sg3E6oPExymEOx1JT2bHGEG63kroLkvySew inU0Bli9dHs+sPkZxj0fJVDEspv19qzFIdi/Gk5jWtvJG+8/BxkjcFUuM1sKuhxcefxI9nwkqma VPmW6QkJC47WfyVp0xA== X-Google-Smtp-Source: AGHT+IGSHYTHK44+Q/MF6SQhcldsTZ4F1l/P7eqEkOlosij1EkIh4u00KVkeS7lVD1xE0j+H0LySU44dB3+wV1E= X-Received: from wrbl4.prod.google.com ([2002:adf:a384:0:b0:42c:c2d6:2ac]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:2502:b0:429:d3c9:b889 with SMTP id ffacd0b85a97d-42f721fc486mr463737f8f.1.1764707293233; Tue, 02 Dec 2025 12:28:13 -0800 (PST) Date: Tue, 02 Dec 2025 20:27:58 +0000 In-Reply-To: <20251202-inline-helpers-v1-0-879dae33a66a@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251202-inline-helpers-v1-0-879dae33a66a@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=1823; i=aliceryhl@google.com; h=from:subject:message-id; bh=XPDdA9qRPdYsgm5Nsj04MFhk3BGo//T0H/GXZaSfcnk=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBpL0vXEU3vfH0ez+fWa3XM7NL47DJ51BTPUHznb cVT8tuwz0eJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCaS9L1wAKCRAEWL7uWMY5 Rr5AEACDSrcLuNKFX5r42YLqnad/vd877J0UJtGVWFmfriVe5P6Y6yqMkDEJ4ZMuWqwR0CnkulQ iiVfYamb7odLVlOubnoHlMlj7AWP0dyPUvMSZSqNcAWaYbGW483ZRaOxEZ4MVgvv0mP7aVNfzTR AXKUkJtJQsTI3jC+m/+ONvFAwhiG/+ExbJiOQBnFIeA/dL5OLSlAzjBaKqkw0Hsjntsfgglh0AU gB0JD/vBPuKmQIVm80pytBPt6BxtuYQkBmnCsIF5IvJhOPmqYIUPGzEKdP/jNzFe+mQ1tYdDU9M 7OjSUVfVipG9L/SkRnOOZrtyBAw3ajN898MH9PL9BL76iul3Xq5SYqIYisYp1EkVSeAV1nI1nQ0 PVAfjmGqhZJAhvdAiFK4JJe89ZSYbg8lS2SM2eCdcS/YvCndmO7zlcsm4tN/dEbSsIrdJe9t/Jq 3T2AaP31hxuZkqIDVjDWYWot0m/f4ZO1DDvtY2NCpcCuooX/eqS89rOiC+QPd3ohGU3j6DpUhyb mfcdw6Kb2qywcFJVSyoT3wGIFQxVosTnahQ4kEHggZST0Fg4rfiMos2UpeMS03pTZ2tJtDFKVcy ZpVWYPeUkijjVyO5ttcLvgrMtnNiUzhdEYUK8ju5sJYNmGueW9AmruIzbdTRORaqrYkAvlfEdrO 8DhJzFKcq8IlMyA== X-Mailer: b4 0.14.2 Message-ID: <20251202-inline-helpers-v1-3-879dae33a66a@google.com> Subject: [PATCH 3/4] kbuild: rust: add `CONFIG_RUSTC_CLANG_LLVM_COMPATIBLE` From: Alice Ryhl To: Miguel Ojeda Cc: Boqun Feng , Gary Guo , "=?utf-8?q?Bj=C3=B6rn_Roy_Baron?=" , Benno Lossin , Andreas Hindborg , Trevor Gross , Danilo Krummrich , Alexandre Courbot , Will Deacon , Peter Zijlstra , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Nicolas Schier , Andrew Morton , Uladzislau Rezki , rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-kbuild@vger.kernel.org, linux-mm@kvack.org, nouveau@lists.freedesktop.org, Alice Ryhl , Matthew Maurer Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Gary Guo This config detects if Rust and Clang have matching LLVM major version. All IR or bitcode operations (e.g. LTO) rely on LLVM major version to be matching, otherwise it may generate errors, or worse, miscompile silently due to change of IR semantics. It's usually suggested to use the exact same LLVM version, but this can be difficult to guarantee. Rust's suggestion [1] is also major-version only, so I think this check is sufficient for the kernel. Link: https://doc.rust-lang.org/rustc/linker-plugin-lto.html [1] Reviewed-by: Andreas Hindborg Signed-off-by: Gary Guo Signed-off-by: Matthew Maurer Signed-off-by: Alice Ryhl --- init/Kconfig | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/init/Kconfig b/init/Kconfig index cab3ad28ca49e7ac930207c9cde8d431d55dc7af..d35d0ddee573d09997087a0f99d= ee7c7c9a66ae3 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -82,6 +82,21 @@ config RUSTC_LLVM_VERSION int default $(rustc-llvm-version) =20 +config RUSTC_LLVM_MAJOR_VERSION + int + default $(shell,expr $(rustc-llvm-version) / 10000) + +config RUSTC_CLANG_LLVM_COMPATIBLE + bool + default y if CC_IS_CLANG && RUSTC_LLVM_MAJOR_VERSION =3D $(shell,expr $(c= c-version) / 10000) + help + This indicates whether Rust and Clang use LLVM of the same major + version. + + Operations involving handling LLVM IR or bitcode (e.g. cross-language + LTO) requires the same LLVM major version to work properly. For best + compatibility it is recommended that the exact same LLVM is used. + config CC_CAN_LINK bool default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) = $(USERCFLAGS) $(USERLDFLAGS) $(m64-flag)) if 64BIT --=20 2.52.0.158.g65b55ccf14-goog From nobody Sun Feb 8 00:38:34 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4A092F360C for ; Tue, 2 Dec 2025 20:28:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764707299; cv=none; b=htq6hmsUjk7a4iXXVQVRwZyMy11VXDKPvs7UTNDlMf4yRdehRlXhZQ0UOXLCw6yKMc9Pzzq5k/68GMjQN6D3ixbvBbjkqtm0OrXeOhSd21CPPUuDZM1cuSNB86PZGz4RDrNlU0BADtYwf3fFtDzqI0smzWIxrRXDQn1/Q0Aw1MA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764707299; c=relaxed/simple; bh=j2kuLAZlOu1qDYVFIam1mftfSDlTjSfi8WMZfY1GIbw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Ih+HWHai8w1ec19j9r6b3BYZqB2DBKSw8oWpf/DyzU+p0wEJxYKJ+3+65CNipmNCZNB0AVWqZoik/UXPxwwNkwMTTC5jEJeBYlF1yCbsbBxjMqSHjYZtjxLm3eAUvl940ZnRof+3kEfhZeCTZxk3jSPHNYfczGt3G8x5bHttg00= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=l1p1tkbr; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--aliceryhl.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="l1p1tkbr" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-477a11d9e67so34845625e9.2 for ; Tue, 02 Dec 2025 12:28:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1764707295; x=1765312095; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Id46d5Vmg/QweH1w+t+fx2ST+00n7i1ciKT4MxLQ6Gk=; b=l1p1tkbrtwaNVarZP97p9WynmYMXEtrFL6jKjGKN40AMTLHPXx2IUd0DOZ+nv0P0Bl CN4bBtUAyPA4jvL6LSGrLzdMSjoFgUmJ5xLpu2Y2ddo8533hc9upa/a15xW4IFQYiWI4 HVfjq6wat6+V9EFnZQOOkOmotgNfAJal7ydzUF7WCX5ERK1i2lYt0+O+Uemn3n/Z08Vh 7CrkEgk3wGOy6ZCG90Xb55la1jVJJi+b6D3X3HL3v5n0s+cwsSmrK8JQ7uGle3tduXGM Gs7ePxsJSOsJ802vzHjIDvRkDmIDtBn2ZnNF61qoMt+cUeOCOCrjhT3SttcxB/UrzL2e j3Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764707295; x=1765312095; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Id46d5Vmg/QweH1w+t+fx2ST+00n7i1ciKT4MxLQ6Gk=; b=d6REOMdrSDFpuIXXM2vFZ1x7KlwP7VLpYOYmL3/vmtvQJS5WFdw8Va8RUaBMtQYR5N kCu39U430EjRLVkTMQ7W9inOBdEG6OVZaOcXLZY/vsZwkIvfkWYajpzNq45TOmnOvNwk Y8iAsjy/Rue3pRD7Uhpt3L6yrcrIKWexYDlykbci+8oMUVVe2O7+g7Qjncc4DaMuWjOE s3L5iiU+QZKAfl4c2w54EPX1YjHrq3a1P34M18xuIqtS2o2d7T+6V9ODx8UXWN55c4hw ffZr+NpwZtSPK/QTqELMPRvabhGf+5Rgn3zYjtjaZXU3k9+ILIVTHAbT/CYuk01lRRi0 1s+Q== X-Forwarded-Encrypted: i=1; AJvYcCUQns3eUC4NR1QJT5ey3bvXwEdoywk8yRLzNEacTTKGrYBr+SIzyvGGqvO6UTHvgs2EkNwT3ETVPXzb1+4=@vger.kernel.org X-Gm-Message-State: AOJu0Yx7Tkk4KIDNCp7uKmI1mgkTUea1Rh7Cb0EsaJi2D6Uq5HKg+bXe 2jP8MuAhglScvPgfVu+3KcRL32yIJstEBbB+u2CnjPWV3pvOq7t/kePbr26GjhrYCuvMhV2VDyq c5eGBMDTBjARHKXwCPA== X-Google-Smtp-Source: AGHT+IEV8MdqGcyCgbe2mt3hk2i0zy9sL5yMlrGz8Wfzitbo0qZDeAoS7Bz5wRmT2BDTK8lUXG2WtOVPXNGQznc= X-Received: from wmqg19.prod.google.com ([2002:a05:600c:4ed3:b0:477:a1f9:138c]) (user=aliceryhl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4e91:b0:477:7ae0:cd6e with SMTP id 5b1f17b1804b1-4792a5ddcc9mr8270815e9.5.1764707294808; Tue, 02 Dec 2025 12:28:14 -0800 (PST) Date: Tue, 02 Dec 2025 20:27:59 +0000 In-Reply-To: <20251202-inline-helpers-v1-0-879dae33a66a@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20251202-inline-helpers-v1-0-879dae33a66a@google.com> X-Developer-Key: i=aliceryhl@google.com; a=openpgp; fpr=49F6C1FAA74960F43A5B86A1EE7A392FDE96209F X-Developer-Signature: v=1; a=openpgp-sha256; l=8979; i=aliceryhl@google.com; h=from:subject:message-id; bh=ZnZgTS3QcWRawBw++AAPhDFQtHAEQIZjfTFyfZ+xDmk=; b=owEBbQKS/ZANAwAKAQRYvu5YxjlGAcsmYgBpL0vY58X0j2N0y6zci+722T9v5vmhEw8pSHYzy bBoMvTYi1uJAjMEAAEKAB0WIQSDkqKUTWQHCvFIvbIEWL7uWMY5RgUCaS9L2AAKCRAEWL7uWMY5 RgpGD/0fasxYXiW6xVDDz8y4znLsMtXtDdRWNS+6bJ+27GT03cNM49SunHDvdbd1REYSO2YPMBz kPaPyVggZHDWnyck1v+MiaG+x4LW+ZRIkW8vaUrh+aymRIXxt76FsCU9rUOcgAZ16qMAGGpu6N+ phlcZRw3rBj1By8r+yuXf8PdJKxCF4DbMGLkzT41x5F1+oHfjrHV4azKoBtDZzOLfrQf14dJ+wE MXz5NrtQdnDi8+fGcMM6T2TA4rimlfzl817O9Ho/Z4sbqYoYciomv8XvLM3z5hS9pzvjfB8UYF5 u5m/OyCU61eDcrbiyQJbjny4mgxknTjmeUmsF5u1ihqApi8vbB/ZlYHyZL9MgJEkFKqHAyZKga2 qEGd/jKKrArjdJwDk4TLcMwofyvdezUNx56N+IwbaQ8nKUWxV01veHHnkhvHWBCrkKPP5LvnemU woHsvvPGxiX3D6uXJDgzNM2ZaO4uNVFBDnjOkWEUCRvzPFg4muptJlGYfXtrbLGar7KWHu+0scl +2UMIdbTQV7p5t32aiJn3k4MV+FSMnPlbBEFJohWVT0dFSUgOeB1jnuJHp6oFzzhJtevrHVJ5iD N2Q7FYIYgHL3/LwM4YB/7bDJs6WhgzHaAsvSbU71oiP9t2Fmzvrx0ndD1YMaaBeWrcRhhwR+AH9 PlvCH9Zp+KU5/LA== X-Mailer: b4 0.14.2 Message-ID: <20251202-inline-helpers-v1-4-879dae33a66a@google.com> Subject: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust From: Alice Ryhl To: Miguel Ojeda Cc: Boqun Feng , Gary Guo , "=?utf-8?q?Bj=C3=B6rn_Roy_Baron?=" , Benno Lossin , Andreas Hindborg , Trevor Gross , Danilo Krummrich , Alexandre Courbot , Will Deacon , Peter Zijlstra , Mark Rutland , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , Nicolas Schier , Andrew Morton , Uladzislau Rezki , rust-for-linux@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-kbuild@vger.kernel.org, linux-mm@kvack.org, nouveau@lists.freedesktop.org, Alice Ryhl , Matthew Maurer Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Gary Guo A new experimental Kconfig option, `RUST_INLINE_HELPERS` is added to allow C helpers (which were created to allow Rust to call into inline/macro C functions without having to re-implement the logic in Rust) to be inlined into Rust crates without performing global LTO. If the option is enabled, the following is performed: * For helpers, instead of compiling them to an object file to be linked into vmlinux, they're compiled to LLVM IR. * The LLVM IR is compiled to bitcode (This is step is not necessary, but is a performance optimisation to prevent LLVM from always have to reparse the same IR). * When a Rust crate is compiled, instead of generating an object file, we ask LLVM bitcode to be generated. * llvm-link is invoked with --internalize to combine the helper bitcode with the crate bitcode. This step is similar to LTO, but this is much faster since it only needs to inline the helpers. * clang is invoked to turn the combined bitcode into a final object file. The --internalize flag tells llvm-link to treat all symbols in helpers.bc using `internal` linkage. This matches the behavior of `clang` on `static inline` functions, and avoids exporting the symbol from the object file. To ensure that RUST_INLINE_HELPERS is not incompatible with BTF, we pass the -g0 flag when building helpers. See commit 5daa0c35a1f0 ("rust: Disallow BTF generation with Rust + LTO") for details. We have an intended triple mismatch of `aarch64-unknown-none` vs `aarch64-unknown-linux-gnu`, so we suppress the warning. Co-developed-by: Boqun Feng Signed-off-by: Boqun Feng Co-developed-by: Matthew Maurer Signed-off-by: Matthew Maurer Signed-off-by: Gary Guo Co-developed-by: Alice Ryhl Signed-off-by: Alice Ryhl --- Makefile | 4 +++- lib/Kconfig.debug | 15 +++++++++++++++ rust/Makefile | 26 ++++++++++++++++++++++---- rust/exports.c | 5 ++++- scripts/Makefile.build | 5 ++++- 5 files changed, 48 insertions(+), 7 deletions(-) diff --git a/Makefile b/Makefile index 96ddbaae7e12de71bcfabef4639de3a13a6e4815..5834bfd568548d1bee34b328dcc= ce5d60f85526f 100644 --- a/Makefile +++ b/Makefile @@ -517,6 +517,8 @@ OBJCOPY =3D $(LLVM_PREFIX)llvm-objcopy$(LLVM_SUFFIX) OBJDUMP =3D $(LLVM_PREFIX)llvm-objdump$(LLVM_SUFFIX) READELF =3D $(LLVM_PREFIX)llvm-readelf$(LLVM_SUFFIX) STRIP =3D $(LLVM_PREFIX)llvm-strip$(LLVM_SUFFIX) +LLVM_LINK =3D $(LLVM_PREFIX)llvm-link$(LLVM_SUFFIX) +LLVM_AS =3D $(LLVM_PREFIX)llvm-as$(LLVM_SUFFIX) else CC =3D $(CROSS_COMPILE)gcc LD =3D $(CROSS_COMPILE)ld @@ -625,7 +627,7 @@ export RUSTC_BOOTSTRAP :=3D 1 export CLIPPY_CONF_DIR :=3D $(srctree) =20 export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPI= LE LD CC HOSTPKG_CONFIG -export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN +export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN= LLVM_LINK LLVM_AS export HOSTRUSTC KBUILD_HOSTRUSTFLAGS export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX Y= ACC AWK INSTALLKERNEL export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 3034e294d50df55c4003c5fa5df442f59e711bd8..e63c5eb57b049aff988419ccd12= dfd99d59f5080 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -3427,6 +3427,21 @@ config RUST_KERNEL_DOCTESTS =20 If unsure, say N. =20 +config RUST_INLINE_HELPERS + bool "Inline C helpers into Rust crates (EXPERIMENTAL)" + depends on RUST && RUSTC_CLANG_LLVM_COMPATIBLE + depends on EXPERT + help + Links C helpers into Rust crates through LLVM IR. + + If this option is enabled, instead of generating object files dire= ctly, + rustc is asked to produce LLVM IR, which is then linked together w= ith + the LLVM IR of C helpers, before object file is generated. + + This requires a matching LLVM version for Clang and rustc. + + If unsure, say N. + endmenu # "Rust" =20 endmenu # Kernel hacking diff --git a/rust/Makefile b/rust/Makefile index d7d19c21b671dea10242b1772a8bcf0bf5dcc1cd..2344e2662ce2928058221595413= 2c09f63cd8c9d 100644 --- a/rust/Makefile +++ b/rust/Makefile @@ -6,15 +6,19 @@ rustdoc_output :=3D $(objtree)/Documentation/output/rust/= rustdoc obj-$(CONFIG_RUST) +=3D core.o compiler_builtins.o ffi.o always-$(CONFIG_RUST) +=3D exports_core_generated.h =20 +ifdef CONFIG_RUST_INLINE_HELPERS +always-$(CONFIG_RUST) +=3D helpers/helpers.bc +else +obj-$(CONFIG_RUST) +=3D helpers/helpers.o +always-$(CONFIG_RUST) +=3D exports_helpers_generated.h +endif # Missing prototypes are expected in the helpers since these are exported # for Rust only, thus there is no header nor prototypes. -obj-$(CONFIG_RUST) +=3D helpers/helpers.o CFLAGS_REMOVE_helpers/helpers.o =3D -Wmissing-prototypes -Wmissing-declara= tions =20 always-$(CONFIG_RUST) +=3D bindings/bindings_generated.rs bindings/binding= s_helpers_generated.rs obj-$(CONFIG_RUST) +=3D bindings.o pin_init.o kernel.o -always-$(CONFIG_RUST) +=3D exports_helpers_generated.h \ - exports_bindings_generated.h exports_kernel_generated.h +always-$(CONFIG_RUST) +=3D exports_bindings_generated.h exports_kernel_gen= erated.h =20 always-$(CONFIG_RUST) +=3D uapi/uapi_generated.rs obj-$(CONFIG_RUST) +=3D uapi.o @@ -468,6 +472,13 @@ $(obj)/bindings/bindings_helpers_generated.rs: private= bindgen_target_extra =3D ; $(obj)/bindings/bindings_helpers_generated.rs: $(src)/helpers/helpers.c FO= RCE $(call if_changed_dep,bindgen) =20 +quiet_cmd_rust_helper =3D HELPER $@ + cmd_rust_helper =3D \ + $(CC) $(filter-out $(CFLAGS_REMOVE_helpers/helpers.o), $(c_flags)) -c -g0= $< -emit-llvm -o $@ + +$(obj)/helpers/helpers.bc: $(obj)/helpers/helpers.c FORCE + +$(call if_changed_dep,rust_helper) + rust_exports =3D $(NM) -p --defined-only $(1) | awk '$$2~/(T|R|D|B)/ && $$= 3!~/__(pfx|cfi|odr_asan)/ { printf $(2),$$3 }' =20 quiet_cmd_exports =3D EXPORTS $@ @@ -547,11 +558,13 @@ quiet_cmd_rustc_library =3D $(if $(skip_clippy),RUSTC= ,$(RUSTC_OR_CLIPPY_QUIET)) L OBJTREE=3D$(abspath $(objtree)) \ $(if $(skip_clippy),$(RUSTC),$(RUSTC_OR_CLIPPY)) \ $(filter-out $(skip_flags),$(rust_flags)) $(rustc_target_flags) \ - --emit=3Ddep-info=3D$(depfile) --emit=3Dobj=3D$@ \ + --emit=3Ddep-info=3D$(depfile) --emit=3D$(if $(link_helper),llvm-bc=3D$(= patsubst %.o,%.bc,$@),obj=3D$@) \ --emit=3Dmetadata=3D$(dir $@)$(patsubst %.o,lib%.rmeta,$(notdir $@)) \ --crate-type rlib -L$(objtree)/$(obj) \ --crate-name $(patsubst %.o,%,$(notdir $@)) $< \ --sysroot=3D/dev/null \ + $(if $(link_helper),;$(LLVM_LINK) --internalize --suppress-warnings $(pat= subst %.o,%.bc,$@) $(obj)/helpers/helpers.bc -o $(patsubst %.o,%.m.bc,$@); \ + $(CC) $(CLANG_FLAGS) $(KBUILD_CFLAGS) -Wno-override-module -c $(patsubst= %.o,%.m.bc,$@) -o $@) \ $(if $(rustc_objcopy),;$(OBJCOPY) $(rustc_objcopy) $@) \ $(cmd_objtool) =20 @@ -678,4 +691,9 @@ $(obj)/kernel.o: $(obj)/kernel/generated_arch_warn_asm.= rs $(obj)/kernel/generate endif endif =20 +ifdef CONFIG_RUST_INLINE_HELPERS +$(obj)/kernel.o: private link_helper =3D 1 +$(obj)/kernel.o: $(obj)/helpers/helpers.bc +endif + endif # CONFIG_RUST diff --git a/rust/exports.c b/rust/exports.c index 587f0e776aba52854080f15aa91094b55996c072..1b52460b0f4eeef6df9081abb9b= 7e054a28c3c21 100644 --- a/rust/exports.c +++ b/rust/exports.c @@ -16,10 +16,13 @@ #define EXPORT_SYMBOL_RUST_GPL(sym) extern int sym; EXPORT_SYMBOL_GPL(sym) =20 #include "exports_core_generated.h" -#include "exports_helpers_generated.h" #include "exports_bindings_generated.h" #include "exports_kernel_generated.h" =20 +#ifndef CONFIG_RUST_INLINE_HELPERS +#include "exports_helpers_generated.h" +#endif + // For modules using `rust/build_error.rs`. #ifdef CONFIG_RUST_BUILD_ASSERT_ALLOW EXPORT_SYMBOL_RUST_GPL(rust_build_error); diff --git a/scripts/Makefile.build b/scripts/Makefile.build index d0ee33a487be95f8ba9a5c964ebecfbebc6c4bf8..04eaf2b4fbca2245f904a6dc787= 5cb3275aa7df6 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -343,7 +343,10 @@ rust_common_cmd =3D \ # would not match each other. =20 quiet_cmd_rustc_o_rs =3D $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@ - cmd_rustc_o_rs =3D $(rust_common_cmd) --emit=3Dobj=3D$@ $< $(cmd_obj= tool) + cmd_rustc_o_rs =3D $(rust_common_cmd) --emit=3D$(if $(CONFIG_RUST_IN= LINE_HELPERS),llvm-bc=3D$(patsubst %.o,%.bc,$@),obj=3D$@) $< \ + $(if $(CONFIG_RUST_INLINE_HELPERS),;$(LLVM_LINK) --internalize --suppress= -warnings $(patsubst %.o,%.bc,$@) $(objtree)/rust/helpers/helpers.bc -o $(p= atsubst %.o,%.m.bc,$@); \ + $(CC) $(CLANG_FLAGS) $(KBUILD_CFLAGS) -Wno-override-module -c $(patsubst= %.o,%.m.bc,$@) -o $@) \ + $(cmd_objtool) =20 define rule_rustc_o_rs $(call cmd_and_fixdep,rustc_o_rs) --=20 2.52.0.158.g65b55ccf14-goog