From nobody Tue Oct 7 01:59:24 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E7E44288C09 for ; Tue, 15 Jul 2025 22:57:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752620267; cv=none; b=oH1+2KMz78lOJ3EzkLyMrdo2MTpYcOTjvqPnC2jfZxq7TkIlaiVSRn+81JnHF9BmIYm7OhUfIA3CLNQcXUvypxsKwJG9vCiAbKolCXIK2kqV9WkVcVntqW90Ocpw8YspVxRLeirSsQ6d+K48HpgrDeAJ7cE2sf2+wIMJAtc97mU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752620267; c=relaxed/simple; bh=gER9ScMtOtwC9xpclIhEAteq0OGt+Y3IoBSJHGzYfCY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PVfilZotErSfGmosJJlHUSld+cSZ61455mJs1p0YXZt7GYuDhQ1qn/etp5i5m/xPH8wZyoEUIhX38+Ew1ZhLPSMPW9RSNXfSPqpUwO6m6ey4QgRYiRuL8kR7bAAvjBuYTjbNYt8/saoxuGkAowTqx/5SwBE2H66/Xly2Ku0ANSs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--samitolvanen.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=N7WGLG5V; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--samitolvanen.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="N7WGLG5V" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-312df02acf5so254488a91.1 for ; Tue, 15 Jul 2025 15:57:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1752620265; x=1753225065; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=v5AK/1PeqQC+sa8XwDnM8NGzttuQcxnY7sFwTeAuNhw=; b=N7WGLG5VXtZxhb0b3DJQ43WbkHcs54n6udzwT6FaynY2KlSfEZCEn8w3pKUXvBHgtN AlKI42hRztyrCyXaZlaEJNBQPnT5y3HI14H7HsXwcRzY1++pZ4CdwIZ8S6wl56RMZed7 E2be85hprhQs+lZOhf9g5DxtNgIOq9cKZwbng5hYjl6sMhAt9oB3JENeJC0Kim1Jojtv HWizTgfSKFm9i9sQODKPScnxtQ2oWnWmLF67SnmfyU2enJggxKJx2CquvUayo/XFL/5U rwywBSyt/SDJNFMEaeciNwocW3s28I6ElWNPrP//Fs91ZYt46sgROjX9wgXpWVyeY0b3 12hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752620265; x=1753225065; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=v5AK/1PeqQC+sa8XwDnM8NGzttuQcxnY7sFwTeAuNhw=; b=HzyQyWxKWrSHL9uOglik6LFf5LZw/G/jYb12hVD+WDznCcm/yUr1AeuAzR3DQGlMYF TdCGtztj1Vcz2//F8l48W/Bc9S2ITx2ydIZ4isdCIUPETiBZYSMNQAqza0QzIYSepoB9 zr5eKIHsnHqXObJLqFdsGrT5ODjtzwypfKmi3E8fuUHU1+RRC5vJo3QR7bi6dDASO8MB nuwGyU46GWbZXU4dAz43iKfuQ4QNu4VjluBZFNdYGmN5UR11Gc2Cw7YKKHa74PamBGUK o9jNqGppp0l3UYQFI0chMf11bVnvBxw7u297W7dq9SKI/I/r1XQzcmEZCMJztDAWfhLS rdOQ== X-Forwarded-Encrypted: i=1; AJvYcCVRl1eOK8rk+mkUeJLtlLRYxwVAbE+oYsS3crQr+rlOLrqMvQ8stlRCrAoV+7XsjX2eUHbeQ1j8mI39XyA=@vger.kernel.org X-Gm-Message-State: AOJu0YyrK9nFCJMK6UdcqEuHNrp8kogLDq9l6sTC88fAy/V2wgDIr4hb 4mmhuefk6aEzB0s9bY8l4YnT/EYCWHQKxL/vgas0Fa0ee7Y+k00X9IEh+d3Cs/Z+PdpRI4WeF1U bFE2dpdpot2OhI75ZHzzlNi6XApRrMQ== X-Google-Smtp-Source: AGHT+IFRJnQuf3ZfTFNheXhqqmEyyVMwsn+nHtXsPsgY0nnTYfz2CvtHbU+B8gUhLr16qjC7a3OLa11W0DsbaBWGlp0= X-Received: from pjwx13.prod.google.com ([2002:a17:90a:c2cd:b0:313:230:89ed]) (user=samitolvanen job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1b4d:b0:2ff:4a8d:74f9 with SMTP id 98e67ed59e1d1-31c9e73efe6mr1347174a91.10.1752620265306; Tue, 15 Jul 2025 15:57:45 -0700 (PDT) Date: Tue, 15 Jul 2025 22:57:35 +0000 In-Reply-To: <20250715225733.3921432-5-samitolvanen@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250715225733.3921432-5-samitolvanen@google.com> X-Developer-Key: i=samitolvanen@google.com; a=openpgp; fpr=35CCFB63B283D6D3AEB783944CB5F6848BBC56EE X-Developer-Signature: v=1; a=openpgp-sha256; l=5258; i=samitolvanen@google.com; h=from:subject; bh=fmHbTT6n3AcyMrdoZaFttPRZqXRsHYHqk54PFgD4xF8=; b=owGbwMvMwCUWxa662nLh8irG02pJDBlld+4dcv66/MPU6FXsLAsCTrLUa5wV/ZDUqjj/UHHN/ Lot76ukO0pZGMS4GGTFFFlavq7euvu7U+qrz0USMHNYmUCGMHBxCsBEVEIY/vBdZ+ipVq/R0pQ9 tOfUjd1iHpZX99WWGoSlML2v6ku//5/hn9K0Umljzpvnz/78GHw3sNjAfoFfaTj36uMJC18yXFs fywsA X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250715225733.3921432-6-samitolvanen@google.com> Subject: [PATCH bpf-next v10 1/3] cfi: add C CFI type macro From: Sami Tolvanen To: bpf@vger.kernel.org, Puranjay Mohan , Alexei Starovoitov , Daniel Borkmann Cc: Catalin Marinas , Will Deacon , Andrii Nakryiko , Mark Rutland , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Maxwell Bland , Sami Tolvanen , Dao Huang Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Mark Rutland Currently x86 and riscv open-code 4 instances of the same logic to define a u32 variable with the KCFI typeid of a given function. Replace the duplicate logic with a common macro. Signed-off-by: Mark Rutland Co-developed-by: Maxwell Bland Signed-off-by: Maxwell Bland Co-developed-by: Sami Tolvanen Signed-off-by: Sami Tolvanen Tested-by: Dao Huang Acked-by: Will Deacon --- arch/riscv/kernel/cfi.c | 35 +++-------------------------------- arch/x86/kernel/alternative.c | 31 +++---------------------------- include/linux/cfi_types.h | 23 +++++++++++++++++++++++ 3 files changed, 29 insertions(+), 60 deletions(-) diff --git a/arch/riscv/kernel/cfi.c b/arch/riscv/kernel/cfi.c index 64bdd3e1ab8c..e7aec5f36dd5 100644 --- a/arch/riscv/kernel/cfi.c +++ b/arch/riscv/kernel/cfi.c @@ -4,6 +4,7 @@ * * Copyright (C) 2023 Google LLC */ +#include #include #include =20 @@ -82,41 +83,11 @@ struct bpf_insn; /* Must match bpf_func_t / DEFINE_BPF_PROG_RUN() */ extern unsigned int __bpf_prog_runX(const void *ctx, const struct bpf_insn *insn); - -/* - * Force a reference to the external symbol so the compiler generates - * __kcfi_typid. - */ -__ADDRESSABLE(__bpf_prog_runX); - -/* u32 __ro_after_init cfi_bpf_hash =3D __kcfi_typeid___bpf_prog_runX; */ -asm ( -" .pushsection .data..ro_after_init,\"aw\",@progbits \n" -" .type cfi_bpf_hash,@object \n" -" .globl cfi_bpf_hash \n" -" .p2align 2, 0x0 \n" -"cfi_bpf_hash: \n" -" .word __kcfi_typeid___bpf_prog_runX \n" -" .size cfi_bpf_hash, 4 \n" -" .popsection \n" -); +DEFINE_CFI_TYPE(cfi_bpf_hash, __bpf_prog_runX); =20 /* Must match bpf_callback_t */ extern u64 __bpf_callback_fn(u64, u64, u64, u64, u64); - -__ADDRESSABLE(__bpf_callback_fn); - -/* u32 __ro_after_init cfi_bpf_subprog_hash =3D __kcfi_typeid___bpf_callba= ck_fn; */ -asm ( -" .pushsection .data..ro_after_init,\"aw\",@progbits \n" -" .type cfi_bpf_subprog_hash,@object \n" -" .globl cfi_bpf_subprog_hash \n" -" .p2align 2, 0x0 \n" -"cfi_bpf_subprog_hash: \n" -" .word __kcfi_typeid___bpf_callback_fn \n" -" .size cfi_bpf_subprog_hash, 4 \n" -" .popsection \n" -); +DEFINE_CFI_TYPE(cfi_bpf_subprog_hash, __bpf_callback_fn); =20 u32 cfi_get_func_hash(void *func) { diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index ea1d984166cd..a555665b4d9c 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -2,6 +2,7 @@ #define pr_fmt(fmt) "SMP alternatives: " fmt =20 #include +#include #include #include #include @@ -1189,37 +1190,11 @@ struct bpf_insn; /* Must match bpf_func_t / DEFINE_BPF_PROG_RUN() */ extern unsigned int __bpf_prog_runX(const void *ctx, const struct bpf_insn *insn); - -KCFI_REFERENCE(__bpf_prog_runX); - -/* u32 __ro_after_init cfi_bpf_hash =3D __kcfi_typeid___bpf_prog_runX; */ -asm ( -" .pushsection .data..ro_after_init,\"aw\",@progbits \n" -" .type cfi_bpf_hash,@object \n" -" .globl cfi_bpf_hash \n" -" .p2align 2, 0x0 \n" -"cfi_bpf_hash: \n" -" .long __kcfi_typeid___bpf_prog_runX \n" -" .size cfi_bpf_hash, 4 \n" -" .popsection \n" -); +DEFINE_CFI_TYPE(cfi_bpf_hash, __bpf_prog_runX); =20 /* Must match bpf_callback_t */ extern u64 __bpf_callback_fn(u64, u64, u64, u64, u64); - -KCFI_REFERENCE(__bpf_callback_fn); - -/* u32 __ro_after_init cfi_bpf_subprog_hash =3D __kcfi_typeid___bpf_callba= ck_fn; */ -asm ( -" .pushsection .data..ro_after_init,\"aw\",@progbits \n" -" .type cfi_bpf_subprog_hash,@object \n" -" .globl cfi_bpf_subprog_hash \n" -" .p2align 2, 0x0 \n" -"cfi_bpf_subprog_hash: \n" -" .long __kcfi_typeid___bpf_callback_fn \n" -" .size cfi_bpf_subprog_hash, 4 \n" -" .popsection \n" -); +DEFINE_CFI_TYPE(cfi_bpf_subprog_hash, __bpf_callback_fn); =20 u32 cfi_get_func_hash(void *func) { diff --git a/include/linux/cfi_types.h b/include/linux/cfi_types.h index 6b8713675765..e5567c0fd0b3 100644 --- a/include/linux/cfi_types.h +++ b/include/linux/cfi_types.h @@ -41,5 +41,28 @@ SYM_TYPED_START(name, SYM_L_GLOBAL, SYM_A_ALIGN) #endif =20 +#else /* __ASSEMBLY__ */ + +#ifdef CONFIG_CFI_CLANG +#define DEFINE_CFI_TYPE(name, func) \ + /* \ + * Force a reference to the function so the compiler generates \ + * __kcfi_typeid_. \ + */ \ + __ADDRESSABLE(func); \ + /* u32 name __ro_after_init =3D __kcfi_typeid_ */ \ + extern u32 name; \ + asm ( \ + " .pushsection .data..ro_after_init,\"aw\",@progbits \n" \ + " .type " #name ",@object \n" \ + " .globl " #name " \n" \ + " .p2align 2, 0x0 \n" \ + #name ": \n" \ + " .4byte __kcfi_typeid_" #func " \n" \ + " .size " #name ", 4 \n" \ + " .popsection \n" \ + ); +#endif + #endif /* __ASSEMBLY__ */ #endif /* _LINUX_CFI_TYPES_H */ --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Tue Oct 7 01:59:24 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8CF7D2D322A for ; Tue, 15 Jul 2025 22:57:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752620269; cv=none; b=on8eGrDzIUl09AD0M43usYx0jM8c8N64Ek4cjiPNCNZTSfDrWKZY26uhPFEDrK8M27SNdwkZ0PyF97WvY07FuqpLuSW6Ai7YXxh+28s5q9EWaJWdc6AtVDZBiAHmmEhWgId5b9rsVcnhAD3KJgUAENHqgEj43vS+KRnmbnPsEG8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752620269; c=relaxed/simple; bh=JYMMqOIguwvmYlpCtBUkSC4Y7ybrNECAIqn/EkjfUiY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ERrWWz3n5tBQZKS7IyqeA75WCphcTxGAqyTJyoyqyOWbmWiuZOMdDXgLexNiSAzWLhiVX425++fBTd7GhJFyG16nUJ5iaP/bwRw5O2HEXXppxjG2VJStC8TpUbODh+91Kje/yNq/doNzErnZvLm5Lfb8tHYZtKwBeu+Qe/11yVk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--samitolvanen.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=CEVvLB/H; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--samitolvanen.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="CEVvLB/H" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-74b4d2f67d5so4892640b3a.3 for ; Tue, 15 Jul 2025 15:57:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1752620267; x=1753225067; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=bxkEzNpl+LQg8WYHJm151LdjKk0iMvXzxlrj1qGda2U=; b=CEVvLB/HzNwqHk5mmu3tKc+jV4Oy+1+ts9+uYwxSSCugOqlaDSXw7sVVWsAZKX/ykP EUCErI5TMs35GzeIu5iOVpJvYvS5peV36klYe1N3Nvulx/YxRTrL/+iWvg2OFx+JVJSm rnHESZ1sDNFuEbf+iLYfrUW3eGHN5ZdqYCvrlDy8q+HgoPEhaiQtCAub/J/5ydIiMyj3 n5BgAjwT2cRpBaYOY4fE0+anHkx5Fb/GZBJuVnNGrqdjZlXx2AUySyhAJVfDkczcGjmM 0/bl2BbyjwbdbeNlhtpVv0cauVfkkiwaXPgpSF/Hs4p3oQbHSkIEcP3ompXts+A/mQ3V 5WsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752620267; x=1753225067; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=bxkEzNpl+LQg8WYHJm151LdjKk0iMvXzxlrj1qGda2U=; b=N612P/Xdpes+64SDHQenfNYZYEKTDR9wi0UOrYyov2u4EdTwBiiOhq1uOiDtSDIiOw tKFsUFDMUC6JzgTce3QyZzlPuOUvaFYgbt8ZFkkXQpWCwQ/gOKkeDbqA9i5uyoOtDyar lnnrGCRmxY79dPnCMuso+6lKtfiPr9/K+Nd4ql5DVlIT/mXB0kIRBkYNHT5vyqGnuh0j uRDmh2aWnPFus1+Rzl6jNx9ao1zeiqjeaTt/ktfFFK0bJnX4fUaZigvE8aKhOLtAjSA/ ZFfoaFO+78F76PMONW1HMN524+DzfBNfkbYFOMVfncJvOrly74hvmIalqLpcYmBC5JtL 4xPQ== X-Forwarded-Encrypted: i=1; AJvYcCUV6wtta0Bq4DsMsz4+76OHDT8HSryU2pfJlvR1TEtZbvXFN8oWDntpBtd4vrlkkKP89Y1eBDVVR80Lk4s=@vger.kernel.org X-Gm-Message-State: AOJu0YwAap7Q8bYqcv3bc3V91fLcABmdszXjwe5GuSBCTwBrmSPAY5Ob T7khRABXe7oP3U6iXVYFuU8xWnwI4B/PquWE5W4B1hx2J1vJpZT7+2x/ackeRNxkrDpkX0rITcE P508Lg5Aj7ZCaYQWQkzfVxBtB2MCaWg== X-Google-Smtp-Source: AGHT+IG2nwRuZU8nKSGZKv/KCQaDAPwv7UTHMunOLKPdpbGG0wYtSG/FaNgQ997rCgAEFOJqVAawtb6zF7I7itb5eWs= X-Received: from pgac23.prod.google.com ([2002:a05:6a02:2957:b0:b2e:ce0c:b3fb]) (user=samitolvanen job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:72a4:b0:235:2cd8:6cd1 with SMTP id adf61e73a8af0-237d754df40mr2164567637.34.1752620266993; Tue, 15 Jul 2025 15:57:46 -0700 (PDT) Date: Tue, 15 Jul 2025 22:57:36 +0000 In-Reply-To: <20250715225733.3921432-5-samitolvanen@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250715225733.3921432-5-samitolvanen@google.com> X-Developer-Key: i=samitolvanen@google.com; a=openpgp; fpr=35CCFB63B283D6D3AEB783944CB5F6848BBC56EE X-Developer-Signature: v=1; a=openpgp-sha256; l=7361; i=samitolvanen@google.com; h=from:subject; bh=JYMMqOIguwvmYlpCtBUkSC4Y7ybrNECAIqn/EkjfUiY=; b=kA0DAAoWWgclqzmhp3oByyZiAGh23N+jP/g5zdO96cF7ZHZ4RED3bDnow5av8UA0IbpUzoer8 Yh1BAAWCgAdFiEEhPWrtbv3QmXq83IYWgclqzmhp3oFAmh23N8ACgkQWgclqzmhp3pszQEA1Eo4 g3Xc1MqFDX6ocKGbT/3AM0t8FXnN6Zbjsw9b+BMBAJqjNShp+eL0Gptqhtbb9qzV483Nl6NngZZ WhhyOp/wJ X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250715225733.3921432-7-samitolvanen@google.com> Subject: [PATCH bpf-next v10 2/3] cfi: Move BPF CFI types and helpers to generic code From: Sami Tolvanen To: bpf@vger.kernel.org, Puranjay Mohan , Alexei Starovoitov , Daniel Borkmann Cc: Catalin Marinas , Will Deacon , Andrii Nakryiko , Mark Rutland , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Maxwell Bland , Sami Tolvanen Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Instead of duplicating the same code for each architecture, move the CFI type hash variables for BPF function types and related helper functions to generic CFI code, and allow architectures to override the function definitions if needed. Signed-off-by: Sami Tolvanen --- arch/riscv/include/asm/cfi.h | 16 --------------- arch/riscv/kernel/cfi.c | 24 ----------------------- arch/x86/include/asm/cfi.h | 9 --------- arch/x86/kernel/alternative.c | 12 ------------ include/linux/cfi.h | 37 +++++++++++++++++++++++++++-------- kernel/cfi.c | 25 +++++++++++++++++++++++ 6 files changed, 54 insertions(+), 69 deletions(-) diff --git a/arch/riscv/include/asm/cfi.h b/arch/riscv/include/asm/cfi.h index fb9696d7a3f2..4508aaa7a2fd 100644 --- a/arch/riscv/include/asm/cfi.h +++ b/arch/riscv/include/asm/cfi.h @@ -14,27 +14,11 @@ struct pt_regs; #ifdef CONFIG_CFI_CLANG enum bug_trap_type handle_cfi_failure(struct pt_regs *regs); #define __bpfcall -static inline int cfi_get_offset(void) -{ - return 4; -} - -#define cfi_get_offset cfi_get_offset -extern u32 cfi_bpf_hash; -extern u32 cfi_bpf_subprog_hash; -extern u32 cfi_get_func_hash(void *func); #else static inline enum bug_trap_type handle_cfi_failure(struct pt_regs *regs) { return BUG_TRAP_TYPE_NONE; } - -#define cfi_bpf_hash 0U -#define cfi_bpf_subprog_hash 0U -static inline u32 cfi_get_func_hash(void *func) -{ - return 0; -} #endif /* CONFIG_CFI_CLANG */ =20 #endif /* _ASM_RISCV_CFI_H */ diff --git a/arch/riscv/kernel/cfi.c b/arch/riscv/kernel/cfi.c index e7aec5f36dd5..6ec9dbd7292e 100644 --- a/arch/riscv/kernel/cfi.c +++ b/arch/riscv/kernel/cfi.c @@ -4,7 +4,6 @@ * * Copyright (C) 2023 Google LLC */ -#include #include #include =20 @@ -76,26 +75,3 @@ enum bug_trap_type handle_cfi_failure(struct pt_regs *re= gs) =20 return report_cfi_failure(regs, regs->epc, &target, type); } - -#ifdef CONFIG_CFI_CLANG -struct bpf_insn; - -/* Must match bpf_func_t / DEFINE_BPF_PROG_RUN() */ -extern unsigned int __bpf_prog_runX(const void *ctx, - const struct bpf_insn *insn); -DEFINE_CFI_TYPE(cfi_bpf_hash, __bpf_prog_runX); - -/* Must match bpf_callback_t */ -extern u64 __bpf_callback_fn(u64, u64, u64, u64, u64); -DEFINE_CFI_TYPE(cfi_bpf_subprog_hash, __bpf_callback_fn); - -u32 cfi_get_func_hash(void *func) -{ - u32 hash; - - if (get_kernel_nofault(hash, func - cfi_get_offset())) - return 0; - - return hash; -} -#endif diff --git a/arch/x86/include/asm/cfi.h b/arch/x86/include/asm/cfi.h index 3e51ba459154..64506a91971b 100644 --- a/arch/x86/include/asm/cfi.h +++ b/arch/x86/include/asm/cfi.h @@ -116,8 +116,6 @@ struct pt_regs; #ifdef CONFIG_CFI_CLANG enum bug_trap_type handle_cfi_failure(struct pt_regs *regs); #define __bpfcall -extern u32 cfi_bpf_hash; -extern u32 cfi_bpf_subprog_hash; =20 static inline int cfi_get_offset(void) { @@ -134,7 +132,6 @@ static inline int cfi_get_offset(void) } #define cfi_get_offset cfi_get_offset =20 -extern u32 cfi_get_func_hash(void *func); extern int cfi_get_func_arity(void *func); =20 #ifdef CONFIG_FINEIBT @@ -153,12 +150,6 @@ static inline enum bug_trap_type handle_cfi_failure(st= ruct pt_regs *regs) { return BUG_TRAP_TYPE_NONE; } -#define cfi_bpf_hash 0U -#define cfi_bpf_subprog_hash 0U -static inline u32 cfi_get_func_hash(void *func) -{ - return 0; -} static inline int cfi_get_func_arity(void *func) { return 0; diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index a555665b4d9c..9f6b7dab2d9a 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -2,7 +2,6 @@ #define pr_fmt(fmt) "SMP alternatives: " fmt =20 #include -#include #include #include #include @@ -1185,17 +1184,6 @@ bool cfi_bhi __ro_after_init =3D false; #endif =20 #ifdef CONFIG_CFI_CLANG -struct bpf_insn; - -/* Must match bpf_func_t / DEFINE_BPF_PROG_RUN() */ -extern unsigned int __bpf_prog_runX(const void *ctx, - const struct bpf_insn *insn); -DEFINE_CFI_TYPE(cfi_bpf_hash, __bpf_prog_runX); - -/* Must match bpf_callback_t */ -extern u64 __bpf_callback_fn(u64, u64, u64, u64, u64); -DEFINE_CFI_TYPE(cfi_bpf_subprog_hash, __bpf_callback_fn); - u32 cfi_get_func_hash(void *func) { u32 hash; diff --git a/include/linux/cfi.h b/include/linux/cfi.h index 1db17ecbb86c..53ed07cb9258 100644 --- a/include/linux/cfi.h +++ b/include/linux/cfi.h @@ -11,16 +11,9 @@ #include #include =20 +#ifdef CONFIG_CFI_CLANG extern bool cfi_warn; =20 -#ifndef cfi_get_offset -static inline int cfi_get_offset(void) -{ - return 0; -} -#endif - -#ifdef CONFIG_CFI_CLANG enum bug_trap_type report_cfi_failure(struct pt_regs *regs, unsigned long = addr, unsigned long *target, u32 type); =20 @@ -29,6 +22,34 @@ static inline enum bug_trap_type report_cfi_failure_noad= dr(struct pt_regs *regs, { return report_cfi_failure(regs, addr, NULL, 0); } + +#ifndef cfi_get_offset +/* + * Returns the CFI prefix offset. By default, the compiler emits only + * a 4-byte CFI type hash before the function. If an architecture + * uses -fpatchable-function-entry=3DN,M where M>0 to change the prefix + * offset, they must override this function. + */ +static inline int cfi_get_offset(void) +{ + return 4; +} +#endif + +u32 cfi_get_func_hash(void *func); + +/* CFI type hashes for BPF function types */ +extern u32 cfi_bpf_hash; +extern u32 cfi_bpf_subprog_hash; + +#else /* CONFIG_CFI_CLANG */ + +static inline int cfi_get_offset(void) { return 0; } +static inline u32 cfi_get_func_hash(void *func) { return 0; } + +#define cfi_bpf_hash 0U +#define cfi_bpf_subprog_hash 0U + #endif /* CONFIG_CFI_CLANG */ =20 #ifdef CONFIG_ARCH_USES_CFI_TRAPS diff --git a/kernel/cfi.c b/kernel/cfi.c index 422fa4f958ae..68b3760c15a6 100644 --- a/kernel/cfi.c +++ b/kernel/cfi.c @@ -5,6 +5,8 @@ * Copyright (C) 2022 Google LLC */ =20 +#include +#include #include =20 bool cfi_warn __ro_after_init =3D IS_ENABLED(CONFIG_CFI_PERMISSIVE); @@ -27,6 +29,29 @@ enum bug_trap_type report_cfi_failure(struct pt_regs *re= gs, unsigned long addr, return BUG_TRAP_TYPE_BUG; } =20 +u32 __weak cfi_get_func_hash(void *func) +{ + u32 hash; + + if (get_kernel_nofault(hash, func - cfi_get_offset())) + return 0; + + return hash; +} + +/* + * Declare two non-existent functions with types that match bpf_func_t and + * bpf_callback_t pointers, and use DEFINE_CFI_TYPE to define type hash + * variables for each function type. The cfi_bpf_* variables are used by + * arch-specific BPF JIT implementations to ensure indirectly callable JIT + * code has matching CFI type hashes. + */ +typeof(*(bpf_func_t)0) __bpf_prog_runX; +DEFINE_CFI_TYPE(cfi_bpf_hash, __bpf_prog_runX); + +typeof(*(bpf_callback_t)0) __bpf_callback_fn; +DEFINE_CFI_TYPE(cfi_bpf_subprog_hash, __bpf_callback_fn); + #ifdef CONFIG_ARCH_USES_CFI_TRAPS static inline unsigned long trap_address(s32 *p) { --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Tue Oct 7 01:59:24 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A6322D5C7C for ; Tue, 15 Jul 2025 22:57:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752620270; cv=none; b=O3jbLsgVS/TlEFcnAEJgsqt2JFPIPJRwXx+mripWv+iO0gNWGsq+yJcHTQFnsz6O8U7vCc6TU6FhblaGZlvdtezcGCuL59qyfUnRz6dCks4DmEAYdrmvZ8wX/MNyDFGVj9IFDj851U/hOO4Q4ey3hIjdCcDVb9Fu4mGhP7R9Im0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752620270; c=relaxed/simple; bh=cfGraB+ty8ghSuPDkOx2L0sZYlF2Nl9H3u/A7CqoDTs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=cKVG8frf8rFOms2Udz7CHCpFPMxnJ5okeSVRPCeR9gT5UBnStNucpJMhR23kELMTe1fhx4Qy3VuYCyao+xVXIm+QaPpElezKicCKVNgUHKnjJt/IVIhADWVjbDpqEB7lzO5o5jgfO3LoCQE5Ga4bUjgKkpQq7fhhKYf0K3nwg2g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--samitolvanen.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=FxpsSWQ3; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--samitolvanen.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FxpsSWQ3" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-756bb07b029so384725b3a.1 for ; Tue, 15 Jul 2025 15:57:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1752620268; x=1753225068; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=UMeLoHHXfTV79ojOu6a6TdIa46QGFM3PiTHew+k00C8=; b=FxpsSWQ3M+sE//NvWX7+zutQyeGf72aQIYMI7rlwV4QZBnjHwr5GEEgcDsop9L62su qMWN6XYqVsfiNgdq3bRgVyyQwcv9cdFyX8y85v6S9YpZEUiVj4LCfOEnBuKbVOmonftr pmiRvVh7NpHMgNVs/FYcx3rUClrS6zpMOGprr42a+2CQwptK0GnpiD0pi2PfVYnsjsqu TpXepB3Wo+e7HJ9bQ6Wo/7U7W3CRYfR+fDip8TYYYrLYj/z2gPWTn3T6ueNMawvxI2KL 0/T0wIjCdFcqydifUvilNnHiHZ0+HSgNJ077BQt+iITgdga8oUKpPDZsgqEJ2w9Xe7Mi y6mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752620268; x=1753225068; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UMeLoHHXfTV79ojOu6a6TdIa46QGFM3PiTHew+k00C8=; b=KOeuRh5Jt9MPtEIvGWDxwCtLv96p44VdByorUAx3JhRxhFb0I9itqio2eX88Z7Onsz sOxr7Ey2EzEc7EyDz65VLiYuTr1LZA1FISih4N3I6BGURNMMlWXbfnN57MchSMcP4B9C VQONCMWAqhpZnWmqxmSVntfc8T/fJGOpUh3g/mbp39sZJxtdiQjE4Ihkoc1hCli+fkM/ qho7/AZ1HZfnpKhbl/kERti8jlEzq31K+gL0fAMCff8QIC8pWNaBUMRQbg1b3hFRnQW4 ZQD69IB6mToefYwSlXK6Zq1QiTbw1wqI/uA2TC9bZAn4DYkiUi9cwbGFkj1IsfuP0Gtd MbRQ== X-Forwarded-Encrypted: i=1; AJvYcCUlXmFje7JUIjo0h7MLHty+qqgyVyO9jfiI7EtVExv6kekrihkGvufpOa1ZNx6oqu1mUdyjGCXLqtw2Eqc=@vger.kernel.org X-Gm-Message-State: AOJu0YxgmH3KBoHzzFkGrh1mEAITwHBaasVNgVAc3aiI86qpNUfrESwB LT3YbnPwVy9gjcxMhbDvQlp5xaCxgGWd8GYTrzSA/sZji6kqZsA4pvEPXjVCVwdt64S7y8Agv+q dcZFQ9ATexyA+5cjbIeXURX7VhBKinQ== X-Google-Smtp-Source: AGHT+IG3vbzi3NS1evCtARelMjUlrEo+9oD75X7Kycti+9xG6ucYm6MbJrliwfPbGT6yJzXUkmX1s4WzbSQSu1uIXYw= X-Received: from pfbcw5.prod.google.com ([2002:a05:6a00:4505:b0:748:f4a1:ae2e]) (user=samitolvanen job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:33aa:b0:22f:bba6:5dee with SMTP id adf61e73a8af0-23813350a48mr649865637.34.1752620268643; Tue, 15 Jul 2025 15:57:48 -0700 (PDT) Date: Tue, 15 Jul 2025 22:57:37 +0000 In-Reply-To: <20250715225733.3921432-5-samitolvanen@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250715225733.3921432-5-samitolvanen@google.com> X-Developer-Key: i=samitolvanen@google.com; a=openpgp; fpr=35CCFB63B283D6D3AEB783944CB5F6848BBC56EE X-Developer-Signature: v=1; a=openpgp-sha256; l=4074; i=samitolvanen@google.com; h=from:subject; bh=rKFd0jTYWs9nu9VBagQnyYYRJmgpx3uHjv+Tze4TT2E=; b=owGbwMvMwCUWxa662nLh8irG02pJDBlld+5/zxS2cDSfHqLN/vbuV84Dj1mDfnw4sbZX68E19 qcvVLK9OkpZGMS4GGTFFFlavq7euvu7U+qrz0USMHNYmUCGMHBxCsBErGUZ/ucd6+sWWrUn72aQ i8bsr+f7t+RMz9/wsbnj7z1n9hUW+R6MDBM+2oewJ+8In9Tk5rx9vcjNHY3LRL0viWbyvjKfFtL IxQcA X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250715225733.3921432-8-samitolvanen@google.com> Subject: [PATCH bpf-next v10 3/3] arm64/cfi,bpf: Support kCFI + BPF on arm64 From: Sami Tolvanen To: bpf@vger.kernel.org, Puranjay Mohan , Alexei Starovoitov , Daniel Borkmann Cc: Catalin Marinas , Will Deacon , Andrii Nakryiko , Mark Rutland , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Maxwell Bland , Puranjay Mohan , Sami Tolvanen , Dao Huang Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Puranjay Mohan Currently, bpf_dispatcher_*_func() is marked with `__nocfi` therefore calling BPF programs from this interface doesn't cause CFI warnings. When BPF programs are called directly from C: from BPF helpers or struct_ops, CFI warnings are generated. Implement proper CFI prologues for the BPF programs and callbacks and drop __nocfi for arm64. Fix the trampoline generation code to emit kCFI prologue when a struct_ops trampoline is being prepared. Signed-off-by: Puranjay Mohan Co-developed-by: Maxwell Bland Signed-off-by: Maxwell Bland Co-developed-by: Sami Tolvanen Signed-off-by: Sami Tolvanen Tested-by: Dao Huang Acked-by: Will Deacon --- arch/arm64/include/asm/cfi.h | 7 +++++++ arch/arm64/net/bpf_jit_comp.c | 22 +++++++++++++++++++--- 2 files changed, 26 insertions(+), 3 deletions(-) create mode 100644 arch/arm64/include/asm/cfi.h diff --git a/arch/arm64/include/asm/cfi.h b/arch/arm64/include/asm/cfi.h new file mode 100644 index 000000000000..ab90f0351b7a --- /dev/null +++ b/arch/arm64/include/asm/cfi.h @@ -0,0 +1,7 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_ARM64_CFI_H +#define _ASM_ARM64_CFI_H + +#define __bpfcall + +#endif /* _ASM_ARM64_CFI_H */ diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index b6c42b5c9668..7509ac6e76bf 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -166,6 +167,12 @@ static inline void emit_bti(u32 insn, struct jit_ctx *= ctx) emit(insn, ctx); } =20 +static inline void emit_kcfi(u32 hash, struct jit_ctx *ctx) +{ + if (IS_ENABLED(CONFIG_CFI_CLANG)) + emit(hash, ctx); +} + /* * Kernel addresses in the vmalloc space use at most 48 bits, and the * remaining bits are guaranteed to be 0x1. So we can compose the address @@ -476,7 +483,6 @@ static int build_prologue(struct jit_ctx *ctx, bool ebp= f_from_cbpf) const bool is_main_prog =3D !bpf_is_subprog(prog); const u8 fp =3D bpf2a64[BPF_REG_FP]; const u8 arena_vm_base =3D bpf2a64[ARENA_VM_START]; - const int idx0 =3D ctx->idx; int cur_offset; =20 /* @@ -502,6 +508,9 @@ static int build_prologue(struct jit_ctx *ctx, bool ebp= f_from_cbpf) * */ =20 + emit_kcfi(is_main_prog ? cfi_bpf_hash : cfi_bpf_subprog_hash, ctx); + const int idx0 =3D ctx->idx; + /* bpf function may be invoked by 3 instruction types: * 1. bl, attached via freplace to bpf prog via short jump * 2. br, attached via freplace to bpf prog via long jump @@ -2055,9 +2064,9 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog = *prog) jit_data->ro_header =3D ro_header; } =20 - prog->bpf_func =3D (void *)ctx.ro_image; + prog->bpf_func =3D (void *)ctx.ro_image + cfi_get_offset(); prog->jited =3D 1; - prog->jited_len =3D prog_size; + prog->jited_len =3D prog_size - cfi_get_offset(); =20 if (!prog->is_func || extra_pass) { int i; @@ -2431,6 +2440,12 @@ static int prepare_trampoline(struct jit_ctx *ctx, s= truct bpf_tramp_image *im, /* return address locates above FP */ retaddr_off =3D stack_size + 8; =20 + if (flags & BPF_TRAMP_F_INDIRECT) { + /* + * Indirect call for bpf_struct_ops + */ + emit_kcfi(cfi_get_func_hash(func_addr), ctx); + } /* bpf trampoline may be invoked by 3 instruction types: * 1. bl, attached to bpf prog or kernel function via short jump * 2. br, attached to bpf prog or kernel function via long jump @@ -2947,6 +2962,7 @@ void bpf_jit_free(struct bpf_prog *prog) sizeof(jit_data->header->size)); kfree(jit_data); } + prog->bpf_func -=3D cfi_get_offset(); hdr =3D bpf_jit_binary_pack_hdr(prog); bpf_jit_binary_pack_free(hdr, NULL); WARN_ON_ONCE(!bpf_prog_kallsyms_verify_off(prog)); --=20 2.50.0.727.gbf7dc18ff4-goog