From nobody Sat Feb 7 15:10:46 2026 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EA6528C2D0 for ; Tue, 13 May 2025 11:12:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747134735; cv=none; b=T02A3pVUmhnGDYpEOtOW0b2JldvN8j/PVosjcoXYvMb2opTvz646x2MGpJ4+FX8uoNBuw/Nw/6CGVlT2WnvhZYyaZ4RaUMZ9/pUooSL2Cyh4ZyTvxWIQ+qa/txgxjJSTMZpj0FvdORfTC7rVTRTwPVBmYgxWQ0TgwZhDgfe8uIY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747134735; c=relaxed/simple; bh=Zhxtw4SO7aK2WDdKFZdJ6jQZ39UBzSaBN1zdKJ8JqV0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=X3P7d96sNtv1FcffUhIwdZBRcfUT0PtdvcdjkG1WNyvCEFmOCMMvnUrP4e2TpMeUBZ0S3wpqpSKhpxUkSwpaVZJHzjD2LV4iVMhxxSjnHCWyRD4RxJn2uyZk8VkWI5QCrfgfoxcrp7lXMRXBTj2vyBwAR7EO7D0XAy9aA6/pBck= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=pwxmHKYH; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="pwxmHKYH" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-3a0b6cb5606so2451423f8f.0 for ; Tue, 13 May 2025 04:12:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747134732; x=1747739532; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=WPb+/2cCvFDpZsAPmT7NZHK5ljAWIgqQu8Z9jqyoi/w=; b=pwxmHKYHu/GXEotciPfuaD9Qo4S7Pm3lQDhMgo6eGEHwYHElQktFrMbU+n8/61Kqy6 glMrWgvUeyd2iaSaXeffLunnyVzbi/7nq8GSu4j6Z9waLNsE4OKZzmKJgCf5Ku2cgJmj xq8k7DGj4ZsP3U30nZC0kp9JZjnkUfivKZ+NThfs6JZngW9sezJQJIfhPbXfRa9yIvD/ 4KRbXv7sSL+gZfMP69GGF9b/gD63IGrWPhcwqL4rpy/BDUdA5aHM/rUlHQ5YqAaeYiTq M9hggQ++mRXAyEb5xRlQlGp1Rckik8w6U45QcrEwDuQrmW+FKpzu4udSxMd+ZR4x2vSv JHAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747134732; x=1747739532; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WPb+/2cCvFDpZsAPmT7NZHK5ljAWIgqQu8Z9jqyoi/w=; b=pN+CoBnrha8IfedNGn0amWTO0KSi2KMy1ymmJjSz86ao5f+c5kXCSWA1uLvIVRDMkn f0c8s6B36IYNSg9X8NukurFZ+/iudlNRd/AJ0zdVbZVIxQS6uFZoxj7CMTAaLCQCC5pm nJIYpNWBa19W8lLODyNNxTYhVrT0szlySlhR3sP4XtOZigYCPY1Jfx8BZNbZ66Bw8GCO cl/j9S7BQPN3AO13c/8w98Zze4kj51wiXehJIz9n4wIOVED0SoEsbKr5mKMN9U+gQget QR8r1JhLYJwGGGDb9vNfWzwNv0jkUPEIsbgv21BpcmYlDS2ea+sWcUe/gb50jBh+5PjH RV6w== X-Gm-Message-State: AOJu0YwsReha6TPU9OgoSEKX1KtwjmW0yw/8bfGO4sxQ761GRNVapSib jABXAnxmA/ni4BvJrh9mWw6GW8D/kW8z3+gW7tB0LUoKiqMzbvPqwKFEw6+S7+YXfqew7/kIWRP VmfeK1GFYGNlEdlTT20YuJKNMHjL5hm9pK6YhwS1rpxbWVy54YwGQQQEuOeM22mPX6S9eh0zGQR p95D9OqMBNXqBnhq43fK03B9hJrhDhZw== X-Google-Smtp-Source: AGHT+IEeQ0RtH/AvWRg0Hd+5L1HThkUD9VnsIM0XqHiZr7NQluCa4rbxX9PABd6/av4XKTxpIR7+T9nI X-Received: from wrgb4.prod.google.com ([2002:a05:6000:3c4:b0:39a:bfde:20c2]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:4205:b0:3a0:8c45:d30e with SMTP id ffacd0b85a97d-3a1f6469589mr12214481f8f.35.1747134732583; Tue, 13 May 2025 04:12:12 -0700 (PDT) Date: Tue, 13 May 2025 13:11:59 +0200 In-Reply-To: <20250513111157.717727-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250513111157.717727-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2286; i=ardb@kernel.org; h=from:subject; bh=Bo0IJENrodyFP+G2e/kdJdC192ghpizsoVViW26/V9I=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUNZ47+B22tTeQ0BL2MFnaNnmbk2v+7ae9NcNrhzx/R2z T1Ppco6SlkYxDgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwEROFzEydH8sVOEz8s/Mj3/o /XS/9VzWg1yvnl1l3nV2O8PcbYevmDMydBr7n3R9td0hzZ3RI/3G3JllHB+CZ7ZfnqNgtMOj/oo eEwA= X-Mailer: git-send-email 2.49.0.1045.g170613ef41-goog Message-ID: <20250513111157.717727-9-ardb+git@google.com> Subject: [RFC PATCH v2 1/6] x86/boot: Defer initialization of VM space related global variables From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The global pseudo-constants page_offset_base, vmalloc_base and vmemmap_base are not used extremely early during the boot, and cannot be used safely until after the KASLR memory randomization code in kernel_randomize_memory() executes, which may update their values. So there is no point in setting these variables extremely early, and it can wait until after the kernel itself is mapped and running from its permanent virtual mapping. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/startup/map_kernel.c | 3 --- arch/x86/kernel/head64.c | 9 ++++++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/x86/boot/startup/map_kernel.c b/arch/x86/boot/startup/map= _kernel.c index 099ae2559336..905e8734b5a3 100644 --- a/arch/x86/boot/startup/map_kernel.c +++ b/arch/x86/boot/startup/map_kernel.c @@ -29,9 +29,6 @@ static inline bool check_la57_support(void) __pgtable_l5_enabled =3D 1; pgdir_shift =3D 48; ptrs_per_p4d =3D 512; - page_offset_base =3D __PAGE_OFFSET_BASE_L5; - vmalloc_base =3D __VMALLOC_BASE_L5; - vmemmap_base =3D __VMEMMAP_BASE_L5; =20 return true; } diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 510fb41f55fc..14f7dda20954 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -62,13 +62,10 @@ EXPORT_SYMBOL(ptrs_per_p4d); #ifdef CONFIG_DYNAMIC_MEMORY_LAYOUT unsigned long page_offset_base __ro_after_init =3D __PAGE_OFFSET_BASE_L4; EXPORT_SYMBOL(page_offset_base); -SYM_PIC_ALIAS(page_offset_base); unsigned long vmalloc_base __ro_after_init =3D __VMALLOC_BASE_L4; EXPORT_SYMBOL(vmalloc_base); -SYM_PIC_ALIAS(vmalloc_base); unsigned long vmemmap_base __ro_after_init =3D __VMEMMAP_BASE_L4; EXPORT_SYMBOL(vmemmap_base); -SYM_PIC_ALIAS(vmemmap_base); #endif =20 /* Wipe all early page tables except for the kernel symbol map */ @@ -244,6 +241,12 @@ asmlinkage __visible void __init __noreturn x86_64_sta= rt_kernel(char * real_mode /* Kill off the identity-map trampoline */ reset_early_page_tables(); =20 + if (pgtable_l5_enabled()) { + page_offset_base =3D __PAGE_OFFSET_BASE_L5; + vmalloc_base =3D __VMALLOC_BASE_L5; + vmemmap_base =3D __VMEMMAP_BASE_L5; + } + clear_bss(); =20 /* --=20 2.49.0.1045.g170613ef41-goog From nobody Sat Feb 7 15:10:46 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2D58B28CF5F for ; Tue, 13 May 2025 11:12:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747134738; cv=none; b=AYNWDXLMr1nfh+sRkwoBiNco39Xl/rl9iwLW2pRshtIO/LMRpMLJrSdJVgzekhxzdFDbCY6UTWxx0WksDFsUieEyyNSD8ze6C+R9Ndy72cLh9T3ThNc9EuQTMI1VW1HtV0/TcwC17e32vIt/MHEm/xPJq66YqwDEZ1+Af6iD17Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747134738; c=relaxed/simple; bh=SiE9cxLDiw7N8srrflo7E3I2uPLPal/N84998cgDVx4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=kY0XIpiXBDsXxSznnJew+ou187jc/GCLWqnfEjYgx2uLSSL+Uv8kl/GSBeg3e26xbXC0oCPi3fyAXrju17QYuoi4fg0oeu2h3MfSdKZveMyZUhuffkkDAey2AXaysBSVHP60jjLeZGKG1JA+hA8KFAnRO+HbE91bLOXdPY3X5y4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=UmmHGgw1; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="UmmHGgw1" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-442d472cf7fso32541425e9.3 for ; Tue, 13 May 2025 04:12:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747134734; x=1747739534; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=bGXY26rkuYFN33Dk+vazAfhid/+q25E0zXYiCdmIXuo=; b=UmmHGgw1s8e1Oj9CyLWcoWTKWR8OwqsloV6EX6IfxRRtB/y8FX2w44tskeJmeVA7o+ u1/q06hzPPMax1NmwzTKkX0PG5LlVtz/3JBoNC8CdV6PziOcdObE+BUNcCMDmemCAFGX QPML6q/v8hEesWKeJrw0tfMam1HbA9Ps5ByHFQCIODpcgOOjCBi0f+7T58uktd5d0KIS fHyh/1M1ThjJjM4KwlpU0UEHpbBn4tAcAxI+ZojYd6LTMzaRbP0w2zLA8wmAsYvzV7gj i3S8UoPZBxLy+HlZzn0RNGXkJeiQ5Rv/9yZ6+4r953d/oQtpc1b6VduaMV76JHs6ZhMC yR1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747134734; x=1747739534; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=bGXY26rkuYFN33Dk+vazAfhid/+q25E0zXYiCdmIXuo=; b=u0m9m9SKfwMOyz7m0YOeom1eUIpzpLmL647EtPt45xbQbVZVzSakgwDJX64G9XlPwZ T2tJVzXzQFtVd0sJKhQYS+aobEeV9HtStW5fIvrmF/IfZJjbFvvKYQeFHdeFtAUwTWVw JGBMGsRaPLqj+0MKlM0XPAU+7WJhUSFT+Dmgynm57ls8hj6VUUntMDRMeoXel2L8czzR 2LvV9fOcsPk12F+RZWWkh0K6w2qzjOj0IaEIWhDKGbQF+7LT7wspi7wS/pPbUOUAhp28 7PeygTe33A0wiYHpz26hT54Xheo/knkKss44HIsnFPaRx8z7trWLifPVTESGzhJAaW4N U9hA== X-Gm-Message-State: AOJu0Yz2NDSy/E9BVNuLCsSakmDcnQHhJow5b7L5rRb6mQhHsw6+1/DY oLJyQcbr072YWFw0ubvfqfXjlgOT3yh4salSgovNEO4AtYY91aXdkqhYXct4ImqHmvJ85AiJYAM KDooe82jF4qw+1untFGr8F0F/uKj5By+fBrCSFkumjONAMU/rJaKzEK9eu0tdtQCkH917zo81i6 aD3Y1dg0+M0/LQW8cejGa/zRqk/ylyPw== X-Google-Smtp-Source: AGHT+IGKS18rlNuMSDx8foLA5/omTQ9ArqIDwz9wP8fDSpdTQarXd3VgQ9KmTdRfYBeY1a3vEY9F/xMM X-Received: from wmbdo8.prod.google.com ([2002:a05:600c:6808:b0:442:cd42:1f7]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:46d3:b0:43c:f470:7605 with SMTP id 5b1f17b1804b1-442d6d377f0mr169499755e9.12.1747134734613; Tue, 13 May 2025 04:12:14 -0700 (PDT) Date: Tue, 13 May 2025 13:12:00 +0200 In-Reply-To: <20250513111157.717727-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250513111157.717727-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=7295; i=ardb@kernel.org; h=from:subject; bh=CTJz8NdzmD/WIIGiFevnMSJGKvermr1seghTmPDFq64=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUNZk0HGovNuyNqi6UvWSzhfqfDsv3mJOfrRjB9ZBg/mG UwPkYzqKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABMJncTwP3it7OXk0soXPNFz TKpu/ti1P25nlJJdMneWQK7tzkpmbob/iZmrZ9XrqNWIM3DL9tzvu+raxzWVa/dLI2njsxO/rhX nAwA= X-Mailer: git-send-email 2.49.0.1045.g170613ef41-goog Message-ID: <20250513111157.717727-10-ardb+git@google.com> Subject: [RFC PATCH v2 2/6] x86/cpu: Use a new feature flag for 5 level paging From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Currently, the LA57 CPU feature flag is taken to mean two different things at once: - whether the CPU implements the LA57 extension, and is therefore capable of supporting 5 level paging; - whether 5 level paging is currently in use. This means the LA57 capability of the hardware is hidden when a LA57 capable CPU is forced to run with 4 levels of paging. It also means the the ordinary CPU capability detection code will happily set the LA57 capability and it needs to be cleared explicitly afterwards to avoid inconsistencies. Separate the two so that the CPU hardware capability can be identified unambigously in all cases. Signed-off-by: Ard Biesheuvel --- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/page_64.h | 2 +- arch/x86/include/asm/pgtable_64_types.h | 2 +- arch/x86/kernel/cpu/common.c | 16 ++-------------- arch/x86/kvm/x86.h | 4 ++-- drivers/iommu/amd/init.c | 4 ++-- drivers/iommu/intel/svm.c | 4 ++-- tools/testing/selftests/kvm/x86/set_sregs_test.c | 2 +- 8 files changed, 12 insertions(+), 23 deletions(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpuf= eatures.h index 6c2c152d8a67..13162cac8957 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -481,6 +481,7 @@ #define X86_FEATURE_AMD_HETEROGENEOUS_CORES (21*32 + 6) /* Heterogeneous C= ore Topology */ #define X86_FEATURE_AMD_WORKLOAD_CLASS (21*32 + 7) /* Workload Classificat= ion */ #define X86_FEATURE_PREFER_YMM (21*32 + 8) /* Avoid ZMM registers due to = downclocking */ +#define X86_FEATURE_5LEVEL_PAGING (21*32 + 9) /* Whether 5 levels of page = tables are in use */ =20 /* * BUG word(s) diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h index d3aab6f4e59a..acfa61ad0725 100644 --- a/arch/x86/include/asm/page_64.h +++ b/arch/x86/include/asm/page_64.h @@ -86,7 +86,7 @@ static __always_inline unsigned long task_size_max(void) unsigned long ret; =20 alternative_io("movq %[small],%0","movq %[large],%0", - X86_FEATURE_LA57, + X86_FEATURE_5LEVEL_PAGING, "=3Dr" (ret), [small] "i" ((1ul << 47)-PAGE_SIZE), [large] "i" ((1ul << 56)-PAGE_SIZE)); diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm= /pgtable_64_types.h index 5bb782d856f2..88dc719b7d37 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -34,7 +34,7 @@ static inline bool pgtable_l5_enabled(void) return __pgtable_l5_enabled; } #else -#define pgtable_l5_enabled() cpu_feature_enabled(X86_FEATURE_LA57) +#define pgtable_l5_enabled() cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) #endif /* USE_EARLY_PGTABLE_L5 */ =20 #else diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index f0f85482a73b..bbec5c4cd8ed 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1675,20 +1675,8 @@ static void __init early_identify_cpu(struct cpuinfo= _x86 *c) setup_clear_cpu_cap(X86_FEATURE_PCID); #endif =20 - /* - * Later in the boot process pgtable_l5_enabled() relies on - * cpu_feature_enabled(X86_FEATURE_LA57). If 5-level paging is not - * enabled by this point we need to clear the feature bit to avoid - * false-positives at the later stage. - * - * pgtable_l5_enabled() can be false here for several reasons: - * - 5-level paging is disabled compile-time; - * - it's 32-bit kernel; - * - machine doesn't support 5-level paging; - * - user specified 'no5lvl' in kernel command line. - */ - if (!pgtable_l5_enabled()) - setup_clear_cpu_cap(X86_FEATURE_LA57); + if (IS_ENABLED(CONFIG_X86_5LEVEL) && (native_read_cr4() & X86_CR4_LA57)) + setup_force_cpu_cap(X86_FEATURE_5LEVEL_PAGING); =20 detect_nopl(); } diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 9dc32a409076..d2c093f17ae5 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -243,7 +243,7 @@ static inline u8 vcpu_virt_addr_bits(struct kvm_vcpu *v= cpu) =20 static inline u8 max_host_virt_addr_bits(void) { - return kvm_cpu_cap_has(X86_FEATURE_LA57) ? 57 : 48; + return kvm_cpu_cap_has(X86_FEATURE_5LEVEL_PAGING) ? 57 : 48; } =20 /* @@ -603,7 +603,7 @@ static inline bool __kvm_is_valid_cr4(struct kvm_vcpu *= vcpu, unsigned long cr4) __reserved_bits |=3D X86_CR4_FSGSBASE; \ if (!__cpu_has(__c, X86_FEATURE_PKU)) \ __reserved_bits |=3D X86_CR4_PKE; \ - if (!__cpu_has(__c, X86_FEATURE_LA57)) \ + if (!__cpu_has(__c, X86_FEATURE_5LEVEL_PAGING)) \ __reserved_bits |=3D X86_CR4_LA57; \ if (!__cpu_has(__c, X86_FEATURE_UMIP)) \ __reserved_bits |=3D X86_CR4_UMIP; \ diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c index dd9e26b7b718..1d129969c4fd 100644 --- a/drivers/iommu/amd/init.c +++ b/drivers/iommu/amd/init.c @@ -3084,7 +3084,7 @@ static int __init early_amd_iommu_init(void) goto out; =20 /* 5 level guest page table */ - if (cpu_feature_enabled(X86_FEATURE_LA57) && + if (cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) && FIELD_GET(FEATURE_GATS, amd_iommu_efr) =3D=3D GUEST_PGTABLE_5_LEVEL) amd_iommu_gpt_level =3D PAGE_MODE_5_LEVEL; =20 @@ -3683,7 +3683,7 @@ __setup("ivrs_acpihid", parse_ivrs_acpihid); bool amd_iommu_pasid_supported(void) { /* CPU page table size should match IOMMU guest page table size */ - if (cpu_feature_enabled(X86_FEATURE_LA57) && + if (cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) && amd_iommu_gpt_level !=3D PAGE_MODE_5_LEVEL) return false; =20 diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index ba93123cb4eb..1f615e6d06ec 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -37,7 +37,7 @@ void intel_svm_check(struct intel_iommu *iommu) return; } =20 - if (cpu_feature_enabled(X86_FEATURE_LA57) && + if (cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) && !cap_fl5lp_support(iommu->cap)) { pr_err("%s SVM disabled, incompatible paging mode\n", iommu->name); @@ -165,7 +165,7 @@ static int intel_svm_set_dev_pasid(struct iommu_domain = *domain, return PTR_ERR(dev_pasid); =20 /* Setup the pasid table: */ - sflags =3D cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0; + sflags =3D cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) ? PASID_FLAG_FL= 5LP : 0; ret =3D __domain_setup_first_level(iommu, dev, pasid, FLPT_DEFAULT_DID, mm->pgd, sflags, old); diff --git a/tools/testing/selftests/kvm/x86/set_sregs_test.c b/tools/testi= ng/selftests/kvm/x86/set_sregs_test.c index f4095a3d1278..de78665fa675 100644 --- a/tools/testing/selftests/kvm/x86/set_sregs_test.c +++ b/tools/testing/selftests/kvm/x86/set_sregs_test.c @@ -52,7 +52,7 @@ static uint64_t calc_supported_cr4_feature_bits(void) =20 if (kvm_cpu_has(X86_FEATURE_UMIP)) cr4 |=3D X86_CR4_UMIP; - if (kvm_cpu_has(X86_FEATURE_LA57)) + if (kvm_cpu_has(X86_FEATURE_5LEVEL_PAGING)) cr4 |=3D X86_CR4_LA57; if (kvm_cpu_has(X86_FEATURE_VMX)) cr4 |=3D X86_CR4_VMXE; --=20 2.49.0.1045.g170613ef41-goog From nobody Sat Feb 7 15:10:46 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5A4A328D82E for ; Tue, 13 May 2025 11:12:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747134740; cv=none; b=p3/dGeesUlReP6qrGvNUBMGVAftmyP4JmAnt6TU2keXtYNZDK//ayPMclRXbBq+cEV2ekhR4zQB5W6JZZtqTv2W9xkiPRsW+BUdvKugB6CtpPs2M3oo1kEALtEZcozl6lYr95XfUeHvmEUsEykS/rexvqanoKQ+n1W2Yx0VHVOo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747134740; c=relaxed/simple; bh=xVxttg+Ki0H7qV3QXQdqHqVoJRGNW84rRIVdoWDdCoA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=cDQHFs2quOUz2Ej1lj3Dd9rJ2Wk2JpyHbrIAJsuQ4dvb3HtOWsMKztlrde89cHnao2Q2DqWCSb0WAjnfm17SK9/XLbNzARy36FsbcvxCQe2KAb/k/l4lqLuisFizryNiUDvIaFVFjSjpF7sYRKtBxtRDy59PSdrvKRJVcWuBpwk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=DrnORwE/; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="DrnORwE/" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-43d0830c3f7so35117725e9.2 for ; Tue, 13 May 2025 04:12:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747134736; x=1747739536; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=dZTpfF8cA7D0pCmiXbDGe34Zc8XEwsH5jH7PDUOh2wE=; b=DrnORwE/I7EyaWnxPwP51R7pEMOUgGJtij3HHB5P/tsEDrj7krsV0jQiwuiqw0JxOx p7gUx55Qgk9PumBLe+C4hyy26XVu5jQHO27YcVjURz+tOyworRMEZ0gxCe3T9H5iUhoi GGQPFn6FbbpoMnWp0xClxQt8fXYhJny/Edf9yTwKzbZKdBI5ESvDZBNBWA856avVTFk0 rcHQJLZO6TJvYcUC+qzl0Ls3PyrBaP0xaj5TkxyuEF4ybpkvIjl2B60X91+/KY1/axI4 F8rDnCcESPyNWekexLh7NsoyM8Ty+Ddh+Icoy7768UjLpG5rN2NKWfBHAdAPzfPqi1vf htqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747134736; x=1747739536; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dZTpfF8cA7D0pCmiXbDGe34Zc8XEwsH5jH7PDUOh2wE=; b=UlhcNlKM5pyUkmK7urJdeSZ9swpz15ed75KFh3zjKoZRmjPker35IgmgHY2N6o/Baw EEW46qqUz2Py+Abo5L9fcBNGZe6X9KrdEqhHbIIVrvhPeoZ1ifAZEDH4Is2oTG+mOZWj x6flyv1Wuk80T3h5actoNKQTtydYRf1noZ5/zPjmE3hcJ2ub/3cJpkTjZ6T5wECnED/U /X6UwRYGbrj3FkOE6CUo31Jeoqi6nKcgz6Ivqc+C+DyaDjhLp18NFtwqV3KzcpEtU6Oi FftdVDZAN0R8abW8leFE3DwHnZNUpqnYBNrUoUEtt5oA/tNMjctVx2HJebrvVeV9Wfyt alPg== X-Gm-Message-State: AOJu0Yz+tNC/KiY2M1pSiSi4YKxmRtmUcs4H6yDhRz5rfRFkslGaXJsk WfUc6jbTbjcY6YZrgGRD6jy/giIkl6VKU62BbhVHQIeGmwwvjQdsvIPLeuWslCzTmn4h70TDPRl twMBZ2BMlyMThWSqvYQ/enikDO7RDVsKQbDq53UwBJyE7hCzlXmqYbiBPpJnXwHPAZUsfNtx9BL yt2L5yy3RcwxEuYf060ulKjFEQB3pbvA== X-Google-Smtp-Source: AGHT+IFbjhW7iHTgI2oIXprmoB/IDk/7xkxLNv3b93TPgxieC7Ak5juYx7DpQTTS9RTcnBdGeYrvWr5B X-Received: from wmcn15.prod.google.com ([2002:a05:600c:c0cf:b0:43c:fe64:2de3]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:45d1:b0:43c:f63c:babb with SMTP id 5b1f17b1804b1-442d6d18a81mr110937645e9.1.1747134736552; Tue, 13 May 2025 04:12:16 -0700 (PDT) Date: Tue, 13 May 2025 13:12:01 +0200 In-Reply-To: <20250513111157.717727-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250513111157.717727-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2966; i=ardb@kernel.org; h=from:subject; bh=dwocxnQDWmEMf8E6vMAUpfYBmpwDUYGcnjgv0Zwk36w=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUNZk8ns9e0OhbzK1dfW2ct9aV7GaJLa4tsrZHS4NsiOm VH32/aOUhYGMQ4GWTFFFoHZf9/tPD1RqtZ5lizMHFYmkCEMXJwCMBHPOIZ/2i9exXQ9DD+itDL+ T8eZ7WwT74e+avNYueLArCnb7kUetWRkePWeM9WZeeHMywcKdCaUCMS0/vxaL8n7hHeZ45ed8b6 TGQE= X-Mailer: git-send-email 2.49.0.1045.g170613ef41-goog Message-ID: <20250513111157.717727-11-ardb+git@google.com> Subject: [RFC PATCH v2 3/6] x86/cpu: Allow caps to be set arbitrarily early From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel cpu_feature_enabled() uses a ternary alternative, where the late variant is based on code patching and the early variant accesses the capability field in boot_cpu_data directly. This allows cpu_feature_enabled() to be called quite early, but it still requires that the CPU feature detection code runs before being able to rely on the return value of cpu_feature_enabled(). This is a problem for the implementation of pgtable_l5_enabled(), which is based on cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING), and may be called extremely early. Currently, there is a hacky workaround where some source files that may execute before (but also after) CPU feature detection have a different version of pgtable_l5_enabled(), based on the USE_EARLY_PGTABLE_L5 preprocessor macro. Instead, let's make it possible to set CPU feature arbitrarily early, so that the X86_FEATURE_5LEVEL_PAGING capability can be set before even entering C code. This involves relying on static initialization of boot_cpu_data and the cpu_caps_set/cpu_caps_cleared arrays, so they all need to reside in .data. This ensures that they won't be cleared along with the rest of BSS. Note that forcing a capability involves setting it in both boot_cpu_data.x86_capability[] and cpu_caps_set[]. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/cpu/common.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index bbec5c4cd8ed..aaa6d9e51ef1 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -704,8 +704,8 @@ static const char *table_lookup_model(struct cpuinfo_x8= 6 *c) } =20 /* Aligned to unsigned long to avoid split lock in atomic bitmap ops */ -__u32 cpu_caps_cleared[NCAPINTS + NBUGINTS] __aligned(sizeof(unsigned long= )); -__u32 cpu_caps_set[NCAPINTS + NBUGINTS] __aligned(sizeof(unsigned long)); +__u32 __read_mostly cpu_caps_cleared[NCAPINTS + NBUGINTS] __aligned(sizeof= (unsigned long)); +__u32 __read_mostly cpu_caps_set[NCAPINTS + NBUGINTS] __aligned(sizeof(uns= igned long)); =20 #ifdef CONFIG_X86_32 /* The 32-bit entry code needs to find cpu_entry_area. */ @@ -1628,9 +1628,6 @@ static void __init cpu_parse_early_param(void) */ static void __init early_identify_cpu(struct cpuinfo_x86 *c) { - memset(&c->x86_capability, 0, sizeof(c->x86_capability)); - c->extended_cpuid_level =3D 0; - if (!have_cpuid_p()) identify_cpu_without_cpuid(c); =20 @@ -1842,7 +1839,8 @@ static void identify_cpu(struct cpuinfo_x86 *c) c->x86_virt_bits =3D 32; #endif c->x86_cache_alignment =3D c->x86_clflush_size; - memset(&c->x86_capability, 0, sizeof(c->x86_capability)); + if (c !=3D &boot_cpu_data) + memset(&c->x86_capability, 0, sizeof(c->x86_capability)); #ifdef CONFIG_X86_VMX_FEATURE_NAMES memset(&c->vmx_capability, 0, sizeof(c->vmx_capability)); #endif --=20 2.49.0.1045.g170613ef41-goog From nobody Sat Feb 7 15:10:46 2026 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2DD1B28C87E for ; Tue, 13 May 2025 11:12:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747134742; cv=none; b=pbxqlU3kVxEPLeZlv6TcbQcZZO2tN+hHRBNGoBqtKvYE4v+PKM4YfYI74ndQxpgBb2jR+etg81FOI4/QwJhzxsz5B9j3Soj7ed+WkVyKnWFQ18ebGjkZM20YtDmxEwOb5kXZs/G4Lc5UDHIfaAgTRIZ0gXBTxQE0yQcWPLUqUSw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747134742; c=relaxed/simple; bh=sbJ8Fw1r90yZtfHFFE9JF4oqF4T7ODzQ/pSatIIn69I=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sr7IBRgfKIY5oElDPPOG1nbM2YVDeT0IEUF4P/1hBTrjBS596M7Ia24GYCgv/fNGErfauVdf/XKa1Gtj0LzpkEAUftfUSc7YmkMGq8gKY4uyly4zkTui+DjTVjlsicbU/sbB71sUmBXqtmT9jWlocvJOcEz6UzFm/CdGRlvcoWY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=P4SklP7I; arc=none smtp.client-ip=209.85.221.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="P4SklP7I" Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-3a1f9ddf04bso2301442f8f.3 for ; Tue, 13 May 2025 04:12:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747134738; x=1747739538; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=NrcABwo7ZQTn+m5mWd/OtSEyPzIuybBaQHAplVXjEhE=; b=P4SklP7IkMjj8zSxKS/J7lQsSGX1d/QUKzNR/d1O8ZaO0gYRPjtvWBT4WvqoEkUZEU QKFnQKPeObZoDrq+hvvgmOShkNuBnVcP2ap4/IuGgEt/BB3SYINNAylF6Mf9opLskKxA 9DfqfBZCMFJpHgpMqR08h8yAovrc9MbqPs1F8682jXlWyAAPWKK5p71Xc7Ry4N0ufkHk Yk+8uhTipWPuE79mZvi7ENUBGj8AWbYFzil5j8ebiq9Yq9V4QoOPDZ5rUJgIy5IO3bzQ ky7n2/SldLljJdrZZlattK+EJpRbU+w2ir202YpoeKuZ0DkHmPb8vctWCXdEDWJTdW0o heYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747134738; x=1747739538; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NrcABwo7ZQTn+m5mWd/OtSEyPzIuybBaQHAplVXjEhE=; b=vt+spq2yvtJRgyOQQm+cb17edRvpdbWBOtgusMZ5S1wjQl1y62jH/06tn7X3zWcVbC zoQfdR/wngeRu48T0xmhuiuRi8SMJdoF3+BTXYK/vefsxN4+akSay3lS7n7OgmGC0yue hpqV493Y0Jjnrj7eK+mUhmLM4FCT7i0e/PhvhL9lfQTRHFNq7fkL06Uih3S6OOI65GWw +PoiNf40ZiMmNYnaxOwycidKY1NjeJjmO1HRP5/P69ZPNeFwL1AVIVcokB4dWDOMJ5tZ U6cikMVFKtQWtx05GYagkh/r8gafp7oqoPj+5GX4hFBG6Dryu6rhU36VVq+zlOa+318u lr1w== X-Gm-Message-State: AOJu0YyKH/tvI36bO+Ogcyuba5RZcfPXg8n9DIsqsuANta9UeVmwycUi FiB0QRV5p4LoC0eyyZx35fvjaV8d2qzR7g/cT58juNA5bluBdZovzUpFJiMX+m8CYv9P0z5RRjY 3s0M2flZOjHqQcMOzjfA645Ipirizf6BL79BDJN6dNQZW20zPBwZpMHM3pNdyiUsPDBHxOKnVgO uTq3OX+2KOGQJ/H/n/PKWBYazo39Y8YA== X-Google-Smtp-Source: AGHT+IHI5HQZ+9PFF0vfq7SzZfXG0glKF0P9kl036c9kIW6+c1GIQEQf573tAmGCBebB5nf93P3UElvt X-Received: from wmbhj9.prod.google.com ([2002:a05:600c:5289:b0:43b:bd03:5d2]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a5d:5f4a:0:b0:3a0:b563:2451 with SMTP id ffacd0b85a97d-3a1f64276b1mr12064732f8f.5.1747134738590; Tue, 13 May 2025 04:12:18 -0700 (PDT) Date: Tue, 13 May 2025 13:12:02 +0200 In-Reply-To: <20250513111157.717727-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250513111157.717727-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=4607; i=ardb@kernel.org; h=from:subject; bh=9j53WpWB7md8IeK5be3OARrIP92+C+LWx2Fx91Qse0c=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUNZk5nvXoDgNGEO3hti0/iXqMlP5TleEMI22/p8Ade2E GZmS42OUhYGMQ4GWTFFFoHZf9/tPD1RqtZ5lizMHFYmkCEMXJwCMJF9BYwMW9YsrMj/zziXvbmz K27Nk+ocS6dNkj05j+bIst96dzw7jeF/wk3dA8ZS+gwBu3S+PrvgOKlogmr6L+uMr0uED8YFBZR xAgA= X-Mailer: git-send-email 2.49.0.1045.g170613ef41-goog Message-ID: <20250513111157.717727-12-ardb+git@google.com> Subject: [RFC PATCH v2 4/6] x86/boot: Set 5-level paging CPU cap before entering C code From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel In order for pgtable_l5_enabled() to be reliable wherever it is used and however early, set the associated CPU capability from asm code before entering the startup C code. Signed-off-by: Ard Biesheuvel --- arch/x86/include/asm/cpufeature.h | 12 +++++++++--- arch/x86/kernel/asm-offsets.c | 8 ++++++++ arch/x86/kernel/asm-offsets_32.c | 9 --------- arch/x86/kernel/cpu/common.c | 3 --- arch/x86/kernel/head_64.S | 15 +++++++++++++++ 5 files changed, 32 insertions(+), 15 deletions(-) diff --git a/arch/x86/include/asm/cpufeature.h b/arch/x86/include/asm/cpufe= ature.h index 893cbca37fe9..1b5de40e7bf7 100644 --- a/arch/x86/include/asm/cpufeature.h +++ b/arch/x86/include/asm/cpufeature.h @@ -2,10 +2,10 @@ #ifndef _ASM_X86_CPUFEATURE_H #define _ASM_X86_CPUFEATURE_H =20 +#ifdef __KERNEL__ +#ifndef __ASSEMBLER__ #include =20 -#if defined(__KERNEL__) && !defined(__ASSEMBLER__) - #include #include #include @@ -137,5 +137,11 @@ static __always_inline bool _static_cpu_has(u16 bit) #define CPU_FEATURE_TYPEVAL boot_cpu_data.x86_vendor, boot_cpu_data.x86, \ boot_cpu_data.x86_model =20 -#endif /* defined(__KERNEL__) && !defined(__ASSEMBLER__) */ +#else /* !defined(__ASSEMBLER__) */ + .macro setup_force_cpu_cap, cap:req + btsl $\cap % 32, boot_cpu_data+CPUINFO_x86_capability+4*(\cap / 32)(%rip) + btsl $\cap % 32, cpu_caps_set+4*(\cap / 32)(%rip) + .endm +#endif /* !defined(__ASSEMBLER__) */ +#endif /* defined(__KERNEL__) */ #endif /* _ASM_X86_CPUFEATURE_H */ diff --git a/arch/x86/kernel/asm-offsets.c b/arch/x86/kernel/asm-offsets.c index ad4ea6fb3b6c..6259b474073b 100644 --- a/arch/x86/kernel/asm-offsets.c +++ b/arch/x86/kernel/asm-offsets.c @@ -33,6 +33,14 @@ =20 static void __used common(void) { + OFFSET(CPUINFO_x86, cpuinfo_x86, x86); + OFFSET(CPUINFO_x86_vendor, cpuinfo_x86, x86_vendor); + OFFSET(CPUINFO_x86_model, cpuinfo_x86, x86_model); + OFFSET(CPUINFO_x86_stepping, cpuinfo_x86, x86_stepping); + OFFSET(CPUINFO_cpuid_level, cpuinfo_x86, cpuid_level); + OFFSET(CPUINFO_x86_capability, cpuinfo_x86, x86_capability); + OFFSET(CPUINFO_x86_vendor_id, cpuinfo_x86, x86_vendor_id); + BLANK(); OFFSET(TASK_threadsp, task_struct, thread.sp); #ifdef CONFIG_STACKPROTECTOR diff --git a/arch/x86/kernel/asm-offsets_32.c b/arch/x86/kernel/asm-offsets= _32.c index 2b411cd00a4e..e0a292db97b2 100644 --- a/arch/x86/kernel/asm-offsets_32.c +++ b/arch/x86/kernel/asm-offsets_32.c @@ -12,15 +12,6 @@ void foo(void); =20 void foo(void) { - OFFSET(CPUINFO_x86, cpuinfo_x86, x86); - OFFSET(CPUINFO_x86_vendor, cpuinfo_x86, x86_vendor); - OFFSET(CPUINFO_x86_model, cpuinfo_x86, x86_model); - OFFSET(CPUINFO_x86_stepping, cpuinfo_x86, x86_stepping); - OFFSET(CPUINFO_cpuid_level, cpuinfo_x86, cpuid_level); - OFFSET(CPUINFO_x86_capability, cpuinfo_x86, x86_capability); - OFFSET(CPUINFO_x86_vendor_id, cpuinfo_x86, x86_vendor_id); - BLANK(); - OFFSET(PT_EBX, pt_regs, bx); OFFSET(PT_ECX, pt_regs, cx); OFFSET(PT_EDX, pt_regs, dx); diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index aaa6d9e51ef1..ea49322ba151 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1672,9 +1672,6 @@ static void __init early_identify_cpu(struct cpuinfo_= x86 *c) setup_clear_cpu_cap(X86_FEATURE_PCID); #endif =20 - if (IS_ENABLED(CONFIG_X86_5LEVEL) && (native_read_cr4() & X86_CR4_LA57)) - setup_force_cpu_cap(X86_FEATURE_5LEVEL_PAGING); - detect_nopl(); } =20 diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 069420853304..b4742942bece 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -27,6 +27,7 @@ #include #include #include +#include =20 /* * We are not able to switch in one step to the final KERNEL ADDRESS SPACE @@ -58,6 +59,20 @@ SYM_CODE_START_NOALIGN(startup_64) */ mov %rsi, %r15 =20 +#ifdef CONFIG_X86_5LEVEL + /* + * Set the X86_FEATURE_5LEVEL_PAGING capability before calling into the + * C code, so that it is guaranteed to have a consistent view of any + * global pseudo-constants that are derived from pgtable_l5_enabled(). + */ + mov %cr4, %rax + btl $X86_CR4_LA57_BIT, %eax + jnc 0f + + setup_force_cpu_cap X86_FEATURE_5LEVEL_PAGING +0: +#endif + /* Set up the stack for verify_cpu() */ leaq __top_init_kernel_stack(%rip), %rsp =20 --=20 2.49.0.1045.g170613ef41-goog From nobody Sat Feb 7 15:10:46 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2DD4428D8DF for ; Tue, 13 May 2025 11:12:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747134744; cv=none; b=emlSqJM8aECAh0q2PDBuxIUX5CwIyxuPf0BsJZDJoAapQxI4ZIfNyFBse8OitaFw15U9WKkKvpiZluU2OqF8pqHkJEFCGHOYgHw1csVEcLsn6KG3vpqLAD2Xw7P+6jmBA4MdNbqo9kRIai6HhFcTMNLtLCpbaeWmEYZU2SDgACs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747134744; c=relaxed/simple; bh=/otbjljyOWw0mpgcPxCo5xMQ7Y+NFUvSH6e6PfiY9Q8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=S/wculW5BhcQ+FbDuZxHIQCF7xdUnFLbK11TGl3VvJ2yao/osanKqsgxjtlFYgaS0x4rM10puChngUFw2fAIEWGIwjiCXGxQFGOfXb6qPPNbuFYhBFEu/JeaiPqvlZPtz4kfx4+4xVYEniE3ol2AF76tAf+72bUYKKRkfj4TZr8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=0GV0gaRh; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="0GV0gaRh" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-43d0a037f97so28475595e9.2 for ; Tue, 13 May 2025 04:12:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747134740; x=1747739540; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=SAYpIXNnR+AmlE/t9wgpjF89DkEAXCG+Ztf2OXxlSPs=; b=0GV0gaRhXv2V8/aL7NTy8I4YzmPPDazV9arzrjixVF07ExgKEW9pDQd385VR0PIZXP DzbXNy5OLcgABjbsCZoKbI8ztix9YByXuOw4SyTmmL6nPNF0h344Dni91xFPqYIJw77Q FCf+OQAqdzO6Da5Et7V2mLHM+Qai25PGcFKbUCkSsDUTTLfm6LAIyihSMlByTjsaRbgg P56N3cszfKjVn7xSV29468e3X98rEtnWFaEqFAKCAtxiT7NhV6GVxdTSblKl3OwjDB93 VXth148NE3OkrzqmzTng6pR3iRROjyqPXtK8hPlXxEh4J0SNOoJm9x8GaJ/WhgeEsG+P kBLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747134740; x=1747739540; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=SAYpIXNnR+AmlE/t9wgpjF89DkEAXCG+Ztf2OXxlSPs=; b=V4FIdkjYsjSNBuhnClgRs9RGhwEJzI+nl8MjQLdFKAfoGUKpLDQcU+eoSbBASHe3r1 DWMG+jFXyyR2bk4URK1AInCxUm1v6YIoSDEYiSNnXQ+WNFxj4QpCKek1tKeMI7ov75XI vpKBTsP5f4hBT86xpL6dXBwZ5bFWsuRM+A0efFKZYbN2DiXMXTJ8ngXnhmJFmZgp7jiN MDAxxz8PLO7p6j8Vv6CSj8wOcXYzd5CUbeuf+avPdF11cuBnA2wwSt6NTtQO+iHUbhpH kkpH1vcjy95Y3zikNRVp0SuixLdBIocOyIjhjZFZ570dCA+1LlH1R4duYS9771Vaij05 caIw== X-Gm-Message-State: AOJu0YyVlNN71J5Dm+hyOJR+g2U9CIkT6eCWsKrCNBAZrq9QyjKtk0yh RZIIRsj5M0WmaJ4PJHhGSRvLsZdzgRxnao88rA5n4Tk5VwPzdhRJ8tgiivpjPJQaqQlvDslFhNN QWuLDie/CcpHSrW0GVNohJVkmYFoniagyRBqAEr58a1YSRKLl8dGbwySuIFrH01O/Xswm5qn/0k kEqrpaB+19+q+g7xc973/9wh60dqHpRw== X-Google-Smtp-Source: AGHT+IFt5PSCcOC0+ikPBQdDo1XaZ3CAUrpM1xnhokHORtbwjon7haTdiVems+KqyFhVBkO1UYlhcxNf X-Received: from wmbei9.prod.google.com ([2002:a05:600c:3f09:b0:441:aaa8:fb65]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3586:b0:43c:f85d:1245 with SMTP id 5b1f17b1804b1-442d9cacc19mr143906065e9.17.1747134740472; Tue, 13 May 2025 04:12:20 -0700 (PDT) Date: Tue, 13 May 2025 13:12:03 +0200 In-Reply-To: <20250513111157.717727-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250513111157.717727-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=5778; i=ardb@kernel.org; h=from:subject; bh=xwv05gUBGQB+6b5sVJVLL3t/PSiOFjAnhqzD0OrgXww=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUNZkyU2h5VtiW7N5EhHjr++h2b7n2aatCu5/+1qkdz3W Rb2WUEdpSwMYhwMsmKKLAKz/77beXqiVK3zLFmYOaxMIEMYuDgFYCLvGxkZ/sy686ZVNp/h7hNN 7+xtKT+nSwnsOWu+5//KOLFNp3tKShj+cPT2G2tWvV8qZyuaslhL9lh3vlKdo/LJvd9eHTGK2vG OFQA= X-Mailer: git-send-email 2.49.0.1045.g170613ef41-goog Message-ID: <20250513111157.717727-13-ardb+git@google.com> Subject: [RFC PATCH v2 5/6] x86/boot: Drop the early variant of pgtable_l5_enabled() From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Now that cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) is guaranteed to produce the correct value even during early boot, there is no longer a need for an early variant and so it can be dropped. For the decompressor, fall back to testing the CR4.LA57 control register bit directly. Note that this removes the need to disable KASAN temporarily while applying alternatives, given that any constant or VA space dimension derived from pgtable_l5_enabled() will now always produce a consistent value. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/misc.h | 7 ++++--- arch/x86/boot/startup/sme.c | 9 --------- arch/x86/include/asm/pgtable_64_types.h | 14 ++------------ arch/x86/kernel/alternative.c | 12 ------------ arch/x86/kernel/cpu/common.c | 2 -- arch/x86/kernel/head64.c | 3 --- arch/x86/mm/kasan_init_64.c | 3 --- 7 files changed, 6 insertions(+), 44 deletions(-) diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/mis= c.h index db1048621ea2..72b830b8a69c 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -16,9 +16,6 @@ =20 #define __NO_FORTIFY =20 -/* cpu_feature_enabled() cannot be used this early */ -#define USE_EARLY_PGTABLE_L5 - /* * Boot stub deals with identity mappings, physical and virtual addresses = are * the same, so override these defines. @@ -28,6 +25,10 @@ #define __pa(x) ((unsigned long)(x)) #define __va(x) ((void *)((unsigned long)(x))) =20 +#ifdef CONFIG_X86_5LEVEL +#define pgtable_l5_enabled() (native_read_cr4() & X86_CR4_LA57) +#endif + #include #include #include diff --git a/arch/x86/boot/startup/sme.c b/arch/x86/boot/startup/sme.c index 753cd2094080..c791f6b8a92f 100644 --- a/arch/x86/boot/startup/sme.c +++ b/arch/x86/boot/startup/sme.c @@ -25,15 +25,6 @@ #undef CONFIG_PARAVIRT_XXL #undef CONFIG_PARAVIRT_SPINLOCKS =20 -/* - * This code runs before CPU feature bits are set. By default, the - * pgtable_l5_enabled() function uses bit X86_FEATURE_LA57 to determine if - * 5-level paging is active, so that won't work here. USE_EARLY_PGTABLE_L5 - * is provided to handle this situation and, instead, use a variable that - * has been set by the early boot code. - */ -#define USE_EARLY_PGTABLE_L5 - #include #include #include diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm= /pgtable_64_types.h index 88dc719b7d37..83cd6c4b9a3f 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -24,19 +24,9 @@ typedef struct { pmdval_t pmd; } pmd_t; extern unsigned int __pgtable_l5_enabled; =20 #ifdef CONFIG_X86_5LEVEL -#ifdef USE_EARLY_PGTABLE_L5 -/* - * cpu_feature_enabled() is not available in early boot code. - * Use variable instead. - */ -static inline bool pgtable_l5_enabled(void) -{ - return __pgtable_l5_enabled; -} -#else +#ifndef pgtable_l5_enabled #define pgtable_l5_enabled() cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) -#endif /* USE_EARLY_PGTABLE_L5 */ - +#endif #else #define pgtable_l5_enabled() 0 #endif /* CONFIG_X86_5LEVEL */ diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index bf82c6f7d690..f4a8b81aac43 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -456,16 +456,6 @@ void __init_or_module noinline apply_alternatives(stru= ct alt_instr *start, =20 DPRINTK(ALT, "alt table %px, -> %px", start, end); =20 - /* - * In the case CONFIG_X86_5LEVEL=3Dy, KASAN_SHADOW_START is defined using - * cpu_feature_enabled(X86_FEATURE_LA57) and is therefore patched here. - * During the process, KASAN becomes confused seeing partial LA57 - * conversion and triggers a false-positive out-of-bound report. - * - * Disable KASAN until the patching is complete. - */ - kasan_disable_current(); - /* * The scan order should be from start to end. A later scanned * alternative code can overwrite previously scanned alternative code. @@ -533,8 +523,6 @@ void __init_or_module noinline apply_alternatives(struc= t alt_instr *start, =20 text_poke_early(instr, insn_buff, insn_buff_sz); } - - kasan_enable_current(); } =20 static inline bool is_jcc32(struct insn *insn) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index ea49322ba151..e1f8a7de07cc 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1,6 +1,4 @@ // SPDX-License-Identifier: GPL-2.0-only -/* cpu_feature_enabled() cannot be used this early */ -#define USE_EARLY_PGTABLE_L5 =20 #include #include diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 14f7dda20954..84b5df539a94 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -5,9 +5,6 @@ * Copyright (C) 2000 Andrea Arcangeli SuSE */ =20 -/* cpu_feature_enabled() cannot be used this early */ -#define USE_EARLY_PGTABLE_L5 - #include #include #include diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 0539efd0d216..7c4fafbd52cc 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -1,9 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 #define pr_fmt(fmt) "kasan: " fmt =20 -/* cpu_feature_enabled() cannot be used this early */ -#define USE_EARLY_PGTABLE_L5 - #include #include #include --=20 2.49.0.1045.g170613ef41-goog From nobody Sat Feb 7 15:10:46 2026 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1307628DB4C for ; Tue, 13 May 2025 11:12:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747134745; cv=none; b=rgYjJGz8f/9gcy6l75PCE3xe2tM2iOy+nrydgFjISEtUnAqSs53gFgSFbk3im2Ztew/TgcU+d8Y8+F9htGPXHboDNxG9mzyuKDIUAJOUUO/n7u77h+BCh1SocJPCnP04iWcqOjHglmbsIEhNOfW48NKlWVUrkJ4JFVQ5ErWY/Sw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747134745; c=relaxed/simple; bh=VZ0ZLmOePVFOuTmwTCqhvNytrOl8syhg+VWczGqwaVk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hVi324HqnfKnAlcYjDDf5cUYCyFVtQERUfEdozlu8mPx69qmR1ykcTT7fxIO1ekUcvviiuWjAtXg0OD0j8CnHqjsK/++7lEkdfIKWKUumJ9QO9EOHC0vYXkC1q+SdBhBPMPfgIJQzW1GaN+nP15Gc9a77Uf90BM+JmJpGODBFFg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=GiDcDAea; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="GiDcDAea" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-3a0db717b55so2828226f8f.3 for ; Tue, 13 May 2025 04:12:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747134742; x=1747739542; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=mzGj7/7HNCkOX8Jy6N1ShBJ0VGLtjdGMeKMS0UkPr2U=; b=GiDcDAeajXCwTYtHjPackOjbUzf2TxNoUtJ5EbQ/oS0Z8vOElAFthCc0wzvUlmA72V d9/+7jvQPY2j0lk4jo3soh6tzc4y5UK0LswcbveJHviI5k+Sp0lrOZ4dIfsQqQCI2Vu5 iOPPgLkHZz9of/y/jz0uwLyBEot3GLpOURW5008FobzXwnBCa+ZDVKOx4S0hx2b+9+sF BIfnWGh3oZpMvPQsX4ri5myKRi5dKhgvGFPTiZsRT9I2feXNa/XCh1HtiMNV2mljBzEV bs0KBdu+aF7pKWAev34WwJUupOt5K9lmL5vv3+rldGNovzh7Gufo+CjUbbHj5E1b6KrB 1brg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747134742; x=1747739542; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mzGj7/7HNCkOX8Jy6N1ShBJ0VGLtjdGMeKMS0UkPr2U=; b=GJ4w5dqEcsS7V02dkgmDfDx/lZbIfXCYdh1WxunSzwguzqOXWxXONbwvxPg1LHEeGI N4YjmegccM0GwpVJXRuH8ozyJkbMZjUxZz7ogapEfMRhAnKpFSfFNuOuPc/D2DTd4Xs4 SR3SxvUtqI3fG7QI36/1kPBIGEPjpKR6fi137jGvoXi7ZViE+Dd9/f4q05QT6aUXMMeK ZWOrOdGKpx/7IsX2rhp6ne93lx+X5MUWLxYHUYINNLSGsORAGLBS4iZQLRHDS+7GPDV3 P9Fn7R7oWLgkk/Z+bwV36FvNJm9N0gQdKlDiDhcnbJQE4w4qIE73no/u/cXNl7c+oet2 ipag== X-Gm-Message-State: AOJu0YzgZ8D+wfBxIQudMtblBA4jrHKSXuxq6YR3WlTbnJAIdNYRxLXI rIYpUkXRar+fPwWJu+f+yX/pF5zLNWKYTk6DaeEh7ipfZeLYgvgr4l3N4KGAUTsb38CpWHxqQup TRFDZXXqIyryClMXzIx3sPqHRzmA1SfO9AZbRBnDznD0LmSz/oEdA/tvrT/R5kngLMmhDlITwln mGBAh4D6WEHGb9tJaR2bnrA/ySR8ahlg== X-Google-Smtp-Source: AGHT+IESX49CIEDe9BU8khZDCxb5erpT7tCrhSDtiDit1VRdndDnzur7Lk8/6OehzO6FEl3BXIpR/H1Y X-Received: from wrgb2.prod.google.com ([2002:a05:6000:3c2:b0:3a1:a98e:dc0b]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a5d:5f51:0:b0:3a0:83b1:b3c1 with SMTP id ffacd0b85a97d-3a1f6432c26mr12712980f8f.15.1747134742479; Tue, 13 May 2025 04:12:22 -0700 (PDT) Date: Tue, 13 May 2025 13:12:04 +0200 In-Reply-To: <20250513111157.717727-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250513111157.717727-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=5646; i=ardb@kernel.org; h=from:subject; bh=p3CnmI29Ai+wS7jnpKGLR8D1i2u4QBwZebDaCkr/b04=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUNZk5X346ddMaoXRM6vS7XcuS0q8NKDxSzLJd9cTdI52 Lu75O3+jlIWBjEOBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjARCW6Gf/bhe5tuZHbJ/lhs /IJp6zP2j8tvlEx64zMtJfeNXdT2klWMDMsaVR+c5Dqhb9e/t+D66dTUPYoKE7+ZRzQLKD1e8jT biAUA X-Mailer: git-send-email 2.49.0.1045.g170613ef41-goog Message-ID: <20250513111157.717727-14-ardb+git@google.com> Subject: [RFC PATCH v2 6/6] x86/boot: Drop 5-level paging related variables and early updates From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The variable __pgtable_l5_enabled is no longer used so it can be dropped. Along with it, drop ptrs_per_p4d and pgdir_shift, and replace any references to those with expressions based on pgtable_l5_enabled(). This ensures that all observers see values that are mutually consistent. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/misc.h | 1 - arch/x86/boot/compressed/pgtable_64.c | 12 ----------- arch/x86/boot/startup/map_kernel.c | 21 +------------------- arch/x86/include/asm/pgtable_64_types.h | 9 ++------- arch/x86/kernel/head64.c | 8 -------- 5 files changed, 3 insertions(+), 48 deletions(-) diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/mis= c.h index 72b830b8a69c..3d5c6322def8 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -190,7 +190,6 @@ static inline int count_immovable_mem_regions(void) { r= eturn 0; } #endif =20 /* ident_map_64.c */ -extern unsigned int __pgtable_l5_enabled, pgdir_shift, ptrs_per_p4d; extern void kernel_add_identity_map(unsigned long start, unsigned long end= ); =20 /* Used by PAGE_KERN* macros: */ diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compress= ed/pgtable_64.c index 5a6c7a190e5b..591d28f2feb6 100644 --- a/arch/x86/boot/compressed/pgtable_64.c +++ b/arch/x86/boot/compressed/pgtable_64.c @@ -10,13 +10,6 @@ #define BIOS_START_MIN 0x20000U /* 128K, less than this is insane */ #define BIOS_START_MAX 0x9f000U /* 640K, absolute maximum */ =20 -#ifdef CONFIG_X86_5LEVEL -/* __pgtable_l5_enabled needs to be in .data to avoid being cleared along = with .bss */ -unsigned int __section(".data") __pgtable_l5_enabled; -unsigned int __section(".data") pgdir_shift =3D 39; -unsigned int __section(".data") ptrs_per_p4d =3D 1; -#endif - /* Buffer to preserve trampoline memory */ static char trampoline_save[TRAMPOLINE_32BIT_SIZE]; =20 @@ -127,11 +120,6 @@ asmlinkage void configure_5level_paging(struct boot_pa= rams *bp, void *pgtable) native_cpuid_eax(0) >=3D 7 && (native_cpuid_ecx(7) & (1 << (X86_FEATURE_LA57 & 31)))) { l5_required =3D true; - - /* Initialize variables for 5-level paging */ - __pgtable_l5_enabled =3D 1; - pgdir_shift =3D 48; - ptrs_per_p4d =3D 512; } =20 /* diff --git a/arch/x86/boot/startup/map_kernel.c b/arch/x86/boot/startup/map= _kernel.c index 905e8734b5a3..056de4766006 100644 --- a/arch/x86/boot/startup/map_kernel.c +++ b/arch/x86/boot/startup/map_kernel.c @@ -14,25 +14,6 @@ extern pmd_t early_dynamic_pgts[EARLY_DYNAMIC_PAGE_TABLES][PTRS_PER_PMD]; extern unsigned int next_early_pgt; =20 -static inline bool check_la57_support(void) -{ - if (!IS_ENABLED(CONFIG_X86_5LEVEL)) - return false; - - /* - * 5-level paging is detected and enabled at kernel decompression - * stage. Only check if it has been enabled there. - */ - if (!(native_read_cr4() & X86_CR4_LA57)) - return false; - - __pgtable_l5_enabled =3D 1; - pgdir_shift =3D 48; - ptrs_per_p4d =3D 512; - - return true; -} - static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdval_t *pmd, unsigned long p2v_offset) @@ -102,7 +83,7 @@ unsigned long __head __startup_64(unsigned long p2v_offs= et, bool la57; int i; =20 - la57 =3D check_la57_support(); + la57 =3D pgtable_l5_enabled(); =20 /* Is the address too large? */ if (physaddr >> MAX_PHYSMEM_BITS) diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm= /pgtable_64_types.h index 83cd6c4b9a3f..26deb6831235 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -21,8 +21,6 @@ typedef unsigned long pgprotval_t; typedef struct { pteval_t pte; } pte_t; typedef struct { pmdval_t pmd; } pmd_t; =20 -extern unsigned int __pgtable_l5_enabled; - #ifdef CONFIG_X86_5LEVEL #ifndef pgtable_l5_enabled #define pgtable_l5_enabled() cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) @@ -31,9 +29,6 @@ extern unsigned int __pgtable_l5_enabled; #define pgtable_l5_enabled() 0 #endif /* CONFIG_X86_5LEVEL */ =20 -extern unsigned int pgdir_shift; -extern unsigned int ptrs_per_p4d; - #endif /* !__ASSEMBLER__ */ =20 #define SHARED_KERNEL_PMD 0 @@ -43,7 +38,7 @@ extern unsigned int ptrs_per_p4d; /* * PGDIR_SHIFT determines what a top-level page table entry can map */ -#define PGDIR_SHIFT pgdir_shift +#define PGDIR_SHIFT (pgtable_l5_enabled() ? 48 : 39) #define PTRS_PER_PGD 512 =20 /* @@ -51,7 +46,7 @@ extern unsigned int ptrs_per_p4d; */ #define P4D_SHIFT 39 #define MAX_PTRS_PER_P4D 512 -#define PTRS_PER_P4D ptrs_per_p4d +#define PTRS_PER_P4D (pgtable_l5_enabled() ? MAX_PTRS_PER_P4D : 1) #define P4D_SIZE (_AC(1, UL) << P4D_SHIFT) #define P4D_MASK (~(P4D_SIZE - 1)) =20 diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 84b5df539a94..68f6a31b4d8e 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -48,14 +48,6 @@ unsigned int __initdata next_early_pgt; SYM_PIC_ALIAS(next_early_pgt); pmdval_t early_pmd_flags =3D __PAGE_KERNEL_LARGE & ~(_PAGE_GLOBAL | _PAGE_= NX); =20 -#ifdef CONFIG_X86_5LEVEL -unsigned int __pgtable_l5_enabled __ro_after_init; -unsigned int pgdir_shift __ro_after_init =3D 39; -EXPORT_SYMBOL(pgdir_shift); -unsigned int ptrs_per_p4d __ro_after_init =3D 1; -EXPORT_SYMBOL(ptrs_per_p4d); -#endif - #ifdef CONFIG_DYNAMIC_MEMORY_LAYOUT unsigned long page_offset_base __ro_after_init =3D __PAGE_OFFSET_BASE_L4; EXPORT_SYMBOL(page_offset_base); --=20 2.49.0.1045.g170613ef41-goog