From nobody Sat Feb 7 22:06:39 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D6CB25DD07 for ; Wed, 14 May 2025 10:43:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747219386; cv=none; b=dh4VqVq6piTKrizaUs/Aezy9aIQmThEioSCb5qgjp6xx6+r0ETu85tc4LUFoKM3ZO1TCz/QUA3yWh8UOyKGw4H8KGGHgBtnb/a+ZIHDKwTJX3CoqmqCqdvPD0qruzpArmx7PZ0wvGJghD9TZdWasHf8nOgxl/7oNTj2lmHP3pZc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747219386; c=relaxed/simple; bh=WyGZ/24tIZKLg0DtCp8ZCs/LqUIOLp/iW/2ONkTOUgU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=IzM9YsBU/CUw8gQoeZxD2dyLp6HaibFjS9ej3Il65RRaHkazYZ0OHL2rNFRZIPS2cJFYKc1c29MHHDvJPpDhLCnZU0V8UucB4Fs0E7Vf3pf7x1ajcNjzZaU2vF7uKNkIAPVwPm9X9YGrK2TXtWa54SqrSe0oYWEgrb3cuWpDRBs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=twmy2FU9; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="twmy2FU9" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-43cf44b66f7so38647165e9.1 for ; Wed, 14 May 2025 03:43:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747219383; x=1747824183; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ckYKmlenvli+0Pxosp0NBNxHU7WK7xjiIFwUQud7L+M=; b=twmy2FU9WqpiWUunN6K3tO3r0CMofgSnxWmUESTa70Vc6pBRF0hufty3kg1taxLF2V 5tidVLjsoAZRPUmI2foguZTd779re+XFOQbGgPsXbgIRx61i0ziksVLIsYe0H9YL0xIY OruTyqp5kP/96806gdQeyE+Uf9YJ33+VK+6Lm+hWex3BLtgzGglRt6NGxbC0ltAH7UAh R3Z1Jyg+FOtSY5P7VEQNZUSwIvpGnVkcvFShhi8Tw+29lRD9SfO/OVuuC8RZPmmbB6Dv 9PXShzdcMpzSLux5jJrO9L3i7tFrr/6JhlgPgGzQtjA9WvDA0S8UZxssgTK2zcd3rgQb 8EXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747219383; x=1747824183; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ckYKmlenvli+0Pxosp0NBNxHU7WK7xjiIFwUQud7L+M=; b=mlfq02XimetlarczL0pRNEcXgBSreXRW9UYn9qs5QqWDWKScJym3VwVJMnBuxNoHn8 ZrsEkoco+VpI9uGGyBpDgHAgK3j7n8+kv1iODZeILqCyRRLrkpjc6HXRJQZYkZEbmkx6 pjIJjSPKgwHl7Bn2OJhcsZDn4zKbs10VP/Jq9VI+n2tEfdkQE7iApjhDQaFGOjf2DLqR PcgZB9S1lfFuvqm08EFoj2BjU60kRyztwcHFJYtbBu5yXDhfWyRks5KFjwkR//ld4Slg WGlk9orfxVZ2mTquuu2aNhBKFtrLaINs1AiZWIko4A4xpcKkUePG+2obigb7VP/HlFdK kj+Q== X-Gm-Message-State: AOJu0YzDUdw4pKudnAQaTQ83sZgcZfGeKjjNLhK1t5rppnLXMcpQNfeJ 5QRAh7Bk6Xat2oMAgf3HbVEf0BDwHI0GOuJ4mWr/zJZL/Y9G7N/1q6zgW4ir1LnA54yNFqAHTGv BfiFZDYfCg7PJtzbRFBrhW/mgcpdmnXnAOAPOuu/WpS5JIID+mWRGD3wFhq4J1roB7dZiXyLh9P Jz7C2e8ceRzKHE1bjuqdMecU+S5e2erQ== X-Google-Smtp-Source: AGHT+IHbw3td/F9MyBKMNyMdIHKKo8hqKv1EMrvJQzvgVFUp+PGwcQKl6dlPm5huSDTMMlh0d6sLisHn X-Received: from wmbjb12.prod.google.com ([2002:a05:600c:54ec:b0:442:f45c:c10c]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600d:1b:b0:442:f4d4:53a with SMTP id 5b1f17b1804b1-442f4d43e20mr13951605e9.2.1747219383053; Wed, 14 May 2025 03:43:03 -0700 (PDT) Date: Wed, 14 May 2025 12:42:44 +0200 In-Reply-To: <20250514104242.1275040-9-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250514104242.1275040-9-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=8344; i=ardb@kernel.org; h=from:subject; bh=t77kZe3wNkywK6OUBs9RIua9fCaypVtIiPtrUFlNqNY=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUOleAn/jnci6sWaHorvmLv0t7xvL9rAevLDLuFmj4ybN 7uf+y/uKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABNxb2L4p/Jmij3jlw7VK1YV rTP+ZX+NnrzQwjT95Yudvuu3HZubeobhrxB3wqzODLMnXmyMcjIrP1zJkJVe+FpX+5B0xa5bjoE H2AE= X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250514104242.1275040-10-ardb+git@google.com> Subject: [PATCH v3 1/7] x86/cpu: Use a new feature flag for 5 level paging From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds , Brian Gerst Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Currently, the LA57 CPU feature flag is taken to mean two different things at once: - whether the CPU implements the LA57 extension, and is therefore capable of supporting 5 level paging; - whether 5 level paging is currently in use. This means the LA57 capability of the hardware is hidden when a LA57 capable CPU is forced to run with 4 levels of paging. It also means the the ordinary CPU capability detection code will happily set the LA57 capability and it needs to be cleared explicitly afterwards to avoid inconsistencies. Separate the two so that the CPU hardware capability can be identified unambigously in all cases. To avoid breaking existing users that might assume that 5 level paging is being used when the "la57" string is visible in /proc/cpuinfo, repurpose that string to mean that 5-level paging is in use, and add a new string la57_capable to indicate that the CPU feature is implemented by the hardware. Signed-off-by: Ard Biesheuvel --- arch/x86/include/asm/cpufeatures.h | 3 ++- arch/x86/include/asm/page_64.h | 2 +- arch/x86/include/asm/pgtable_64_types.h | 2 +- arch/x86/kernel/cpu/common.c | 16 ++-------------- arch/x86/kvm/x86.h | 4 ++-- drivers/iommu/amd/init.c | 4 ++-- drivers/iommu/intel/svm.c | 4 ++-- tools/testing/selftests/kvm/x86/set_sregs_test.c | 2 +- 8 files changed, 13 insertions(+), 24 deletions(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpuf= eatures.h index f67a93fc9391..d59bee5907e7 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -395,7 +395,7 @@ #define X86_FEATURE_AVX512_BITALG (16*32+12) /* "avx512_bitalg" Support fo= r VPOPCNT[B,W] and VPSHUF-BITQMB instructions */ #define X86_FEATURE_TME (16*32+13) /* "tme" Intel Total Memory Encryptio= n */ #define X86_FEATURE_AVX512_VPOPCNTDQ (16*32+14) /* "avx512_vpopcntdq" POPC= NT for vectors of DW/QW */ -#define X86_FEATURE_LA57 (16*32+16) /* "la57" 5-level page tables */ +#define X86_FEATURE_LA57 (16*32+16) /* "la57_hw" 5-level page tables */ #define X86_FEATURE_RDPID (16*32+22) /* "rdpid" RDPID instruction */ #define X86_FEATURE_BUS_LOCK_DETECT (16*32+24) /* "bus_lock_detect" Bus Lo= ck detect */ #define X86_FEATURE_CLDEMOTE (16*32+25) /* "cldemote" CLDEMOTE instructio= n */ @@ -483,6 +483,7 @@ #define X86_FEATURE_PREFER_YMM (21*32+ 8) /* Avoid ZMM registers due to d= ownclocking */ #define X86_FEATURE_APX (21*32+ 9) /* Advanced Performance Extensions */ #define X86_FEATURE_INDIRECT_THUNK_ITS (21*32+10) /* Use thunk for indirec= t branches in lower half of cacheline */ +#define X86_FEATURE_5LEVEL_PAGING (21*32+11) /* "la57" Whether 5 levels of= page tables are in use */ =20 /* * BUG word(s) diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h index d3aab6f4e59a..acfa61ad0725 100644 --- a/arch/x86/include/asm/page_64.h +++ b/arch/x86/include/asm/page_64.h @@ -86,7 +86,7 @@ static __always_inline unsigned long task_size_max(void) unsigned long ret; =20 alternative_io("movq %[small],%0","movq %[large],%0", - X86_FEATURE_LA57, + X86_FEATURE_5LEVEL_PAGING, "=3Dr" (ret), [small] "i" ((1ul << 47)-PAGE_SIZE), [large] "i" ((1ul << 56)-PAGE_SIZE)); diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm= /pgtable_64_types.h index e83721db18c9..d31ae12663bb 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -34,7 +34,7 @@ static inline bool pgtable_l5_enabled(void) return __pgtable_l5_enabled; } #else -#define pgtable_l5_enabled() cpu_feature_enabled(X86_FEATURE_LA57) +#define pgtable_l5_enabled() cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) #endif /* USE_EARLY_PGTABLE_L5 */ =20 #else diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 114aaaf6ae8a..6f7827015834 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1755,20 +1755,8 @@ static void __init early_identify_cpu(struct cpuinfo= _x86 *c) setup_clear_cpu_cap(X86_FEATURE_PCID); #endif =20 - /* - * Later in the boot process pgtable_l5_enabled() relies on - * cpu_feature_enabled(X86_FEATURE_LA57). If 5-level paging is not - * enabled by this point we need to clear the feature bit to avoid - * false-positives at the later stage. - * - * pgtable_l5_enabled() can be false here for several reasons: - * - 5-level paging is disabled compile-time; - * - it's 32-bit kernel; - * - machine doesn't support 5-level paging; - * - user specified 'no5lvl' in kernel command line. - */ - if (!pgtable_l5_enabled()) - setup_clear_cpu_cap(X86_FEATURE_LA57); + if (IS_ENABLED(CONFIG_X86_5LEVEL) && (native_read_cr4() & X86_CR4_LA57)) + setup_force_cpu_cap(X86_FEATURE_5LEVEL_PAGING); =20 detect_nopl(); } diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 9dc32a409076..d2c093f17ae5 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -243,7 +243,7 @@ static inline u8 vcpu_virt_addr_bits(struct kvm_vcpu *v= cpu) =20 static inline u8 max_host_virt_addr_bits(void) { - return kvm_cpu_cap_has(X86_FEATURE_LA57) ? 57 : 48; + return kvm_cpu_cap_has(X86_FEATURE_5LEVEL_PAGING) ? 57 : 48; } =20 /* @@ -603,7 +603,7 @@ static inline bool __kvm_is_valid_cr4(struct kvm_vcpu *= vcpu, unsigned long cr4) __reserved_bits |=3D X86_CR4_FSGSBASE; \ if (!__cpu_has(__c, X86_FEATURE_PKU)) \ __reserved_bits |=3D X86_CR4_PKE; \ - if (!__cpu_has(__c, X86_FEATURE_LA57)) \ + if (!__cpu_has(__c, X86_FEATURE_5LEVEL_PAGING)) \ __reserved_bits |=3D X86_CR4_LA57; \ if (!__cpu_has(__c, X86_FEATURE_UMIP)) \ __reserved_bits |=3D X86_CR4_UMIP; \ diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c index 14aa0d77df26..083fca8f8b97 100644 --- a/drivers/iommu/amd/init.c +++ b/drivers/iommu/amd/init.c @@ -3084,7 +3084,7 @@ static int __init early_amd_iommu_init(void) goto out; =20 /* 5 level guest page table */ - if (cpu_feature_enabled(X86_FEATURE_LA57) && + if (cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) && FIELD_GET(FEATURE_GATS, amd_iommu_efr) =3D=3D GUEST_PGTABLE_5_LEVEL) amd_iommu_gpt_level =3D PAGE_MODE_5_LEVEL; =20 @@ -3691,7 +3691,7 @@ __setup("ivrs_acpihid", parse_ivrs_acpihid); bool amd_iommu_pasid_supported(void) { /* CPU page table size should match IOMMU guest page table size */ - if (cpu_feature_enabled(X86_FEATURE_LA57) && + if (cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) && amd_iommu_gpt_level !=3D PAGE_MODE_5_LEVEL) return false; =20 diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index ba93123cb4eb..1f615e6d06ec 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -37,7 +37,7 @@ void intel_svm_check(struct intel_iommu *iommu) return; } =20 - if (cpu_feature_enabled(X86_FEATURE_LA57) && + if (cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) && !cap_fl5lp_support(iommu->cap)) { pr_err("%s SVM disabled, incompatible paging mode\n", iommu->name); @@ -165,7 +165,7 @@ static int intel_svm_set_dev_pasid(struct iommu_domain = *domain, return PTR_ERR(dev_pasid); =20 /* Setup the pasid table: */ - sflags =3D cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0; + sflags =3D cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) ? PASID_FLAG_FL= 5LP : 0; ret =3D __domain_setup_first_level(iommu, dev, pasid, FLPT_DEFAULT_DID, mm->pgd, sflags, old); diff --git a/tools/testing/selftests/kvm/x86/set_sregs_test.c b/tools/testi= ng/selftests/kvm/x86/set_sregs_test.c index f4095a3d1278..de78665fa675 100644 --- a/tools/testing/selftests/kvm/x86/set_sregs_test.c +++ b/tools/testing/selftests/kvm/x86/set_sregs_test.c @@ -52,7 +52,7 @@ static uint64_t calc_supported_cr4_feature_bits(void) =20 if (kvm_cpu_has(X86_FEATURE_UMIP)) cr4 |=3D X86_CR4_UMIP; - if (kvm_cpu_has(X86_FEATURE_LA57)) + if (kvm_cpu_has(X86_FEATURE_5LEVEL_PAGING)) cr4 |=3D X86_CR4_LA57; if (kvm_cpu_has(X86_FEATURE_VMX)) cr4 |=3D X86_CR4_VMXE; --=20 2.49.0.1101.gccaa498523-goog From nobody Sat Feb 7 22:06:39 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6AFC1270ECD for ; Wed, 14 May 2025 10:43:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747219388; cv=none; b=PmSP8Rhpgby096JnvtAS/iMwrD1ynFioStRBjJGoeV+T1Aiyqdd8qqsfDyuLUlnR8i7AQTRNCWHzjqNvlAzHKXVdOuEbDMSouQoPCHMg0kDtjeSOa+AfKTcIBWl7wy2JrSXp7zx/FfmkHHGhMskiEpo+BWtI5TYhQxpQGWdBIW8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747219388; c=relaxed/simple; bh=Qy4swRHdqb7GIWMI2+IQEAUo68bT990/p7joY7lPE/A=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=OKFFUpdkcsgB0sXWKwMscoKd7bLv16YNiI6prmGQ3TvRhzVDU76rRwu/yw7I0Q9ElnjDG7hReN3sPFknbKYHIslLgekCTNdcj5umtizkOsTF+wVncohPutA/ccyVSYwapar8qoUB5fRIEywoM1d8bfow+MU5HB+1ZxW5YjTLzfc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lbF/mS48; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lbF/mS48" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-442cdf07ad9so28427405e9.2 for ; Wed, 14 May 2025 03:43:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747219385; x=1747824185; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=mjxA9/NacXyJblHrDl8e4BLWViKbKXNacaYSz8P49QU=; b=lbF/mS48jqFEz73lAtKZd1zTZDWmm2G8L/iL15yEc2uUn9YDL0OJIZj6djyJWRKRx6 3oq2r2kvVRQHPRqTYXg7GMwl/FgTAm+vPTneTjV2LFT/mcX2KktLvGt1sUpE2334xfGk E76iTyYIrhc3+iGf+IE758FN/rvs211PoXFduBuWyubU6cGvcx7PoiQn1V7mHcMzu7d+ ilvVU0k2VN/EjWkoaAIbhQ2N7GeyWTU/ZkfXXU6dmO8yVyZ4eJue33yKHumJX0xNJVNz grhB+0uCfor3i9ibzQfQaOcLR+cSTkC2+ZbomSZn2VJ0cjMowmPrWqbJNNuj5YKGDnow myfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747219385; x=1747824185; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mjxA9/NacXyJblHrDl8e4BLWViKbKXNacaYSz8P49QU=; b=RTYceBchlLZmpWFDnfRHmyj/V4hUC/4/JbtWqCwzSa7pgcbIZXDlnvCaypZpKIKylf esijwpfpHRTUaNpTbQkewF0ecS1rTPGq8abiPFKnW6c8qsoMpaF0IiogABKNRtTyQtmW NdBEl8RyQl+ztdiOecHnM7PgqkK9ou6MfDysEGtut7d85bGt3qwK0I4MyJjYbeaHyG5X pyg38Q6q5d2V/p3S76rnGLzZrSlOqGhsXeFufqR1UtZymz8daWZ6B+oBpPBm00QYCzmg 9H9iEfnEzjUoEkoaQCNDHyJJJ0cRrGQMaXLWOMbzx46h70Is3zdMf+BXlfed6D7aB6Mb TkfQ== X-Gm-Message-State: AOJu0YwzsJgj/38wc/zhq7AplI7q8pMnfiWrj0NDqSwI+yNM+7zZWN4P URJukhV0YJdAzAzbqviybSNFa5Va/k6gz2YvNAIm8AxGF0z7WJ3cYe7SGZJRXZWdsgxlVftCDMm NodQcU78J7Fq7gE4k5sbiLhVXJQ6Gqe0xCf6OprYcSqj3THN0Ymm6lUTqy53+DVCCgLnbdUpioN NR4Sk+At9/nLYEHIrUmO0N+dnmIWqMJw== X-Google-Smtp-Source: AGHT+IFX2kxbrlLnmW0JE9ieUKAGCEn6noplLWKDg1OfwKIu+zbE5NRggR/lgchqPwskq9maGr9GcMuV X-Received: from wmrm4.prod.google.com ([2002:a05:600c:37c4:b0:43d:1dd4:37f2]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:698c:b0:43d:ea:51d2 with SMTP id 5b1f17b1804b1-442f20d6550mr30021405e9.14.1747219384897; Wed, 14 May 2025 03:43:04 -0700 (PDT) Date: Wed, 14 May 2025 12:42:45 +0200 In-Reply-To: <20250514104242.1275040-9-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250514104242.1275040-9-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3118; i=ardb@kernel.org; h=from:subject; bh=p2NOqIg6W9MB52am73AAAFc8VSowwxVLbOld/DsZJJY=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUOleCl33B+xtF/p5ZdUkuOW6TafV7eoffbXITj4dE9Bp cSxE80dpSwMYhwMsmKKLAKz/77beXqiVK3zLFmYOaxMIEMYuDgFYCK32Rn+pwrN59r1r/OPhfCe X8++tN+NmhVzP37Fw7V7j0z5MeNsYQQjw/mN86bHBHl+C6s+2v0vZX686TLr+t/T5GdIMD2J1Fo 2kQMA X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250514104242.1275040-11-ardb+git@google.com> Subject: [PATCH v3 2/7] x86/cpu: Allow caps to be set arbitrarily early From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds , Brian Gerst Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel cpu_feature_enabled() uses a ternary alternative, where the late variant is based on code patching and the early variant accesses the capability field in boot_cpu_data directly. This allows cpu_feature_enabled() to be called quite early, but it still requires that the CPU feature detection code runs before being able to rely on the return value of cpu_feature_enabled(). This is a problem for the implementation of pgtable_l5_enabled(), which is based on cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING), and may be called extremely early. Currently, there is a hacky workaround where some source files that may execute before (but also after) CPU feature detection have a different version of pgtable_l5_enabled(), based on the USE_EARLY_PGTABLE_L5 preprocessor macro. Instead, let's make it possible to set CPU feature arbitrarily early, so that the X86_FEATURE_5LEVEL_PAGING capability can be set before even entering C code. This involves relying on static initialization of boot_cpu_data and the cpu_caps_set/cpu_caps_cleared arrays, so they all need to reside in .data. This ensures that they won't be cleared along with the rest of BSS. Note that forcing a capability involves setting it in both boot_cpu_data.x86_capability[] and cpu_caps_set[]. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/cpu/common.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 6f7827015834..f6f206743d6a 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -704,8 +704,8 @@ static const char *table_lookup_model(struct cpuinfo_x8= 6 *c) } =20 /* Aligned to unsigned long to avoid split lock in atomic bitmap ops */ -__u32 cpu_caps_cleared[NCAPINTS + NBUGINTS] __aligned(sizeof(unsigned long= )); -__u32 cpu_caps_set[NCAPINTS + NBUGINTS] __aligned(sizeof(unsigned long)); +__u32 __read_mostly cpu_caps_cleared[NCAPINTS + NBUGINTS] __aligned(sizeof= (unsigned long)); +__u32 __read_mostly cpu_caps_set[NCAPINTS + NBUGINTS] __aligned(sizeof(uns= igned long)); =20 #ifdef CONFIG_X86_32 /* The 32-bit entry code needs to find cpu_entry_area. */ @@ -1708,9 +1708,6 @@ static void __init cpu_parse_early_param(void) */ static void __init early_identify_cpu(struct cpuinfo_x86 *c) { - memset(&c->x86_capability, 0, sizeof(c->x86_capability)); - c->extended_cpuid_level =3D 0; - if (!have_cpuid_p()) identify_cpu_without_cpuid(c); =20 @@ -1922,7 +1919,6 @@ static void identify_cpu(struct cpuinfo_x86 *c) c->x86_virt_bits =3D 32; #endif c->x86_cache_alignment =3D c->x86_clflush_size; - memset(&c->x86_capability, 0, sizeof(c->x86_capability)); #ifdef CONFIG_X86_VMX_FEATURE_NAMES memset(&c->vmx_capability, 0, sizeof(c->vmx_capability)); #endif @@ -2084,6 +2080,7 @@ void identify_secondary_cpu(unsigned int cpu) *c =3D boot_cpu_data; c->cpu_index =3D cpu; =20 + memset(&c->x86_capability, 0, sizeof(c->x86_capability)); identify_cpu(c); #ifdef CONFIG_X86_32 enable_sep_cpu(); --=20 2.49.0.1101.gccaa498523-goog From nobody Sat Feb 7 22:06:39 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 59F0D272E7A for ; Wed, 14 May 2025 10:43:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747219390; cv=none; b=W5jR54bhtK9QGzL6XQBzhcsoYDtNTe+FVUHT+G5hoXE/11/o0qqVX3b3mE2GrWXVUUXml7mu3V7v/odfBJBaXGlPnpQT5DU4/1196UIgk7HQ05fPxBre1j4Yfi3mck/rmCBrLwKATVu08nkaCX6rAVAoyMiinZz7K9X9GIGofG4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747219390; c=relaxed/simple; bh=y8s4bUb0IxHXfTiRhIemGG3ymnvZm5HHi84sBsqj+KE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=EFjPj0Fn3bFGhu+3lcW0EuhO8XWMqH/brozAixwAXmy+Yekfy0eqCF23viEcR8Bc2y6P7kDnB0FZ0N9sPZB6YpSg+gCqbM28hsm72IP0cL26P/tXoHlNu/qlsHUZxgLVOG51el/3Qz6TnFD9sD8pVHgySTq2ybdpaGGlEEXavhs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=2lZBn4Jf; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="2lZBn4Jf" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-442d472cf7fso40457855e9.3 for ; Wed, 14 May 2025 03:43:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747219387; x=1747824187; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=80flvzp8IqtGF+d/LETZ/1H5RL6Q/J5ql9Ob81N5sfU=; b=2lZBn4JfFRnV21ywWCkHoJkkRdj+BF8lbw8X2jC4V4YukIx8/PZKTM73A8r8y1Ohw+ H5qwe4ja6o+zoUbWtbYbchVsOEELyKpKcNUNBslrbOkHLd5gIbpBl8HS/n5RJyh5Hm4o fWaHA5q5XXGnjSSHSJisx8e4N/cOlY6+F/OSr/FFX9knPb2BPn/lQv+rXPUMmZ8w0ptr cgyrDPPmf8FiRpiPRIOJYVjIq2a2ykElUzHyIsinFx35SCTj0BX2tgZfWZXd7t6Vy75Q nq1j0wjGwVlQgw+YB/15NuG7xKTlFRTOf8MFKXaxOM9pdaCk+oiROnIB0fd5/t+G1oDt KCeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747219387; x=1747824187; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=80flvzp8IqtGF+d/LETZ/1H5RL6Q/J5ql9Ob81N5sfU=; b=a4Xkdtayzcq09PwIt6/p8Ij1nG7vSK7cwtD4omsn6r/e7qa8iA5+Jnty6WCxBR44JU 644XbaolEv+ig1bVe2PwnWlkKlDcYV80NOefvlbdff92TRlfoWPUvwA+mdZxm59/1hZp MMp4+FuhLB+0hybHFYChMAUozCb82aZ4LpYojnya0l3pBJwC2SXyNUce4QjtwUlI/HfQ zqLRkLQxlGBd4B9YDdkumZDgrRCczmE/ScCeydoXO5dh/Sr9Y57Klq+9vqYBNpUjvezo AybfWAoayN4JOWWoJaN42riWx4XOGVMRu8wtVrg3VQWGtz7GBsJLrBbbfVhgOWZaJ9Hx 2X5A== X-Gm-Message-State: AOJu0YwTI75sLCrbVxWRvUGB/zUvdBzSrXCUDkv/Fy/J49tq0FBfZFTb haJHPYPaS9f7OSQmADS+SdaPUGjOs+D4OYOJgqqQK+H+cQ757+XgVSeWmBpPYHrL2Tz4Su3ysa1 sJgz1XM5Vh1/NygNpL/+akuyfsxx+xnR6+/v4qoNVxzZVy0+ohHCC6L8sZyxIVbz1rLzJiQ+bRn +Jx/rbYrF4E4WN1qxIs4+Jsf4Bnm5OCw== X-Google-Smtp-Source: AGHT+IGvStNzVZduVnJp5jbauCg8hcm+MkhkDAG+ox0YT0WOikip+U1D4kw7j4LLGRzWddDifNmPGi1e X-Received: from wmpz14.prod.google.com ([2002:a05:600c:a0e:b0:442:cab2:15c3]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:a40a:b0:439:9b2a:1b2f with SMTP id 5b1f17b1804b1-442f285da1dmr23267445e9.3.1747219386877; Wed, 14 May 2025 03:43:06 -0700 (PDT) Date: Wed, 14 May 2025 12:42:46 +0200 In-Reply-To: <20250514104242.1275040-9-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250514104242.1275040-9-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1881; i=ardb@kernel.org; h=from:subject; bh=gBGqkNnrw35rPAatB03sACotWjygpNGNTzKZnb9kfW4=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUOleNmpnreVusbbpf6s5Zr5ZSODRbTIb+7O8DNy/3wvZ 28O3jito5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAEwksoKRYVXMdx9Lt8eOa2Zn 676TkeA4xV/9zjhI0ll36gKVuQctpzL8U7x6yK/x9cS34Sd3+Es/8AmVEmA4Hvul/Mer/zy+W7X 6OQA= X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250514104242.1275040-12-ardb+git@google.com> Subject: [PATCH v3 3/7] x86/asm-offsets: Export struct cpuinfo_x86 layout for asm use From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds , Brian Gerst Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Expose struct cpuinfo_x86 via asm-offsets for x86_64 too so that it will be possible to set CPU capabilities from asm code. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/asm-offsets.c | 8 ++++++++ arch/x86/kernel/asm-offsets_32.c | 9 --------- 2 files changed, 8 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/asm-offsets.c b/arch/x86/kernel/asm-offsets.c index ad4ea6fb3b6c..6259b474073b 100644 --- a/arch/x86/kernel/asm-offsets.c +++ b/arch/x86/kernel/asm-offsets.c @@ -33,6 +33,14 @@ =20 static void __used common(void) { + OFFSET(CPUINFO_x86, cpuinfo_x86, x86); + OFFSET(CPUINFO_x86_vendor, cpuinfo_x86, x86_vendor); + OFFSET(CPUINFO_x86_model, cpuinfo_x86, x86_model); + OFFSET(CPUINFO_x86_stepping, cpuinfo_x86, x86_stepping); + OFFSET(CPUINFO_cpuid_level, cpuinfo_x86, cpuid_level); + OFFSET(CPUINFO_x86_capability, cpuinfo_x86, x86_capability); + OFFSET(CPUINFO_x86_vendor_id, cpuinfo_x86, x86_vendor_id); + BLANK(); OFFSET(TASK_threadsp, task_struct, thread.sp); #ifdef CONFIG_STACKPROTECTOR diff --git a/arch/x86/kernel/asm-offsets_32.c b/arch/x86/kernel/asm-offsets= _32.c index 2b411cd00a4e..e0a292db97b2 100644 --- a/arch/x86/kernel/asm-offsets_32.c +++ b/arch/x86/kernel/asm-offsets_32.c @@ -12,15 +12,6 @@ void foo(void); =20 void foo(void) { - OFFSET(CPUINFO_x86, cpuinfo_x86, x86); - OFFSET(CPUINFO_x86_vendor, cpuinfo_x86, x86_vendor); - OFFSET(CPUINFO_x86_model, cpuinfo_x86, x86_model); - OFFSET(CPUINFO_x86_stepping, cpuinfo_x86, x86_stepping); - OFFSET(CPUINFO_cpuid_level, cpuinfo_x86, cpuid_level); - OFFSET(CPUINFO_x86_capability, cpuinfo_x86, x86_capability); - OFFSET(CPUINFO_x86_vendor_id, cpuinfo_x86, x86_vendor_id); - BLANK(); - OFFSET(PT_EBX, pt_regs, bx); OFFSET(PT_ECX, pt_regs, cx); OFFSET(PT_EDX, pt_regs, dx); --=20 2.49.0.1101.gccaa498523-goog From nobody Sat Feb 7 22:06:39 2026 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7634B274664 for ; Wed, 14 May 2025 10:43:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747219392; cv=none; b=hNdlLmhRndehEPFu4reqMW6pPUQ/ilqTFETmcyvDLpaxXA7/kQaOdpn8wVV30IvFYkOFbei9RX8P/DMfrTkqMj19jXY/Hp0ZLps9NS8BJfeymWYjbxgyJFxaBJEEVhRJxKfX0eQYDwBaoOhC6HSZWfi1YyLNnVmHH6gIPnACh10= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747219392; c=relaxed/simple; bh=e/MBru9i0x32CmO404oOZFVYF445BP/6nlpIjIUC0bE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WpWEcfOzO9UOmsXp85Jgj0M+QjoFCbdnZneDl3d1Xg7hD+ut8gJKc35+8m2KZKYXyNZLBq0ZXbuW3DYCKB0RwZKVKkj7y2/loiR5CcRyAJQX2pwudVjM7/9FoigcOq0DwXt7r5yhdg/rcakN64eH5F/9dsFyFqhSMj3mT/819M4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=u67KQVst; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="u67KQVst" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-3a0b589a2c6so2381027f8f.0 for ; Wed, 14 May 2025 03:43:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747219389; x=1747824189; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=0VzLG+4jCrLkOaKar5PHuA0l15B/Zk/YdAoRJV8V4Fc=; b=u67KQVst2Q1dy21PBaOZBAWxZ5H2l20euVnVKzZWcOPoPE/y5Tg8A+CX2Y7pqLnMxF Y2W1e1rs4eHYYaGfhPaVUiTMI+RdfV4+4JCLzimFmZptuM+B0TJHiRNZRehbrVJiMbUa /xyvTUKfoHkh13q/A0PbpH3JJga/GD+o4vWjBbGmB2PNPxBAroYL7qA+w5oaZ+agVd8+ GwZEXQrzVgat6Xr72C9miPcG6ggwcLi8YwgrnKGjsDGtQXkUfYZJ47Bd4CB6gGctbjWf uuOKklCuxUuTs+Has8Zh+cl6zYRaN67b70BuErGI5KujBqaSq7dStIcYtf8FZZSUc63n bWYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747219389; x=1747824189; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=0VzLG+4jCrLkOaKar5PHuA0l15B/Zk/YdAoRJV8V4Fc=; b=nrXnBbpoeQ1qF3EdpoVdZF7IID1aw/fi7JIgUy3gn7CmvSbHBseu3SQ+YdGTv8Re9t i1SqOnCBb2KdYr+UOnUljN/CPa8FavSRh76BUDOmpNd5k/nGHFbDYqlBmBYz4ZZdD5cc niTkGKz8Hnw69NwXl+ejTHRgF/mPTKZRFYrnaYwDrBfhruWPrZoSPlTxZElINl9YR1bU 00jv7NqYC/MMARVoVH4YtqvUaVGr3fHVPRuZwR6M7RR2uGs4bpxbN446XWlcfdoBsy0b yF+QYzhuPxdyML1evBnqUwD0w1y26KbIAqHUCd7rce2vr6iAprurzhDbxO6sjVIT9rnq Juqw== X-Gm-Message-State: AOJu0YzVR5UoRUz3CE0v0sytiMZvnmBzpG0rFuF2XwJDbALB92UNVxbS ZYAZCZ2HjAEOMKobYdfjo6yiE8ZsgA7/SMkrJJ8dwGnreFALjpMlcI8pgKht2ji5zhl4hkzDLIq IRGKUVHlm/ezBACoDOWAzbvdQZBIoC4GRNL3wZJvJMZF12v+0z9UTByWUTz5eh2SW/sIxYFpIPj bvvwusTS2fxESTVm21EZKzIabiE5902w== X-Google-Smtp-Source: AGHT+IH8fauKfOvfi56CYHzvmseXFVCjYtUpAPd9UQRZNBx+gw+gkPFtrnqib8RaW8T8pocK6W8ylX30 X-Received: from wmbek10.prod.google.com ([2002:a05:600c:3eca:b0:442:c98d:df37]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:2011:b0:3a0:be75:1bb1 with SMTP id ffacd0b85a97d-3a34991e1d9mr2312697f8f.42.1747219388810; Wed, 14 May 2025 03:43:08 -0700 (PDT) Date: Wed, 14 May 2025 12:42:47 +0200 In-Reply-To: <20250514104242.1275040-9-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250514104242.1275040-9-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2969; i=ardb@kernel.org; h=from:subject; bh=J+4pNRWP3H+GWumxi+lQpFWOEQDCqKW7T9fIG9bC9eA=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUOleHm1sEieapNJAse3r5dLr/IedJpymn3JucV+Vj+7s gsmp6l1lLIwiHEwyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgIncX8DwVzDXcPM0ydJSz8YP wRVs2tdl5/NxnZEp+rJc+lWe39fpbQx/JVarrRdsDPD4/G9zbdAxxQkz5v0tdb+n2r44VdTlyu9 0ZgA= X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250514104242.1275040-13-ardb+git@google.com> Subject: [PATCH v3 4/7] x86/boot: Set 5-level paging CPU cap before entering C code From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds , Brian Gerst Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel In order for pgtable_l5_enabled() to be reliable wherever it is used and however early, set the associated CPU capability from asm code before entering the startup C code. Signed-off-by: Ard Biesheuvel --- arch/x86/include/asm/cpufeature.h | 12 +++++++++--- arch/x86/kernel/cpu/common.c | 3 --- arch/x86/kernel/head_64.S | 15 +++++++++++++++ 3 files changed, 24 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/cpufeature.h b/arch/x86/include/asm/cpufe= ature.h index 893cbca37fe9..1b5de40e7bf7 100644 --- a/arch/x86/include/asm/cpufeature.h +++ b/arch/x86/include/asm/cpufeature.h @@ -2,10 +2,10 @@ #ifndef _ASM_X86_CPUFEATURE_H #define _ASM_X86_CPUFEATURE_H =20 +#ifdef __KERNEL__ +#ifndef __ASSEMBLER__ #include =20 -#if defined(__KERNEL__) && !defined(__ASSEMBLER__) - #include #include #include @@ -137,5 +137,11 @@ static __always_inline bool _static_cpu_has(u16 bit) #define CPU_FEATURE_TYPEVAL boot_cpu_data.x86_vendor, boot_cpu_data.x86, \ boot_cpu_data.x86_model =20 -#endif /* defined(__KERNEL__) && !defined(__ASSEMBLER__) */ +#else /* !defined(__ASSEMBLER__) */ + .macro setup_force_cpu_cap, cap:req + btsl $\cap % 32, boot_cpu_data+CPUINFO_x86_capability+4*(\cap / 32)(%rip) + btsl $\cap % 32, cpu_caps_set+4*(\cap / 32)(%rip) + .endm +#endif /* !defined(__ASSEMBLER__) */ +#endif /* defined(__KERNEL__) */ #endif /* _ASM_X86_CPUFEATURE_H */ diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index f6f206743d6a..c8954dc2fb26 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1752,9 +1752,6 @@ static void __init early_identify_cpu(struct cpuinfo_= x86 *c) setup_clear_cpu_cap(X86_FEATURE_PCID); #endif =20 - if (IS_ENABLED(CONFIG_X86_5LEVEL) && (native_read_cr4() & X86_CR4_LA57)) - setup_force_cpu_cap(X86_FEATURE_5LEVEL_PAGING); - detect_nopl(); } =20 diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 069420853304..191d5947a762 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -27,6 +27,7 @@ #include #include #include +#include =20 /* * We are not able to switch in one step to the final KERNEL ADDRESS SPACE @@ -58,6 +59,20 @@ SYM_CODE_START_NOALIGN(startup_64) */ mov %rsi, %r15 =20 +#ifdef CONFIG_X86_5LEVEL + /* + * Set the X86_FEATURE_5LEVEL_PAGING capability before calling into the + * C code, to give it a consistent view of any global pseudo-constants + * that are derived from pgtable_l5_enabled(). + */ + mov %cr4, %rax + btl $X86_CR4_LA57_BIT, %eax + jnc 0f + + setup_force_cpu_cap X86_FEATURE_5LEVEL_PAGING +0: +#endif + /* Set up the stack for verify_cpu() */ leaq __top_init_kernel_stack(%rip), %rsp =20 --=20 2.49.0.1101.gccaa498523-goog From nobody Sat Feb 7 22:06:39 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47EBC2749CF for ; Wed, 14 May 2025 10:43:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747219394; cv=none; b=PWgY4q6cXDLkZS2hzu/IwmUVeMN4uW9m6eGGbv/dGGo+1aEqaT36ARukcGfYwOKeFD6FdgDyEDxhSoBRR1vvA5+kMpp1ZGnGV6OIJSn9OvV+TzPmExMKcapZyBfDd2UBoRabnE/pSkQ1N7H9BYj321KqmLp6R6WxZpvst5nSvlE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747219394; c=relaxed/simple; bh=xxoNEMSTUCYLsCNwLM9WUsxu7gCDKHVup2TQhrh3AoA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=itPLPEuQYJQXRx1PHtZJjP6sxuWUNa0GSbh/9WqVeUU78xy7CZ2H4tttB80yGHGnmvHZksNIFr8qrpPFibaM8x1ZQfj82BjSnNi1ySm0wcOgBKxrz40SDhm7TrHTdOq3nc5EA5djzn+3xgteqS5AXdtn4fzoMfsw6emnQQqnl70= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=4I2UHtM2; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="4I2UHtM2" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-43cf5196c25so32305065e9.0 for ; Wed, 14 May 2025 03:43:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747219390; x=1747824190; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=KoCln9Dir8XSG1+T49pb7GLSaixX/5M9T2Zma60JTSY=; b=4I2UHtM2ZwB1pbLjbsOQgVM9oFnRH5QRZMiKHa1t/RBUhLhtCnWglJvIpffKp2+RFJ nQAdPhXoCCC8L4P4snjjCFr+/p7aRUZx6dzsUSGS9PrMShT+qGR6Cb+9YKuJuKTwaZ/R gXDKMoNrhyqohwpS6HfJVapiCtQnP0rSXe92BrBY3/9pdBoG54Cw5zvery1Gp2C+t9EO ykgL9eSb5Gev4FFsImopEshKikCV5WfBKcBtqKzkO1i2UvL5ffAMRDxUKwf+FOlEqh6W JbFyqjiCVs+cYcVM4Te+2XYrC8GdAjik3d14YLfpiBAhiToFvbYbXNlSuP9ceOLrWcSt 2isA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747219390; x=1747824190; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KoCln9Dir8XSG1+T49pb7GLSaixX/5M9T2Zma60JTSY=; b=chNfmZToZ7+0I5iLjkcC9fRR4oap4buW7sFBowd8ooPybXpB9FjUWoxbKkDEqwnEqk SGRBBSBr+AxrtSA1iqMPCPfhqBs172FuW1yqw1ooVKX5J0RH+J3+uvxPMS8hdfms6eIH 8CQA1SagsLWNZXwm9tNCbRj3l0EitxeA2RLKfqimPnUsZr4o6DvGCxr4APjH5l/v3vrc DodofnNPswKdjdxVUSdBAsn9Jji9ANbAMVVzgpi2b6FkE/8PcpJ1HOuo6zbXW4p9BtEu JOR7wk/gXoCgocJlw1Kq7xJLGwUXB+3HTqYXZKZI7iZZLUFFA8k5VIHUNdt3+xDNWD8x a9tA== X-Gm-Message-State: AOJu0YxDaJX8dfPeNBgDuiPy1yGBQBFWEbxxCMbwS5SWGwlBQfNzW9UX kkN7yEHsle+cdYqEl1hNFOHBpn7pubrYNVbUHzb1kBUu/AbhihXBho16hINV62YgO+I65CVM9fx YIDEtJcQFnxlZFwRMTlzRQLfHom4qYVM71sk1Q179aMaqVevS9/csMp0C4iMKoFJJTy4QGtwKzJ mafTkfKctCOxXRMnDsTz3U0RL7NEzitA== X-Google-Smtp-Source: AGHT+IFVekG9SdJs73x4qWOsN3ineFqG79bfgsNiuf/qRSGuIOZSQJ9IVALPJiKp04XquGiIKH1rfm56 X-Received: from wmbfm5.prod.google.com ([2002:a05:600c:c05:b0:43d:4ba5:b5d6]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:8012:b0:43d:82c:2b11 with SMTP id 5b1f17b1804b1-442f216074bmr22373505e9.23.1747219390733; Wed, 14 May 2025 03:43:10 -0700 (PDT) Date: Wed, 14 May 2025 12:42:48 +0200 In-Reply-To: <20250514104242.1275040-9-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250514104242.1275040-9-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=5778; i=ardb@kernel.org; h=from:subject; bh=yugaVGpx7jAlY4kkCR2uhnsWUOwUL6x2MzNG/46sYME=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUOleKXozk0RO2ttLs8ufTbPy6XMapf/qhKPrWeP+Wdck 9x4WeNfRykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZiI30GG/xFBVhJdAdOuNH+e G7kv+kuGntCj08bz0l7/Vj2/5kqf8x6Gf5orXZ0+Z6qrnOiqb1b8d7zeeC3zi+qzekzHmE51VsQ tZQcA X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250514104242.1275040-14-ardb+git@google.com> Subject: [PATCH v3 5/7] x86/boot: Drop the early variant of pgtable_l5_enabled() From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds , Brian Gerst Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Now that cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) is guaranteed to produce the correct value even during early boot, there is no longer a need for an early variant and so it can be dropped. For the decompressor, fall back to testing the CR4.LA57 control register bit directly. Note that this removes the need to disable KASAN temporarily while applying alternatives, given that any constant or VA space dimension derived from pgtable_l5_enabled() will now always produce a consistent value. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/misc.h | 7 ++++--- arch/x86/boot/startup/sme.c | 9 --------- arch/x86/include/asm/pgtable_64_types.h | 14 ++------------ arch/x86/kernel/alternative.c | 12 ------------ arch/x86/kernel/cpu/common.c | 2 -- arch/x86/kernel/head64.c | 3 --- arch/x86/mm/kasan_init_64.c | 3 --- 7 files changed, 6 insertions(+), 44 deletions(-) diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/mis= c.h index db1048621ea2..72b830b8a69c 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -16,9 +16,6 @@ =20 #define __NO_FORTIFY =20 -/* cpu_feature_enabled() cannot be used this early */ -#define USE_EARLY_PGTABLE_L5 - /* * Boot stub deals with identity mappings, physical and virtual addresses = are * the same, so override these defines. @@ -28,6 +25,10 @@ #define __pa(x) ((unsigned long)(x)) #define __va(x) ((void *)((unsigned long)(x))) =20 +#ifdef CONFIG_X86_5LEVEL +#define pgtable_l5_enabled() (native_read_cr4() & X86_CR4_LA57) +#endif + #include #include #include diff --git a/arch/x86/boot/startup/sme.c b/arch/x86/boot/startup/sme.c index 70ea1748c0a7..a6c25d005991 100644 --- a/arch/x86/boot/startup/sme.c +++ b/arch/x86/boot/startup/sme.c @@ -25,15 +25,6 @@ #undef CONFIG_PARAVIRT_XXL #undef CONFIG_PARAVIRT_SPINLOCKS =20 -/* - * This code runs before CPU feature bits are set. By default, the - * pgtable_l5_enabled() function uses bit X86_FEATURE_LA57 to determine if - * 5-level paging is active, so that won't work here. USE_EARLY_PGTABLE_L5 - * is provided to handle this situation and, instead, use a variable that - * has been set by the early boot code. - */ -#define USE_EARLY_PGTABLE_L5 - #include #include #include diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm= /pgtable_64_types.h index d31ae12663bb..3e94da790cb7 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -24,19 +24,9 @@ typedef struct { pmdval_t pmd; } pmd_t; extern unsigned int __pgtable_l5_enabled; =20 #ifdef CONFIG_X86_5LEVEL -#ifdef USE_EARLY_PGTABLE_L5 -/* - * cpu_feature_enabled() is not available in early boot code. - * Use variable instead. - */ -static inline bool pgtable_l5_enabled(void) -{ - return __pgtable_l5_enabled; -} -#else +#ifndef pgtable_l5_enabled #define pgtable_l5_enabled() cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) -#endif /* USE_EARLY_PGTABLE_L5 */ - +#endif #else #define pgtable_l5_enabled() 0 #endif /* CONFIG_X86_5LEVEL */ diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 2385528792b2..e39823d8d1ae 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -589,16 +589,6 @@ void __init_or_module noinline apply_alternatives(stru= ct alt_instr *start, =20 DPRINTK(ALT, "alt table %px, -> %px", start, end); =20 - /* - * In the case CONFIG_X86_5LEVEL=3Dy, KASAN_SHADOW_START is defined using - * cpu_feature_enabled(X86_FEATURE_LA57) and is therefore patched here. - * During the process, KASAN becomes confused seeing partial LA57 - * conversion and triggers a false-positive out-of-bound report. - * - * Disable KASAN until the patching is complete. - */ - kasan_disable_current(); - /* * The scan order should be from start to end. A later scanned * alternative code can overwrite previously scanned alternative code. @@ -666,8 +656,6 @@ void __init_or_module noinline apply_alternatives(struc= t alt_instr *start, =20 text_poke_early(instr, insn_buff, insn_buff_sz); } - - kasan_enable_current(); } =20 static inline bool is_jcc32(struct insn *insn) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index c8954dc2fb26..08a586606e24 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1,6 +1,4 @@ // SPDX-License-Identifier: GPL-2.0-only -/* cpu_feature_enabled() cannot be used this early */ -#define USE_EARLY_PGTABLE_L5 =20 #include #include diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 14f7dda20954..84b5df539a94 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -5,9 +5,6 @@ * Copyright (C) 2000 Andrea Arcangeli SuSE */ =20 -/* cpu_feature_enabled() cannot be used this early */ -#define USE_EARLY_PGTABLE_L5 - #include #include #include diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 0539efd0d216..7c4fafbd52cc 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -1,9 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 #define pr_fmt(fmt) "kasan: " fmt =20 -/* cpu_feature_enabled() cannot be used this early */ -#define USE_EARLY_PGTABLE_L5 - #include #include #include --=20 2.49.0.1101.gccaa498523-goog From nobody Sat Feb 7 22:06:39 2026 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 62D72274FC2 for ; Wed, 14 May 2025 10:43:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747219396; cv=none; b=CUtOB/8VFgVOQL5Pou1VXZ73RkkYWPqbHP3nBBY1Y0yny/d041xtyMe/je8C0xfg6gJD9GXYGCQ7hoM1wzfp3CUEhYmD7FCRAY7aopG7qZpkxQ5WWkMVlTH5ws8wUPhW1zvlvJV9IspTy1TmPpNM0kru2j8uneNv4DgxkO+krXs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747219396; c=relaxed/simple; bh=wVgytFlfrZYDQdk7biHLFWpfSUdJ1/Hab/NTcZzWFsE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=MavVAcpIF/ZmwhM2gyEfnuG4FsYWGqdm5dpIt47TA6akto7kSYrBbWrU2BqSZCV1ieuyq3yKFJcQNN2osYwctw8uDhxH33zC6bnh31mA08BNDwJHPsh2+MSW0jq97tIwznvp/WnlOsUzMeJyFABPxhVCmslIy8CHu9wp9s4TP/s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=C4gNuFur; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="C4gNuFur" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-3a0b6cb5606so3061184f8f.0 for ; Wed, 14 May 2025 03:43:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747219392; x=1747824192; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=VWJj8+iqJAgJOO5WqllK6FIvG4mC5sXfm+BBNrQO94U=; b=C4gNuFurX07oEhTN+azdqQx+KKGIRxMmzieLHUYQsJXEe4aFxG3EJgJZlra5k0cXm9 wa1B2ZBS0ZKchgq4v4C2mR8JoqXLwtVDBp/SsStNOptx1cIAJOivJxOrysRku5kaa9lK afVETKTje0HjtvB3oenN/xN5Ek7K2XEP6CbRbO6wlEKdD6IyePwrT5SbddClER+Rkwki LWO+JJ/GzqGbr1ZvPMh55MAN2mjOmyylI0iUsd2LBtV+lBhCHtPcewLYbn7D9leKfOkF rrstck9sjXjupTGkGJKtT+iE1N+YKx6/S8ibBuAUfl4/kH9kKfcrRYzSx9p+O7Mq/Mc8 XsTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747219392; x=1747824192; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VWJj8+iqJAgJOO5WqllK6FIvG4mC5sXfm+BBNrQO94U=; b=wrXzJQvIA1fkWlv6qSOUfWuooXW08fMmRoq7VVDK40mj+yDNTWAnvq/x1Q1cyWvAY3 aLP75ALlckzrfx/rTNBcPjVNbkKaxfjua02iCPFK9rnRvMWzfFOTQ0q1rQCkiSUlTDjH 6k4jOjB6h7LEtPqd2yeSEyXByfchG57NgTusJ4XIYWQ4AxEsLmg22NQFlH1rll11ml1y MNrbIVdLLsj9bvxpfZlWNevr7lkgbOrNL7q2u+/RAYXeLOUbI1TIyc45MKQom9MIvqCk ZrDga966sc7go4xpoiVo3pseBlLKi6eLyUCuUR6rzGf7jOVmoFFrctJ6vjGSUd5N8Plc 9HDA== X-Gm-Message-State: AOJu0YzBpIF9HG9P6KIvaiPxLOj3HRK5z4I+kibOgh5VzrLjXTFIETn/ /gUZJkJuYHyzmF8gPXYaHbcvOmKp8/kcSoxBWVEyRaFftgzLtTsifoCWNYVf2gf/8jSpwWhsG/a q1+Ju3a5X9fgzBBHV34VyybzObMIbaT1MEJcGQspGmbhRjo7BEXvcrjd4f0uJRTqWA7htFm09ij JgoFTDYsnlyZn90WC61lr5AaUXzf6K2A== X-Google-Smtp-Source: AGHT+IEKogkGnauwUE1wTbaou6kMGdNu8wrl6JgiLLpViruNP1QShp21+4kbLzoUYQ9q/gzJRYLmr5+j X-Received: from wmbfl10.prod.google.com ([2002:a05:600c:b8a:b0:440:68cb:bd4]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:40c9:b0:391:39fb:59c8 with SMTP id ffacd0b85a97d-3a3496c1ebdmr2427927f8f.25.1747219392654; Wed, 14 May 2025 03:43:12 -0700 (PDT) Date: Wed, 14 May 2025 12:42:49 +0200 In-Reply-To: <20250514104242.1275040-9-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250514104242.1275040-9-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=5643; i=ardb@kernel.org; h=from:subject; bh=ZJqS2ncMmT4amBTlidYVOfxHc+2GJc7TrdutOHDYkMc=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUOleNUXpb8B/58wSixdnbvr7JJ0Yf7dFg/OuMoXRcT9u sf4ZnZWRykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZiIUiMjwwGZjPI4q0af+Q8c 2WV8Nu7ck/x/ssj9l/f+f8jmtGhscGf4X/T/ku+XKq07U9eG2gv7CXbukpyxkGPVowx+i+1rii+ ksQAA X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250514104242.1275040-15-ardb+git@google.com> Subject: [PATCH v3 6/7] x86/boot: Drop 5-level paging related variables and early updates From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds , Brian Gerst Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The variable __pgtable_l5_enabled is no longer used so it can be dropped. Along with it, drop ptrs_per_p4d and pgdir_shift, and replace any references to those with expressions based on pgtable_l5_enabled(). This ensures that all observers see values that are mutually consistent. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/misc.h | 1 - arch/x86/boot/compressed/pgtable_64.c | 12 ----------- arch/x86/boot/startup/map_kernel.c | 21 +------------------- arch/x86/include/asm/pgtable_64_types.h | 9 ++------- arch/x86/kernel/head64.c | 8 -------- 5 files changed, 3 insertions(+), 48 deletions(-) diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/mis= c.h index 72b830b8a69c..3d5c6322def8 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -190,7 +190,6 @@ static inline int count_immovable_mem_regions(void) { r= eturn 0; } #endif =20 /* ident_map_64.c */ -extern unsigned int __pgtable_l5_enabled, pgdir_shift, ptrs_per_p4d; extern void kernel_add_identity_map(unsigned long start, unsigned long end= ); =20 /* Used by PAGE_KERN* macros: */ diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compress= ed/pgtable_64.c index 5a6c7a190e5b..591d28f2feb6 100644 --- a/arch/x86/boot/compressed/pgtable_64.c +++ b/arch/x86/boot/compressed/pgtable_64.c @@ -10,13 +10,6 @@ #define BIOS_START_MIN 0x20000U /* 128K, less than this is insane */ #define BIOS_START_MAX 0x9f000U /* 640K, absolute maximum */ =20 -#ifdef CONFIG_X86_5LEVEL -/* __pgtable_l5_enabled needs to be in .data to avoid being cleared along = with .bss */ -unsigned int __section(".data") __pgtable_l5_enabled; -unsigned int __section(".data") pgdir_shift =3D 39; -unsigned int __section(".data") ptrs_per_p4d =3D 1; -#endif - /* Buffer to preserve trampoline memory */ static char trampoline_save[TRAMPOLINE_32BIT_SIZE]; =20 @@ -127,11 +120,6 @@ asmlinkage void configure_5level_paging(struct boot_pa= rams *bp, void *pgtable) native_cpuid_eax(0) >=3D 7 && (native_cpuid_ecx(7) & (1 << (X86_FEATURE_LA57 & 31)))) { l5_required =3D true; - - /* Initialize variables for 5-level paging */ - __pgtable_l5_enabled =3D 1; - pgdir_shift =3D 48; - ptrs_per_p4d =3D 512; } =20 /* diff --git a/arch/x86/boot/startup/map_kernel.c b/arch/x86/boot/startup/map= _kernel.c index 905e8734b5a3..056de4766006 100644 --- a/arch/x86/boot/startup/map_kernel.c +++ b/arch/x86/boot/startup/map_kernel.c @@ -14,25 +14,6 @@ extern pmd_t early_dynamic_pgts[EARLY_DYNAMIC_PAGE_TABLES][PTRS_PER_PMD]; extern unsigned int next_early_pgt; =20 -static inline bool check_la57_support(void) -{ - if (!IS_ENABLED(CONFIG_X86_5LEVEL)) - return false; - - /* - * 5-level paging is detected and enabled at kernel decompression - * stage. Only check if it has been enabled there. - */ - if (!(native_read_cr4() & X86_CR4_LA57)) - return false; - - __pgtable_l5_enabled =3D 1; - pgdir_shift =3D 48; - ptrs_per_p4d =3D 512; - - return true; -} - static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdval_t *pmd, unsigned long p2v_offset) @@ -102,7 +83,7 @@ unsigned long __head __startup_64(unsigned long p2v_offs= et, bool la57; int i; =20 - la57 =3D check_la57_support(); + la57 =3D pgtable_l5_enabled(); =20 /* Is the address too large? */ if (physaddr >> MAX_PHYSMEM_BITS) diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm= /pgtable_64_types.h index 3e94da790cb7..dc3a08b539d8 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -21,8 +21,6 @@ typedef unsigned long pgprotval_t; typedef struct { pteval_t pte; } pte_t; typedef struct { pmdval_t pmd; } pmd_t; =20 -extern unsigned int __pgtable_l5_enabled; - #ifdef CONFIG_X86_5LEVEL #ifndef pgtable_l5_enabled #define pgtable_l5_enabled() cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) @@ -31,9 +29,6 @@ extern unsigned int __pgtable_l5_enabled; #define pgtable_l5_enabled() 0 #endif /* CONFIG_X86_5LEVEL */ =20 -extern unsigned int pgdir_shift; -extern unsigned int ptrs_per_p4d; - #endif /* !__ASSEMBLER__ */ =20 #ifdef CONFIG_X86_5LEVEL @@ -41,7 +36,7 @@ extern unsigned int ptrs_per_p4d; /* * PGDIR_SHIFT determines what a top-level page table entry can map */ -#define PGDIR_SHIFT pgdir_shift +#define PGDIR_SHIFT (pgtable_l5_enabled() ? 48 : 39) #define PTRS_PER_PGD 512 =20 /* @@ -49,7 +44,7 @@ extern unsigned int ptrs_per_p4d; */ #define P4D_SHIFT 39 #define MAX_PTRS_PER_P4D 512 -#define PTRS_PER_P4D ptrs_per_p4d +#define PTRS_PER_P4D (pgtable_l5_enabled() ? MAX_PTRS_PER_P4D : 1) #define P4D_SIZE (_AC(1, UL) << P4D_SHIFT) #define P4D_MASK (~(P4D_SIZE - 1)) =20 diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 84b5df539a94..68f6a31b4d8e 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -48,14 +48,6 @@ unsigned int __initdata next_early_pgt; SYM_PIC_ALIAS(next_early_pgt); pmdval_t early_pmd_flags =3D __PAGE_KERNEL_LARGE & ~(_PAGE_GLOBAL | _PAGE_= NX); =20 -#ifdef CONFIG_X86_5LEVEL -unsigned int __pgtable_l5_enabled __ro_after_init; -unsigned int pgdir_shift __ro_after_init =3D 39; -EXPORT_SYMBOL(pgdir_shift); -unsigned int ptrs_per_p4d __ro_after_init =3D 1; -EXPORT_SYMBOL(ptrs_per_p4d); -#endif - #ifdef CONFIG_DYNAMIC_MEMORY_LAYOUT unsigned long page_offset_base __ro_after_init =3D __PAGE_OFFSET_BASE_L4; EXPORT_SYMBOL(page_offset_base); --=20 2.49.0.1101.gccaa498523-goog From nobody Sat Feb 7 22:06:39 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5258B27510A for ; Wed, 14 May 2025 10:43:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747219399; cv=none; b=Vp7Xji9uVaPBd/Fk6V0PyZYIiHR5ttT0gWS6rM/amZCaPU/OKg6JG1LTKXMw/mgUzfaVLwAcfU3XU8b3EUFl0zxJ3bG480Rj/AmJzSbrkbVYga0HQe4Q5i+WkXFuAzrOWPlgLswF4hO+018OznTRKWsns6qr68uv3if4xgZW2kg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747219399; c=relaxed/simple; bh=hWgX7qXvIT3HZAzoYoIDN1xkBbDXH1YUNgUPdd4zZ7U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=oRkt86gZpbi56iCnv1qSn8h9Hv6fIBOca8rDxrows4oUmFfHvNPQ0INeFDXV6fzOK8QM8hrooP6O72ZhVz3FSX6IfOJqLhAtdbXPPmF4cz1Fj3zxOBeG/oGYsbiDDIpQpw/NsrlJSnMV/QO2ETApAZWYLqX6202JEw/GtatT8Wg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=QOjgwXIk; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="QOjgwXIk" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-441c96c1977so43514705e9.0 for ; Wed, 14 May 2025 03:43:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747219394; x=1747824194; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=gOB58JVAymn9DIOyvp9CG9R9zCpbYXA0Lhh4NhuM7zs=; b=QOjgwXIkRU946IQUPUpCoHZtdXagisq4V/wWkZLEpXMIZj7XmrHmmei+dwRSCh0koD Fbp5OdrA+Pu5vL3mA9S8eq02FNU9uG1dNX3GHwz/QvqX+k0pcdPNXFuxMExNU4susc2a hVl84sXNT6cEg3JCrBdHD9GlVMYg5HK1N+U6N6jZJioNgNzL3TThZME1M3C5V0y6NgEQ 7vo0U/GLdTtben8JhiMWz7mS5jK5K+gVexwwkqMeLGkbgLERrj+5AHMY48dIOiuCpANE Hd2L9zaJodJY6M445I2Ph/Og1RHbp7bq3KEtBeO27PfLiANTF0Bs0Sgo8/ySeMs+ZcZr i8Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747219394; x=1747824194; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=gOB58JVAymn9DIOyvp9CG9R9zCpbYXA0Lhh4NhuM7zs=; b=pQAFC7p60DxAxkqkF5Me5slXu++i9eP5Zkje5cpQjL+SEP2kLpSB4dkHLffukrJ9Ne Shv9OiTXEpcRQrbD0wJ58Oh/sZnPsYdzDebVdL5ZyV+GEDK2Yd4itdp1zDUZjjOOME2Z wSTYvBlXWX91E22IhPVCqWs3E9dptlCwU3kpckvNzuG7VtuV/ijE3qbUTdmJ/++ZPrSG mqZOIGUZEDMJUiiq0F86T7UEf0fcmISxyL+jQ277ciRUr+Qfm1wZl5jIie1QHMHLo4UK JQ6mAMUTx3SlRy/cSuFgy8JJNyHg+aAhgKrQdQzkxS7+OBv0ce55PNQD6wC96qrBmm+v 9Lsw== X-Gm-Message-State: AOJu0Yw/pMrpGiyzdQsVva5a3snOP1Wp2+IxtL07iCq2k9+P4aIxC6mB z7GMT0FdKzLrNH+hAJirWy7r5SJzcAlkXBNgIwOsePN6Kd/JCVGCjBzL6/UoPVu0ICC/UtOEbtQ uo3/UND0EwYdGexSWAtg6eEYz2v6wZOyC+nd91/hitzxsYUcThaECn+3UunsQJ8ULGD9vIMLFn6 WuDkSYUZTrSNLvrIzC2sRwVg9OoEMgkA== X-Google-Smtp-Source: AGHT+IH9JnZfF11Zn9zczUS7Jbcn8wqjwD3wJ09uDKapVoiH1A30r9OSnDQs2RLOqgnOkbpg+QzLIHOy X-Received: from wryv6.prod.google.com ([2002:a5d:59c6:0:b0:39a:bec7:76db]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:1acb:b0:3a3:4bd8:4c6 with SMTP id ffacd0b85a97d-3a34bd80595mr1452005f8f.59.1747219394674; Wed, 14 May 2025 03:43:14 -0700 (PDT) Date: Wed, 14 May 2025 12:42:50 +0200 In-Reply-To: <20250514104242.1275040-9-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250514104242.1275040-9-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1080; i=ardb@kernel.org; h=from:subject; bh=x4z2VkW3ceTma4YBru6lhP3tx3BetAlSCCBd3CfJhOE=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUOlePWzkK1+1yQeBzuddHitFuz+12q64uLX/hGPGZc76 uaWr23sKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABPhO8PIsDjqSbYGv9muzhPv 5K6pRW1dHPExf/M5t88JOdOUorQ8jBgZjm+ViQsr3uj70KC61UXcRCi5tdX626WHAkpSzPULbL3 ZAA== X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250514104242.1275040-16-ardb+git@google.com> Subject: [PATCH v3 7/7] x86/cpu: Make CPU capability overrides __ro_after_init From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds , Brian Gerst Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel CPU capabilities are set at init time so they can be made R/O once the boot completes. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/cpu/common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 08a586606e24..19b310bd55ae 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -702,8 +702,8 @@ static const char *table_lookup_model(struct cpuinfo_x8= 6 *c) } =20 /* Aligned to unsigned long to avoid split lock in atomic bitmap ops */ -__u32 __read_mostly cpu_caps_cleared[NCAPINTS + NBUGINTS] __aligned(sizeof= (unsigned long)); -__u32 __read_mostly cpu_caps_set[NCAPINTS + NBUGINTS] __aligned(sizeof(uns= igned long)); +__u32 __ro_after_init cpu_caps_cleared[NCAPINTS + NBUGINTS] __aligned(size= of(unsigned long)); +__u32 __ro_after_init cpu_caps_set[NCAPINTS + NBUGINTS] __aligned(sizeof(u= nsigned long)); =20 #ifdef CONFIG_X86_32 /* The 32-bit entry code needs to find cpu_entry_area. */ --=20 2.49.0.1101.gccaa498523-goog