From nobody Fri Dec 19 17:16:03 2025 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 534921C5D4E for ; Sat, 17 May 2025 09:16:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747473414; cv=none; b=UgXzkbmXmY2GDxtQD94/qx4XWxtbcmV8FwAN6STCtc06ZpsBqqZeinotifBZYCTgk/ktr6NaHYcziDL//u1qhYjhRAkyAt+jSf4FfXUVpe9NMPayasPYGzmhB4pfEaaqaVwRxavO+EPNFIt0c9mXat4XWmpDOLgrVzqvJdVWc0I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747473414; c=relaxed/simple; bh=A2PWt6apuTm9JwluZpTScl/wfgfGu9Nf4nGzK5k3BNU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=iY4ZXu3yHOhCkfS4Wm9/KvMzCiU9rpj5i9gP05+jJdz0Llpn8JFIhOrPFpuvUCihyUWJAv/YpDJS2OfxV3dwes1SB1CzqPT83/CJkwHgBCy2LFuguGnUgKgbggBrMYbrQbHrOkBrmS7gNN+SFcBnHKshg11nNLjf7VTHQkdTDU0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Hgqaa8D6; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Hgqaa8D6" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-43cfda30a3cso15528465e9.3 for ; Sat, 17 May 2025 02:16:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747473410; x=1748078210; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ySqGRgKPqgwX6++IKUIQ9Ziefq9nh8F/5/YDVXWW7hc=; b=Hgqaa8D6GCijHPERO6QnqnSG5Qeoq96SNCWFveAJQ+Jzmx0sNrPgJiyewNq+Tc2v1H aHxJrAbK8fgeg3cAgAIWubRBA5lQPwsqTBkBQdSrhkKU2MTrkfyhyLoojLbsWsTaZ50x C4kl9TfbKF8B1woGEQBhXqOn9P0MqE0I4O9nG43sMG6DQevvHXDU32Puow1mthP4KDxs 3dZmfjZQSa6pKHAUId70RvRKFLiKRZiX74rtg4qVT3yTCdR2IzUTHJCTKWbHmJVm8SLc fsx+JSRA/seqDVs9JYaywtWCu6d20/0YyFbSwaVR6i8iBRqrjIgYYcesCB8OMIIcY738 sNCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747473410; x=1748078210; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ySqGRgKPqgwX6++IKUIQ9Ziefq9nh8F/5/YDVXWW7hc=; b=hGbWsls6Q8B41hVzscOYhC3ysTXm92xeIKBAPzd0PjvYrutPpeFhMLDJSVu9p8ZCol 1bGaxIDgRxhvQAn3TjYIg1qO45R3ES1rhdLChzB2OhRcUsx+rOskz2PE0f+2cciaRpXa 3ZkwoOVhtMqYmMFQhLHl0M2Yub48XouNiALXSEmHeFNZ3cKRcY9jfoSh49ahgWssXH41 SJoMjT8hsnATcjORA38NtN8IhXC9vqENces5YhBz/mOXVUKwagy9SR1CnHuOWMmSF0sl Pu6Wt6nm2JgNrDEKn1CVQqWrI0Xp7fI1kYK1wrSYogOlGwMuezols7flKllJuUzNvlrK 0bpw== X-Gm-Message-State: AOJu0YzbRRvGLiHOHG01d4ivFAtluMZT/fdqfm+ss0jKwVLsU3U47CXl tACiqYb5pZ7ZQFZM2TeDoyNV30q9OELIapzhReXqZ6LYzElW1alKsPKKabmAcHy7MOd6XXssv5J QqsMPkVjAfPG0Ois8337fqy6ZDcTGzSezPGNm7LWaajf6vWv2/0JaB97aN83z96IT/CwLbuRCJy c31wQZ9lEdT70futCjSlaqsqHFZUt6KIzs1w== X-Google-Smtp-Source: AGHT+IEdsXPLrHmfzWnR5aagQ68Up8bfq5fBEgvzRrU1XlcXC2St/uxPUjjIbI45MsV2X7UYNYtdubIg X-Received: from wmrm14.prod.google.com ([2002:a05:600c:37ce:b0:43c:f7c3:c16e]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3c87:b0:442:d9f2:c6ef with SMTP id 5b1f17b1804b1-442fefd5f98mr59544035e9.2.1747473410746; Sat, 17 May 2025 02:16:50 -0700 (PDT) Date: Sat, 17 May 2025 11:16:41 +0200 In-Reply-To: <20250517091639.3807875-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250517091639.3807875-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=6631; i=ardb@kernel.org; h=from:subject; bh=3LxUTlSUbspQmEIJOCH5/fP+eOwl8D45b7vP/53ouJk=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUMj+Mfe04YZv4vynNf8uMmZ/5klLkPo4t9qxd74dietg +13tih1lLIwiHEwyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgIns3cPI0HT4yTN19bMGd+7v n9bhZ7TDLlBsbf+xmUeV+8Umtzi+LmRkeKyzftXTd4Fxsqek7jMX++VJS+xU/pOkMUdL20fN6VA 7FwA= X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250517091639.3807875-9-ardb+git@google.com> Subject: [PATCH v4 1/6] x86/cpu: Use a new feature flag for 5 level paging From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds , Brian Gerst , "Kirill A. Shutemov" Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Currently, the LA57 CPU feature flag is taken to mean two different things at once: - whether the CPU implements the LA57 extension, and is therefore capable of supporting 5 level paging; - whether 5 level paging is currently in use. This means the LA57 capability of the hardware is hidden when a LA57 capable CPU is forced to run with 4 levels of paging. It also means the the ordinary CPU capability detection code will happily set the LA57 capability and it needs to be cleared explicitly afterwards to avoid inconsistencies. Separate the two so that the CPU hardware capability can be identified unambigously in all cases. To avoid breaking existing users that might assume that 5 level paging is being used when the "la57" string is visible in /proc/cpuinfo, repurpose that string to mean that 5-level paging is in use, and add a new string la57_capable to indicate that the CPU feature is implemented by the hardware. Signed-off-by: Ard Biesheuvel --- arch/x86/include/asm/cpufeatures.h | 3 ++- arch/x86/include/asm/page_64.h | 2 +- arch/x86/include/asm/pgtable_64_types.h | 2 +- arch/x86/kernel/cpu/common.c | 16 ++-------------- drivers/iommu/amd/init.c | 4 ++-- drivers/iommu/intel/svm.c | 4 ++-- 6 files changed, 10 insertions(+), 21 deletions(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpuf= eatures.h index f67a93fc9391..5c19bee0af11 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -395,7 +395,7 @@ #define X86_FEATURE_AVX512_BITALG (16*32+12) /* "avx512_bitalg" Support fo= r VPOPCNT[B,W] and VPSHUF-BITQMB instructions */ #define X86_FEATURE_TME (16*32+13) /* "tme" Intel Total Memory Encryptio= n */ #define X86_FEATURE_AVX512_VPOPCNTDQ (16*32+14) /* "avx512_vpopcntdq" POPC= NT for vectors of DW/QW */ -#define X86_FEATURE_LA57 (16*32+16) /* "la57" 5-level page tables */ +#define X86_FEATURE_LA57 (16*32+16) /* 57-bit linear addressing */ #define X86_FEATURE_RDPID (16*32+22) /* "rdpid" RDPID instruction */ #define X86_FEATURE_BUS_LOCK_DETECT (16*32+24) /* "bus_lock_detect" Bus Lo= ck detect */ #define X86_FEATURE_CLDEMOTE (16*32+25) /* "cldemote" CLDEMOTE instructio= n */ @@ -483,6 +483,7 @@ #define X86_FEATURE_PREFER_YMM (21*32+ 8) /* Avoid ZMM registers due to d= ownclocking */ #define X86_FEATURE_APX (21*32+ 9) /* Advanced Performance Extensions */ #define X86_FEATURE_INDIRECT_THUNK_ITS (21*32+10) /* Use thunk for indirec= t branches in lower half of cacheline */ +#define X86_FEATURE_5LEVEL_PAGING (21*32+11) /* "la57" Whether 5 levels of= page tables are in use */ =20 /* * BUG word(s) diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h index d3aab6f4e59a..acfa61ad0725 100644 --- a/arch/x86/include/asm/page_64.h +++ b/arch/x86/include/asm/page_64.h @@ -86,7 +86,7 @@ static __always_inline unsigned long task_size_max(void) unsigned long ret; =20 alternative_io("movq %[small],%0","movq %[large],%0", - X86_FEATURE_LA57, + X86_FEATURE_5LEVEL_PAGING, "=3Dr" (ret), [small] "i" ((1ul << 47)-PAGE_SIZE), [large] "i" ((1ul << 56)-PAGE_SIZE)); diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm= /pgtable_64_types.h index eee06f77b245..bf4c33ae24d7 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -34,7 +34,7 @@ static inline bool pgtable_l5_enabled(void) return __pgtable_l5_enabled; } #else -#define pgtable_l5_enabled() cpu_feature_enabled(X86_FEATURE_LA57) +#define pgtable_l5_enabled() cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) #endif /* USE_EARLY_PGTABLE_L5 */ =20 #else diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 8feb8fd2957a..67cdbd916830 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1755,20 +1755,8 @@ static void __init early_identify_cpu(struct cpuinfo= _x86 *c) setup_clear_cpu_cap(X86_FEATURE_PCID); #endif =20 - /* - * Later in the boot process pgtable_l5_enabled() relies on - * cpu_feature_enabled(X86_FEATURE_LA57). If 5-level paging is not - * enabled by this point we need to clear the feature bit to avoid - * false-positives at the later stage. - * - * pgtable_l5_enabled() can be false here for several reasons: - * - 5-level paging is disabled compile-time; - * - it's 32-bit kernel; - * - machine doesn't support 5-level paging; - * - user specified 'no5lvl' in kernel command line. - */ - if (!pgtable_l5_enabled()) - setup_clear_cpu_cap(X86_FEATURE_LA57); + if (native_read_cr4() & X86_CR4_LA57) + setup_force_cpu_cap(X86_FEATURE_5LEVEL_PAGING); =20 detect_nopl(); } diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c index 14aa0d77df26..083fca8f8b97 100644 --- a/drivers/iommu/amd/init.c +++ b/drivers/iommu/amd/init.c @@ -3084,7 +3084,7 @@ static int __init early_amd_iommu_init(void) goto out; =20 /* 5 level guest page table */ - if (cpu_feature_enabled(X86_FEATURE_LA57) && + if (cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) && FIELD_GET(FEATURE_GATS, amd_iommu_efr) =3D=3D GUEST_PGTABLE_5_LEVEL) amd_iommu_gpt_level =3D PAGE_MODE_5_LEVEL; =20 @@ -3691,7 +3691,7 @@ __setup("ivrs_acpihid", parse_ivrs_acpihid); bool amd_iommu_pasid_supported(void) { /* CPU page table size should match IOMMU guest page table size */ - if (cpu_feature_enabled(X86_FEATURE_LA57) && + if (cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) && amd_iommu_gpt_level !=3D PAGE_MODE_5_LEVEL) return false; =20 diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index ba93123cb4eb..1f615e6d06ec 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -37,7 +37,7 @@ void intel_svm_check(struct intel_iommu *iommu) return; } =20 - if (cpu_feature_enabled(X86_FEATURE_LA57) && + if (cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) && !cap_fl5lp_support(iommu->cap)) { pr_err("%s SVM disabled, incompatible paging mode\n", iommu->name); @@ -165,7 +165,7 @@ static int intel_svm_set_dev_pasid(struct iommu_domain = *domain, return PTR_ERR(dev_pasid); =20 /* Setup the pasid table: */ - sflags =3D cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0; + sflags =3D cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) ? PASID_FLAG_FL= 5LP : 0; ret =3D __domain_setup_first_level(iommu, dev, pasid, FLPT_DEFAULT_DID, mm->pgd, sflags, old); --=20 2.49.0.1101.gccaa498523-goog From nobody Fri Dec 19 17:16:03 2025 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9442B1DF271 for ; Sat, 17 May 2025 09:16:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747473416; cv=none; b=dtscmdmWMIkNFe3FMccnwxRr7xj/xng1ccHJ8nrNe6ViTkONyW27Qb3kYziO4oJY4rZADLGvD+Nl8HN9/rCUM73h29Iuz06TX4KW1wDI/8BLVBqcEb9uF8nl2j8rMmqcjTMIQMfwjWwfIxtgIoh/9m0rSTGqWGQYLUpFFFFotrM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747473416; c=relaxed/simple; bh=YTRMDon1bbH1VEcrlHc63BkhAwnfRloZ/j1aJBIxrwI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=j6SnxcoopwSN9ndTLRJqnsLl46d4KidktGVpjEJeqLZLskNjH/89nSA4SgTNy0R13bWIUNr1G3me/zyxevgfB5k4haZ3lt8jiAatLLAAI8CfIW6d++FXg7FGVduVPOGEZf8EHCptn+OMsXT0c0FN8eOmpfTXYWbWuWf4Zorr1P0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=A47o9X4f; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="A47o9X4f" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-43e9b0fd00cso14957515e9.0 for ; Sat, 17 May 2025 02:16:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747473413; x=1748078213; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ZDoNlRqiItndadEJ074/sC5RRxG1d1oYzgqZU9Rxc24=; b=A47o9X4f4GesbzXb3cE23y3GfzoTipEJ40keQiMplKvF8R7cpfUDHMFYgyLNXv+iZt DDR9xdLLoGffQxXCOPzrs7MjlMUOy0JGoHWcivYTuG7mIJORd3864FiJ29E9Dvv810yM 0hAOVlMtjmae4Y94BcnzGzYxqf+9XSn/HF9SxqW6sT9pIaR74Q5ayAC6w5nS76leL1DC +XZX4Z9Zb7McJpFDuYS4+VVMaugI1jZ0WZNO4BKPG55jywlcTvHOHzHaVqg6mY7X3QFE fyTOV7UorJEwovSm5+151qhsr5Tv1ddrOge+cQ6lMFENyRKRGYkjuxAS+PEPxshrOWIX Pstw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747473413; x=1748078213; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ZDoNlRqiItndadEJ074/sC5RRxG1d1oYzgqZU9Rxc24=; b=EVSE7BvRMTIYKV5Iphc7eMu2+Z+yNdBLYu60u1FzumFnw3n6QQm8syUVFm6XOcBv9i 05Bvds7H8FjB2u0lt9mwZqK3rxKeCJwNJBY2MRe8XxxAG60xG7L4tvx3Qlo4PqaFELJq LwaUcG6Pg9YAn9rAVNiz26nULXKKGvex0qRaHNjWyqC8DSQByegeqcaphR8BNKM1uNwr pcffWLmNSKIG1qnj2DYoQFUTjvdqJjD3Hm5xOzT2zN6slBS5WlCvW5fcW942ChOyPU6H 1abTQC6XOJEXdcvT0jKM6XRstYbIvC7ljPWtieqvP7T/M4pu/gG1fDSr8Smv06aKzP9X 8B6g== X-Gm-Message-State: AOJu0YxhzJo2cCL5rHvQjFNYYkNqKuzdqM0iS2FfGx9YQQd51KS/+QwS MGrVmXAFl8MslPD+dQVAR0x6aLW3mSXHQV1tsqjCGL9oZGiJZ2iHD9rw3afsVPBYYbpaVDlvWqT t5cvjN9zgekaKjicReRRzmByIjz6Gqr19QAB72Tv21PZksyG1ibZslwrB9wFXXOdREx4gbPYZec zG4+qVXp7g5Cs9sIvqiuNajossKYTElliv4w== X-Google-Smtp-Source: AGHT+IFwO+w4TLTta2LbHnxhnJljQlZZ+ZqN82hjZFvLHt6QcRo8zELLQbatUM8N6m0AxQuQvDJYte3F X-Received: from wmbea10.prod.google.com ([2002:a05:600c:674a:b0:43c:eba5:f9b3]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3d07:b0:442:f904:1305 with SMTP id 5b1f17b1804b1-442fd93d54emr62847335e9.6.1747473413005; Sat, 17 May 2025 02:16:53 -0700 (PDT) Date: Sat, 17 May 2025 11:16:42 +0200 In-Reply-To: <20250517091639.3807875-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250517091639.3807875-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1352; i=ardb@kernel.org; h=from:subject; bh=C31qclDK+T9suY/4QZ4rdvfNkVOVPbLM73aWDom88co=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUMj+GeywYdb4cfS2Ob2HfG00HuvfPeZ1gGh+qNMW3O2r XabesCro5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAEyk+wkjw8vT18rTbwgfPJl3 9Z6PkmbQppDC8ut9tovbA7btVH3T/52R4UZQ7dWkhQs7fL0PpfksdbBuXMGqfLP18mSGj3xLlt9 p5gAA X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250517091639.3807875-10-ardb+git@google.com> Subject: [PATCH v4 2/6] x86/cpu: Move CPU capability override arrays from BSS to __ro_after_init From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds , Brian Gerst , "Kirill A. Shutemov" Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel In order to allow CPU capability overrides to be set arbitrarily early during the boot, the underlying data objects should not be wiped along with the rest of BSS, and so they will need to be moved out. Given that CPU capabilities are set at init time, and shouldn't be modified after that, move them into __ro_after_init, which is part of the statically initialized kernel image. Signed-off-by: Ard Biesheuvel Reviewed-by: Brian Gerst --- arch/x86/kernel/cpu/common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 67cdbd916830..579d5b84e183 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -704,8 +704,8 @@ static const char *table_lookup_model(struct cpuinfo_x8= 6 *c) } =20 /* Aligned to unsigned long to avoid split lock in atomic bitmap ops */ -__u32 cpu_caps_cleared[NCAPINTS + NBUGINTS] __aligned(sizeof(unsigned long= )); -__u32 cpu_caps_set[NCAPINTS + NBUGINTS] __aligned(sizeof(unsigned long)); +__u32 __ro_after_init cpu_caps_cleared[NCAPINTS + NBUGINTS] __aligned(size= of(unsigned long)); +__u32 __ro_after_init cpu_caps_set[NCAPINTS + NBUGINTS] __aligned(sizeof(u= nsigned long)); =20 #ifdef CONFIG_X86_32 /* The 32-bit entry code needs to find cpu_entry_area. */ --=20 2.49.0.1101.gccaa498523-goog From nobody Fri Dec 19 17:16:03 2025 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CACD11E1A3B for ; Sat, 17 May 2025 09:16:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747473418; cv=none; b=kJ5V86jeOsG3KAl65/6i369Yv/FgGUM9GAxRSxQc4W+prFHkkmCuj3DwBDDlHRjusFaFXPe1aoFvZKcX+nDXEvj8XkpmbQDqEAmqS9YX1z/Vfd1mbfWUNi/dlu5JyIVDW7uXHqfdxXojzhMPtSX0JEE7RN1gr33PPhN8igQX324= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747473418; c=relaxed/simple; bh=ZWEpqE9ktHgh077E7w6k+BuB8NwD4CPevNMgyEcZOpU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=MM1/pRnH6oyKv/nINxhT6c8ikAfDo6P3eaDHnOIHexrcHF2cqtUy0DOdrOn2nUTVbZAbi/5zwkvvzjjeaaw87V6eT0LTbqj3jX/hd9MQXcW24B1y+3vDrCirMNT7qa0FEXC/HqNgLIffulM0EIuAplz478lTbkoM03tP/Cvel0c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=FV4ilwoK; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FV4ilwoK" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-445135eb689so104285e9.2 for ; Sat, 17 May 2025 02:16:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747473415; x=1748078215; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=9M+b2tkrDVZQ50Sd4UaKGxHHMqCit92UPEaKkgiQe7Q=; b=FV4ilwoKgbk3ArTbtQ/gZ3jrj2PQu6AnGHWfR8SjFWKiSs+K8tvplKR2UHbRqALWtk Hfu/ZLvBv/xbqj4/Hv0Bmu/zNbRCYSOih1KkkRm/8jUNoFe3b5+aW+0tKj0ArNouVJz5 HYGEZzm5l8vbjmxfcQzK34JYxFliPDxRhKjFD2kT3TCPlU7xx5IgSjJaUfHsl790jJmH jcKKRHwm27O5Nape2uo+4vqjhEti1h1dXfltIlf7zL8fA2aqoSUDCoTU+bNdcut6EWJf JaQfFTq3a2lGdTlacyBW+NoHG5Cu2MBxn1dyEYkdQUiapMmfjI42nrhs+YbklWyFyAsJ K0Zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747473415; x=1748078215; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9M+b2tkrDVZQ50Sd4UaKGxHHMqCit92UPEaKkgiQe7Q=; b=jvjT9WTWRxRbn0rzLUKwNozQ296VR1X4YgyEBKL7Hd+nQHMJocLMtQ3DkAukg1LNe/ NF6x/GpP6dbQmrXr9zwzTvwZjuTg/Vj0g3kW/eERHgalup9aVsfWotenLCXM5B2uoCZR mPOFd3PqCB1um4zV3pCTa6cszdAj9M3N5QBM00M4CD9zHJo0asrfUCDntoBFXLbeG3LT 6ZhYV8cLAZpxhBBAv7FxmRqHJ3GzhaeHvhZ+O6zel1l+VLht1s+Gtd1x51+W319YY3DT dvjnAxC9lI6YnepHWqG51oW0x6Wws8Ow0t2lFSb6AmPZo1MqUrBAhjI7uOI/HtIkQndg K8Kw== X-Gm-Message-State: AOJu0YyQafXhm75WZ55rNu/FL2xvGNBYiH03ZS1GFWTs+1qM998Z4S6V jO7oMNmTBJvQCBxC4EreI3gai2h1BnQpVb0XFLRdajm7yZupc9a4kEoF4pqV9b0C+5p9ky6frn7 TJK/SOgm93LOEO/83VEhqxZum3x6P9WLI+FGMepq0RVkKtyj32A5SV+GDXugNaxMiyliIV11b6t 0ITEpkcj4+1cSD5oKVAJe/LIOMNBXpI/HOmA== X-Google-Smtp-Source: AGHT+IHHOrp2s3H+1dqsxzAqJxAtSzcdGgUM7/yqWj7wrZnUVpJcPOX6JJnGfYcgfgqUM93D+pbzljVp X-Received: from wmbay14.prod.google.com ([2002:a05:600c:1e0e:b0:442:ddce:d53c]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3e0a:b0:43c:e478:889 with SMTP id 5b1f17b1804b1-442fd5a2c09mr79651695e9.0.1747473415200; Sat, 17 May 2025 02:16:55 -0700 (PDT) Date: Sat, 17 May 2025 11:16:43 +0200 In-Reply-To: <20250517091639.3807875-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250517091639.3807875-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2436; i=ardb@kernel.org; h=from:subject; bh=Es9VKr7BFTHF7N5t1z+PF2rgAMg95nxzCelW5j1DcSU=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUMj+BeH/onzaybzTN/2SdJ1s+etDXm5bl9Mb9/L8t6Qd it2X86XjlIWBjEOBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjCRC8cYGRbXbIjjW+WzWe7T q51vxa3dqy9vb1ieyPX/qYJXRkuDmyIjw7nSP9Esv7fsvfa8xio5RG1+ivI1xp0OMo93X5hVdeX kWjYA X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250517091639.3807875-11-ardb+git@google.com> Subject: [PATCH v4 3/6] x86/cpu: Allow caps to be set arbitrarily early From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds , Brian Gerst , "Kirill A. Shutemov" Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel cpu_feature_enabled() uses a ternary alternative, where the late variant is based on code patching and the early variant accesses the capability field in boot_cpu_data directly. This allows cpu_feature_enabled() to be called quite early, but it still requires that the CPU feature detection code runs before being able to rely on the return value of cpu_feature_enabled(). This is a problem for the implementation of pgtable_l5_enabled(), which is based on cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING), and may be called extremely early. Currently, there is a hacky workaround where some source files that may execute before (but also after) CPU feature detection have a different version of pgtable_l5_enabled(), based on the USE_EARLY_PGTABLE_L5 preprocessor macro. Instead, let's make it possible to set CPU feature arbitrarily early, so that the X86_FEATURE_5LEVEL_PAGING capability can be set before even entering C code, by making sure that boot_cpu_data.x86_capability[] is not [redundantly] wiped again when detecting CPU features. Note that forcing a capability requires setting it in cpu_caps_set[] too, which has been moved out of BSS in a preceding patch. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/cpu/common.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 579d5b84e183..7392a75d85c3 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1708,9 +1708,6 @@ static void __init cpu_parse_early_param(void) */ static void __init early_identify_cpu(struct cpuinfo_x86 *c) { - memset(&c->x86_capability, 0, sizeof(c->x86_capability)); - c->extended_cpuid_level =3D 0; - if (!cpuid_feature()) identify_cpu_without_cpuid(c); =20 @@ -1922,7 +1919,6 @@ static void identify_cpu(struct cpuinfo_x86 *c) c->x86_virt_bits =3D 32; #endif c->x86_cache_alignment =3D c->x86_clflush_size; - memset(&c->x86_capability, 0, sizeof(c->x86_capability)); #ifdef CONFIG_X86_VMX_FEATURE_NAMES memset(&c->vmx_capability, 0, sizeof(c->vmx_capability)); #endif @@ -2084,6 +2080,7 @@ void identify_secondary_cpu(unsigned int cpu) *c =3D boot_cpu_data; c->cpu_index =3D cpu; =20 + memset(&c->x86_capability, 0, sizeof(c->x86_capability)); identify_cpu(c); #ifdef CONFIG_X86_32 enable_sep_cpu(); --=20 2.49.0.1101.gccaa498523-goog From nobody Fri Dec 19 17:16:03 2025 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D29E91E521E for ; Sat, 17 May 2025 09:16:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747473420; cv=none; b=imwdfBlD0DJuVwLnggLhzYyemfZwWB+Br6iIeM1PGnPuZmnMIBv57OQCKwvftmbdgnxH8s59sTJLCmV9wX4viyDi0u7uqkuvAVK2Mf1PH31imPiAOLWVn6NGVuzqKhh6ikEilEWe+ccLDjj8oNVIjY56Aa/drLtIDSjRinV0j+w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747473420; c=relaxed/simple; bh=LrYCuiDJiGB848WGTFaxS545ElR8sRVOtr/mTgwPJn4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=HyKq1oXYFrjDJt4lys2b7XNrN4vQDCFNbvCM2lSCWMGw5yMbJMIeBnnwagJkbrTWuQg3bw7EhZw89dTsYyQ08gpTflR2I/qHAYG++r6BjTdX6Hr5lCy2ke9QOIjqDsHQv3XDtyPIquLtL2os2BysfEOutAKc7y4IrJtAQLHrZbE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=zJOzmc1i; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="zJOzmc1i" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-441c96c1977so18505235e9.0 for ; Sat, 17 May 2025 02:16:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747473417; x=1748078217; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=EvPK23kMajR02E8bh8W17jGFJeTrq3SL6S8Nz/0H/6g=; b=zJOzmc1ip/PAoOShiv0hPgsQ5pgjBo4lpe0+EvVNk6eQQeiW0nuDAH8KfhrS39vQik V9uRdzFy6y/iJcOnuMIjMAXvVrwPTZqI8miRdGu3uPyypGz5kaJ7VWqWhMxrxNd9nbc5 TxNxDHiE1s0BGlYouteKwDwIOfm/CdpOEsNsURl0+vRqLK7ASy32t17zCxYBwBRyy6F/ IsUfD49vvan7i9tmzMFgtIsRn51ig8T4MWzEiX+RYUHakgOxRt6IuYUUJkuf8MJSv1rC z1fyUS5SE0Tiy4rHA5U9EKMSakZ9CmT3B+QVQUwF+ful2TojdGWkN+RjlknSB1//g/eF IZJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747473417; x=1748078217; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=EvPK23kMajR02E8bh8W17jGFJeTrq3SL6S8Nz/0H/6g=; b=hkZosJvW9hM4uNPNWF8BNvsgErR1Gwpm9OnfJiXyIMpeT9Dr+4D0BraWKuDFVHqX0U OsvZshJRCHIjprvs+qxVV8uUKXEGivXQyuD/l6Ce6EFjS1Iqtz3r0sCtoJxkEQbSDbss ek3dLp2kFkON51tcQXxxTKV2M8CxFcgzZFjmH1QNvNEMmK0aZprw8vobTnhizL35+htv vr9YPThlMuLDlGe8vIC0G7tjWmLUCuCfAQG65zsChFqNrIsFXeAhr5x0p1eLOMjOyUCd 6zNOwZJSRHpi847DtP3xAPfPRVSgXA1gxkLI5KeC76r2dFMPJYoLz/f69N7SX5EVmvjI +XNA== X-Gm-Message-State: AOJu0Yy04hby65swiBZB1o6/G2Gjs31/XQrkEc/KC31HXpI8WM7U4IKP yIZ93H3dRJ1s09bMKb0TohvTc0bXRsaLJUGnEpA1O4nBG3WJdly4vWVVnuPG+sEcg414UIjMziv /fc9MhoStILAEO+yAssIp/3dg8tJSBtBT2S9YiAX+8anq7+BeSbsDzndQD/a1Eigq+pJRAK4cEP kIOJzgvbgbevaGUsga0N/9I+ai6I/TzsGJZw== X-Google-Smtp-Source: AGHT+IHgcrTX567r8tsWZ4BIFPA+SZMwnFNqHDNGcBlQeKcxYYes6HeXNfVSIa4ZFft89GzjSotTfp5e X-Received: from wmrn39.prod.google.com ([2002:a05:600c:5027:b0:440:5e01:286b]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:1881:b0:39a:cb5b:78fe with SMTP id ffacd0b85a97d-3a35fe952cdmr4466053f8f.34.1747473417221; Sat, 17 May 2025 02:16:57 -0700 (PDT) Date: Sat, 17 May 2025 11:16:44 +0200 In-Reply-To: <20250517091639.3807875-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250517091639.3807875-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2896; i=ardb@kernel.org; h=from:subject; bh=9ddViHhv7qt25KUQb0TRCKOa0MYESX6EDtdopTEwpsc=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUMj+PfHN9d2Z0r82uh9/4Hn83vLjI0dTKfpPeGI05u/5 WCtpmNuRykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZiIvwjD/9roxvnih5Lm7L29 xLOBn+vagzClLzxTd3wOTv4r4BkVuI/hN/tTPQXD36L3Q5tucjQ8Olb+cZ/dm0lJzPN3fsjqeP/ tHDcA X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250517091639.3807875-12-ardb+git@google.com> Subject: [PATCH v4 4/6] x86/boot: Set 5-level paging CPU cap before entering C code From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds , Brian Gerst , "Kirill A. Shutemov" Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel In order for pgtable_l5_enabled() to be reliable wherever it is used and however early, set the associated CPU capability from asm code before entering the startup C code. Signed-off-by: Ard Biesheuvel --- arch/x86/include/asm/cpufeature.h | 12 +++++++++--- arch/x86/kernel/cpu/common.c | 3 --- arch/x86/kernel/head_64.S | 13 +++++++++++++ 3 files changed, 22 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/cpufeature.h b/arch/x86/include/asm/cpufe= ature.h index 893cbca37fe9..1b5de40e7bf7 100644 --- a/arch/x86/include/asm/cpufeature.h +++ b/arch/x86/include/asm/cpufeature.h @@ -2,10 +2,10 @@ #ifndef _ASM_X86_CPUFEATURE_H #define _ASM_X86_CPUFEATURE_H =20 +#ifdef __KERNEL__ +#ifndef __ASSEMBLER__ #include =20 -#if defined(__KERNEL__) && !defined(__ASSEMBLER__) - #include #include #include @@ -137,5 +137,11 @@ static __always_inline bool _static_cpu_has(u16 bit) #define CPU_FEATURE_TYPEVAL boot_cpu_data.x86_vendor, boot_cpu_data.x86, \ boot_cpu_data.x86_model =20 -#endif /* defined(__KERNEL__) && !defined(__ASSEMBLER__) */ +#else /* !defined(__ASSEMBLER__) */ + .macro setup_force_cpu_cap, cap:req + btsl $\cap % 32, boot_cpu_data+CPUINFO_x86_capability+4*(\cap / 32)(%rip) + btsl $\cap % 32, cpu_caps_set+4*(\cap / 32)(%rip) + .endm +#endif /* !defined(__ASSEMBLER__) */ +#endif /* defined(__KERNEL__) */ #endif /* _ASM_X86_CPUFEATURE_H */ diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 7392a75d85c3..6846a83fa1b6 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1752,9 +1752,6 @@ static void __init early_identify_cpu(struct cpuinfo_= x86 *c) setup_clear_cpu_cap(X86_FEATURE_PCID); #endif =20 - if (native_read_cr4() & X86_CR4_LA57) - setup_force_cpu_cap(X86_FEATURE_5LEVEL_PAGING); - detect_nopl(); } =20 diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 069420853304..21a713fac86c 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -27,6 +27,7 @@ #include #include #include +#include =20 /* * We are not able to switch in one step to the final KERNEL ADDRESS SPACE @@ -58,6 +59,18 @@ SYM_CODE_START_NOALIGN(startup_64) */ mov %rsi, %r15 =20 + /* + * Set the X86_FEATURE_5LEVEL_PAGING capability before calling into the + * C code, to give it a consistent view of any global pseudo-constants + * that are derived from pgtable_l5_enabled(). + */ + mov %cr4, %rax + btl $X86_CR4_LA57_BIT, %eax + jnc 0f + + setup_force_cpu_cap X86_FEATURE_5LEVEL_PAGING +0: + /* Set up the stack for verify_cpu() */ leaq __top_init_kernel_stack(%rip), %rsp =20 --=20 2.49.0.1101.gccaa498523-goog From nobody Fri Dec 19 17:16:03 2025 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A79DE1E7648 for ; Sat, 17 May 2025 09:17:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747473422; cv=none; b=YA36xeJ4bzGWm1R7pQT1ugBauhv3sNnnxOPCdISch+knwEswR97KL0m7s1q1EqfIk0H9kpC6HERB2kkMcHT/gHzXFCNaRV2v4ONfBWpMGlbdlxA1xDgnkPtlSgHTUXwLWGv3kO0z5M+Kkk3ObrV2Vj94j/EngsgkkOO6Hrulvd0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747473422; c=relaxed/simple; bh=3EwEj/6lMOnvgATJ6r0vVrXxYmo66nwJZa3pw0bSmpc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=KhRa+ggYnAGVJqilbTuekf3U0B+rA5HG12uaH/Sua1KROlCqQBJU5GHJ1Eg1LG7hg54nNU6HMfwOVaiL71oJfFtvQytryI/PB9GmFpMDeqzIfE7+rN3ifKQF8E5rT2c9VFOkXjwkxuqTh+kWxWyeQSkuVGiDxWQDNOF+8yep4Gs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=YAxTjO6t; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YAxTjO6t" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-43d007b2c79so22261745e9.2 for ; Sat, 17 May 2025 02:17:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747473419; x=1748078219; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=x08rrTDWY9cAUMy0PmmEcseVvr8AaWm8xVAdcnsEt+s=; b=YAxTjO6t26LDMFzM+qEJNew+m1syZ0eJ+S/1MKOUu1qvu1DtvucOs2uw+6sgSA0P0m bzcQxplyARrNtq9AIzywucisDlLZAarBJkIlGs+ShcmG3C5Cv0+3SdIy1aOfcUBygADE 2F2aBMRXyp784x+0ip1KHaQ/0l+F11SxInBcvMXntqsURZFcvTMdsMSuJJ5YH/BWcBA+ pHb40K0QI3yjylgV+Z2lFphy/WJRukDr5+ATfTXvXaqc/iJEAOvI5gsFdUmkrNWvlqns iH8rncFdlEdNzCGHXjmeVyASCjgU702w2j6/8Z3m0Ll6GhAvY46gwu/HCAJvQBlGEd/L FNuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747473419; x=1748078219; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=x08rrTDWY9cAUMy0PmmEcseVvr8AaWm8xVAdcnsEt+s=; b=lk9uJqrlKNtdWY51TmiW43jusAQtxwDMuo7Mq3vtSjlD2CkKZvhuFHfQFyidYLxSr7 U4GKEK+a/KxLX5ean4Ovt5JT96SqqZ0dhab8Jy6BiTdwFKqQMAGnMp80b6e7L/UW8BxE X6w+h+iBjgbNc/xnRap60wddAptWds4gW/V1B/7Lvi1zuxq/hg39KzDb8+2tyt/3ZCK6 5+hZ/URrotP99fnB+MqO3NDQYWTZPD7b9xGU8YhZzieftWeuIUW4ete5RaHyrHftqlNX 40nUdhqUBN/Q/CyhC9fB9NI7NQXyHHQnGHqMx3X47kL0qc0oKqnx+TWK6keZDHpjskZa 0NBg== X-Gm-Message-State: AOJu0YzXZA3m2Md4nMIFtdgzz2ZHiwOLkHfTf2E2qOO9hXTyf/lk40bw jhBv4qBIlsAfd43CuGgMDh0wLSj69wjVejJ3S5ovb+gf3xzWONWP05VyKmoK9meySjY9XyzNtcE lfhVHKbniYOAOTwDELCh2BSdJPUu4Vz0ydcgzqUXdZJUv8zS1ll38M3WmxhytfVIukeXKqCvHC7 ts3K8GUWToWxmnFQhdogfSexI/0B9Kxq2vwQ== X-Google-Smtp-Source: AGHT+IHKZJlY9Uv9u+cGzDV11hKu8wrzHdh212AZ6dF6ZNutFJXQwheB6bdOCI8Fb16m2tP/nCBgQxNr X-Received: from wmsd12.prod.google.com ([2002:a05:600c:3acc:b0:442:cab2:15c3]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1d81:b0:43c:fc04:6d35 with SMTP id 5b1f17b1804b1-442fd606b8emr59059445e9.4.1747473419209; Sat, 17 May 2025 02:16:59 -0700 (PDT) Date: Sat, 17 May 2025 11:16:45 +0200 In-Reply-To: <20250517091639.3807875-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250517091639.3807875-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=5985; i=ardb@kernel.org; h=from:subject; bh=QLFdRUPvLnW8mE/Zv58BLsDVSP21dwXqh8N0Ru2s09I=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUMj+M/2KVzeL2vUeX7wzZHwLzt+9rW/YOmklPzfqe94K 2/fD2bsKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABOZ+IWR4e6aafKHlrCyJd+c 69vMc3/Lp8qjaa2TzzB4n+OTW/Lc/RzDP5N9G71XX5mzxuAiW3HUNXfu8vs/Z8cevJE63Vzu/T7 BxbwA X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250517091639.3807875-13-ardb+git@google.com> Subject: [PATCH v4 5/6] x86/boot: Drop the early variant of pgtable_l5_enabled() From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds , Brian Gerst , "Kirill A. Shutemov" Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Now that cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) is guaranteed to produce the correct value even during early boot, there is no longer a need for an early variant and so it can be dropped. For the decompressor, fall back to testing the CR4.LA57 control register bit directly. Note that this removes the need to disable KASAN temporarily while applying alternatives, given that any constant or VA space dimension derived from pgtable_l5_enabled() will now always consistently produce the correct value. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/misc.h | 5 ++--- arch/x86/boot/startup/sme.c | 9 -------- arch/x86/include/asm/pgtable_64_types.h | 22 ++++---------------- arch/x86/kernel/alternative.c | 12 ----------- arch/x86/kernel/cpu/common.c | 2 -- arch/x86/kernel/head64.c | 3 --- arch/x86/mm/kasan_init_64.c | 3 --- 7 files changed, 6 insertions(+), 50 deletions(-) diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/mis= c.h index db1048621ea2..65e7ff5d7ded 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -16,9 +16,6 @@ =20 #define __NO_FORTIFY =20 -/* cpu_feature_enabled() cannot be used this early */ -#define USE_EARLY_PGTABLE_L5 - /* * Boot stub deals with identity mappings, physical and virtual addresses = are * the same, so override these defines. @@ -28,6 +25,8 @@ #define __pa(x) ((unsigned long)(x)) #define __va(x) ((void *)((unsigned long)(x))) =20 +#define pgtable_l5_enabled() (native_read_cr4() & X86_CR4_LA57) + #include #include #include diff --git a/arch/x86/boot/startup/sme.c b/arch/x86/boot/startup/sme.c index 70ea1748c0a7..a6c25d005991 100644 --- a/arch/x86/boot/startup/sme.c +++ b/arch/x86/boot/startup/sme.c @@ -25,15 +25,6 @@ #undef CONFIG_PARAVIRT_XXL #undef CONFIG_PARAVIRT_SPINLOCKS =20 -/* - * This code runs before CPU feature bits are set. By default, the - * pgtable_l5_enabled() function uses bit X86_FEATURE_LA57 to determine if - * 5-level paging is active, so that won't work here. USE_EARLY_PGTABLE_L5 - * is provided to handle this situation and, instead, use a variable that - * has been set by the early boot code. - */ -#define USE_EARLY_PGTABLE_L5 - #include #include #include diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm= /pgtable_64_types.h index bf4c33ae24d7..a3f7ec94012b 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -23,29 +23,15 @@ typedef struct { pmdval_t pmd; } pmd_t; =20 extern unsigned int __pgtable_l5_enabled; =20 -#ifdef CONFIG_X86_5LEVEL -#ifdef USE_EARLY_PGTABLE_L5 -/* - * cpu_feature_enabled() is not available in early boot code. - * Use variable instead. - */ -static inline bool pgtable_l5_enabled(void) -{ - return __pgtable_l5_enabled; -} -#else -#define pgtable_l5_enabled() cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) -#endif /* USE_EARLY_PGTABLE_L5 */ - -#else -#define pgtable_l5_enabled() 0 -#endif /* CONFIG_X86_5LEVEL */ - extern unsigned int pgdir_shift; extern unsigned int ptrs_per_p4d; =20 #endif /* !__ASSEMBLER__ */ =20 +#ifndef pgtable_l5_enabled +#define pgtable_l5_enabled() cpu_feature_enabled(X86_FEATURE_5LEVEL_PAGING) +#endif + #ifdef CONFIG_X86_5LEVEL =20 /* diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 2385528792b2..e39823d8d1ae 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -589,16 +589,6 @@ void __init_or_module noinline apply_alternatives(stru= ct alt_instr *start, =20 DPRINTK(ALT, "alt table %px, -> %px", start, end); =20 - /* - * In the case CONFIG_X86_5LEVEL=3Dy, KASAN_SHADOW_START is defined using - * cpu_feature_enabled(X86_FEATURE_LA57) and is therefore patched here. - * During the process, KASAN becomes confused seeing partial LA57 - * conversion and triggers a false-positive out-of-bound report. - * - * Disable KASAN until the patching is complete. - */ - kasan_disable_current(); - /* * The scan order should be from start to end. A later scanned * alternative code can overwrite previously scanned alternative code. @@ -666,8 +656,6 @@ void __init_or_module noinline apply_alternatives(struc= t alt_instr *start, =20 text_poke_early(instr, insn_buff, insn_buff_sz); } - - kasan_enable_current(); } =20 static inline bool is_jcc32(struct insn *insn) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 6846a83fa1b6..65ee1de785ac 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1,6 +1,4 @@ // SPDX-License-Identifier: GPL-2.0-only -/* cpu_feature_enabled() cannot be used this early */ -#define USE_EARLY_PGTABLE_L5 =20 #include #include diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 9f617be64fa9..455f12850778 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -5,9 +5,6 @@ * Copyright (C) 2000 Andrea Arcangeli SuSE */ =20 -/* cpu_feature_enabled() cannot be used this early */ -#define USE_EARLY_PGTABLE_L5 - #include #include #include diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 0539efd0d216..7c4fafbd52cc 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -1,9 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 #define pr_fmt(fmt) "kasan: " fmt =20 -/* cpu_feature_enabled() cannot be used this early */ -#define USE_EARLY_PGTABLE_L5 - #include #include #include --=20 2.49.0.1101.gccaa498523-goog From nobody Fri Dec 19 17:16:03 2025 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F194F1EE03D for ; Sat, 17 May 2025 09:17:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747473424; cv=none; b=uAplWDdc3C7HDcJPKw7UcrJ8yOHIw77CGMvDAKCvDbzcXTLy38uJAghtx+LkBjPHV6R3fBksnS17JeayKeB776mVxKlfCEBUtAaFuttunVkuGIzMU7/aow4TuSE3fLMzOnm89Rw3ko0VNlt9xbR09NyLDG03xLnxyxngzGMntsc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747473424; c=relaxed/simple; bh=0t8T1QbYeQN2hvCtMCujog7MqzzzxGy3Vuk4n4AIUGs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Xq6ga28PsxOgjGvFTc98qDUvSa1bJvOZhcxsPG/uQ2DAHtCQbtvYEpGb3M2yvHqxXS7TyTtjiEbuwGCrACwq1HctGKLtxbVb3w9YUShtZ4zXe+zjB2n1XWgxGH1jZulPUm9Ex1uSS3vb5/OcqNGaQIceZxortcvUvomthLk7tpI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Y/PLFQyQ; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Y/PLFQyQ" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-3a367b3bc13so84452f8f.3 for ; Sat, 17 May 2025 02:17:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747473421; x=1748078221; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=1yfXKW9qA3ci94z4bkzfydUvCwbLjyFz+98FQfnm8O8=; b=Y/PLFQyQd1kO0RijHZRw0P78hqh288/O4+6Z4Lieoy+X09CSKtCQZsu7MZykzRv6Ww Tb8ODpVaiJ1N9JflzRpPNCn1nNPfsBxTOpmc19mZHU8rEQcNSBdMpUCfKIoQmMItuImI i2eFXa5YxkSfPcdCz8vrqJhTh4P0n//LkrTyXp269QoSblEGYqA+oK6Dapi02iepW8ys pK8e0pd2aOgQeo7tEvDHaP28u89euI/RmBZqsbTGmWGzBtt0m1JFuLeSURdCkPZp9hT7 q8FyTX727RAEEBKhnLWKRtFfdmfQFYGgxA9seNk1y1BpTg6o40Xl1NWBSCfpjsxmcWrb UUjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747473421; x=1748078221; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1yfXKW9qA3ci94z4bkzfydUvCwbLjyFz+98FQfnm8O8=; b=wMPzRzo041xxCS4QSDkWMW+r4sLJM5M5YSUqPBHJPG/pOp7lN4A5b2nwirZpcLbhIl 5N8vQOqxFq1nUuSa6RdW0YwIyPH+TEDzvWnLDtxeJ06inxSbCYY6uBazocvIde2M1cxP DukFOoa26PdkJNu5C/0cl89ZTEUC4WWGZ4bHDXkt/MVi4Zc2U1YhNgNpC0NDcDCt3Cab L+kRhZtdwUJzRDpI0fEnT4QeH0ek0MZL7kK6b9PLNKh7VNKKYOEgZzBP1v348ED1K+fV 5EFfNRZ5vMx8ZaVg3U0b9QepzABBGlhCqrITpxGdEod2Svu1KaB4Is+05/2oKtFImzpX 9l4g== X-Gm-Message-State: AOJu0Yynl066XIoQV3HdFWR6dMmuDL5ubOX2MjVeMPs/fzyueK5hyfFN TvOM0Pzyhl7davjggwMZxDcFmYK0mJp+vK4jXK8au4NlL12O3YrITIbBlRYFuGmaVaoTkWdKR3S 5WrvSgV4ezE88LN9MTBgjZCw8/spSny3Z6IlS9phyTZu5SL2RmZN5sRM3Q5Q6DKMMZ08c4ugjw+ 995SyU7RCVlf0wYZ6a3fo28k8b1qupRdzLgA== X-Google-Smtp-Source: AGHT+IGGrQXFH7FGCf2kNULNg9gKfn9HqOMF6Sks7YuziECohlfEXUWRIhlJ3RLm6GwWWCniuGJpeAse X-Received: from wmbfj10.prod.google.com ([2002:a05:600c:c8a:b0:440:60ac:3f40]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:230a:b0:3a3:64d2:cee2 with SMTP id ffacd0b85a97d-3a364d2d10dmr2619421f8f.45.1747473421223; Sat, 17 May 2025 02:17:01 -0700 (PDT) Date: Sat, 17 May 2025 11:16:46 +0200 In-Reply-To: <20250517091639.3807875-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250517091639.3807875-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=5410; i=ardb@kernel.org; h=from:subject; bh=rA4TspQiCNXQdmEQ/LEBR6E/QrtzS3QcW5m4BFJZm+A=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUMj+K+6uFvvpC2XGnn/rOF9P/fUudVPlty8dT1z0uwec efDAqcyOkpZGMQ4GGTFFFkEZv99t/P0RKla51myMHNYmUCGMHBxCsBEGqUY/qna3Tn3NtnjI+fT Frb5L8q+uu2wK1nFMj3tjmT7+5jHvvwM/31muotEVN7v9BEL/Je54nHpsVX8VtrO2XH/vJIaN2y cwwUA X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250517091639.3807875-14-ardb+git@google.com> Subject: [PATCH v4 6/6] x86/boot: Drop 5-level paging related variables and early updates From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Ingo Molnar , Linus Torvalds , Brian Gerst , "Kirill A. Shutemov" Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The variable __pgtable_l5_enabled is no longer used so it can be dropped. Along with it, drop ptrs_per_p4d and pgdir_shift, and replace any references to those with expressions based on pgtable_l5_enabled(). This ensures that all observers see values that are mutually consistent. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/misc.h | 1 - arch/x86/boot/compressed/pgtable_64.c | 12 ----------- arch/x86/boot/startup/map_kernel.c | 21 +------------------- arch/x86/include/asm/pgtable_64_types.h | 9 ++------- arch/x86/kernel/head64.c | 8 -------- 5 files changed, 3 insertions(+), 48 deletions(-) diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/mis= c.h index 65e7ff5d7ded..8c3e9114a639 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -188,7 +188,6 @@ static inline int count_immovable_mem_regions(void) { r= eturn 0; } #endif =20 /* ident_map_64.c */ -extern unsigned int __pgtable_l5_enabled, pgdir_shift, ptrs_per_p4d; extern void kernel_add_identity_map(unsigned long start, unsigned long end= ); =20 /* Used by PAGE_KERN* macros: */ diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compress= ed/pgtable_64.c index 5a6c7a190e5b..591d28f2feb6 100644 --- a/arch/x86/boot/compressed/pgtable_64.c +++ b/arch/x86/boot/compressed/pgtable_64.c @@ -10,13 +10,6 @@ #define BIOS_START_MIN 0x20000U /* 128K, less than this is insane */ #define BIOS_START_MAX 0x9f000U /* 640K, absolute maximum */ =20 -#ifdef CONFIG_X86_5LEVEL -/* __pgtable_l5_enabled needs to be in .data to avoid being cleared along = with .bss */ -unsigned int __section(".data") __pgtable_l5_enabled; -unsigned int __section(".data") pgdir_shift =3D 39; -unsigned int __section(".data") ptrs_per_p4d =3D 1; -#endif - /* Buffer to preserve trampoline memory */ static char trampoline_save[TRAMPOLINE_32BIT_SIZE]; =20 @@ -127,11 +120,6 @@ asmlinkage void configure_5level_paging(struct boot_pa= rams *bp, void *pgtable) native_cpuid_eax(0) >=3D 7 && (native_cpuid_ecx(7) & (1 << (X86_FEATURE_LA57 & 31)))) { l5_required =3D true; - - /* Initialize variables for 5-level paging */ - __pgtable_l5_enabled =3D 1; - pgdir_shift =3D 48; - ptrs_per_p4d =3D 512; } =20 /* diff --git a/arch/x86/boot/startup/map_kernel.c b/arch/x86/boot/startup/map= _kernel.c index 905e8734b5a3..056de4766006 100644 --- a/arch/x86/boot/startup/map_kernel.c +++ b/arch/x86/boot/startup/map_kernel.c @@ -14,25 +14,6 @@ extern pmd_t early_dynamic_pgts[EARLY_DYNAMIC_PAGE_TABLES][PTRS_PER_PMD]; extern unsigned int next_early_pgt; =20 -static inline bool check_la57_support(void) -{ - if (!IS_ENABLED(CONFIG_X86_5LEVEL)) - return false; - - /* - * 5-level paging is detected and enabled at kernel decompression - * stage. Only check if it has been enabled there. - */ - if (!(native_read_cr4() & X86_CR4_LA57)) - return false; - - __pgtable_l5_enabled =3D 1; - pgdir_shift =3D 48; - ptrs_per_p4d =3D 512; - - return true; -} - static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdval_t *pmd, unsigned long p2v_offset) @@ -102,7 +83,7 @@ unsigned long __head __startup_64(unsigned long p2v_offs= et, bool la57; int i; =20 - la57 =3D check_la57_support(); + la57 =3D pgtable_l5_enabled(); =20 /* Is the address too large? */ if (physaddr >> MAX_PHYSMEM_BITS) diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm= /pgtable_64_types.h index a3f7ec94012b..a873dec1a615 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -21,11 +21,6 @@ typedef unsigned long pgprotval_t; typedef struct { pteval_t pte; } pte_t; typedef struct { pmdval_t pmd; } pmd_t; =20 -extern unsigned int __pgtable_l5_enabled; - -extern unsigned int pgdir_shift; -extern unsigned int ptrs_per_p4d; - #endif /* !__ASSEMBLER__ */ =20 #ifndef pgtable_l5_enabled @@ -37,7 +32,7 @@ extern unsigned int ptrs_per_p4d; /* * PGDIR_SHIFT determines what a top-level page table entry can map */ -#define PGDIR_SHIFT pgdir_shift +#define PGDIR_SHIFT (pgtable_l5_enabled() ? 48 : 39) #define PTRS_PER_PGD 512 =20 /* @@ -45,7 +40,7 @@ extern unsigned int ptrs_per_p4d; */ #define P4D_SHIFT 39 #define MAX_PTRS_PER_P4D 512 -#define PTRS_PER_P4D ptrs_per_p4d +#define PTRS_PER_P4D (pgtable_l5_enabled() ? MAX_PTRS_PER_P4D : 1) #define P4D_SIZE (_AC(1, UL) << P4D_SHIFT) #define P4D_MASK (~(P4D_SIZE - 1)) =20 diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 455f12850778..137c93498601 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -48,14 +48,6 @@ unsigned int __initdata next_early_pgt; SYM_PIC_ALIAS(next_early_pgt); pmdval_t early_pmd_flags =3D __PAGE_KERNEL_LARGE & ~(_PAGE_GLOBAL | _PAGE_= NX); =20 -#ifdef CONFIG_X86_5LEVEL -unsigned int __pgtable_l5_enabled __ro_after_init; -unsigned int pgdir_shift __ro_after_init =3D 39; -EXPORT_SYMBOL(pgdir_shift); -unsigned int ptrs_per_p4d __ro_after_init =3D 1; -EXPORT_SYMBOL(ptrs_per_p4d); -#endif - unsigned long page_offset_base __ro_after_init =3D __PAGE_OFFSET_BASE_L4; EXPORT_SYMBOL(page_offset_base); unsigned long vmalloc_base __ro_after_init =3D __VMALLOC_BASE_L4; --=20 2.49.0.1101.gccaa498523-goog