From nobody Thu May 14 08:07:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90FEDC433EF for ; Tue, 19 Apr 2022 11:17:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350977AbiDSLUP (ORCPT ); Tue, 19 Apr 2022 07:20:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349339AbiDSLUJ (ORCPT ); Tue, 19 Apr 2022 07:20:09 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6B991F615; Tue, 19 Apr 2022 04:17:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1650367047; x=1681903047; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ApOZ19KKY/ZegfPl38ecJIUqs9w6pgZB1hZZKzFij3s=; b=BRnt9L87HfORXI/r+ybbi1Xfnq1hUAEHA0lvbHZ4UtNgVtZQJZ8InIIo gtwE0IWi1v0dsBktznltjJprAVgT8hYksmtn2yHH1Skj1wBYVWMbHop6m U0383P4ZqI2Ordkz1nT2qCPs9bxSry+CXh7lfS17hzDkg/UDC7Cr3xMJ/ bcMhFBlT+VgzUhqZMBDEEGeg5xsPfWGVezUxHc9R92LrmiYw9hhmPy9cm wfJpXWu/tCazxmN41YZU7TW/jlw0IuYGNPv5VonOaPR7azNqRlRd7ze0j oIJvR0J5hIqtoc4uYtlBx5WTxQvXe+v6T4Ql3xPwK4bYdx6a/fvOnaHxH A==; X-IronPort-AV: E=McAfee;i="6400,9594,10321"; a="350189753" X-IronPort-AV: E=Sophos;i="5.90,272,1643702400"; d="scan'208";a="350189753" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Apr 2022 04:17:16 -0700 X-IronPort-AV: E=Sophos;i="5.90,272,1643702400"; d="scan'208";a="554683738" Received: from csambran-mobl.amr.corp.intel.com (HELO khuang2-desk.gar.corp.intel.com) ([10.254.58.20]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Apr 2022 04:17:14 -0700 From: Kai Huang To: kvm@vger.kernel.org Cc: linux-kernel@vger.kernel.org, pbonzini@redhat.com, seanjc@google.com, vkuznets@redhat.com, jmattson@google.com, joro@8bytes.org, wanpengli@tencent.com Subject: [PATCH 1/3] KVM: x86/mmu: Rename reset_rsvds_bits_mask() Date: Tue, 19 Apr 2022 23:17:02 +1200 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Rename reset_rsvds_bits_mask() to reset_guest_rsvds_bits_mask() to make it clearer that it resets the reserved bits check for guest's page table entries. Signed-off-by: Kai Huang --- arch/x86/kvm/mmu/mmu.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 69a30d6d1e2b..2931785f1e73 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4369,8 +4369,8 @@ static bool guest_can_use_gbpages(struct kvm_vcpu *vc= pu) guest_cpuid_has(vcpu, X86_FEATURE_GBPAGES); } =20 -static void reset_rsvds_bits_mask(struct kvm_vcpu *vcpu, - struct kvm_mmu *context) +static void reset_guest_rsvds_bits_mask(struct kvm_vcpu *vcpu, + struct kvm_mmu *context) { __reset_rsvds_bits_mask(&context->guest_rsvd_check, vcpu->arch.reserved_gpa_bits, @@ -4669,7 +4669,7 @@ static void reset_guest_paging_metadata(struct kvm_vc= pu *vcpu, if (!is_cr0_pg(mmu)) return; =20 - reset_rsvds_bits_mask(vcpu, mmu); + reset_guest_rsvds_bits_mask(vcpu, mmu); update_permission_bitmask(mmu, false); update_pkru_bitmask(mmu); } --=20 2.35.1 From nobody Thu May 14 08:07:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F802C4332F for ; Tue, 19 Apr 2022 11:17:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351005AbiDSLUV (ORCPT ); Tue, 19 Apr 2022 07:20:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47748 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350951AbiDSLUK (ORCPT ); Tue, 19 Apr 2022 07:20:10 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E022C1F622; Tue, 19 Apr 2022 04:17:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1650367047; x=1681903047; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kWejiPF82YrH+Ty7R0Fkl07BkhwqHcnwr3vq1zjTRTA=; b=kqnboPV1GyLfUiuZpyPlEGY1AH+L6U2OOXYRx9J5yY79XUmYEChEOt7s XextP0iuQiVh050e/6zY0CpFykego4hyrHIOhXI54fA2XPEkb4HTdCS83 kbY90qufC7T8BDh37pXSl96PS5RdcznVqK/fixYMvVOmIAsxtCFpd4gom q2ODnTed63mRUdNdej12ubQ805HlYlO45UbXlpA3cpcmQC1XhoZ7NaTfh IWpwkBHw5OH+xof5kICiiB7rfj5SQTC21fcYMWQfubWp31XKw+kJQNowO 5uaMO6OSupuZu/ILQYK/ScqA5a2qRAhximb4klm/9YRWL2JRuQ3gWyh7z Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10321"; a="350189761" X-IronPort-AV: E=Sophos;i="5.90,272,1643702400"; d="scan'208";a="350189761" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Apr 2022 04:17:18 -0700 X-IronPort-AV: E=Sophos;i="5.90,272,1643702400"; d="scan'208";a="554683751" Received: from csambran-mobl.amr.corp.intel.com (HELO khuang2-desk.gar.corp.intel.com) ([10.254.58.20]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Apr 2022 04:17:16 -0700 From: Kai Huang To: kvm@vger.kernel.org Cc: linux-kernel@vger.kernel.org, pbonzini@redhat.com, seanjc@google.com, vkuznets@redhat.com, jmattson@google.com, joro@8bytes.org, wanpengli@tencent.com Subject: [PATCH 2/3] KVM: x86/mmu: Add shadow_me_value and repurpose shadow_me_mask Date: Tue, 19 Apr 2022 23:17:03 +1200 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Intel Multi-Key Total Memory Encryption (MKTME) repurposes couple of high bits of physical address bits as 'KeyID' bits. Intel Trust Domain Extentions (TDX) further steals part of MKTME KeyID bits as TDX private KeyID bits. TDX private KeyID bits cannot be set in any mapping in the host kernel since they can only be accessed by software running inside a new CPU isolated mode. And unlike to AMD's SME, host kernel doesn't set any legacy MKTME KeyID bits to any mapping either. Therefore, it's not legitimate for KVM to set any KeyID bits in SPTE which maps guest memory. KVM maintains shadow_zero_check bits to represent which bits must be zero for SPTE which maps guest memory. MKTME KeyID bits should be set to shadow_zero_check. Currently, shadow_me_mask is used by AMD to set the sme_me_mask to SPTE, and shadow_me_shadow is excluded from shadow_zero_check. So initializing shadow_me_mask to represent all MKTME keyID bits doesn't work for VMX (as oppositely, they must be set to shadow_zero_check). Introduce a new 'shadow_me_value' to replace existing shadow_me_mask, and repurpose shadow_me_mask as 'all possible memory encryption bits'. The new schematic of them will be: - shadow_me_value: the memory encryption bit(s) that will be set to the SPTE (the original shadow_me_mask). - shadow_me_mask: all possible memory encryption bits (which is a super set of shadow_me_value). - For now, shadow_me_value is supposed to be set by SVM and VMX respectively, and it is a constant during KVM's life time. This perhaps doesn't fit MKTME but for now host kernel doesn't support it (and perhaps will never do). - Bits in shadow_me_mask are set to shadow_zero_check, except the bits in shadow_me_value. Introduce a new helper kvm_mmu_set_me_spte_mask() to initialize them. Replace shadow_me_mask with shadow_me_value in almost all code paths, except the one in PT64_PERM_MASK, which is used by need_remote_flush() to determine whether remote TLB flush is needed. This should still use shadow_me_mask as any encryption bit change should need a TLB flush. And for AMD, move initializing shadow_me_value/shadow_me_mask from kvm_mmu_reset_all_pte_masks() to svm_hardware_setup(). Signed-off-by: Kai Huang --- arch/x86/kvm/mmu.h | 1 + arch/x86/kvm/mmu/mmu.c | 16 ++++++++++++---- arch/x86/kvm/mmu/spte.c | 21 ++++++++++++++++----- arch/x86/kvm/mmu/spte.h | 1 + arch/x86/kvm/svm/svm.c | 3 +++ 5 files changed, 33 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 671cfeccf04e..60c669494c4c 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -66,6 +66,7 @@ static __always_inline u64 rsvd_bits(int s, int e) } =20 void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_= mask); +void kvm_mmu_set_me_spte_mask(u64 me_value, u64 me_mask); void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only); =20 void kvm_init_mmu(struct kvm_vcpu *vcpu); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 2931785f1e73..410bce591c59 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3489,7 +3489,7 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vc= pu) * or a PAE 3-level page table. In either case we need to be aware that * the shadow page table may be a PAE or a long mode page table. */ - pm_mask =3D PT_PRESENT_MASK | shadow_me_mask; + pm_mask =3D PT_PRESENT_MASK | shadow_me_value; if (mmu->root_role.level >=3D PT64_ROOT_4LEVEL) { pm_mask |=3D PT_ACCESSED_MASK | PT_WRITABLE_MASK | PT_USER_MASK; =20 @@ -4458,8 +4458,16 @@ static void reset_shadow_zero_bits_mask(struct kvm_v= cpu *vcpu, return; =20 for (i =3D context->root_role.level; --i >=3D 0;) { - shadow_zero_check->rsvd_bits_mask[0][i] &=3D ~shadow_me_mask; - shadow_zero_check->rsvd_bits_mask[1][i] &=3D ~shadow_me_mask; + /* + * So far shadow_me_value is a constant during KVM's life + * time. Bits in shadow_me_value are allowed to be set. + * Bits in shadow_me_mask but not in shadow_me_value are + * not allowed to be set. + */ + shadow_zero_check->rsvd_bits_mask[0][i] |=3D shadow_me_mask; + shadow_zero_check->rsvd_bits_mask[1][i] |=3D shadow_me_mask; + shadow_zero_check->rsvd_bits_mask[0][i] &=3D ~shadow_me_value; + shadow_zero_check->rsvd_bits_mask[1][i] &=3D ~shadow_me_value; } =20 } @@ -5546,7 +5554,7 @@ static int __kvm_mmu_create(struct kvm_vcpu *vcpu, st= ruct kvm_mmu *mmu) if (!tdp_enabled) set_memory_decrypted((unsigned long)mmu->pae_root, 1); else - WARN_ON_ONCE(shadow_me_mask); + WARN_ON_ONCE(shadow_me_value); =20 for (i =3D 0; i < 4; ++i) mmu->pae_root[i] =3D INVALID_PAE_ROOT; diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 4739b53c9734..a6da94f441c4 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -33,6 +33,7 @@ u64 __read_mostly shadow_mmio_value; u64 __read_mostly shadow_mmio_mask; u64 __read_mostly shadow_mmio_access_mask; u64 __read_mostly shadow_present_mask; +u64 __read_mostly shadow_me_value; u64 __read_mostly shadow_me_mask; u64 __read_mostly shadow_acc_track_mask; =20 @@ -140,7 +141,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_pa= ge *sp, pte_access &=3D ~ACC_WRITE_MASK; =20 if (!kvm_is_mmio_pfn(pfn)) - spte |=3D shadow_me_mask; + spte |=3D shadow_me_value; =20 spte |=3D (u64)pfn << PAGE_SHIFT; =20 @@ -256,7 +257,7 @@ u64 make_nonleaf_spte(u64 *child_pt, bool ad_disabled) u64 spte =3D SPTE_MMU_PRESENT_MASK; =20 spte |=3D __pa(child_pt) | shadow_present_mask | PT_WRITABLE_MASK | - shadow_user_mask | shadow_x_mask | shadow_me_mask; + shadow_user_mask | shadow_x_mask | shadow_me_value; =20 if (ad_disabled) spte |=3D SPTE_TDP_AD_DISABLED_MASK; @@ -357,6 +358,17 @@ void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mm= io_mask, u64 access_mask) } EXPORT_SYMBOL_GPL(kvm_mmu_set_mmio_spte_mask); =20 +void kvm_mmu_set_me_spte_mask(u64 me_value, u64 me_mask) +{ + /* shadow_me_value must be a subset of shadow_me_mask */ + if (WARN_ON(me_value & ~me_mask)) + me_value =3D me_mask =3D 0; + + shadow_me_value =3D me_value; + shadow_me_mask =3D me_mask; +} +EXPORT_SYMBOL_GPL(kvm_mmu_set_me_spte_mask); + void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only) { shadow_user_mask =3D VMX_EPT_READABLE_MASK; @@ -366,8 +378,6 @@ void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_e= xec_only) shadow_x_mask =3D VMX_EPT_EXECUTABLE_MASK; shadow_present_mask =3D has_exec_only ? 0ull : VMX_EPT_READABLE_MASK; shadow_acc_track_mask =3D VMX_EPT_RWX_MASK; - shadow_me_mask =3D 0ull; - shadow_host_writable_mask =3D EPT_SPTE_HOST_WRITABLE; shadow_mmu_writable_mask =3D EPT_SPTE_MMU_WRITABLE; =20 @@ -418,7 +428,8 @@ void kvm_mmu_reset_all_pte_masks(void) shadow_x_mask =3D 0; shadow_present_mask =3D PT_PRESENT_MASK; shadow_acc_track_mask =3D 0; - shadow_me_mask =3D sme_me_mask; + shadow_me_mask =3D 0; + shadow_me_value =3D 0; =20 shadow_host_writable_mask =3D DEFAULT_SPTE_HOST_WRITABLE; shadow_mmu_writable_mask =3D DEFAULT_SPTE_MMU_WRITABLE; diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 73f12615416f..c45fb1cd2644 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -149,6 +149,7 @@ extern u64 __read_mostly shadow_mmio_value; extern u64 __read_mostly shadow_mmio_mask; extern u64 __read_mostly shadow_mmio_access_mask; extern u64 __read_mostly shadow_present_mask; +extern u64 __read_mostly shadow_me_value; extern u64 __read_mostly shadow_me_mask; =20 /* diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index fc1725b7d05f..c9485b5bd1dd 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4891,6 +4891,9 @@ static __init int svm_hardware_setup(void) get_npt_level(), PG_LEVEL_1G); pr_info("kvm: Nested Paging %sabled\n", npt_enabled ? "en" : "dis"); =20 + /* Setup shadow_me_value and shadow_me_mask */ + kvm_mmu_set_me_spte_mask(sme_me_mask, sme_me_mask); + /* Note, SEV setup consumes npt_enabled. */ sev_hardware_setup(); =20 --=20 2.35.1 From nobody Thu May 14 08:07:54 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67B73C433EF for ; Tue, 19 Apr 2022 11:17:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350990AbiDSLUR (ORCPT ); Tue, 19 Apr 2022 07:20:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350952AbiDSLUK (ORCPT ); Tue, 19 Apr 2022 07:20:10 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6986915813; Tue, 19 Apr 2022 04:17:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1650367048; x=1681903048; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qXUC0JW6X2U4JInupot3/lviCS3TrbWUtQI18BZne9k=; b=eZgSSppmO3JL3B+aw1vf9rI8M8KwzKqrjSftgUkYk4CSR8c0PUXTIzuo e/1gL5Agm9a35+KEjvhbwcKr6FiYVSX1gmtNHpgy+I6xUMKw0uxc93lOv qihKX2vox+IV2/3DzVM2QIgWJuEE1NAWZRjpSXh0CZ2+Q0zhmGa+jQYWD KwveiJKMffKNv3PLhWPptZGMpWAc+zsVZLBliaDTp84eoqiG5bx2L19y+ 3ygfC9kWqsXhpkQZQuB982hQFYBQqGkI9uwBJ8tRzu5zIjWxBu3mLsEFV iEeYYpScAqsvbV1oVmWjYYS/LdfZr/w6n4YUwv8IX4ipkQu1A357uV9p0 g==; X-IronPort-AV: E=McAfee;i="6400,9594,10321"; a="350189765" X-IronPort-AV: E=Sophos;i="5.90,272,1643702400"; d="scan'208";a="350189765" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Apr 2022 04:17:20 -0700 X-IronPort-AV: E=Sophos;i="5.90,272,1643702400"; d="scan'208";a="554683759" Received: from csambran-mobl.amr.corp.intel.com (HELO khuang2-desk.gar.corp.intel.com) ([10.254.58.20]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Apr 2022 04:17:18 -0700 From: Kai Huang To: kvm@vger.kernel.org Cc: linux-kernel@vger.kernel.org, pbonzini@redhat.com, seanjc@google.com, vkuznets@redhat.com, jmattson@google.com, joro@8bytes.org, wanpengli@tencent.com Subject: [PATCH 3/3] KVM: VMX: Include MKTME KeyID bits to shadow_zero_check Date: Tue, 19 Apr 2022 23:17:04 +1200 Message-Id: <27bc10e97a3c0b58a4105ff9107448c190328239.1650363789.git.kai.huang@intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Intel MKTME KeyID bits (including Intel TDX private KeyID bits) should never be set to SPTE. Set shadow_me_value to 0 and shadow_me_mask to include all MKTME KeyID bits to include them to shadow_zero_check. Signed-off-by: Kai Huang --- arch/x86/kvm/mmu.h | 19 +++++++++++++++++++ arch/x86/kvm/mmu/spte.c | 19 ------------------- arch/x86/kvm/vmx/vmx.c | 31 +++++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+), 19 deletions(-) diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 60c669494c4c..ce953885362b 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -65,6 +65,25 @@ static __always_inline u64 rsvd_bits(int s, int e) return ((2ULL << (e - s)) - 1) << s; } =20 +static inline u8 kvm_get_shadow_phys_bits(void) +{ + /* + * boot_cpu_data.x86_phys_bits is reduced when MKTME or SME are detected + * in CPU detection code, but the processor treats those reduced bits as + * 'keyID' thus they are not reserved bits. Therefore KVM needs to look at + * the physical address bits reported by CPUID. + */ + if (likely(boot_cpu_data.extended_cpuid_level >=3D 0x80000008)) + return cpuid_eax(0x80000008) & 0xff; + + /* + * Quite weird to have VMX or SVM but not MAXPHYADDR; probably a VM with + * custom CPUID. Proceed with whatever the kernel found since these feat= ures + * aren't virtualizable (SME/SEV also require CPUIDs higher than 0x800000= 08). + */ + return boot_cpu_data.x86_phys_bits; +} + void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_= mask); void kvm_mmu_set_me_spte_mask(u64 me_value, u64 me_mask); void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only); diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index a6da94f441c4..c2b785f08cf7 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -283,25 +283,6 @@ u64 kvm_mmu_changed_pte_notifier_make_spte(u64 old_spt= e, kvm_pfn_t new_pfn) return new_spte; } =20 -static u8 kvm_get_shadow_phys_bits(void) -{ - /* - * boot_cpu_data.x86_phys_bits is reduced when MKTME or SME are detected - * in CPU detection code, but the processor treats those reduced bits as - * 'keyID' thus they are not reserved bits. Therefore KVM needs to look at - * the physical address bits reported by CPUID. - */ - if (likely(boot_cpu_data.extended_cpuid_level >=3D 0x80000008)) - return cpuid_eax(0x80000008) & 0xff; - - /* - * Quite weird to have VMX or SVM but not MAXPHYADDR; probably a VM with - * custom CPUID. Proceed with whatever the kernel found since these feat= ures - * aren't virtualizable (SME/SEV also require CPUIDs higher than 0x800000= 08). - */ - return boot_cpu_data.x86_phys_bits; -} - u64 mark_spte_for_access_track(u64 spte) { if (spte_ad_enabled(spte)) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index cf8581978bce..65494caf75f0 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7883,6 +7883,31 @@ static __init void vmx_setup_user_return_msrs(void) kvm_add_user_return_msr(vmx_uret_msrs_list[i]); } =20 +static void __init vmx_setup_me_spte_mask(void) +{ + u64 me_mask =3D 0; + + /* + * kvm_get_shadow_phys_bits() returns shadow_phys_bits. Use + * the former to avoid exposing shadow_phys_bits. + * + * On pre-MKTME system, boot_cpu_data.x86_phys_bits equals to + * shadow_phys_bits. On MKTME and/or TDX capable systems, + * boot_cpu_data.x86_phys_bits holds the actual physical address + * w/o the KeyID bits, and shadow_phys_bits equals to MAXPHYADDR + * reported by CPUID. Those bits between are KeyID bits. + */ + if (boot_cpu_data.x86_phys_bits !=3D kvm_get_shadow_phys_bits()) + me_mask =3D rsvd_bits(boot_cpu_data.x86_phys_bits, + kvm_get_shadow_phys_bits() - 1); + /* + * Unlike to SME, host kernel doesn't support setting up any + * MKTME KeyID to any mapping for Intel platforms. No memory + * encryption bits should be included into the SPTE. + */ + kvm_mmu_set_me_spte_mask(0, me_mask); +} + static struct kvm_x86_init_ops vmx_init_ops __initdata; =20 static __init int hardware_setup(void) @@ -7985,6 +8010,12 @@ static __init int hardware_setup(void) kvm_mmu_set_ept_masks(enable_ept_ad_bits, cpu_has_vmx_ept_execute_only()); =20 + /* + * Setup shadow_me_value/shadow_me_mask to include MKTME KeyID + * bits to shadow_zero_check. + */ + vmx_setup_me_spte_mask(); + kvm_configure_mmu(enable_ept, 0, vmx_get_max_tdp_level(), ept_caps_to_lpage_level(vmx_capability.ept)); =20 --=20 2.35.1