From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51822C433EF for ; Fri, 4 Feb 2022 11:57:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358641AbiBDL5f (ORCPT ); Fri, 4 Feb 2022 06:57:35 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:54067 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230501AbiBDL5Y (ORCPT ); Fri, 4 Feb 2022 06:57:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975844; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UoW9C6WTQmDXSZguRIVvtIoVzfzF+dJipQYj4ctsU3Q=; b=RAVt5pJc/S0Rvl7d8EqKRKwzJJKxA/wQ62sXhluk5uxj0+de9wXPXsBxThMCoA0Ng1tssU WvtXH0SeCmtO/uhaGrCwV67jLtonUHxHbiCAZyAaJ+sAtus9OiEjuNUsqbgvFCpqYgS9Xi isbTgnXIqqel55+3hViDgarJ9DgOUl4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-257-4rbSrh1JPqmLmI6pKzLEqA-1; Fri, 04 Feb 2022 06:57:21 -0500 X-MC-Unique: 4rbSrh1JPqmLmI6pKzLEqA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EB76461259; Fri, 4 Feb 2022 11:57:19 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7F7A81081172; Fri, 4 Feb 2022 11:57:19 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 01/23] KVM: MMU: pass uses_nx directly to reset_shadow_zero_bits_mask Date: Fri, 4 Feb 2022 06:56:56 -0500 Message-Id: <20220204115718.14934-2-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" reset_shadow_zero_bits_mask has a very unintuitive way of deciding whether the shadow pages will use the NX bit. The function is used in two cases, shadow paging and shadow NPT; shadow paging has a use for EFER.NX and needs to force it enabled, while shadow NPT only needs it depending on L1's setting. The actual root problem here is that is_efer_nx, despite being part of the "base" role, only matches the format of the shadow pages in the NPT case. For now, just remove the ugly variable initialization and move the call to reset_shadow_zero_bits_mask out of shadow_mmu_init_context. The parameter can then be removed after the root problem in the role is fixed. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 296f8723f9ae..9424ae90f1ef 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4410,18 +4410,9 @@ static inline u64 reserved_hpa_bits(void) * follow the features in guest. */ static void reset_shadow_zero_bits_mask(struct kvm_vcpu *vcpu, - struct kvm_mmu *context) + struct kvm_mmu *context, + bool uses_nx) { - /* - * KVM uses NX when TDP is disabled to handle a variety of scenarios, - * notably for huge SPTEs if iTLB multi-hit mitigation is enabled and - * to generate correct permissions for CR0.WP=3D0/CR4.SMEP=3D1/EFER.NX=3D= 0. - * The iTLB multi-hit workaround can be toggled at any time, so assume - * NX can be used by any non-nested shadow MMU to avoid having to reset - * MMU contexts. Note, KVM forces EFER.NX=3D1 when TDP is disabled. - */ - bool uses_nx =3D is_efer_nx(context) || !tdp_enabled; - /* @amd adds a check on bit of SPTEs, which KVM shouldn't use anyways. */ bool is_amd =3D true; /* KVM doesn't use 2-level page tables for the shadow MMU. */ @@ -4829,8 +4820,6 @@ static void shadow_mmu_init_context(struct kvm_vcpu *= vcpu, struct kvm_mmu *conte =20 reset_guest_paging_metadata(vcpu, context); context->shadow_root_level =3D new_role.base.level; - - reset_shadow_zero_bits_mask(vcpu, context); } =20 static void kvm_init_shadow_mmu(struct kvm_vcpu *vcpu, @@ -4841,6 +4830,16 @@ static void kvm_init_shadow_mmu(struct kvm_vcpu *vcp= u, kvm_calc_shadow_mmu_root_page_role(vcpu, regs, false); =20 shadow_mmu_init_context(vcpu, context, regs, new_role); + + /* + * KVM uses NX when TDP is disabled to handle a variety of scenarios, + * notably for huge SPTEs if iTLB multi-hit mitigation is enabled and + * to generate correct permissions for CR0.WP=3D0/CR4.SMEP=3D1/EFER.NX=3D= 0. + * The iTLB multi-hit workaround can be toggled at any time, so assume + * NX can be used by any non-nested shadow MMU to avoid having to reset + * MMU contexts. Note, KVM forces EFER.NX=3D1 when TDP is disabled. + */ + reset_shadow_zero_bits_mask(vcpu, context, true); } =20 static union kvm_mmu_role @@ -4872,6 +4871,7 @@ void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, u= nsigned long cr0, __kvm_mmu_new_pgd(vcpu, nested_cr3, new_role.base); =20 shadow_mmu_init_context(vcpu, context, ®s, new_role); + reset_shadow_zero_bits_mask(vcpu, context, is_efer_nx(context)); } EXPORT_SYMBOL_GPL(kvm_init_shadow_npt_mmu); =20 --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A94B3C433EF for ; Fri, 4 Feb 2022 11:57:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358547AbiBDL5h (ORCPT ); Fri, 4 Feb 2022 06:57:37 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:24957 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358516AbiBDL5Z (ORCPT ); Fri, 4 Feb 2022 06:57:25 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975844; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tpMro66VpU3af97OW3y5FG1bjk0vC2VXc8qIJQfmWX8=; b=NRGnvSWtL+XnsIhuswn58tULCE072IXHz9j2GzUtfBoVg/iAtWNnw9xTXD3kW7lBzk8hsb EOE+1raZ33cQ2iGFz6dcj71KXNSxvxrppIEmVHsUzOBUzMIUgeOCOBRaxwIPYWrat93tGb xXQYPfwLbA5wZDLTTzz9pgEJvJate7g= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-290-XnIDk1q2M3KMRAyCv3C8qQ-1; Fri, 04 Feb 2022 06:57:21 -0500 X-MC-Unique: XnIDk1q2M3KMRAyCv3C8qQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7C687190B2AD; Fri, 4 Feb 2022 11:57:20 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 112B71084186; Fri, 4 Feb 2022 11:57:20 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 02/23] KVM: MMU: nested EPT cannot be used in SMM Date: Fri, 4 Feb 2022 06:56:57 -0500 Message-Id: <20220204115718.14934-3-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The role.base.smm flag is always zero, do not bother copying it over from vcpu->arch.root_mmu. Signed-off-by: Paolo Bonzini Reviewed-by: David Matlack --- arch/x86/kvm/mmu/mmu.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 9424ae90f1ef..b0065ae3cea8 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4881,9 +4881,6 @@ kvm_calc_shadow_ept_root_page_role(struct kvm_vcpu *v= cpu, bool accessed_dirty, { union kvm_mmu_role role =3D {0}; =20 - /* SMM flag is inherited from root_mmu */ - role.base.smm =3D vcpu->arch.root_mmu.mmu_role.base.smm; - role.base.level =3D level; role.base.has_4_byte_gpte =3D false; role.base.direct =3D false; --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B823DC433F5 for ; Fri, 4 Feb 2022 11:57:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358544AbiBDL51 (ORCPT ); Fri, 4 Feb 2022 06:57:27 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:49961 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358505AbiBDL5X (ORCPT ); Fri, 4 Feb 2022 06:57:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975843; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EpNiTSjqJIiQYBK9fyczhlCSJngnEHfTdSweeKvpGZc=; b=Sslla06Y+7iQUutuRFWPNrlCMEDe4jc0cbGAu3X4U3Ztlb1mX8wnHlwUW647z7rAtspBmI IywxAeqFLh5/S2IgecHlDOwL6mB/4QBl8NIJobDO2PgvWeKvcpMmtkdn3JshIQKsUTBfsX ptrfUhlCbBp5RQ8p7lxlXg4lKDr/fUg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-380-ptUzOL1INVuDCuqhKWQAEA-1; Fri, 04 Feb 2022 06:57:22 -0500 X-MC-Unique: ptUzOL1INVuDCuqhKWQAEA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0F58F1054F90; Fri, 4 Feb 2022 11:57:21 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 967C81081172; Fri, 4 Feb 2022 11:57:20 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 03/23] KVM: MMU: remove valid from extended role Date: Fri, 4 Feb 2022 06:56:58 -0500 Message-Id: <20220204115718.14934-4-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The level field of the MMU role can act as a marker for validity instead: it is guaranteed to be nonzero so a zero value means the role is invalid and the MMU properties will be computed again. Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/kvm_host.h | 4 +--- arch/x86/kvm/mmu/mmu.c | 9 +++------ 2 files changed, 4 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index e7e5bd9a984d..4ec7d1e3aa36 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -342,8 +342,7 @@ union kvm_mmu_page_role { * kvm_mmu_extended_role complements kvm_mmu_page_role, tracking properties * relevant to the current MMU configuration. When loading CR0, CR4, or = EFER, * including on nested transitions, if nothing in the full role changes th= en - * MMU re-configuration can be skipped. @valid bit is set on first usage s= o we - * don't treat all-zero structure as valid data. + * MMU re-configuration can be skipped. * * The properties that are tracked in the extended role but not the page r= ole * are for things that either (a) do not affect the validity of the shadow= page @@ -360,7 +359,6 @@ union kvm_mmu_page_role { union kvm_mmu_extended_role { u32 word; struct { - unsigned int valid:1; unsigned int execonly:1; unsigned int cr0_pg:1; unsigned int cr4_pae:1; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index b0065ae3cea8..0039b2f21286 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4683,8 +4683,6 @@ static union kvm_mmu_extended_role kvm_calc_mmu_role_= ext(struct kvm_vcpu *vcpu, ext.efer_lma =3D ____is_efer_lma(regs); } =20 - ext.valid =3D 1; - return ext; } =20 @@ -4891,7 +4889,6 @@ kvm_calc_shadow_ept_root_page_role(struct kvm_vcpu *v= cpu, bool accessed_dirty, /* EPT, and thus nested EPT, does not consume CR0, CR4, nor EFER. */ role.ext.word =3D 0; role.ext.execonly =3D execonly; - role.ext.valid =3D 1; =20 return role; } @@ -5039,9 +5036,9 @@ void kvm_mmu_after_set_cpuid(struct kvm_vcpu *vcpu) * problem is swept under the rug; KVM's CPUID API is horrific and * it's all but impossible to solve it without introducing a new API. */ - vcpu->arch.root_mmu.mmu_role.ext.valid =3D 0; - vcpu->arch.guest_mmu.mmu_role.ext.valid =3D 0; - vcpu->arch.nested_mmu.mmu_role.ext.valid =3D 0; + vcpu->arch.root_mmu.mmu_role.base.level =3D 0; + vcpu->arch.guest_mmu.mmu_role.base.level =3D 0; + vcpu->arch.nested_mmu.mmu_role.base.level =3D 0; kvm_mmu_reset_context(vcpu); =20 /* --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73507C433F5 for ; Fri, 4 Feb 2022 11:57:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358595AbiBDL5c (ORCPT ); Fri, 4 Feb 2022 06:57:32 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:46156 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358510AbiBDL5Y (ORCPT ); Fri, 4 Feb 2022 06:57:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975843; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=23LmqLrLqw9sHYk580lY5bQARRIz3tEWlrRaxErbyko=; b=OHnf5oWE8syqk8EyXNRfbRd3Pk7AtKXsESd3vCktZEL3BvFuBLWW52wRWPgFYj+u2Y3z4j 1BNW6h/K86fTIOVLJouXWQnBF2gWK16RRa34aSbC7E6c9PLNyd8oJcd1/ZLe78Tq4v4rni 9gbGl/+Is5Gndxwxtkjro1yXAtk320I= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-327-aJmuQr2pNgieNtln78GC6g-1; Fri, 04 Feb 2022 06:57:22 -0500 X-MC-Unique: aJmuQr2pNgieNtln78GC6g-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 962AE190B2A1; Fri, 4 Feb 2022 11:57:21 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 299AA1081172; Fri, 4 Feb 2022 11:57:21 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 04/23] KVM: MMU: constify uses of struct kvm_mmu_role_regs Date: Fri, 4 Feb 2022 06:56:59 -0500 Message-Id: <20220204115718.14934-5-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" struct kvm_mmu_role_regs is computed just once and then accessed. Use const to enforce this. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0039b2f21286..3add9d8b0630 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -208,7 +208,7 @@ struct kvm_mmu_role_regs { * the single source of truth for the MMU's state. */ #define BUILD_MMU_ROLE_REGS_ACCESSOR(reg, name, flag) \ -static inline bool __maybe_unused ____is_##reg##_##name(struct kvm_mmu_rol= e_regs *regs)\ +static inline bool __maybe_unused ____is_##reg##_##name(const struct kvm_m= mu_role_regs *regs)\ { \ return !!(regs->reg & flag); \ } @@ -255,7 +255,7 @@ static struct kvm_mmu_role_regs vcpu_to_role_regs(struc= t kvm_vcpu *vcpu) return regs; } =20 -static int role_regs_to_root_level(struct kvm_mmu_role_regs *regs) +static int role_regs_to_root_level(const struct kvm_mmu_role_regs *regs) { if (!____is_cr0_pg(regs)) return 0; @@ -4666,7 +4666,7 @@ static void paging32_init_context(struct kvm_mmu *con= text) } =20 static union kvm_mmu_extended_role kvm_calc_mmu_role_ext(struct kvm_vcpu *= vcpu, - struct kvm_mmu_role_regs *regs) + const struct kvm_mmu_role_regs *regs) { union kvm_mmu_extended_role ext =3D {0}; =20 @@ -4687,7 +4687,7 @@ static union kvm_mmu_extended_role kvm_calc_mmu_role_= ext(struct kvm_vcpu *vcpu, } =20 static union kvm_mmu_role kvm_calc_mmu_role_common(struct kvm_vcpu *vcpu, - struct kvm_mmu_role_regs *regs, + const struct kvm_mmu_role_regs *regs, bool base_only) { union kvm_mmu_role role =3D {0}; @@ -4723,7 +4723,8 @@ static inline int kvm_mmu_get_tdp_level(struct kvm_vc= pu *vcpu) =20 static union kvm_mmu_role kvm_calc_tdp_mmu_root_page_role(struct kvm_vcpu *vcpu, - struct kvm_mmu_role_regs *regs, bool base_only) + const struct kvm_mmu_role_regs *regs, + bool base_only) { union kvm_mmu_role role =3D kvm_calc_mmu_role_common(vcpu, regs, base_onl= y); =20 @@ -4769,7 +4770,8 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu) =20 static union kvm_mmu_role kvm_calc_shadow_root_page_role_common(struct kvm_vcpu *vcpu, - struct kvm_mmu_role_regs *regs, bool base_only) + const struct kvm_mmu_role_regs *regs, + bool base_only) { union kvm_mmu_role role =3D kvm_calc_mmu_role_common(vcpu, regs, base_onl= y); =20 @@ -4782,7 +4784,8 @@ kvm_calc_shadow_root_page_role_common(struct kvm_vcpu= *vcpu, =20 static union kvm_mmu_role kvm_calc_shadow_mmu_root_page_role(struct kvm_vcpu *vcpu, - struct kvm_mmu_role_regs *regs, bool base_only) + const struct kvm_mmu_role_regs *regs, + bool base_only) { union kvm_mmu_role role =3D kvm_calc_shadow_root_page_role_common(vcpu, regs, base_only); @@ -4940,7 +4943,7 @@ static void init_kvm_softmmu(struct kvm_vcpu *vcpu) } =20 static union kvm_mmu_role -kvm_calc_nested_mmu_role(struct kvm_vcpu *vcpu, struct kvm_mmu_role_regs *= regs) +kvm_calc_nested_mmu_role(struct kvm_vcpu *vcpu, const struct kvm_mmu_role_= regs *regs) { union kvm_mmu_role role; =20 --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC6FBC433EF for ; Fri, 4 Feb 2022 11:57:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358822AbiBDL5w (ORCPT ); Fri, 4 Feb 2022 06:57:52 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44880 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358537AbiBDL51 (ORCPT ); Fri, 4 Feb 2022 06:57:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975846; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HdszhqD355DEoGMlDegfttlPu8jVJiAVeRKaATVPk1o=; b=WRKsXYRxNP/EGKqOmYAiwHdbCfMWQLXxNU67fH77w+twNiG2/94DMWti3h6QbLHjxMQOGA D8Nb3SteEzcMCvYwut9vJS470MRmYPmc3i/M0ZZabYY1fnyaXzfS5Vu7QJKyfv0X1SFng+ SYPB/WbdMs6TVuO1+1yWykmwYzG8N2U= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-364-R2EQWIJYPiGy16cgnl3zfQ-1; Fri, 04 Feb 2022 06:57:23 -0500 X-MC-Unique: R2EQWIJYPiGy16cgnl3zfQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2881A61261; Fri, 4 Feb 2022 11:57:22 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id B05901081172; Fri, 4 Feb 2022 11:57:21 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 05/23] KVM: MMU: pull computation of kvm_mmu_role_regs to kvm_init_mmu Date: Fri, 4 Feb 2022 06:57:00 -0500 Message-Id: <20220204115718.14934-6-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The init_kvm_*mmu functions, with the exception of shadow NPT, do not need to know the full values of CR0/CR4/EFER; they only need to know the bits that make up the "role". This cleanup however will take quite a few incremental steps. As a start, pull the common computation of the struct kvm_mmu_role_regs into their caller: all of them extract the struct from the vcpu as the very first step. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 33 +++++++++++++++++---------------- 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 3add9d8b0630..577e70509510 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4736,12 +4736,12 @@ kvm_calc_tdp_mmu_root_page_role(struct kvm_vcpu *vc= pu, return role; } =20 -static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu) +static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, + const struct kvm_mmu_role_regs *regs) { struct kvm_mmu *context =3D &vcpu->arch.root_mmu; - struct kvm_mmu_role_regs regs =3D vcpu_to_role_regs(vcpu); union kvm_mmu_role new_role =3D - kvm_calc_tdp_mmu_root_page_role(vcpu, ®s, false); + kvm_calc_tdp_mmu_root_page_role(vcpu, regs, false); =20 if (new_role.as_u64 =3D=3D context->mmu_role.as_u64) return; @@ -4755,7 +4755,7 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu) context->get_guest_pgd =3D get_cr3; context->get_pdptr =3D kvm_pdptr_read; context->inject_page_fault =3D kvm_inject_page_fault; - context->root_level =3D role_regs_to_root_level(®s); + context->root_level =3D role_regs_to_root_level(regs); =20 if (!is_cr0_pg(context)) context->gva_to_gpa =3D nonpaging_gva_to_gpa; @@ -4803,7 +4803,7 @@ kvm_calc_shadow_mmu_root_page_role(struct kvm_vcpu *v= cpu, } =20 static void shadow_mmu_init_context(struct kvm_vcpu *vcpu, struct kvm_mmu = *context, - struct kvm_mmu_role_regs *regs, + const struct kvm_mmu_role_regs *regs, union kvm_mmu_role new_role) { if (new_role.as_u64 =3D=3D context->mmu_role.as_u64) @@ -4824,7 +4824,7 @@ static void shadow_mmu_init_context(struct kvm_vcpu *= vcpu, struct kvm_mmu *conte } =20 static void kvm_init_shadow_mmu(struct kvm_vcpu *vcpu, - struct kvm_mmu_role_regs *regs) + const struct kvm_mmu_role_regs *regs) { struct kvm_mmu *context =3D &vcpu->arch.root_mmu; union kvm_mmu_role new_role =3D @@ -4845,7 +4845,7 @@ static void kvm_init_shadow_mmu(struct kvm_vcpu *vcpu, =20 static union kvm_mmu_role kvm_calc_shadow_npt_root_page_role(struct kvm_vcpu *vcpu, - struct kvm_mmu_role_regs *regs) + const struct kvm_mmu_role_regs *regs) { union kvm_mmu_role role =3D kvm_calc_shadow_root_page_role_common(vcpu, regs, false); @@ -4930,12 +4930,12 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu,= bool execonly, } EXPORT_SYMBOL_GPL(kvm_init_shadow_ept_mmu); =20 -static void init_kvm_softmmu(struct kvm_vcpu *vcpu) +static void init_kvm_softmmu(struct kvm_vcpu *vcpu, + const struct kvm_mmu_role_regs *regs) { struct kvm_mmu *context =3D &vcpu->arch.root_mmu; - struct kvm_mmu_role_regs regs =3D vcpu_to_role_regs(vcpu); =20 - kvm_init_shadow_mmu(vcpu, ®s); + kvm_init_shadow_mmu(vcpu, regs); =20 context->get_guest_pgd =3D get_cr3; context->get_pdptr =3D kvm_pdptr_read; @@ -4959,10 +4959,9 @@ kvm_calc_nested_mmu_role(struct kvm_vcpu *vcpu, cons= t struct kvm_mmu_role_regs * return role; } =20 -static void init_kvm_nested_mmu(struct kvm_vcpu *vcpu) +static void init_kvm_nested_mmu(struct kvm_vcpu *vcpu, const struct kvm_mm= u_role_regs *regs) { - struct kvm_mmu_role_regs regs =3D vcpu_to_role_regs(vcpu); - union kvm_mmu_role new_role =3D kvm_calc_nested_mmu_role(vcpu, ®s); + union kvm_mmu_role new_role =3D kvm_calc_nested_mmu_role(vcpu, regs); struct kvm_mmu *g_context =3D &vcpu->arch.nested_mmu; =20 if (new_role.as_u64 =3D=3D g_context->mmu_role.as_u64) @@ -5002,12 +5001,14 @@ static void init_kvm_nested_mmu(struct kvm_vcpu *vc= pu) =20 void kvm_init_mmu(struct kvm_vcpu *vcpu) { + struct kvm_mmu_role_regs regs =3D vcpu_to_role_regs(vcpu); + if (mmu_is_nested(vcpu)) - init_kvm_nested_mmu(vcpu); + init_kvm_nested_mmu(vcpu, ®s); else if (tdp_enabled) - init_kvm_tdp_mmu(vcpu); + init_kvm_tdp_mmu(vcpu, ®s); else - init_kvm_softmmu(vcpu); + init_kvm_softmmu(vcpu, ®s); } EXPORT_SYMBOL_GPL(kvm_init_mmu); =20 --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB1F2C433EF for ; Fri, 4 Feb 2022 11:57:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358750AbiBDL5q (ORCPT ); Fri, 4 Feb 2022 06:57:46 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:37000 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358540AbiBDL51 (ORCPT ); Fri, 4 Feb 2022 06:57:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975846; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OzIDUAXsJoa+BiSWuGe751GcZgmjbrKEWZp98e47DYA=; b=RV2fs8FiEN5OBPCsZkZxqwvyHUT2SbYBvyltOdy16lB9wZ9VtXi2PTDFY9O0mz6RLjHjNp f1qlcMUgc4tRnQlhiYP0BTCeiP7ajWc7gGfdELHWrW7CraMbQ9aFv/4Yse4xFaZg/fzE08 cIaUjoNKorT0tacwdpbA3U+GLlEB3po= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-657-ziQl9z8bOUeB2IVJ1VGaRQ-1; Fri, 04 Feb 2022 06:57:23 -0500 X-MC-Unique: ziQl9z8bOUeB2IVJ1VGaRQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B05268710F8; Fri, 4 Feb 2022 11:57:22 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 429281084184; Fri, 4 Feb 2022 11:57:22 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 06/23] KVM: MMU: load new PGD once nested two-dimensional paging is initialized Date: Fri, 4 Feb 2022 06:57:01 -0500 Message-Id: <20220204115718.14934-7-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" __kvm_mmu_new_pgd looks at the MMU's root_level and shadow_root_level via fast_pgd_switch. It makes no sense to call it before updating these fields, even though it was done like that ever since nested VMX grew the ability to use fast CR3 switch (commit 50c28f21d045, "kvm: x86: Use fast CR3 switch for nested VMX"). Pull it to the end of the initialization of the shadow nested MMUs. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 41 +++++++++++++++++++---------------------- 1 file changed, 19 insertions(+), 22 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 577e70509510..b8ab16323629 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4869,10 +4869,9 @@ void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, = unsigned long cr0, =20 new_role =3D kvm_calc_shadow_npt_root_page_role(vcpu, ®s); =20 - __kvm_mmu_new_pgd(vcpu, nested_cr3, new_role.base); - shadow_mmu_init_context(vcpu, context, ®s, new_role); reset_shadow_zero_bits_mask(vcpu, context, is_efer_nx(context)); + __kvm_mmu_new_pgd(vcpu, nested_cr3, new_role.base); } EXPORT_SYMBOL_GPL(kvm_init_shadow_npt_mmu); =20 @@ -4906,27 +4905,25 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu,= bool execonly, kvm_calc_shadow_ept_root_page_role(vcpu, accessed_dirty, execonly, level); =20 - __kvm_mmu_new_pgd(vcpu, new_eptp, new_role.base); - - if (new_role.as_u64 =3D=3D context->mmu_role.as_u64) - return; - - context->mmu_role.as_u64 =3D new_role.as_u64; - - context->shadow_root_level =3D level; - - context->ept_ad =3D accessed_dirty; - context->page_fault =3D ept_page_fault; - context->gva_to_gpa =3D ept_gva_to_gpa; - context->sync_page =3D ept_sync_page; - context->invlpg =3D ept_invlpg; - context->root_level =3D level; - context->direct_map =3D false; + if (new_role.as_u64 !=3D context->mmu_role.as_u64) { + context->mmu_role.as_u64 =3D new_role.as_u64; + + context->shadow_root_level =3D level; + + context->ept_ad =3D accessed_dirty; + context->page_fault =3D ept_page_fault; + context->gva_to_gpa =3D ept_gva_to_gpa; + context->sync_page =3D ept_sync_page; + context->invlpg =3D ept_invlpg; + context->root_level =3D level; + context->direct_map =3D false; + update_permission_bitmask(context, true); + context->pkru_mask =3D 0; + reset_rsvds_bits_mask_ept(vcpu, context, execonly, huge_page_level); + reset_ept_shadow_zero_bits_mask(context, execonly); + } =20 - update_permission_bitmask(context, true); - context->pkru_mask =3D 0; - reset_rsvds_bits_mask_ept(vcpu, context, execonly, huge_page_level); - reset_ept_shadow_zero_bits_mask(context, execonly); + __kvm_mmu_new_pgd(vcpu, new_eptp, new_role.base); } EXPORT_SYMBOL_GPL(kvm_init_shadow_ept_mmu); =20 --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7370EC433EF for ; Fri, 4 Feb 2022 11:57:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358789AbiBDL5t (ORCPT ); Fri, 4 Feb 2022 06:57:49 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:20128 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358543AbiBDL51 (ORCPT ); Fri, 4 Feb 2022 06:57:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975847; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fkKYEfyzy0ipD48Cu9xppCXPtPIRhSzneIy4U+9hHCY=; b=AIo5TW79uwGWMP/IVVQaYhCSrEutQETteVbwm5Q4EM3VE80xgKHmuarH182Q760SJuqwCx PKs2731uwA99SXEn8+kFFdxS59Gx9EL7Ml18RHjRubOpP5s2A1idNjJCaNcbIi5TOsILeR NU5H/qLNCqb7P1v7t94LKYISjASp8Cg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-398-4yVODtwiNpSUmarTOQWBJA-1; Fri, 04 Feb 2022 06:57:24 -0500 X-MC-Unique: 4yVODtwiNpSUmarTOQWBJA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 41931190B2A6; Fri, 4 Feb 2022 11:57:23 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id CA5A31081172; Fri, 4 Feb 2022 11:57:22 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 07/23] KVM: MMU: remove kvm_mmu_calc_root_page_role Date: Fri, 4 Feb 2022 06:57:02 -0500 Message-Id: <20220204115718.14934-8-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Since the guest PGD is now loaded after the MMU has been set up completely, the desired role for a cache hit is simply the current mmu_role. There is no need to compute it again, so __kvm_mmu_new_pgd can be folded in kvm_mmu_new_pgd. As an aside, the !tdp_enabled case in the function was dead code, and that also gets mopped up as a side effect. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 29 ++++------------------------- 1 file changed, 4 insertions(+), 25 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index b8ab16323629..42475e4c2a48 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -190,8 +190,6 @@ struct kmem_cache *mmu_page_header_cache; static struct percpu_counter kvm_total_used_mmu_pages; =20 static void mmu_spte_set(u64 *sptep, u64 spte); -static union kvm_mmu_page_role -kvm_mmu_calc_root_page_role(struct kvm_vcpu *vcpu); =20 struct kvm_mmu_role_regs { const unsigned long cr0; @@ -4159,9 +4157,9 @@ static bool fast_pgd_switch(struct kvm_vcpu *vcpu, gp= a_t new_pgd, return false; } =20 -static void __kvm_mmu_new_pgd(struct kvm_vcpu *vcpu, gpa_t new_pgd, - union kvm_mmu_page_role new_role) +void kvm_mmu_new_pgd(struct kvm_vcpu *vcpu, gpa_t new_pgd) { + union kvm_mmu_page_role new_role =3D vcpu->arch.mmu->mmu_role.base; if (!fast_pgd_switch(vcpu, new_pgd, new_role)) { kvm_mmu_free_roots(vcpu, vcpu->arch.mmu, KVM_MMU_ROOT_CURRENT); return; @@ -4196,11 +4194,6 @@ static void __kvm_mmu_new_pgd(struct kvm_vcpu *vcpu,= gpa_t new_pgd, __clear_sp_write_flooding_count( to_shadow_page(vcpu->arch.mmu->root_hpa)); } - -void kvm_mmu_new_pgd(struct kvm_vcpu *vcpu, gpa_t new_pgd) -{ - __kvm_mmu_new_pgd(vcpu, new_pgd, kvm_mmu_calc_root_page_role(vcpu)); -} EXPORT_SYMBOL_GPL(kvm_mmu_new_pgd); =20 static unsigned long get_cr3(struct kvm_vcpu *vcpu) @@ -4871,7 +4864,7 @@ void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, u= nsigned long cr0, =20 shadow_mmu_init_context(vcpu, context, ®s, new_role); reset_shadow_zero_bits_mask(vcpu, context, is_efer_nx(context)); - __kvm_mmu_new_pgd(vcpu, nested_cr3, new_role.base); + kvm_mmu_new_pgd(vcpu, nested_cr3); } EXPORT_SYMBOL_GPL(kvm_init_shadow_npt_mmu); =20 @@ -4923,7 +4916,7 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, b= ool execonly, reset_ept_shadow_zero_bits_mask(context, execonly); } =20 - __kvm_mmu_new_pgd(vcpu, new_eptp, new_role.base); + kvm_mmu_new_pgd(vcpu, new_eptp); } EXPORT_SYMBOL_GPL(kvm_init_shadow_ept_mmu); =20 @@ -5009,20 +5002,6 @@ void kvm_init_mmu(struct kvm_vcpu *vcpu) } EXPORT_SYMBOL_GPL(kvm_init_mmu); =20 -static union kvm_mmu_page_role -kvm_mmu_calc_root_page_role(struct kvm_vcpu *vcpu) -{ - struct kvm_mmu_role_regs regs =3D vcpu_to_role_regs(vcpu); - union kvm_mmu_role role; - - if (tdp_enabled) - role =3D kvm_calc_tdp_mmu_root_page_role(vcpu, ®s, true); - else - role =3D kvm_calc_shadow_mmu_root_page_role(vcpu, ®s, true); - - return role.base; -} - void kvm_mmu_after_set_cpuid(struct kvm_vcpu *vcpu) { /* --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 725A1C4332F for ; Fri, 4 Feb 2022 11:57:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358726AbiBDL5n (ORCPT ); Fri, 4 Feb 2022 06:57:43 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:21435 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358549AbiBDL52 (ORCPT ); Fri, 4 Feb 2022 06:57:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975848; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9Aa81Pa80imIk4UaTF0wxzL7ZPYrOTOis7eBwCIeEGQ=; b=CCSyzt3hPhk5AifpcS5xLS1SK6/gS2j6cdnA2GphCb+S1FC4pipk7XuHPbxkpNWSiReNba fq89xNP8La3MURv40Orlm+n6h4s+UYvOr4Uz9hr8On+0OpOIaSoFlH7/znUl4k1uFbRZLD yjABxrPLUgUjpbVyipO3dWTUXmOmTMw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-480-qj1A0x2mMGyZvYX7FywFtQ-1; Fri, 04 Feb 2022 06:57:24 -0500 X-MC-Unique: qj1A0x2mMGyZvYX7FywFtQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C6F0061280; Fri, 4 Feb 2022 11:57:23 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5BCA11084184; Fri, 4 Feb 2022 11:57:23 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 08/23] KVM: MMU: rephrase unclear comment Date: Fri, 4 Feb 2022 06:57:03 -0500 Message-Id: <20220204115718.14934-9-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" If accessed bits are not supported there simple isn't any distinction between accessed and non-accessed gPTEs, so the comment does not make much sense. Rephrase it in terms of what happens if accessed bits *are* supported. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 5b5bdac97c7b..6bb9a377bf89 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -193,7 +193,7 @@ static bool FNAME(prefetch_invalid_gpte)(struct kvm_vcp= u *vcpu, if (!FNAME(is_present_gpte)(gpte)) goto no_present; =20 - /* if accessed bit is not supported prefetch non accessed gpte */ + /* if accessed bit is supported, prefetch only accessed gpte */ if (PT_HAVE_ACCESSED_DIRTY(vcpu->arch.mmu) && !(gpte & PT_GUEST_ACCESSED_MASK)) goto no_present; --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6863EC433F5 for ; Fri, 4 Feb 2022 11:57:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358702AbiBDL5l (ORCPT ); Fri, 4 Feb 2022 06:57:41 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:41412 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358536AbiBDL50 (ORCPT ); Fri, 4 Feb 2022 06:57:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975846; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/0KvnVSZbYuXChqPICSoFkHZMPZz7M/WQK9Wm/Hf92g=; b=XZjel3en4QnXZRz606tbdCI/OEDvyRkzBxHu69O2xt/abnxgEZ7TDvp7b7GIL8uGus58lM q4XMyPAYIAkRcqycbaYetg7rRXKlQxqZfO2Qd1k1ioajiGESk0cI0xuPURlfbYVXr0t4bv OQPKfshYG4mk+FVoIGO0Lvk3grdcjbg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-115-2r59kty6OcywRixb3SyHTQ-1; Fri, 04 Feb 2022 06:57:25 -0500 X-MC-Unique: 2r59kty6OcywRixb3SyHTQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5822F1015DBE; Fri, 4 Feb 2022 11:57:24 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id E12861081172; Fri, 4 Feb 2022 11:57:23 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 09/23] KVM: MMU: remove "bool base_only" arguments Date: Fri, 4 Feb 2022 06:57:04 -0500 Message-Id: <20220204115718.14934-10-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The argument is always false now that kvm_mmu_calc_root_page_role has been removed. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 63 +++++++++++++++--------------------------- 1 file changed, 22 insertions(+), 41 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 42475e4c2a48..dd69cfc8c4f6 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4658,46 +4658,30 @@ static void paging32_init_context(struct kvm_mmu *c= ontext) context->direct_map =3D false; } =20 -static union kvm_mmu_extended_role kvm_calc_mmu_role_ext(struct kvm_vcpu *= vcpu, - const struct kvm_mmu_role_regs *regs) -{ - union kvm_mmu_extended_role ext =3D {0}; - - if (____is_cr0_pg(regs)) { - ext.cr0_pg =3D 1; - ext.cr4_pae =3D ____is_cr4_pae(regs); - ext.cr4_smep =3D ____is_cr4_smep(regs); - ext.cr4_smap =3D ____is_cr4_smap(regs); - ext.cr4_pse =3D ____is_cr4_pse(regs); - - /* PKEY and LA57 are active iff long mode is active. */ - ext.cr4_pke =3D ____is_efer_lma(regs) && ____is_cr4_pke(regs); - ext.cr4_la57 =3D ____is_efer_lma(regs) && ____is_cr4_la57(regs); - ext.efer_lma =3D ____is_efer_lma(regs); - } - - return ext; -} - static union kvm_mmu_role kvm_calc_mmu_role_common(struct kvm_vcpu *vcpu, - const struct kvm_mmu_role_regs *regs, - bool base_only) + const struct kvm_mmu_role_regs *regs) { union kvm_mmu_role role =3D {0}; =20 role.base.access =3D ACC_ALL; if (____is_cr0_pg(regs)) { + role.ext.cr0_pg =3D 1; role.base.efer_nx =3D ____is_efer_nx(regs); role.base.cr0_wp =3D ____is_cr0_wp(regs); + + role.ext.cr4_pae =3D ____is_cr4_pae(regs); + role.ext.cr4_smep =3D ____is_cr4_smep(regs); + role.ext.cr4_smap =3D ____is_cr4_smap(regs); + role.ext.cr4_pse =3D ____is_cr4_pse(regs); + + /* PKEY and LA57 are active iff long mode is active. */ + role.ext.cr4_pke =3D ____is_efer_lma(regs) && ____is_cr4_pke(regs); + role.ext.cr4_la57 =3D ____is_efer_lma(regs) && ____is_cr4_la57(regs); + role.ext.efer_lma =3D ____is_efer_lma(regs); } role.base.smm =3D is_smm(vcpu); role.base.guest_mode =3D is_guest_mode(vcpu); =20 - if (base_only) - return role; - - role.ext =3D kvm_calc_mmu_role_ext(vcpu, regs); - return role; } =20 @@ -4716,10 +4700,9 @@ static inline int kvm_mmu_get_tdp_level(struct kvm_v= cpu *vcpu) =20 static union kvm_mmu_role kvm_calc_tdp_mmu_root_page_role(struct kvm_vcpu *vcpu, - const struct kvm_mmu_role_regs *regs, - bool base_only) + const struct kvm_mmu_role_regs *regs) { - union kvm_mmu_role role =3D kvm_calc_mmu_role_common(vcpu, regs, base_onl= y); + union kvm_mmu_role role =3D kvm_calc_mmu_role_common(vcpu, regs); =20 role.base.ad_disabled =3D (shadow_accessed_mask =3D=3D 0); role.base.level =3D kvm_mmu_get_tdp_level(vcpu); @@ -4734,7 +4717,7 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, { struct kvm_mmu *context =3D &vcpu->arch.root_mmu; union kvm_mmu_role new_role =3D - kvm_calc_tdp_mmu_root_page_role(vcpu, regs, false); + kvm_calc_tdp_mmu_root_page_role(vcpu, regs); =20 if (new_role.as_u64 =3D=3D context->mmu_role.as_u64) return; @@ -4763,10 +4746,9 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, =20 static union kvm_mmu_role kvm_calc_shadow_root_page_role_common(struct kvm_vcpu *vcpu, - const struct kvm_mmu_role_regs *regs, - bool base_only) + const struct kvm_mmu_role_regs *regs) { - union kvm_mmu_role role =3D kvm_calc_mmu_role_common(vcpu, regs, base_onl= y); + union kvm_mmu_role role =3D kvm_calc_mmu_role_common(vcpu, regs); =20 role.base.smep_andnot_wp =3D role.ext.cr4_smep && !____is_cr0_wp(regs); role.base.smap_andnot_wp =3D role.ext.cr4_smap && !____is_cr0_wp(regs); @@ -4777,11 +4759,10 @@ kvm_calc_shadow_root_page_role_common(struct kvm_vc= pu *vcpu, =20 static union kvm_mmu_role kvm_calc_shadow_mmu_root_page_role(struct kvm_vcpu *vcpu, - const struct kvm_mmu_role_regs *regs, - bool base_only) + const struct kvm_mmu_role_regs *regs) { union kvm_mmu_role role =3D - kvm_calc_shadow_root_page_role_common(vcpu, regs, base_only); + kvm_calc_shadow_root_page_role_common(vcpu, regs); =20 role.base.direct =3D !____is_cr0_pg(regs); =20 @@ -4821,7 +4802,7 @@ static void kvm_init_shadow_mmu(struct kvm_vcpu *vcpu, { struct kvm_mmu *context =3D &vcpu->arch.root_mmu; union kvm_mmu_role new_role =3D - kvm_calc_shadow_mmu_root_page_role(vcpu, regs, false); + kvm_calc_shadow_mmu_root_page_role(vcpu, regs); =20 shadow_mmu_init_context(vcpu, context, regs, new_role); =20 @@ -4841,7 +4822,7 @@ kvm_calc_shadow_npt_root_page_role(struct kvm_vcpu *v= cpu, const struct kvm_mmu_role_regs *regs) { union kvm_mmu_role role =3D - kvm_calc_shadow_root_page_role_common(vcpu, regs, false); + kvm_calc_shadow_root_page_role_common(vcpu, regs); =20 role.base.direct =3D false; role.base.level =3D kvm_mmu_get_tdp_level(vcpu); @@ -4937,7 +4918,7 @@ kvm_calc_nested_mmu_role(struct kvm_vcpu *vcpu, const= struct kvm_mmu_role_regs * { union kvm_mmu_role role; =20 - role =3D kvm_calc_shadow_root_page_role_common(vcpu, regs, false); + role =3D kvm_calc_shadow_root_page_role_common(vcpu, regs); =20 /* * Nested MMUs are used only for walking L2's gva->gpa, they never have --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F532C4332F for ; Fri, 4 Feb 2022 11:58:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358984AbiBDL6J (ORCPT ); Fri, 4 Feb 2022 06:58:09 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:30522 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358576AbiBDL5b (ORCPT ); Fri, 4 Feb 2022 06:57:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975850; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0hbqReHi1E1mWk0ofe1hvygtQARV0sfpY7xX+td/2o8=; b=gkV9SvE+iqpgGgDPOjwOykvAFM+R2hS5+65GjnEBAndUH/VvAnCbZ9FykJetxhLJQV+6Um cwnypb+W50zGkA7viSD6GBXIFK0mZRpd5AiFm8aqeB2196oo1xyLOawlzsmFCbsitJiV8k PRHHhF5W8ZK9lRJZl68I3JN7kOzDfxM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-665-lcEWjBuTOs-yq1ebAeEY0Q-1; Fri, 04 Feb 2022 06:57:26 -0500 X-MC-Unique: lcEWjBuTOs-yq1ebAeEY0Q-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4D1708710F8; Fri, 4 Feb 2022 11:57:25 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id BF2E06E20B; Fri, 4 Feb 2022 11:57:24 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 10/23] KVM: MMU: split cpu_role from mmu_role Date: Fri, 4 Feb 2022 06:57:05 -0500 Message-Id: <20220204115718.14934-11-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Snapshot the state of the processor registers that govern page walk into a new field of struct kvm_mmu. This is a more natural representation than having it *mostly* in mmu_role but not exclusively; the delta right now is represented in other fields, such as root_level. For example, already in this patch we can replace role_regs_to_root_level with the "level" field of the CPU role. The nested MMU now has only the CPU role; and in fact the new function kvm_calc_cpu_role is analogous to the previous kvm_calc_nested_mmu_role, except that it has role.base.direct equal to CR0.PG. It is not clear what the code meant by "setting role.base.direct to true to detect bogus usage of the nested MMU". Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 100 ++++++++++++++++++++------------ arch/x86/kvm/mmu/paging_tmpl.h | 2 +- 3 files changed, 64 insertions(+), 39 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 4ec7d1e3aa36..427ee486309c 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -432,6 +432,7 @@ struct kvm_mmu { void (*invlpg)(struct kvm_vcpu *vcpu, gva_t gva, hpa_t root_hpa); hpa_t root_hpa; gpa_t root_pgd; + union kvm_mmu_role cpu_role; union kvm_mmu_role mmu_role; u8 root_level; u8 shadow_root_level; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index dd69cfc8c4f6..f98444e1d834 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -230,7 +230,7 @@ BUILD_MMU_ROLE_REGS_ACCESSOR(efer, lma, EFER_LMA); #define BUILD_MMU_ROLE_ACCESSOR(base_or_ext, reg, name) \ static inline bool __maybe_unused is_##reg##_##name(struct kvm_mmu *mmu) \ { \ - return !!(mmu->mmu_role. base_or_ext . reg##_##name); \ + return !!(mmu->cpu_role. base_or_ext . reg##_##name); \ } BUILD_MMU_ROLE_ACCESSOR(ext, cr0, pg); BUILD_MMU_ROLE_ACCESSOR(base, cr0, wp); @@ -4658,6 +4658,38 @@ static void paging32_init_context(struct kvm_mmu *co= ntext) context->direct_map =3D false; } =20 +static union kvm_mmu_role +kvm_calc_cpu_role(struct kvm_vcpu *vcpu, const struct kvm_mmu_role_regs *r= egs) +{ + union kvm_mmu_role role =3D {0}; + + role.base.access =3D ACC_ALL; + role.base.smm =3D is_smm(vcpu); + role.base.guest_mode =3D is_guest_mode(vcpu); + role.base.direct =3D !____is_cr0_pg(regs); + if (!role.base.direct) { + role.base.efer_nx =3D ____is_efer_nx(regs); + role.base.cr0_wp =3D ____is_cr0_wp(regs); + role.base.smep_andnot_wp =3D ____is_cr4_smep(regs) && !____is_cr0_wp(reg= s); + role.base.smap_andnot_wp =3D ____is_cr4_smap(regs) && !____is_cr0_wp(reg= s); + role.base.has_4_byte_gpte =3D !____is_cr4_pae(regs); + role.base.level =3D role_regs_to_root_level(regs); + + role.ext.cr0_pg =3D 1; + role.ext.cr4_pae =3D ____is_cr4_pae(regs); + role.ext.cr4_smep =3D ____is_cr4_smep(regs); + role.ext.cr4_smap =3D ____is_cr4_smap(regs); + role.ext.cr4_pse =3D ____is_cr4_pse(regs); + + /* PKEY and LA57 are active iff long mode is active. */ + role.ext.cr4_pke =3D ____is_efer_lma(regs) && ____is_cr4_pke(regs); + role.ext.cr4_la57 =3D ____is_efer_lma(regs) && ____is_cr4_la57(regs); + role.ext.efer_lma =3D ____is_efer_lma(regs); + } + + return role; +} + static union kvm_mmu_role kvm_calc_mmu_role_common(struct kvm_vcpu *vcpu, const struct kvm_mmu_role_regs *regs) { @@ -4716,13 +4748,16 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, const struct kvm_mmu_role_regs *regs) { struct kvm_mmu *context =3D &vcpu->arch.root_mmu; - union kvm_mmu_role new_role =3D + union kvm_mmu_role cpu_role =3D kvm_calc_cpu_role(vcpu, regs); + union kvm_mmu_role mmu_role =3D kvm_calc_tdp_mmu_root_page_role(vcpu, regs); =20 - if (new_role.as_u64 =3D=3D context->mmu_role.as_u64) + if (cpu_role.as_u64 =3D=3D context->cpu_role.as_u64 && + mmu_role.as_u64 =3D=3D context->mmu_role.as_u64) return; =20 - context->mmu_role.as_u64 =3D new_role.as_u64; + context->cpu_role.as_u64 =3D cpu_role.as_u64; + context->mmu_role.as_u64 =3D mmu_role.as_u64; context->page_fault =3D kvm_tdp_page_fault; context->sync_page =3D nonpaging_sync_page; context->invlpg =3D NULL; @@ -4777,13 +4812,15 @@ kvm_calc_shadow_mmu_root_page_role(struct kvm_vcpu = *vcpu, } =20 static void shadow_mmu_init_context(struct kvm_vcpu *vcpu, struct kvm_mmu = *context, - const struct kvm_mmu_role_regs *regs, - union kvm_mmu_role new_role) + union kvm_mmu_role cpu_role, + union kvm_mmu_role mmu_role) { - if (new_role.as_u64 =3D=3D context->mmu_role.as_u64) + if (cpu_role.as_u64 =3D=3D context->cpu_role.as_u64 && + mmu_role.as_u64 =3D=3D context->mmu_role.as_u64) return; =20 - context->mmu_role.as_u64 =3D new_role.as_u64; + context->cpu_role.as_u64 =3D cpu_role.as_u64; + context->mmu_role.as_u64 =3D mmu_role.as_u64; =20 if (!is_cr0_pg(context)) nonpaging_init_context(context); @@ -4791,20 +4828,21 @@ static void shadow_mmu_init_context(struct kvm_vcpu= *vcpu, struct kvm_mmu *conte paging64_init_context(context); else paging32_init_context(context); - context->root_level =3D role_regs_to_root_level(regs); + context->root_level =3D cpu_role.base.level; =20 reset_guest_paging_metadata(vcpu, context); - context->shadow_root_level =3D new_role.base.level; + context->shadow_root_level =3D mmu_role.base.level; } =20 static void kvm_init_shadow_mmu(struct kvm_vcpu *vcpu, const struct kvm_mmu_role_regs *regs) { struct kvm_mmu *context =3D &vcpu->arch.root_mmu; - union kvm_mmu_role new_role =3D + union kvm_mmu_role cpu_role =3D kvm_calc_cpu_role(vcpu, regs); + union kvm_mmu_role mmu_role =3D kvm_calc_shadow_mmu_root_page_role(vcpu, regs); =20 - shadow_mmu_init_context(vcpu, context, regs, new_role); + shadow_mmu_init_context(vcpu, context, cpu_role, mmu_role); =20 /* * KVM uses NX when TDP is disabled to handle a variety of scenarios, @@ -4839,11 +4877,10 @@ void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu,= unsigned long cr0, .cr4 =3D cr4 & ~X86_CR4_PKE, .efer =3D efer, }; - union kvm_mmu_role new_role; - - new_role =3D kvm_calc_shadow_npt_root_page_role(vcpu, ®s); + union kvm_mmu_role cpu_role =3D kvm_calc_cpu_role(vcpu, ®s); + union kvm_mmu_role mmu_role =3D kvm_calc_shadow_npt_root_page_role(vcpu, = ®s);; =20 - shadow_mmu_init_context(vcpu, context, ®s, new_role); + shadow_mmu_init_context(vcpu, context, cpu_role, mmu_role); reset_shadow_zero_bits_mask(vcpu, context, is_efer_nx(context)); kvm_mmu_new_pgd(vcpu, nested_cr3); } @@ -4862,7 +4899,6 @@ kvm_calc_shadow_ept_root_page_role(struct kvm_vcpu *v= cpu, bool accessed_dirty, role.base.guest_mode =3D true; role.base.access =3D ACC_ALL; =20 - /* EPT, and thus nested EPT, does not consume CR0, CR4, nor EFER. */ role.ext.word =3D 0; role.ext.execonly =3D execonly; =20 @@ -4879,7 +4915,9 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, b= ool execonly, kvm_calc_shadow_ept_root_page_role(vcpu, accessed_dirty, execonly, level); =20 - if (new_role.as_u64 !=3D context->mmu_role.as_u64) { + if (new_role.as_u64 !=3D context->cpu_role.as_u64) { + /* EPT, and thus nested EPT, does not consume CR0, CR4, nor EFER. */ + context->cpu_role.as_u64 =3D new_role.as_u64; context->mmu_role.as_u64 =3D new_role.as_u64; =20 context->shadow_root_level =3D level; @@ -4913,32 +4951,15 @@ static void init_kvm_softmmu(struct kvm_vcpu *vcpu, context->inject_page_fault =3D kvm_inject_page_fault; } =20 -static union kvm_mmu_role -kvm_calc_nested_mmu_role(struct kvm_vcpu *vcpu, const struct kvm_mmu_role_= regs *regs) -{ - union kvm_mmu_role role; - - role =3D kvm_calc_shadow_root_page_role_common(vcpu, regs); - - /* - * Nested MMUs are used only for walking L2's gva->gpa, they never have - * shadow pages of their own and so "direct" has no meaning. Set it - * to "true" to try to detect bogus usage of the nested MMU. - */ - role.base.direct =3D true; - role.base.level =3D role_regs_to_root_level(regs); - return role; -} - static void init_kvm_nested_mmu(struct kvm_vcpu *vcpu, const struct kvm_mm= u_role_regs *regs) { - union kvm_mmu_role new_role =3D kvm_calc_nested_mmu_role(vcpu, regs); + union kvm_mmu_role new_role =3D kvm_calc_cpu_role(vcpu, regs); struct kvm_mmu *g_context =3D &vcpu->arch.nested_mmu; =20 - if (new_role.as_u64 =3D=3D g_context->mmu_role.as_u64) + if (new_role.as_u64 =3D=3D g_context->cpu_role.as_u64) return; =20 - g_context->mmu_role.as_u64 =3D new_role.as_u64; + g_context->cpu_role.as_u64 =3D new_role.as_u64; g_context->get_guest_pgd =3D get_cr3; g_context->get_pdptr =3D kvm_pdptr_read; g_context->inject_page_fault =3D kvm_inject_page_fault; @@ -4997,6 +5018,9 @@ void kvm_mmu_after_set_cpuid(struct kvm_vcpu *vcpu) * problem is swept under the rug; KVM's CPUID API is horrific and * it's all but impossible to solve it without introducing a new API. */ + vcpu->arch.root_mmu.cpu_role.base.level =3D 0; + vcpu->arch.guest_mmu.cpu_role.base.level =3D 0; + vcpu->arch.nested_mmu.cpu_role.base.level =3D 0; vcpu->arch.root_mmu.mmu_role.base.level =3D 0; vcpu->arch.guest_mmu.mmu_role.base.level =3D 0; vcpu->arch.nested_mmu.mmu_role.base.level =3D 0; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 6bb9a377bf89..b9f472f27077 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -323,7 +323,7 @@ static inline bool FNAME(is_last_gpte)(struct kvm_mmu *= mmu, * is not reserved and does not indicate a large page at this level, * so clear PT_PAGE_SIZE_MASK in gpte if that is the case. */ - gpte &=3D level - (PT32_ROOT_LEVEL + mmu->mmu_role.ext.cr4_pse); + gpte &=3D level - (PT32_ROOT_LEVEL + mmu->cpu_role.ext.cr4_pse); #endif /* * PG_LEVEL_4K always terminates. The RHS has bit 7 set --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A5E2C433FE for ; Fri, 4 Feb 2022 11:58:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358963AbiBDL6H (ORCPT ); Fri, 4 Feb 2022 06:58:07 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:22651 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358515AbiBDL5a (ORCPT ); Fri, 4 Feb 2022 06:57:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975850; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oChNsJadBLIffGVGmDXFiHjemHn5uBlLZG2TbVnS7ts=; b=N0DXoidWFGIW6fU92PRp/bv0nWZqhLzgwC/qOHRpQD/TewYqmJ73Y/KHiFmW+MDqLfgtZI RD0bI2t4Nhho6iAYqrKnRnwKmW76AYFf/rB9HMFbpxthjOwbqBzNzNvh4YcZafF1iBMaid 3S0pU84XwmHOXRenwo4rIc+n7Cf4SMw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-665-56riODwuNlSJmXs5Na8ZVw-1; Fri, 04 Feb 2022 06:57:27 -0500 X-MC-Unique: 56riODwuNlSJmXs5Na8ZVw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C47E7190B2A0; Fri, 4 Feb 2022 11:57:25 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 567A46E1F1; Fri, 4 Feb 2022 11:57:25 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 11/23] KVM: MMU: do not recompute root level from kvm_mmu_role_regs Date: Fri, 4 Feb 2022 06:57:06 -0500 Message-Id: <20220204115718.14934-12-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The root_level can be found in the cpu_role (in fact the field is superfluous and could be removed, but one thing at a time). Since there is only one usage left of role_regs_to_root_level, inline it into kvm_calc_cpu_role. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 23 ++++++++--------------- 1 file changed, 8 insertions(+), 15 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index f98444e1d834..74789295f922 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -253,19 +253,6 @@ static struct kvm_mmu_role_regs vcpu_to_role_regs(stru= ct kvm_vcpu *vcpu) return regs; } =20 -static int role_regs_to_root_level(const struct kvm_mmu_role_regs *regs) -{ - if (!____is_cr0_pg(regs)) - return 0; - else if (____is_efer_lma(regs)) - return ____is_cr4_la57(regs) ? PT64_ROOT_5LEVEL : - PT64_ROOT_4LEVEL; - else if (____is_cr4_pae(regs)) - return PT32E_ROOT_LEVEL; - else - return PT32_ROOT_LEVEL; -} - static inline bool kvm_available_flush_tlb_with_range(void) { return kvm_x86_ops.tlb_remote_flush_with_range; @@ -4673,7 +4660,13 @@ kvm_calc_cpu_role(struct kvm_vcpu *vcpu, const struc= t kvm_mmu_role_regs *regs) role.base.smep_andnot_wp =3D ____is_cr4_smep(regs) && !____is_cr0_wp(reg= s); role.base.smap_andnot_wp =3D ____is_cr4_smap(regs) && !____is_cr0_wp(reg= s); role.base.has_4_byte_gpte =3D !____is_cr4_pae(regs); - role.base.level =3D role_regs_to_root_level(regs); + + if (____is_efer_lma(regs)) + role.base.level =3D ____is_cr4_la57(regs) ? PT64_ROOT_5LEVEL : PT64_ROO= T_4LEVEL; + else if (____is_cr4_pae(regs)) + role.base.level =3D PT32E_ROOT_LEVEL; + else + role.base.level =3D PT32_ROOT_LEVEL; =20 role.ext.cr0_pg =3D 1; role.ext.cr4_pae =3D ____is_cr4_pae(regs); @@ -4766,7 +4759,7 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, context->get_guest_pgd =3D get_cr3; context->get_pdptr =3D kvm_pdptr_read; context->inject_page_fault =3D kvm_inject_page_fault; - context->root_level =3D role_regs_to_root_level(regs); + context->root_level =3D cpu_role.base.level; =20 if (!is_cr0_pg(context)) context->gva_to_gpa =3D nonpaging_gva_to_gpa; --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B53B3C433EF for ; Fri, 4 Feb 2022 11:58:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358622AbiBDL6A (ORCPT ); Fri, 4 Feb 2022 06:58:00 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:50547 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358578AbiBDL5b (ORCPT ); Fri, 4 Feb 2022 06:57:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975850; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KpW0MeScL9+nni+CpA1MY2/RDHQVdxcrB5fLNku5iQQ=; b=HWfj79+ql/SPlTZjsWRwSoBZ6+dW/BuvULB5ShDpXfZx1iAvZ4DN9NbV9+gr2psPbmYCNO 7YXNua2lJ+eRPMihJBunWv3p/YnrtfHli7kGUjOG7BU2+4+iMG93Hh52JBnT/jE4eZ/wAs gv20yGaXpEI0TrYR66LNsQzS4rEwRL8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-84-klwgJG1wOd2VnCfCSUvipg-1; Fri, 04 Feb 2022 06:57:27 -0500 X-MC-Unique: klwgJG1wOd2VnCfCSUvipg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 58E83190B2A1; Fri, 4 Feb 2022 11:57:26 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id DEFA46E202; Fri, 4 Feb 2022 11:57:25 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 12/23] KVM: MMU: remove ept_ad field Date: Fri, 4 Feb 2022 06:57:07 -0500 Message-Id: <20220204115718.14934-13-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The similar field in the CPU role, ad_disabled, is initialized to the opposite value for shadow EPT, and left zero for "normal" EPT because guest page tables always have A/D bits. So, read it from the CPU role, like other page-format fields; it just has to be inverted to account for the different polarity. In the MMU role, instead, the ad_disabled bit is set according to shadow_accessed_mask, so it would have been incorrect to replace PT_HAVE_ACCESSED_DIRTY with just !mmu->mmu_role.base.ad_disabled. However, with the separation of CPU and MMU roles, we might even get rid of the PT_HAVE_ACCESSED_DIRTY macro altogether. I am not doing this because the macro has a small effect in terms of dead code elimination: text data bss dec hex 103544 16665 112 120321 1d601 # as of this patch 103746 16665 112 120523 1d6cb # without PT_HAVE_ACCESSED_DIRTY Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/mmu/mmu.c | 1 - arch/x86/kvm/mmu/paging_tmpl.h | 2 +- 3 files changed, 1 insertion(+), 3 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 427ee486309c..795b345361c8 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -436,7 +436,6 @@ struct kvm_mmu { union kvm_mmu_role mmu_role; u8 root_level; u8 shadow_root_level; - u8 ept_ad; bool direct_map; struct kvm_mmu_root_info prev_roots[KVM_MMU_NUM_PREV_ROOTS]; =20 diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 74789295f922..d6b5d8c1c0dc 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4915,7 +4915,6 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, b= ool execonly, =20 context->shadow_root_level =3D level; =20 - context->ept_ad =3D accessed_dirty; context->page_fault =3D ept_page_fault; context->gva_to_gpa =3D ept_gva_to_gpa; context->sync_page =3D ept_sync_page; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index b9f472f27077..1b5c7d03f94b 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -64,7 +64,7 @@ #define PT_LEVEL_BITS PT64_LEVEL_BITS #define PT_GUEST_DIRTY_SHIFT 9 #define PT_GUEST_ACCESSED_SHIFT 8 - #define PT_HAVE_ACCESSED_DIRTY(mmu) ((mmu)->ept_ad) + #define PT_HAVE_ACCESSED_DIRTY(mmu) (!(mmu)->cpu_role.base.ad_disabled) #define CMPXCHG cmpxchg64 #define PT_MAX_FULL_LEVELS PT64_ROOT_MAX_LEVEL #else --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EF36C433F5 for ; Fri, 4 Feb 2022 11:58:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358920AbiBDL6D (ORCPT ); Fri, 4 Feb 2022 06:58:03 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:33962 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358551AbiBDL5b (ORCPT ); Fri, 4 Feb 2022 06:57:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975851; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yhlsGmiUXZA8gEHUfw5Hj0xsoP5XAliQCBV1I2fhGDs=; b=ePx5UWpYQ/7OnS9fz8EJuYo6IC7TKOShIY8keKipCP2EYYu/+/FyI3m1Q68wY+J3ScVm4I 9ej0xYwG02IsvYavQkUKHcUG7dXLN1Gl1g4x6GjmwSvzH3aRrzLAhS5LbjNX7EeJwPsT7v +O2LmjrTdmVxg5A5aXYfNuYU+V3cHlg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-607-eKSqtdHiPeSBIvq2G4wmEQ-1; Fri, 04 Feb 2022 06:57:28 -0500 X-MC-Unique: eKSqtdHiPeSBIvq2G4wmEQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0108B83DD24; Fri, 4 Feb 2022 11:57:27 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 72F2E6E1FD; Fri, 4 Feb 2022 11:57:26 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 13/23] KVM: MMU: remove kvm_calc_shadow_root_page_role_common Date: Fri, 4 Feb 2022 06:57:08 -0500 Message-Id: <20220204115718.14934-14-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" kvm_calc_shadow_root_page_role_common is the same as kvm_calc_cpu_role except for the level, which is overwritten afterwards in kvm_calc_shadow_mmu_root_page_role and kvm_calc_shadow_npt_root_page_role. role.base.direct is already set correctly for the CPU role, and CR0.PG=3D1 is required for VMRUN so it will also be correct for nested NPT. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 21 ++------------------- 1 file changed, 2 insertions(+), 19 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index d6b5d8c1c0dc..19abf1e4cee9 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4772,27 +4772,11 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, reset_tdp_shadow_zero_bits_mask(context); } =20 -static union kvm_mmu_role -kvm_calc_shadow_root_page_role_common(struct kvm_vcpu *vcpu, - const struct kvm_mmu_role_regs *regs) -{ - union kvm_mmu_role role =3D kvm_calc_mmu_role_common(vcpu, regs); - - role.base.smep_andnot_wp =3D role.ext.cr4_smep && !____is_cr0_wp(regs); - role.base.smap_andnot_wp =3D role.ext.cr4_smap && !____is_cr0_wp(regs); - role.base.has_4_byte_gpte =3D ____is_cr0_pg(regs) && !____is_cr4_pae(regs= ); - - return role; -} - static union kvm_mmu_role kvm_calc_shadow_mmu_root_page_role(struct kvm_vcpu *vcpu, const struct kvm_mmu_role_regs *regs) { - union kvm_mmu_role role =3D - kvm_calc_shadow_root_page_role_common(vcpu, regs); - - role.base.direct =3D !____is_cr0_pg(regs); + union kvm_mmu_role role =3D kvm_calc_cpu_role(vcpu, regs); =20 if (!____is_efer_lma(regs)) role.base.level =3D PT32E_ROOT_LEVEL; @@ -4853,9 +4837,8 @@ kvm_calc_shadow_npt_root_page_role(struct kvm_vcpu *v= cpu, const struct kvm_mmu_role_regs *regs) { union kvm_mmu_role role =3D - kvm_calc_shadow_root_page_role_common(vcpu, regs); + kvm_calc_cpu_role(vcpu, regs); =20 - role.base.direct =3D false; role.base.level =3D kvm_mmu_get_tdp_level(vcpu); =20 return role; --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DCC8C433F5 for ; Fri, 4 Feb 2022 11:57:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358852AbiBDL54 (ORCPT ); Fri, 4 Feb 2022 06:57:56 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:37959 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358559AbiBDL5c (ORCPT ); Fri, 4 Feb 2022 06:57:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975851; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MuYSDO7EkngVPWTa07Xx0ZALA/kIXduI1AnC+SV/f2g=; b=VoS1zadXWFagI5lkc4fwr2YAxtVST6nGSXp6ZP5FE+m5D2RiAiLRbt0OXkzdXBSyCp9kNM uDhgHVYKtpCk3lr2FO+SLDDlHJfglA+a1Ntw6XkpQEiQQT3EC7j/alYBRvdaKcAIaeRq8P qC00Rl9mEtu/a4N8uGCo+RnCnDch2HE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-607-NwmUUIaJO7qMNStcMXGMJw-1; Fri, 04 Feb 2022 06:57:28 -0500 X-MC-Unique: NwmUUIaJO7qMNStcMXGMJw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 83EAE1015DA1; Fri, 4 Feb 2022 11:57:27 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1652A6E1FF; Fri, 4 Feb 2022 11:57:27 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 14/23] KVM: MMU: cleanup computation of MMU roles for two-dimensional paging Date: Fri, 4 Feb 2022 06:57:09 -0500 Message-Id: <20220204115718.14934-15-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Inline kvm_calc_mmu_role_common into its sole caller, and simplify it by removing the computation of unnecessary bits. Extended bits are unnecessary because page walking uses the CPU role, and EFER.NX/CR0.WP can be set to one unconditionally---matching the format of shadow pages rather than the format of guest pages. The MMU role for two dimensional paging does still depend on the CPU role, even if only barely so, due to SMM and guest mode; for consistency, pass it down to kvm_calc_tdp_mmu_root_page_role instead of querying the vcpu with is_smm or is_guest_mode. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 39 ++++++++------------------------------- 1 file changed, 8 insertions(+), 31 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 19abf1e4cee9..1650fc291284 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4683,33 +4683,6 @@ kvm_calc_cpu_role(struct kvm_vcpu *vcpu, const struc= t kvm_mmu_role_regs *regs) return role; } =20 -static union kvm_mmu_role kvm_calc_mmu_role_common(struct kvm_vcpu *vcpu, - const struct kvm_mmu_role_regs *regs) -{ - union kvm_mmu_role role =3D {0}; - - role.base.access =3D ACC_ALL; - if (____is_cr0_pg(regs)) { - role.ext.cr0_pg =3D 1; - role.base.efer_nx =3D ____is_efer_nx(regs); - role.base.cr0_wp =3D ____is_cr0_wp(regs); - - role.ext.cr4_pae =3D ____is_cr4_pae(regs); - role.ext.cr4_smep =3D ____is_cr4_smep(regs); - role.ext.cr4_smap =3D ____is_cr4_smap(regs); - role.ext.cr4_pse =3D ____is_cr4_pse(regs); - - /* PKEY and LA57 are active iff long mode is active. */ - role.ext.cr4_pke =3D ____is_efer_lma(regs) && ____is_cr4_pke(regs); - role.ext.cr4_la57 =3D ____is_efer_lma(regs) && ____is_cr4_la57(regs); - role.ext.efer_lma =3D ____is_efer_lma(regs); - } - role.base.smm =3D is_smm(vcpu); - role.base.guest_mode =3D is_guest_mode(vcpu); - - return role; -} - static inline int kvm_mmu_get_tdp_level(struct kvm_vcpu *vcpu) { /* tdp_root_level is architecture forced level, use it if nonzero */ @@ -4725,10 +4698,15 @@ static inline int kvm_mmu_get_tdp_level(struct kvm_= vcpu *vcpu) =20 static union kvm_mmu_role kvm_calc_tdp_mmu_root_page_role(struct kvm_vcpu *vcpu, - const struct kvm_mmu_role_regs *regs) + union kvm_mmu_role cpu_role) { - union kvm_mmu_role role =3D kvm_calc_mmu_role_common(vcpu, regs); + union kvm_mmu_role role =3D {0}; =20 + role.base.access =3D ACC_ALL; + role.base.cr0_wp =3D true; + role.base.efer_nx =3D true; + role.base.smm =3D cpu_role.base.smm; + role.base.guest_mode =3D cpu_role.base.guest_mode; role.base.ad_disabled =3D (shadow_accessed_mask =3D=3D 0); role.base.level =3D kvm_mmu_get_tdp_level(vcpu); role.base.direct =3D true; @@ -4742,8 +4720,7 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, { struct kvm_mmu *context =3D &vcpu->arch.root_mmu; union kvm_mmu_role cpu_role =3D kvm_calc_cpu_role(vcpu, regs); - union kvm_mmu_role mmu_role =3D - kvm_calc_tdp_mmu_root_page_role(vcpu, regs); + union kvm_mmu_role mmu_role =3D kvm_calc_tdp_mmu_root_page_role(vcpu, cpu= _role); =20 if (cpu_role.as_u64 =3D=3D context->cpu_role.as_u64 && mmu_role.as_u64 =3D=3D context->mmu_role.as_u64) --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8D5FC433F5 for ; Fri, 4 Feb 2022 11:58:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358795AbiBDL6V (ORCPT ); Fri, 4 Feb 2022 06:58:21 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:54177 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358593AbiBDL5c (ORCPT ); Fri, 4 Feb 2022 06:57:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975852; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NJOP7FoyOCifyX0DKl4Q1LzWVmhR0iMvwQ8KGPl/v9U=; b=DcWyt+dXbGw0ndTnQ/TA2rwcx6oR/aOufqNOqzxYXrvizdLyM5T1VkLqWu1sq2voRaxyma i3uDIkXn97Pu4hg6XB86/ll4I25bl20XN9N1/0iIl4izeUxVRq8TG+CnIdTaQ6ruyYxDrs ZIxMdBREPtGSEKzKo6n+obc3QiSKD9I= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-629-9R-M2s5UO3K2MisQaDVVWw-1; Fri, 04 Feb 2022 06:57:29 -0500 X-MC-Unique: 9R-M2s5UO3K2MisQaDVVWw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 182548710FB; Fri, 4 Feb 2022 11:57:28 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9E32B6E1F1; Fri, 4 Feb 2022 11:57:27 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 15/23] KVM: MMU: cleanup computation of MMU roles for shadow paging Date: Fri, 4 Feb 2022 06:57:10 -0500 Message-Id: <20220204115718.14934-16-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Pass the already-computed CPU role, instead of redoing it. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 1650fc291284..817e6cc916fc 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4751,13 +4751,11 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, =20 static union kvm_mmu_role kvm_calc_shadow_mmu_root_page_role(struct kvm_vcpu *vcpu, - const struct kvm_mmu_role_regs *regs) + union kvm_mmu_role role) { - union kvm_mmu_role role =3D kvm_calc_cpu_role(vcpu, regs); - - if (!____is_efer_lma(regs)) + if (!role.ext.efer_lma) role.base.level =3D PT32E_ROOT_LEVEL; - else if (____is_cr4_la57(regs)) + else if (role.ext.cr4_la57) role.base.level =3D PT64_ROOT_5LEVEL; else role.base.level =3D PT64_ROOT_4LEVEL; @@ -4794,7 +4792,7 @@ static void kvm_init_shadow_mmu(struct kvm_vcpu *vcpu, struct kvm_mmu *context =3D &vcpu->arch.root_mmu; union kvm_mmu_role cpu_role =3D kvm_calc_cpu_role(vcpu, regs); union kvm_mmu_role mmu_role =3D - kvm_calc_shadow_mmu_root_page_role(vcpu, regs); + kvm_calc_shadow_mmu_root_page_role(vcpu, cpu_role); =20 shadow_mmu_init_context(vcpu, context, cpu_role, mmu_role); =20 @@ -4811,11 +4809,8 @@ static void kvm_init_shadow_mmu(struct kvm_vcpu *vcp= u, =20 static union kvm_mmu_role kvm_calc_shadow_npt_root_page_role(struct kvm_vcpu *vcpu, - const struct kvm_mmu_role_regs *regs) + union kvm_mmu_role role) { - union kvm_mmu_role role =3D - kvm_calc_cpu_role(vcpu, regs); - role.base.level =3D kvm_mmu_get_tdp_level(vcpu); =20 return role; @@ -4831,7 +4826,7 @@ void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, u= nsigned long cr0, .efer =3D efer, }; union kvm_mmu_role cpu_role =3D kvm_calc_cpu_role(vcpu, ®s); - union kvm_mmu_role mmu_role =3D kvm_calc_shadow_npt_root_page_role(vcpu, = ®s);; + union kvm_mmu_role mmu_role =3D kvm_calc_shadow_npt_root_page_role(vcpu, = cpu_role); =20 shadow_mmu_init_context(vcpu, context, cpu_role, mmu_role); reset_shadow_zero_bits_mask(vcpu, context, is_efer_nx(context)); --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D6A2C433EF for ; Fri, 4 Feb 2022 11:58:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359043AbiBDL6P (ORCPT ); Fri, 4 Feb 2022 06:58:15 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:23172 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358605AbiBDL5d (ORCPT ); Fri, 4 Feb 2022 06:57:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975853; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RyK+8TNqNAO/o5qLawKAV0/5MaLr0gp+ky512a4ENu4=; b=JQgEbMBW3bX9F40VZ003i9GIx9rmHR1gf9Qblxy7g8uhJVYtSymtmMf6T3zJgs1EKIX7mp w4cJL4PUyNZW58Cr4o0ql9u5E+Cr40ocImAl1lfkUbHpF63jLv30vKu1lY4VG9y2J30Cb8 UtBOkofK5RTRp/xK9RC04BetxyRFcmk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-346-ZK4wex1EM_-DGQLTbEDiCg-1; Fri, 04 Feb 2022 06:57:29 -0500 X-MC-Unique: ZK4wex1EM_-DGQLTbEDiCg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A1EA28710F1; Fri, 4 Feb 2022 11:57:28 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 338FB6E1FD; Fri, 4 Feb 2022 11:57:28 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 16/23] KVM: MMU: remove extended bits from mmu_role Date: Fri, 4 Feb 2022 06:57:11 -0500 Message-Id: <20220204115718.14934-17-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" mmu_role represents the role of the root of the page tables. It does not need any extended bits, as those govern only KVM's page table walking; the is_* functions used for page table walking always use the CPU role. Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/mmu/mmu.c | 63 ++++++++++++++++----------------- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 2 +- 4 files changed, 34 insertions(+), 35 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 795b345361c8..121eefdb9991 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -433,7 +433,7 @@ struct kvm_mmu { hpa_t root_hpa; gpa_t root_pgd; union kvm_mmu_role cpu_role; - union kvm_mmu_role mmu_role; + union kvm_mmu_page_role mmu_role; u8 root_level; u8 shadow_root_level; bool direct_map; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 817e6cc916fc..0cb46a74e561 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2045,7 +2045,7 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct k= vm_vcpu *vcpu, int collisions =3D 0; LIST_HEAD(invalid_list); =20 - role =3D vcpu->arch.mmu->mmu_role.base; + role =3D vcpu->arch.mmu->mmu_role; role.level =3D level; role.direct =3D direct; role.access =3D access; @@ -3278,7 +3278,7 @@ void kvm_mmu_free_guest_mode_roots(struct kvm_vcpu *v= cpu, struct kvm_mmu *mmu) * This should not be called while L2 is active, L2 can't invalidate * _only_ its own roots, e.g. INVVPID unconditionally exits. */ - WARN_ON_ONCE(mmu->mmu_role.base.guest_mode); + WARN_ON_ONCE(mmu->mmu_role.guest_mode); =20 for (i =3D 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) { root_hpa =3D mmu->prev_roots[i].hpa; @@ -4146,7 +4146,7 @@ static bool fast_pgd_switch(struct kvm_vcpu *vcpu, gp= a_t new_pgd, =20 void kvm_mmu_new_pgd(struct kvm_vcpu *vcpu, gpa_t new_pgd) { - union kvm_mmu_page_role new_role =3D vcpu->arch.mmu->mmu_role.base; + union kvm_mmu_page_role new_role =3D vcpu->arch.mmu->mmu_role; if (!fast_pgd_switch(vcpu, new_pgd, new_role)) { kvm_mmu_free_roots(vcpu, vcpu->arch.mmu, KVM_MMU_ROOT_CURRENT); return; @@ -4696,21 +4696,21 @@ static inline int kvm_mmu_get_tdp_level(struct kvm_= vcpu *vcpu) return max_tdp_level; } =20 -static union kvm_mmu_role +static union kvm_mmu_page_role kvm_calc_tdp_mmu_root_page_role(struct kvm_vcpu *vcpu, union kvm_mmu_role cpu_role) { - union kvm_mmu_role role =3D {0}; + union kvm_mmu_page_role role =3D {0}; =20 - role.base.access =3D ACC_ALL; - role.base.cr0_wp =3D true; - role.base.efer_nx =3D true; - role.base.smm =3D cpu_role.base.smm; - role.base.guest_mode =3D cpu_role.base.guest_mode; - role.base.ad_disabled =3D (shadow_accessed_mask =3D=3D 0); - role.base.level =3D kvm_mmu_get_tdp_level(vcpu); - role.base.direct =3D true; - role.base.has_4_byte_gpte =3D false; + role.access =3D ACC_ALL; + role.cr0_wp =3D true; + role.efer_nx =3D true; + role.smm =3D cpu_role.base.smm; + role.guest_mode =3D cpu_role.base.guest_mode; + role.ad_disabled =3D (shadow_accessed_mask =3D=3D 0); + role.level =3D kvm_mmu_get_tdp_level(vcpu); + role.direct =3D true; + role.has_4_byte_gpte =3D false; =20 return role; } @@ -4720,14 +4720,14 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, { struct kvm_mmu *context =3D &vcpu->arch.root_mmu; union kvm_mmu_role cpu_role =3D kvm_calc_cpu_role(vcpu, regs); - union kvm_mmu_role mmu_role =3D kvm_calc_tdp_mmu_root_page_role(vcpu, cpu= _role); + union kvm_mmu_page_role mmu_role =3D kvm_calc_tdp_mmu_root_page_role(vcpu= , cpu_role); =20 if (cpu_role.as_u64 =3D=3D context->cpu_role.as_u64 && - mmu_role.as_u64 =3D=3D context->mmu_role.as_u64) + mmu_role.word =3D=3D context->mmu_role.word) return; =20 context->cpu_role.as_u64 =3D cpu_role.as_u64; - context->mmu_role.as_u64 =3D mmu_role.as_u64; + context->mmu_role.word =3D mmu_role.word; context->page_fault =3D kvm_tdp_page_fault; context->sync_page =3D nonpaging_sync_page; context->invlpg =3D NULL; @@ -4749,7 +4749,7 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, reset_tdp_shadow_zero_bits_mask(context); } =20 -static union kvm_mmu_role +static union kvm_mmu_page_role kvm_calc_shadow_mmu_root_page_role(struct kvm_vcpu *vcpu, union kvm_mmu_role role) { @@ -4760,19 +4760,19 @@ kvm_calc_shadow_mmu_root_page_role(struct kvm_vcpu = *vcpu, else role.base.level =3D PT64_ROOT_4LEVEL; =20 - return role; + return role.base; } =20 static void shadow_mmu_init_context(struct kvm_vcpu *vcpu, struct kvm_mmu = *context, union kvm_mmu_role cpu_role, - union kvm_mmu_role mmu_role) + union kvm_mmu_page_role mmu_role) { if (cpu_role.as_u64 =3D=3D context->cpu_role.as_u64 && - mmu_role.as_u64 =3D=3D context->mmu_role.as_u64) + mmu_role.word =3D=3D context->mmu_role.word) return; =20 context->cpu_role.as_u64 =3D cpu_role.as_u64; - context->mmu_role.as_u64 =3D mmu_role.as_u64; + context->mmu_role.word =3D mmu_role.word; =20 if (!is_cr0_pg(context)) nonpaging_init_context(context); @@ -4783,7 +4783,7 @@ static void shadow_mmu_init_context(struct kvm_vcpu *= vcpu, struct kvm_mmu *conte context->root_level =3D cpu_role.base.level; =20 reset_guest_paging_metadata(vcpu, context); - context->shadow_root_level =3D mmu_role.base.level; + context->shadow_root_level =3D mmu_role.level; } =20 static void kvm_init_shadow_mmu(struct kvm_vcpu *vcpu, @@ -4791,7 +4791,7 @@ static void kvm_init_shadow_mmu(struct kvm_vcpu *vcpu, { struct kvm_mmu *context =3D &vcpu->arch.root_mmu; union kvm_mmu_role cpu_role =3D kvm_calc_cpu_role(vcpu, regs); - union kvm_mmu_role mmu_role =3D + union kvm_mmu_page_role mmu_role =3D kvm_calc_shadow_mmu_root_page_role(vcpu, cpu_role); =20 shadow_mmu_init_context(vcpu, context, cpu_role, mmu_role); @@ -4807,13 +4807,12 @@ static void kvm_init_shadow_mmu(struct kvm_vcpu *vc= pu, reset_shadow_zero_bits_mask(vcpu, context, true); } =20 -static union kvm_mmu_role +static union kvm_mmu_page_role kvm_calc_shadow_npt_root_page_role(struct kvm_vcpu *vcpu, union kvm_mmu_role role) { role.base.level =3D kvm_mmu_get_tdp_level(vcpu); - - return role; + return role.base; } =20 void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, unsigned long cr0, @@ -4826,7 +4825,7 @@ void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, u= nsigned long cr0, .efer =3D efer, }; union kvm_mmu_role cpu_role =3D kvm_calc_cpu_role(vcpu, ®s); - union kvm_mmu_role mmu_role =3D kvm_calc_shadow_npt_root_page_role(vcpu, = cpu_role); + union kvm_mmu_page_role mmu_role =3D kvm_calc_shadow_npt_root_page_role(v= cpu, cpu_role); =20 shadow_mmu_init_context(vcpu, context, cpu_role, mmu_role); reset_shadow_zero_bits_mask(vcpu, context, is_efer_nx(context)); @@ -4866,7 +4865,7 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, b= ool execonly, if (new_role.as_u64 !=3D context->cpu_role.as_u64) { /* EPT, and thus nested EPT, does not consume CR0, CR4, nor EFER. */ context->cpu_role.as_u64 =3D new_role.as_u64; - context->mmu_role.as_u64 =3D new_role.as_u64; + context->mmu_role.word =3D new_role.base.word; =20 context->shadow_root_level =3D level; =20 @@ -4968,9 +4967,9 @@ void kvm_mmu_after_set_cpuid(struct kvm_vcpu *vcpu) vcpu->arch.root_mmu.cpu_role.base.level =3D 0; vcpu->arch.guest_mmu.cpu_role.base.level =3D 0; vcpu->arch.nested_mmu.cpu_role.base.level =3D 0; - vcpu->arch.root_mmu.mmu_role.base.level =3D 0; - vcpu->arch.guest_mmu.mmu_role.base.level =3D 0; - vcpu->arch.nested_mmu.mmu_role.base.level =3D 0; + vcpu->arch.root_mmu.mmu_role.level =3D 0; + vcpu->arch.guest_mmu.mmu_role.level =3D 0; + vcpu->arch.nested_mmu.mmu_role.level =3D 0; kvm_mmu_reset_context(vcpu); =20 /* diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 1b5c7d03f94b..847c4339e4d9 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -1025,7 +1025,7 @@ static gpa_t FNAME(gva_to_gpa)(struct kvm_vcpu *vcpu,= struct kvm_mmu *mmu, */ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp) { - union kvm_mmu_page_role mmu_role =3D vcpu->arch.mmu->mmu_role.base; + union kvm_mmu_page_role mmu_role =3D vcpu->arch.mmu->mmu_role; int i; bool host_writable; gpa_t first_pte_gpa; diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 8def8f810cb0..dd4c78833016 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -209,7 +209,7 @@ static void tdp_mmu_init_child_sp(struct kvm_mmu_page *= child_sp, =20 hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu) { - union kvm_mmu_page_role role =3D vcpu->arch.mmu->mmu_role.base; + union kvm_mmu_page_role role =3D vcpu->arch.mmu->mmu_role; struct kvm *kvm =3D vcpu->kvm; struct kvm_mmu_page *root; =20 --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70627C433EF for ; Fri, 4 Feb 2022 11:58:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358814AbiBDL63 (ORCPT ); Fri, 4 Feb 2022 06:58:29 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:24542 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358648AbiBDL5g (ORCPT ); Fri, 4 Feb 2022 06:57:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975856; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dAlo3b8oAZQXk/zcIbEROEPp7sP/CVGo95J3iYCaLXk=; b=H8r2n993xemqWpx9hErqzY/xgxlFFOhHul8dof4upz5/YleH07cywfrGM1TpThBziVzAFm RSee8pkjJ7yuSTHTq58BVJ8/QpcG1g2YtPiQPQekx22nx1dV+mv1CBBqqr66jYq+7D6aXv pbFqWktJTtuDZUW+hRyPG+Se6dQE9TU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-204-9Mh7QANsMt61Mk8W0foUMQ-1; Fri, 04 Feb 2022 06:57:30 -0500 X-MC-Unique: 9Mh7QANsMt61Mk8W0foUMQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3616A1054F90; Fri, 4 Feb 2022 11:57:29 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id BC5D36E1F1; Fri, 4 Feb 2022 11:57:28 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 17/23] KVM: MMU: remove redundant bits from extended role Date: Fri, 4 Feb 2022 06:57:12 -0500 Message-Id: <20220204115718.14934-18-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Before the separation of the CPU and the MMU role, CR0.PG was not available in the base MMU role, because two-dimensional paging always used direct=3D1 in the MMU role. However, now that the raw role is snapshotted in mmu->cpu_role, CR0.PG *can* be found (though inverted) as !cpu_role.base.direct. There is no need to store it again in union kvm_mmu_extended_role; instead, write an is_cr0_pg accessor by hand that takes care of the inversion. Likewise, CR4.PAE is now always present in the CPU role as !cpu_role.base.has_4_byte_gpte. The inversion makes certain tests on the MMU role easier, and is easily hidden by the is_cr4_pae accessor when operating on the CPU role. Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/kvm_host.h | 2 -- arch/x86/kvm/mmu/mmu.c | 14 ++++++++++---- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 121eefdb9991..b0085c54786c 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -360,8 +360,6 @@ union kvm_mmu_extended_role { u32 word; struct { unsigned int execonly:1; - unsigned int cr0_pg:1; - unsigned int cr4_pae:1; unsigned int cr4_pse:1; unsigned int cr4_pke:1; unsigned int cr4_smap:1; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0cb46a74e561..b3856551607d 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -232,16 +232,24 @@ static inline bool __maybe_unused is_##reg##_##name(s= truct kvm_mmu *mmu) \ { \ return !!(mmu->cpu_role. base_or_ext . reg##_##name); \ } -BUILD_MMU_ROLE_ACCESSOR(ext, cr0, pg); BUILD_MMU_ROLE_ACCESSOR(base, cr0, wp); BUILD_MMU_ROLE_ACCESSOR(ext, cr4, pse); -BUILD_MMU_ROLE_ACCESSOR(ext, cr4, pae); BUILD_MMU_ROLE_ACCESSOR(ext, cr4, smep); BUILD_MMU_ROLE_ACCESSOR(ext, cr4, smap); BUILD_MMU_ROLE_ACCESSOR(ext, cr4, pke); BUILD_MMU_ROLE_ACCESSOR(ext, cr4, la57); BUILD_MMU_ROLE_ACCESSOR(base, efer, nx); =20 +static inline bool is_cr0_pg(struct kvm_mmu *mmu) +{ + return !(mmu->cpu_role.base.direct); +} + +static inline bool is_cr4_pae(struct kvm_mmu *mmu) +{ + return !(mmu->cpu_role.base.has_4_byte_gpte); +} + static struct kvm_mmu_role_regs vcpu_to_role_regs(struct kvm_vcpu *vcpu) { struct kvm_mmu_role_regs regs =3D { @@ -4668,8 +4676,6 @@ kvm_calc_cpu_role(struct kvm_vcpu *vcpu, const struct= kvm_mmu_role_regs *regs) else role.base.level =3D PT32_ROOT_LEVEL; =20 - role.ext.cr0_pg =3D 1; - role.ext.cr4_pae =3D ____is_cr4_pae(regs); role.ext.cr4_smep =3D ____is_cr4_smep(regs); role.ext.cr4_smap =3D ____is_cr4_smap(regs); role.ext.cr4_pse =3D ____is_cr4_pse(regs); --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29E9FC433EF for ; Fri, 4 Feb 2022 11:58:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359093AbiBDL6Y (ORCPT ); Fri, 4 Feb 2022 06:58:24 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:57261 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358631AbiBDL5e (ORCPT ); Fri, 4 Feb 2022 06:57:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975853; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OfQ/zYI/xARDMrodHdk46Yk2GZI0WBamte9e9IL8TAQ=; b=G5YY5m3Ra0ADJ+5MubiwqxYNVze6pICMqx7y1TsOM4BxriBqkJ8aSwmampfIXKBUZ889cK Vnb3fcQDgc7ANdXY2Wj+yQi74qTQWyz/Wb+FnwBikeCMjMYhZ+YcnK5U/74gq8u1dtMK5i Hn+jvMPtsyO7ennGUZiE2eLTfckIkMM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-549-4SOzFpx2PMmMy3FMcvH2jg-1; Fri, 04 Feb 2022 06:57:30 -0500 X-MC-Unique: 4SOzFpx2PMmMy3FMcvH2jg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C20681054F91; Fri, 4 Feb 2022 11:57:29 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 50B496E1EA; Fri, 4 Feb 2022 11:57:29 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 18/23] KVM: MMU: fetch shadow EFER.NX from MMU role Date: Fri, 4 Feb 2022 06:57:13 -0500 Message-Id: <20220204115718.14934-19-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Now that the MMU role is separate from the CPU role, it contains a truthful description of the format of the shadow pages. This includes whether the shadow pages use the NX bit, so use the MMU role instead of hardcoding it in the callers of reset_shadow_zero_bits_mask. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index b3856551607d..bba712d1a6d7 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4398,13 +4398,13 @@ static inline u64 reserved_hpa_bits(void) * follow the features in guest. */ static void reset_shadow_zero_bits_mask(struct kvm_vcpu *vcpu, - struct kvm_mmu *context, - bool uses_nx) + struct kvm_mmu *context) { /* @amd adds a check on bit of SPTEs, which KVM shouldn't use anyways. */ bool is_amd =3D true; /* KVM doesn't use 2-level page tables for the shadow MMU. */ bool is_pse =3D false; + bool uses_nx =3D context->mmu_role.efer_nx; struct rsvd_bits_validate *shadow_zero_check; int i; =20 @@ -4810,7 +4810,7 @@ static void kvm_init_shadow_mmu(struct kvm_vcpu *vcpu, * NX can be used by any non-nested shadow MMU to avoid having to reset * MMU contexts. Note, KVM forces EFER.NX=3D1 when TDP is disabled. */ - reset_shadow_zero_bits_mask(vcpu, context, true); + reset_shadow_zero_bits_mask(vcpu, context); } =20 static union kvm_mmu_page_role @@ -4834,7 +4834,7 @@ void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, u= nsigned long cr0, union kvm_mmu_page_role mmu_role =3D kvm_calc_shadow_npt_root_page_role(v= cpu, cpu_role); =20 shadow_mmu_init_context(vcpu, context, cpu_role, mmu_role); - reset_shadow_zero_bits_mask(vcpu, context, is_efer_nx(context)); + reset_shadow_zero_bits_mask(vcpu, context); kvm_mmu_new_pgd(vcpu, nested_cr3); } EXPORT_SYMBOL_GPL(kvm_init_shadow_npt_mmu); --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF67FC433EF for ; Fri, 4 Feb 2022 11:58:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234861AbiBDL6R (ORCPT ); Fri, 4 Feb 2022 06:58:17 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:30027 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358600AbiBDL5d (ORCPT ); Fri, 4 Feb 2022 06:57:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975852; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e2E56FroKAIKAOPkFQHe02XUrHi9MAiTwjfpyI4L/KE=; b=DiqiLhDqeHbklwZdaaUgNEVUghlYjngUMxrT+oebqkbWz6Im3z57vZ4DLITr3YchlULzFx /qveOXmoLX9VpXa8fkRLCJ0YoMQhaCnW4PwirisrG+ERSWwxT+J4IHPHV6xKxvT0ZwD+nT T0OQRkjMaCMh0zZhIvWWDE477IYzyGY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-76-bh0r1GCTNkmrJHhXPsaXZA-1; Fri, 04 Feb 2022 06:57:31 -0500 X-MC-Unique: bh0r1GCTNkmrJHhXPsaXZA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 55C7D190B2AE; Fri, 4 Feb 2022 11:57:30 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id DC1DD6E1EA; Fri, 4 Feb 2022 11:57:29 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 19/23] KVM: MMU: simplify and/or inline computation of shadow MMU roles Date: Fri, 4 Feb 2022 06:57:14 -0500 Message-Id: <20220204115718.14934-20-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Shadow MMUs can compute their role from cpu_role.base, simply by adjusting the root level. It's one line of code, so do not place it in a separate function. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 32 +++++++------------------------- 1 file changed, 7 insertions(+), 25 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index bba712d1a6d7..01027da82e23 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4755,20 +4755,6 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, reset_tdp_shadow_zero_bits_mask(context); } =20 -static union kvm_mmu_page_role -kvm_calc_shadow_mmu_root_page_role(struct kvm_vcpu *vcpu, - union kvm_mmu_role role) -{ - if (!role.ext.efer_lma) - role.base.level =3D PT32E_ROOT_LEVEL; - else if (role.ext.cr4_la57) - role.base.level =3D PT64_ROOT_5LEVEL; - else - role.base.level =3D PT64_ROOT_4LEVEL; - - return role.base; -} - static void shadow_mmu_init_context(struct kvm_vcpu *vcpu, struct kvm_mmu = *context, union kvm_mmu_role cpu_role, union kvm_mmu_page_role mmu_role) @@ -4797,9 +4783,10 @@ static void kvm_init_shadow_mmu(struct kvm_vcpu *vcp= u, { struct kvm_mmu *context =3D &vcpu->arch.root_mmu; union kvm_mmu_role cpu_role =3D kvm_calc_cpu_role(vcpu, regs); - union kvm_mmu_page_role mmu_role =3D - kvm_calc_shadow_mmu_root_page_role(vcpu, cpu_role); + union kvm_mmu_page_role mmu_role; =20 + mmu_role =3D cpu_role.base; + mmu_role.level =3D max_t(u32, mmu_role.level, PT32E_ROOT_LEVEL); shadow_mmu_init_context(vcpu, context, cpu_role, mmu_role); =20 /* @@ -4813,14 +4800,6 @@ static void kvm_init_shadow_mmu(struct kvm_vcpu *vcp= u, reset_shadow_zero_bits_mask(vcpu, context); } =20 -static union kvm_mmu_page_role -kvm_calc_shadow_npt_root_page_role(struct kvm_vcpu *vcpu, - union kvm_mmu_role role) -{ - role.base.level =3D kvm_mmu_get_tdp_level(vcpu); - return role.base; -} - void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, unsigned long cr0, unsigned long cr4, u64 efer, gpa_t nested_cr3) { @@ -4831,7 +4810,10 @@ void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, = unsigned long cr0, .efer =3D efer, }; union kvm_mmu_role cpu_role =3D kvm_calc_cpu_role(vcpu, ®s); - union kvm_mmu_page_role mmu_role =3D kvm_calc_shadow_npt_root_page_role(v= cpu, cpu_role); + union kvm_mmu_page_role mmu_role; + + mmu_role =3D cpu_role.base; + mmu_role.level =3D kvm_mmu_get_tdp_level(vcpu); =20 shadow_mmu_init_context(vcpu, context, cpu_role, mmu_role); reset_shadow_zero_bits_mask(vcpu, context); --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA81DC433EF for ; Fri, 4 Feb 2022 11:58:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359122AbiBDL6h (ORCPT ); Fri, 4 Feb 2022 06:58:37 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22735 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358705AbiBDL5l (ORCPT ); Fri, 4 Feb 2022 06:57:41 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975861; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AlOU5hIq1A/HFrJv2NU8RmCYxQMuiUK6gea68jpdViM=; b=KvY89htQaiJq2OF5Z9EUY0ucl8epW1v2MvB4t9MO5TBen0+gd7zIT1Uuwd1dLgip3iMpME Mw1fPmyiiRmij5T7iIT6r7eTc7wu6P5ggn0uOK2hDVKMC65JCCkcr9txKVoipYgWF/+Y6H K/iHUNGZb1JQgpsybsKMN+Pg1RrVXB8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-400-HCxM_3_uO_mBxd-EB0azoA-1; Fri, 04 Feb 2022 06:57:38 -0500 X-MC-Unique: HCxM_3_uO_mBxd-EB0azoA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E9A2B190B2A0; Fri, 4 Feb 2022 11:57:36 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id B03027CAD8; Fri, 4 Feb 2022 11:57:30 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 20/23] KVM: MMU: pull CPU role computation to kvm_init_mmu Date: Fri, 4 Feb 2022 06:57:15 -0500 Message-Id: <20220204115718.14934-21-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Do not lead init_kvm_*mmu into the temptation of poking into struct kvm_mmu_role_regs, by passing to it directly the CPU role. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 01027da82e23..6f9d876ce429 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4721,11 +4721,9 @@ kvm_calc_tdp_mmu_root_page_role(struct kvm_vcpu *vcp= u, return role; } =20 -static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, - const struct kvm_mmu_role_regs *regs) +static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, union kvm_mmu_role cpu= _role) { struct kvm_mmu *context =3D &vcpu->arch.root_mmu; - union kvm_mmu_role cpu_role =3D kvm_calc_cpu_role(vcpu, regs); union kvm_mmu_page_role mmu_role =3D kvm_calc_tdp_mmu_root_page_role(vcpu= , cpu_role); =20 if (cpu_role.as_u64 =3D=3D context->cpu_role.as_u64 && @@ -4779,10 +4777,9 @@ static void shadow_mmu_init_context(struct kvm_vcpu = *vcpu, struct kvm_mmu *conte } =20 static void kvm_init_shadow_mmu(struct kvm_vcpu *vcpu, - const struct kvm_mmu_role_regs *regs) + union kvm_mmu_role cpu_role) { struct kvm_mmu *context =3D &vcpu->arch.root_mmu; - union kvm_mmu_role cpu_role =3D kvm_calc_cpu_role(vcpu, regs); union kvm_mmu_page_role mmu_role; =20 mmu_role =3D cpu_role.base; @@ -4874,20 +4871,19 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu,= bool execonly, EXPORT_SYMBOL_GPL(kvm_init_shadow_ept_mmu); =20 static void init_kvm_softmmu(struct kvm_vcpu *vcpu, - const struct kvm_mmu_role_regs *regs) + union kvm_mmu_role cpu_role) { struct kvm_mmu *context =3D &vcpu->arch.root_mmu; =20 - kvm_init_shadow_mmu(vcpu, regs); + kvm_init_shadow_mmu(vcpu, cpu_role); =20 context->get_guest_pgd =3D get_cr3; context->get_pdptr =3D kvm_pdptr_read; context->inject_page_fault =3D kvm_inject_page_fault; } =20 -static void init_kvm_nested_mmu(struct kvm_vcpu *vcpu, const struct kvm_mm= u_role_regs *regs) +static void init_kvm_nested_mmu(struct kvm_vcpu *vcpu, union kvm_mmu_role = new_role) { - union kvm_mmu_role new_role =3D kvm_calc_cpu_role(vcpu, regs); struct kvm_mmu *g_context =3D &vcpu->arch.nested_mmu; =20 if (new_role.as_u64 =3D=3D g_context->cpu_role.as_u64) @@ -4928,13 +4924,14 @@ static void init_kvm_nested_mmu(struct kvm_vcpu *vc= pu, const struct kvm_mmu_role void kvm_init_mmu(struct kvm_vcpu *vcpu) { struct kvm_mmu_role_regs regs =3D vcpu_to_role_regs(vcpu); + union kvm_mmu_role cpu_role =3D kvm_calc_cpu_role(vcpu, ®s); =20 if (mmu_is_nested(vcpu)) - init_kvm_nested_mmu(vcpu, ®s); + init_kvm_nested_mmu(vcpu, cpu_role); else if (tdp_enabled) - init_kvm_tdp_mmu(vcpu, ®s); + init_kvm_tdp_mmu(vcpu, cpu_role); else - init_kvm_softmmu(vcpu, ®s); + init_kvm_softmmu(vcpu, cpu_role); } EXPORT_SYMBOL_GPL(kvm_init_mmu); =20 --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C13BCC433EF for ; Fri, 4 Feb 2022 11:58:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359115AbiBDL6f (ORCPT ); Fri, 4 Feb 2022 06:58:35 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:37667 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358711AbiBDL5m (ORCPT ); Fri, 4 Feb 2022 06:57:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975861; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XYRG8JlQ0s5Ybgj55U9BkAxflxr/b4DfF/gBfKvQtJ8=; b=EJaoLjFo0kkyJr/Sig6AWqAev/funkFk34MU0ZtS2XJxTP1NL3VKxAd9PVpD5Y1CRkRrwY 5bPRI4COaI8MLyPLYsK73w0IYVdc9CfRfdcOJP6xFRygLQ2+84xMdx26XGcakz15/5sFve FllAJm/J0dZHtyk+6T2UN4O+g4bmn3k= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-457-JO849tj3PW-eilvJ1qxS-w-1; Fri, 04 Feb 2022 06:57:38 -0500 X-MC-Unique: JO849tj3PW-eilvJ1qxS-w-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6EC4D1015DA0; Fri, 4 Feb 2022 11:57:37 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id E861C7CAF6; Fri, 4 Feb 2022 11:57:36 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 21/23] KVM: MMU: store shadow_root_level into mmu_role Date: Fri, 4 Feb 2022 06:57:16 -0500 Message-Id: <20220204115718.14934-22-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" mmu_role.level is always the same value as shadow_level: - kvm_mmu_get_tdp_level(vcpu) when going through init_kvm_tdp_mmu - the level argument when going through kvm_init_shadow_ept_mmu - it's assigned directly from new_role.base.level when going through shadow_mmu_init_context Remove the duplication and get the level directly from the role. Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/mmu.h | 2 +- arch/x86/kvm/mmu/mmu.c | 36 +++++++++++++++------------------ arch/x86/kvm/mmu/tdp_mmu.c | 2 +- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/vmx/vmx.c | 2 +- 6 files changed, 20 insertions(+), 25 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index b0085c54786c..867fc82f1de5 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -433,7 +433,6 @@ struct kvm_mmu { union kvm_mmu_role cpu_role; union kvm_mmu_page_role mmu_role; u8 root_level; - u8 shadow_root_level; bool direct_map; struct kvm_mmu_root_info prev_roots[KVM_MMU_NUM_PREV_ROOTS]; =20 diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 51faa2c76ca5..43b99308cb0e 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -112,7 +112,7 @@ static inline void kvm_mmu_load_pgd(struct kvm_vcpu *vc= pu) return; =20 static_call(kvm_x86_load_mmu_pgd)(vcpu, root_hpa, - vcpu->arch.mmu->shadow_root_level); + vcpu->arch.mmu->mmu_role.level); } =20 struct kvm_page_fault { diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 6f9d876ce429..4d1fa87718f8 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2143,7 +2143,7 @@ static void shadow_walk_init_using_root(struct kvm_sh= adow_walk_iterator *iterato { iterator->addr =3D addr; iterator->shadow_addr =3D root; - iterator->level =3D vcpu->arch.mmu->shadow_root_level; + iterator->level =3D vcpu->arch.mmu->mmu_role.level; =20 if (iterator->level >=3D PT64_ROOT_4LEVEL && vcpu->arch.mmu->root_level < PT64_ROOT_4LEVEL && @@ -3254,7 +3254,7 @@ void kvm_mmu_free_roots(struct kvm_vcpu *vcpu, struct= kvm_mmu *mmu, &invalid_list); =20 if (free_active_root) { - if (mmu->shadow_root_level >=3D PT64_ROOT_4LEVEL && + if (mmu->mmu_role.level >=3D PT64_ROOT_4LEVEL && (mmu->root_level >=3D PT64_ROOT_4LEVEL || mmu->direct_map)) { mmu_free_root_page(kvm, &mmu->root_hpa, &invalid_list); } else if (mmu->pae_root) { @@ -3329,7 +3329,7 @@ static hpa_t mmu_alloc_root(struct kvm_vcpu *vcpu, gf= n_t gfn, gva_t gva, static int mmu_alloc_direct_roots(struct kvm_vcpu *vcpu) { struct kvm_mmu *mmu =3D vcpu->arch.mmu; - u8 shadow_root_level =3D mmu->shadow_root_level; + u8 shadow_root_level =3D mmu->mmu_role.level; hpa_t root; unsigned i; int r; @@ -3479,7 +3479,7 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vc= pu) */ if (mmu->root_level >=3D PT64_ROOT_4LEVEL) { root =3D mmu_alloc_root(vcpu, root_gfn, 0, - mmu->shadow_root_level, false); + mmu->mmu_role.level, false); mmu->root_hpa =3D root; goto set_root_pgd; } @@ -3495,7 +3495,7 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vc= pu) * the shadow page table may be a PAE or a long mode page table. */ pm_mask =3D PT_PRESENT_MASK | shadow_me_mask; - if (mmu->shadow_root_level >=3D PT64_ROOT_4LEVEL) { + if (mmu->mmu_role.level >=3D PT64_ROOT_4LEVEL) { pm_mask |=3D PT_ACCESSED_MASK | PT_WRITABLE_MASK | PT_USER_MASK; =20 if (WARN_ON_ONCE(!mmu->pml4_root)) { @@ -3504,7 +3504,7 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vc= pu) } mmu->pml4_root[0] =3D __pa(mmu->pae_root) | pm_mask; =20 - if (mmu->shadow_root_level =3D=3D PT64_ROOT_5LEVEL) { + if (mmu->mmu_role.level =3D=3D PT64_ROOT_5LEVEL) { if (WARN_ON_ONCE(!mmu->pml5_root)) { r =3D -EIO; goto out_unlock; @@ -3529,9 +3529,9 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vc= pu) mmu->pae_root[i] =3D root | pm_mask; } =20 - if (mmu->shadow_root_level =3D=3D PT64_ROOT_5LEVEL) + if (mmu->mmu_role.level =3D=3D PT64_ROOT_5LEVEL) mmu->root_hpa =3D __pa(mmu->pml5_root); - else if (mmu->shadow_root_level =3D=3D PT64_ROOT_4LEVEL) + else if (mmu->mmu_role.level =3D=3D PT64_ROOT_4LEVEL) mmu->root_hpa =3D __pa(mmu->pml4_root); else mmu->root_hpa =3D __pa(mmu->pae_root); @@ -3547,7 +3547,7 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vc= pu) static int mmu_alloc_special_roots(struct kvm_vcpu *vcpu) { struct kvm_mmu *mmu =3D vcpu->arch.mmu; - bool need_pml5 =3D mmu->shadow_root_level > PT64_ROOT_4LEVEL; + bool need_pml5 =3D mmu->mmu_role.level > PT64_ROOT_4LEVEL; u64 *pml5_root =3D NULL; u64 *pml4_root =3D NULL; u64 *pae_root; @@ -3559,7 +3559,7 @@ static int mmu_alloc_special_roots(struct kvm_vcpu *v= cpu) * on demand, as running a 32-bit L1 VMM on 64-bit KVM is very rare. */ if (mmu->direct_map || mmu->root_level >=3D PT64_ROOT_4LEVEL || - mmu->shadow_root_level < PT64_ROOT_4LEVEL) + mmu->mmu_role.level < PT64_ROOT_4LEVEL) return 0; =20 /* @@ -4145,7 +4145,7 @@ static bool fast_pgd_switch(struct kvm_vcpu *vcpu, gp= a_t new_pgd, * having to deal with PDPTEs. We may add support for 32-bit hosts/VMs * later if necessary. */ - if (mmu->shadow_root_level >=3D PT64_ROOT_4LEVEL && + if (mmu->mmu_role.level >=3D PT64_ROOT_4LEVEL && mmu->root_level >=3D PT64_ROOT_4LEVEL) return cached_root_available(vcpu, new_pgd, new_role); =20 @@ -4408,17 +4408,17 @@ static void reset_shadow_zero_bits_mask(struct kvm_= vcpu *vcpu, struct rsvd_bits_validate *shadow_zero_check; int i; =20 - WARN_ON_ONCE(context->shadow_root_level < PT32E_ROOT_LEVEL); + WARN_ON_ONCE(context->mmu_role.level < PT32E_ROOT_LEVEL); =20 shadow_zero_check =3D &context->shadow_zero_check; __reset_rsvds_bits_mask(shadow_zero_check, reserved_hpa_bits(), - context->shadow_root_level, uses_nx, + context->mmu_role.level, uses_nx, guest_can_use_gbpages(vcpu), is_pse, is_amd); =20 if (!shadow_me_mask) return; =20 - for (i =3D context->shadow_root_level; --i >=3D 0;) { + for (i =3D context->mmu_role.level; --i >=3D 0;) { shadow_zero_check->rsvd_bits_mask[0][i] &=3D ~shadow_me_mask; shadow_zero_check->rsvd_bits_mask[1][i] &=3D ~shadow_me_mask; } @@ -4445,7 +4445,7 @@ reset_tdp_shadow_zero_bits_mask(struct kvm_mmu *conte= xt) =20 if (boot_cpu_is_amd()) __reset_rsvds_bits_mask(shadow_zero_check, reserved_hpa_bits(), - context->shadow_root_level, false, + context->mmu_role.level, false, boot_cpu_has(X86_FEATURE_GBPAGES), false, true); else @@ -4456,7 +4456,7 @@ reset_tdp_shadow_zero_bits_mask(struct kvm_mmu *conte= xt) if (!shadow_me_mask) return; =20 - for (i =3D context->shadow_root_level; --i >=3D 0;) { + for (i =3D context->mmu_role.level; --i >=3D 0;) { shadow_zero_check->rsvd_bits_mask[0][i] &=3D ~shadow_me_mask; shadow_zero_check->rsvd_bits_mask[1][i] &=3D ~shadow_me_mask; } @@ -4735,7 +4735,6 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, u= nion kvm_mmu_role cpu_role) context->page_fault =3D kvm_tdp_page_fault; context->sync_page =3D nonpaging_sync_page; context->invlpg =3D NULL; - context->shadow_root_level =3D kvm_mmu_get_tdp_level(vcpu); context->direct_map =3D true; context->get_guest_pgd =3D get_cr3; context->get_pdptr =3D kvm_pdptr_read; @@ -4773,7 +4772,6 @@ static void shadow_mmu_init_context(struct kvm_vcpu *= vcpu, struct kvm_mmu *conte context->root_level =3D cpu_role.base.level; =20 reset_guest_paging_metadata(vcpu, context); - context->shadow_root_level =3D mmu_role.level; } =20 static void kvm_init_shadow_mmu(struct kvm_vcpu *vcpu, @@ -4852,8 +4850,6 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, b= ool execonly, context->cpu_role.as_u64 =3D new_role.as_u64; context->mmu_role.word =3D new_role.base.word; =20 - context->shadow_root_level =3D level; - context->page_fault =3D ept_page_fault; context->gva_to_gpa =3D ept_gva_to_gpa; context->sync_page =3D ept_sync_page; diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index dd4c78833016..9fb6d983bae9 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1697,7 +1697,7 @@ int kvm_tdp_mmu_get_walk(struct kvm_vcpu *vcpu, u64 a= ddr, u64 *sptes, gfn_t gfn =3D addr >> PAGE_SHIFT; int leaf =3D -1; =20 - *root_level =3D vcpu->arch.mmu->shadow_root_level; + *root_level =3D vcpu->arch.mmu->mmu_role.level; =20 tdp_mmu_for_each_pte(iter, mmu, gfn, gfn + 1) { leaf =3D iter.level; diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 7b5345a66117..5a1d552b535b 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -3815,7 +3815,7 @@ static void svm_load_mmu_pgd(struct kvm_vcpu *vcpu, h= pa_t root_hpa, hv_track_root_tdp(vcpu, root_hpa); =20 cr3 =3D vcpu->arch.cr3; - } else if (vcpu->arch.mmu->shadow_root_level >=3D PT64_ROOT_4LEVEL) { + } else if (vcpu->arch.mmu->mmu_role.level >=3D PT64_ROOT_4LEVEL) { cr3 =3D __sme_set(root_hpa) | kvm_get_active_pcid(vcpu); } else { /* PCID in the guest should be impossible with a 32-bit MMU. */ diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 8ac5a6fa7720..5e2c865a04ff 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -2965,7 +2965,7 @@ static void vmx_flush_tlb_current(struct kvm_vcpu *vc= pu) =20 if (enable_ept) ept_sync_context(construct_eptp(vcpu, root_hpa, - mmu->shadow_root_level)); + mmu->mmu_role.level)); else vpid_sync_context(vmx_get_current_vpid(vcpu)); } --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 018A2C433F5 for ; Fri, 4 Feb 2022 11:58:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359107AbiBDL6d (ORCPT ); Fri, 4 Feb 2022 06:58:33 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:34347 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358722AbiBDL5m (ORCPT ); Fri, 4 Feb 2022 06:57:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975862; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iWySmgXD0THqSjFsrBr0x42/E6z0T7ApfXzZ7tuhiKQ=; b=CT6wLoKRW7Yv6s90TVzXyKZa37twVYb+kOI7T712TlUDndh/f/CNTY0HyLhxXJ4rdBaKfb zaZKVDL0EXgAtKXNib215cfKFdztVl3C5suSVEuftTPMt9lzTlV/vjTH9eTmpVpNd6BvGr N/pKwPyqSdUjylpT9qBV48aZ4NWSEc0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-459-fG1B0xU_MRSbsTm2xZOd6Q-1; Fri, 04 Feb 2022 06:57:39 -0500 X-MC-Unique: fG1B0xU_MRSbsTm2xZOd6Q-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 015F683DD22; Fri, 4 Feb 2022 11:57:38 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 89A567CAD8; Fri, 4 Feb 2022 11:57:37 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 22/23] KVM: MMU: use cpu_role for root_level Date: Fri, 4 Feb 2022 06:57:17 -0500 Message-Id: <20220204115718.14934-23-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Remove another duplicate field of struct kvm_mmu. This time it's the root level for page table walking; we were already initializing it mostly as cpu_role.base.level, but the field still existed; remove it. Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/mmu/mmu.c | 22 +++++++++------------- arch/x86/kvm/mmu/mmu_audit.c | 6 +++--- arch/x86/kvm/mmu/paging_tmpl.h | 4 ++-- 4 files changed, 14 insertions(+), 19 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 867fc82f1de5..c86a2beee92a 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -432,7 +432,6 @@ struct kvm_mmu { gpa_t root_pgd; union kvm_mmu_role cpu_role; union kvm_mmu_page_role mmu_role; - u8 root_level; bool direct_map; struct kvm_mmu_root_info prev_roots[KVM_MMU_NUM_PREV_ROOTS]; =20 diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 4d1fa87718f8..5a6541d6a424 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2146,7 +2146,7 @@ static void shadow_walk_init_using_root(struct kvm_sh= adow_walk_iterator *iterato iterator->level =3D vcpu->arch.mmu->mmu_role.level; =20 if (iterator->level >=3D PT64_ROOT_4LEVEL && - vcpu->arch.mmu->root_level < PT64_ROOT_4LEVEL && + vcpu->arch.mmu->cpu_role.base.level < PT64_ROOT_4LEVEL && !vcpu->arch.mmu->direct_map) iterator->level =3D PT32E_ROOT_LEVEL; =20 @@ -3255,7 +3255,7 @@ void kvm_mmu_free_roots(struct kvm_vcpu *vcpu, struct= kvm_mmu *mmu, =20 if (free_active_root) { if (mmu->mmu_role.level >=3D PT64_ROOT_4LEVEL && - (mmu->root_level >=3D PT64_ROOT_4LEVEL || mmu->direct_map)) { + (mmu->cpu_role.base.level >=3D PT64_ROOT_4LEVEL || mmu->direct_map))= { mmu_free_root_page(kvm, &mmu->root_hpa, &invalid_list); } else if (mmu->pae_root) { for (i =3D 0; i < 4; ++i) { @@ -3453,7 +3453,7 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vc= pu) * On SVM, reading PDPTRs might access guest memory, which might fault * and thus might sleep. Grab the PDPTRs before acquiring mmu_lock. */ - if (mmu->root_level =3D=3D PT32E_ROOT_LEVEL) { + if (mmu->cpu_role.base.level =3D=3D PT32E_ROOT_LEVEL) { for (i =3D 0; i < 4; ++i) { pdptrs[i] =3D mmu->get_pdptr(vcpu, i); if (!(pdptrs[i] & PT_PRESENT_MASK)) @@ -3477,7 +3477,7 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vc= pu) * Do we shadow a long mode page table? If so we need to * write-protect the guests page table root. */ - if (mmu->root_level >=3D PT64_ROOT_4LEVEL) { + if (mmu->cpu_role.base.level >=3D PT64_ROOT_4LEVEL) { root =3D mmu_alloc_root(vcpu, root_gfn, 0, mmu->mmu_role.level, false); mmu->root_hpa =3D root; @@ -3516,7 +3516,7 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vc= pu) for (i =3D 0; i < 4; ++i) { WARN_ON_ONCE(IS_VALID_PAE_ROOT(mmu->pae_root[i])); =20 - if (mmu->root_level =3D=3D PT32E_ROOT_LEVEL) { + if (mmu->cpu_role.base.level =3D=3D PT32E_ROOT_LEVEL) { if (!(pdptrs[i] & PT_PRESENT_MASK)) { mmu->pae_root[i] =3D INVALID_PAE_ROOT; continue; @@ -3558,7 +3558,7 @@ static int mmu_alloc_special_roots(struct kvm_vcpu *v= cpu) * equivalent level in the guest's NPT to shadow. Allocate the tables * on demand, as running a 32-bit L1 VMM on 64-bit KVM is very rare. */ - if (mmu->direct_map || mmu->root_level >=3D PT64_ROOT_4LEVEL || + if (mmu->direct_map || mmu->cpu_role.base.level >=3D PT64_ROOT_4LEVEL || mmu->mmu_role.level < PT64_ROOT_4LEVEL) return 0; =20 @@ -3655,7 +3655,7 @@ void kvm_mmu_sync_roots(struct kvm_vcpu *vcpu) =20 vcpu_clear_mmio_info(vcpu, MMIO_GVA_ANY); =20 - if (vcpu->arch.mmu->root_level >=3D PT64_ROOT_4LEVEL) { + if (vcpu->arch.mmu->cpu_role.base.level >=3D PT64_ROOT_4LEVEL) { hpa_t root =3D vcpu->arch.mmu->root_hpa; sp =3D to_shadow_page(root); =20 @@ -4146,7 +4146,7 @@ static bool fast_pgd_switch(struct kvm_vcpu *vcpu, gp= a_t new_pgd, * later if necessary. */ if (mmu->mmu_role.level >=3D PT64_ROOT_4LEVEL && - mmu->root_level >=3D PT64_ROOT_4LEVEL) + mmu->cpu_role.base.level >=3D PT64_ROOT_4LEVEL) return cached_root_available(vcpu, new_pgd, new_role); =20 return false; @@ -4335,7 +4335,7 @@ static void reset_rsvds_bits_mask(struct kvm_vcpu *vc= pu, { __reset_rsvds_bits_mask(&context->guest_rsvd_check, vcpu->arch.reserved_gpa_bits, - context->root_level, is_efer_nx(context), + context->cpu_role.base.level, is_efer_nx(context), guest_can_use_gbpages(vcpu), is_cr4_pse(context), guest_cpuid_is_amd_or_hygon(vcpu)); @@ -4739,7 +4739,6 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, u= nion kvm_mmu_role cpu_role) context->get_guest_pgd =3D get_cr3; context->get_pdptr =3D kvm_pdptr_read; context->inject_page_fault =3D kvm_inject_page_fault; - context->root_level =3D cpu_role.base.level; =20 if (!is_cr0_pg(context)) context->gva_to_gpa =3D nonpaging_gva_to_gpa; @@ -4769,7 +4768,6 @@ static void shadow_mmu_init_context(struct kvm_vcpu *= vcpu, struct kvm_mmu *conte paging64_init_context(context); else paging32_init_context(context); - context->root_level =3D cpu_role.base.level; =20 reset_guest_paging_metadata(vcpu, context); } @@ -4854,7 +4852,6 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, b= ool execonly, context->gva_to_gpa =3D ept_gva_to_gpa; context->sync_page =3D ept_sync_page; context->invlpg =3D ept_invlpg; - context->root_level =3D level; context->direct_map =3D false; update_permission_bitmask(context, true); context->pkru_mask =3D 0; @@ -4889,7 +4886,6 @@ static void init_kvm_nested_mmu(struct kvm_vcpu *vcpu= , union kvm_mmu_role new_ro g_context->get_guest_pgd =3D get_cr3; g_context->get_pdptr =3D kvm_pdptr_read; g_context->inject_page_fault =3D kvm_inject_page_fault; - g_context->root_level =3D new_role.base.level; =20 /* * L2 page tables are never shadowed, so there is no need to sync diff --git a/arch/x86/kvm/mmu/mmu_audit.c b/arch/x86/kvm/mmu/mmu_audit.c index f31fdb874f1f..eb9c59fcb957 100644 --- a/arch/x86/kvm/mmu/mmu_audit.c +++ b/arch/x86/kvm/mmu/mmu_audit.c @@ -59,11 +59,11 @@ static void mmu_spte_walk(struct kvm_vcpu *vcpu, inspec= t_spte_fn fn) if (!VALID_PAGE(vcpu->arch.mmu->root_hpa)) return; =20 - if (vcpu->arch.mmu->root_level >=3D PT64_ROOT_4LEVEL) { + if (vcpu->arch.mmu->cpu_role.base.level >=3D PT64_ROOT_4LEVEL) { hpa_t root =3D vcpu->arch.mmu->root_hpa; =20 sp =3D to_shadow_page(root); - __mmu_spte_walk(vcpu, sp, fn, vcpu->arch.mmu->root_level); + __mmu_spte_walk(vcpu, sp, fn, vcpu->arch.mmu->cpu_role.base.level); return; } =20 @@ -119,7 +119,7 @@ static void audit_mappings(struct kvm_vcpu *vcpu, u64 *= sptep, int level) hpa =3D pfn << PAGE_SHIFT; if ((*sptep & PT64_BASE_ADDR_MASK) !=3D hpa) audit_printk(vcpu->kvm, "levels %d pfn %llx hpa %llx " - "ent %llxn", vcpu->arch.mmu->root_level, pfn, + "ent %llxn", vcpu->arch.mmu->cpu_role.base.level, pfn, hpa, *sptep); } =20 diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 847c4339e4d9..dd0b6f83171f 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -361,7 +361,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker= *walker, =20 trace_kvm_mmu_pagetable_walk(addr, access); retry_walk: - walker->level =3D mmu->root_level; + walker->level =3D mmu->cpu_role.base.level; pte =3D mmu->get_guest_pgd(vcpu); have_ad =3D PT_HAVE_ACCESSED_DIRTY(mmu); =20 @@ -656,7 +656,7 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, struct k= vm_page_fault *fault, WARN_ON_ONCE(gw->gfn !=3D base_gfn); direct_access =3D gw->pte_access; =20 - top_level =3D vcpu->arch.mmu->root_level; + top_level =3D vcpu->arch.mmu->cpu_role.base.level; if (top_level =3D=3D PT32E_ROOT_LEVEL) top_level =3D PT32_ROOT_LEVEL; /* --=20 2.31.1 From nobody Mon Jun 29 18:39:59 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7568C433EF for ; Fri, 4 Feb 2022 11:58:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359131AbiBDL6k (ORCPT ); Fri, 4 Feb 2022 06:58:40 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:37721 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358740AbiBDL5p (ORCPT ); Fri, 4 Feb 2022 06:57:45 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643975864; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sArPDuM16yDbcx1bDIyT7nyZtdEl0hoR3jX/2tqV6pM=; b=RnDd6BYMbo+k4aJxIIyhis1IFs5zz6OPlIZijMMfQ7AU33ogp8OM4ppUnm8WjPJkDP1PGk VQv2igHAeNooRLW6SDPdVdXu5Y7qdUv23G07htpSiEhyud6LbhQdT220EIzvPoOuvvVdS1 FP3VB7sqwzLMtGYqr4YemZRVfhchpQA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-387-cy8ZjzniPEy0wBs-N2eeZg-1; Fri, 04 Feb 2022 06:57:39 -0500 X-MC-Unique: cy8ZjzniPEy0wBs-N2eeZg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 88FD81015DA2; Fri, 4 Feb 2022 11:57:38 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1C8277CAEF; Fri, 4 Feb 2022 11:57:38 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: dmatlack@google.com, seanjc@google.com, vkuznets@redhat.com Subject: [PATCH 23/23] KVM: MMU: replace direct_map with mmu_role.direct Date: Fri, 4 Feb 2022 06:57:18 -0500 Message-Id: <20220204115718.14934-24-pbonzini@redhat.com> In-Reply-To: <20220204115718.14934-1-pbonzini@redhat.com> References: <20220204115718.14934-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" direct_map is always equal to the role's direct field: - for shadow paging, direct_map is true if CR0.PG=3D0 and mmu_role.direct is copied from cpu_role.base.direct - for TDP, it is always true and mmu_role.direct is also always true - for shadow EPT, it is always false and mmu_role.direct is also always false Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/mmu/mmu.c | 30 ++++++++++++++---------------- arch/x86/kvm/x86.c | 12 ++++++------ 4 files changed, 21 insertions(+), 23 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index c86a2beee92a..647b3f6d02d0 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -432,7 +432,6 @@ struct kvm_mmu { gpa_t root_pgd; union kvm_mmu_role cpu_role; union kvm_mmu_page_role mmu_role; - bool direct_map; struct kvm_mmu_root_info prev_roots[KVM_MMU_NUM_PREV_ROOTS]; =20 /* diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5a6541d6a424..ce55fad99671 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2045,7 +2045,7 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct k= vm_vcpu *vcpu, int direct, unsigned int access) { - bool direct_mmu =3D vcpu->arch.mmu->direct_map; + bool direct_mmu =3D vcpu->arch.mmu->mmu_role.direct; union kvm_mmu_page_role role; struct hlist_head *sp_list; unsigned quadrant; @@ -2147,7 +2147,7 @@ static void shadow_walk_init_using_root(struct kvm_sh= adow_walk_iterator *iterato =20 if (iterator->level >=3D PT64_ROOT_4LEVEL && vcpu->arch.mmu->cpu_role.base.level < PT64_ROOT_4LEVEL && - !vcpu->arch.mmu->direct_map) + !vcpu->arch.mmu->mmu_role.direct) iterator->level =3D PT32E_ROOT_LEVEL; =20 if (iterator->level =3D=3D PT32E_ROOT_LEVEL) { @@ -2523,7 +2523,7 @@ static int kvm_mmu_unprotect_page_virt(struct kvm_vcp= u *vcpu, gva_t gva) gpa_t gpa; int r; =20 - if (vcpu->arch.mmu->direct_map) + if (vcpu->arch.mmu->mmu_role.direct) return 0; =20 gpa =3D kvm_mmu_gva_to_gpa_read(vcpu, gva, NULL); @@ -3255,7 +3255,8 @@ void kvm_mmu_free_roots(struct kvm_vcpu *vcpu, struct= kvm_mmu *mmu, =20 if (free_active_root) { if (mmu->mmu_role.level >=3D PT64_ROOT_4LEVEL && - (mmu->cpu_role.base.level >=3D PT64_ROOT_4LEVEL || mmu->direct_map))= { + (mmu->cpu_role.base.level >=3D PT64_ROOT_4LEVEL || + mmu->mmu_role.direct)) { mmu_free_root_page(kvm, &mmu->root_hpa, &invalid_list); } else if (mmu->pae_root) { for (i =3D 0; i < 4; ++i) { @@ -3558,7 +3559,8 @@ static int mmu_alloc_special_roots(struct kvm_vcpu *v= cpu) * equivalent level in the guest's NPT to shadow. Allocate the tables * on demand, as running a 32-bit L1 VMM on 64-bit KVM is very rare. */ - if (mmu->direct_map || mmu->cpu_role.base.level >=3D PT64_ROOT_4LEVEL || + if (mmu->mmu_role.direct || + mmu->cpu_role.base.level >=3D PT64_ROOT_4LEVEL || mmu->mmu_role.level < PT64_ROOT_4LEVEL) return 0; =20 @@ -3647,7 +3649,7 @@ void kvm_mmu_sync_roots(struct kvm_vcpu *vcpu) int i; struct kvm_mmu_page *sp; =20 - if (vcpu->arch.mmu->direct_map) + if (vcpu->arch.mmu->mmu_role.direct) return; =20 if (!VALID_PAGE(vcpu->arch.mmu->root_hpa)) @@ -3872,7 +3874,7 @@ static bool kvm_arch_setup_async_pf(struct kvm_vcpu *= vcpu, gpa_t cr2_or_gpa, =20 arch.token =3D (vcpu->arch.apf.id++ << 12) | vcpu->vcpu_id; arch.gfn =3D gfn; - arch.direct_map =3D vcpu->arch.mmu->direct_map; + arch.direct_map =3D vcpu->arch.mmu->mmu_role.direct; arch.cr3 =3D vcpu->arch.mmu->get_guest_pgd(vcpu); =20 return kvm_setup_async_pf(vcpu, cr2_or_gpa, @@ -4090,7 +4092,6 @@ static void nonpaging_init_context(struct kvm_mmu *co= ntext) context->gva_to_gpa =3D nonpaging_gva_to_gpa; context->sync_page =3D nonpaging_sync_page; context->invlpg =3D NULL; - context->direct_map =3D true; } =20 static inline bool is_root_usable(struct kvm_mmu_root_info *root, gpa_t pg= d, @@ -4641,7 +4642,6 @@ static void paging64_init_context(struct kvm_mmu *con= text) context->gva_to_gpa =3D paging64_gva_to_gpa; context->sync_page =3D paging64_sync_page; context->invlpg =3D paging64_invlpg; - context->direct_map =3D false; } =20 static void paging32_init_context(struct kvm_mmu *context) @@ -4650,7 +4650,6 @@ static void paging32_init_context(struct kvm_mmu *con= text) context->gva_to_gpa =3D paging32_gva_to_gpa; context->sync_page =3D paging32_sync_page; context->invlpg =3D paging32_invlpg; - context->direct_map =3D false; } =20 static union kvm_mmu_role @@ -4735,7 +4734,6 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu, u= nion kvm_mmu_role cpu_role) context->page_fault =3D kvm_tdp_page_fault; context->sync_page =3D nonpaging_sync_page; context->invlpg =3D NULL; - context->direct_map =3D true; context->get_guest_pgd =3D get_cr3; context->get_pdptr =3D kvm_pdptr_read; context->inject_page_fault =3D kvm_inject_page_fault; @@ -4852,7 +4850,7 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, b= ool execonly, context->gva_to_gpa =3D ept_gva_to_gpa; context->sync_page =3D ept_sync_page; context->invlpg =3D ept_invlpg; - context->direct_map =3D false; + update_permission_bitmask(context, true); context->pkru_mask =3D 0; reset_rsvds_bits_mask_ept(vcpu, context, execonly, huge_page_level); @@ -4967,13 +4965,13 @@ int kvm_mmu_load(struct kvm_vcpu *vcpu) { int r; =20 - r =3D mmu_topup_memory_caches(vcpu, !vcpu->arch.mmu->direct_map); + r =3D mmu_topup_memory_caches(vcpu, !vcpu->arch.mmu->mmu_role.direct); if (r) goto out; r =3D mmu_alloc_special_roots(vcpu); if (r) goto out; - if (vcpu->arch.mmu->direct_map) + if (vcpu->arch.mmu->mmu_role.direct) r =3D mmu_alloc_direct_roots(vcpu); else r =3D mmu_alloc_shadow_roots(vcpu); @@ -5176,7 +5174,7 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t c= r2_or_gpa, u64 error_code, void *insn, int insn_len) { int r, emulation_type =3D EMULTYPE_PF; - bool direct =3D vcpu->arch.mmu->direct_map; + bool direct =3D vcpu->arch.mmu->mmu_role.direct; =20 if (WARN_ON(!VALID_PAGE(vcpu->arch.mmu->root_hpa))) return RET_PF_RETRY; @@ -5207,7 +5205,7 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t c= r2_or_gpa, u64 error_code, * paging in both guests. If true, we simply unprotect the page * and resume the guest. */ - if (vcpu->arch.mmu->direct_map && + if (vcpu->arch.mmu->mmu_role.direct && (error_code & PFERR_NESTED_GUEST_PAGE) =3D=3D PFERR_NESTED_GUEST_PAGE= ) { kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(cr2_or_gpa)); return 1; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 657aa646871e..b910fa34e57e 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -7978,7 +7978,7 @@ static bool reexecute_instruction(struct kvm_vcpu *vc= pu, gpa_t cr2_or_gpa, WARN_ON_ONCE(!(emulation_type & EMULTYPE_PF))) return false; =20 - if (!vcpu->arch.mmu->direct_map) { + if (!vcpu->arch.mmu->mmu_role.direct) { /* * Write permission should be allowed since only * write access need to be emulated. @@ -8011,7 +8011,7 @@ static bool reexecute_instruction(struct kvm_vcpu *vc= pu, gpa_t cr2_or_gpa, kvm_release_pfn_clean(pfn); =20 /* The instructions are well-emulated on direct mmu. */ - if (vcpu->arch.mmu->direct_map) { + if (vcpu->arch.mmu->mmu_role.direct) { unsigned int indirect_shadow_pages; =20 write_lock(&vcpu->kvm->mmu_lock); @@ -8079,7 +8079,7 @@ static bool retry_instruction(struct x86_emulate_ctxt= *ctxt, vcpu->arch.last_retry_eip =3D ctxt->eip; vcpu->arch.last_retry_addr =3D cr2_or_gpa; =20 - if (!vcpu->arch.mmu->direct_map) + if (!vcpu->arch.mmu->mmu_role.direct) gpa =3D kvm_mmu_gva_to_gpa_write(vcpu, cr2_or_gpa, NULL); =20 kvm_mmu_unprotect_page(vcpu->kvm, gpa_to_gfn(gpa)); @@ -8359,7 +8359,7 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gp= a_t cr2_or_gpa, ctxt->exception.address =3D cr2_or_gpa; =20 /* With shadow page tables, cr2 contains a GVA or nGPA. */ - if (vcpu->arch.mmu->direct_map) { + if (vcpu->arch.mmu->mmu_role.direct) { ctxt->gpa_available =3D true; ctxt->gpa_val =3D cr2_or_gpa; } @@ -12196,7 +12196,7 @@ void kvm_arch_async_page_ready(struct kvm_vcpu *vcp= u, struct kvm_async_pf *work) { int r; =20 - if ((vcpu->arch.mmu->direct_map !=3D work->arch.direct_map) || + if ((vcpu->arch.mmu->mmu_role.direct !=3D work->arch.direct_map) || work->wakeup_all) return; =20 @@ -12204,7 +12204,7 @@ void kvm_arch_async_page_ready(struct kvm_vcpu *vcp= u, struct kvm_async_pf *work) if (unlikely(r)) return; =20 - if (!vcpu->arch.mmu->direct_map && + if (!vcpu->arch.mmu->mmu_role.direct && work->arch.cr3 !=3D vcpu->arch.mmu->get_guest_pgd(vcpu)) return; =20 --=20 2.31.1