From nobody Wed Sep 17 09:00:43 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CCA3C4332F for ; Wed, 21 Dec 2022 22:26:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235091AbiLUW0L (ORCPT ); Wed, 21 Dec 2022 17:26:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235022AbiLUWZj (ORCPT ); Wed, 21 Dec 2022 17:25:39 -0500 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 518A727B09 for ; Wed, 21 Dec 2022 14:24:45 -0800 (PST) Received: by mail-pj1-x104a.google.com with SMTP id z4-20020a17090ab10400b002195a146546so1946742pjq.9 for ; Wed, 21 Dec 2022 14:24:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=4TdMmKeh9xdWHOqsXuzcwDMeXjRCekoXNuBTvbORk4Y=; b=KtV0pVIMfC+oSFEy+6xnaTAWg/otjtJQ3S9SFSJiaqziSxkFyLCWYd++OxLcdI3jLy 4C3EcV/Ykpd+HnLaclxh7dAhhVAMH+ziDlp47PaFQ91e+QjdlZFuVFPdGMEr8SQDDGbx 4/Qpcj76ezTxKGP/PoSZw5IxJNR4zFVMJw+5YccygGPHkQL/5G9lr6mY+qVDAXiRMPvO ySvjpodRibFvIKqH+VUZxLjHX8bpBBDQMz2OR1ENE/Bah4ZAlIy3N86pUGBWVap6Wjn7 WB8Yrqmpcq1Rq2w5AxKxOvBTJUTDeI9MXknD17Fi+uhVGb3ypwV5HytB2WWzpjdGu1z2 BZTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=4TdMmKeh9xdWHOqsXuzcwDMeXjRCekoXNuBTvbORk4Y=; b=wDCiLKFB4K04unlAzNer3tPMisU2OuWi9PATLf63dwxQZvWr4EvtcgL5fltiaLASlW X3sqkfvgYo3DGKrZmkFAzENB/VffT9G5BklvN5PyQSoQj8TmLzGHhBZf2xO7CjK77fnP rSP22t8XqjM/iTrVu3mZIWmYKaGqEJ/TN3dK2oCUiRW7NY2aFPwppPmL931Y82ZdD2+W u7V7u8VSB0B7cMwNx1nCA35C5BT58th/WJ9ZaGd+oX7C4u0CKKLNUABHZGTWzrFaybYE sFIVzifcUTvHpLCWtTPvVu0ItLtZo5bQDgF5pWlYKP4KkJy+/jgZuUrxKQVk8bXQ3OcR EcuQ== X-Gm-Message-State: AFqh2kpg1z9rQ16tHhPOyQfgtvIWysg6EjLK3Jiz8rNx/Lu6jYwBvv0V Et6z59OsPxWz4fmkeDyvtKvjCMYEQFvQloblu8wNy91ZS9EmLAbVbnZ0ahW4Y2WiLMGJYEPF/A6 GlG2HgRRkDefyjIRVHzn5Dp03GyVK4zvrAof1b3jewzBV8gf8QurGk0AYH+tmLGoIH4h0WCLV X-Google-Smtp-Source: AMrXdXvNqVXX6zU/V4HwnaQF1OmpHJ80B/IXAmbw1M/S+cN3+VYASEke/ukeHnemFxM/uBOEi3zro3UdIF9T X-Received: from sweer.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:e45]) (user=bgardon job=sendgmr) by 2002:a17:902:ab4e:b0:189:5c50:ce5 with SMTP id ij14-20020a170902ab4e00b001895c500ce5mr204757plb.14.1671661485557; Wed, 21 Dec 2022 14:24:45 -0800 (PST) Date: Wed, 21 Dec 2022 22:24:18 +0000 In-Reply-To: <20221221222418.3307832-1-bgardon@google.com> Mime-Version: 1.0 References: <20221221222418.3307832-1-bgardon@google.com> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog Message-ID: <20221221222418.3307832-15-bgardon@google.com> Subject: [RFC 14/14] KVM: x86/MMU: Add kvm_shadow_mmu_ to the last few functions in shadow_mmu.h From: Ben Gardon To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Peter Xu , Sean Christopherson , David Matlack , Vipin Sharma , Nagareddy Reddy , Ben Gardon Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Fix up the names of the last few Shadow MMU functions in shadow_mmu.h. This gives a clean and obvious interface between the shared x86 MMU code and the Shadow MMU. There are still a few functions exported from paging_tmpl.h that are left as-is, but changing those will need to be done separately, if at all. No functional change intended. Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/mmu.c | 23 ++++++++++-------- arch/x86/kvm/mmu/shadow_mmu.c | 44 +++++++++++++++++++---------------- arch/x86/kvm/mmu/shadow_mmu.h | 16 +++++++------ 3 files changed, 46 insertions(+), 37 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index ceb3146016d0..8f3b96af470d 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -921,9 +921,11 @@ static int fast_page_fault(struct kvm_vcpu *vcpu, stru= ct kvm_page_fault *fault) u64 new_spte; =20 if (is_tdp_mmu(vcpu->arch.mmu)) - sptep =3D kvm_tdp_mmu_fast_pf_get_last_sptep(vcpu, fault->addr, &spte); + sptep =3D kvm_tdp_mmu_fast_pf_get_last_sptep(vcpu, + fault->addr, &spte); else - sptep =3D fast_pf_get_last_sptep(vcpu, fault->addr, &spte); + sptep =3D kvm_shadow_mmu_fast_pf_get_last_sptep(vcpu, + fault->addr, &spte); =20 if (!is_shadow_present_pte(spte)) break; @@ -1113,7 +1115,7 @@ static int mmu_alloc_direct_roots(struct kvm_vcpu *vc= pu) root =3D kvm_tdp_mmu_get_vcpu_root_hpa(vcpu); mmu->root.hpa =3D root; } else if (shadow_root_level >=3D PT64_ROOT_4LEVEL) { - root =3D mmu_alloc_root(vcpu, 0, 0, shadow_root_level); + root =3D kvm_shadow_mmu_alloc_root(vcpu, 0, 0, shadow_root_level); mmu->root.hpa =3D root; } else if (shadow_root_level =3D=3D PT32E_ROOT_LEVEL) { if (WARN_ON_ONCE(!mmu->pae_root)) { @@ -1124,8 +1126,8 @@ static int mmu_alloc_direct_roots(struct kvm_vcpu *vc= pu) for (i =3D 0; i < 4; ++i) { WARN_ON_ONCE(IS_VALID_PAE_ROOT(mmu->pae_root[i])); =20 - root =3D mmu_alloc_root(vcpu, i << (30 - PAGE_SHIFT), 0, - PT32_ROOT_LEVEL); + root =3D kvm_shadow_mmu_alloc_root(vcpu, + i << (30 - PAGE_SHIFT), 0, PT32_ROOT_LEVEL); mmu->pae_root[i] =3D root | PT_PRESENT_MASK | shadow_me_value; } @@ -1665,7 +1667,7 @@ void kvm_mmu_new_pgd(struct kvm_vcpu *vcpu, gpa_t new= _pgd) * count. Otherwise, clear the write flooding count. */ if (!new_role.direct) - __clear_sp_write_flooding_count( + kvm_shadow_mmu_clear_sp_write_flooding_count( to_shadow_page(vcpu->arch.mmu->root.hpa)); } EXPORT_SYMBOL_GPL(kvm_mmu_new_pgd); @@ -2447,13 +2449,13 @@ int kvm_mmu_load(struct kvm_vcpu *vcpu) r =3D mmu_topup_memory_caches(vcpu, !vcpu->arch.mmu->root_role.direct); if (r) goto out; - r =3D mmu_alloc_special_roots(vcpu); + r =3D kvm_shadow_mmu_alloc_special_roots(vcpu); if (r) goto out; if (vcpu->arch.mmu->root_role.direct) r =3D mmu_alloc_direct_roots(vcpu); else - r =3D mmu_alloc_shadow_roots(vcpu); + r =3D kvm_shadow_mmu_alloc_shadow_roots(vcpu); if (r) goto out; =20 @@ -2679,7 +2681,8 @@ static int __kvm_mmu_create(struct kvm_vcpu *vcpu, st= ruct kvm_mmu *mmu) * generally doesn't use PAE paging and can skip allocating the PDP * table. The main exception, handled here, is SVM's 32-bit NPT. The * other exception is for shadowing L1's 32-bit or PAE NPT on 64-bit - * KVM; that horror is handled on-demand by mmu_alloc_special_roots(). + * KVM; that horror is handled on-demand by + * kvm_shadow_mmu_alloc_special_roots(). */ if (tdp_enabled && kvm_mmu_get_tdp_level(vcpu) > PT32E_ROOT_LEVEL) return 0; @@ -2820,7 +2823,7 @@ int kvm_mmu_init_vm(struct kvm *kvm) if (r < 0) return r; =20 - node->track_write =3D kvm_mmu_pte_write; + node->track_write =3D kvm_shadow_mmu_pte_write; node->track_flush_slot =3D kvm_mmu_invalidate_zap_pages_in_memslot; kvm_page_track_register_notifier(kvm, node); =20 diff --git a/arch/x86/kvm/mmu/shadow_mmu.c b/arch/x86/kvm/mmu/shadow_mmu.c index 1c6ff6fe3d2c..6f3e201af670 100644 --- a/arch/x86/kvm/mmu/shadow_mmu.c +++ b/arch/x86/kvm/mmu/shadow_mmu.c @@ -1402,14 +1402,14 @@ static int mmu_sync_children(struct kvm_vcpu *vcpu,= struct kvm_mmu_page *parent, return 0; } =20 -void __clear_sp_write_flooding_count(struct kvm_mmu_page *sp) +void kvm_shadow_mmu_clear_sp_write_flooding_count(struct kvm_mmu_page *sp) { atomic_set(&sp->write_flooding_count, 0); } =20 static void clear_sp_write_flooding_count(u64 *spte) { - __clear_sp_write_flooding_count(sptep_to_sp(spte)); + kvm_shadow_mmu_clear_sp_write_flooding_count(sptep_to_sp(spte)); } =20 /* @@ -1480,7 +1480,7 @@ static struct kvm_mmu_page *kvm_mmu_find_shadow_page(= struct kvm *kvm, kvm_flush_remote_tlbs(kvm); } =20 - __clear_sp_write_flooding_count(sp); + kvm_shadow_mmu_clear_sp_write_flooding_count(sp); =20 goto out; } @@ -1605,12 +1605,13 @@ static union kvm_mmu_page_role kvm_mmu_child_role(u= 64 *sptep, bool direct, * Concretely, a 4-byte PDE consumes bits 31:22, while an 8-byte PDE * consumes bits 29:21. To consume bits 31:30, KVM's uses 4 shadow * PDPTEs; those 4 PAE page directories are pre-allocated and their - * quadrant is assigned in mmu_alloc_root(). A 4-byte PTE consumes - * bits 21:12, while an 8-byte PTE consumes bits 20:12. To consume - * bit 21 in the PTE (the child here), KVM propagates that bit to the - * quadrant, i.e. sets quadrant to '0' or '1'. The parent 8-byte PDE - * covers bit 21 (see above), thus the quadrant is calculated from the - * _least_ significant bit of the PDE index. + * quadrant is assigned in kvm_shadow_mmu_alloc_root(). + * A 4-byte PTE consumes bits 21:12, while an 8-byte PTE consumes + * bits 20:12. To consume bit 21 in the PTE (the child here), KVM + * propagates that bit to the quadrant, i.e. sets quadrant to + * '0' or '1'. The parent 8-byte PDE covers bit 21 (see above), thus + * the quadrant is calculated from the _least_ significant bit of the + * PDE index. */ if (role.has_4_byte_gpte) { WARN_ON_ONCE(role.level !=3D PG_LEVEL_4K); @@ -2377,7 +2378,8 @@ int kvm_shadow_mmu_direct_map(struct kvm_vcpu *vcpu, = struct kvm_page_fault *faul * - Must be called between walk_shadow_page_lockless_{begin,end}. * - The returned sptep must not be used after walk_shadow_page_lockless_= end. */ -u64 *fast_pf_get_last_sptep(struct kvm_vcpu *vcpu, gpa_t gpa, u64 *spte) +u64 *kvm_shadow_mmu_fast_pf_get_last_sptep(struct kvm_vcpu *vcpu, gpa_t gp= a, + u64 *spte) { struct kvm_shadow_walk_iterator iterator; u64 old_spte; @@ -2430,7 +2432,8 @@ static int mmu_check_root(struct kvm_vcpu *vcpu, gfn_= t root_gfn) return ret; } =20 -hpa_t mmu_alloc_root(struct kvm_vcpu *vcpu, gfn_t gfn, int quadrant, u8 le= vel) +hpa_t kvm_shadow_mmu_alloc_root(struct kvm_vcpu *vcpu, gfn_t gfn, int quad= rant, + u8 level) { union kvm_mmu_page_role role =3D vcpu->arch.mmu->root_role; struct kvm_mmu_page *sp; @@ -2447,7 +2450,7 @@ hpa_t mmu_alloc_root(struct kvm_vcpu *vcpu, gfn_t gfn= , int quadrant, u8 level) return __pa(sp->spt); } =20 -static int mmu_first_shadow_root_alloc(struct kvm *kvm) +static int kvm_shadow_mmu_first_shadow_root_alloc(struct kvm *kvm) { struct kvm_memslots *slots; struct kvm_memory_slot *slot; @@ -2508,7 +2511,7 @@ static int mmu_first_shadow_root_alloc(struct kvm *kv= m) return r; } =20 -int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) +int kvm_shadow_mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) { struct kvm_mmu *mmu =3D vcpu->arch.mmu; u64 pdptrs[4], pm_mask; @@ -2537,7 +2540,7 @@ int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) } } =20 - r =3D mmu_first_shadow_root_alloc(vcpu->kvm); + r =3D kvm_shadow_mmu_first_shadow_root_alloc(vcpu->kvm); if (r) return r; =20 @@ -2551,8 +2554,8 @@ int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) * write-protect the guests page table root. */ if (mmu->cpu_role.base.level >=3D PT64_ROOT_4LEVEL) { - root =3D mmu_alloc_root(vcpu, root_gfn, 0, - mmu->root_role.level); + root =3D kvm_shadow_mmu_alloc_root(vcpu, root_gfn, 0, + mmu->root_role.level); mmu->root.hpa =3D root; goto set_root_pgd; } @@ -2605,7 +2608,8 @@ int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) */ quadrant =3D (mmu->cpu_role.base.level =3D=3D PT32_ROOT_LEVEL) ? i : 0; =20 - root =3D mmu_alloc_root(vcpu, root_gfn, quadrant, PT32_ROOT_LEVEL); + root =3D kvm_shadow_mmu_alloc_root(vcpu, root_gfn, quadrant, + PT32_ROOT_LEVEL); mmu->pae_root[i] =3D root | pm_mask; } =20 @@ -2624,7 +2628,7 @@ int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) return r; } =20 -int mmu_alloc_special_roots(struct kvm_vcpu *vcpu) +int kvm_shadow_mmu_alloc_special_roots(struct kvm_vcpu *vcpu) { struct kvm_mmu *mmu =3D vcpu->arch.mmu; bool need_pml5 =3D mmu->root_role.level > PT64_ROOT_4LEVEL; @@ -2997,8 +3001,8 @@ static u64 *get_written_sptes(struct kvm_mmu_page *sp= , gpa_t gpa, int *nspte) return spte; } =20 -void kvm_mmu_pte_write(struct kvm_vcpu *vcpu, gpa_t gpa, const u8 *new, - int bytes, struct kvm_page_track_notifier_node *node) +void kvm_shadow_mmu_pte_write(struct kvm_vcpu *vcpu, gpa_t gpa, const u8 *= new, + int bytes, struct kvm_page_track_notifier_node *node) { gfn_t gfn =3D gpa >> PAGE_SHIFT; struct kvm_mmu_page *sp; diff --git a/arch/x86/kvm/mmu/shadow_mmu.h b/arch/x86/kvm/mmu/shadow_mmu.h index 2ded3d674cb0..a3e6daa36236 100644 --- a/arch/x86/kvm/mmu/shadow_mmu.h +++ b/arch/x86/kvm/mmu/shadow_mmu.h @@ -26,7 +26,7 @@ struct pte_list_desc { /* Only exported for debugfs.c. */ unsigned int pte_list_count(struct kvm_rmap_head *rmap_head); =20 -void __clear_sp_write_flooding_count(struct kvm_mmu_page *sp); +void kvm_shadow_mmu_clear_sp_write_flooding_count(struct kvm_mmu_page *sp); =20 bool __kvm_shadow_mmu_prepare_zap_page(struct kvm *kvm, struct kvm_mmu_pag= e *sp, struct list_head *invalid_list, @@ -41,17 +41,19 @@ int kvm_shadow_mmu_make_pages_available(struct kvm_vcpu= *vcpu); int kvm_shadow_mmu_unprotect_page_virt(struct kvm_vcpu *vcpu, gva_t gva); =20 int kvm_shadow_mmu_direct_map(struct kvm_vcpu *vcpu, struct kvm_page_fault= *fault); -u64 *fast_pf_get_last_sptep(struct kvm_vcpu *vcpu, gpa_t gpa, u64 *spte); +u64 *kvm_shadow_mmu_fast_pf_get_last_sptep(struct kvm_vcpu *vcpu, gpa_t gp= a, + u64 *spte); =20 -hpa_t mmu_alloc_root(struct kvm_vcpu *vcpu, gfn_t gfn, int quadrant, u8 le= vel); -int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu); -int mmu_alloc_special_roots(struct kvm_vcpu *vcpu); +hpa_t kvm_shadow_mmu_alloc_root(struct kvm_vcpu *vcpu, gfn_t gfn, int quad= rant, + u8 level); +int kvm_shadow_mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu); +int kvm_shadow_mmu_alloc_special_roots(struct kvm_vcpu *vcpu); =20 int kvm_shadow_mmu_get_walk(struct kvm_vcpu *vcpu, u64 addr, u64 *sptes, int *root_level); =20 -void kvm_mmu_pte_write(struct kvm_vcpu *vcpu, gpa_t gpa, const u8 *new, - int bytes, struct kvm_page_track_notifier_node *node); +void kvm_shadow_mmu_pte_write(struct kvm_vcpu *vcpu, gpa_t gpa, const u8 *= new, + int bytes, struct kvm_page_track_notifier_node *node); =20 void kvm_shadow_mmu_zap_obsolete_pages(struct kvm *kvm); bool kvm_shadow_mmu_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t = gfn_end); --=20 2.39.0.314.g84b9a713c41-goog