From nobody Sun Apr 26 16:02:03 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88A37CCA473 for ; Fri, 24 Jun 2022 21:31:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231518AbiFXVa7 (ORCPT ); Fri, 24 Jun 2022 17:30:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57994 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231423AbiFXVax (ORCPT ); Fri, 24 Jun 2022 17:30:53 -0400 Received: from mail-pf1-x449.google.com (mail-pf1-x449.google.com [IPv6:2607:f8b0:4864:20::449]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EEB8A7FD0E for ; Fri, 24 Jun 2022 14:30:52 -0700 (PDT) Received: by mail-pf1-x449.google.com with SMTP id n21-20020a056a000d5500b005251893308cso1638806pfv.6 for ; Fri, 24 Jun 2022 14:30:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=ICaGYy1bxClqsMr84xS9Z2iY/dNy9kl//MDmpzal8X4=; b=EuR98mP+tn9jie3sAeJxuGzTMcy/XECRgpnOsmsm5vvlMkYMIto/8JtRhSIjw9lMSf 1x72TWYnxmt53MX597QAUXAq1GcS98OkIDfi5g2oRBRx9l9QtSw/6RcIxm+URxRXvP7A 1qDqUQ72llWr2Neo171cQpFRwtwycRmuHG6T90ZCbPwzPrLL4EDDS96RBNYkIIKzd9uj O9x+voPh91LaBga+5WFN0RBy6XY3RMGuQnBI6G2z3Uakvdo+U9W9iv5+brYZpY7s8Pv+ liteH/FB2uTh1Ekc4XVQ0iB+xF7Volgpuxat8y7kvd/oRhGdgjWw5BdmzZwrajmTeDE/ 1pKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=ICaGYy1bxClqsMr84xS9Z2iY/dNy9kl//MDmpzal8X4=; b=8B81/rJWasQZaCP9xNXtuZTEp3DBr+crW5iirwx+b0f6q7ShDUzT9lRIbQcA0gdRwY 186iNUFb+WyIvxh5+YCpEd4d0ApTLzkQTaIMdnDR8080CFmvgUPw5t0JcIBNv+/5RabS L2iVkuogl4jV8huwAzfTqhtP0TBtXB8vt9PHZbI5LYTdc4xReVBVmcI1J3xezsfpa4G5 sx4aWwaFOlvq4t7W/wO9Fm5PyuJvIyM38jvCsGZhlV6Mg68nKmio8qIyjEbFK274c4wF H0q9KCenl92fAd7N7lpu/8AN7Srm1aeQmi25lX2VVjax4Kku4lBESZ25dW9Q85BktsPm QFWg== X-Gm-Message-State: AJIora9xZYVvLYmoaHrdNfoIO974+xPWXDjorw3297EVgPaAKyJxOJjH W247K2fPrCmImgLDXYHCcIiPts+xrdc= X-Google-Smtp-Source: AGRyM1soHmDol5uisTNLRvpaxaglJLD4MnzM/G9seXjCj78n330EZb2VxQUAG9Uc7TQCeviIawphfk/SLPk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:174b:b0:525:4eea:8ff2 with SMTP id j11-20020a056a00174b00b005254eea8ff2mr1147112pfc.23.1656106252545; Fri, 24 Jun 2022 14:30:52 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 24 Jun 2022 21:30:36 +0000 In-Reply-To: <20220624213039.2872507-1-seanjc@google.com> Message-Id: <20220624213039.2872507-2-seanjc@google.com> Mime-Version: 1.0 References: <20220624213039.2872507-1-seanjc@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v2 1/4] KVM: x86/mmu: Add optimized helper to retrieve an SPTE's index From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, David Matlack Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add spte_index() to dedup all the code that calculates a SPTE's index into its parent's page table and/or spt array. Opportunistically tweak the calculation to avoid pointer arithmetic, which is subtle (subtract in 8-byte chunks) and less performant (requires the compiler to generate the subtraction). Suggested-by: David Matlack Signed-off-by: Sean Christopherson Reviewed-by: David Matlack --- arch/x86/kvm/mmu/mmu.c | 22 ++++++++++------------ arch/x86/kvm/mmu/paging_tmpl.h | 4 ++-- arch/x86/kvm/mmu/spte.h | 6 ++++++ 3 files changed, 18 insertions(+), 14 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index bd74a287b54a..b04e9ce2469a 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -1036,7 +1036,7 @@ static void rmap_remove(struct kvm *kvm, u64 *spte) struct kvm_rmap_head *rmap_head; =20 sp =3D sptep_to_sp(spte); - gfn =3D kvm_mmu_page_get_gfn(sp, spte - sp->spt); + gfn =3D kvm_mmu_page_get_gfn(sp, spte_index(spte)); =20 /* * Unlike rmap_add, rmap_remove does not run in the context of a vCPU @@ -1587,7 +1587,7 @@ static void __rmap_add(struct kvm *kvm, int rmap_count; =20 sp =3D sptep_to_sp(spte); - kvm_mmu_page_set_translation(sp, spte - sp->spt, gfn, access); + kvm_mmu_page_set_translation(sp, spte_index(spte), gfn, access); kvm_update_page_stats(kvm, sp->role.level, 1); =20 rmap_head =3D gfn_to_rmap(gfn, sp->role.level, slot); @@ -1714,11 +1714,9 @@ static void kvm_mmu_mark_parents_unsync(struct kvm_m= mu_page *sp) static void mark_unsync(u64 *spte) { struct kvm_mmu_page *sp; - unsigned int index; =20 sp =3D sptep_to_sp(spte); - index =3D spte - sp->spt; - if (__test_and_set_bit(index, sp->unsync_child_bitmap)) + if (__test_and_set_bit(spte_index(spte), sp->unsync_child_bitmap)) return; if (sp->unsync_children++) return; @@ -2201,7 +2199,7 @@ static union kvm_mmu_page_role kvm_mmu_child_role(u64= *sptep, bool direct, unsig */ if (role.has_4_byte_gpte) { WARN_ON_ONCE(role.level !=3D PG_LEVEL_4K); - role.quadrant =3D (sptep - parent_sp->spt) % 2; + role.quadrant =3D spte_index(sptep) & 1; } =20 return role; @@ -2826,7 +2824,7 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct= kvm_memory_slot *slot, rmap_add(vcpu, slot, sptep, gfn, pte_access); } else { /* Already rmapped but the pte_access bits may have changed. */ - kvm_mmu_page_set_access(sp, sptep - sp->spt, pte_access); + kvm_mmu_page_set_access(sp, spte_index(sptep), pte_access); } =20 return ret; @@ -2842,7 +2840,7 @@ static int direct_pte_prefetch_many(struct kvm_vcpu *= vcpu, int i, ret; gfn_t gfn; =20 - gfn =3D kvm_mmu_page_get_gfn(sp, start - sp->spt); + gfn =3D kvm_mmu_page_get_gfn(sp, spte_index(start)); slot =3D gfn_to_memslot_dirty_bitmap(vcpu, gfn, access & ACC_WRITE_MASK); if (!slot) return -1; @@ -2868,7 +2866,7 @@ static void __direct_pte_prefetch(struct kvm_vcpu *vc= pu, =20 WARN_ON(!sp->role.direct); =20 - i =3D (sptep - sp->spt) & ~(PTE_PREFETCH_NUM - 1); + i =3D spte_index(sptep) & ~(PTE_PREFETCH_NUM - 1); spte =3D sp->spt + i; =20 for (i =3D 0; i < PTE_PREFETCH_NUM; i++, spte++) { @@ -6146,8 +6144,8 @@ static struct kvm_mmu_page *shadow_mmu_get_sp_for_spl= it(struct kvm *kvm, u64 *hu unsigned int access; gfn_t gfn; =20 - gfn =3D kvm_mmu_page_get_gfn(huge_sp, huge_sptep - huge_sp->spt); - access =3D kvm_mmu_page_get_access(huge_sp, huge_sptep - huge_sp->spt); + gfn =3D kvm_mmu_page_get_gfn(huge_sp, spte_index(huge_sptep)); + access =3D kvm_mmu_page_get_access(huge_sp, spte_index(huge_sptep)); =20 /* * Note, huge page splitting always uses direct shadow pages, regardless @@ -6221,7 +6219,7 @@ static int shadow_mmu_try_split_huge_page(struct kvm = *kvm, u64 spte; =20 /* Grab information for the tracepoint before dropping the MMU lock. */ - gfn =3D kvm_mmu_page_get_gfn(huge_sp, huge_sptep - huge_sp->spt); + gfn =3D kvm_mmu_page_get_gfn(huge_sp, spte_index(huge_sptep)); level =3D huge_sp->role.level; spte =3D *huge_sptep; =20 diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 2448fa8d8438..d06dee7d38a8 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -595,7 +595,7 @@ static void FNAME(pte_prefetch)(struct kvm_vcpu *vcpu, = struct guest_walker *gw, if (sp->role.direct) return __direct_pte_prefetch(vcpu, sp, sptep); =20 - i =3D (sptep - sp->spt) & ~(PTE_PREFETCH_NUM - 1); + i =3D spte_index(sptep) & ~(PTE_PREFETCH_NUM - 1); spte =3D sp->spt + i; =20 for (i =3D 0; i < PTE_PREFETCH_NUM; i++, spte++) { @@ -933,7 +933,7 @@ static void FNAME(invlpg)(struct kvm_vcpu *vcpu, gva_t = gva, hpa_t root_hpa) break; =20 pte_gpa =3D FNAME(get_level1_sp_gpa)(sp); - pte_gpa +=3D (sptep - sp->spt) * sizeof(pt_element_t); + pte_gpa +=3D spte_index(sptep) * sizeof(pt_element_t); =20 mmu_page_zap_pte(vcpu->kvm, sp, sptep, NULL); if (is_shadow_present_pte(old_spte)) diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index b5c855f5514f..ba3dccb202bc 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -190,6 +190,12 @@ static inline bool is_removed_spte(u64 spte) return spte =3D=3D REMOVED_SPTE; } =20 +/* Get an SPTE's index into its parent's page table (and the spt array). */ +static inline int spte_index(u64 *sptep) +{ + return ((unsigned long)sptep / sizeof(*sptep)) & (SPTE_ENT_PER_PAGE - 1); +} + /* * In some cases, we need to preserve the GFN of a non-present or reserved * SPTE when we usurp the upper five bits of the physical address space to --=20 2.37.0.rc0.161.g10f37bed90-goog From nobody Sun Apr 26 16:02:03 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA225C43334 for ; Fri, 24 Jun 2022 21:31:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231538AbiFXVbA (ORCPT ); Fri, 24 Jun 2022 17:31:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231459AbiFXVaz (ORCPT ); Fri, 24 Jun 2022 17:30:55 -0400 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF3157FD39 for ; Fri, 24 Jun 2022 14:30:54 -0700 (PDT) Received: by mail-pg1-x549.google.com with SMTP id 15-20020a63020f000000b003fca9ebc5cbso1546806pgc.22 for ; Fri, 24 Jun 2022 14:30:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=aVL8GM5UlCYwkYpjstIne4ze+uj4PFzbjfg18IJWQlI=; b=MNWyh9kWGEe9f3b2ZEyrpQTABz+TEx0AgE7yK61YYChhZL7DpPrgNJw4jZlShrPH/T 6yxzaoxZaBvuHcdqqjG151wMMqI9GWvT+vTYyU8+pqY1vAmC0Vby0UZt58enP6mxuhOz hwgvzRX3qsiRUe9uezVcYWRZwbNL4DurMF99wwND4yZ9krO96vGb41YJMkO76M3cDHEM metf14q/h4Gfwimqb6G6rfwRVgJxjeupcvFyno3QqtZq/vLSk66tFvf5CNz38R+j3yxN bFAYk1w781QG4K555lmuC80JPNPT6FcRFRaYpj2+WrdQ5XXODK5MFTqSBdkf10lJvPo+ bHcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=aVL8GM5UlCYwkYpjstIne4ze+uj4PFzbjfg18IJWQlI=; b=kPFQW2WJCVFhgf1pyruhcVXQYpWqacsthxk5cgEX6PafU+J4Db+nsKZqOvNV27/rWm dUaKtEfW/+Ne0pFuBwyYJZ03CFd/JE+ICaiW1VmgpacZ4S3a18ZgZMk8g0fIATDVNlmW y2Rmmx20srFtAAIlyKt3BjruUNTOUIGrazMg8vRK+WEkbdFRn2LdLpHL8NGSimh1XTh6 BcGynVJeQJJ0CsFJIDrrhFIB+SbHDRnBw4wSi5DhMzB/6f9yf8I85r62q3izCYEl3Eq7 ezTJc/7liVUzrZMkyj5Ic1yUlWdMuBj3nBnkRsPKGUsto7LUzh+UBzuRgfR/2jHcFnD6 JsTA== X-Gm-Message-State: AJIora/eJPJVIUa3JJJAewDOY37ZEBqK1xEn+Hmy36c0xZw9jTbgQNjD 5VWJehEHdF2oiqTi5juGBCIJbqNrYXE= X-Google-Smtp-Source: AGRyM1sMeT4iPH4Z55LvwsYjBdCB6PioE05w9SsEBuG3UpD3DTmlohwidwuec0jKYnkeFNx6hlYHlWLKjBQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:e809:b0:16a:22dc:d23a with SMTP id u9-20020a170902e80900b0016a22dcd23amr1098100plg.119.1656106254367; Fri, 24 Jun 2022 14:30:54 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 24 Jun 2022 21:30:37 +0000 In-Reply-To: <20220624213039.2872507-1-seanjc@google.com> Message-Id: <20220624213039.2872507-3-seanjc@google.com> Mime-Version: 1.0 References: <20220624213039.2872507-1-seanjc@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v2 2/4] KVM: x86/mmu: Expand quadrant comment for PG_LEVEL_4K shadow pages From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, David Matlack Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Tweak the comment above the computation of the quadrant for PG_LEVEL_4K shadow pages to explicitly call out how and why KVM uses role.quadrant to consume gPTE bits. Opportunistically wrap an unnecessarily long line. No functional change intended. Link: https://lore.kernel.org/all/YqvWvBv27fYzOFdE@google.com Cc: David Matlack Signed-off-by: Sean Christopherson Reviewed-by: David Matlack --- arch/x86/kvm/mmu/mmu.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index b04e9ce2469a..83ca71361acd 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2166,7 +2166,8 @@ static struct kvm_mmu_page *kvm_mmu_get_shadow_page(s= truct kvm_vcpu *vcpu, return __kvm_mmu_get_shadow_page(vcpu->kvm, vcpu, &caches, gfn, role); } =20 -static union kvm_mmu_page_role kvm_mmu_child_role(u64 *sptep, bool direct,= unsigned int access) +static union kvm_mmu_page_role kvm_mmu_child_role(u64 *sptep, bool direct, + unsigned int access) { struct kvm_mmu_page *parent_sp =3D sptep_to_sp(sptep); union kvm_mmu_page_role role; @@ -2193,9 +2194,15 @@ static union kvm_mmu_page_role kvm_mmu_child_role(u6= 4 *sptep, bool direct, unsig * uses 2 PAE page tables, each mapping a 2MiB region. For these, * @role.quadrant encodes which half of the region they map. * - * Note, the 4 PAE page directories are pre-allocated and the quadrant - * assigned in mmu_alloc_root(). So only page tables need to be handled - * here. + * Concretely, a 4-byte PDE consumes bits 31:22, while an 8-byte PDE + * consumes bits 29:21. To consume bits 31:30, KVM's uses 4 shadow + * PDPTEs; those 4 PAE page directories are pre-allocated and their + * quadrant is assigned in mmu_alloc_root(). A 4-byte PTE consumes + * bits 21:12, while an 8-byte PTE consumes bits 20:12. To consume + * bit 21 in the PTE (the child here), KVM propagates that bit to the + * quadrant, i.e. sets quadrant to '0' or '1'. The parent 8-byte PDE + * covers bit 21 (see above), thus the quadrant is calculated from the + * _least_ significant bit of the PDE index. */ if (role.has_4_byte_gpte) { WARN_ON_ONCE(role.level !=3D PG_LEVEL_4K); --=20 2.37.0.rc0.161.g10f37bed90-goog From nobody Sun Apr 26 16:02:03 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1E6CC43334 for ; Fri, 24 Jun 2022 21:31:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231190AbiFXVbE (ORCPT ); Fri, 24 Jun 2022 17:31:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231467AbiFXVa5 (ORCPT ); Fri, 24 Jun 2022 17:30:57 -0400 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AA32581269 for ; Fri, 24 Jun 2022 14:30:56 -0700 (PDT) Received: by mail-pg1-x549.google.com with SMTP id o22-20020a637316000000b0040d238478aeso1557602pgc.2 for ; Fri, 24 Jun 2022 14:30:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=+jnXhzHCPLlyWGuqZQ0mGasvqcBiUClEYUi7f6uXGnA=; b=oMU1MlW+3HAUB8TWEcHWIqU6v3ZgrNxHm+Tp3iBDMZrqPmr/WGzoH7HQuTSZdLaO52 Gb1sdyM9iEwOq8CHr2baCpaLbkv06tIYZyXwdAyxq3b+9mOXSiCveMZcDoIhpN7XNFoP AZz1KEI2BDu64hW6HRfrsp4y039H0MhY5iqm2PIwz8x4excaZ6az3/sRqSm3mhfSVDbO ZRVccTtvLbqt+A1utFOFQShRJA2OcrbxF2gm0h390trx4zGXGf4iEvd5XVXiTV+mycc4 g5TfbT4xRTTs7G7PskXxxsTv1ArM8+7fgiD2df4FrniuyouldzbJ9a4LkzRRa+YWbE7o JHlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=+jnXhzHCPLlyWGuqZQ0mGasvqcBiUClEYUi7f6uXGnA=; b=QWE0lVxrZtsF5sh+3CnlKPSy4HFlk0UjHMU93DbbLD5pV8Aogb4kdQpO7bwtT3yNvy 80NQb4DAJAkYUyrqps7TOMxY3TAWayDOIUbIqaLL+DrOnMe9l1kqn+ShNyDJ57Nhc86O WT5s8Tib2cBRpjqiJZUZ4ADR6Cv/esPaZrdtA+RrULBgMgvCocg6HGWzxpVwlVLz6kfm L/3zgrfKeLfn3+jW453PPyD1qM+uUOT7XxhvXyiPSao/yfU3ulDpZTwvhfCHgwX5zG2K rrXL+jIt1t596k25Vg2wlM+637PK4IxvQ8ZeOfO8ERYw0BDWk6Nw2ccBST11AO/W8nVr 2mhw== X-Gm-Message-State: AJIora/LBUTgMFtYe1rscLYvGTxesk6rK97YfydBUs/O1fEZsEylcabi 8UM9nx/8m3LoMiDTE+G3pnmz3mmAVKc= X-Google-Smtp-Source: AGRyM1tzdjiNisYNqq4YKLQi4OaXM7gDixiQ7ghRnaWpvtULrzdW//5VP9BJ3ETreJf22QjtlykemyEnLqM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90a:4a97:b0:1ea:fa24:467c with SMTP id f23-20020a17090a4a9700b001eafa24467cmr389269pjh.1.1656106255902; Fri, 24 Jun 2022 14:30:55 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 24 Jun 2022 21:30:38 +0000 In-Reply-To: <20220624213039.2872507-1-seanjc@google.com> Message-Id: <20220624213039.2872507-4-seanjc@google.com> Mime-Version: 1.0 References: <20220624213039.2872507-1-seanjc@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v2 3/4] KVM: x86/mmu: Use "unsigned int", not "u32", for SPTEs' @access info From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, David Matlack Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use an "unsigned int" for @access parameters instead of a "u32", mostly to be consistent throughout KVM, but also because "u32" is misleading. @access can actually squeeze into a u8, i.e. doesn't need 32 bits, but is as an "unsigned int" because sp->role.access is an unsigned int. No functional change intended. Link: https://lore.kernel.org/all/YqyZxEfxXLsHGoZ%2F@google.com Reviewed-by: David Matlack Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 83ca71361acd..eae5c801e442 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -717,7 +717,8 @@ static u32 kvm_mmu_page_get_access(struct kvm_mmu_page = *sp, int index) return sp->role.access; } =20 -static void kvm_mmu_page_set_translation(struct kvm_mmu_page *sp, int inde= x, gfn_t gfn, u32 access) +static void kvm_mmu_page_set_translation(struct kvm_mmu_page *sp, int inde= x, + gfn_t gfn, unsigned int access) { if (sp_has_gptes(sp)) { sp->shadowed_translation[index] =3D (gfn << PAGE_SHIFT) | access; @@ -735,7 +736,8 @@ static void kvm_mmu_page_set_translation(struct kvm_mmu= _page *sp, int index, gfn sp->gfn, kvm_mmu_page_get_gfn(sp, index), gfn); } =20 -static void kvm_mmu_page_set_access(struct kvm_mmu_page *sp, int index, u3= 2 access) +static void kvm_mmu_page_set_access(struct kvm_mmu_page *sp, int index, + unsigned int access) { gfn_t gfn =3D kvm_mmu_page_get_gfn(sp, index); =20 @@ -1580,7 +1582,7 @@ static bool kvm_test_age_rmapp(struct kvm *kvm, struc= t kvm_rmap_head *rmap_head, static void __rmap_add(struct kvm *kvm, struct kvm_mmu_memory_cache *cache, const struct kvm_memory_slot *slot, - u64 *spte, gfn_t gfn, u32 access) + u64 *spte, gfn_t gfn, unsigned int access) { struct kvm_mmu_page *sp; struct kvm_rmap_head *rmap_head; @@ -1601,7 +1603,7 @@ static void __rmap_add(struct kvm *kvm, } =20 static void rmap_add(struct kvm_vcpu *vcpu, const struct kvm_memory_slot *= slot, - u64 *spte, gfn_t gfn, u32 access) + u64 *spte, gfn_t gfn, unsigned int access) { struct kvm_mmu_memory_cache *cache =3D &vcpu->arch.mmu_pte_list_desc_cach= e; =20 --=20 2.37.0.rc0.161.g10f37bed90-goog From nobody Sun Apr 26 16:02:03 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54272C43334 for ; Fri, 24 Jun 2022 21:31:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231467AbiFXVbG (ORCPT ); Fri, 24 Jun 2022 17:31:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231508AbiFXVa6 (ORCPT ); Fri, 24 Jun 2022 17:30:58 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0139A81702 for ; Fri, 24 Jun 2022 14:30:58 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id u13-20020a170902e5cd00b0016a53274671so1843212plf.15 for ; Fri, 24 Jun 2022 14:30:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=FBTaEZTEg5GSVpewRBYKdCI+FLeByH1Vcwz0M7m4O+0=; b=SWbZ73EHxyjh8o5NT5+YNokKeaatVJ46ekafm3q2KaueYufnteAQ+nNnv0g8VuQmzQ dHdFDYemFGBvCT+4wkiipM0eUtXvZUQYQ/2yu1xl/ziQv8ITCGWREJevCH/qZKYhaPqx JGlwkxDPC2I9qUSoxy/vzMyPDYGWqWUePXYQNHNloS7yfJiyjPmc5xjvxftfm3WGFA4x Ws8ly60Yx3TPDrBH5Iy6hhgA1rHtTn7701i3Tn/rhUObgTU8LfBPMX/VmGPMyk4xFrsf cd+bqJ0ZwVcILvOgSExuBkQ2KKX58+0nCQINRuuQjWpSwdslQIsAuu7Bxm1i+TquTdws OeYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=FBTaEZTEg5GSVpewRBYKdCI+FLeByH1Vcwz0M7m4O+0=; b=htIXv32aeleqn0UBWhSuRzLvsCD6nk9u4NvxUYNYsr1/onBL0QyAvil2D3rFFRIufZ sxaAI7dUp8PO0f2sus/9W6/Ly4V7WFCfUjUjvDVyMycxO0FSxdMxSEQWKfWyl0Fbareu sLFV1SrgEtRwF8ZKql4NHP+k71ETNEeGoXEVjVtvw7UD9u2E5yYu8x/xvk5aEgGfelFh Am/pyrTcNgJ3ndAV4ih0H1OPC+rcoH9OXA73Bl2ICDRxc03PpSXoyBa4Xp8FXYUVhGu5 y/bKRlS218I9RPCxdpDd8GM1Ny1po6WF8g6fk6SOpL0EPY6dN6g/flpozqY3Y/K30PAx CjVA== X-Gm-Message-State: AJIora/n4Re7a7QZpxn2YMzp6kg9rzDhgiaOv+6mvmh4CkGs+pE/8M12 mhx6iAb7XL+dn2Pc/zBE9O32C1d59xY= X-Google-Smtp-Source: AGRyM1sugnfEsmqVDhbHA4jyTxXo0iuQG8dGD8pRvqXU8Z2EFcOSud0AzOY1pj5YaZMeBTqxatGprCqYfyI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:d2c4:b0:16a:5c48:8312 with SMTP id n4-20020a170902d2c400b0016a5c488312mr1169374plc.45.1656106257561; Fri, 24 Jun 2022 14:30:57 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 24 Jun 2022 21:30:39 +0000 In-Reply-To: <20220624213039.2872507-1-seanjc@google.com> Message-Id: <20220624213039.2872507-5-seanjc@google.com> Mime-Version: 1.0 References: <20220624213039.2872507-1-seanjc@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v2 4/4] KVM: x86/mmu: Buffer nested MMU split_desc_cache only by default capacity From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, David Matlack Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Buffer split_desc_cache, the cache used to allcoate rmap list entries, only by the default cache capacity (currently 40), not by doubling the minimum (513). Aliasing L2 GPAs to L1 GPAs is uncommon, thus eager page splitting is unlikely to need 500+ entries. And because each object is (currently) a non-trivial 128 bytes (see struct pte_list_desc), those extra ~500 entries means KVM is in all likelihood wasting ~64kb of memory. Link: https://lore.kernel.org/all/YrTDcrsn0%2F+alpzf@google.com Reviewed-by: David Matlack Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index eae5c801e442..52664c3caaab 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -6123,17 +6123,26 @@ static bool need_topup_split_caches_or_resched(stru= ct kvm *kvm) =20 static int topup_split_caches(struct kvm *kvm) { - int r; - - lockdep_assert_held(&kvm->slots_lock); - /* - * Setting capacity =3D=3D min would cause KVM to drop mmu_lock even if - * just one object was consumed from the cache, so make capacity - * larger than min. + * Allocating rmap list entries when splitting huge pages for nested + * MMUs is uncommon as KVM needs to use a list if and only if there is + * more than one rmap entry for a gfn, i.e. requires an L1 gfn to be + * aliased by multiple L2 gfns and/or from multiple nested roots with + * different roles. Aliasing gfns when using TDP is atypical for VMMs; + * a few gfns are often aliased during boot, e.g. when remapping BIOS, + * but aliasing rarely occurs post-boot or for many gfns. If there is + * only one rmap entry, rmap->val points directly at that one entry and + * doesn't need to allocate a list. Buffer the cache by the default + * capacity so that KVM doesn't have to drop mmu_lock to topup if KVM + * encounters an aliased gfn or two. */ - r =3D __kvm_mmu_topup_memory_cache(&kvm->arch.split_desc_cache, - 2 * SPLIT_DESC_CACHE_MIN_NR_OBJECTS, + const int capacity =3D SPLIT_DESC_CACHE_MIN_NR_OBJECTS + + KVM_ARCH_NR_OBJS_PER_MEMORY_CACHE; + int r; + + lockdep_assert_held(&kvm->slots_lock); + + r =3D __kvm_mmu_topup_memory_cache(&kvm->arch.split_desc_cache, capacity, SPLIT_DESC_CACHE_MIN_NR_OBJECTS); if (r) return r; --=20 2.37.0.rc0.161.g10f37bed90-goog