From nobody Fri Jun 19 16:54:19 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7B44C433F5 for ; Fri, 1 Apr 2022 06:36:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245466AbiDAGii (ORCPT ); Fri, 1 Apr 2022 02:38:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44338 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236461AbiDAGid (ORCPT ); Fri, 1 Apr 2022 02:38:33 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 302DB18C0FB for ; Thu, 31 Mar 2022 23:36:44 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id j1-20020a170903028100b0014b1f9e0068so1017673plr.8 for ; Thu, 31 Mar 2022 23:36:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=ENE9zx/RQL0mygOY8ggwW7MkJ6Wz+nE6nyB1FT/1jsI=; b=g/PXhHA9mEdDh3slFWre1e8TY+VvyivH7kjt/GwKq6ffkDtuX8VopYGCymUtyb6RhB ewGF1UtAPU9Sa4dDc2iZ/Vy4PCNdUU1oab2oehFVqdpd50H5GyiHav8G1r3hK+dGQCEd 2M03mMQ3XT30DZuUkxXGbS8wIldDzpmGyJbXpYUzJ+B0h0Vq5/FZqP5GXso0g7lZAPDa Z9wniQhVdLTYYSQpK2p/N6gEsY02XpfE5LgltNjUxFVZwNvPXeoTjlGQpvck6zw5v86S GSJgvXA2Y+rWm0Zj/zC6O02GcPTDMoUv5jdOBqHc2jB9el+gIeA2f0LV9z9IhvWcNZ+u Z4MA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=ENE9zx/RQL0mygOY8ggwW7MkJ6Wz+nE6nyB1FT/1jsI=; b=RDDwwbl3tBYRtBtreHWp19gCxs+Pu2oJtNvo3+2rS4nqxUEZQmuXX4vvGilazv7LMi Cd7upfAZVEMqsy7TAnHD6egDMrULRpUQK2YfITGiOLAwcH5v9fJbWgSivSUE8okSENwA zSzUCeP9nVtwXs+qPaNOqzLcGkG2Z5IVzJAhMcMIJPl+dbwIGy8LbsiMX+2Pjt1vfdAe 7Ewwg3EL70/tJ+PM9wX9MVCOUd6ftuJ677U2OkmFjOaOT0fun9VUAVJbqnkmy1dFzlN0 piXLXQb2XM7IiffOwWxTGUmBI1OgL1mUStFVjkPenyDRbbuVTtBP0dMb5Z2FubXBjL92 cN1w== X-Gm-Message-State: AOAM5336ZCg8mUpgu7lZXxmeYVq115WgPMf73uCIwMRdyXOXJBc46WIz mI7Xd1sH2AXOjwbyZE11fnQ3k9Qqo2JX X-Google-Smtp-Source: ABdhPJzv9YI6PtocdAi7hN0IW4mYt7gmW+cmzFUcODAijax0zAODiJkHDyITC84noPld2tbNypH2tj2bhqUm X-Received: from mizhang-super.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1071]) (user=mizhang job=sendgmr) by 2002:a63:f54b:0:b0:384:64d1:fa45 with SMTP id e11-20020a63f54b000000b0038464d1fa45mr13816423pgk.95.1648795003619; Thu, 31 Mar 2022 23:36:43 -0700 (PDT) Reply-To: Mingwei Zhang Date: Fri, 1 Apr 2022 06:36:31 +0000 In-Reply-To: <20220401063636.2414200-1-mizhang@google.com> Message-Id: <20220401063636.2414200-2-mizhang@google.com> Mime-Version: 1.0 References: <20220401063636.2414200-1-mizhang@google.com> X-Mailer: git-send-email 2.35.1.1094.g7c7d902a7c-goog Subject: [PATCH v3 1/6] KVM: x86/mmu: Set lpage_disallowed in TDP MMU before setting SPTE From: Mingwei Zhang To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Mingwei Zhang , Yosry Ahmed , Ben Gardon , David Matlack , Jing Zhang , Peter Xu Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sean Christopherson Set lpage_disallowed in TDP MMU shadow pages before making the SP visible to other readers, i.e. before setting its SPTE. This will allow KVM to query lpage_disallowed when determining if a shadow page can be replaced by a NX huge page without violating the rules of the mitigation. Reviewed-by: Mingwei Zhang Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 14 ++++++++++---- arch/x86/kvm/mmu/mmu_internal.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 20 ++++++++++++-------- 3 files changed, 23 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 1361eb4599b4..5cb845fae56e 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -812,14 +812,20 @@ static void account_shadowed(struct kvm *kvm, struct = kvm_mmu_page *sp) kvm_mmu_gfn_disallow_lpage(slot, gfn); } =20 -void account_huge_nx_page(struct kvm *kvm, struct kvm_mmu_page *sp) +void __account_huge_nx_page(struct kvm *kvm, struct kvm_mmu_page *sp) { - if (sp->lpage_disallowed) - return; - ++kvm->stat.nx_lpage_splits; list_add_tail(&sp->lpage_disallowed_link, &kvm->arch.lpage_disallowed_mmu_pages); +} + +static void account_huge_nx_page(struct kvm *kvm, struct kvm_mmu_page *sp) +{ + if (sp->lpage_disallowed) + return; + + __account_huge_nx_page(kvm, sp); + sp->lpage_disallowed =3D true; } =20 diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index 1bff453f7cbe..4a0087efa1e3 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -168,7 +168,7 @@ void disallowed_hugepage_adjust(struct kvm_page_fault *= fault, u64 spte, int cur_ =20 void *mmu_memory_cache_alloc(struct kvm_mmu_memory_cache *mc); =20 -void account_huge_nx_page(struct kvm *kvm, struct kvm_mmu_page *sp); +void __account_huge_nx_page(struct kvm *kvm, struct kvm_mmu_page *sp); void unaccount_huge_nx_page(struct kvm *kvm, struct kvm_mmu_page *sp); =20 #endif /* __KVM_X86_MMU_INTERNAL_H */ diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index b3b6426725d4..f05423545e6d 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1122,16 +1122,13 @@ static int tdp_mmu_map_handle_target_level(struct k= vm_vcpu *vcpu, * @kvm: kvm instance * @iter: a tdp_iter instance currently on the SPTE that should be set * @sp: The new TDP page table to install. - * @account_nx: True if this page table is being installed to split a - * non-executable huge page. * @shared: This operation is running under the MMU lock in read mode. * * Returns: 0 if the new page table was installed. Non-0 if the page table * could not be installed (e.g. the atomic compare-exchange faile= d). */ static int tdp_mmu_link_sp(struct kvm *kvm, struct tdp_iter *iter, - struct kvm_mmu_page *sp, bool account_nx, - bool shared) + struct kvm_mmu_page *sp, bool shared) { u64 spte =3D make_nonleaf_spte(sp->spt, !shadow_accessed_mask); int ret =3D 0; @@ -1146,8 +1143,6 @@ static int tdp_mmu_link_sp(struct kvm *kvm, struct td= p_iter *iter, =20 spin_lock(&kvm->arch.tdp_mmu_pages_lock); list_add(&sp->link, &kvm->arch.tdp_mmu_pages); - if (account_nx) - account_huge_nx_page(kvm, sp); spin_unlock(&kvm->arch.tdp_mmu_pages_lock); =20 return 0; @@ -1160,6 +1155,7 @@ static int tdp_mmu_link_sp(struct kvm *kvm, struct td= p_iter *iter, int kvm_tdp_mmu_map(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { struct kvm_mmu *mmu =3D vcpu->arch.mmu; + struct kvm *kvm =3D vcpu->kvm; struct tdp_iter iter; struct kvm_mmu_page *sp; int ret; @@ -1210,10 +1206,18 @@ int kvm_tdp_mmu_map(struct kvm_vcpu *vcpu, struct k= vm_page_fault *fault) sp =3D tdp_mmu_alloc_sp(vcpu); tdp_mmu_init_child_sp(sp, &iter); =20 - if (tdp_mmu_link_sp(vcpu->kvm, &iter, sp, account_nx, true)) { + sp->lpage_disallowed =3D account_nx; + + if (tdp_mmu_link_sp(kvm, &iter, sp, true)) { tdp_mmu_free_sp(sp); break; } + + if (account_nx) { + spin_lock(&kvm->arch.tdp_mmu_pages_lock); + __account_huge_nx_page(kvm, sp); + spin_unlock(&kvm->arch.tdp_mmu_pages_lock); + } } } =20 @@ -1501,7 +1505,7 @@ static int tdp_mmu_split_huge_page(struct kvm *kvm, s= truct tdp_iter *iter, * correctness standpoint since the translation will be the same either * way. */ - ret =3D tdp_mmu_link_sp(kvm, iter, sp, false, shared); + ret =3D tdp_mmu_link_sp(kvm, iter, sp, shared); if (ret) goto out; =20 --=20 2.35.1.1094.g7c7d902a7c-goog From nobody Fri Jun 19 16:54:19 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CDFD9C433FE for ; Fri, 1 Apr 2022 06:37:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245543AbiDAGit (ORCPT ); Fri, 1 Apr 2022 02:38:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245455AbiDAGie (ORCPT ); Fri, 1 Apr 2022 02:38:34 -0400 Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com [IPv6:2607:f8b0:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DBE6118D9AE for ; Thu, 31 Mar 2022 23:36:45 -0700 (PDT) Received: by mail-pf1-x44a.google.com with SMTP id b19-20020a621b13000000b004fa68b3677bso1079012pfb.20 for ; Thu, 31 Mar 2022 23:36:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=79d9al9/Ox8ojgYPVafCO8rjr+Q8aeYemdUQb3D6s/c=; b=N3b6Yhe/1EBQnTkfpidMiqiMv9Vb0c2ONl+P0zVk+ldqlgRmqSNSMQzz6W5kdeqGiG 5N2WnohG0La3eMJjqlEt/3fAoG0FKkeCcNZbs53PdEx+CCYJRIq4GDWaNN4awB4rzbS8 beRw0rfxKpYYKnLouF0wkgn1rl0G6pTUde4p0fRb+Uz4rxrjqGOn4WdlcfElTdecYPvN MHTF3uzCRELJYPqahjhEgG+R2bBQpgMsZXZzdiauYrMLnVWuCIS0EDl1VSlcTykTG9iu 1DuIBZKF/lDYPzhGhzXvyQKokiMKcVHqjG0BUPKFSBMhqXKkVHzcP4YkMG8UTy/6pxky yrPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=79d9al9/Ox8ojgYPVafCO8rjr+Q8aeYemdUQb3D6s/c=; b=RV+J4cnxVswp+KVCIl1pFHgYlWi60h1X1iMV0r3W1XH/141/IsD95EiX2iB3LMrNkV kDt1oycnB1v+AhTblizP5Qpn/m+dC5wYWdt4DT/D0CqNBk34Yp24QiXYJuqqmrDs3k3Q ZhT4t/JX7g1s6UgsxTnT6eaDf0/wiITG13kzZmLWFh/k3xcbBzAxzSQIWTKCAq7wpJwo 9Gjysiunj41LWjXGRIm+70EYobPiLOY/Jcm9ta4xZyPFQ4H3USzEVRBBchCXS2W31PRR aEz8YxEpsYmWD1pLHupDbysqeSVSx9h4ZA0fFir9d600oMJJsrphX4JNYzn0pBDtckTw 0lqw== X-Gm-Message-State: AOAM530vjuD4FV6jPb36fojZop/c7VeEQQXvM4752Tc54rTAbj9ZLb2q K+mg8vfQoPQciVeoNF4qGb2fxhD2OGRd X-Google-Smtp-Source: ABdhPJy3jxKjcEEonO+XJQJHQsbp/0IBv5UTK/0rCshy1zEI4VBGvkOvFuAhzl/cNBA6e0+bHVc8Bje4cTQi X-Received: from mizhang-super.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1071]) (user=mizhang job=sendgmr) by 2002:a63:1d62:0:b0:382:1421:c7 with SMTP id d34-20020a631d62000000b00382142100c7mr13700229pgm.416.1648795005383; Thu, 31 Mar 2022 23:36:45 -0700 (PDT) Reply-To: Mingwei Zhang Date: Fri, 1 Apr 2022 06:36:32 +0000 In-Reply-To: <20220401063636.2414200-1-mizhang@google.com> Message-Id: <20220401063636.2414200-3-mizhang@google.com> Mime-Version: 1.0 References: <20220401063636.2414200-1-mizhang@google.com> X-Mailer: git-send-email 2.35.1.1094.g7c7d902a7c-goog Subject: [PATCH v3 2/6] KVM: x86/mmu: Track the number of TDP MMU pages, but not the actual pages From: Mingwei Zhang To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Mingwei Zhang , Yosry Ahmed , Ben Gardon , David Matlack , Jing Zhang , Peter Xu Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sean Christopherson Track the number of TDP MMU "shadow" pages instead of tracking the pages themselves. With the NX huge page list manipulation moved out of the common linking flow, elminating the list-based tracking means the happy path of adding a shadow page doesn't need to acquire a spinlock and can instead inc/dec an atomic. Keep the tracking as the WARN during TDP MMU teardown on leaked shadow pages is very, very useful for detecting KVM bugs. Tracking the number of pages will also make it trivial to expose the counter to userspace as a stat in the future, which may or may not be desirable. Note, the TDP MMU needs to use a separate counter (and stat if that ever comes to be) from the existing n_used_mmu_pages. The TDP MMU doesn't bother supporting the shrinker nor does it honor KVM_SET_NR_MMU_PAGES (because the TDP MMU consumes so few pages relative to shadow paging), and including TDP MMU pages in that counter would break both the shrinker and shadow MMUs, e.g. if a VM is using nested TDP. Cc: Yosry Ahmed Reviewed-by: Mingwei Zhang Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 11 +++-------- arch/x86/kvm/mmu/tdp_mmu.c | 16 ++++++++-------- 2 files changed, 11 insertions(+), 16 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 9694dd5e6ccc..d0dd5ed2e209 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1192,6 +1192,9 @@ struct kvm_arch { */ bool tdp_mmu_enabled; =20 + /* The number of TDP MMU pages across all roots. */ + atomic64_t tdp_mmu_pages; + /* * List of struct kvm_mmu_pages being used as roots. * All struct kvm_mmu_pages in the list should have @@ -1212,18 +1215,10 @@ struct kvm_arch { */ struct list_head tdp_mmu_roots; =20 - /* - * List of struct kvmp_mmu_pages not being used as roots. - * All struct kvm_mmu_pages in the list should have - * tdp_mmu_page set and a tdp_mmu_root_count of 0. - */ - struct list_head tdp_mmu_pages; - /* * Protects accesses to the following fields when the MMU lock * is held in read mode: * - tdp_mmu_roots (above) - * - tdp_mmu_pages (above) * - the link field of struct kvm_mmu_pages used by the TDP MMU * - lpage_disallowed_mmu_pages * - the lpage_disallowed_link field of struct kvm_mmu_pages used diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index f05423545e6d..5ca78a89d8ed 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -24,7 +24,6 @@ bool kvm_mmu_init_tdp_mmu(struct kvm *kvm) =20 INIT_LIST_HEAD(&kvm->arch.tdp_mmu_roots); spin_lock_init(&kvm->arch.tdp_mmu_pages_lock); - INIT_LIST_HEAD(&kvm->arch.tdp_mmu_pages); kvm->arch.tdp_mmu_zap_wq =3D alloc_workqueue("kvm", WQ_UNBOUND|WQ_MEM_RECLAIM|WQ_CPU_INTENSIVE, 0); =20 @@ -51,7 +50,7 @@ void kvm_mmu_uninit_tdp_mmu(struct kvm *kvm) flush_workqueue(kvm->arch.tdp_mmu_zap_wq); destroy_workqueue(kvm->arch.tdp_mmu_zap_wq); =20 - WARN_ON(!list_empty(&kvm->arch.tdp_mmu_pages)); + WARN_ON(atomic64_read(&kvm->arch.tdp_mmu_pages)); WARN_ON(!list_empty(&kvm->arch.tdp_mmu_roots)); =20 /* @@ -381,14 +380,17 @@ static void handle_changed_spte_dirty_log(struct kvm = *kvm, int as_id, gfn_t gfn, static void tdp_mmu_unlink_sp(struct kvm *kvm, struct kvm_mmu_page *sp, bool shared) { + atomic64_dec(&kvm->arch.tdp_mmu_pages); + + if (!sp->lpage_disallowed) + return; + if (shared) spin_lock(&kvm->arch.tdp_mmu_pages_lock); else lockdep_assert_held_write(&kvm->mmu_lock); =20 - list_del(&sp->link); - if (sp->lpage_disallowed) - unaccount_huge_nx_page(kvm, sp); + unaccount_huge_nx_page(kvm, sp); =20 if (shared) spin_unlock(&kvm->arch.tdp_mmu_pages_lock); @@ -1141,9 +1143,7 @@ static int tdp_mmu_link_sp(struct kvm *kvm, struct td= p_iter *iter, tdp_mmu_set_spte(kvm, iter, spte); } =20 - spin_lock(&kvm->arch.tdp_mmu_pages_lock); - list_add(&sp->link, &kvm->arch.tdp_mmu_pages); - spin_unlock(&kvm->arch.tdp_mmu_pages_lock); + atomic64_inc(&kvm->arch.tdp_mmu_pages); =20 return 0; } --=20 2.35.1.1094.g7c7d902a7c-goog From nobody Fri Jun 19 16:54:19 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D622CC433FE for ; Fri, 1 Apr 2022 06:37:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245501AbiDAGi5 (ORCPT ); Fri, 1 Apr 2022 02:38:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44618 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245460AbiDAGig (ORCPT ); Fri, 1 Apr 2022 02:38:36 -0400 Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com [IPv6:2607:f8b0:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85B5718F202 for ; Thu, 31 Mar 2022 23:36:47 -0700 (PDT) Received: by mail-pf1-x44a.google.com with SMTP id y26-20020aa793da000000b004fb7c6f5d10so1122853pff.1 for ; Thu, 31 Mar 2022 23:36:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=jEiThSY5io7iqeivSdVrWS3wQB6IdCOB+6U4bME4fgY=; b=sps9qGKyNmf+5b3n2bkwDIVDtx5Yq0BVKRxCnUepqnxV9GAILe7KG+LbBRCJa2lkL6 9w3aOe7cIwtMG+dVHztyRSADk6zzm+RzxFs2RqBUjRZFhzBcDPP+bMyJO5n+y5qQrGjG QleH2pwjAoMZvRLAqoN3H2Lqqi8a1K3rFRYd4lJxspD5FIXKG6bI/t/K6jfKLoTJlvGq 2lEMI4HhR1zQdkFyWlulmdZ1HO2BFJAcAbDfe9C2+ey8lgx+IID3v/ZTBJFFcfQSuo+Z u6pHXykmiIgt0F+X2Zfbs9HgoOsZmzPqGI+C0URmCxzcJ0IEoxuoykh5zxJgWo8xIr9e 1kfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=jEiThSY5io7iqeivSdVrWS3wQB6IdCOB+6U4bME4fgY=; b=o3k/TVAsdyW7545P0lSZSua+/IZNpWeDzZnm1s/uaXPbf48oS9ZiD+8nu/nCis6MWX 6DdQxK+fsWAAeB/KfaQYpq8GzpOp20KBx4Yq6f1AaYvNrLFgtRgo5GuZrI+J4yDancwM dTJdHC1Q5LHV+GLhh2c44RUoiQItdL4SrdhsEvyPsXRVmH4PqCqBmqrNWpeyDQEWoVXy hOl3lQ3+6/pUJ+74gutz3anMYscOtWq2Z5SBHfZfdpnQhNm+AJxJTZ7r2B8mY2hd5GvZ chy4gr8/rQlt+tda3uOjAyl3rlOMVCURmDebnTiwkid915H8ZnE4Yxbh0ucgojfsgJfh hogw== X-Gm-Message-State: AOAM532hP4zFk0FHmtJshSxrXEVnlmtBu5gKw99KTlVGvmyFqAoocZ0u NM2GFa3tGOEREMcMAN2b7KOfMugu2U/g X-Google-Smtp-Source: ABdhPJwo9sODgkPXK77NyeA2JigKrPu7Ax7hOoyKYNGMGTjTyYZjOrtTdGXzsPU1XHtnHlWu9KuX2om/jf0v X-Received: from mizhang-super.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1071]) (user=mizhang job=sendgmr) by 2002:a05:6a00:2908:b0:4fa:9297:f631 with SMTP id cg8-20020a056a00290800b004fa9297f631mr42950687pfb.3.1648795007009; Thu, 31 Mar 2022 23:36:47 -0700 (PDT) Reply-To: Mingwei Zhang Date: Fri, 1 Apr 2022 06:36:33 +0000 In-Reply-To: <20220401063636.2414200-1-mizhang@google.com> Message-Id: <20220401063636.2414200-4-mizhang@google.com> Mime-Version: 1.0 References: <20220401063636.2414200-1-mizhang@google.com> X-Mailer: git-send-email 2.35.1.1094.g7c7d902a7c-goog Subject: [PATCH v3 3/6] KVM: x86/mmu: explicitly check nx_hugepage in disallowed_hugepage_adjust() From: Mingwei Zhang To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Mingwei Zhang , Yosry Ahmed , Ben Gardon , David Matlack , Jing Zhang , Peter Xu Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Explicitly check if a NX huge page is disallowed when determining if a page fault needs to be forced to use a smaller sized page. KVM incorrectly assumes that the NX huge page mitigation is the only scenario where KVM will create a shadow page instead of a huge page. Any scenario that causes KVM to zap leaf SPTEs may result in having a SP that can be made huge without violating the NX huge page mitigation. E.g. disabling of dirty logging, zapping from mmu_notifier due to page migration, guest MTRR changes that affect the viability of a huge page, etc... Fixes: b8e8c8303ff2 ("kvm: mmu: ITLB_MULTIHIT mitigation") Reviewed-by: Ben Gardon Signed-off-by: Mingwei Zhang Reviewed-by (and Acked-by and Tested-by) tags should be dropped when there = is a --- arch/x86/kvm/mmu/mmu.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5cb845fae56e..033609e8b332 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2896,6 +2896,16 @@ void disallowed_hugepage_adjust(struct kvm_page_faul= t *fault, u64 spte, int cur_ cur_level =3D=3D fault->goal_level && is_shadow_present_pte(spte) && !is_large_pte(spte)) { + struct kvm_mmu_page *sp; + u64 page_mask; + + sp =3D to_shadow_page(spte & PT64_BASE_ADDR_MASK); + + /* Prevent lpage_disallowed read from moving ahead. */ + smp_rmb(); + + if (!sp->lpage_disallowed) + return; /* * A small SPTE exists for this pfn, but FNAME(fetch) * and __direct_map would like to create a large PTE @@ -2903,8 +2913,8 @@ void disallowed_hugepage_adjust(struct kvm_page_fault= *fault, u64 spte, int cur_ * patching back for them into pfn the next 9 bits of * the address. */ - u64 page_mask =3D KVM_PAGES_PER_HPAGE(cur_level) - - KVM_PAGES_PER_HPAGE(cur_level - 1); + page_mask =3D KVM_PAGES_PER_HPAGE(cur_level) - + KVM_PAGES_PER_HPAGE(cur_level - 1); fault->pfn |=3D fault->gfn & page_mask; fault->goal_level--; } --=20 2.35.1.1094.g7c7d902a7c-goog From nobody Fri Jun 19 16:54:19 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C086AC433F5 for ; Fri, 1 Apr 2022 06:37:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245553AbiDAGjE (ORCPT ); Fri, 1 Apr 2022 02:39:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45342 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245480AbiDAGio (ORCPT ); Fri, 1 Apr 2022 02:38:44 -0400 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [IPv6:2607:f8b0:4864:20::1049]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 309BF18FAF4 for ; Thu, 31 Mar 2022 23:36:49 -0700 (PDT) Received: by mail-pj1-x1049.google.com with SMTP id e15-20020a17090ab38f00b001c9989ae56cso1009613pjr.9 for ; Thu, 31 Mar 2022 23:36:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=zA1hhg2v3lONrCX4JFcIbrA6dugCDeMMabOIS+UXvt0=; b=RXpbAJNaZOQPRyOFrsI7PdtluW2DTEvL1zNv/NNNaBPnnoSu4LJpKois3txuZ9gyUQ FOfa91I8HAgcDZxNrKLdIBXiqFsC7opatEQGzDH5sjtixKXs9yaOG5M3ArS6BxZ8UfFC PPVRKKqwQksQp8X86ES2PNinMfWLe4uI4x6UPLIBEq4LgG0ZbhY+w5WQ80xUl+aNoIxC nWN+GaAcv0OQWIbex/CzEy3Voq1cxvcjGgPbhnpoZF7oIni4G6ULtW48Zz0aXLvTuY9A I9zab5jIkgJGZV7VRtjVwFEtApKEvmvDoUYmsQOwz0yMLILx4rQz5C8H5iy6UxElbh7N Cp7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=zA1hhg2v3lONrCX4JFcIbrA6dugCDeMMabOIS+UXvt0=; b=gP6q18XiZQIs8SKV46JEAypLQhjU5qJThBDnIdLHG0/iT6ckpAZFmhfpkBodfjjN/I cYMUjl9gJJZUbKGwGIf/GD5iHv7Z8HMvWPozhfE8oMzkPj42lkc3cZ7FAIgUhKKt4xuT 3NKk9qy5BIeRPVO7OG8C9QzisTp4vAmF7g8jlzlfGT+fA0h6SqYRVkolY/9KUPvl4KO8 a55YK3rSFbAbHWbh6oKcIEzicZpM0/BWASBujk+6psvurgZt7N+RKPxg4+SAoRkQuuNB R0UrAsZAx9ZyhEsu6flO0zyRHo3LZUYWAE4bTPuG0QFPXzDBm4lFI1TwCPePb/gcrzPh xHgg== X-Gm-Message-State: AOAM533j5BDqGGSZc4TXcvO9ivoCRuNCcEwr67cX92WbXO63C6QjZDkL sTS2kt02s95wOywjzqdANKVcm3K5guKZ X-Google-Smtp-Source: ABdhPJxSwDEJqS5VukOG+zSAw6wXw4vSsoAWgyTHY2cPhIoFUZZiodY5hfmMCC80lOVeAqOeTzv1xYwsWgzc X-Received: from mizhang-super.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1071]) (user=mizhang job=sendgmr) by 2002:a17:902:6b42:b0:156:e1f:ba75 with SMTP id g2-20020a1709026b4200b001560e1fba75mr8998795plt.55.1648795008695; Thu, 31 Mar 2022 23:36:48 -0700 (PDT) Reply-To: Mingwei Zhang Date: Fri, 1 Apr 2022 06:36:34 +0000 In-Reply-To: <20220401063636.2414200-1-mizhang@google.com> Message-Id: <20220401063636.2414200-5-mizhang@google.com> Mime-Version: 1.0 References: <20220401063636.2414200-1-mizhang@google.com> X-Mailer: git-send-email 2.35.1.1094.g7c7d902a7c-goog Subject: [PATCH v3 4/6] KVM: selftests: Dump VM stats in binary stats test From: Mingwei Zhang To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Mingwei Zhang , Yosry Ahmed , Ben Gardon , David Matlack , Jing Zhang , Peter Xu Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ben Gardon Add kvm_util library functions to read KVM stats through the binary stats interface and then dump them to stdout when running the binary stats test. Subsequent commits will extend the kvm_util code and use it to make assertions in a test for NX hugepages. CC: Jing Zhang Signed-off-by: Ben Gardon --- .../selftests/kvm/include/kvm_util_base.h | 1 + .../selftests/kvm/kvm_binary_stats_test.c | 3 + tools/testing/selftests/kvm/lib/kvm_util.c | 143 ++++++++++++++++++ 3 files changed, 147 insertions(+) diff --git a/tools/testing/selftests/kvm/include/kvm_util_base.h b/tools/te= sting/selftests/kvm/include/kvm_util_base.h index 92cef0ffb19e..c5f4a67772cb 100644 --- a/tools/testing/selftests/kvm/include/kvm_util_base.h +++ b/tools/testing/selftests/kvm/include/kvm_util_base.h @@ -400,6 +400,7 @@ void assert_on_unhandled_exception(struct kvm_vm *vm, u= int32_t vcpuid); =20 int vm_get_stats_fd(struct kvm_vm *vm); int vcpu_get_stats_fd(struct kvm_vm *vm, uint32_t vcpuid); +void dump_vm_stats(struct kvm_vm *vm); =20 uint32_t guest_get_vcpuid(void); =20 diff --git a/tools/testing/selftests/kvm/kvm_binary_stats_test.c b/tools/te= sting/selftests/kvm/kvm_binary_stats_test.c index 17f65d514915..afc4701ce8dd 100644 --- a/tools/testing/selftests/kvm/kvm_binary_stats_test.c +++ b/tools/testing/selftests/kvm/kvm_binary_stats_test.c @@ -174,6 +174,9 @@ static void vm_stats_test(struct kvm_vm *vm) stats_test(stats_fd); close(stats_fd); TEST_ASSERT(fcntl(stats_fd, F_GETFD) =3D=3D -1, "Stats fd not freed"); + + /* Dump VM stats */ + dump_vm_stats(vm); } =20 static void vcpu_stats_test(struct kvm_vm *vm, int vcpu_id) diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/sel= ftests/kvm/lib/kvm_util.c index 1665a220abcb..4d21c3b46780 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -2556,3 +2556,146 @@ int vcpu_get_stats_fd(struct kvm_vm *vm, uint32_t v= cpuid) =20 return ioctl(vcpu->fd, KVM_GET_STATS_FD, NULL); } + +/* Caller is responsible for freeing the returned kvm_stats_header. */ +static struct kvm_stats_header *read_vm_stats_header(int stats_fd) +{ + struct kvm_stats_header *header; + ssize_t ret; + + /* Read kvm stats header */ + header =3D malloc(sizeof(*header)); + TEST_ASSERT(header, "Allocate memory for stats header"); + + ret =3D read(stats_fd, header, sizeof(*header)); + TEST_ASSERT(ret =3D=3D sizeof(*header), "Read stats header"); + + return header; +} + +static void dump_header(int stats_fd, struct kvm_stats_header *header) +{ + ssize_t ret; + char *id; + + printf("flags: %u\n", header->flags); + printf("name size: %u\n", header->name_size); + printf("num_desc: %u\n", header->num_desc); + printf("id_offset: %u\n", header->id_offset); + printf("desc_offset: %u\n", header->desc_offset); + printf("data_offset: %u\n", header->data_offset); + + /* Read kvm stats id string */ + id =3D malloc(header->name_size); + TEST_ASSERT(id, "Allocate memory for id string"); + ret =3D pread(stats_fd, id, header->name_size, header->id_offset); + TEST_ASSERT(ret =3D=3D header->name_size, "Read id string"); + + printf("id: %s\n", id); + + free(id); +} + +static ssize_t stats_desc_size(struct kvm_stats_header *header) +{ + return sizeof(struct kvm_stats_desc) + header->name_size; +} + +/* Caller is responsible for freeing the returned kvm_stats_desc. */ +static struct kvm_stats_desc *read_vm_stats_desc(int stats_fd, + struct kvm_stats_header *header) +{ + struct kvm_stats_desc *stats_desc; + size_t size_desc; + ssize_t ret; + + size_desc =3D header->num_desc * stats_desc_size(header); + + /* Allocate memory for stats descriptors */ + stats_desc =3D malloc(size_desc); + TEST_ASSERT(stats_desc, "Allocate memory for stats descriptors"); + + /* Read kvm stats descriptors */ + ret =3D pread(stats_fd, stats_desc, size_desc, header->desc_offset); + TEST_ASSERT(ret =3D=3D size_desc, "Read KVM stats descriptors"); + + return stats_desc; +} + +/* Caller is responsible for freeing the memory *data. */ +static int read_stat_data(int stats_fd, struct kvm_stats_header *header, + struct kvm_stats_desc *desc, uint64_t **data) +{ + u64 *stats_data; + ssize_t ret; + + stats_data =3D malloc(desc->size * sizeof(*stats_data)); + + ret =3D pread(stats_fd, stats_data, desc->size * sizeof(*stats_data), + header->data_offset + desc->offset); + + /* ret is in bytes. */ + ret =3D ret / sizeof(*stats_data); + + TEST_ASSERT(ret =3D=3D desc->size, + "Read data of KVM stats: %s", desc->name); + + *data =3D stats_data; + + return ret; +} + +static void dump_stat(int stats_fd, struct kvm_stats_header *header, + struct kvm_stats_desc *desc) +{ + u64 *stats_data; + ssize_t ret; + int i; + + printf("\tflags: %u\n", desc->flags); + printf("\texponent: %u\n", desc->exponent); + printf("\tsize: %u\n", desc->size); + printf("\toffset: %u\n", desc->offset); + printf("\tbucket_size: %u\n", desc->bucket_size); + printf("\tname: %s\n", (char *)&desc->name); + + ret =3D read_stat_data(stats_fd, header, desc, &stats_data); + + printf("\tdata: %lu", *stats_data); + for (i =3D 1; i < ret; i++) + printf(", %lu", *(stats_data + i)); + printf("\n\n"); + + free(stats_data); +} + +void dump_vm_stats(struct kvm_vm *vm) +{ + struct kvm_stats_desc *stats_desc; + struct kvm_stats_header *header; + struct kvm_stats_desc *desc; + size_t size_desc; + int stats_fd; + int i; + + stats_fd =3D vm_get_stats_fd(vm); + + header =3D read_vm_stats_header(stats_fd); + dump_header(stats_fd, header); + + stats_desc =3D read_vm_stats_desc(stats_fd, header); + + size_desc =3D stats_desc_size(header); + + /* Read kvm stats data one by one */ + for (i =3D 0; i < header->num_desc; ++i) { + desc =3D (void *)stats_desc + (i * size_desc); + dump_stat(stats_fd, header, desc); + } + + free(stats_desc); + free(header); + + close(stats_fd); +} + --=20 2.35.1.1094.g7c7d902a7c-goog From nobody Fri Jun 19 16:54:19 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17E07C433F5 for ; Fri, 1 Apr 2022 06:37:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245602AbiDAGjV (ORCPT ); Fri, 1 Apr 2022 02:39:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45348 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245495AbiDAGip (ORCPT ); Fri, 1 Apr 2022 02:38:45 -0400 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 379A81905A8 for ; Thu, 31 Mar 2022 23:36:51 -0700 (PDT) Received: by mail-pj1-x104a.google.com with SMTP id m24-20020a17090b069800b001c7de0243a2so813305pjz.9 for ; Thu, 31 Mar 2022 23:36:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=4FXxXLJZLQEs++ONkysSkU83dsOxk/dTRI/0WYDJ8+o=; b=RA/ibjQbIQjR1lmfHqDORV7no8EtnU3osUeN4xQ1ZUnoqlRqOqgvjK5NGtrBnfvt1p a/eEFVUoOEVlBXA5InpwA1aNCl9qwhm8AIooMqcXR7iottezC6NwjTwpz/JNPwyBpjDz b6TT4zTOG31I9iPQqxfhPINrEk+7WLYwNeM0yYQ5b/1E6pWszelF4Aikq6UxBEWW0Uip F6eZdHAVN8AGyprizZD6uW38b8URzIJFsnW90Cruf7IiTC5lg9Osw3jpnlenLiShRORr dUH/4iKkCROza6IQpjZ+DE5OCFZ9RQ8lk9w6Q8I1mNC0a9W2VZMvasflYjNjpOpSEK8y oaqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=4FXxXLJZLQEs++ONkysSkU83dsOxk/dTRI/0WYDJ8+o=; b=inK6ARsblHx6PIvlgcrGTSjxzZiBLrEuN0okiSM/4d9vhoJsTFpgRN1drqqffHBcil x9XwXwRfiLaWuq+tWxiLe0RN1fWvpOzTPLTZqLoTL0NZLqOR/M/mt3Ni+Egz6RrMjrW5 Zq9bS7dFbIhGpf4387rfBEQTk4XIg2vU82/3kGJg08BVWwbkvaMxhE4udMSLgxGsgDZX +t5pVSHMm7yRbMgGuVAX9mtdXMmCiddhic0S8KgKShQavGQOlaU0Oonyis5r6W8XtjMW gqpIeiFVLa2mzSXOerDGdFs8sb6dqYU2LwCW7VttobL6POoUQldqfQg8QtY9WBcKwr4X WJ0A== X-Gm-Message-State: AOAM5306gAHapfQdiQNZIPEKRmsw9z/rqRFvrsAVssCYUa3rBX7xlx4l E5CzwJ2m2ddx3dkOZKwAe/MxE6mmom/+ X-Google-Smtp-Source: ABdhPJxRZKW1kTO4N7PzjBjiTYtM5m2OPqUDst+Fqm/l7UQ+k1okKkthp4T79hWu30Kzi/4orYUFMKOyYFDl X-Received: from mizhang-super.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1071]) (user=mizhang job=sendgmr) by 2002:a17:90a:858b:b0:1c6:5bc8:781a with SMTP id m11-20020a17090a858b00b001c65bc8781amr332690pjn.0.1648795010348; Thu, 31 Mar 2022 23:36:50 -0700 (PDT) Reply-To: Mingwei Zhang Date: Fri, 1 Apr 2022 06:36:35 +0000 In-Reply-To: <20220401063636.2414200-1-mizhang@google.com> Message-Id: <20220401063636.2414200-6-mizhang@google.com> Mime-Version: 1.0 References: <20220401063636.2414200-1-mizhang@google.com> X-Mailer: git-send-email 2.35.1.1094.g7c7d902a7c-goog Subject: [PATCH v3 5/6] KVM: selftests: Test reading a single stat From: Mingwei Zhang To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Mingwei Zhang , Yosry Ahmed , Ben Gardon , David Matlack , Jing Zhang , Peter Xu Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ben Gardon Retrieve the value of a single stat by name in the binary stats test to ensure the kvm_util library functions work. CC: Jing Zhang Signed-off-by: Ben Gardon --- .../selftests/kvm/include/kvm_util_base.h | 1 + .../selftests/kvm/kvm_binary_stats_test.c | 3 ++ tools/testing/selftests/kvm/lib/kvm_util.c | 53 +++++++++++++++++++ 3 files changed, 57 insertions(+) diff --git a/tools/testing/selftests/kvm/include/kvm_util_base.h b/tools/te= sting/selftests/kvm/include/kvm_util_base.h index c5f4a67772cb..09ee70c0df26 100644 --- a/tools/testing/selftests/kvm/include/kvm_util_base.h +++ b/tools/testing/selftests/kvm/include/kvm_util_base.h @@ -401,6 +401,7 @@ void assert_on_unhandled_exception(struct kvm_vm *vm, u= int32_t vcpuid); int vm_get_stats_fd(struct kvm_vm *vm); int vcpu_get_stats_fd(struct kvm_vm *vm, uint32_t vcpuid); void dump_vm_stats(struct kvm_vm *vm); +uint64_t vm_get_single_stat(struct kvm_vm *vm, const char *stat_name); =20 uint32_t guest_get_vcpuid(void); =20 diff --git a/tools/testing/selftests/kvm/kvm_binary_stats_test.c b/tools/te= sting/selftests/kvm/kvm_binary_stats_test.c index afc4701ce8dd..97bde355f105 100644 --- a/tools/testing/selftests/kvm/kvm_binary_stats_test.c +++ b/tools/testing/selftests/kvm/kvm_binary_stats_test.c @@ -177,6 +177,9 @@ static void vm_stats_test(struct kvm_vm *vm) =20 /* Dump VM stats */ dump_vm_stats(vm); + + /* Read a single stat. */ + printf("remote_tlb_flush: %lu\n", vm_get_single_stat(vm, "remote_tlb_flus= h")); } =20 static void vcpu_stats_test(struct kvm_vm *vm, int vcpu_id) diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/sel= ftests/kvm/lib/kvm_util.c index 4d21c3b46780..1d3493d7fd55 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -2699,3 +2699,56 @@ void dump_vm_stats(struct kvm_vm *vm) close(stats_fd); } =20 +static int vm_get_stat_data(struct kvm_vm *vm, const char *stat_name, + uint64_t **data) +{ + struct kvm_stats_desc *stats_desc; + struct kvm_stats_header *header; + struct kvm_stats_desc *desc; + size_t size_desc; + int stats_fd; + int ret =3D -EINVAL; + int i; + + *data =3D NULL; + + stats_fd =3D vm_get_stats_fd(vm); + + header =3D read_vm_stats_header(stats_fd); + + stats_desc =3D read_vm_stats_desc(stats_fd, header); + + size_desc =3D stats_desc_size(header); + + /* Read kvm stats data one by one */ + for (i =3D 0; i < header->num_desc; ++i) { + desc =3D (void *)stats_desc + (i * size_desc); + + if (strcmp(desc->name, stat_name)) + continue; + + ret =3D read_stat_data(stats_fd, header, desc, data); + } + + free(stats_desc); + free(header); + + close(stats_fd); + + return ret; +} + +uint64_t vm_get_single_stat(struct kvm_vm *vm, const char *stat_name) +{ + uint64_t *data; + uint64_t value; + int ret; + + ret =3D vm_get_stat_data(vm, stat_name, &data); + TEST_ASSERT(ret =3D=3D 1, "Stat %s expected to have 1 element, but has %d= ", + stat_name, ret); + value =3D *data; + free(data); + return value; +} + --=20 2.35.1.1094.g7c7d902a7c-goog From nobody Fri Jun 19 16:54:19 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7621C433EF for ; Fri, 1 Apr 2022 06:37:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243961AbiDAGjQ (ORCPT ); Fri, 1 Apr 2022 02:39:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45310 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245512AbiDAGiq (ORCPT ); Fri, 1 Apr 2022 02:38:46 -0400 Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com [IPv6:2607:f8b0:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AAC1919236D for ; Thu, 31 Mar 2022 23:36:52 -0700 (PDT) Received: by mail-pf1-x44a.google.com with SMTP id 63-20020a621742000000b004fa914b7f34so1138172pfx.0 for ; Thu, 31 Mar 2022 23:36:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=uzcMfPPC4r9DDGZEKc9Z+WT8d8M153ZB7YHzP50uvCQ=; b=sejzPoMfvygP+Yln6s7tbom3x27vqmjv+uyd79uPjN3jxvfF1wmqSlZxhPazjIOTrR V63nVwYUn6riNiFEpx242Cla0yUMB5ZPL3hVfLMlzCtGRGChXeobeN7IWM5exWkIZzLm 7qUSYt+8YqmjecCpxTc5MFGv7uGJhFp4qE4tYaXHhGqVq9gND/05SlJ32xq3jeGdd21h nzs6jh+kRm8cXA9GYM00Z0hVDvMr4UtSfVPYFyWqhfdtH/b1wHlfcLU0X4aXEvpFpraI DmEv5f0Zw9L4DOeapIvTN0h33kQyhVbYUr04noqzZ0mzlgqxRgsAC+vahAdD71/5k0DT vO3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=uzcMfPPC4r9DDGZEKc9Z+WT8d8M153ZB7YHzP50uvCQ=; b=6PKb5ou1ES19cCrLfVMsRg5z+cdsmAhWIm9fjglonLGetSpDxV69f39bPwDbLRuTZD vNn9AlNlJwISRKt/IO8cUnJB0qxZw9Z/9sZyAHjGkk8rkmzVtIMXtTGINbCS/l7tdFas 7/TDEiLlEhhk3W9G1LrLUopXj1ifp1tYnKFavxxEkSOIZEn0Ok5tpi4wGw6f7oG9u520 99dN2Ify43xDPCdpTlCY5j+6RRnSgi7LKp2NyfXeU+rGHYwoE1WiLmn/T1jRdJ4GseoQ KbZlk14nKzT/dbopi5Zggcpq7KURb7gc+YteRauXACSdQLS2a0kVUeco1jxSWP+ZUrgd XZlA== X-Gm-Message-State: AOAM531lzdJA55hPBfmjuT5CvVwhXMPlDvhaSrQPBqfLgjxb7o3ADjNf Q5GRaeJ0MMOT2M2yMbhWXI5lSe5TlIt5 X-Google-Smtp-Source: ABdhPJyAOejzhvbgsx7VN0zVrvPkMlMTUNrgRtrz5gzaixv2702AoUZ4mGJxU7txTGviK1moyXxJiP62V/yX X-Received: from mizhang-super.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1071]) (user=mizhang job=sendgmr) by 2002:a05:6a00:140f:b0:4e0:6995:9c48 with SMTP id l15-20020a056a00140f00b004e069959c48mr9243091pfu.59.1648795012083; Thu, 31 Mar 2022 23:36:52 -0700 (PDT) Reply-To: Mingwei Zhang Date: Fri, 1 Apr 2022 06:36:36 +0000 In-Reply-To: <20220401063636.2414200-1-mizhang@google.com> Message-Id: <20220401063636.2414200-7-mizhang@google.com> Mime-Version: 1.0 References: <20220401063636.2414200-1-mizhang@google.com> X-Mailer: git-send-email 2.35.1.1094.g7c7d902a7c-goog Subject: [PATCH v3 6/6] selftests: KVM: use page stats to check if dirty logging works properly From: Mingwei Zhang To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Mingwei Zhang , Yosry Ahmed , Ben Gardon , David Matlack , Jing Zhang , Peter Xu Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When dirty logging is enabled, KVM will remap all accessed pages in NPT/EPT at 4K. This property could be used to check if the page stats metrics work properly in KVM mmu. At the same time, this logic might be used the other way around: using page stats to verify if dirty logging really splits all huge pages. Moreover, when dirty logging is disabled, KVM zaps corresponding SPTEs and we could check whether the large pages come back when guest touches the pages again. So add page stats checking in dirty logging performance selftest. In particular, add checks in three locations: - just after vm is created; - after populating memory into vm but before enabling dirty logging; - finish dirty logging but before disabling it; - behind the final iteration after disabling dirty logging. Tested using commands: - ./dirty_log_perf_test -s anonymous_hugetlb_1gb - ./dirty_log_perf_test -s anonymous_hugetlb_2mb - ./dirty_log_perf_test -s anonymous_thp Cc: Sean Christopherson Cc: David Matlack Cc: Jing Zhang Cc: Peter Xu Suggested-by: Ben Gardon Reviewed-by: Ben Gardon Signed-off-by: Mingwei Zhang --- .../selftests/kvm/dirty_log_perf_test.c | 53 +++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/tools/testing/selftests/kvm/dirty_log_perf_test.c b/tools/test= ing/selftests/kvm/dirty_log_perf_test.c index c9d9e513ca04..dd48aabfff5c 100644 --- a/tools/testing/selftests/kvm/dirty_log_perf_test.c +++ b/tools/testing/selftests/kvm/dirty_log_perf_test.c @@ -25,6 +25,10 @@ #define GICR_BASE_GPA 0x80A0000ULL #endif =20 +#ifdef __x86_64__ +#include "processor.h" +#endif + /* How many host loops to run by default (one KVM_GET_DIRTY_LOG for each l= oop)*/ #define TEST_HOST_LOOP_N 2UL =20 @@ -191,6 +195,14 @@ static void run_test(enum vm_guest_mode mode, void *ar= g) p->slots, p->backing_src, p->partition_vcpu_memory_access); =20 +#ifdef __x86_64__ + TEST_ASSERT(vm_get_single_stat(vm, "pages_4k") =3D=3D 0, + "4K page is non zero"); + TEST_ASSERT(vm_get_single_stat(vm, "pages_2m") =3D=3D 0, + "2M page is non zero"); + TEST_ASSERT(vm_get_single_stat(vm, "pages_1g") =3D=3D 0, + "1G page is non zero"); +#endif perf_test_set_wr_fract(vm, p->wr_fract); =20 guest_num_pages =3D (nr_vcpus * guest_percpu_mem_size) >> vm_get_page_shi= ft(vm); @@ -232,6 +244,17 @@ static void run_test(enum vm_guest_mode mode, void *ar= g) pr_info("Populate memory time: %ld.%.9lds\n", ts_diff.tv_sec, ts_diff.tv_nsec); =20 +#ifdef __x86_64__ + TEST_ASSERT(vm_get_single_stat(vm, "pages_4k") !=3D 0, + "4K page is zero"); + if (p->backing_src =3D=3D VM_MEM_SRC_ANONYMOUS_THP || + p->backing_src =3D=3D VM_MEM_SRC_ANONYMOUS_HUGETLB_2MB) + TEST_ASSERT(vm_get_single_stat(vm, "pages_2m") !=3D 0, + "2M page is zero"); + if (p->backing_src =3D=3D VM_MEM_SRC_ANONYMOUS_HUGETLB_1GB) + TEST_ASSERT(vm_get_single_stat(vm, "pages_1g") !=3D 0, + "1G page is zero"); +#endif /* Enable dirty logging */ clock_gettime(CLOCK_MONOTONIC, &start); enable_dirty_logging(vm, p->slots); @@ -277,6 +300,14 @@ static void run_test(enum vm_guest_mode mode, void *ar= g) iteration, ts_diff.tv_sec, ts_diff.tv_nsec); } } +#ifdef __x86_64__ + TEST_ASSERT(vm_get_single_stat(vm, "pages_4k") !=3D 0, + "4K page is zero after dirty logging"); + TEST_ASSERT(vm_get_single_stat(vm, "pages_2m") =3D=3D 0, + "2M page is non-zero after dirty logging"); + TEST_ASSERT(vm_get_single_stat(vm, "pages_1g") =3D=3D 0, + "1G page is non-zero after dirty logging"); +#endif =20 /* Disable dirty logging */ clock_gettime(CLOCK_MONOTONIC, &start); @@ -285,6 +316,28 @@ static void run_test(enum vm_guest_mode mode, void *ar= g) pr_info("Disabling dirty logging time: %ld.%.9lds\n", ts_diff.tv_sec, ts_diff.tv_nsec); =20 + /* + * Increment iteration to run the vcpus again to ensure all pages come + * back. + */ + iteration++; + pr_info("Starting the final iteration to get all pages back.\n"); + for (vcpu_id =3D 0; vcpu_id < nr_vcpus; vcpu_id++) { + while (READ_ONCE(vcpu_last_completed_iteration[vcpu_id]) + !=3D iteration) + ; + } + +#ifdef __x86_64__ + if (p->backing_src =3D=3D VM_MEM_SRC_ANONYMOUS_THP || + p->backing_src =3D=3D VM_MEM_SRC_ANONYMOUS_HUGETLB_2MB) + TEST_ASSERT(vm_get_single_stat(vm, "pages_2m") !=3D 0, + "2M page is zero"); + if (p->backing_src =3D=3D VM_MEM_SRC_ANONYMOUS_HUGETLB_1GB) + TEST_ASSERT(vm_get_single_stat(vm, "pages_1g") !=3D 0, + "1G page is zero"); +#endif + /* Tell the vcpu thread to quit */ host_quit =3D true; perf_test_join_vcpu_threads(nr_vcpus); --=20 2.35.1.1094.g7c7d902a7c-goog