From nobody Sun Feb 8 16:16:40 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E72DBE732EB for ; Thu, 28 Sep 2023 16:31:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231524AbjI1QbJ (ORCPT ); Thu, 28 Sep 2023 12:31:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46514 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231696AbjI1QbD (ORCPT ); Thu, 28 Sep 2023 12:31:03 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 134231AD for ; Thu, 28 Sep 2023 09:30:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695918612; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D9/RV6Hb0/JilJv3mz6TN4Le2K6hQ3WCfVt/3YM6beE=; b=C0rs+wK3kJNX0cfHh2/rNwBwj3KADNRmuR+HyLEZld7iZYv7+ablLpXOJruc5HuzymaYAH gTP2t0hIjLHxYKUrsGk0tGf6LCMxolmsvx/yvd93tVdsMMaQie5KLk15afAtRozDmHNWub II2FDRXRbM+/8O3EB4HVePrvqHcL+Vc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-204-jWNZzshsOFiCkpO4G53Aiw-1; Thu, 28 Sep 2023 12:30:02 -0400 X-MC-Unique: jWNZzshsOFiCkpO4G53Aiw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3F17B858293; Thu, 28 Sep 2023 16:30:00 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 26BEE140E969; Thu, 28 Sep 2023 16:30:00 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH 1/3] KVM: x86/mmu: remove unnecessary "bool shared" argument from functions Date: Thu, 28 Sep 2023 12:29:57 -0400 Message-Id: <20230928162959.1514661-2-pbonzini@redhat.com> In-Reply-To: <20230928162959.1514661-1-pbonzini@redhat.com> References: <20230928162959.1514661-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Neither tdp_mmu_next_root nor kvm_tdp_mmu_put_root need to know if the lock is taken for read or write. Either way, protection is achieved via RCU and tdp_mmu_pages_lock. Remove the argument and just assert that the lock is taken. Signed-off-by: Paolo Bonzini Reviewed-by: Maxim Levitsky --- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 34 +++++++++++++++++++++------------- arch/x86/kvm/mmu/tdp_mmu.h | 3 +-- 3 files changed, 23 insertions(+), 16 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index f7901cb4d2fa..64b1bdba943e 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3548,7 +3548,7 @@ static void mmu_free_root_page(struct kvm *kvm, hpa_t= *root_hpa, return; =20 if (is_tdp_mmu_page(sp)) - kvm_tdp_mmu_put_root(kvm, sp, false); + kvm_tdp_mmu_put_root(kvm, sp); else if (!--sp->root_count && sp->role.invalid) kvm_mmu_prepare_zap_page(kvm, sp, invalid_list); =20 diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 6cd4dd631a2f..ab0876015be7 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -73,10 +73,13 @@ static void tdp_mmu_free_sp_rcu_callback(struct rcu_hea= d *head) tdp_mmu_free_sp(sp); } =20 -void kvm_tdp_mmu_put_root(struct kvm *kvm, struct kvm_mmu_page *root, - bool shared) +void kvm_tdp_mmu_put_root(struct kvm *kvm, struct kvm_mmu_page *root) { - kvm_lockdep_assert_mmu_lock_held(kvm, shared); + /* + * Either read or write is okay, but the lock is needed because + * writers might not take tdp_mmu_pages_lock. + */ + lockdep_assert_held(&kvm->mmu_lock); =20 if (!refcount_dec_and_test(&root->tdp_mmu_root_count)) return; @@ -106,10 +109,16 @@ void kvm_tdp_mmu_put_root(struct kvm *kvm, struct kvm= _mmu_page *root, */ static struct kvm_mmu_page *tdp_mmu_next_root(struct kvm *kvm, struct kvm_mmu_page *prev_root, - bool shared, bool only_valid) + bool only_valid) { struct kvm_mmu_page *next_root; =20 + /* + * While the roots themselves are RCU-protected, fields such as + * role.invalid are protected by mmu_lock. + */ + lockdep_assert_held(&kvm->mmu_lock); + rcu_read_lock(); =20 if (prev_root) @@ -132,7 +141,7 @@ static struct kvm_mmu_page *tdp_mmu_next_root(struct kv= m *kvm, rcu_read_unlock(); =20 if (prev_root) - kvm_tdp_mmu_put_root(kvm, prev_root, shared); + kvm_tdp_mmu_put_root(kvm, prev_root); =20 return next_root; } @@ -144,13 +153,12 @@ static struct kvm_mmu_page *tdp_mmu_next_root(struct = kvm *kvm, * recent root. (Unless keeping a live reference is desirable.) * * If shared is set, this function is operating under the MMU lock in read - * mode. In the unlikely event that this thread must free a root, the lock - * will be temporarily dropped and reacquired in write mode. + * mode. */ #define __for_each_tdp_mmu_root_yield_safe(_kvm, _root, _as_id, _shared, _= only_valid)\ - for (_root =3D tdp_mmu_next_root(_kvm, NULL, _shared, _only_valid); \ - _root; \ - _root =3D tdp_mmu_next_root(_kvm, _root, _shared, _only_valid)) \ + for (_root =3D tdp_mmu_next_root(_kvm, NULL, _only_valid); \ + _root; \ + _root =3D tdp_mmu_next_root(_kvm, _root, _only_valid)) \ if (kvm_lockdep_assert_mmu_lock_held(_kvm, _shared) && \ kvm_mmu_page_as_id(_root) !=3D _as_id) { \ } else @@ -159,9 +167,9 @@ static struct kvm_mmu_page *tdp_mmu_next_root(struct kv= m *kvm, __for_each_tdp_mmu_root_yield_safe(_kvm, _root, _as_id, _shared, true) =20 #define for_each_tdp_mmu_root_yield_safe(_kvm, _root, _shared) \ - for (_root =3D tdp_mmu_next_root(_kvm, NULL, _shared, false); \ - _root; \ - _root =3D tdp_mmu_next_root(_kvm, _root, _shared, false)) \ + for (_root =3D tdp_mmu_next_root(_kvm, NULL, false); \ + _root; \ + _root =3D tdp_mmu_next_root(_kvm, _root, false)) if (!kvm_lockdep_assert_mmu_lock_held(_kvm, _shared)) { \ } else =20 diff --git a/arch/x86/kvm/mmu/tdp_mmu.h b/arch/x86/kvm/mmu/tdp_mmu.h index 733a3aef3a96..20d97aa46c49 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.h +++ b/arch/x86/kvm/mmu/tdp_mmu.h @@ -17,8 +17,7 @@ __must_check static inline bool kvm_tdp_mmu_get_root(stru= ct kvm_mmu_page *root) return refcount_inc_not_zero(&root->tdp_mmu_root_count); } =20 -void kvm_tdp_mmu_put_root(struct kvm *kvm, struct kvm_mmu_page *root, - bool shared); +void kvm_tdp_mmu_put_root(struct kvm *kvm, struct kvm_mmu_page *root); =20 bool kvm_tdp_mmu_zap_leafs(struct kvm *kvm, gfn_t start, gfn_t end, bool f= lush); bool kvm_tdp_mmu_zap_sp(struct kvm *kvm, struct kvm_mmu_page *sp); --=20 2.39.1 From nobody Sun Feb 8 16:16:40 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEE0AE732EB for ; Thu, 28 Sep 2023 16:31:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231811AbjI1QbG (ORCPT ); Thu, 28 Sep 2023 12:31:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230212AbjI1Qaz (ORCPT ); Thu, 28 Sep 2023 12:30:55 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1AF7DBF for ; Thu, 28 Sep 2023 09:30:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695918604; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UdQ8vFIkrjKHPjmY3S0MSAG68uzsObzQSGb5suxbmVU=; b=MEebHRP8dn6t2WqcPNXax43vxe0BoR78W2KS/jhrCual1N7s6HLKpORY7aL4FGytvp6tVs 2U3simylN27g86QkgO2mVmqWJjg2DrGVQQ7lYVIpqSW57i05cG04uCRW/oQFOaudGtb/fA oYrdCzIBxgtOtokdQrE2xLfU17tcjnY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-223-rVPCVgeKMlSnVbd3S2fAzQ-1; Thu, 28 Sep 2023 12:30:00 -0400 X-MC-Unique: rVPCVgeKMlSnVbd3S2fAzQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5F475185A78E; Thu, 28 Sep 2023 16:30:00 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 466F9140E969; Thu, 28 Sep 2023 16:30:00 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH 2/3] KVM: x86/mmu: remove unnecessary "bool shared" argument from iterators Date: Thu, 28 Sep 2023 12:29:58 -0400 Message-Id: <20230928162959.1514661-3-pbonzini@redhat.com> In-Reply-To: <20230928162959.1514661-1-pbonzini@redhat.com> References: <20230928162959.1514661-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The "bool shared" argument is more or less unnecessary in the for_each_*_tdp_mmu_root_yield_safe() macros. Many users check for the lock before calling it; all of them either call small functions that do the check, or end up calling tdp_mmu_set_spte_atomic() and tdp_mmu_iter_set_spte(). Add a few assertions to make up for the lost check in for_each_*_tdp_mmu_root_yield_safe(), but even this is probably overkill and mostly for documentation reasons. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/tdp_mmu.c | 42 +++++++++++++++++++------------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index ab0876015be7..b9abfa78808a 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -155,23 +155,20 @@ static struct kvm_mmu_page *tdp_mmu_next_root(struct = kvm *kvm, * If shared is set, this function is operating under the MMU lock in read * mode. */ -#define __for_each_tdp_mmu_root_yield_safe(_kvm, _root, _as_id, _shared, _= only_valid)\ +#define __for_each_tdp_mmu_root_yield_safe(_kvm, _root, _as_id, _only_vali= d)\ for (_root =3D tdp_mmu_next_root(_kvm, NULL, _only_valid); \ _root; \ _root =3D tdp_mmu_next_root(_kvm, _root, _only_valid)) \ - if (kvm_lockdep_assert_mmu_lock_held(_kvm, _shared) && \ - kvm_mmu_page_as_id(_root) !=3D _as_id) { \ + if (kvm_mmu_page_as_id(_root) !=3D _as_id) { \ } else =20 -#define for_each_valid_tdp_mmu_root_yield_safe(_kvm, _root, _as_id, _share= d) \ - __for_each_tdp_mmu_root_yield_safe(_kvm, _root, _as_id, _shared, true) +#define for_each_valid_tdp_mmu_root_yield_safe(_kvm, _root, _as_id) \ + __for_each_tdp_mmu_root_yield_safe(_kvm, _root, _as_id, true) =20 -#define for_each_tdp_mmu_root_yield_safe(_kvm, _root, _shared) \ +#define for_each_tdp_mmu_root_yield_safe(_kvm, _root) \ for (_root =3D tdp_mmu_next_root(_kvm, NULL, false); \ _root; \ _root =3D tdp_mmu_next_root(_kvm, _root, false)) - if (!kvm_lockdep_assert_mmu_lock_held(_kvm, _shared)) { \ - } else =20 /* * Iterate over all TDP MMU roots. Requires that mmu_lock be held for wri= te, @@ -840,7 +837,8 @@ bool kvm_tdp_mmu_zap_leafs(struct kvm *kvm, gfn_t start= , gfn_t end, bool flush) { struct kvm_mmu_page *root; =20 - for_each_tdp_mmu_root_yield_safe(kvm, root, false) + lockdep_assert_held_write(&kvm->mmu_lock); + for_each_tdp_mmu_root_yield_safe(kvm, root) flush =3D tdp_mmu_zap_leafs(kvm, root, start, end, true, flush); =20 return flush; @@ -862,7 +860,8 @@ void kvm_tdp_mmu_zap_all(struct kvm *kvm) * is being destroyed or the userspace VMM has exited. In both cases, * KVM_RUN is unreachable, i.e. no vCPUs will ever service the request. */ - for_each_tdp_mmu_root_yield_safe(kvm, root, false) + lockdep_assert_held_write(&kvm->mmu_lock); + for_each_tdp_mmu_root_yield_safe(kvm, root) tdp_mmu_zap_root(kvm, root, false); } =20 @@ -876,7 +875,7 @@ void kvm_tdp_mmu_zap_invalidated_roots(struct kvm *kvm) =20 read_lock(&kvm->mmu_lock); =20 - for_each_tdp_mmu_root_yield_safe(kvm, root, true) { + for_each_tdp_mmu_root_yield_safe(kvm, root) { if (!root->tdp_mmu_scheduled_root_to_zap) continue; =20 @@ -899,7 +898,7 @@ void kvm_tdp_mmu_zap_invalidated_roots(struct kvm *kvm) * the root must be reachable by mmu_notifiers while it's being * zapped */ - kvm_tdp_mmu_put_root(kvm, root, true); + kvm_tdp_mmu_put_root(kvm, root); } =20 read_unlock(&kvm->mmu_lock); @@ -1133,7 +1132,9 @@ bool kvm_tdp_mmu_unmap_gfn_range(struct kvm *kvm, str= uct kvm_gfn_range *range, { struct kvm_mmu_page *root; =20 - __for_each_tdp_mmu_root_yield_safe(kvm, root, range->slot->as_id, false, = false) + lockdep_assert_held_write(&kvm->mmu_lock); + + __for_each_tdp_mmu_root_yield_safe(kvm, root, range->slot->as_id, false) flush =3D tdp_mmu_zap_leafs(kvm, root, range->start, range->end, range->may_block, flush); =20 @@ -1322,7 +1323,7 @@ bool kvm_tdp_mmu_wrprot_slot(struct kvm *kvm, =20 lockdep_assert_held_read(&kvm->mmu_lock); =20 - for_each_valid_tdp_mmu_root_yield_safe(kvm, root, slot->as_id, true) + for_each_valid_tdp_mmu_root_yield_safe(kvm, root, slot->as_id) spte_set |=3D wrprot_gfn_range(kvm, root, slot->base_gfn, slot->base_gfn + slot->npages, min_level); =20 @@ -1354,6 +1355,8 @@ static struct kvm_mmu_page *tdp_mmu_alloc_sp_for_spli= t(struct kvm *kvm, { struct kvm_mmu_page *sp; =20 + kvm_lockdep_assert_mmu_lock_held(kvm, shared); + /* * Since we are allocating while under the MMU lock we have to be * careful about GFP flags. Use GFP_NOWAIT to avoid blocking on direct @@ -1504,11 +1507,10 @@ void kvm_tdp_mmu_try_split_huge_pages(struct kvm *k= vm, int r =3D 0; =20 kvm_lockdep_assert_mmu_lock_held(kvm, shared); - - for_each_valid_tdp_mmu_root_yield_safe(kvm, root, slot->as_id, shared) { + for_each_valid_tdp_mmu_root_yield_safe(kvm, root, slot->as_id) { r =3D tdp_mmu_split_huge_pages_root(kvm, root, start, end, target_level,= shared); if (r) { - kvm_tdp_mmu_put_root(kvm, root, shared); + kvm_tdp_mmu_put_root(kvm, root); break; } } @@ -1568,8 +1570,7 @@ bool kvm_tdp_mmu_clear_dirty_slot(struct kvm *kvm, bool spte_set =3D false; =20 lockdep_assert_held_read(&kvm->mmu_lock); - - for_each_valid_tdp_mmu_root_yield_safe(kvm, root, slot->as_id, true) + for_each_valid_tdp_mmu_root_yield_safe(kvm, root, slot->as_id) spte_set |=3D clear_dirty_gfn_range(kvm, root, slot->base_gfn, slot->base_gfn + slot->npages); =20 @@ -1703,8 +1704,7 @@ void kvm_tdp_mmu_zap_collapsible_sptes(struct kvm *kv= m, struct kvm_mmu_page *root; =20 lockdep_assert_held_read(&kvm->mmu_lock); - - for_each_valid_tdp_mmu_root_yield_safe(kvm, root, slot->as_id, true) + for_each_valid_tdp_mmu_root_yield_safe(kvm, root, slot->as_id) zap_collapsible_spte_range(kvm, root, slot); } =20 --=20 2.39.1 From nobody Sun Feb 8 16:16:40 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2AD2E732E9 for ; Thu, 28 Sep 2023 16:30:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231389AbjI1Qav (ORCPT ); Thu, 28 Sep 2023 12:30:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46470 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231255AbjI1Qar (ORCPT ); Thu, 28 Sep 2023 12:30:47 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B75D1A8 for ; Thu, 28 Sep 2023 09:30:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695918604; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kglJ7MDXIf3EPYGcUm7QWoca/YSI3KryqQDz+PiB8OQ=; b=b0vMuiiIvoDrapdR7kL2Bd+2dKQet1JZOlhT/UhL0fKKn7I3uKJUYIRvHH+KVDwk0DHdSx FEZofVGXfir1jV9WqquZ7p6j0Ip1DJZ7wHBHnaQwHuNSOJXbjbTMtxRzIkp/W8poTa5zfC OnF9Gxg6uoEe4ORBUOLsK7HFskZYfps= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-606-DlzphYcnOs-th9PdQp_fug-1; Thu, 28 Sep 2023 12:30:01 -0400 X-MC-Unique: DlzphYcnOs-th9PdQp_fug-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7E845280BC9D; Thu, 28 Sep 2023 16:30:00 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6639C140E969; Thu, 28 Sep 2023 16:30:00 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH 3/3] KVM: x86/mmu: always take tdp_mmu_pages_lock Date: Thu, 28 Sep 2023 12:29:59 -0400 Message-Id: <20230928162959.1514661-4-pbonzini@redhat.com> In-Reply-To: <20230928162959.1514661-1-pbonzini@redhat.com> References: <20230928162959.1514661-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" It is cheap to take tdp_mmu_pages_lock in all write-side critical sections. We already do it all the time when zapping with read_lock(), so it is not a problem to do it from the kvm_tdp_mmu_zap_all() path (aka kvm_arch_flush_shadow_all(), aka VM destruction and MMU notifier release). Signed-off-by: Paolo Bonzini --- Documentation/virt/kvm/locking.rst | 6 ++---- arch/x86/kvm/mmu/tdp_mmu.c | 17 ++++++----------- 2 files changed, 8 insertions(+), 15 deletions(-) diff --git a/Documentation/virt/kvm/locking.rst b/Documentation/virt/kvm/lo= cking.rst index 3a034db5e55f..381eb0e7d947 100644 --- a/Documentation/virt/kvm/locking.rst +++ b/Documentation/virt/kvm/locking.rst @@ -43,10 +43,8 @@ On x86: =20 - vcpu->mutex is taken outside kvm->arch.hyperv.hv_lock and kvm->arch.xen.= xen_lock =20 -- kvm->arch.mmu_lock is an rwlock. kvm->arch.tdp_mmu_pages_lock and - kvm->arch.mmu_unsync_pages_lock are taken inside kvm->arch.mmu_lock, and - cannot be taken without already holding kvm->arch.mmu_lock (typically wi= th - ``read_lock`` for the TDP MMU, thus the need for additional spinlocks). +- kvm->arch.mmu_lock is an rwlock and is taken outside + kvm->arch.tdp_mmu_pages_lock and kvm->arch.mmu_unsync_pages_lock =20 Everything else is a leaf: no other lock is taken inside the critical sections. diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index b9abfa78808a..f61bc842067f 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -285,24 +285,19 @@ static void tdp_unaccount_mmu_page(struct kvm *kvm, s= truct kvm_mmu_page *sp) * the MMU lock and the operation must synchronize with other * threads that might be adding or removing pages. */ -static void tdp_mmu_unlink_sp(struct kvm *kvm, struct kvm_mmu_page *sp, - bool shared) +static void tdp_mmu_unlink_sp(struct kvm *kvm, struct kvm_mmu_page *sp) { + lockdep_assert_held(&kvm->mmu_lock); + tdp_unaccount_mmu_page(kvm, sp); =20 if (!sp->nx_huge_page_disallowed) return; =20 - if (shared) - spin_lock(&kvm->arch.tdp_mmu_pages_lock); - else - lockdep_assert_held_write(&kvm->mmu_lock); - + spin_lock(&kvm->arch.tdp_mmu_pages_lock); sp->nx_huge_page_disallowed =3D false; untrack_possible_nx_huge_page(kvm, sp); - - if (shared) - spin_unlock(&kvm->arch.tdp_mmu_pages_lock); + spin_unlock(&kvm->arch.tdp_mmu_pages_lock); } =20 /** @@ -331,7 +326,7 @@ static void handle_removed_pt(struct kvm *kvm, tdp_ptep= _t pt, bool shared) =20 trace_kvm_mmu_prepare_zap_page(sp); =20 - tdp_mmu_unlink_sp(kvm, sp, shared); + tdp_mmu_unlink_sp(kvm, sp); =20 for (i =3D 0; i < SPTE_ENT_PER_PAGE; i++) { tdp_ptep_t sptep =3D pt + i; --=20 2.39.1