From nobody Mon Feb 9 09:33:14 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linux.intel.com ARC-Seal: i=1; a=rsa-sha256; t=1663252779; cv=none; d=zohomail.com; s=zohoarc; b=TKRsl+0ej6ApMKjceL7PAkuz156ChpOETbBr1xtkV7s0iAZWhmGCTI843xhXMIXyt9j/fYKgy3pygO7tbqjMulBjN8CMkozdXK6mIc28517NJVdyAKAQFiNmCM0DUEAdMS6+e+WoJdc1HXbyQUMOSxE1/5mp3CTeWEzhhmu8Gqo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1663252779; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=He0Ijmf21PGBrboF0204SJXtyhXPPMbhvE53Ph3lTJc=; b=W+kgDWkhJC7A6kpSmFO+mF7CFoomJjLWe8azo86uQf2DAIfF4vv6t7J67/yUWaEHf78NRDvIxXhK6GOOeFm8UadlAJRF4Ix51F4ZhuXLE4QsD7MiRHlcld40a+yyOIwKBL3jNSdHdGAEPO8NEPGX1BnO3bBEtehosqCiGLOY2kw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1663252779812126.87568518970659; Thu, 15 Sep 2022 07:39:39 -0700 (PDT) Received: from localhost ([::1]:58134 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oYq1i-00082H-Jd for importer@patchew.org; Thu, 15 Sep 2022 10:39:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42834) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oYpx1-00051F-6D for qemu-devel@nongnu.org; Thu, 15 Sep 2022 10:34:48 -0400 Received: from mga07.intel.com ([134.134.136.100]:64428) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oYpwy-0000r5-IR for qemu-devel@nongnu.org; Thu, 15 Sep 2022 10:34:46 -0400 Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2022 07:34:42 -0700 Received: from chaop.bj.intel.com ([10.240.193.75]) by fmsmga005.fm.intel.com with ESMTP; 15 Sep 2022 07:34:32 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663252484; x=1694788484; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Raca1wOUdVWzIsRmNLgx+j7A1SHMFBohtFeko5o9+Ds=; b=g4YRqYfKhXl0B+zPWYsq0goUrdVwIOGv4OhfFDzCAL4qB9CKBEcy9WsM 3xrxeS7o+y+320h/TFH6/0MrIe84RRmjoFKgkcwucBP0kTVf4cGijIPak 7stCcw3rxyhqlHVbbfRYXXS2Q38btCPVVpu+iBHO+6WKQQUARYrJhLXcH IC525LcMqzfOkgQXlxlPgLOzLAbMXISzcnWmpgfJwcaZaSmjpNvvj82KY qeiad2rhG9DcH8DDM/wrgTF9DLj851WnOH7N/lAi4Fjuf59nrKwznOaHM zqhS5ztunuJeykH9qDQqAkTnSXEXvlZ4eQ1OenWVFRpkTkkg9zEIHUNIi w==; X-IronPort-AV: E=McAfee;i="6500,9779,10470"; a="362690365" X-IronPort-AV: E=Sophos;i="5.93,318,1654585200"; d="scan'208";a="362690365" X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,318,1654585200"; d="scan'208";a="945977075" From: Chao Peng To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-api@vger.kernel.org, linux-doc@vger.kernel.org, qemu-devel@nongnu.org Cc: Paolo Bonzini , Jonathan Corbet , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, "H . Peter Anvin" , Hugh Dickins , Jeff Layton , "J . Bruce Fields" , Andrew Morton , Shuah Khan , Mike Rapoport , Steven Price , "Maciej S . Szmigiero" , Vlastimil Babka , Vishal Annapurve , Yu Zhang , Chao Peng , "Kirill A . Shutemov" , luto@kernel.org, jun.nakajima@intel.com, dave.hansen@intel.com, ak@linux.intel.com, david@redhat.com, aarcange@redhat.com, ddutile@redhat.com, dhildenb@redhat.com, Quentin Perret , Michael Roth , mhocko@suse.com, Muchun Song , wei.w.wang@intel.com Subject: [PATCH v8 4/8] KVM: Use gfn instead of hva for mmu_notifier_retry Date: Thu, 15 Sep 2022 22:29:09 +0800 Message-Id: <20220915142913.2213336-5-chao.p.peng@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220915142913.2213336-1-chao.p.peng@linux.intel.com> References: <20220915142913.2213336-1-chao.p.peng@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: none client-ip=134.134.136.100; envelope-from=chao.p.peng@linux.intel.com; helo=mga07.intel.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @intel.com) X-ZM-MESSAGEID: 1663252781094100002 Content-Type: text/plain; charset="utf-8" Currently in mmu_notifier validate path, hva range is recorded and then checked against in the mmu_notifier_retry_hva() of the page fault path. However, for the to be introduced private memory, a page fault may not have a hva associated, checking gfn(gpa) makes more sense. For existing non private memory case, gfn is expected to continue to work. The only downside is when aliasing multiple gfns to a single hva, the current algorithm of checking multiple ranges could result in a much larger range being rejected. Such aliasing should be uncommon, so the impact is expected small. The patch also fixes a bug in kvm_zap_gfn_range() which has already been using gfn when calling kvm_mmu_invalidate_begin/end() while these functions accept hva in current code. Signed-off-by: Chao Peng --- arch/x86/kvm/mmu/mmu.c | 2 +- include/linux/kvm_host.h | 18 +++++++--------- virt/kvm/kvm_main.c | 45 ++++++++++++++++++++++++++-------------- 3 files changed, 39 insertions(+), 26 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index e418ef3ecfcb..08abad4f3e6f 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4203,7 +4203,7 @@ static bool is_page_fault_stale(struct kvm_vcpu *vcpu, return true; =20 return fault->slot && - mmu_invalidate_retry_hva(vcpu->kvm, mmu_seq, fault->hva); + mmu_invalidate_retry_gfn(vcpu->kvm, mmu_seq, fault->gfn); } =20 static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault = *fault) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index eac1787b899b..2125b50f6345 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -776,8 +776,8 @@ struct kvm { struct mmu_notifier mmu_notifier; unsigned long mmu_invalidate_seq; long mmu_invalidate_in_progress; - unsigned long mmu_invalidate_range_start; - unsigned long mmu_invalidate_range_end; + gfn_t mmu_invalidate_range_start; + gfn_t mmu_invalidate_range_end; #endif struct list_head devices; u64 manual_dirty_log_protect; @@ -1366,10 +1366,8 @@ void kvm_mmu_free_memory_cache(struct kvm_mmu_memory= _cache *mc); void *kvm_mmu_memory_cache_alloc(struct kvm_mmu_memory_cache *mc); #endif =20 -void kvm_mmu_invalidate_begin(struct kvm *kvm, unsigned long start, - unsigned long end); -void kvm_mmu_invalidate_end(struct kvm *kvm, unsigned long start, - unsigned long end); +void kvm_mmu_invalidate_begin(struct kvm *kvm, gfn_t start, gfn_t end); +void kvm_mmu_invalidate_end(struct kvm *kvm, gfn_t start, gfn_t end); =20 long kvm_arch_dev_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg); @@ -1938,9 +1936,9 @@ static inline int mmu_invalidate_retry(struct kvm *kv= m, unsigned long mmu_seq) return 0; } =20 -static inline int mmu_invalidate_retry_hva(struct kvm *kvm, +static inline int mmu_invalidate_retry_gfn(struct kvm *kvm, unsigned long mmu_seq, - unsigned long hva) + gfn_t gfn) { lockdep_assert_held(&kvm->mmu_lock); /* @@ -1950,8 +1948,8 @@ static inline int mmu_invalidate_retry_hva(struct kvm= *kvm, * positives, due to shortcuts when handing concurrent invalidations. */ if (unlikely(kvm->mmu_invalidate_in_progress) && - hva >=3D kvm->mmu_invalidate_range_start && - hva < kvm->mmu_invalidate_range_end) + gfn >=3D kvm->mmu_invalidate_range_start && + gfn < kvm->mmu_invalidate_range_end) return 1; if (kvm->mmu_invalidate_seq !=3D mmu_seq) return 1; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 12dc0dc57b06..fa9dd2d2c001 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -540,8 +540,7 @@ static void kvm_mmu_notifier_invalidate_range(struct mm= u_notifier *mn, =20 typedef bool (*hva_handler_t)(struct kvm *kvm, struct kvm_gfn_range *range= ); =20 -typedef void (*on_lock_fn_t)(struct kvm *kvm, unsigned long start, - unsigned long end); +typedef void (*on_lock_fn_t)(struct kvm *kvm, gfn_t start, gfn_t end); =20 typedef void (*on_unlock_fn_t)(struct kvm *kvm); =20 @@ -628,7 +627,8 @@ static __always_inline int __kvm_handle_hva_range(struc= t kvm *kvm, locked =3D true; KVM_MMU_LOCK(kvm); if (!IS_KVM_NULL_FN(range->on_lock)) - range->on_lock(kvm, range->start, range->end); + range->on_lock(kvm, gfn_range.start, + gfn_range.end); if (IS_KVM_NULL_FN(range->handler)) break; } @@ -715,15 +715,9 @@ static void kvm_mmu_notifier_change_pte(struct mmu_not= ifier *mn, kvm_handle_hva_range(mn, address, address + 1, pte, kvm_set_spte_gfn); } =20 -void kvm_mmu_invalidate_begin(struct kvm *kvm, unsigned long start, - unsigned long end) +static inline void update_invalidate_range(struct kvm *kvm, gfn_t start, + gfn_t end) { - /* - * The count increase must become visible at unlock time as no - * spte can be established without taking the mmu_lock and - * count is also read inside the mmu_lock critical section. - */ - kvm->mmu_invalidate_in_progress++; if (likely(kvm->mmu_invalidate_in_progress =3D=3D 1)) { kvm->mmu_invalidate_range_start =3D start; kvm->mmu_invalidate_range_end =3D end; @@ -744,6 +738,28 @@ void kvm_mmu_invalidate_begin(struct kvm *kvm, unsigne= d long start, } } =20 +static void mark_invalidate_in_progress(struct kvm *kvm, gfn_t start, gfn_= t end) +{ + /* + * The count increase must become visible at unlock time as no + * spte can be established without taking the mmu_lock and + * count is also read inside the mmu_lock critical section. + */ + kvm->mmu_invalidate_in_progress++; +} + +static bool kvm_mmu_handle_gfn_range(struct kvm *kvm, struct kvm_gfn_range= *range) +{ + update_invalidate_range(kvm, range->start, range->end); + return kvm_unmap_gfn_range(kvm, range); +} + +void kvm_mmu_invalidate_begin(struct kvm *kvm, gfn_t start, gfn_t end) +{ + mark_invalidate_in_progress(kvm, start, end); + update_invalidate_range(kvm, start, end); +} + static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, const struct mmu_notifier_range *range) { @@ -752,8 +768,8 @@ static int kvm_mmu_notifier_invalidate_range_start(stru= ct mmu_notifier *mn, .start =3D range->start, .end =3D range->end, .pte =3D __pte(0), - .handler =3D kvm_unmap_gfn_range, - .on_lock =3D kvm_mmu_invalidate_begin, + .handler =3D kvm_mmu_handle_gfn_range, + .on_lock =3D mark_invalidate_in_progress, .on_unlock =3D kvm_arch_guest_memory_reclaimed, .flush_on_ret =3D true, .may_block =3D mmu_notifier_range_blockable(range), @@ -791,8 +807,7 @@ static int kvm_mmu_notifier_invalidate_range_start(stru= ct mmu_notifier *mn, return 0; } =20 -void kvm_mmu_invalidate_end(struct kvm *kvm, unsigned long start, - unsigned long end) +void kvm_mmu_invalidate_end(struct kvm *kvm, gfn_t start, gfn_t end) { /* * This sequence increase will notify the kvm page fault that --=20 2.25.1