From nobody Tue Dec 16 05:55:03 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C810C253920 for ; Thu, 8 May 2025 14:10:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746713419; cv=none; b=Ea3IYwmzJLf36XWcqZIzs+YC8Awf6gblMZzR96gzVN/rjn3sFhATdiwVLX2XFttVKRsnuxu98SWTU0DI90ki2vib2UFvSCgpPHExn6gc3FxSE3naTeO81zzozJUBGkiJPrY4CEMC/4sEcnVqNYOHXOGjg1w/26TMu5EzVPkmEck= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746713419; c=relaxed/simple; bh=G4MulOxIAWodwoabJCbtlh+WMIsmZESdSAC65RXZU30=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fLn0wETSr5AktUDbOlTC/Al4PXAlL16as6wyGyI5d2DA4qJG8ft5wsDdLkfpkyt+3u4FYSNCwt6fFgcBEWJb/9reyGhH1jt/xNAwN7GXRBNVOHOIkQZE65+Iiv2xQxtLqUWzNRdlEsEke1gOlnPaO8WjC6ox6+Mt80QBdk3jlhs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=uVQEdqAh; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="uVQEdqAh" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-30aab0f21a3so989004a91.3 for ; Thu, 08 May 2025 07:10:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1746713417; x=1747318217; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=r0z8Ayy/An5FIdcoS5gnzj4EHpvTo5w/T7c83ie1jKg=; b=uVQEdqAh+T5JiTz9GotyZ73+vQhKxxPi8KSIAk8wybD1+0g06OtcgNvjV+3hoburEx 6JLNAvRVlQCW2k36SML/XgcYYbkXpevzHlCrlla1o3VrHFcGN1HV1/UAPzib9AVF3JS/ tpH9OEQcB79yzvo5oBbsTKoPYKk4knpWx79Zjvv2i2tNuABjSNliXw2allmr1ZSIHasz 7LpyLhSrbzRy7nldDvSArOrMo0gr7lCTaI0KDQedLcYdE5gJvbvTpw1fZVz3kLGOlWi9 X6NDKaq1GJ3YTaHm7Hk1KUicD899x9uwCbAMnT7e0HR9CMuHDnrRl27X1/3jaE51lycq pfBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746713417; x=1747318217; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=r0z8Ayy/An5FIdcoS5gnzj4EHpvTo5w/T7c83ie1jKg=; b=UJjLAoxihOtK0H/l71Nn2RDmWn3bWdJnz0xhxuXleSOdNT4306D6DcNAd7jVjV+bAy kes2W7z88A+nXTLD3irbeeXPygy5MEWEr5KGH4WzQaUdgUNtgCJTMAHEEFCzhKECd93x BAApl01dB2/qinrpoPO2EcdpTRnicYDySQH/iMJqBti2RH7gO/0eyxFbM1g9VvMbcvSh 39n+QCvEZ+HSrsHac29bcNq/oUX9Di/8u12gxl53bP2sKZdYNIoPcpi1rnMlYH3qNPxA 3mXzi2FKCy4YoG9RF4wy1bsP2Hknk65rO/KmIdaJdKyrZ3L75WR5h9RbAFZ8jvQhZpQj nnFA== X-Forwarded-Encrypted: i=1; AJvYcCVEsqsEq2xSb0uIBmRoF/Uu2WjoLgV5M5TbJpX4KuNZxvCntfJXI8OTzTgUW638qCPNeQtoCuYUDj8QtNc=@vger.kernel.org X-Gm-Message-State: AOJu0YzUul4XDymvqXAKARH1F29GPWtwGLWXUaziM422432fmYFXjOwD h94k74c/KYtWiXLd7bdWgvtHzYjDIdtXh5E3Gsj3GcxDq8BB2BLKMh5pGw1uc6HKeN6EpxYSzJj e6g== X-Google-Smtp-Source: AGHT+IEtwm0yhtMPGtbie+qGTsHKAvlMoQCo2CofB6XzmclxdV1LwoF+6M5dpkArbOuyw8zzgEGn1qZQuNk= X-Received: from pjtd8.prod.google.com ([2002:a17:90b:48:b0:2ee:4a90:3d06]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4b87:b0:2ee:90a1:5d42 with SMTP id 98e67ed59e1d1-30aac097e49mr13855976a91.0.1746713417034; Thu, 08 May 2025 07:10:17 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 8 May 2025 07:10:08 -0700 In-Reply-To: <20250508141012.1411952-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250508141012.1411952-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1015.ga840276032-goog Message-ID: <20250508141012.1411952-2-seanjc@google.com> Subject: [PATCH v2 1/5] KVM: Bound the number of dirty ring entries in a single reset at INT_MAX From: Sean Christopherson To: Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Xu , Yan Zhao , Maxim Levitsky , Sean Christopherson Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Cap the number of ring entries that are reset in a single ioctl to INT_MAX to ensure userspace isn't confused by a wrap into negative space, and so that, in a truly pathological scenario, KVM doesn't miss a TLB flush due to the count wrapping to zero. While the size of the ring is fixed at 0x10000 entries and KVM (currently) supports at most 4096, userspace is allowed to harvest entries from the ring while the reset is in-progress, i.e. it's possible for the ring to always have harvested entries. Opportunistically return an actual error code from the helper so that a future fix to handle pending signals can gracefully return -EINTR. Cc: Peter Xu Cc: Yan Zhao Cc: Maxim Levitsky Fixes: fb04a1eddb1a ("KVM: X86: Implement ring-based dirty memory tracking") Signed-off-by: Sean Christopherson --- include/linux/kvm_dirty_ring.h | 8 +++++--- virt/kvm/dirty_ring.c | 10 +++++----- virt/kvm/kvm_main.c | 9 ++++++--- 3 files changed, 16 insertions(+), 11 deletions(-) diff --git a/include/linux/kvm_dirty_ring.h b/include/linux/kvm_dirty_ring.h index da4d9b5f58f1..ee61ff6c3fe4 100644 --- a/include/linux/kvm_dirty_ring.h +++ b/include/linux/kvm_dirty_ring.h @@ -49,9 +49,10 @@ static inline int kvm_dirty_ring_alloc(struct kvm *kvm, = struct kvm_dirty_ring *r } =20 static inline int kvm_dirty_ring_reset(struct kvm *kvm, - struct kvm_dirty_ring *ring) + struct kvm_dirty_ring *ring, + int *nr_entries_reset) { - return 0; + return -ENOENT; } =20 static inline void kvm_dirty_ring_push(struct kvm_vcpu *vcpu, @@ -82,7 +83,8 @@ int kvm_dirty_ring_alloc(struct kvm *kvm, struct kvm_dirt= y_ring *ring, * called with kvm->slots_lock held, returns the number of * processed pages. */ -int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring); +int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring, + int *nr_entries_reset); =20 /* * returns =3D0: successfully pushed diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c index d14ffc7513ee..77986f34eff8 100644 --- a/virt/kvm/dirty_ring.c +++ b/virt/kvm/dirty_ring.c @@ -105,19 +105,19 @@ static inline bool kvm_dirty_gfn_harvested(struct kvm= _dirty_gfn *gfn) return smp_load_acquire(&gfn->flags) & KVM_DIRTY_GFN_F_RESET; } =20 -int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring) +int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring, + int *nr_entries_reset) { u32 cur_slot, next_slot; u64 cur_offset, next_offset; unsigned long mask; - int count =3D 0; struct kvm_dirty_gfn *entry; bool first_round =3D true; =20 /* This is only needed to make compilers happy */ cur_slot =3D cur_offset =3D mask =3D 0; =20 - while (true) { + while (likely((*nr_entries_reset) < INT_MAX)) { entry =3D &ring->dirty_gfns[ring->reset_index & (ring->size - 1)]; =20 if (!kvm_dirty_gfn_harvested(entry)) @@ -130,7 +130,7 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_di= rty_ring *ring) kvm_dirty_gfn_set_invalid(entry); =20 ring->reset_index++; - count++; + (*nr_entries_reset)++; /* * Try to coalesce the reset operations when the guest is * scanning pages in the same slot. @@ -167,7 +167,7 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_di= rty_ring *ring) =20 trace_kvm_dirty_ring_reset(ring); =20 - return count; + return 0; } =20 void kvm_dirty_ring_push(struct kvm_vcpu *vcpu, u32 slot, u64 offset) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 16fe54cf2808..f9de2c8b9c1e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -4903,15 +4903,18 @@ static int kvm_vm_ioctl_reset_dirty_pages(struct kv= m *kvm) { unsigned long i; struct kvm_vcpu *vcpu; - int cleared =3D 0; + int cleared =3D 0, r; =20 if (!kvm->dirty_ring_size) return -EINVAL; =20 mutex_lock(&kvm->slots_lock); =20 - kvm_for_each_vcpu(i, vcpu, kvm) - cleared +=3D kvm_dirty_ring_reset(vcpu->kvm, &vcpu->dirty_ring); + kvm_for_each_vcpu(i, vcpu, kvm) { + r =3D kvm_dirty_ring_reset(vcpu->kvm, &vcpu->dirty_ring, &cleared); + if (r) + break; + } =20 mutex_unlock(&kvm->slots_lock); =20 --=20 2.49.0.1015.ga840276032-goog From nobody Tue Dec 16 05:55:03 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C47DD253F15 for ; Thu, 8 May 2025 14:10:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746713421; cv=none; b=J8nl8/jIoxgu3ZeqjD6FcrJWosBqhWBNB0qHvVnvZSQAed6RybEHQM1BXrqLcvM576dn2x6BBTn31cXhGSEs/+r02q+WhyOfs3fmBc5Yxljd4SNADjLjN+eEu3zA3Lx1eu0/i06dNPrvkNLTH6VIshKia4S5UDNbxC2cmIrNFy8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746713421; c=relaxed/simple; bh=h9ceh9DWtwvOAv8lZI5QIIzEnFfIR1nnvswMVkuMh7c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=MtPutXHPY2Rj8ZXoIz7wolIISQHzKWTdmX6dqWEcj8RMyVv9aKScTMxG0HjNKHYKfS8srrb4AnuU2SoIG0WLx2Uf2Srcu+Hb1gLRektdmuXFShW5uRCN62J2g3rnhKZwdcWV24WPx2Qx/ZmeLdyH9i7NxaZpcg1EGq3on1DYqXs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bm6axfZm; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bm6axfZm" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-7395d07a3dcso785863b3a.3 for ; Thu, 08 May 2025 07:10:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1746713419; x=1747318219; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=p+56ubm47OlXcm/VaWogaGlH7q4WP+P+2pKjZu9cp4w=; b=bm6axfZmRXgeuF9fa6kEQJoma3Qim35CdnI18deLf9b2MaHRq1Z0ouEjlfGuk+Jv6J GRT4f8NL/JVvw7LKwHhyuIxPK53pYd0tKVbjFT+JxDH8ZSEeG5NG0vZ1pCNwV0Ic2uMk +/dW4LwVu+Mc7JoQ5ms0DVqsVQ4kaAZui1imiDkYSX6Rh9M4wmjdGNyjOJR7hZ2lSO6u XSGQpQ3itYTHSfAt2Y/IDTn7OC1x3QzheU7QXT9yWpwImj0U1bzcAjwOUeBMLQSdSVMt wm2L+YdvtlqTtY5ChjG6VJ7AgPc6xN0ve0lUaZNMXtC47/4jS5iqUcSZOuGrhOVEIwpf vAFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746713419; x=1747318219; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=p+56ubm47OlXcm/VaWogaGlH7q4WP+P+2pKjZu9cp4w=; b=SQIX2jt2D3RPNrr1YcmL5Cg9YnBv9y/meR3GFP7R+13PuFApqA85e+lgqelnF+Yjik umcLcXaO68AtXzI4Gdf5CQLLIYXuGRHbAc15l0aI964H9n7/SXdEbeZKxsKToD2KS3yC AMJagskYddkYhtcf0Bi+7e+WHPlOyXjG3UUhGuTKN9TM1xaO+xdWW/5ezRmESKzqc7/x tN0XOdYgHh14CrkdsnFOSHC6dOdwFCIDsSDy6LdU3V0qyD3a9H1l5JWZcv/FDEIRirZ4 vMPpSC6t0xOMnL1DVzIOe8OMIFTEZllQ+WkHWFkWK/xhVQv357JSz0uZSaOG/UoRBqS1 11ng== X-Forwarded-Encrypted: i=1; AJvYcCWOwOAZm4V1urb5hUq9D35Awf7xjdCRFfY+IWzM4iTmY7Qfcfw7R6wwHdsax+8p6SHE3Hwy9LTkOKrdsFI=@vger.kernel.org X-Gm-Message-State: AOJu0Yx/xu8BBse0itB1s/q7LNVPcI/izcXwvqTVqGjaoLvPtLsymwgK 3hWenh8dfkiaQvcs9ljI1xIcRK5cWlcxx6G+zxPnd2n2OVVLXVKI4GUApJScTL9vvSJ2CP6A7FI wNg== X-Google-Smtp-Source: AGHT+IG2eg6x/bjzTx5zXZD+HM1Dnr1Anhc1D/g6qFT5/6izbWgde1cjoEAZ96/ZeW59opbMXnoIvhkIYXc= X-Received: from pfbdu10.prod.google.com ([2002:a05:6a00:2b4a:b0:740:41eb:584c]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:4c0c:b0:736:41ec:aaad with SMTP id d2e1a72fcca58-740a99cdbe4mr5209318b3a.14.1746713418848; Thu, 08 May 2025 07:10:18 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 8 May 2025 07:10:09 -0700 In-Reply-To: <20250508141012.1411952-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250508141012.1411952-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1015.ga840276032-goog Message-ID: <20250508141012.1411952-3-seanjc@google.com> Subject: [PATCH v2 2/5] KVM: Bail from the dirty ring reset flow if a signal is pending From: Sean Christopherson To: Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Xu , Yan Zhao , Maxim Levitsky , Sean Christopherson Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Abort a dirty ring reset if the current task has a pending signal, as the hard limit of INT_MAX entries doesn't ensure KVM will respond to a signal in a timely fashion. Fixes: fb04a1eddb1a ("KVM: X86: Implement ring-based dirty memory tracking") Signed-off-by: Sean Christopherson --- virt/kvm/dirty_ring.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c index 77986f34eff8..e844e869e8c7 100644 --- a/virt/kvm/dirty_ring.c +++ b/virt/kvm/dirty_ring.c @@ -118,6 +118,9 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_di= rty_ring *ring, cur_slot =3D cur_offset =3D mask =3D 0; =20 while (likely((*nr_entries_reset) < INT_MAX)) { + if (signal_pending(current)) + return -EINTR; + entry =3D &ring->dirty_gfns[ring->reset_index & (ring->size - 1)]; =20 if (!kvm_dirty_gfn_harvested(entry)) --=20 2.49.0.1015.ga840276032-goog From nobody Tue Dec 16 05:55:03 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF187265611 for ; Thu, 8 May 2025 14:10:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746713423; cv=none; b=E2hWGxsSJhCTSzasICTHbE4Ly0GncBpW092Gbb0YjJIojwdZmOFYWzYiahh/EDuBwwPxXigJ2JChl3e5TZLYcp4o4ipdkAnriWVQKVThYNzbkOXpCDiEbRJJaYClB/qoufRvUolyDCkkumPOntw839Sv/8ecokqJGKJgV5vA0ns= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746713423; c=relaxed/simple; bh=rBZ8oHeY+uQoBWsAP98N8CQDCX40XW60zl1C5BKmQ7U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=DhtHQOzdY+9U2ltVzVbrh/GQXGKl/f5yZLtXcB/FMkbz3gzFr9f2iw0g7va9fNmqpEhh/dIWDzbgx4yBClJ/1bU94sz+tDS+PUcebOa8r6f9hb+e2ZBZippxItmEQdwLiyYXRd89gdG0TEtqg8vNvh8a6gmb5enyihpen6p+kiE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=yZQgNjfL; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="yZQgNjfL" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-736b5f9279cso1085034b3a.2 for ; Thu, 08 May 2025 07:10:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1746713421; x=1747318221; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=KJVX1DCqX7vbNbrylbBP6BivBKN94o2apUJMFPAVyIY=; b=yZQgNjfLY8tMCqmk/kgtHcZGrwP+oXxFgh1o3ToR8ifTATCjTnpwLEfFcf607mTV3l ZKlrYYUPZMyWKFDSlyJ1YcM/zxjw1TtNtt2VveUG71ZdDu7TtoI6uW+XVxdAfpiOyjKE FqTJbwgF701mkifSYOt/vGBVrz3fAKOtst4iqCGApXJGiank5KlFqK/i2CJJoY5FXdqS +g8lxUb/hqKZMsbXEFIsYuL590X1UlCZ0HVKePbLWnr08onXu/dUAZyk42JHNLaM8SgP FCySqyIxKoWwc+LFFamibqBJqckq+43yB4lnaqpgNksC/mwNe2Vk7w/NXdPY+lW+p0sW XuwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746713421; x=1747318221; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KJVX1DCqX7vbNbrylbBP6BivBKN94o2apUJMFPAVyIY=; b=MXaVaMLEr7P8Ssyl1+oOxMF7BxAihYx0WPBPiuEZqzt+zdV8fNeKYF3glRpVI+pyNZ of/2KeBa9quNQFiHrnGtXfUgbhDWhJTX7hSgzHtpSxy/TBvoPR1XojDSkkwyX+Oq4USf uqaBupdDdChDxNnDIkKAGDb4jE/cpntDKgM3g77umzxlI6SLQbiMWB7vXYL4+FbFKLVB cP9UQIOHMpkH4RzmVVO6HlkStsrx9XJFvm/f8YNMzWXvqaHr/XcSYGWgrq1e/DDKtpG2 WVpn2ALFJKCH2CiFMoiwhCyjMlpBzi3wwf6rCK0eYpWZdnpkQ7cNSFrOgyzKkNqsVMIg yONQ== X-Forwarded-Encrypted: i=1; AJvYcCXrpYrCz1NZcs6mHXBIrTAhpOyRcf3pl45+Wq6LlbKtvWfjoVeSh1Ii45UgbruivwNHc5vutO6mbZeokSs=@vger.kernel.org X-Gm-Message-State: AOJu0Yxqi1BGcJ2fSeQUBwkhyGsXQHfXMLS70zzhb2l9ptA+zqU+Y22r il93IuqiqtJemqEy6Z+t33hchklZ866dM/zyNvEC0sDzyRzN6htFh2kwjMGdur+4oJD8asphZjo pug== X-Google-Smtp-Source: AGHT+IGeOozbRNnQDObWfA03lfKGbI9CmCxprWmItBaqzUSnplsOV8hYwrU5Or4chHdRZ8SUt7GQWEf4fi8= X-Received: from pful19.prod.google.com ([2002:a05:6a00:1413:b0:740:addd:daba]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:2908:b0:72d:3b2e:fef9 with SMTP id d2e1a72fcca58-7409d00d6fbmr11961256b3a.20.1746713420600; Thu, 08 May 2025 07:10:20 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 8 May 2025 07:10:10 -0700 In-Reply-To: <20250508141012.1411952-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250508141012.1411952-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1015.ga840276032-goog Message-ID: <20250508141012.1411952-4-seanjc@google.com> Subject: [PATCH v2 3/5] KVM: Conditionally reschedule when resetting the dirty ring From: Sean Christopherson To: Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Xu , Yan Zhao , Maxim Levitsky , Sean Christopherson Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When resetting a dirty ring, conditionally reschedule on each iteration after the first. The recently introduced hard limit mitigates the issue of an endless reset, but isn't sufficient to completely prevent RCU stalls, soft lockups, etc., nor is the hard limit intended to guard against such badness. Note! Take care to check for reschedule even in the "continue" paths, as a pathological scenario (or malicious userspace) could dirty the same gfn over and over, i.e. always hit the continue path. rcu: INFO: rcu_sched self-detected stall on CPU rcu: 4-....: (5249 ticks this GP) idle=3D51e4/1/0x4000000000000000 softi= rq=3D309/309 fqs=3D2563 rcu: (t=3D5250 jiffies g=3D-319 q=3D608 ncpus=3D24) CPU: 4 UID: 1000 PID: 1067 Comm: dirty_log_test Tainted: G L = 6.13.0-rc3-17fa7a24ea1e-HEAD-vm #814 Tainted: [L]=3DSOFTLOCKUP Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015 RIP: 0010:kvm_arch_mmu_enable_log_dirty_pt_masked+0x26/0x200 [kvm] Call Trace: kvm_reset_dirty_gfn.part.0+0xb4/0xe0 [kvm] kvm_dirty_ring_reset+0x58/0x220 [kvm] kvm_vm_ioctl+0x10eb/0x15d0 [kvm] __x64_sys_ioctl+0x8b/0xb0 do_syscall_64+0x5b/0x160 entry_SYSCALL_64_after_hwframe+0x4b/0x53 Tainted: [L]=3DSOFTLOCKUP Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015 RIP: 0010:kvm_arch_mmu_enable_log_dirty_pt_masked+0x17/0x200 [kvm] Call Trace: kvm_reset_dirty_gfn.part.0+0xb4/0xe0 [kvm] kvm_dirty_ring_reset+0x58/0x220 [kvm] kvm_vm_ioctl+0x10eb/0x15d0 [kvm] __x64_sys_ioctl+0x8b/0xb0 do_syscall_64+0x5b/0x160 entry_SYSCALL_64_after_hwframe+0x4b/0x53 Fixes: fb04a1eddb1a ("KVM: X86: Implement ring-based dirty memory tracking") Signed-off-by: Sean Christopherson --- virt/kvm/dirty_ring.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c index e844e869e8c7..97cca0c02fd1 100644 --- a/virt/kvm/dirty_ring.c +++ b/virt/kvm/dirty_ring.c @@ -134,6 +134,16 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_d= irty_ring *ring, =20 ring->reset_index++; (*nr_entries_reset)++; + + /* + * While the size of each ring is fixed, it's possible for the + * ring to be constantly re-dirtied/harvested while the reset + * is in-progress (the hard limit exists only to guard against + * wrapping the count into negative space). + */ + if (!first_round) + cond_resched(); + /* * Try to coalesce the reset operations when the guest is * scanning pages in the same slot. --=20 2.49.0.1015.ga840276032-goog From nobody Tue Dec 16 05:55:03 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7633266B4E for ; Thu, 8 May 2025 14:10:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746713425; cv=none; b=lBQlIWtaxPwznlIHyeHg3uFSPDZ2JmVb9H5gjqTY/iWY7vqzVwwWsOBa2jtbuZ0lS+0kG6fo6gc+SSHffxODqJf3+5Oxww72uH38TtCUAB+MxF0UZzE5mEHMC2y3C/q5QWb9WmzjxJ9ZSH54mJJOPhA0/gLuJSiP0LmFnEaISYo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746713425; c=relaxed/simple; bh=awSJS8WConudI9j96iI1tSZciOtqzk/WMv+Azjyuw5A=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jiVzEBnXGKSiG9TJYlfANUfcWkH6J0YNnrzhSVpspOtukdnm+8EvgHASnn1/4P/5Z4FykEf1Ii89GyRAsTptNTz2f8UviUNRJMVuihQIXf3KvHQ1cdU8ISfGbJW34oHCD7mKBOTMuT3QKtEMPsBUufkgH6DtRL+qEB3UJOUZlBs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=JkkAsPuL; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="JkkAsPuL" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-73bfc657aefso852733b3a.1 for ; Thu, 08 May 2025 07:10:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1746713423; x=1747318223; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=ickIymo1Jt+rHn8phqijkm5gyPMPS3XGUhpL5W5szxc=; b=JkkAsPuLfXVqfoTGbXl996mYyAQa5A2IVn1pgWLhb3oB3Of9WQXA5TOJV8oO42N4G3 EGQzQB2SCUstHYbetR0vHko+tauX4E3RHMO2G3aQGYQIvoRmanHZY5nmAcxMusFVQEIJ Z7TIYSVOH8GrHzAUXpjKles2P2v8mLOjRP/ZikR0iUnN0Lw0yhjpjHPaGeqP3NB7GULL u/A7pZ0h6nxRBcGoBfY7DmLbB5oK38f3PXOqRHcio3LNgAf7F5oDY3BP37gtza+0wU8N I29evWZCKHLnaHx1rcRXaHA0AksnonIv8j134C4r/TCZXbIXay1n9xN6RaRrCUNAjNZv Poog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746713423; x=1747318223; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ickIymo1Jt+rHn8phqijkm5gyPMPS3XGUhpL5W5szxc=; b=SFpFVU6uwelmCnCOGozOq5Lr/nVkGSl7/wqLUi985EoVMA4fsVxM0Y2lU3b90yIE0o HBpnHJplP0NFr4gGONwnPIwvgTKMs72WIpgDOOilTKaYHcdg+PbSlQUT/uzWYaQ3gVlo nCC6ZWZm1CPRnIYBrJ017vrw2FkJLXsxPGpHpvCTQuARNLx49T1XpnD99sWRwPE+VwvI FArOSEFAE63a9IccKkJsuGLORo3Udm/nkxarWA+lvpgKlpcFl9xrE9qq/OA/8Ld0RkYP t9rn2kV9/Jyf0XAN9Tv4zh+Rn36QdhpeZ+shfs72SDDZ4DXaRVAdsRagl2kXT0eQxciB ykbA== X-Forwarded-Encrypted: i=1; AJvYcCXxurjaLlQpUOHYi6gnPzWc4EV2upoIvp5VrLct9CdCkyEJaCjVeefdE2Y4OdKKd+kDZKN4FDl7YB+OEq0=@vger.kernel.org X-Gm-Message-State: AOJu0YzujxjMjThwhENwFMyzAyecHF+AxbapLwbKjIw9s+AHYfyNtf8I a0skK/Pg8q/W9FrHPY3BQSR7pUTBT63/8JAmslgGA2VsFuz1BFSoZZV55YY1mc4nUabHQMgGwQV rFA== X-Google-Smtp-Source: AGHT+IHLiymXHmxCwlVp1XiwMzwsV4UPdVj2tb8haL51LmOJpuCCiwihi6XjLonfWdsyG+RYO6/C9IKGO2I= X-Received: from pfbgs18.prod.google.com ([2002:a05:6a00:4d92:b0:739:45ba:a49a]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:35cd:b0:732:2923:b70f with SMTP id d2e1a72fcca58-740a99c4bd6mr4703644b3a.11.1746713422488; Thu, 08 May 2025 07:10:22 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 8 May 2025 07:10:11 -0700 In-Reply-To: <20250508141012.1411952-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250508141012.1411952-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1015.ga840276032-goog Message-ID: <20250508141012.1411952-5-seanjc@google.com> Subject: [PATCH v2 4/5] KVM: Check for empty mask of harvested dirty ring entries in caller From: Sean Christopherson To: Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Xu , Yan Zhao , Maxim Levitsky , Sean Christopherson Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When resetting a dirty ring, explicitly check that there is work to be done before calling kvm_reset_dirty_gfn(), e.g. if no harvested entries are found and/or on the loop's first iteration, and delete the extremely misleading comment "This is only needed to make compilers happy". KVM absolutely relies on mask to be zero-initialized, i.e. the comment is an outright lie. Furthermore, the compiler is right to complain that KVM is calling a function with uninitialized data, as there are no guarantees the implementation details of kvm_reset_dirty_gfn() will be visible to kvm_dirty_ring_reset(). While the flaw could be fixed by simply deleting (or rewording) the comment, and duplicating the check is unfortunate, checking mask in the caller will allow for additional cleanups. Opportunisticaly drop the zero-initialization of cur_slot and cur_offset. If a bug were introduced where either the slot or offset was consumed before mask is set to a non-zero value, then it is highly desirable for the compiler (or some other sanitizer) to yell. Cc: Peter Xu Cc: Yan Zhao Cc: Maxim Levitsky Signed-off-by: Sean Christopherson Reviewed-by: Pankaj Gupta --- virt/kvm/dirty_ring.c | 44 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 35 insertions(+), 9 deletions(-) diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c index 97cca0c02fd1..a3434be8f00d 100644 --- a/virt/kvm/dirty_ring.c +++ b/virt/kvm/dirty_ring.c @@ -55,9 +55,6 @@ static void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot= , u64 offset, u64 mask) struct kvm_memory_slot *memslot; int as_id, id; =20 - if (!mask) - return; - as_id =3D slot >> 16; id =3D (u16)slot; =20 @@ -108,15 +105,24 @@ static inline bool kvm_dirty_gfn_harvested(struct kvm= _dirty_gfn *gfn) int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring, int *nr_entries_reset) { + /* + * To minimize mmu_lock contention, batch resets for harvested entries + * whose gfns are in the same slot, and are within N frame numbers of + * each other, where N is the number of bits in an unsigned long. For + * simplicity, process the current set of entries when the next entry + * can't be included in the batch. + * + * Track the current batch slot, the gfn offset into the slot for the + * batch, and the bitmask of gfns that need to be reset (relative to + * offset). Note, the offset may be adjusted backwards, e.g. so that + * a sequence of gfns X, X-1, ... X-N can be batched. + */ u32 cur_slot, next_slot; u64 cur_offset, next_offset; - unsigned long mask; + unsigned long mask =3D 0; struct kvm_dirty_gfn *entry; bool first_round =3D true; =20 - /* This is only needed to make compilers happy */ - cur_slot =3D cur_offset =3D mask =3D 0; - while (likely((*nr_entries_reset) < INT_MAX)) { if (signal_pending(current)) return -EINTR; @@ -164,14 +170,34 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_= dirty_ring *ring, continue; } } - kvm_reset_dirty_gfn(kvm, cur_slot, cur_offset, mask); + + /* + * Reset the slot for all the harvested entries that have been + * gathered, but not yet fully processed. + */ + if (mask) + kvm_reset_dirty_gfn(kvm, cur_slot, cur_offset, mask); + + /* + * The current slot was reset or this is the first harvested + * entry, (re)initialize the metadata. + */ cur_slot =3D next_slot; cur_offset =3D next_offset; mask =3D 1; first_round =3D false; } =20 - kvm_reset_dirty_gfn(kvm, cur_slot, cur_offset, mask); + /* + * Perform a final reset if there are harvested entries that haven't + * been processed, which is guaranteed if at least one harvested was + * found. The loop only performs a reset when the "next" entry can't + * be batched with "current" the entry(s), and that reset processes the + * _current_ entry(s), i.e. the last harvested entry, a.k.a. next, will + * always be left pending. + */ + if (mask) + kvm_reset_dirty_gfn(kvm, cur_slot, cur_offset, mask); =20 /* * The request KVM_REQ_DIRTY_RING_SOFT_FULL will be cleared --=20 2.49.0.1015.ga840276032-goog From nobody Tue Dec 16 05:55:03 2025 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D353266F10 for ; Thu, 8 May 2025 14:10:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746713427; cv=none; b=mrb+By4CskMuHq4imfeje7o1Yl9pHfjsqecxH3oNqFP07Nhho+nI6by0BjsRb7jPJn77rUrThzu6js/cQmMENO8B1+XWDqB23XhVT5Nx2W2LLQ6WOdqcuhD0AXytL3prVW3A8l8MDYzIZLbrFABXs23FmNRZCJPV6T8R4BujwnU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746713427; c=relaxed/simple; bh=RaCWxBRdYBPrhmSkEzciipjWTledXPE2JsIOwJIPUVc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=rRy/MvWRD3ajGu53sCWUaD87mRXc2Y3n08y34i/kb2e6Tu2CWGyBuXFrujKR3m0ulCapzshj9fEWISgZ8i2CK1zfkTAd2p6oetJ0x+MISyV7RYCrtZDZEqJEXIl7Y9k3u43HX0t7AqfnJxgae3+VfyeMFL/w2Q/M1kC9RoPN13k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=QDI6i43V; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="QDI6i43V" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-30a3c3014fcso892630a91.3 for ; Thu, 08 May 2025 07:10:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1746713425; x=1747318225; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=4KGvQ+Q6heMHcK36CVSIx+UJlHy7CJTk2MzDhPy5Mfw=; b=QDI6i43Vy/TKdQgzyIHXEngulWNLc8swjDSmohQmH4KiDjobOStg4pwWY8qHJ1I7eK zylfHqArpSfkdSuspiPCpNU71V3KoCdlH2pshWoyS8ekNOw4Z5BMzQcEs1IfWwkLiHRg QbwRm3MOFIbYdgvIVV9H0IYV+SqUOxYhULgjwqOrU8ohWiFYPciihKjthNkjD4rzAWiq 3HDouEl3zLvm42eYixTWR+HXdx78zxDhnbj8LA/UlWfJvbOK/vfwihKaRV9m+Ggavcqr S1M0dAcP1Zi5m64LNAfocQIbYND4Jo9qL/MHB59uT1+p2hP2cjdaA3RN+LoHzu0Po+2d pg9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746713425; x=1747318225; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=4KGvQ+Q6heMHcK36CVSIx+UJlHy7CJTk2MzDhPy5Mfw=; b=A8OKmnQY4gnRLhwP9Fy/h7oBRAsZddNhpI6QXaM+DWkL3jVe65gY9RVlLQYvJIiLrq 5xbyUi3K5kNzfRd3aueIucDsqMUfUQWa1hlhtI5W4VW4eZ2UKLYPgFo+HujbvO35aA6S z8HlGOdvTRvDyePMTTZlqbJuCcxNCSgdDyf7nFwO/MhgUt+/pju5mSVVXAwLsrAUcT3r SYUYVt+4R/mAkEWVW+1jEvUaISEfR8k+7r306paeBFW+24+mZLGmMqFlnhlseXbzFGHL H7Cl+jZmraynCJbxKxSzMSrK5QsdRFLsIowI/7SM62exVTTzjuuc479sOjjXnGqW1D6N OhPw== X-Forwarded-Encrypted: i=1; AJvYcCXpMoCcLAjcNoIwRWO68z68BT7jDT0MmD/gWIjEqNETU09LRYC/lXPJYdwcMWH7ko/J+WU/m43YWs1WMzg=@vger.kernel.org X-Gm-Message-State: AOJu0Yzl9qwVvrLWCT8JmzqWeUiW/TM4La3nejyHy9TR0+txKvFs241/ +LlsUhC8OnpeQIELpRi30U4hc/sT+ERBh5JyxiPuBzsW+zxBW8ubaRNGmI2a8JJ1tIuwgtz+qks xkQ== X-Google-Smtp-Source: AGHT+IEtWJ3nmgNRQEmk4BVb5RHnwtUoCyvuNs/Epm6J06x22EncYKPdDDsnMCNUw8J0F0H1iuL5jdh5x00= X-Received: from pjuu13.prod.google.com ([2002:a17:90b:586d:b0:2fa:1481:81f5]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3d86:b0:2f7:4cce:ae37 with SMTP id 98e67ed59e1d1-30aac1b3f9emr12785433a91.18.1746713424759; Thu, 08 May 2025 07:10:24 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 8 May 2025 07:10:12 -0700 In-Reply-To: <20250508141012.1411952-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250508141012.1411952-1-seanjc@google.com> X-Mailer: git-send-email 2.49.0.1015.ga840276032-goog Message-ID: <20250508141012.1411952-6-seanjc@google.com> Subject: [PATCH v2 5/5] KVM: Use mask of harvested dirty ring entries to coalesce dirty ring resets From: Sean Christopherson To: Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Xu , Yan Zhao , Maxim Levitsky , Sean Christopherson Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use "mask" instead of a dedicated boolean to track whether or not there is at least one to-be-reset entry for the current slot+offset. In the body of the loop, mask is zero only on the first iteration, i.e. !mask is equivalent to first_round. Opportunstically combine the adjacent "if (mask)" statements into a single if-statement. No function change intended. Cc: Peter Xu Cc: Yan Zhao Cc: Maxim Levitsky Signed-off-by: Sean Christopherson Reviewed-by: James Houghton Reviewed-by: Pankaj Gupta --- virt/kvm/dirty_ring.c | 60 +++++++++++++++++++++---------------------- 1 file changed, 29 insertions(+), 31 deletions(-) diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c index a3434be8f00d..934828d729e5 100644 --- a/virt/kvm/dirty_ring.c +++ b/virt/kvm/dirty_ring.c @@ -121,7 +121,6 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_di= rty_ring *ring, u64 cur_offset, next_offset; unsigned long mask =3D 0; struct kvm_dirty_gfn *entry; - bool first_round =3D true; =20 while (likely((*nr_entries_reset) < INT_MAX)) { if (signal_pending(current)) @@ -141,42 +140,42 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_= dirty_ring *ring, ring->reset_index++; (*nr_entries_reset)++; =20 - /* - * While the size of each ring is fixed, it's possible for the - * ring to be constantly re-dirtied/harvested while the reset - * is in-progress (the hard limit exists only to guard against - * wrapping the count into negative space). - */ - if (!first_round) + if (mask) { + /* + * While the size of each ring is fixed, it's possible + * for the ring to be constantly re-dirtied/harvested + * while the reset is in-progress (the hard limit exists + * only to guard against the count becoming negative). + */ cond_resched(); =20 - /* - * Try to coalesce the reset operations when the guest is - * scanning pages in the same slot. - */ - if (!first_round && next_slot =3D=3D cur_slot) { - s64 delta =3D next_offset - cur_offset; + /* + * Try to coalesce the reset operations when the guest + * is scanning pages in the same slot. + */ + if (next_slot =3D=3D cur_slot) { + s64 delta =3D next_offset - cur_offset; =20 - if (delta >=3D 0 && delta < BITS_PER_LONG) { - mask |=3D 1ull << delta; - continue; - } + if (delta >=3D 0 && delta < BITS_PER_LONG) { + mask |=3D 1ull << delta; + continue; + } =20 - /* Backwards visit, careful about overflows! */ - if (delta > -BITS_PER_LONG && delta < 0 && - (mask << -delta >> -delta) =3D=3D mask) { - cur_offset =3D next_offset; - mask =3D (mask << -delta) | 1; - continue; + /* Backwards visit, careful about overflows! */ + if (delta > -BITS_PER_LONG && delta < 0 && + (mask << -delta >> -delta) =3D=3D mask) { + cur_offset =3D next_offset; + mask =3D (mask << -delta) | 1; + continue; + } } - } =20 - /* - * Reset the slot for all the harvested entries that have been - * gathered, but not yet fully processed. - */ - if (mask) + /* + * Reset the slot for all the harvested entries that + * have been gathered, but not yet fully processed. + */ kvm_reset_dirty_gfn(kvm, cur_slot, cur_offset, mask); + } =20 /* * The current slot was reset or this is the first harvested @@ -185,7 +184,6 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_di= rty_ring *ring, cur_slot =3D next_slot; cur_offset =3D next_offset; mask =3D 1; - first_round =3D false; } =20 /* --=20 2.49.0.1015.ga840276032-goog