From nobody Sun Mar 22 14:17:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1773962116; cv=none; d=zohomail.com; s=zohoarc; b=LOp0n7rcgihPEZ2S4vpojFRV/p2nshTzmRXYUcGdOKbd6hM9Dw68Ft5PWZiN4mZyWzTVRZce4ARxNGQI0i3OqvVrV0BKlboUS09jRbGaO1BVZRJS3t4DmvZ7mmuAzo3s8uTrC07GGpse6V3/2FC76Ci4pmo+jcHPPXx8XZhm/yE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773962116; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fZkKChfolbOCjcYtzW/j8UchHmlgfocdKxXn+D4VI3U=; b=EJvxfY6mieM0pjvCq4jrotRHX9TPPkTgUStUKKHAPAWYbzOdP3SNryb4EWmslAz1Fagw2B/iFRZSRPTnnOyBOcRykoL6zVORfHbb2MGu/MHK2HtcgXikUfsgbmIfM/Ghx79JcfzeoGBqYRRtE8hnd635TZH4CzAjepsX7LDY4Oo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773962116625860.2636064921867; Thu, 19 Mar 2026 16:15:16 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w3MYH-0001Lw-HO; Thu, 19 Mar 2026 19:13:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYF-0001Ks-Et for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYC-0001Oc-Jm for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:15 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-232-sK9PT_VeMEys0EfKrufeTg-1; Thu, 19 Mar 2026 19:13:10 -0400 Received: by mail-qt1-f199.google.com with SMTP id d75a77b69052e-50b274f94f8so85628311cf.1 for ; Thu, 19 Mar 2026 16:13:10 -0700 (PDT) Received: from x1.local ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-50b36e5bee3sm6717161cf.21.2026.03.19.16.13.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Mar 2026 16:13:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773961991; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fZkKChfolbOCjcYtzW/j8UchHmlgfocdKxXn+D4VI3U=; b=UkZy+34WyN6XK472kkWSqo1ha0wbSE0Vdn5hG3f0nqy45PRuA5dgWLo4UUk3Cy72uo8x8q R6O8z2t/gr9BbWZ+9mfXiBEd8lGVDHWNZuNCgf/KpAVc4TyOKB4xpjXUYrQ+z/zyM2s/ib L9W8fR6OMB9wEvpjGwNRaZEPZnXUR7o= X-MC-Unique: sK9PT_VeMEys0EfKrufeTg-1 X-Mimecast-MFC-AGG-ID: sK9PT_VeMEys0EfKrufeTg_1773961990 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1773961989; x=1774566789; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fZkKChfolbOCjcYtzW/j8UchHmlgfocdKxXn+D4VI3U=; b=Xx1Rrqz/YcXCXGowT42t2OEfy2TjbgUYVSz1xFLyl/NyO4w1qYrj0yQUDNYtr9lwT8 siNAYGz2Is5SVip6wW2BaplPq3JxBbAKheh967JLrz5BxTIhY7bF1IIHgVyXc2COY6TL JOncZI5W7/TzzMxPiY1pCyA7XvdqfrRVK/j8zUFqaYeg4qAa/ksXOZM6dyrfGIRU24qd jRqg8aXT/4SMoHano5gSj+sUMVw3+sTC5FEyxxDsVZroWn2ijd3JjRTUXTOJ4FORL1Bg JpV7IxYLFkzVob1TiuhcZtV4dvfVsqvUn7zZQLNiLLOEh7HpkMlNCm0Yp26bBRjv4HHS vpBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773961989; x=1774566789; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=fZkKChfolbOCjcYtzW/j8UchHmlgfocdKxXn+D4VI3U=; b=cBdw6Vdue4lGy19Kklq2TFBr2+1o9mctuxMm9uKSfs/kS+AzNqZFiSMv1zE727nHEe IPK9JdvIZ2ZL50wC1fcSBLwqrHO04R9NTfkZ08BBf075QfoimFa/6i5akk8zx4JeRjOX cqC9ObnHB1dETa5s8eyKV+jh7Kt15CrXo94JqjM0XPLHlg7FgKFBh4Q1IDsTJM0YS5+P df0vUMo679DNwuqLWvp96mnFzTC6dfj0n3OOWX3EGkH6BMtqaaK/0iGLiV0Q2GrjJTH4 hllGv0ERVQQqOr0SsjoiYtJxA9/HVqmUoHapeUQkzOGv+vvdBbgsVwfnYNH+2f9IVY0V hqLQ== X-Gm-Message-State: AOJu0YwhNPn7ZinkmEO7TZ4sbDf2i6U+3i/jcbgTq1Wnrn5T7vw/0lLJ XbJTIh/0TVX44R1NgYzmWdkuU9q12OBCvneC9JyWqUL/n0T7ETjDNC4WQyESC/m4TBflRbEPpUM LcRKeCu72I13Pvna4D8bbFe/K9dsKuWt5uAP/0d0yHE61GBIJO6Wfln311iEtM/4DGOo2o2uXel r83psPNWVUHVOyPRH9L56A7F2zwaIHeQhZv36N1A== X-Gm-Gg: ATEYQzysoWyQacYRUPFdG5ZiujtGVwcBD13NtQspntUvOYs7SV7Aop8roN9BMD8QdKB nhTcuSxmmjbOZSE1c5pyvgxUVHcrzErlc0FxnAKYBtM5Fp1jYab9X+Uq9vcI7QKIxTvoy1PrQ3k 0hIRCjJ8sm59JiTlik7bU9un2Hx8U/df8nG+Sl53gs7tddBe0P/eak7AbiYFUwXNCLo6sW7INxe 8pYK83+Rfgr8m1zUnqEGaYk+ZHzyxYN3d2FE/8YG1j77nCaJuozGAv+OEUappO5mmrGxfm+uOLb b0fCqRU1uq3VK4yBJM3Hea+4XpbbW3ScFdzvh2JRZJg89mLtG4gnodobSc+oopGvEKCLM7JoyXj WyJkpSYh7ATk1RA== X-Received: by 2002:a05:622a:5c17:b0:50b:36b6:d84d with SMTP id d75a77b69052e-50b3715666fmr17221191cf.26.1773961989512; Thu, 19 Mar 2026 16:13:09 -0700 (PDT) X-Received: by 2002:a05:622a:5c17:b0:50b:36b6:d84d with SMTP id d75a77b69052e-50b3715666fmr17220691cf.26.1773961988965; Thu, 19 Mar 2026 16:13:08 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juraj Marcin , Kirti Wankhede , "Maciej S . Szmigiero" , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Joao Martins , Alex Williamson , Yishai Hadas , Fabiano Rosas , Pranav Tyagi , peterx@redhat.com, Zhiyi Guo , Markus Armbruster , Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , qemu-stable@nongnu.org Subject: [PATCH RFC 01/12] migration: Fix low possibility downtime violation Date: Thu, 19 Mar 2026 19:12:51 -0400 Message-ID: <20260319231302.123135-2-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260319231302.123135-1-peterx@redhat.com> References: <20260319231302.123135-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.819, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.903, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1773962119562154100 Content-Type: text/plain; charset="utf-8" When QEMU queried the estimated version of pending data and thinks it's ready to converge, it'll send another accurate query to make sure of it. It is needed to make sure we collect the latest reports and that equation still holds true. However we missed one tiny little difference here on "<" v.s. "<=3D" when comparing pending_size (A) to threshold_size (B).. QEMU src only re-query if A --- migration/migration.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 5c9aaa6e58..dfc60372cf 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3242,7 +3242,7 @@ static MigIterateState migration_iteration_run(Migrat= ionState *s) * postcopy started, so ESTIMATE should always match with EXACT * during postcopy phase. */ - if (pending_size < s->threshold_size) { + if (pending_size <=3D s->threshold_size) { qemu_savevm_state_pending_exact(&must_precopy, &can_postcopy); pending_size =3D must_precopy + can_postcopy; trace_migrate_pending_exact(pending_size, must_precopy, --=20 2.50.1 From nobody Sun Mar 22 14:17:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1773962140; cv=none; d=zohomail.com; s=zohoarc; b=NAy2IB98zr5OATzEtTV5vZ9891oepZHQYyUxXFMi7djtlSiheLpbCKORTsBdVGo2hx4Nday2Cs+oDguKVhzuRBoU4QnAxK+HzUmmccGLKLfM+Ij7FzkJ/OrrnMKvruLNWL0kPK4WMM1T9obTVjqD1puu3FoqP9gEQFOMrA3iVao= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773962140; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=i/IRRC3VrN6zBxxQuScl2KCrKe2VxypUJ2dujj/8UDw=; b=UpQoFZ/dg6Pfg1SeT4CtX0CuZ3C46J0uujcNO7kUrAhWzamj5zy1J4gROlkCudYKbpISXTsyfz06jUBMUiAhECZPP0dzjfHqdoIu+XhViQyEb+F83tFIz3gkWtxjElKTdzvKzycJkcivSPgH1/rORDzuE4bgtAYHNCE5lkk7Tjg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773962140582371.3480142869789; Thu, 19 Mar 2026 16:15:40 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w3MYI-0001MS-R9; Thu, 19 Mar 2026 19:13:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYG-0001LJ-Sp for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYE-0001Op-WE for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:16 -0400 Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-42-b1geWkosNPWp9Kv8OJdr9A-1; Thu, 19 Mar 2026 19:13:12 -0400 Received: by mail-qt1-f200.google.com with SMTP id d75a77b69052e-50b323c43fdso41141131cf.2 for ; Thu, 19 Mar 2026 16:13:12 -0700 (PDT) Received: from x1.local ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-50b36e5bee3sm6717161cf.21.2026.03.19.16.13.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Mar 2026 16:13:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773961994; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i/IRRC3VrN6zBxxQuScl2KCrKe2VxypUJ2dujj/8UDw=; b=BuKkQIZqfS/7gWb6Fqx0qIiC2QqmHVYNSMucMIFxXNy0WRYCLcZB7+pmsDBZ2Nu23kxvha NoxVN6agAOlG/nQaNg4E8aMLIkVYYcNTb3LjwmxxWU8rtnm2G8lpJyjj3GYhcN46od/Xjg ceTVZILsaR93ZH3D//Eov9F0CXbT5z8= X-MC-Unique: b1geWkosNPWp9Kv8OJdr9A-1 X-Mimecast-MFC-AGG-ID: b1geWkosNPWp9Kv8OJdr9A_1773961992 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1773961992; x=1774566792; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=i/IRRC3VrN6zBxxQuScl2KCrKe2VxypUJ2dujj/8UDw=; b=l4iKYsi5U0IePAfnc3h4Ads/89I9t9RRH6PDI+R9A5OW4EHNqGsZWYSrXCwPVX1P1a zETsnWyygNXyNJjJgT41pQLLQcUnotzZ6AF4t3yMnFeMUw9/s1zTXF4qyxxt4vRajY7D HoTIjx+YGUsrVp9Wh11nvOxIautR3EMvZaq8sRibtxjzbAsUPFr2wZomL/crLifYxRSr TY3eGewMlUAkFo9qBw4qCFaCGPFmcnAZmiBkcmcyBuk9xlxJBPlbZYzLjzxLPJuI4u/I m9LCa8WxDWPG9J7Beuqi7HOIjn2wC0nW5z8B/ZLK4ywlZy4bLBkBYQqf+bRExOIxQwJi KEMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773961992; x=1774566792; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=i/IRRC3VrN6zBxxQuScl2KCrKe2VxypUJ2dujj/8UDw=; b=E6pCpmWx427sWeOoQZrHcd/3ep+1rhPIePncFwbvzBf1MLDdxjr7hr5j8fQN/iXSnU S1tYW3eQiSCX5fqcfPZ7ibvyIGUd+4PLFDnXnfFAU5nC25VBH7JO9/XyuGWS1xE8MfEV tgCVDeBQLkjtGs4eeaoZuYnG5/s4CNEWq0vaLyFhlTTdFgPzN0eFRe0sHl+C8uwhvSYd 2hb7bHl1veBITDTIhNJnD32fHnl7zhYse+6l3ENgF5pKCqMLjAnEbetm/pC/0cSKAYYJ 9UWVYoDqCUHg/tvvcxgzhM2K+/Ui0mCnOc0+T8AOw8hTvja+e66o36X6AKFsDenxOERa J0KQ== X-Gm-Message-State: AOJu0Yy5v2KxagFxFLZ3yvC+zXX3kl8X1AhUxbkFbErwobCahKg51EGa u4dP4TfQt0LNY8oFu8SRHAA3nlB9YkgEBP2Yfr82ofSZocotGk1BfU51X3aPsefdBXsLk3VoFeB ZRX2XBoMLVKe+qV3JmOcwz5d/iKBXN8iNm24aP7X3KrbU7B9PQyyekdyIFM0gOi/NRGPWIpANOn NWto2/wcS0M2FEquVSwRzcoOpWbKXDqRi/UqDqig== X-Gm-Gg: ATEYQzz05FB60KmJJv8SYoZcqiCB7JL57/UIMnsrXFWj84v3u5lgUplBLQh4m0zkPSi nbVxU/6oOOzvZ/cvJzNxwJbIIiWye8z2/7EksMSzajNd1HEJUcjW3NXlOe1iO7kd+iUFG4094UT ltWxKSgI86aZyyGuPLKrRANfXBuRr4QTNTU4+PqnP1QghEA3p9gz197485IVIkZoVxyt9a1tAXm kgEXpmjlsuuz3oGJHjRCqt90A+q7DUeeNcNdudVqdZBFYOeXjmtM8BI/OnfXn5NCnOafgP1cgwE D+TT1C9xlCH6X2b5xjN/oOLkhr4TxhKA+GoOuIibTdtZaguAPEAP14wW+q/JZj1jE2g1srk9sDL U4BpgaDfleu8e3w== X-Received: by 2002:ac8:7c43:0:b0:509:1339:3dbb with SMTP id d75a77b69052e-50b3751cd2fmr16096811cf.48.1773961991822; Thu, 19 Mar 2026 16:13:11 -0700 (PDT) X-Received: by 2002:ac8:7c43:0:b0:509:1339:3dbb with SMTP id d75a77b69052e-50b3751cd2fmr16096211cf.48.1773961991092; Thu, 19 Mar 2026 16:13:11 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juraj Marcin , Kirti Wankhede , "Maciej S . Szmigiero" , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Joao Martins , Alex Williamson , Yishai Hadas , Fabiano Rosas , Pranav Tyagi , peterx@redhat.com, Zhiyi Guo , Markus Armbruster , Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Libvirt Mailing List Subject: [PATCH RFC 02/12] migration/qapi: Rename MigrationStats to MigrationRAMStats Date: Thu, 19 Mar 2026 19:12:52 -0400 Message-ID: <20260319231302.123135-3-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260319231302.123135-1-peterx@redhat.com> References: <20260319231302.123135-1-peterx@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.819, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.903, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1773962143900154100 This stats is only about RAM, so make it accurate. This paves way for statistics for all devices. Note that even if this is part of qapi/, this should not be part of ABI of at least query-migrate, because the structure is not changed, and this stats is always reported only under the "ram" section. Cc: Daniel P. Berrang=C3=A9 Cc: Markus Armbruster Cc: Libvirt Mailing List Signed-off-by: Peter Xu Acked-by: Markus Armbruster --- docs/about/removed-features.rst | 2 +- qapi/migration.json | 8 ++++---- migration/migration-stats.h | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.= rst index 6f4447993c..7c4f4325f7 100644 --- a/docs/about/removed-features.rst +++ b/docs/about/removed-features.rst @@ -699,7 +699,7 @@ was superseded by ``sections``. ``query-migrate`` return value member ``skipped`` (removed in 9.1) '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' =20 -Member ``skipped`` of the ``MigrationStats`` struct hasn't been used +Member ``skipped`` of the ``MigrationRAMStats`` struct hasn't been used for more than 10 years. Removed with no replacement. =20 ``migrate`` command option ``inc`` (removed in 9.1) diff --git a/qapi/migration.json b/qapi/migration.json index 7134d4ce47..cfc6ccee26 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -12,7 +12,7 @@ { 'include': 'sockets.json' } =20 ## -# @MigrationStats: +# @MigrationRAMStats: # # Detailed migration status. # @@ -64,7 +64,7 @@ # # Since: 0.14 ## -{ 'struct': 'MigrationStats', +{ 'struct': 'MigrationRAMStats', 'data': {'transferred': 'int', 'remaining': 'int', 'total': 'int' , 'duplicate': 'int', 'normal': 'int', @@ -209,7 +209,7 @@ # If this field is not returned, no migration process has been # initiated # -# @ram: `MigrationStats` containing detailed migration status, only +# @ram: `MigrationRAMStats` containing detailed migration status, only # returned if status is 'active' or 'completed'(since 1.2) # # @xbzrle-cache: `XBZRLECacheStats` containing detailed XBZRLE @@ -309,7 +309,7 @@ # Since: 0.14 ## { 'struct': 'MigrationInfo', - 'data': {'*status': 'MigrationStatus', '*ram': 'MigrationStats', + 'data': {'*status': 'MigrationStatus', '*ram': 'MigrationRAMStats', '*vfio': 'VfioStats', '*xbzrle-cache': 'XBZRLECacheStats', '*total-time': 'int', diff --git a/migration/migration-stats.h b/migration/migration-stats.h index c0f50144c9..1153520f7a 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -27,7 +27,7 @@ =20 /* * These are the ram migration statistic counters. It is loosely - * based on MigrationStats. + * based on MigrationRAMStats. */ typedef struct { /* --=20 2.50.1 From nobody Sun Mar 22 14:17:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1773962088; cv=none; d=zohomail.com; s=zohoarc; b=RbPS4tPky4J1NybCgkOQpN0l6kvqPUvKs/ZpKMHrA86ZcQF5YtaWeSKTBm5bsUl0sXEgIRUlZ/3ygphJRzE/kKT0QvrtKBPl9kcdtHECZHQ+BRSos60KjEWyeF3JWLlkmcrN7/c+Ied+wzMaYcx7ToNEpWQfFQ42Og0VOxETqiY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773962088; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=7CYpOBSw137WiL2qGC9SqFhoCArm6dY0CTE+aFxRoEQ=; b=UVw0Ps7yMYJpycwBaXwDexqL3YyQB45wwOkmHNfs3wdTMHTxsU6lNbiQ7taWjlC8O74jGpUbgLzrz20oBXd6VjXem5N5ln6ijAcHA9ksjjsDfInBcINzur/aC+o6tbfNkLGFCNRp3aUCND/uiADfNTfsMnyAFViOVUmdmCrt920= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773962088492626.8947072508471; Thu, 19 Mar 2026 16:14:48 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w3MYJ-0001Mk-Bo; Thu, 19 Mar 2026 19:13:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYI-0001M0-4M for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYG-0001PH-Gr for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:17 -0400 Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-630-hBW159fKNMyhENBhL-PmCw-1; Thu, 19 Mar 2026 19:13:15 -0400 Received: by mail-qt1-f200.google.com with SMTP id d75a77b69052e-50911c94db1so83482231cf.0 for ; Thu, 19 Mar 2026 16:13:15 -0700 (PDT) Received: from x1.local ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-50b36e5bee3sm6717161cf.21.2026.03.19.16.13.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Mar 2026 16:13:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773961996; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7CYpOBSw137WiL2qGC9SqFhoCArm6dY0CTE+aFxRoEQ=; b=gT05cvnxZqavhW7fjHozwfdVGCoFfl40SH/W/kQsPan91dhzhGKds+XtcmKDZEgHjFDiVT ihdyb4d+WMPK6W7k/h0wUHbkkVOVr5k3E628H11cgpe1ZevG3Ame8q7NyPx5Qn+/sqS8SS LmpO3lPR95X/HBQj5H3bDLnJNOPB4Mk= X-MC-Unique: hBW159fKNMyhENBhL-PmCw-1 X-Mimecast-MFC-AGG-ID: hBW159fKNMyhENBhL-PmCw_1773961994 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1773961994; x=1774566794; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7CYpOBSw137WiL2qGC9SqFhoCArm6dY0CTE+aFxRoEQ=; b=EeW4pNeEQvgQPZDcfuf2K7XyxURrO2Ajxz6N07+DiOSx7Ik4ZJB4FFBm3MdX4rIBWK 8Fx+cp8ydUVfGJEph2rq3bqG+5WD0gFvL2FB66/DuSaUgwULRCOx8QjvAldnx5g2ZPPh qFj6ya5NPDVNlo19ITSPtmdJ4CC5+bUmxC+1ApaUJ7nc4lTv6ooysUXI1kV69l/ver/E GCqKmDsQBy7n50CyMqHvCRX6JsZ1XghlEgDVClw5anrAuXZBBcMp1gZXkVcYzdbgaq0m 9aEkIiTcU+GBCvFSKhv6M02PgwSpDesVzhBDx41EV13Gq9Bl1GgMq6pz6lc4gl/hLXSF eX4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773961994; x=1774566794; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=7CYpOBSw137WiL2qGC9SqFhoCArm6dY0CTE+aFxRoEQ=; b=oB/uj3jaCryzJCQCr4aFbYsKXzDceHEws5cOq1bOwwe5Eq4zcnMQUaR8jtECvpIAiN F9L2YFfwMnINFjRj+LQNKPvDadmx7LxF6FwnF5Oq95iI8N3SKtSH/Al/YXPGuyBQ5DS2 GnmDp6QPlWdUeZ2L65tAIm5ySpu6tV+7EySG3ailqHR0vYGhWSafXENaQl11TbUt18zD VvjNuY2EZ24DZz/K4vszuqYZN8zk/gPnw1KtJIttlPJv9pwP2w20HDTU2I9fpI5jFOp6 Y3dxmT9wEwncdgEuMbkxTxi2PQ9Y2HJItasSdp6vXHam49QSMX/FZSChgeZwsBlkn52J tSRg== X-Gm-Message-State: AOJu0YyyEVlfwoVyXt9QYDPajxk545yB04ENey5zumt3rYgVjMnRrkNR +xVS4/hhEZHw2myTRA67LbrNXHcjrsu0whtHUBpsj4lPvvW1emjzB8D8Qk00xKgazZNnWqpTNf+ 8x/2J1YTA/L/QhL/4vFJ/3DwLbNiHz0pjA0sAvCBIBkHEfaLeL0QNgy6JDiMHZ4sRZTEMeAL+ip v4c/bLebgsy0dgFu4G7bYM8MFo/ZyUvAUB89OylQ== X-Gm-Gg: ATEYQzx3PyViHPJ2HvoPYUdcum2veVHnfbtZhH/k2KhIdcZvDfFGj9hvA83EFNVbhwz vZlJ//GXZt3QgWOulzIbrVPLDuAOCoecJ0PqyL1p88SW1+khgA3RNzPdh0/qyhZmbzcqrQUgCz7 y9X5ikKEtnIDnXEgqFw0kNuZEHiyL4Ukj8o66sA1KootcfjoYl5K2AbxRtqDKilecKshfN3zJQp PAPdxBaATvjM7yCt/GFtw5SUY3b65W9dSmrl/UOcjl465/9aRncN5My811S+pqydLIgZhndJYwA 9v0pmPtaKOGG95tQgPXkwzginU5B83hiSOaZmhAj4n3qzMugwEnw0dNiYy4jo9Bc+CKHPyPeVNY Ydiw9Q/e6k2W/Kg== X-Received: by 2002:a05:622a:118e:b0:509:3fc5:a847 with SMTP id d75a77b69052e-50b3738ea69mr15533601cf.12.1773961994183; Thu, 19 Mar 2026 16:13:14 -0700 (PDT) X-Received: by 2002:a05:622a:118e:b0:509:3fc5:a847 with SMTP id d75a77b69052e-50b3738ea69mr15533181cf.12.1773961993612; Thu, 19 Mar 2026 16:13:13 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juraj Marcin , Kirti Wankhede , "Maciej S . Szmigiero" , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Joao Martins , Alex Williamson , Yishai Hadas , Fabiano Rosas , Pranav Tyagi , peterx@redhat.com, Zhiyi Guo , Markus Armbruster , Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PATCH RFC 03/12] vfio/migration: Throttle vfio_save_block() on data size to read Date: Thu, 19 Mar 2026 19:12:53 -0400 Message-ID: <20260319231302.123135-4-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260319231302.123135-1-peterx@redhat.com> References: <20260319231302.123135-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.819, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.903, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1773962091479154100 Content-Type: text/plain; charset="utf-8" During precopy phase, VFIO maintains two counters for init/dirty data tracking for query estimations. VFIO fetches data during precopy by reading from the VFIO fd, after fetching it'll deduct the read size. Here since the fd's size can dynamically change, I think it means VFIO may read more than what it "thought" were there for fetching. I highly suspect it's also relevant to a weird case in the function of vfio_update_estimated_pending_data(), where when VFIO reads 0 from the FD it will _reset_ the two counters, instead of asserting both of them being zeros, which looks pretty hackish. Just guarantee it from userspace level that VFIO won't read more than what it expects for now. Signed-off-by: Peter Xu --- hw/vfio/migration.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 83327b6573..851ea783f3 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -357,12 +357,18 @@ static int vfio_query_precopy_size(VFIOMigration *mig= ration) } =20 /* Returns the size of saved data on success and -errno on error */ -static ssize_t vfio_save_block(QEMUFile *f, VFIOMigration *migration) +static ssize_t vfio_save_block(QEMUFile *f, VFIOMigration *migration, + bool precopy) { - ssize_t data_size; + ssize_t data_size =3D migration->data_buffer_size; + + if (precopy) { + /* Limit the buffer size to make sure cached stats don't overflow = */ + data_size =3D MIN(data_size, migration->precopy_init_size + + migration->precopy_dirty_size); + } =20 - data_size =3D read(migration->data_fd, migration->data_buffer, - migration->data_buffer_size); + data_size =3D read(migration->data_fd, migration->data_buffer, data_si= ze); if (data_size < 0) { /* * Pre-copy emptied all the device state for now. For more informa= tion, @@ -623,7 +629,7 @@ static int vfio_save_iterate(QEMUFile *f, void *opaque) migration->event_save_iterate_started =3D true; } =20 - data_size =3D vfio_save_block(f, migration); + data_size =3D vfio_save_block(f, migration, true); if (data_size < 0) { return data_size; } @@ -667,7 +673,7 @@ static int vfio_save_complete_precopy(QEMUFile *f, void= *opaque) } =20 do { - data_size =3D vfio_save_block(f, vbasedev->migration); + data_size =3D vfio_save_block(f, vbasedev->migration, false); if (data_size < 0) { return data_size; } --=20 2.50.1 From nobody Sun Mar 22 14:17:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1773962121; cv=none; d=zohomail.com; s=zohoarc; b=BdmfjqoZiTKyKzB7kdNI9ydoqFHG3XhjG69PRHSjbGZhV/aQZIToW2IYrREtrbbxvF/JxBZmBkSkW+oOmQI34JUQqhOLdsGCEptnu8cTo4Ej0niNYQsHHCrGu9vGA92RAIHyi8kpdjeCfYh9aZ4Y8lPW2+nurrPsnBSNsPPgHhQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773962121; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=V6xO5KlK8X7T9dIvIeNyHDMz8xdJ4Fx4esbGShwNklI=; b=ldUzx06HCB353sY70IBMM3oXW0F6RjLSBqdO4iR1Hr0bLnve/PKXk8lUByILpfKjt+sUHSFlDfpBXn9cJAZkJ9f5gQOMHxlRPa73glsl+K39MUHtzEMCP1BisKojy7gzzn2CUy0g5RDNVA5uNISOmP/+1N0/P/XrwA6nS7h8yX4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773962121076136.58109482181158; Thu, 19 Mar 2026 16:15:21 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w3MYM-0001NI-6E; Thu, 19 Mar 2026 19:13:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYL-0001N8-1R for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYJ-0001Pw-Ar for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:20 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-100-Wow8rfqkMNmvIh2ID-IQqg-1; Thu, 19 Mar 2026 19:13:17 -0400 Received: by mail-qt1-f199.google.com with SMTP id d75a77b69052e-5091f12ed3bso2799461cf.3 for ; Thu, 19 Mar 2026 16:13:17 -0700 (PDT) Received: from x1.local ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-50b36e5bee3sm6717161cf.21.2026.03.19.16.13.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Mar 2026 16:13:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773961998; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=V6xO5KlK8X7T9dIvIeNyHDMz8xdJ4Fx4esbGShwNklI=; b=ERHvpJO6R8gH/IGU3syxVwIV6V9dXEcQK0nUpahXXZEamHGPud2ZX5C2OgZeobClcuvHzv EC27HiZ3FmfXt13imxsPEbstgWHbl8oiUJiUM0ZQmGmEART4zKDtRnPBYfkdkCO/jnbOaD bn1uZfcVv4apvddVNuYv24uYC1GM/Xg= X-MC-Unique: Wow8rfqkMNmvIh2ID-IQqg-1 X-Mimecast-MFC-AGG-ID: Wow8rfqkMNmvIh2ID-IQqg_1773961997 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1773961996; x=1774566796; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=V6xO5KlK8X7T9dIvIeNyHDMz8xdJ4Fx4esbGShwNklI=; b=T8VWZgPP3Jg2xGqr64pvjvSgRHN8e8SO8d+bvKahPMogsFlAxyQoLp5EVrekLlxate z482gGOlMO72tCxu9EzlDYvAExu1vEu+V4KbuC7V4JqxaoYBah2WciyZ1R1ZVJACd4ra z3s4fgM4TMdPvEKqnEuj0ibotTBN4fmIKYxuALy6kCL8DQtFKrxJMdQM66gRi1QluuT3 GTEmXVdnwvZL3sCUbc2RpOui85pXxjKcrHINICPU1Y1LZJeBCTLiSVbW+/Gd6zQaCgfu fwiKxQ9rr3FKocqnhrxgnEK9Xi5RXHJrtYDNPK5Z0EAW7KoXtehFcmEZPjnC73wziT0F gZeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773961996; x=1774566796; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=V6xO5KlK8X7T9dIvIeNyHDMz8xdJ4Fx4esbGShwNklI=; b=AUkWDUukuqidwwsy9pJ+GhLmkn6V2oewQsDBHAsI27ZhCQI2gl+VWGG5nJyXT+2fo+ BJuy1vPmZqBH3uG+oiyHaLNiqm6+m1oB/jQl8mr5+bSLXI9F6gavnl7wRUNEMbrqh1+5 Ju8fqgWWxV9HqVio/JM4MeU4iA7bbysIFRYHAPk4L+kkF9yELWnS/7Igh+ipdFhu7WDK xNKQXr9ippOqRHejW5vzApDhV8tRA6iVT4HYJIMyjoiYJuCSWcfCXnFk/uoVKc3gTD8F be3ZRDJY/LPTV2XverPmfU4zhX3zeIKhlbOWJYHmGFI4vy8QunZ376emAkzMEMHQ/88U kc5w== X-Gm-Message-State: AOJu0YyHd86JmxBwDDhL0a0UJkyPr8T9PTVNNu+Ukbsk/wtLjgdKvKWk JQI3OKHfbd7GZYsRi/5D5SwZYnEJMikCFTUkAQ+xRirAkDybEMEwg69L2pE6bqEmyFCa8uOOTbJ Y0BGO7B572qIAs5clYJ9ZckX67M83ZldoZep4Y70QLFl8h8ZTGp2w8bGeCVqzBD3Z2k+fT0foJf yPt7uqrCXIYCamabYuqVnagzJRKYQgm4+7lvYHxg== X-Gm-Gg: ATEYQzwl/Od9ZLUZFnNev9UhmkZdIypqrl3nguW4/XeDZzWIp3xsI4U8pvzvtb3ZAPO g3TdsbPdUdgKsCZItjWfewYGpnQrle15H4zRZ4qWRKRjVgLkmdDq+tXspQHZolv1jLPLCpTksje eVAC0m2mJ19n/UcnR75n+ChaG0JBWw/AOiOkGNyEIHYUo0LIh45n/Ig9xxO6WT/87DBYXNS9OD4 x1SKJbQDu8+KcRUBXoJDvsMZDt1BUi893VrZnbD//kRwI+VdyWEE32P83hLmgd8V+arIzGS9Czv TjgYmI6WYAVBBxfTj8UQ4yRXZM9ujTAiGqjyGZQqZAEVSBvE+z2zUFQ+qiAl3/U3u5jJJV64bri xPxe9h4lf9yuH1g== X-Received: by 2002:a05:622a:178f:b0:509:bdb:1539 with SMTP id d75a77b69052e-50b3744891emr17086981cf.23.1773961996385; Thu, 19 Mar 2026 16:13:16 -0700 (PDT) X-Received: by 2002:a05:622a:178f:b0:509:bdb:1539 with SMTP id d75a77b69052e-50b3744891emr17086521cf.23.1773961995839; Thu, 19 Mar 2026 16:13:15 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juraj Marcin , Kirti Wankhede , "Maciej S . Szmigiero" , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Joao Martins , Alex Williamson , Yishai Hadas , Fabiano Rosas , Pranav Tyagi , peterx@redhat.com, Zhiyi Guo , Markus Armbruster , Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PATCH RFC 04/12] vfio/migration: Cache stop size in VFIOMigration Date: Thu, 19 Mar 2026 19:12:54 -0400 Message-ID: <20260319231302.123135-5-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260319231302.123135-1-peterx@redhat.com> References: <20260319231302.123135-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.819, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.903, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1773962123376158501 Content-Type: text/plain; charset="utf-8" Add a field to cache stop size. Note that there's an initial value change in vfio_save_setup for the stop size default, but it shouldn't matter if it is followed with a math of MIN() against VFIO_MIG_DEFAULT_DATA_BUFFER_SIZE. Signed-off-by: Peter Xu --- hw/vfio/vfio-migration-internal.h | 1 + hw/vfio/migration.c | 43 +++++++++++++++++-------------- 2 files changed, 24 insertions(+), 20 deletions(-) diff --git a/hw/vfio/vfio-migration-internal.h b/hw/vfio/vfio-migration-int= ernal.h index 814fbd9eba..08df32c055 100644 --- a/hw/vfio/vfio-migration-internal.h +++ b/hw/vfio/vfio-migration-internal.h @@ -47,6 +47,7 @@ typedef struct VFIOMigration { uint64_t mig_flags; uint64_t precopy_init_size; uint64_t precopy_dirty_size; + uint64_t stopcopy_size; bool multifd_transfer; VFIOMultifd *multifd; bool initial_data_sent; diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 851ea783f3..827d3ded63 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -41,6 +41,12 @@ */ #define VFIO_MIG_DEFAULT_DATA_BUFFER_SIZE (1 * MiB) =20 +/* + * Migration size of VFIO devices can be as little as a few KBs or as big = as + * many GBs. This value should be big enough to cover the worst case. + */ +#define VFIO_MIG_STOP_COPY_SIZE (100 * GiB) + static unsigned long bytes_transferred; =20 static const char *mig_state_to_str(enum vfio_device_mig_state state) @@ -314,8 +320,7 @@ static void vfio_migration_cleanup(VFIODevice *vbasedev) migration->data_fd =3D -1; } =20 -static int vfio_query_stop_copy_size(VFIODevice *vbasedev, - uint64_t *stop_copy_size) +static int vfio_query_stop_copy_size(VFIODevice *vbasedev) { uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature) + sizeof(struct vfio_device_feature_mig_data_s= ize), @@ -323,16 +328,22 @@ static int vfio_query_stop_copy_size(VFIODevice *vbas= edev, struct vfio_device_feature *feature =3D (struct vfio_device_feature *)= buf; struct vfio_device_feature_mig_data_size *mig_data_size =3D (struct vfio_device_feature_mig_data_size *)feature->data; + VFIOMigration *migration =3D vbasedev->migration; =20 feature->argsz =3D sizeof(buf); feature->flags =3D VFIO_DEVICE_FEATURE_GET | VFIO_DEVICE_FEATURE_MIG_DATA_SIZE; =20 if (ioctl(vbasedev->fd, VFIO_DEVICE_FEATURE, feature)) { + /* + * If getting pending migration size fails, VFIO_MIG_STOP_COPY_SIZE + * is reported so downtime limit won't be violated. + */ + migration->stopcopy_size =3D VFIO_MIG_STOP_COPY_SIZE; return -errno; } =20 - *stop_copy_size =3D mig_data_size->stop_copy_length; + migration->stopcopy_size =3D mig_data_size->stop_copy_length; =20 return 0; } @@ -415,6 +426,9 @@ static void vfio_update_estimated_pending_data(VFIOMigr= ation *migration, return; } =20 + /* The total size remaining requires separate accounting */ + migration->stopcopy_size -=3D data_size; + if (migration->precopy_init_size) { uint64_t init_size =3D MIN(migration->precopy_init_size, data_size= ); =20 @@ -469,7 +483,6 @@ static int vfio_save_setup(QEMUFile *f, void *opaque, E= rror **errp) { VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; - uint64_t stop_copy_size =3D VFIO_MIG_DEFAULT_DATA_BUFFER_SIZE; int ret; =20 if (!vfio_multifd_setup(vbasedev, false, errp)) { @@ -478,9 +491,9 @@ static int vfio_save_setup(QEMUFile *f, void *opaque, E= rror **errp) =20 qemu_put_be64(f, VFIO_MIG_FLAG_DEV_SETUP_STATE); =20 - vfio_query_stop_copy_size(vbasedev, &stop_copy_size); + vfio_query_stop_copy_size(vbasedev); migration->data_buffer_size =3D MIN(VFIO_MIG_DEFAULT_DATA_BUFFER_SIZE, - stop_copy_size); + migration->stopcopy_size); migration->data_buffer =3D g_try_malloc0(migration->data_buffer_size); if (!migration->data_buffer) { error_setg(errp, "%s: Failed to allocate migration data buffer", @@ -576,32 +589,22 @@ static void vfio_state_pending_estimate(void *opaque,= uint64_t *must_precopy, migration->precopy_dirty_size); } =20 -/* - * Migration size of VFIO devices can be as little as a few KBs or as big = as - * many GBs. This value should be big enough to cover the worst case. - */ -#define VFIO_MIG_STOP_COPY_SIZE (100 * GiB) - static void vfio_state_pending_exact(void *opaque, uint64_t *must_precopy, uint64_t *can_postcopy) { VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; - uint64_t stop_copy_size =3D VFIO_MIG_STOP_COPY_SIZE; =20 - /* - * If getting pending migration size fails, VFIO_MIG_STOP_COPY_SIZE is - * reported so downtime limit won't be violated. - */ - vfio_query_stop_copy_size(vbasedev, &stop_copy_size); - *must_precopy +=3D stop_copy_size; + vfio_query_stop_copy_size(vbasedev); + *must_precopy +=3D migration->stopcopy_size; =20 if (vfio_device_state_is_precopy(vbasedev)) { vfio_query_precopy_size(migration); } =20 trace_vfio_state_pending_exact(vbasedev->name, *must_precopy, *can_pos= tcopy, - stop_copy_size, migration->precopy_init= _size, + migration->stopcopy_size, + migration->precopy_init_size, migration->precopy_dirty_size); } =20 --=20 2.50.1 From nobody Sun Mar 22 14:17:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1773962049; cv=none; d=zohomail.com; s=zohoarc; b=n+jzOWTiBLsrgvpZu/sUujk+XFVV45P09ioC/RPiPkCQ6e+7Vf7RPP2Q/RHza0Ud7j2wF74PuoCPHs6/H0gl/B+FYldYBL+XuFGgMx+vbs8/tAjB65Udcrug8tqAHdb1ai6HHpsWxjIaAiIh77ZLX6C3oS3JqVTRVjHDDJheTTM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773962049; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=D1BrBdrzK29CHSb+V1d+R+8Ipqmktv7AoIQg4+C/0zE=; b=PRU4ja81Tq/Yhf1KCciv28QWR7i1gE14VPeWnJo0x/PAtr1TRb3LX70kBCNW+75oc26BG3hsQWIE4eSbLw0xQbnlD0q4szN38yQ6hP5l+0HW5jSRHsmYsq3atmXtb1Tkp5TeameZ+DL67b+IiDqxQZm1JiXfptLk5U1HOsMz0kA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773962049003306.148360896801; Thu, 19 Mar 2026 16:14:09 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w3MYQ-0001OF-ED; Thu, 19 Mar 2026 19:13:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYO-0001Nh-Gm for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYL-0001QF-Sj for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:24 -0400 Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-182-NDSYfyt4Pa6nNIMjCdOgKw-1; Thu, 19 Mar 2026 19:13:19 -0400 Received: by mail-qt1-f197.google.com with SMTP id d75a77b69052e-50b31cff27fso9189071cf.3 for ; Thu, 19 Mar 2026 16:13:19 -0700 (PDT) Received: from x1.local ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-50b36e5bee3sm6717161cf.21.2026.03.19.16.13.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Mar 2026 16:13:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773962001; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D1BrBdrzK29CHSb+V1d+R+8Ipqmktv7AoIQg4+C/0zE=; b=YtxRGAUrQiHnPmf6e0aC/WeuOcqEfZpWs+cRDpM+++nd4au+7wKKKz9w8cwrfUx0eaXPk6 U5bzhl+VbTyfjvdJ1hEMWB1501v5NEtmTEdw/w2Boz16BsLnxLeUEsFYLr/7F+obtvdNHz REzErzGdM8IiNzZHLT9xDn0sJGho7T4= X-MC-Unique: NDSYfyt4Pa6nNIMjCdOgKw-1 X-Mimecast-MFC-AGG-ID: NDSYfyt4Pa6nNIMjCdOgKw_1773961999 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1773961999; x=1774566799; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=D1BrBdrzK29CHSb+V1d+R+8Ipqmktv7AoIQg4+C/0zE=; b=jXQGZxiCR+6aDRNM1bqf2zitwB70k4zVx/cewnGk4XbtME3/ySslE+uEmQ1svMfv5o IBy0tS2V+5jYC6WD/2OpiHtoShHe1k+BEpD2fE+q2uQ5bAWc6PLZwRvaGaFE/WS0vdkm Ja4LShKKXA9we8uwpnBS+6gzvrluA6ecLcIEeea3ttDmc1kDxUZUgQtDFaRQ5/SrdLDr WhyRpkfHYOAhvgJw3TEAdQ8MzDADIHU6HThGQ4FFUTQd2wRq4CSx7NCqNHxwV//US9wo al3O/uGQacqzBCji48fHkNYg/FF4BBpv+b01KtF2a5pU+XYDHOkd/aH2aieORlDU4Na3 9JRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773961999; x=1774566799; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=D1BrBdrzK29CHSb+V1d+R+8Ipqmktv7AoIQg4+C/0zE=; b=lcB3ApSU4XmkMxmSSDpVebrIcqBI2EFkdLGnU6VFd5/J7be/zUegZUr/tIUAaGutWe YLVdADjqktlsvwr2awvpquU2oQbsr6rqOeX1jk74sihn7CHxxz5OSVpRlrwgEOHqVPN9 EkGvlL1c7k/0EIhc4N2g2QSQgRaShX1V93ILJuvixtiadGXPAQDpRlpR+3RPsGvdYP+i G7wqnm5zQaL2fzNdydfVcpatct2G7vxl/PsZRLZ8z8mbBV7D7lZ/Hg4UvuUPh2d/myCq MNQ8B3EJxhWkX+QdooW9tar+yRpvcq0ujmX4yndtTZYptYiyKX7n/j5Ep/alv2YiJV6a jcrg== X-Gm-Message-State: AOJu0Yw8nfSh0UwLBL8SUhby/PEuhblo8PRUg8ASHGOFmIZv4SX9r4qS 7muB4/SagyOPfkBAPIiqOMBK/46+8XlE/SVBineKcywI1v5jfdQ+p5y37nAH+PZfRRphN/SLgUr Ue3XfvawcCIbVr/9i0JFj0iuJWb2nyy1QC1DwCuQdE+tX0Il7H5KM310ZmpxJPfsIBHJ7BICAzh r3uSR31mopp+1pP9KF29cHdNS4h0SnEoW1R2f+dA== X-Gm-Gg: ATEYQzwzYc5LGy5jPSWh0YUbWTfQRyo0A18g72R9OpjVjXeWtD92Jui7iJqfM4BcW2O klKO0zjkorjcOHS6lUxA6vNF/aCDUOsTDvdEaq5pPltHCw1tUU2drRJrdrSPA9KpOfOoWi9VAdH PQuJqDekLDaEuWRkRQ2wvs4iDtgaqabBm6IIh4NR0kQdHsbk8U/25wFtsYf531rKrM0aErYISiz QUX0ThTrOmGO2VRl6VtNBarAkjNmoLrIzJSsoGRalZMPFrwYtJeg0ZuXruEArvzXMwYdzj4GsfB WwXPwQypm/vjmoghRN5TjXvi9W39UfpcHMNQz0NKUpNw6NsUHr8dE+XzxPR8JlilMOOLkmJVYGH jy3AbiCa0bFQZZg== X-Received: by 2002:a05:622a:6691:b0:509:aa4:49f0 with SMTP id d75a77b69052e-50b37541c05mr11342181cf.50.1773961998830; Thu, 19 Mar 2026 16:13:18 -0700 (PDT) X-Received: by 2002:a05:622a:6691:b0:509:aa4:49f0 with SMTP id d75a77b69052e-50b37541c05mr11341551cf.50.1773961998114; Thu, 19 Mar 2026 16:13:18 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juraj Marcin , Kirti Wankhede , "Maciej S . Szmigiero" , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Joao Martins , Alex Williamson , Yishai Hadas , Fabiano Rosas , Pranav Tyagi , peterx@redhat.com, Zhiyi Guo , Markus Armbruster , Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Halil Pasic , Christian Borntraeger , Jason Herne , Eric Farman , Matthew Rosato , Richard Henderson , Ilya Leoshkevich , David Hildenbrand , Cornelia Huck , Eric Blake , Vladimir Sementsov-Ogievskiy , John Snow Subject: [PATCH RFC 05/12] migration/treewide: Merge @state_pending_{exact|estimate} APIs Date: Thu, 19 Mar 2026 19:12:55 -0400 Message-ID: <20260319231302.123135-6-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260319231302.123135-1-peterx@redhat.com> References: <20260319231302.123135-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.819, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.903, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1773962051977158500 Content-Type: text/plain; charset="utf-8" These two APIs are a slight duplication. For example, there're a few users that directly pass in the same function. It might also be slightly error prone to provide two hooks, so that it's easier to happen that one module report different things via the two hooks. In reality they should always report the same thing, only about whether we should use a fast-path when the slow path might be too slow, and even if we need to pay with some less accuracy. Let's just merge it into one API, but instead provide a bool showing if the query is a fast query or not. No functional change intended. Export qemu_savevm_query_pending(). We should likely directly use the new API here provided when there're new users to do the query. This will happen very soon. Cc: Halil Pasic Cc: Christian Borntraeger Cc: Jason Herne Cc: Eric Farman Cc: Matthew Rosato Cc: Richard Henderson Cc: Ilya Leoshkevich Cc: David Hildenbrand Cc: Cornelia Huck Cc: Eric Blake Cc: Vladimir Sementsov-Ogievskiy Cc: John Snow Signed-off-by: Peter Xu --- docs/devel/migration/main.rst | 5 ++- docs/devel/migration/vfio.rst | 9 ++---- include/migration/register.h | 52 ++++++++++-------------------- migration/savevm.h | 3 ++ hw/s390x/s390-stattrib.c | 8 ++--- hw/vfio/migration.c | 58 +++++++++++++++++++--------------- migration/block-dirty-bitmap.c | 9 ++---- migration/ram.c | 32 ++++++------------- migration/savevm.c | 43 ++++++++++++------------- hw/vfio/trace-events | 3 +- 10 files changed, 93 insertions(+), 129 deletions(-) diff --git a/docs/devel/migration/main.rst b/docs/devel/migration/main.rst index 234d280249..22c5910d5c 100644 --- a/docs/devel/migration/main.rst +++ b/docs/devel/migration/main.rst @@ -519,9 +519,8 @@ An iterative device must provide: data we must save. The core migration code will use this to determine when to pause the CPUs and complete the migration. =20 - - A ``state_pending_estimate`` function that indicates how much more - data we must save. When the estimated amount is smaller than the - threshold, we call ``state_pending_exact``. + - A ``save_query_pending`` function that indicates how much more + data we must save. =20 - A ``save_live_iterate`` function should send a chunk of data until the point that stream bandwidth limits tell it to stop. Each call diff --git a/docs/devel/migration/vfio.rst b/docs/devel/migration/vfio.rst index 0790e5031d..33768c877c 100644 --- a/docs/devel/migration/vfio.rst +++ b/docs/devel/migration/vfio.rst @@ -50,13 +50,8 @@ VFIO implements the device hooks for the iterative appro= ach as follows: * A ``load_setup`` function that sets the VFIO device on the destination in _RESUMING state. =20 -* A ``state_pending_estimate`` function that reports an estimate of the - remaining pre-copy data that the vendor driver has yet to save for the V= FIO - device. - -* A ``state_pending_exact`` function that reads pending_bytes from the ven= dor - driver, which indicates the amount of data that the vendor driver has ye= t to - save for the VFIO device. +* A ``save_query_pending`` function that reports the remaining pre-copy + data that the vendor driver has yet to save for the VFIO device. =20 * An ``is_active_iterate`` function that indicates ``save_live_iterate`` is active only when the VFIO device is in pre-copy states. diff --git a/include/migration/register.h b/include/migration/register.h index d0f37f5f43..2320c3a981 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -16,6 +16,15 @@ =20 #include "hw/core/vmstate-if.h" =20 +typedef struct MigPendingData { + /* How many bytes are pending for precopy / stopcopy? */ + uint64_t precopy_bytes; + /* How many bytes are pending that can be transferred in postcopy? */ + uint64_t postcopy_bytes; + /* Is this a fastpath query (which can be inaccurate)? */ + bool fastpath; +} MigPendingData ; + /** * struct SaveVMHandlers: handler structure to finely control * migration of complex subsystems and devices, such as RAM, block and @@ -197,46 +206,17 @@ typedef struct SaveVMHandlers { bool (*save_postcopy_prepare)(QEMUFile *f, void *opaque, Error **errp); =20 /** - * @state_pending_estimate - * - * This estimates the remaining data to transfer - * - * Sum of @can_postcopy and @must_postcopy is the whole amount of - * pending data. - * - * @opaque: data pointer passed to register_savevm_live() - * @must_precopy: amount of data that must be migrated in precopy - * or in stopped state, i.e. that must be migrated - * before target start. - * @can_postcopy: amount of data that can be migrated in postcopy - * or in stopped state, i.e. after target start. - * Some can also be migrated during precopy (RAM). - * Some must be migrated after source stops - * (block-dirty-bitmap) - */ - void (*state_pending_estimate)(void *opaque, uint64_t *must_precopy, - uint64_t *can_postcopy); - - /** - * @state_pending_exact - * - * This calculates the exact remaining data to transfer + * @save_query_pending * - * Sum of @can_postcopy and @must_postcopy is the whole amount of - * pending data. + * This estimates the remaining data to transfer on the source side. + * It's highly suggested that the module should implement both fastpath + * and slowpath version of it when it can be slow (for more information + * please check pending->fastpath field). * * @opaque: data pointer passed to register_savevm_live() - * @must_precopy: amount of data that must be migrated in precopy - * or in stopped state, i.e. that must be migrated - * before target start. - * @can_postcopy: amount of data that can be migrated in postcopy - * or in stopped state, i.e. after target start. - * Some can also be migrated during precopy (RAM). - * Some must be migrated after source stops - * (block-dirty-bitmap) + * @pending: pointer to a MigPendingData struct */ - void (*state_pending_exact)(void *opaque, uint64_t *must_precopy, - uint64_t *can_postcopy); + void (*save_query_pending)(void *opaque, MigPendingData *pending); =20 /** * @load_state diff --git a/migration/savevm.h b/migration/savevm.h index b3d1e8a13c..b116933bce 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -14,6 +14,8 @@ #ifndef MIGRATION_SAVEVM_H #define MIGRATION_SAVEVM_H =20 +#include "migration/register.h" + #define QEMU_VM_FILE_MAGIC 0x5145564d #define QEMU_VM_FILE_VERSION_COMPAT 0x00000002 #define QEMU_VM_FILE_VERSION 0x00000003 @@ -43,6 +45,7 @@ int qemu_savevm_state_iterate(QEMUFile *f, bool postcopy); void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(MigrationState *s); +void qemu_savevm_query_pending(MigPendingData *pending, bool fastpath); void qemu_savevm_state_pending_exact(uint64_t *must_precopy, uint64_t *can_postcopy); void qemu_savevm_state_pending_estimate(uint64_t *must_precopy, diff --git a/hw/s390x/s390-stattrib.c b/hw/s390x/s390-stattrib.c index d808ece3b9..b1ec51c77a 100644 --- a/hw/s390x/s390-stattrib.c +++ b/hw/s390x/s390-stattrib.c @@ -187,15 +187,14 @@ static int cmma_save_setup(QEMUFile *f, void *opaque,= Error **errp) return 0; } =20 -static void cmma_state_pending(void *opaque, uint64_t *must_precopy, - uint64_t *can_postcopy) +static void cmma_state_pending(void *opaque, MigPendingData *pending) { S390StAttribState *sas =3D S390_STATTRIB(opaque); S390StAttribClass *sac =3D S390_STATTRIB_GET_CLASS(sas); long long res =3D sac->get_dirtycount(sas); =20 if (res >=3D 0) { - *must_precopy +=3D res; + pending->precopy_bytes +=3D res; } } =20 @@ -340,8 +339,7 @@ static SaveVMHandlers savevm_s390_stattrib_handlers =3D= { .save_setup =3D cmma_save_setup, .save_live_iterate =3D cmma_save_iterate, .save_complete =3D cmma_save_complete, - .state_pending_exact =3D cmma_state_pending, - .state_pending_estimate =3D cmma_state_pending, + .save_query_pending =3D cmma_state_pending, .save_cleanup =3D cmma_save_cleanup, .load_state =3D cmma_load, .is_active =3D cmma_active, diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 827d3ded63..c054c749b0 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -570,42 +570,51 @@ static void vfio_save_cleanup(void *opaque) trace_vfio_save_cleanup(vbasedev->name); } =20 -static void vfio_state_pending_estimate(void *opaque, uint64_t *must_preco= py, - uint64_t *can_postcopy) +static void vfio_state_pending_sync(VFIODevice *vbasedev) { - VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; =20 - if (!vfio_device_state_is_precopy(vbasedev)) { - return; - } + vfio_query_stop_copy_size(vbasedev); =20 - *must_precopy +=3D - migration->precopy_init_size + migration->precopy_dirty_size; + if (vfio_device_state_is_precopy(vbasedev)) { + vfio_query_precopy_size(migration); + } =20 - trace_vfio_state_pending_estimate(vbasedev->name, *must_precopy, - *can_postcopy, - migration->precopy_init_size, - migration->precopy_dirty_size); + /* + * In all cases, all PRECOPY data should be no more than STOPCOPY data. + * Otherwise we have a problem. So far, only dump some errors. + */ + if (migration->precopy_init_size + migration->precopy_dirty_size < + migration->stopcopy_size) { + error_report_once("%s: wrong pending data (init=3D%" PRIx64 + ", dirty=3D%"PRIx64", stop=3D%"PRIx64")", + __func__, migration->precopy_init_size, + migration->precopy_dirty_size, + migration->stopcopy_size); + } } =20 -static void vfio_state_pending_exact(void *opaque, uint64_t *must_precopy, - uint64_t *can_postcopy) +static void vfio_state_pending(void *opaque, MigPendingData *pending) { VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; + uint64_t remain; =20 - vfio_query_stop_copy_size(vbasedev); - *must_precopy +=3D migration->stopcopy_size; - - if (vfio_device_state_is_precopy(vbasedev)) { - vfio_query_precopy_size(migration); + if (pending->fastpath) { + if (!vfio_device_state_is_precopy(vbasedev)) { + return; + } + remain =3D migration->precopy_init_size + migration->precopy_dirty= _size; + } else { + vfio_state_pending_sync(vbasedev); + remain =3D migration->stopcopy_size; } =20 - trace_vfio_state_pending_exact(vbasedev->name, *must_precopy, *can_pos= tcopy, - migration->stopcopy_size, - migration->precopy_init_size, - migration->precopy_dirty_size); + pending->precopy_bytes +=3D remain; + + trace_vfio_state_pending(vbasedev->name, migration->stopcopy_size, + migration->precopy_init_size, + migration->precopy_dirty_size); } =20 static bool vfio_is_active_iterate(void *opaque) @@ -850,8 +859,7 @@ static const SaveVMHandlers savevm_vfio_handlers =3D { .save_prepare =3D vfio_save_prepare, .save_setup =3D vfio_save_setup, .save_cleanup =3D vfio_save_cleanup, - .state_pending_estimate =3D vfio_state_pending_estimate, - .state_pending_exact =3D vfio_state_pending_exact, + .save_query_pending =3D vfio_state_pending, .is_active_iterate =3D vfio_is_active_iterate, .save_live_iterate =3D vfio_save_iterate, .save_complete =3D vfio_save_complete_precopy, diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c index a061aad817..376a9b43ac 100644 --- a/migration/block-dirty-bitmap.c +++ b/migration/block-dirty-bitmap.c @@ -766,9 +766,7 @@ static int dirty_bitmap_save_complete(QEMUFile *f, void= *opaque) return 0; } =20 -static void dirty_bitmap_state_pending(void *opaque, - uint64_t *must_precopy, - uint64_t *can_postcopy) +static void dirty_bitmap_state_pending(void *opaque, MigPendingData *data) { DBMSaveState *s =3D &((DBMState *)opaque)->save; SaveBitmapState *dbms; @@ -788,7 +786,7 @@ static void dirty_bitmap_state_pending(void *opaque, =20 trace_dirty_bitmap_state_pending(pending); =20 - *can_postcopy +=3D pending; + data->postcopy_bytes +=3D pending; } =20 /* First occurrence of this bitmap. It should be created if doesn't exist = */ @@ -1250,8 +1248,7 @@ static SaveVMHandlers savevm_dirty_bitmap_handlers = =3D { .save_setup =3D dirty_bitmap_save_setup, .save_complete =3D dirty_bitmap_save_complete, .has_postcopy =3D dirty_bitmap_has_postcopy, - .state_pending_exact =3D dirty_bitmap_state_pending, - .state_pending_estimate =3D dirty_bitmap_state_pending, + .save_query_pending =3D dirty_bitmap_state_pending, .save_live_iterate =3D dirty_bitmap_save_iterate, .is_active_iterate =3D dirty_bitmap_is_active_iterate, .load_state =3D dirty_bitmap_load, diff --git a/migration/ram.c b/migration/ram.c index 979751f61b..89f761a471 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3443,30 +3443,17 @@ static int ram_save_complete(QEMUFile *f, void *opa= que) return qemu_fflush(f); } =20 -static void ram_state_pending_estimate(void *opaque, uint64_t *must_precop= y, - uint64_t *can_postcopy) -{ - RAMState **temp =3D opaque; - RAMState *rs =3D *temp; - - uint64_t remaining_size =3D rs->migration_dirty_pages * TARGET_PAGE_SI= ZE; - - if (migrate_postcopy_ram()) { - /* We can do postcopy, and all the data is postcopiable */ - *can_postcopy +=3D remaining_size; - } else { - *must_precopy +=3D remaining_size; - } -} - -static void ram_state_pending_exact(void *opaque, uint64_t *must_precopy, - uint64_t *can_postcopy) +static void ram_state_pending(void *opaque, MigPendingData *pending) { RAMState **temp =3D opaque; RAMState *rs =3D *temp; uint64_t remaining_size; =20 - if (!migration_in_postcopy()) { + /* + * Sync is only needed either with: (1) a fast query, or (2) postcopy + * as started (in which case no new dirty will generate anymore). + */ + if (!pending->fastpath && !migration_in_postcopy()) { bql_lock(); WITH_RCU_READ_LOCK_GUARD() { migration_bitmap_sync_precopy(false); @@ -3478,9 +3465,9 @@ static void ram_state_pending_exact(void *opaque, uin= t64_t *must_precopy, =20 if (migrate_postcopy_ram()) { /* We can do postcopy, and all the data is postcopiable */ - *can_postcopy +=3D remaining_size; + pending->postcopy_bytes +=3D remaining_size; } else { - *must_precopy +=3D remaining_size; + pending->precopy_bytes +=3D remaining_size; } } =20 @@ -4703,8 +4690,7 @@ static SaveVMHandlers savevm_ram_handlers =3D { .save_live_iterate =3D ram_save_iterate, .save_complete =3D ram_save_complete, .has_postcopy =3D ram_has_postcopy, - .state_pending_exact =3D ram_state_pending_exact, - .state_pending_estimate =3D ram_state_pending_estimate, + .save_query_pending =3D ram_state_pending, .load_state =3D ram_load, .save_cleanup =3D ram_save_cleanup, .load_setup =3D ram_load_setup, diff --git a/migration/savevm.c b/migration/savevm.c index dd58f2a705..6268e68382 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1762,46 +1762,45 @@ int qemu_savevm_state_complete_precopy(MigrationSta= te *s) return qemu_fflush(f); } =20 -/* Give an estimate of the amount left to be transferred, - * the result is split into the amount for units that can and - * for units that can't do postcopy. - */ -void qemu_savevm_state_pending_estimate(uint64_t *must_precopy, - uint64_t *can_postcopy) +void qemu_savevm_query_pending(MigPendingData *pending, bool fastpath) { SaveStateEntry *se; =20 - *must_precopy =3D 0; - *can_postcopy =3D 0; + pending->precopy_bytes =3D 0; + pending->postcopy_bytes =3D 0; + pending->fastpath =3D fastpath; =20 QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { - if (!se->ops || !se->ops->state_pending_estimate) { + if (!se->ops || !se->ops->save_query_pending) { continue; } if (!qemu_savevm_state_active(se)) { continue; } - se->ops->state_pending_estimate(se->opaque, must_precopy, can_post= copy); + se->ops->save_query_pending(se->opaque, pending); } } =20 +void qemu_savevm_state_pending_estimate(uint64_t *must_precopy, + uint64_t *can_postcopy) +{ + MigPendingData pending; + + qemu_savevm_query_pending(&pending, true); + + *must_precopy =3D pending.precopy_bytes; + *can_postcopy =3D pending.postcopy_bytes; +} + void qemu_savevm_state_pending_exact(uint64_t *must_precopy, uint64_t *can_postcopy) { - SaveStateEntry *se; + MigPendingData pending; =20 - *must_precopy =3D 0; - *can_postcopy =3D 0; + qemu_savevm_query_pending(&pending, false); =20 - QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { - if (!se->ops || !se->ops->state_pending_exact) { - continue; - } - if (!qemu_savevm_state_active(se)) { - continue; - } - se->ops->state_pending_exact(se->opaque, must_precopy, can_postcop= y); - } + *must_precopy =3D pending.precopy_bytes; + *can_postcopy =3D pending.postcopy_bytes; } =20 void qemu_savevm_state_cleanup(void) diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index 846e3625c5..7cf5a9eb2d 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -173,8 +173,7 @@ vfio_save_device_config_state(const char *name) " (%s)" vfio_save_iterate(const char *name, uint64_t precopy_init_size, uint64_t p= recopy_dirty_size) " (%s) precopy initial size %"PRIu64" precopy dirty size= %"PRIu64 vfio_save_iterate_start(const char *name) " (%s)" vfio_save_setup(const char *name, uint64_t data_buffer_size) " (%s) data b= uffer size %"PRIu64 -vfio_state_pending_estimate(const char *name, uint64_t precopy, uint64_t p= ostcopy, uint64_t precopy_init_size, uint64_t precopy_dirty_size) " (%s) pr= ecopy %"PRIu64" postcopy %"PRIu64" precopy initial size %"PRIu64" precopy d= irty size %"PRIu64 -vfio_state_pending_exact(const char *name, uint64_t precopy, uint64_t post= copy, uint64_t stopcopy_size, uint64_t precopy_init_size, uint64_t precopy_= dirty_size) " (%s) precopy %"PRIu64" postcopy %"PRIu64" stopcopy size %"PRI= u64" precopy initial size %"PRIu64" precopy dirty size %"PRIu64 +vfio_state_pending(const char *name, uint64_t stopcopy_size, uint64_t prec= opy_init_size, uint64_t precopy_dirty_size) " (%s) stopcopy size %"PRIu64" = precopy initial size %"PRIu64" precopy dirty size %"PRIu64 vfio_vmstate_change(const char *name, int running, const char *reason, con= st char *dev_state) " (%s) running %d reason %s device state %s" vfio_vmstate_change_prepare(const char *name, int running, const char *rea= son, const char *dev_state) " (%s) running %d reason %s device state %s" =20 --=20 2.50.1 From nobody Sun Mar 22 14:17:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1773962115; cv=none; d=zohomail.com; s=zohoarc; b=cy+yP1RC7d3EWEM4lGj8FXM53mEYYozY0Pfuq4PKTcX0nqf9yXEuSJ5hvtCJn181+yT2iA8cZDuIx4NjqVW+TnTrsdwE5ypEbzyLD4xeuWUyX/ZtSolhBA3DtaLa2V7iqN+79YmD1Rp7UAdNvLq0uvmir7wideXzjhfMJFhRLik= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773962115; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=SvYChAt7BjGSW8L2rQATcp7ZC0XhC8w7Tx3h3LXPv4w=; b=bDEFqz65QM5m0mdJkc+JNTsbyL6azs6oB5EJPlXlD9c+FHvjHYfb7ufLGeFlRV1rlnhhEQMLBWVjdaXEfz18A/vJJLY/pzeFi229aoQ09a2jBtnqa0eTBH6tbKh8ns/ROzJdVtSyHz/Cc4YJJVaVKflV1HV3ynM+LRKFK6Sm36k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17739621151951007.6704732956094; Thu, 19 Mar 2026 16:15:15 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w3MYR-0001OY-NN; Thu, 19 Mar 2026 19:13:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYP-0001Nt-Mt for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:25 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYN-0001SZ-VF for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:25 -0400 Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-689-mHt8bz-4Pcqxj-Vo2Kx5aA-1; Thu, 19 Mar 2026 19:13:22 -0400 Received: by mail-qt1-f197.google.com with SMTP id d75a77b69052e-50939597b85so38900261cf.2 for ; Thu, 19 Mar 2026 16:13:22 -0700 (PDT) Received: from x1.local ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-50b36e5bee3sm6717161cf.21.2026.03.19.16.13.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Mar 2026 16:13:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773962003; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SvYChAt7BjGSW8L2rQATcp7ZC0XhC8w7Tx3h3LXPv4w=; b=V8hAU69HHczmHzn+e+nKYAn9d11lKtdsvDeRBFgkW+tos5k2N7s6IcLNM7r6cDZ8AVElZN duHV7XNIw48xpA04JHAfJtYUGkKxlCuLA9j16Yr0KRC97dHKUYt/gi3QcIwR677y4uj1WV gtOtOWT9k+fUdDVqRPh6E3UDffRC9cs= X-MC-Unique: mHt8bz-4Pcqxj-Vo2Kx5aA-1 X-Mimecast-MFC-AGG-ID: mHt8bz-4Pcqxj-Vo2Kx5aA_1773962001 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1773962001; x=1774566801; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SvYChAt7BjGSW8L2rQATcp7ZC0XhC8w7Tx3h3LXPv4w=; b=i7rzfxdysQTghlXCCHg6LjP8jyUB65Sw5eeqKjWSCc1iXT6IbjAqkuhpWFBuU3lOyA I7J3v/V36+lASdhCWk3ofGJaQCMPBC+SxWvTQFrvITWLpTBkEUVtZdblEg7CKflzO8p1 Hjnhb4AWJRK+0ul4o0AhD3bCgtezP9utbNV5Zvo7aFKoLljJ/5TlIoO7cc7um28m9+En rxorwVkJiYlb7q3RBV3QJi5ZXYM0fLoWkd04ZNfFOPqwXEX+dEyMI5DpNeQQCihgYQT3 AkCH8YygO77+NLRjq+YRE9dFxAfKjVzol1+TCSlLdgcuPg26aSt21AtBAGz2I4VZXIm5 bVVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773962001; x=1774566801; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=SvYChAt7BjGSW8L2rQATcp7ZC0XhC8w7Tx3h3LXPv4w=; b=A/uGSeLQxsGQcD8yV91YJZK7y5V5yBLmyEyicHnWu8GUaE1doe1ZKKt1V9Vly64c2j 1KeXVqD+3ofywwNUuGVBcj9OGO+/m9Etn5/17m3yeAO7SBb5wpSZ00wED9nrbV3t2Ssk AViPIwu7HepcYy+G9GMvKVefkqe0a/yDluJzjoUUPtzb3CGpSzVipvlrkywGEKK0FZs2 K7HhQp/PryrkPQzEPDbDdnNxiF8+urgT13WPrADiJnjp2rk80ocIEsBQRe5LZYyzqN7h X3xEA9mwIEYcA6omy3mSDiw4COGwIZkPq26r5Pl8hfBjVRJ92gr1eeLcy5l3IVIJ2GTC Q3/g== X-Gm-Message-State: AOJu0Yxpmd6XYKE9c1BYHocp3rQXbNI4iB2oA8jSsEUWlGBXjbDZe5Ya 5PwDOlJYsnw40oDTfroaDVO7JVsr+ZrtR44xk/7ei4wt78nxJCZmuBZHtm+ne6uMtgB97bYHmpY Z+ys3+GMD+s0t8/273WlXUXsoDbDI1lcDlCwY5O/fGmrY/UZpaEane+n6nE9sCk3vSC0Uwv10lR h0D56QvQW9zKTDPCXrDAni70MfvV9qLMSTI2PplA== X-Gm-Gg: ATEYQzy3pvmFJonG+exXrTDjDEI9eeF/pufAVNjYE0OGpgbHPg7p7QT4Xr8lRAykP5E lxG3+snpjEOZo61eVvLTd3j/VMV56WjIfxd+0Q7TAAlVJAmZG7PL3hcbXI1lAGLBLOUehETRm/d 5dUBFPtbWd/KvyKZ4VIsqA+294cBYe5/aqtYutbvhh7Io2iRgr//nOzTD72e83ew/yZiv78S+QA woboluo9NW+Oo2XV3TO7emzpmEUwDxkN9qaKO+z3HmKhkKlHpqrecRvK2tMF4XTUjRU6u16dcpm ovAYgnb07NAjmXfGlI5lBN969L8RjUX/7/RiwWVs4MHadHcTKgKpuA6eDjdWQYsTTpZnW3FLhRb UeEfUeW/F08nLTA== X-Received: by 2002:a05:622a:2c9:b0:509:2677:68fc with SMTP id d75a77b69052e-50b3739f85amr17479501cf.10.1773962001103; Thu, 19 Mar 2026 16:13:21 -0700 (PDT) X-Received: by 2002:a05:622a:2c9:b0:509:2677:68fc with SMTP id d75a77b69052e-50b3739f85amr17478871cf.10.1773962000447; Thu, 19 Mar 2026 16:13:20 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juraj Marcin , Kirti Wankhede , "Maciej S . Szmigiero" , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Joao Martins , Alex Williamson , Yishai Hadas , Fabiano Rosas , Pranav Tyagi , peterx@redhat.com, Zhiyi Guo , Markus Armbruster , Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PATCH RFC 06/12] migration: Use the new save_query_pending() API directly Date: Thu, 19 Mar 2026 19:12:56 -0400 Message-ID: <20260319231302.123135-7-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260319231302.123135-1-peterx@redhat.com> References: <20260319231302.123135-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.819, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.903, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1773962117427158500 Content-Type: text/plain; charset="utf-8" It's easier to use the new API directly in the migration iterations. This also paves way for follow up patches to add new data to report directly to the iterator function. When at it, merge the tracepoints too into one. No functional change intended. Signed-off-by: Peter Xu --- migration/savevm.h | 4 ---- migration/migration.c | 16 +++++++--------- migration/savevm.c | 23 ++--------------------- migration/trace-events | 1 + 4 files changed, 10 insertions(+), 34 deletions(-) diff --git a/migration/savevm.h b/migration/savevm.h index b116933bce..3c39190fa4 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -46,10 +46,6 @@ void qemu_savevm_state_cleanup(void); void qemu_savevm_state_complete_postcopy(QEMUFile *f); int qemu_savevm_state_complete_precopy(MigrationState *s); void qemu_savevm_query_pending(MigPendingData *pending, bool fastpath); -void qemu_savevm_state_pending_exact(uint64_t *must_precopy, - uint64_t *can_postcopy); -void qemu_savevm_state_pending_estimate(uint64_t *must_precopy, - uint64_t *can_postcopy); int qemu_savevm_state_complete_precopy_iterable(QEMUFile *f, bool in_postc= opy); bool qemu_savevm_state_postcopy_prepare(QEMUFile *f, Error **errp); void qemu_savevm_state_end(QEMUFile *f); diff --git a/migration/migration.c b/migration/migration.c index dfc60372cf..99c4d09000 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3204,17 +3204,17 @@ typedef enum { */ static MigIterateState migration_iteration_run(MigrationState *s) { - uint64_t must_precopy, can_postcopy, pending_size; Error *local_err =3D NULL; bool in_postcopy =3D (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_DEVICE= || s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE); bool can_switchover =3D migration_can_switchover(s); + MigPendingData pending =3D { }; + uint64_t pending_size; bool complete_ready; =20 /* Fast path - get the estimated amount of pending data */ - qemu_savevm_state_pending_estimate(&must_precopy, &can_postcopy); - pending_size =3D must_precopy + can_postcopy; - trace_migrate_pending_estimate(pending_size, must_precopy, can_postcop= y); + qemu_savevm_query_pending(&pending, true); + pending_size =3D pending.precopy_bytes + pending.postcopy_bytes; =20 if (in_postcopy) { /* @@ -3243,14 +3243,12 @@ static MigIterateState migration_iteration_run(Migr= ationState *s) * during postcopy phase. */ if (pending_size <=3D s->threshold_size) { - qemu_savevm_state_pending_exact(&must_precopy, &can_postcopy); - pending_size =3D must_precopy + can_postcopy; - trace_migrate_pending_exact(pending_size, must_precopy, - can_postcopy); + qemu_savevm_query_pending(&pending, false); + pending_size =3D pending.precopy_bytes + pending.postcopy_byte= s; } =20 /* Should we switch to postcopy now? */ - if (must_precopy <=3D s->threshold_size && + if (pending.precopy_bytes <=3D s->threshold_size && can_switchover && qatomic_read(&s->start_postcopy)) { if (postcopy_start(s, &local_err)) { migrate_error_propagate(s, error_copy(local_err)); diff --git a/migration/savevm.c b/migration/savevm.c index 6268e68382..b3285d480f 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1779,28 +1779,9 @@ void qemu_savevm_query_pending(MigPendingData *pendi= ng, bool fastpath) } se->ops->save_query_pending(se->opaque, pending); } -} - -void qemu_savevm_state_pending_estimate(uint64_t *must_precopy, - uint64_t *can_postcopy) -{ - MigPendingData pending; - - qemu_savevm_query_pending(&pending, true); - - *must_precopy =3D pending.precopy_bytes; - *can_postcopy =3D pending.postcopy_bytes; -} - -void qemu_savevm_state_pending_exact(uint64_t *must_precopy, - uint64_t *can_postcopy) -{ - MigPendingData pending; - - qemu_savevm_query_pending(&pending, false); =20 - *must_precopy =3D pending.precopy_bytes; - *can_postcopy =3D pending.postcopy_bytes; + trace_qemu_savevm_query_pending(fastpath, pending->precopy_bytes, + pending->postcopy_bytes); } =20 void qemu_savevm_state_cleanup(void) diff --git a/migration/trace-events b/migration/trace-events index 60e5087e38..5f836a8652 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -7,6 +7,7 @@ qemu_loadvm_state_section_partend(uint32_t section_id) "%u" qemu_loadvm_state_post_main(int ret) "%d" qemu_loadvm_state_section_startfull(uint32_t section_id, const char *idstr= , uint32_t instance_id, uint32_t version_id) "%u(%s) %u %u" qemu_savevm_send_packaged(void) "" +qemu_savevm_query_pending(bool fast, uint64_t precopy, uint64_t postcopy) = "fast=3D%d, precopy=3D%"PRIu64", postcopy=3D%"PRIu64 loadvm_state_switchover_ack_needed(unsigned int switchover_ack_pending_num= ) "Switchover ack pending num=3D%u" loadvm_state_setup(void) "" loadvm_state_cleanup(void) "" --=20 2.50.1 From nobody Sun Mar 22 14:17:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1773962048; cv=none; d=zohomail.com; s=zohoarc; b=ONnT82BQDjSI2ySKAn0TdEnyC3FfN0E9cIPUcfaPFNZLS386naeLW/uIL7omcu+9rWy5rUKbRI/EqsN8BRq21WI9NZCmxcnLAh8CBxW04LfT8O19nihxz3hUw2yU+gPtPvcMvTU47KfTmTjWQJYUbni2aWweI5t0t7sxyKXf0mg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773962048; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ipcYTYg6wXP+u/zbMbL8eRUnK01vI0ajbEJw8rwturY=; b=ifhxktJ0Y15iQbiXz5JrnYQlLGzgJP2FTZ/+HImYCw1bOrXCoWlLGh8OtoTLpoaWO+im5y23gh4tqj3PrJK/UaFPwcqcCnT/kDinEh0fzf9617P/L07Lmkc3DtK9PFgZzdNIrFYqH++RFNwE0XUP9P7G7x1cr/fHx/cIhimWo7c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17739620488841002.5594918502679; Thu, 19 Mar 2026 16:14:08 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w3MYT-0001P3-Sp; Thu, 19 Mar 2026 19:13:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYS-0001Oa-2w for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYQ-0001WX-8m for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:27 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-113-a9MLxu8QMryATVTzCYvz6Q-1; Thu, 19 Mar 2026 19:13:24 -0400 Received: by mail-qt1-f199.google.com with SMTP id d75a77b69052e-50b389f2560so11103951cf.3 for ; Thu, 19 Mar 2026 16:13:24 -0700 (PDT) Received: from x1.local ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-50b36e5bee3sm6717161cf.21.2026.03.19.16.13.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Mar 2026 16:13:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773962005; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ipcYTYg6wXP+u/zbMbL8eRUnK01vI0ajbEJw8rwturY=; b=GXnl8733qKHNGfvqr2LzNIv0rKIic4f9mjHzcbobZPU8tGURTH/N7tDcpkiF1Os/7BAy7c IIQC8iaBP0HlmS7TNGlqDGMc1JHq857Wgcl5fgRxgNlVQXHmnhyCf4iFMkGKC/CX8pLXbg kJXdoaRKe9iyRlVaMtNlt0nczdcJTKQ= X-MC-Unique: a9MLxu8QMryATVTzCYvz6Q-1 X-Mimecast-MFC-AGG-ID: a9MLxu8QMryATVTzCYvz6Q_1773962004 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1773962003; x=1774566803; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ipcYTYg6wXP+u/zbMbL8eRUnK01vI0ajbEJw8rwturY=; b=Mx5O8P83jLTIYuRaUcCICdcwgG6FmNiric3ilzWjPvhI61AoIP5uHLMcB6QeNVsuiU ypNivFrSlR5cm8+w6l28FswuG69x60lFs9W1e+njq51RLPXdlOpcyF1RpPEmy56Yqk5u qBzMl37QpTsBgdtKbg4vnc96c9j6sSB2a7AK13d8HkA6m/I5knWPlALC1CL5eveRozaM Xxb+JZxveVEq/xhnlMkC6kPERZsfRFrkTQQluZM7zdGjPIvq0miDHCJFlRy13cPq1jb4 4MQoc/qGZZ0lqurjcjEYrQDAP0XBj/QVh8ezq28kAAi3udpyOG3GQKhlEiimnDxqQ3hh mhCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773962003; x=1774566803; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ipcYTYg6wXP+u/zbMbL8eRUnK01vI0ajbEJw8rwturY=; b=hmjhg5Ylgu9cOqh9koBFwDvqfxQ4Ul2x9Ef4ItDWq0qcbypdDQKXk93GYKRJQeIHE8 atn/d4Jx0igrXN6rb7Hj/6MjfJ93rcynAd9us4KXKyOtrRKpoN7P+eJHQVgKHTJ5imLR fnaSvang1c4xA2OW0QNr/tUyhGDm7ExH2gEkidL8Y5Vi0EnYm6XUX3hYff9+X2lOdUoR 9XT9agP8h5b42kvSQlQvPGP06uHqhh5ZOAuGBVJbZWp31jQskIN8XUE4NWbbVnakJxnH pttX1ekiOEjPnz7+E/uMCNRNMi3gsvV1HKRrx1aUMs4gYB+UlY4I4r/FebgFUiLIObAT fAKQ== X-Gm-Message-State: AOJu0YzMafcBFcD3mRHjV4g+xRbKaERaklj7KR0W2gVSoIlyIGV7chZb 6/dZjr6R3rC4HYUSVWHYRDFuAIddQ20cwReaZhlMiWcZlvVBwTPsQ8wZfeEQzicqvNWnLUvoyw9 POavSdosNLoVlbKhgYihMwSCVkxKPabevxWoD+lmeMeIGUNvirqduMa+Se4gv/hFAbr3kT6Hut9 IFcmlm9wxFGoRguYYrj3EhyERcYCRJgq5xT3uyEw== X-Gm-Gg: ATEYQzynfVnQRWRTbvyafYdx+ChntHJNLiLKBb10vy6hA7NCEzRovCNQOFdzdoWR6QI AT2/gZa9Z5EfL81Pr+R0o6R/yBG46TCJIaZlDgfOONqJm4H7EpiMSfT36V/VppJlVJR1f1W2PGI RJqwK+fmNB4ODCBgBJqtgXcFGILkKcpuaPkj4XnnSrvLfsOa5pKaq6GJkWvypw7DUSb1wIso8eH lpS05qpJ4aKfH+J9LNc3kEnAlGTqa+DIG04/VNR2OV2APjyRVJCY4J9bmZTagn+VhyjO2eeWhn4 h6rHg0a6uml0OHvTpbRmAwal4TaI9Ma9G751REwkn8syLBYl3pxzVCSt9lDKr1ApjP7aPS1B2z5 5rAp8b59FwloN1g== X-Received: by 2002:a05:622a:607:b0:509:aa4:49fa with SMTP id d75a77b69052e-50b374b17f9mr16611061cf.34.1773962003302; Thu, 19 Mar 2026 16:13:23 -0700 (PDT) X-Received: by 2002:a05:622a:607:b0:509:aa4:49fa with SMTP id d75a77b69052e-50b374b17f9mr16610341cf.34.1773962002705; Thu, 19 Mar 2026 16:13:22 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juraj Marcin , Kirti Wankhede , "Maciej S . Szmigiero" , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Joao Martins , Alex Williamson , Yishai Hadas , Fabiano Rosas , Pranav Tyagi , peterx@redhat.com, Zhiyi Guo , Markus Armbruster , Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PATCH RFC 07/12] migration: Introduce stopcopy_bytes in save_query_pending() Date: Thu, 19 Mar 2026 19:12:57 -0400 Message-ID: <20260319231302.123135-8-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260319231302.123135-1-peterx@redhat.com> References: <20260319231302.123135-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.819, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.903, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1773962051982158500 Content-Type: text/plain; charset="utf-8" Allow modules to report data that can only be migrated after VM is stopped. When this concept is introduced, we will need to account stopcopy size to be part of pending_size as before. One thing to mention is, when there can be stopcopy size, it means the old "pending_size" may not always be able to reach low enough to kickoff an slow version of query sync. While it used to be almost guaranteed to happen because if we keep iterating, normally pending_size can go to zero for precopy-only because we assume everything reported can be migrated in precopy phase. So we need to make sure QEMU will kickoff a synchronized version of query pending when all precopy data is migrated too. This might be important to VFIO to keep making progress even if the downtime cannot yet be satisfied. So far, this patch should introduce no functional change, as no module yet report stopcopy size. This will pave way for VFIO to properly report its pending data sizes, which was actually buggy today. Will be done in follow up patches. Signed-off-by: Peter Xu --- include/migration/register.h | 12 +++++++++ migration/migration.c | 52 ++++++++++++++++++++++++++++++------ migration/savevm.c | 7 +++-- migration/trace-events | 2 +- 4 files changed, 62 insertions(+), 11 deletions(-) diff --git a/include/migration/register.h b/include/migration/register.h index 2320c3a981..3824958ba5 100644 --- a/include/migration/register.h +++ b/include/migration/register.h @@ -17,12 +17,24 @@ #include "hw/core/vmstate-if.h" =20 typedef struct MigPendingData { + /* + * Modules can only update these fields in a query request via its + * save_query_pending() API. + */ /* How many bytes are pending for precopy / stopcopy? */ uint64_t precopy_bytes; /* How many bytes are pending that can be transferred in postcopy? */ uint64_t postcopy_bytes; + /* How many bytes that can only be transferred when VM stopped? */ + uint64_t stopcopy_bytes; + + /* + * Modules should never update these fields. + */ /* Is this a fastpath query (which can be inaccurate)? */ bool fastpath; + /* Total pending data */ + uint64_t total_bytes; } MigPendingData ; =20 /** diff --git a/migration/migration.c b/migration/migration.c index 99c4d09000..42facb16d1 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3198,6 +3198,44 @@ typedef enum { MIG_ITERATE_BREAK, /* Break the loop */ } MigIterateState; =20 +/* Are we ready to move to the next iteration phase? */ +static bool migration_iteration_next_ready(MigrationState *s, + MigPendingData *pending) +{ + /* + * If the estimated values already suggest us to switchover, mark this + * iteration finished, time to do a slow sync. + */ + if (pending->total_bytes <=3D s->threshold_size) { + return true; + } + + /* + * Since we may have modules reporting stop-only data, we also want to + * re-query with slow mode if all precopy data is moved over. This + * will also mark the current iteration done. + * + * This could happen when e.g. a module (like, VFIO) reports stopcopy + * size too large so it will never yet satisfy the downtime with the + * current setup (above check). Here, slow version of re-query helps + * because we keep trying the best to move whatever we have. + */ + if (pending->precopy_bytes =3D=3D 0) { + return true; + } + + return false; +} + +static void migration_iteration_go_next(MigPendingData *pending) +{ + /* + * Do a slow sync will achieve this. TODO: move RAM iteration code + * into the core layer. + */ + qemu_savevm_query_pending(pending, false); +} + /* * Return true if continue to the next iteration directly, false * otherwise. @@ -3209,12 +3247,10 @@ static MigIterateState migration_iteration_run(Migr= ationState *s) s->state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE); bool can_switchover =3D migration_can_switchover(s); MigPendingData pending =3D { }; - uint64_t pending_size; bool complete_ready; =20 /* Fast path - get the estimated amount of pending data */ qemu_savevm_query_pending(&pending, true); - pending_size =3D pending.precopy_bytes + pending.postcopy_bytes; =20 if (in_postcopy) { /* @@ -3222,7 +3258,7 @@ static MigIterateState migration_iteration_run(Migrat= ionState *s) * postcopy completion doesn't rely on can_switchover, because when * POSTCOPY_ACTIVE it means switchover already happened. */ - complete_ready =3D !pending_size; + complete_ready =3D !pending.total_bytes; if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_DEVICE && (s->postcopy_package_loaded || complete_ready)) { /* @@ -3242,9 +3278,8 @@ static MigIterateState migration_iteration_run(Migrat= ionState *s) * postcopy started, so ESTIMATE should always match with EXACT * during postcopy phase. */ - if (pending_size <=3D s->threshold_size) { - qemu_savevm_query_pending(&pending, false); - pending_size =3D pending.precopy_bytes + pending.postcopy_byte= s; + if (migration_iteration_next_ready(s, &pending)) { + migration_iteration_go_next(&pending); } =20 /* Should we switch to postcopy now? */ @@ -3264,11 +3299,12 @@ static MigIterateState migration_iteration_run(Migr= ationState *s) * (2) Pending size is no more than the threshold specified * (which was calculated from expected downtime) */ - complete_ready =3D can_switchover && (pending_size <=3D s->thresho= ld_size); + complete_ready =3D can_switchover && + (pending.total_bytes <=3D s->threshold_size); } =20 if (complete_ready) { - trace_migration_thread_low_pending(pending_size); + trace_migration_thread_low_pending(pending.total_bytes); migration_completion(s); return MIG_ITERATE_BREAK; } diff --git a/migration/savevm.c b/migration/savevm.c index b3285d480f..812c72b3e5 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1766,8 +1766,7 @@ void qemu_savevm_query_pending(MigPendingData *pendin= g, bool fastpath) { SaveStateEntry *se; =20 - pending->precopy_bytes =3D 0; - pending->postcopy_bytes =3D 0; + memset(pending, 0, sizeof(*pending)); pending->fastpath =3D fastpath; =20 QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { @@ -1780,7 +1779,11 @@ void qemu_savevm_query_pending(MigPendingData *pendi= ng, bool fastpath) se->ops->save_query_pending(se->opaque, pending); } =20 + pending->total_bytes =3D pending->precopy_bytes + + pending->stopcopy_bytes + pending->postcopy_bytes; + trace_qemu_savevm_query_pending(fastpath, pending->precopy_bytes, + pending->stopcopy_bytes, pending->postcopy_bytes); } =20 diff --git a/migration/trace-events b/migration/trace-events index 5f836a8652..175f09f8ad 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -7,7 +7,7 @@ qemu_loadvm_state_section_partend(uint32_t section_id) "%u" qemu_loadvm_state_post_main(int ret) "%d" qemu_loadvm_state_section_startfull(uint32_t section_id, const char *idstr= , uint32_t instance_id, uint32_t version_id) "%u(%s) %u %u" qemu_savevm_send_packaged(void) "" -qemu_savevm_query_pending(bool fast, uint64_t precopy, uint64_t postcopy) = "fast=3D%d, precopy=3D%"PRIu64", postcopy=3D%"PRIu64 +qemu_savevm_query_pending(bool fast, uint64_t precopy, uint64_t stopcopy, = uint64_t postcopy) "fast=3D%d, precopy=3D%"PRIu64", stopcopy=3D%"PRIu64", p= ostcopy=3D%"PRIu64 loadvm_state_switchover_ack_needed(unsigned int switchover_ack_pending_num= ) "Switchover ack pending num=3D%u" loadvm_state_setup(void) "" loadvm_state_cleanup(void) "" --=20 2.50.1 From nobody Sun Mar 22 14:17:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1773962063; cv=none; d=zohomail.com; s=zohoarc; b=NuiPWQJQRa92OR1RnkgZiV3nnj/FgkmvE3Dt+EoTH82J8n4hioK+qzCRTmvORhcoSBguG3eYOsij4aJYnx9fNrIgVR+GxBxHsDeg45JE+6zfXWb0W+dOKmCpmXe2FmW70ieGSmY/tl5g4vra/VWWnUxL16ENnYz3dUCIHEhCdjE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773962063; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=qhGfQJ/1tmgLlZlyqT2rU+eJDb8Vl/eOR9kvn1/xgAk=; b=AaVu9YW20/pJdzHXpKeNks+cKxVarOTfg2gRDAQ8X8jBpchRQGXICGyQV2Z2ujv2Aphw1NavoKgZKaTkWjBrMgSBNOC0NW3+vc4h+matVX9rV4KbjoZIjOxUkyBVgwgkIkZ6VtVxmKyhIVLSG5OPWzrWcf8cu1uCimPZN8n1sqA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773962063801208.08908642217023; Thu, 19 Mar 2026 16:14:23 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w3MYV-0001PG-Bh; Thu, 19 Mar 2026 19:13:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYT-0001P5-V8 for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYS-0001ba-GB for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:29 -0400 Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-59-I2wGZJvDNOS4RksiDia57Q-1; Thu, 19 Mar 2026 19:13:26 -0400 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-899f474fae1so229123626d6.0 for ; Thu, 19 Mar 2026 16:13:26 -0700 (PDT) Received: from x1.local ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-50b36e5bee3sm6717161cf.21.2026.03.19.16.13.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Mar 2026 16:13:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773962007; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qhGfQJ/1tmgLlZlyqT2rU+eJDb8Vl/eOR9kvn1/xgAk=; b=Oxa81qr4bzVtofzce1JkvMmPPMCf3mBdTgw60OY6AiXokdSZSAodbdxIV3Ou6V3tjCSX9c VpAQqdpJku+EAjacsSlzIGe9Yisll2KH+B2j5oZoDLYRRgF7KZm4sNTLzt/T3ERYiZniTc 7LRhqm5NdUGcrLQNn6P5WrSVabrKN0k= X-MC-Unique: I2wGZJvDNOS4RksiDia57Q-1 X-Mimecast-MFC-AGG-ID: I2wGZJvDNOS4RksiDia57Q_1773962006 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1773962005; x=1774566805; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qhGfQJ/1tmgLlZlyqT2rU+eJDb8Vl/eOR9kvn1/xgAk=; b=Ds4YrvIDUyoZFHkUr7kD4h9KTYJK9CrUhFoIYKEpBTOTkcgMkDbwAVXirf4daN5sgb IZ8OPFja0MwpuQWQc0kKnltaKyxCW19tohZ8e7BJnwTAqfv038LbsUqz7r20OxtWbLhe nV9VfA1PXiFE2AnnJA75njVpLl45Y6Tp/f6mmh4OjUNOG6+pFpRJGR72DmyyOvzBu52Z aG9lu979kHencMQO3fRtb5i8ZVss3/2l0kbYbAokI7cjJKWdJmzaJyjApIRs/VOvpFJU FBhtRJu4Mtjvd7R4mpUiERVoadq9gB/ehE+Krc7cpkZ3rLh5xZl6czn2i8oZ57ErjzPo 5pDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773962005; x=1774566805; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=qhGfQJ/1tmgLlZlyqT2rU+eJDb8Vl/eOR9kvn1/xgAk=; b=RWUER4bGm/zi2x2mOYh7V4AL312GrCkwjtvRMRDE/Il6THWBTt8G881lICDYf+4NRE 87R6GTMyfpRKRIfBreDvOlRTlNsRvBC23j4Iils60ujv0BlbNK6G/SNc7rOzNcO9dQcH 1sYhHWlIAUCgAGaYo+hIxoLhIcB+kz/6GRu9bIVGQuEOHPIMBpBaHaBYDyokaLVkqXe1 pjPMnTyJsq4f/SX1t88k4pz8jyBT+qY08bfkqsNJ7QT1LveaBnO7H5z31JwfsnQjZ66x dTSCIUGX0hvGT4nR/1WzYejqVd0JfkiURtlClHAd/Dxn3iwAgNt/2XK4IrEEYibGE/qS b8cQ== X-Gm-Message-State: AOJu0YxJC6kERdO0jUkBKpjKfhXZwscIi7zbtZAKikqK49J4c4koUbBj X4ayX/HA3miYwldkqDH8pfPifMsiMZgg450Zfsno6J1EHFwn6lWSDXULSk5ygyvhUmBdd77rCy2 JC9NxxmUC5i6qaVyhWwjcv4jlW501xYlLHqVRSb5qxt4iAZ78olwud1C7vvdf+T0ozNBoJxGLe3 e8b4yeyw2YOVVnxFv0E79TjX23Wa29H+GQCPL3UA== X-Gm-Gg: ATEYQzzdKTVpxSJnmT+gsQCkxgwUcJiMOJ98hVIcotheTeXZiDLX47VuNCoso1Zg86k YWBfH/ZybPlY7MEWTwtNh9+XMVfq+8+GBpaLgR5G++Z6QRNGi6ZhtrzQP3isxRi7hunm4C5ORJn nzu479aa9As+hE9Z2iTR7yB+OYjXum4R6zOBQNvesgddBnmnLQ5GVT64gwXncGrbA8jyvfhreDO G0eK5SY37HLza8PfMDoT/q+nJ/AZYJ08GSsby4meXKgDR64oEKR0ZfkIJsuilwtPItkZPBxrSNq m7S10X92AvigUy1IgR1Ocv4VRNKEN4l05+NHbYtHr6Vuh+3tgOFnXlt6i/qp7wb38qmIKdCqU+6 /I36EI6hSlF8CQw== X-Received: by 2002:ac8:7dcb:0:b0:506:baf9:5dd9 with SMTP id d75a77b69052e-50b2461b5c6mr75315851cf.17.1773962005186; Thu, 19 Mar 2026 16:13:25 -0700 (PDT) X-Received: by 2002:ac8:7dcb:0:b0:506:baf9:5dd9 with SMTP id d75a77b69052e-50b2461b5c6mr75315341cf.17.1773962004654; Thu, 19 Mar 2026 16:13:24 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juraj Marcin , Kirti Wankhede , "Maciej S . Szmigiero" , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Joao Martins , Alex Williamson , Yishai Hadas , Fabiano Rosas , Pranav Tyagi , peterx@redhat.com, Zhiyi Guo , Markus Armbruster , Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PATCH RFC 08/12] vfio/migration: Fix incorrect reporting for VFIO pending data Date: Thu, 19 Mar 2026 19:12:58 -0400 Message-ID: <20260319231302.123135-9-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260319231302.123135-1-peterx@redhat.com> References: <20260319231302.123135-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.819, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.903, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1773962065125154100 Content-Type: text/plain; charset="utf-8" VFIO used to report different things in its fast/slow version of query pending results. It was likely because it wants to make sure precopy data can reach 0 hence trigger sync queries. Now with stopcopy size reporting facility it doesn't need this hack anymore. Fix this. Copy stable might be too much; just skip it and skip the Fixes. Cc: Avihai Horon Signed-off-by: Peter Xu --- hw/vfio/migration.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index c054c749b0..9dbe5ad9e9 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -591,6 +591,10 @@ static void vfio_state_pending_sync(VFIODevice *vbased= ev) __func__, migration->precopy_init_size, migration->precopy_dirty_size, migration->stopcopy_size); + migration->stopcopy_size =3D 0; + } else { + migration->stopcopy_size -=3D + (migration->precopy_init_size + migration->precopy_dirty_size); } } =20 @@ -598,19 +602,18 @@ static void vfio_state_pending(void *opaque, MigPendi= ngData *pending) { VFIODevice *vbasedev =3D opaque; VFIOMigration *migration =3D vbasedev->migration; - uint64_t remain; =20 if (pending->fastpath) { if (!vfio_device_state_is_precopy(vbasedev)) { return; } - remain =3D migration->precopy_init_size + migration->precopy_dirty= _size; } else { vfio_state_pending_sync(vbasedev); - remain =3D migration->stopcopy_size; } =20 - pending->precopy_bytes +=3D remain; + pending->precopy_bytes +=3D + migration->precopy_init_size + migration->precopy_dirty_size; + pending->stopcopy_bytes +=3D migration->stopcopy_size; =20 trace_vfio_state_pending(vbasedev->name, migration->stopcopy_size, migration->precopy_init_size, --=20 2.50.1 From nobody Sun Mar 22 14:17:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1773962124; cv=none; d=zohomail.com; s=zohoarc; b=VglldzMID8hmMp0v0V8ykkg6CYAGk1IDjQCtPkp0Us+9KbBUUXbZFH8fM/cUM/SCbAFoTvTbU0X1iC7BF9t5v4Qoo7RxWh8RzOrtTFmnyTtrHCMxb8lGIgkdLTYWplje3NVKACwg5Ugi8IoyPBV6ZLBmPRWP7QneGs45pu3Rh3s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773962124; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=bsHlTUEhhNOjys8kon0snbRzLZZAGqbckreegTmuncE=; b=HqwTz4/zM2nZebHEG9KiGwkknhLdz8Ptp2dW47nUzlHpSlAiSViIzvD3j4c9m1yWVLAMwmROQigz1F+uuun7+kkiPr5UcvYGNo3nZKg6zPmurcH0O3pA7FaKUETxlXPGaw98+jZZblCUlCKpBCJD/h6DrNcHd4ni/ZywRLjlu5g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17739621249430.39743463402930956; Thu, 19 Mar 2026 16:15:24 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w3MYX-0001Pt-TR; Thu, 19 Mar 2026 19:13:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYW-0001PZ-Ol for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYV-0001bx-4P for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:32 -0400 Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-235-oDgWHqbxPSGTzaefkCfwYA-1; Thu, 19 Mar 2026 19:13:28 -0400 Received: by mail-qt1-f198.google.com with SMTP id d75a77b69052e-5091ee9f1d8so120914631cf.0 for ; Thu, 19 Mar 2026 16:13:28 -0700 (PDT) Received: from x1.local ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-50b36e5bee3sm6717161cf.21.2026.03.19.16.13.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Mar 2026 16:13:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773962010; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bsHlTUEhhNOjys8kon0snbRzLZZAGqbckreegTmuncE=; b=iR/Vjf4wkM1Phij366R6fM/q4WpUEyYSc11EQJ4tJW7bCLKFALtwkTgLkOrhYP74wR58Ke UmVuiB/acuIWObe3XAM0D7geQmxoz5mTkiKqLsPEeimISNbkKDWDVXKGe5wMdrk+gQxCUW 7hVR7Wrwdiw4EvFsfgmGJgDHdPpqnkE= X-MC-Unique: oDgWHqbxPSGTzaefkCfwYA-1 X-Mimecast-MFC-AGG-ID: oDgWHqbxPSGTzaefkCfwYA_1773962008 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1773962008; x=1774566808; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bsHlTUEhhNOjys8kon0snbRzLZZAGqbckreegTmuncE=; b=tujCAcUQwsBfPjkXU9iPcsuPKfPIsJxF8YEwYJhnHvYEp0zxkZQrG09nb127FcLy36 D2U/R7K8y623lsSj9DtH5QzyEABSa6yq5xMx75zY5BuMXhPm/lnsVdAnqHnUulFGXYXw awsjYrhEVDKWZcEdi+z/spUMsdf+mP4PzqMIVuNnjtQwdXHsTzlnVwy3OuHXDBl/Gs9V NwaUZ8LqPQOPuRkIMzpq4UWVGI1j063g4hP1j+jSgFIhfAF45Ap7RsRPkXZgoDEqbhLa 9N5LMjNAQCjX6ZLuPGrX0VF+CwNWzwzZF8uj5si8SE3yJ1iVOzDYSx2gmwi7PeSnsPJj T/8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773962008; x=1774566808; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=bsHlTUEhhNOjys8kon0snbRzLZZAGqbckreegTmuncE=; b=QmwjI2X9VMI4KvapMaMUnGY6yWfKucxe2bxsXeZ7JaO72ONFZIviym3tr49JhR4Mt2 YZsyME2BJf9ADbiolZHGC3Pf852FE9ckFvxYpEk89SvENGBZfeq6Cn5hhRdWD8WOs2Ra IQ+O+3MrQMgMm6lDoXS85MAdCjrQqck9XT4eGBsQa5RYQV1Xefcf8ydKIXTfB1cZSabV 7XBELB4omrMwTF9C3poD1duQqBPIlbcIhBJ6hKvlzp86nvrSMW5lkOt3UV8lFMoDSkaA TcgQ6ZGBZtdWTYtzgXYasjF4bWoBmK1zNU+gFAYgB7pD2SxWJo5cCNFRCR3Vch9KhkX7 si1w== X-Gm-Message-State: AOJu0YwhT6MLp8HCFAyEi5svTKDaNSa/clHSqI221qJ68KVSPL2QPC7S t6FCqvGROOkwKrpB5Qbkh1PXEM8MKbukB7YSk9quNLEtanKkxXfDBVzi1P/jPnf19CNQInVbs80 9j8j7NBiyMvmspUseM6nJ/JcvVJwSrhBHcWUoUYcx/VL75ue4VYgxbsQaS4KmuUnS9yLC/IRXB7 Ae0w4QHo/ohBg/dv0Lwpg+DM7LsH53ztxkbzI/Wg== X-Gm-Gg: ATEYQzwmxVypaLObHiySIf0UUEUTiZH4BK8OtejvTGEMkYRcgtwsgtkRB1AgN5r1TnH Zam6vC+EHcb+kHZfHSqX2rYtqdfcDk15NljVJFvJoMCnAEku8/Tv2FKUpZfsUMXGc4M0SrCzRRv k8rg5Qu2vGWH8WoyEIKGHmwoU7hqTipJj1Pl7hYvQ9muMx1Oc9Ha56GvFPkM/DVE/dOoup7zsxo ew5VvtxEWUc9bHn47rYGuMadDlwUdeMIT9u/tbe5rZQdGhqPDL/tLzTr8zPJW7HqAolLChxUcCh pEa1gXae/lmiJ19VviphGTWxfTM4PAAJn4LT/uPWSc3xnnBAHQO5uNhY8gn8xQZhdwTBnbJZ7bI sjlG3ko8gNy5v6A== X-Received: by 2002:a05:622a:1181:b0:509:205:230f with SMTP id d75a77b69052e-50b3702fcbfmr15596941cf.4.1773962007876; Thu, 19 Mar 2026 16:13:27 -0700 (PDT) X-Received: by 2002:a05:622a:1181:b0:509:205:230f with SMTP id d75a77b69052e-50b3702fcbfmr15596521cf.4.1773962007255; Thu, 19 Mar 2026 16:13:27 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juraj Marcin , Kirti Wankhede , "Maciej S . Szmigiero" , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Joao Martins , Alex Williamson , Yishai Hadas , Fabiano Rosas , Pranav Tyagi , peterx@redhat.com, Zhiyi Guo , Markus Armbruster , Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= , Yong Huang Subject: [PATCH RFC 09/12] migration: Make iteration counter out of RAM Date: Thu, 19 Mar 2026 19:12:59 -0400 Message-ID: <20260319231302.123135-10-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260319231302.123135-1-peterx@redhat.com> References: <20260319231302.123135-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.819, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.903, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1773962125384158500 Content-Type: text/plain; charset="utf-8" It used to hide in RAM dirty sync path. Now with more modules being able to slow sync on dirty information, keeping it there may not be good anymore because it's not RAM's own concept for iterations: all modules should follow. More importantly, mgmt may try to query dirty info (to make policy decisions like adjusting downtime) by listening to iteration count changes via QMP events. So we must make sure the boost of iterations only happens _after_ the dirty sync operations with whatever form (RAM's dirty bitmap sync, or VFIO's different ioctls to fetch latest dirty info from kernel). Move this to core migration path to manage, together with the event generation, so that it can be well ordered with the sync operations for all modules. This brings a good side effect that we should have an old issue regarding to cpu_throttle_dirty_sync_timer_tick() which can randomly boost iteration counts (because it invokes sync ops). Now it won't, which is actually the right behavior. Said that, we have code (not only QEMU, but likely mgmt too) assuming the 1st iteration will always shows dirty count to 1. Make it initialized with 1 this time, because we'll miss the dirty sync for setup() on boosting this counter now. Cc: Yong Huang Signed-off-by: Peter Xu Reviewed-by: Hyman Huang Reviewed-by: Prasad Pandit --- migration/migration-stats.h | 3 ++- migration/migration.c | 29 ++++++++++++++++++++++++++--- migration/ram.c | 6 ------ 3 files changed, 28 insertions(+), 10 deletions(-) diff --git a/migration/migration-stats.h b/migration/migration-stats.h index 1153520f7a..326ddb0088 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -43,7 +43,8 @@ typedef struct { */ uint64_t dirty_pages_rate; /* - * Number of times we have synchronized guest bitmaps. + * Number of times we have synchronized guest bitmaps. This always + * starts from 1 for the 1st iteration. */ uint64_t dirty_sync_count; /* diff --git a/migration/migration.c b/migration/migration.c index 42facb16d1..ad8a824585 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1654,10 +1654,15 @@ int migrate_init(MigrationState *s, Error **errp) s->threshold_size =3D 0; s->switchover_acked =3D false; s->rdma_migration =3D false; + /* - * set mig_stats memory to zero for a new migration + * set mig_stats memory to zero for a new migration.. except the + * iteration counter, which we want to make sure it returns 1 for the + * first iteration. */ memset(&mig_stats, 0, sizeof(mig_stats)); + mig_stats.dirty_sync_count =3D 1; + migration_reset_vfio_bytes_transferred(); =20 s->postcopy_package_loaded =3D false; @@ -3230,10 +3235,28 @@ static bool migration_iteration_next_ready(Migratio= nState *s, static void migration_iteration_go_next(MigPendingData *pending) { /* - * Do a slow sync will achieve this. TODO: move RAM iteration code - * into the core layer. + * Do a slow sync first before boosting the iteration count. */ qemu_savevm_query_pending(pending, false); + + /* + * Boost dirty sync count to reflect we finished one iteration. + * + * NOTE: we need to make sure when this happens (together with the + * event sent below) all modules have slow-synced the pending data + * above. That means a write mem barrier, but qatomic_add() should be + * enough. + * + * It's because a mgmt could wait on the iteration event to query again + * on pending data for policy changes (e.g. downtime adjustments). The + * ordering will make sure the query will fetch the latest results from + * all the modules. + */ + qatomic_add(&mig_stats.dirty_sync_count, 1); + + if (migrate_events()) { + qapi_event_send_migration_pass(mig_stats.dirty_sync_count); + } } =20 /* diff --git a/migration/ram.c b/migration/ram.c index 89f761a471..29e9608715 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1136,8 +1136,6 @@ static void migration_bitmap_sync(RAMState *rs, bool = last_stage) RAMBlock *block; int64_t end_time; =20 - qatomic_add(&mig_stats.dirty_sync_count, 1); - if (!rs->time_last_bitmap_sync) { rs->time_last_bitmap_sync =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIM= E); } @@ -1172,10 +1170,6 @@ static void migration_bitmap_sync(RAMState *rs, bool= last_stage) rs->num_dirty_pages_period =3D 0; rs->bytes_xfer_prev =3D migration_transferred_bytes(); } - if (migrate_events()) { - uint64_t generation =3D qatomic_read(&mig_stats.dirty_sync_count); - qapi_event_send_migration_pass(generation); - } } =20 void migration_bitmap_sync_precopy(bool last_stage) --=20 2.50.1 From nobody Sun Mar 22 14:17:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1773962051; cv=none; d=zohomail.com; s=zohoarc; b=XTxmhWxutx9l70UFVvWoBUZba5z9WkxMa1fQxcsazDo0iFoF1E5cacCQUTmsU/dOOVhlajrZOga2wEA/Hz3d/Gjl8MWsbC+IF7wGbhUugdWnd4qNc1tYG2SVPj4mZ5U0Avnq9JgsmUFG73BQ1ifWXyRbZh8xynMm8s2zJiRHRPI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773962051; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=1ZTk3pexOhw1yn+1jhPOuPpjgi4ezlZc3e3A0v5mt6c=; b=XDoYS7Ev45qNE14oZFyTUDEHztlPjbVFuBtYaW1YKoEFhDttSx8VOkepbv0xaZ4VoeMkgd4kg82NsLpiatDW3OmiV2/5EMWZlbS8A24GQTNpsL3r6cvDGVbrvhp+8UQLc2MafiZGCmV6Mhows8v81dxcjLF+St9BHsv8/OUlfY0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773962051003716.739499205539; Thu, 19 Mar 2026 16:14:11 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w3MYY-0001QB-TR; Thu, 19 Mar 2026 19:13:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYX-0001Pk-I6 for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYV-0001c4-V4 for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:33 -0400 Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-348-NbPma5CCPAy7x8ilbiBhmA-1; Thu, 19 Mar 2026 19:13:30 -0400 Received: by mail-qt1-f198.google.com with SMTP id d75a77b69052e-50911c94db1so83492001cf.0 for ; Thu, 19 Mar 2026 16:13:30 -0700 (PDT) Received: from x1.local ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-50b36e5bee3sm6717161cf.21.2026.03.19.16.13.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Mar 2026 16:13:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773962011; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1ZTk3pexOhw1yn+1jhPOuPpjgi4ezlZc3e3A0v5mt6c=; b=HrdFn5/kaIA0AUgp9xh9QkcdksrGfdoaArTtJjn7thFfNzqK2IjyyvYhep6HjYb5XEqSLC u4g/OInhhzFlqiouiy/e6+dbSKuTPNRo1HgavLV4fEvb9FZ56U0h40y+JHfWaWUh+pVTRt /synGlJ9lyjFbdWjcSf+vJn3rIFX4VY= X-MC-Unique: NbPma5CCPAy7x8ilbiBhmA-1 X-Mimecast-MFC-AGG-ID: NbPma5CCPAy7x8ilbiBhmA_1773962010 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1773962009; x=1774566809; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1ZTk3pexOhw1yn+1jhPOuPpjgi4ezlZc3e3A0v5mt6c=; b=i8AdNrnFJNpRYwuRJfl6gEXiUD1H9FHvfmeqAvZoHr0Eq7osfPmN3A6IXa4sI/kSUT Rq3k6POBy75nC6MEDo0ywveuTsX4+8LUjkg/9QDPCn1umem2qirKt9XKsdxHU17OVGdK uSQGUgmKjqRWRTvMw202M4AkXWweQw+cu9DplpUXJC8des/tOwhRRbpv2ywQsquhcPn8 IiR2TZQolcQrCHPHR3FAgh81Yoka4AbvfKwtW7CfhDt2MIWgF7lkq/mrKaC2aegoT2xT s4KcZ4AoocDH0CaDY/ThKAovjMhx9MIqHpXo7WNjYZVPwnZIF1qXlLnDUQUljz+lWqN/ P+iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773962009; x=1774566809; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=1ZTk3pexOhw1yn+1jhPOuPpjgi4ezlZc3e3A0v5mt6c=; b=bqwT3Tdjwec/TxWkd91DJBrhej072iYOau7mHTXHa7SiKxloleaCPnP52TWZSbHaPF xSEeWcG0KzU4c6EPJKduSnDWbDe7m+Jk2gWe8WqmyzxtPmRXdwb4TXUruKxqSRnphjlz 3GndJxFwQW1aGgxcprgyrg6675UqJztExCgZQAky3CsXcW2yt53qvBjdhA4nXJrozRC8 1K0FirHboG/ya8W8isXATuH+jlLYq3CL4Lw6hNxHc+pNxGGX+Kbc63E5xwdo4sAF51sg zK3mV74v3jd1Y6bKAV6N3AB5Ij6eqQLHptK3jrApV0wz1QHQK9LYPa4uXzmdtNWGqqLv uKWg== X-Gm-Message-State: AOJu0YyEbheKd67l4g1ha/AznCzTj2DqLCFw/zWU3X+hKfWjgCqht3AC +i9+7CKaV/gLQ4N1ApC9SCvpRWkp/Gfg1rd7FfYiDCAXBtAHTN+MpY9hDCxvTZIwU7OgTq7V97/ O43iDLU2PHBXqhQv5q+EqeLK7iJfoT7fDEpY41hVGbwOy5IhhJNqMoLuM0IlwywOF39UE++ZWU/ jOmznOA9RNqZ2OG4u4aJT1EX0ycLcAUDnlB5NM1g== X-Gm-Gg: ATEYQzwadT5OVcrXACnd4sBPMUQsL5UoUp1hv3NrClN8yZ96fR3wKTgHkQ9VyEgeV+H 07opwkg88Ghhh+0xhaH5OGV+LqL814OZN8Xuooa85rE/D4TDuzAuX/fYT7eRcEeCUuKcTnNu1lJ oa0VL6kw/mxMMJ2oZRW3SRInrFT2gazZlXjzA7ScnRoXTzIJxY7vLicyylHp4PgCcq/l5ETHzwj Rbz7/ZCLjZV700BxH+5V3dLFLG9vjN+a1zUiL8bOjjIQq6P7Bmajr8LEbwqasyu+nkfGzNitYjB B0Dg9rTPoYGcbBP2LEBkx8s/Pcl00YPVYbuCUVpaT85ZtEa042C73jShfk5zjdTrO2LZkcjhrpc Gs1dGmQ2IoENZZg== X-Received: by 2002:a05:622a:1e0b:b0:4ee:13d0:d02b with SMTP id d75a77b69052e-50b37500cdamr14681651cf.50.1773962009566; Thu, 19 Mar 2026 16:13:29 -0700 (PDT) X-Received: by 2002:a05:622a:1e0b:b0:4ee:13d0:d02b with SMTP id d75a77b69052e-50b37500cdamr14681241cf.50.1773962009073; Thu, 19 Mar 2026 16:13:29 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juraj Marcin , Kirti Wankhede , "Maciej S . Szmigiero" , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Joao Martins , Alex Williamson , Yishai Hadas , Fabiano Rosas , Pranav Tyagi , peterx@redhat.com, Zhiyi Guo , Markus Armbruster , Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PATCH RFC 10/12] migration: Introduce a helper to return switchover bw estimate Date: Thu, 19 Mar 2026 19:13:00 -0400 Message-ID: <20260319231302.123135-11-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260319231302.123135-1-peterx@redhat.com> References: <20260319231302.123135-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.819, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.903, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1773962053236158500 Content-Type: text/plain; charset="utf-8" Add a helper to be able to return an estimate of switchover bw, called migration_get_switchover_bw(). Use it to sligitly simplify the current code. This will be used in new codes later to remove expected_downtime. Signed-off-by: Peter Xu --- migration/migration.h | 1 + migration/migration.c | 43 ++++++++++++++++++++++--------------------- 2 files changed, 23 insertions(+), 21 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index b6888daced..bf3ee6cc07 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -586,6 +586,7 @@ void migration_cancel(void); void migration_populate_vfio_info(MigrationInfo *info); void migration_reset_vfio_bytes_transferred(void); void postcopy_temp_page_reset(PostcopyTmpPage *tmp_page); +int64_t migration_downtime_calc_expected(MigrationState *s); =20 /* * Migration thread waiting for return path thread. Return non-zero if an diff --git a/migration/migration.c b/migration/migration.c index ad8a824585..56d605ede9 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -984,6 +984,21 @@ void migrate_send_rp_resume_ack(MigrationIncomingState= *mis, uint32_t value) migrate_send_rp_message(mis, MIG_RP_MSG_RESUME_ACK, sizeof(buf), &buf); } =20 +/* + * Returns the estimated switchover bandwidth (unit: bytes / seconds) + */ +static double migration_get_switchover_bw(MigrationState *s) +{ + uint64_t switchover_bw =3D migrate_avail_switchover_bandwidth(); + + if (switchover_bw) { + /* If user specified, prioritize this value and don't estimate */ + return (double)switchover_bw; + } + + return s->mbps / 8 * 1000 * 1000; +} + bool migration_is_running(void) { MigrationState *s =3D current_migration; @@ -3126,37 +3141,22 @@ static void migration_update_counters(MigrationStat= e *s, { uint64_t transferred, transferred_pages, time_spent; uint64_t current_bytes; /* bytes transferred since the beginning */ - uint64_t switchover_bw; - /* Expected bandwidth when switching over to destination QEMU */ - double expected_bw_per_ms; - double bandwidth; + double switchover_bw; =20 if (current_time < s->iteration_start_time + BUFFER_DELAY) { return; } =20 - switchover_bw =3D migrate_avail_switchover_bandwidth(); current_bytes =3D migration_transferred_bytes(); transferred =3D current_bytes - s->iteration_initial_bytes; time_spent =3D current_time - s->iteration_start_time; - bandwidth =3D (double)transferred / time_spent; - - if (switchover_bw) { - /* - * If the user specified a switchover bandwidth, let's trust the - * user so that can be more accurate than what we estimated. - */ - expected_bw_per_ms =3D (double)switchover_bw / 1000; - } else { - /* If the user doesn't specify bandwidth, we use the estimated */ - expected_bw_per_ms =3D bandwidth; - } - - s->threshold_size =3D expected_bw_per_ms * migrate_downtime_limit(); - s->mbps =3D (((double) transferred * 8.0) / ((double) time_spent / 1000.0)) / 1000.0 / 1000.0; =20 + /* NOTE: only update this after bandwidth (s->mbps) updated */ + switchover_bw =3D migration_get_switchover_bw(s) / 1000; + s->threshold_size =3D switchover_bw * migrate_downtime_limit(); + transferred_pages =3D ram_get_total_transferred_pages() - s->iteration_initial_pages; s->pages_per_second =3D (double) transferred_pages / @@ -3178,7 +3178,8 @@ static void migration_update_counters(MigrationState = *s, =20 trace_migrate_transferred(transferred, time_spent, /* Both in unit bytes/ms */ - bandwidth, switchover_bw / 1000, + (uint64_t)s->mbps, + (uint64_t)switchover_bw, s->threshold_size); } =20 --=20 2.50.1 From nobody Sun Mar 22 14:17:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1773962101; cv=none; d=zohomail.com; s=zohoarc; b=DJD7VwJyXOSK9T6YzWkoICoS467X9bWvuQhI7PjubJL9u44xe4SrX4xDeF6WslhUIjlypTZlJFFw/S5QSqsVfyRiBhSX6M71R59Kp7horXzGDvUJReaUt6Ttc+5m/CLoT3YEAoE/UewEP7Gihy4AhdNn6KoltF47Qx/r00DPyLQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773962101; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=6RAHkTnj5upfaZ8SooBURZ2Qif/9bxeVdrP7OGRyeXk=; b=D6yIsH2Y0us9Vwk1xoKILz/Zg5LErnm5QCNp61LdxrdC8V9n748ZztJZx/qKYKpOv9DKALHb7LBLuMEZhCa7xUT0GkoNsE1dv2HFRJ0PvPAnO6wX2ufeUOYDtiQEPBilkq8vFusaydcy0aSreCjqfXSY6wis9urNFucfRAVI2Yg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773962101591598.5606968541306; Thu, 19 Mar 2026 16:15:01 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w3MYa-0001Qg-Ad; Thu, 19 Mar 2026 19:13:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYZ-0001QT-LC for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYY-0001cN-4N for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:35 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-411-rgZjUUthOei7N9FZm9bKGA-1; Thu, 19 Mar 2026 19:13:32 -0400 Received: by mail-qt1-f199.google.com with SMTP id d75a77b69052e-50937cf66b5so6337291cf.3 for ; Thu, 19 Mar 2026 16:13:32 -0700 (PDT) Received: from x1.local ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-50b36e5bee3sm6717161cf.21.2026.03.19.16.13.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Mar 2026 16:13:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773962013; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6RAHkTnj5upfaZ8SooBURZ2Qif/9bxeVdrP7OGRyeXk=; b=cV7iY6gBJWI64XcB9BhExjZ3Nktsm+SkDSkpqH+GBGW3CwDKi5eyV2VFVzTFZMUVYa/+Tb eEBtFwdDaz35urIzokdolN/yBc90UER6oIKR35ENR/Uo7uXNur3yI/sE0B+F6lr4Pl/rXz BYIbdYaMOmfXyFPTwOR+Gz1dwwbHCJ4= X-MC-Unique: rgZjUUthOei7N9FZm9bKGA-1 X-Mimecast-MFC-AGG-ID: rgZjUUthOei7N9FZm9bKGA_1773962012 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1773962011; x=1774566811; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6RAHkTnj5upfaZ8SooBURZ2Qif/9bxeVdrP7OGRyeXk=; b=bqR1eNad5Fj9d26JyUbmBETAo/HJ31O3wHPCrDlQWKUeKkIVDR1tYgJiS5NqlqaF9F BGMtgzvvLUqPYvWV2DNbeUocKFJeEF8RyvPU4wt3gZW2S9lF/1wtbFExhgsxRmFNjIzX MhQChcPhgPewv+eZWkPllt2dAXq6Vee4JUMrVdXlOx8ORSgHc9z8hWipCR6vrg1GM/b5 xAOlPxNE4euSMoWMd97adNaZ8DlN01KT4xd9s/ezZkqpE1enWEAmp0YRkonP6N8FVim5 rd7bQEBPvPNdgJbb04eExyEqtvjrKazbj6CS6xN38bL++OXFLc2sUElG4Z9AZJmQl41n 5reA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773962011; x=1774566811; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=6RAHkTnj5upfaZ8SooBURZ2Qif/9bxeVdrP7OGRyeXk=; b=Ad75l0AxiZIykufu7FE2K8r4z1C0BstmoNcBjHmgEscy992chu5XRXOfsB0Gvqdbwu YMi9Q0McMmyBkGXeonDwhfKPAYCpeInd/4Tz1RCio8hCnnkwzpvyJ3GzLuhI+Y3myHyY f7YHB5PqmMDSnj1P1u2oHG1jCy+IbvBDRcyHFez3Vl8iReicWTfnvcoadPaeDm+MyGR4 bh+BZyXqCC46RC1qKIwRd18yL/1qE07Yss47oVJygow3+u/GsV+8SG/KuHNXIRwQfqOc VtJUbQfOavDkRWU8ASn3EkMD45e8ldTIZE6EaH60c9yn/D3qnil0Ml44JUuHfrQNTfc+ rZ9A== X-Gm-Message-State: AOJu0YxB1QTxyoadwUj3ANsTBKcbEfLlDznmnO2oLKyf4g15advvMZgN zMDxFQWpAbx8Jws+TLxwY2hMwLBfmPcZy9l131TBk3GW5qKv+ZXtESSpi7zUavfD1f0VIrWBFwT Ir0gYeg8vwN4/yIbx2NcdaEqIZ6L8+Wwi5jSK19/lnVvZUvt6tHqtS3pY5W454TkDaq9JQGzSNo O70AyqjcTSUbEJNs6Fb/kUfn4yp+CygLFQNjNVCQ== X-Gm-Gg: ATEYQzw0Quu6Lmi5g5Pf8ZQcROyttBwGJ4wEbyboXmsOWF8wD+ox0gk03/nfuPThBy0 1P1KAupZCAPbskyQ8H8c5f7Xcsx4+K/BgGi/YuRvRHne6A6BjnfNLVoywnvnUUAeas/nyf6ayQh QK/Vb9ecG5AaSewO9HrK2GYrtAzWX3daVQYs687ZxQgdZohfTDqm0m9c1s3CgUbZ1QAf/KalsOp pzVoMIAUmbi8iK3Z9WU+EcJMWMH2nbo8W3lKE+7yXBza1+jT9nk27HxOPM0iI4c9hpOS+ktYWwM CNnsKlmXlOvW9HWRVijSoSoNIfzYcFR4Wq7FEIO4vorWrJSMy3ncRpIWFhBifDx5rh1c7G304ct AekYY1yo0Sg0GRQ== X-Received: by 2002:a05:622a:1823:b0:501:5152:f49f with SMTP id d75a77b69052e-50b374e59aamr17037791cf.46.1773962011334; Thu, 19 Mar 2026 16:13:31 -0700 (PDT) X-Received: by 2002:a05:622a:1823:b0:501:5152:f49f with SMTP id d75a77b69052e-50b374e59aamr17037201cf.46.1773962010754; Thu, 19 Mar 2026 16:13:30 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juraj Marcin , Kirti Wankhede , "Maciej S . Szmigiero" , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Joao Martins , Alex Williamson , Yishai Hadas , Fabiano Rosas , Pranav Tyagi , peterx@redhat.com, Zhiyi Guo , Markus Armbruster , Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PATCH RFC 11/12] migration: Calculate expected downtime on demand Date: Thu, 19 Mar 2026 19:13:01 -0400 Message-ID: <20260319231302.123135-12-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260319231302.123135-1-peterx@redhat.com> References: <20260319231302.123135-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.819, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.903, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1773962103609154100 Content-Type: text/plain; charset="utf-8" This value does not need to be calculated as frequent. Only calculate it on demand when query-migrate happened. With that we can remove the variable in MigrationState. This paves way for fixing this value to include all modules (not only RAM but others too). Signed-off-by: Peter Xu --- migration/migration.h | 1 - migration/migration.c | 26 ++++++++++++-------------- 2 files changed, 12 insertions(+), 15 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index bf3ee6cc07..ba0f9e0f9c 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -359,7 +359,6 @@ struct MigrationState { /* Timestamp when VM is down (ms) to migrate the last stuff */ int64_t downtime_start; int64_t downtime; - int64_t expected_downtime; bool capabilities[MIGRATION_CAPABILITY__MAX]; int64_t setup_time; =20 diff --git a/migration/migration.c b/migration/migration.c index 56d605ede9..23c78b3a2c 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1041,6 +1041,17 @@ static bool migrate_show_downtime(MigrationState *s) return (s->state =3D=3D MIGRATION_STATUS_COMPLETED) || migration_in_po= stcopy(); } =20 +/* Return expected downtime (unit: milliseconds) */ +int64_t migration_downtime_calc_expected(MigrationState *s) +{ + if (mig_stats.dirty_sync_count <=3D 1) { + return migrate_downtime_limit(); + } + + return 1.0 * mig_stats.dirty_bytes_last_sync / + migration_get_switchover_bw(s) * 1000; +} + static void populate_time_info(MigrationInfo *info, MigrationState *s) { info->has_status =3D true; @@ -1061,7 +1072,7 @@ static void populate_time_info(MigrationInfo *info, M= igrationState *s) info->downtime =3D s->downtime; } else { info->has_expected_downtime =3D true; - info->expected_downtime =3D s->expected_downtime; + info->expected_downtime =3D migration_downtime_calc_expected(s); } } =20 @@ -1649,7 +1660,6 @@ int migrate_init(MigrationState *s, Error **errp) s->mbps =3D 0.0; s->pages_per_second =3D 0.0; s->downtime =3D 0; - s->expected_downtime =3D 0; s->setup_time =3D 0; s->start_postcopy =3D false; s->migration_thread_running =3D false; @@ -3162,16 +3172,6 @@ static void migration_update_counters(MigrationState= *s, s->pages_per_second =3D (double) transferred_pages / (((double) time_spent / 1000.0)); =20 - /* - * if we haven't sent anything, we don't want to - * recalculate. 10000 is a small enough number for our purposes - */ - if (qatomic_read(&mig_stats.dirty_pages_rate) && - transferred > 10000) { - s->expected_downtime =3D - qatomic_read(&mig_stats.dirty_bytes_last_sync) / expected_bw_p= er_ms; - } - migration_rate_reset(); =20 update_iteration_initial_status(s); @@ -3817,8 +3817,6 @@ void migration_start_outgoing(MigrationState *s) bool resume =3D (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_RECOVER_SET= UP); int ret; =20 - s->expected_downtime =3D migrate_downtime_limit(); - if (resume) { /* This is a resumed migration */ rate_limit =3D migrate_max_postcopy_bandwidth(); --=20 2.50.1 From nobody Sun Mar 22 14:17:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1773962063; cv=none; d=zohomail.com; s=zohoarc; b=JI+Frin1loOSw3rUpBTWZV0MBjYufQWdp/MBsyD9YDTkgiisLEdll098T3b8HJanYL6t+fIdrKXIfUXU3fSz7lF+L4R6hQw2pzQhmr/+espPXHJ5fiBXAevaXKySX/sy7Yf2WNp29uPjBD6asrfnNkyoctH7A/PCuZ5BqUEoLo0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773962063; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9JQi7Vu+t66NBcDJkYGi6DnBUrFvI6/hQpALCINNvLc=; b=NcNqombYImQ7eCu616/J6F4N8ToFzA28jS5efwgqjH/7Q5CEQy0C8v4LG1YynN597Jc4Af7deo3qVJGHMHsSb/Mkf1RNkWS0gU1DMIDHrOoWluwgxh1Q7m1QWZYo5zyoP3BnMivnQei2z944tQcxZ7xH4DIrTuD2wouUpVGyL2c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1773962063931563.3614776481223; Thu, 19 Mar 2026 16:14:23 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w3MYd-0001RU-Rs; Thu, 19 Mar 2026 19:13:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYb-0001Qz-PA for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w3MYa-0001cq-8Q for qemu-devel@nongnu.org; Thu, 19 Mar 2026 19:13:37 -0400 Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-593-hSbwh8HSO5uVRAD9QB5GoQ-1; Thu, 19 Mar 2026 19:13:34 -0400 Received: by mail-qv1-f71.google.com with SMTP id 6a1803df08f44-899eacb58a5so7196346d6.1 for ; Thu, 19 Mar 2026 16:13:34 -0700 (PDT) Received: from x1.local ([142.189.10.167]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-50b36e5bee3sm6717161cf.21.2026.03.19.16.13.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Mar 2026 16:13:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773962015; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9JQi7Vu+t66NBcDJkYGi6DnBUrFvI6/hQpALCINNvLc=; b=jCDyBBtxd29JeHNR422vdc6vxsdHc6jt73X7hlGhyosQXYFC5TxZo92TuNUPc1ACCxJVP+ KVY9emOo8UHjdy+tvv3CbCefliW0eM0TvLlY+DrnAlrWtyUYjFFgvTReSy/5XLD7jFWuhB qr+Y6l64vV2jqMsEasR8Ur5TEKxcAOU= X-MC-Unique: hSbwh8HSO5uVRAD9QB5GoQ-1 X-Mimecast-MFC-AGG-ID: hSbwh8HSO5uVRAD9QB5GoQ_1773962014 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1773962013; x=1774566813; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9JQi7Vu+t66NBcDJkYGi6DnBUrFvI6/hQpALCINNvLc=; b=imFri8WHKmcL1H4bqwwDBSJc2grzVsFxdr15q2pw7rZrkvYobzpdqh1KZIMRdZCso7 mdTJs+hS5G78JBvoe50+OEAAS14FBqdmggnCg5Qnmu5YrY15WO2fcN35ualjarJ0yIm/ 7GSdHc/e/tPDl0RH3DyyK8tU0ctQBziH4G1S88KnsVQ/wNW/tPjoDuT4loFjOOYUA4r8 RhOYMZ3RE1bisRxVQ2Ggn0JmnVRo47GsugdvOU/bBKIOPhc4Fe/OXiDRzvHw/R83Ld+4 RVAUGr4/hIUxbttop6Lq743qbsB76moiDtd05KOaZnHzdgHTGcHFaFmsGRV95MMemNY3 D0Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773962013; x=1774566813; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=9JQi7Vu+t66NBcDJkYGi6DnBUrFvI6/hQpALCINNvLc=; b=FsX1uvbWvPwjPbW/veRt26FsQe+s4BKvTr/eawnPmt0B7tAm7Dy2C9Kp0aHIfu8UHD 4C0onFFGwutuMRhV+pYkUqhgu1gYC70xW2cEnA6AqTqUgkVTv3XSATpAM/LJ6VDiohng flJDL2PN8YuFMW1WAhbt4avEvxuH9RYJ2ttO84ngM5M1kUMBTmQA8mmJXHI2RUY8EoZq fLTQEyCeH7E8NpwG/B2liltsNReUS7outCTgFu7azjh5I9THFsqnUaRePSwpYsHD5e4F pK1dWzkcCZHoC2bZmXr36ITkcyQlm8U3PpI0qKlxFhTkRBusLHMvUzzcw2eRiHHeOw/c rPMQ== X-Gm-Message-State: AOJu0YzcY0FZ8MAzDPss4DR+8zOigQEsJ18Me0JbUNl+e5KiRi5Rm595 nEqf1n3divtgkQBoHrIRp1KQKaPrpkXlNikms/5v69Qr973+KCyctH0ah79Ma/PK6fARkc81w+E eG+aDthOCRjaApl0gu1FrJPgD0AXNL+foCtYZUmzcbtEy5iPoStDZK733IZkXEjOAry9lB113Iu H4qSaliHQ4ZKfYkvQr7xoG4/qEJb082o8qKalhaw== X-Gm-Gg: ATEYQzzohKgxCdRp610GPyt2e2oE9xxnrT6UxNwHUICJm8UE2Z9CkouMqXiKlqvHXmC Mu+xzx8KG9m7UyCbRSrRIzNNtUwy2wrhQ9zPeRahRX9ltdmJITFU5L4rYMuQihHHp/sNyy6Noy7 ebSUD+GGmG2J2dl+J10tCja7RWcJwJWv4ABcwnn1Wy8ZqA2qNzcyv27ZuhrQBu5WiL/1ysXmu/f E139FBuVBTccsgnkQw5MypIk8ViFEgds+IGj/SMKpk4/fCPJOlLiMoSjbKOWdYLvjFgHCJeYInc ICWJi82hUyfHPrgrvEO65krdmBC+/lANv47vRgliMETkbH91K668VtrL86B2vKdEGreNGUfUGYk FKR+MbrhfuJfu9g== X-Received: by 2002:a05:622a:199a:b0:509:2b02:c1b5 with SMTP id d75a77b69052e-50b375252ecmr16863811cf.44.1773962013218; Thu, 19 Mar 2026 16:13:33 -0700 (PDT) X-Received: by 2002:a05:622a:199a:b0:509:2b02:c1b5 with SMTP id d75a77b69052e-50b375252ecmr16863301cf.44.1773962012613; Thu, 19 Mar 2026 16:13:32 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juraj Marcin , Kirti Wankhede , "Maciej S . Szmigiero" , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Joao Martins , Alex Williamson , Yishai Hadas , Fabiano Rosas , Pranav Tyagi , peterx@redhat.com, Zhiyi Guo , Markus Armbruster , Avihai Horon , =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= Subject: [PATCH RFC 12/12] migration: Fix calculation of expected_downtime to take VFIO info Date: Thu, 19 Mar 2026 19:13:02 -0400 Message-ID: <20260319231302.123135-13-peterx@redhat.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260319231302.123135-1-peterx@redhat.com> References: <20260319231302.123135-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.819, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.903, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1773962065215154100 Content-Type: text/plain; charset="utf-8" QEMU will provide an expected downtime for the whole system during migration, by remembering the total dirty RAM that we synced the last time, divides the estimated switchover bandwidth. That was flawed when VFIO is taking into account: consider there is a VFIO GPU device that contains GBs of data to migrate during stop phase. Those will not be accounted in this math. Fix it by updating dirty_bytes_last_sync properly only when we go to the next iteration, rather than hide this update in the RAM code. Meanwhile, fetch the total (rather than RAM-only) portion of dirty bytes, so as to include GPU device states too. Update the comment of the field to reflect its new meaning. Now after this change, the expected-downtime to be read from query-migrate should be very accurate even with VFIO devices involved. Signed-off-by: Peter Xu --- migration/migration-stats.h | 10 +++++----- migration/migration.c | 11 ++++++++--- migration/ram.c | 1 - 3 files changed, 13 insertions(+), 9 deletions(-) diff --git a/migration/migration-stats.h b/migration/migration-stats.h index 326ddb0088..14b2773beb 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -31,11 +31,11 @@ */ typedef struct { /* - * Number of bytes that were dirty last time that we synced with - * the guest memory. We use that to calculate the downtime. As - * the remaining dirty amounts to what we know that is still dirty - * since last iteration, not counting what the guest has dirtied - * since we synchronized bitmaps. + * Number of bytes that are still dirty after the last whole-system + * sync on dirty information. We use that to calculate the expected + * downtime. As the remaining dirty amounts to what we know that is + * still dirty since last iteration, not counting what the guest has + * dirtied since then on either RAM or device states. */ uint64_t dirty_bytes_last_sync; /* diff --git a/migration/migration.c b/migration/migration.c index 23c78b3a2c..1c00572d14 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3240,18 +3240,23 @@ static void migration_iteration_go_next(MigPendingD= ata *pending) */ qemu_savevm_query_pending(pending, false); =20 + /* + * Update the dirty information for the whole system for this + * iteration. This value is used to calculate expected downtime. + */ + qatomic_set(&mig_stats.dirty_bytes_last_sync, pending->total_bytes); + /* * Boost dirty sync count to reflect we finished one iteration. * * NOTE: we need to make sure when this happens (together with the * event sent below) all modules have slow-synced the pending data - * above. That means a write mem barrier, but qatomic_add() should be - * enough. + * above and updated corresponding fields (e.g. dirty_bytes_last_sync). * * It's because a mgmt could wait on the iteration event to query again * on pending data for policy changes (e.g. downtime adjustments). The * ordering will make sure the query will fetch the latest results from - * all the modules. + * all the modules on everything. */ qatomic_add(&mig_stats.dirty_sync_count, 1); =20 diff --git a/migration/ram.c b/migration/ram.c index 29e9608715..1bdf121d16 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1148,7 +1148,6 @@ static void migration_bitmap_sync(RAMState *rs, bool = last_stage) RAMBLOCK_FOREACH_NOT_IGNORED(block) { ramblock_sync_dirty_bitmap(rs, block); } - qatomic_set(&mig_stats.dirty_bytes_last_sync, ram_bytes_remain= ing()); } } =20 --=20 2.50.1