From nobody Wed May  7 17:41:29 2025
Delivered-To: importer@patchew.org
Authentication-Results: mx.zohomail.com;
	dkim=pass;
	spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
 permitted sender)
  smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
	dmarc=pass(p=none dis=none)  header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1733446861; cv=none;
	d=zohomail.com; s=zohoarc;
	b=UqIhDmXI3ZXef7NAs0c5nYZ32eaT/nCwtc2qKYMjnI6I8+6rmsZ2Bu9UbhlAa5h8N9JanEif27MFzuf4dVxaByd0Np0t6EHJd1+/10fakvoJC5Lep1Bf6FKLpSo6n9tYwaX3St/4rObznUhSuuxM751sWRelJFN1guB8GzsQjpg=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com;
 s=zohoarc;
	t=1733446861;
 h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To;
	bh=es3XBObQHiGtgZBVTVbSw3WDegXk2q8ebNuAwrDJ0N0=;
	b=O5y/KLN0w0WWPCEOAtK/NWtZCGOvGixymuc410jobQNRboOdMjEVNDPDCu9DH+H/gOZrj1jaqG1VPana+UlkiXmdVAD9ICBumGj/t6lohYf2o0Xhfd5UUEqWkCmSs4v1EAJPTMHahzxuujEIc1MfOR+faMilGeuqNAyRkuAY4lE=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass;
	spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
 permitted sender)
  smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
	dmarc=pass header.from=<peterx@redhat.com> (p=none dis=none)
Return-Path: <qemu-devel-bounces+importer=patchew.org@nongnu.org>
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by
 mx.zohomail.com
	with SMTPS id 1733446861726642.7973142457975;
 Thu, 5 Dec 2024 17:01:01 -0800 (PST)
Received: from localhost ([::1] helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <qemu-devel-bounces@nongnu.org>)
	id 1tJMgE-0003a5-K9; Thu, 05 Dec 2024 19:58:50 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1tJMgD-0003ZD-6A
 for qemu-devel@nongnu.org; Thu, 05 Dec 2024 19:58:49 -0500
Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1tJMg9-0005iM-1i
 for qemu-devel@nongnu.org; Thu, 05 Dec 2024 19:58:48 -0500
Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com
 [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS
 (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id
 us-mta-511-Uv6HYEu0NyCVRxUGyinsqw-1; Thu, 05 Dec 2024 19:58:41 -0500
Received: by mail-qv1-f70.google.com with SMTP id
 6a1803df08f44-6d893f0027fso42036656d6.0
 for <qemu-devel@nongnu.org>; Thu, 05 Dec 2024 16:58:40 -0800 (PST)
Received: from x1n.redhat.com (pool-99-254-114-190.cpe.net.cable.rogers.com.
 [99.254.114.190]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6d8dac016cbsm12635226d6.117.2024.12.05.16.58.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 05 Dec 2024 16:58:38 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
 s=mimecast20190719; t=1733446722;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:
 content-transfer-encoding:content-transfer-encoding:
 in-reply-to:in-reply-to:references:references;
 bh=es3XBObQHiGtgZBVTVbSw3WDegXk2q8ebNuAwrDJ0N0=;
 b=Dn711r+rrs9wHwbEzkNVDvYWJIybIpGIETjizzb9Sb1znKomRCaxlheHQ710RDE793W3Ye
 B+4DAW1HBqvA35h0/Iz6FAKGWMaowLabYfYcrWIBubzqE6ejAWa+XhFTlhQqYiKObpor3+
 KflVG+01NyOhdolNu4OUcDNJZqGf0/Q=
X-MC-Unique: Uv6HYEu0NyCVRxUGyinsqw-1
X-Mimecast-MFC-AGG-ID: Uv6HYEu0NyCVRxUGyinsqw
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1733446720; x=1734051520;
 h=content-transfer-encoding:mime-version:references:in-reply-to
 :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=es3XBObQHiGtgZBVTVbSw3WDegXk2q8ebNuAwrDJ0N0=;
 b=ixnn3rs78YZxjKPeN0ZQW7OS+mfqMdatmQxYe9ASwS8gW4g5K7DVh+lqtd9JnjlHJQ
 qijqaC8/cxbKvMuvlK68cTtEcZsVCFn2isR4qPZxGecGwntLMPgw9JyTfoISEqE0h+dp
 XUgbgoC5CNYP2WGFHPw+kWXluLBv7vT4qXt7roqvfRzmbn29ogHPAHfu5KlCp2weucaJ
 aAbldfFdIsWPrpaDW6Tu04pvCC+EE+5XxioI3Oq1USBLkFh2lH8I0ddwBaBLxXH/LKce
 0jcoch0ya7q+A4hayv685NSi27NcMrYSEWgwE8mLz64ms+OeJ9faJxj8/80WI+PMQL0g
 mBdQ==
X-Gm-Message-State: AOJu0YyXLJ0W/tRMv5csmqtmgjD9K8wfEkWeWspisgpe9iFHuAcbbln5
 0yfbSMQVTkHwfPaTeX0P7g6tBaebJja3qzj5IyDk2cTrnLWkMhPcwAd4gIui96OxMcgYDIlCbwF
 1o/7MSECR5RVeCGHMmMRf31TmMpKnyr3yQGPaZjWECTAb2+X6oNGY/jzwPy5Z22f0D+9mnZ2Bx4
 VnCJ2E+zodtGaRPvWVimiFw3qW4d8Z8rRjTA==
X-Gm-Gg: ASbGnctdT4w+unLpAU7p1Pu3LaeJigmT+JiLXbhOg8yPQX9z9xBwEnaXzu+Qd6GU9oa
 qi+KDE+eRgPzmxlT2ffRMUNkluwKke/+AWsShZlPUU/92Ta3gSqJuFRAGpTEQByDO2Wu4mHwY8r
 fzoYKhsKKx+K67Mz42zSWQCKNBtFrTl+/knYCE+of+7yfE+M3zEVxgMiqKW1r75uthdJNg7V0sT
 y1Ayc4W0GaX8y/pekMW0vQgCdfH09fB7LHS5n/s01SSeE4JuMcSPJXZ3jmkoHB4Qqxqc2EMmBU7
 QHCKEdY1fDxB5hxo0WGSF7Lpdg==
X-Received: by 2002:ad4:5c46:0:b0:6d8:7ed4:336c with SMTP id
 6a1803df08f44-6d8e70a2743mr24885956d6.9.1733446720024;
 Thu, 05 Dec 2024 16:58:40 -0800 (PST)
X-Google-Smtp-Source: 
 AGHT+IEUWRkX/DMHW6ZCiAHXl9KlYYZWdaC3iIDx0eVNrUEzC4vIQRmJBnzXGJUDRyid7bDrenVBIQ==
X-Received: by 2002:ad4:5c46:0:b0:6d8:7ed4:336c with SMTP id
 6a1803df08f44-6d8e70a2743mr24885556d6.9.1733446719594;
 Thu, 05 Dec 2024 16:58:39 -0800 (PST)
From: Peter Xu <peterx@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Maciej S . Szmigiero" <mail@maciej.szmigiero.name>, peterx@redhat.com,
 =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= <clg@redhat.com>,
 Avihai Horon <avihaih@nvidia.com>,
 Alex Williamson <alex.williamson@redhat.com>,
 Fabiano Rosas <farosas@suse.de>, Prasad Pandit <ppandit@redhat.com>
Subject: [PATCH v2 1/7] migration/multifd: Further remove the SYNC on complete
Date: Thu,  5 Dec 2024 19:58:28 -0500
Message-ID: <20241206005834.1050905-2-peterx@redhat.com>
X-Mailer: git-send-email 2.47.0
In-Reply-To: <20241206005834.1050905-1-peterx@redhat.com>
References: <20241206005834.1050905-1-peterx@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17
 as permitted sender) client-ip=209.51.188.17;
 envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org;
 helo=lists.gnu.org;
Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com;
 helo=us-smtp-delivery-124.mimecast.com
X-Spam_score_int: -50
X-Spam_score: -5.1
X-Spam_bar: -----
X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.996,
 DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001,
 RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: qemu-devel@nongnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <https://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=subscribe>
Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org
Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org
X-ZohoMail-DKIM: pass (identity @redhat.com)
X-ZM-MESSAGEID: 1733446863616116600
Content-Type: text/plain; charset="utf-8"

Commit 637280aeb2 ("migration/multifd: Avoid the final FLUSH in
complete()") removed the FLUSH operation on complete() which should avoid
one global sync on destination side, because it's not needed.

However that commit overlooked multifd_ram_flush_and_sync() part of things,
as that's always used together with the FLUSH message on the main channel.

For very old binaries (multifd_flush_after_each_section=3D=3Dtrue), that's
still needed because each EOS received on destination will enforce
all-channel sync once.

For new binaries (multifd_flush_after_each_section=3D=3Dfalse), the flush a=
nd
sync shouldn't be needed just like the FLUSH, with the same reasoning.

Currently, on new binaries we'll still send SYNC messages on multifd
channels, even if FLUSH is omitted at the end.  It'll make all recv threads
hang at SYNC message.

Multifd is still all working fine because luckily recv side cleanup
code (mostly multifd_recv_sync_main()) is smart enough to make sure even if
recv threads are stuck at SYNC it'll get kicked out.  And since this is the
completion phase of migration, nothing else will be sent after the SYNCs.

This may be needed for VFIO in the future because VFIO can have data to
push after ram_save_complete(), hence we don't want the recv thread to be
stuck in SYNC message. Remove this limitation will make src even slightly
faster too to avoid some more code.

Stable branches do not need this patch, as no real bug I can think of that
will go wrong there.. so not attaching Fixes to be clear on the backport
not needed.

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
---
 migration/ram.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 05ff9eb328..7284c34bd8 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -3283,9 +3283,16 @@ static int ram_save_complete(QEMUFile *f, void *opaq=
ue)
         }
     }
=20
-    ret =3D multifd_ram_flush_and_sync();
-    if (ret < 0) {
-        return ret;
+    if (migrate_multifd() &&
+        migrate_multifd_flush_after_each_section()) {
+        /*
+         * Only the old dest QEMU will need this sync, because each EOS
+         * will require one SYNC message on each channel.
+         */
+        ret =3D multifd_ram_flush_and_sync();
+        if (ret < 0) {
+            return ret;
+        }
     }
=20
     if (migrate_mapped_ram()) {
--=20
2.47.0
From nobody Wed May  7 17:41:29 2025
Delivered-To: importer@patchew.org
Authentication-Results: mx.zohomail.com;
	dkim=pass;
	spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
 permitted sender)
  smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
	dmarc=pass(p=none dis=none)  header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1733446779; cv=none;
	d=zohomail.com; s=zohoarc;
	b=IBnURJdyVbtmxftpOPqyUDYpwi2LOcT1PK9eWbPlBfvwqbDyo1Jgx0HJcJ4jGfI/+7SJzk9y/mVsuUvTEtRX5odNPkKxw+gWEcIFJmhJ5o7Ogo0+gOHUdTuXXGvp00xcAf7YfZ/DCWKNQ2g/opytxpP1IdRBwss7ii5Ucu9DY5M=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com;
 s=zohoarc;
	t=1733446779;
 h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To;
	bh=of03+bL8atAVQ8Sx7MKVcupnwu+B9QuMvu36EUykEjA=;
	b=krEA663oVy8VLUqmN4SbIcmTNnAHMP8okiRay97fbCdilUbETIveyTB7MAMB9JSlF9Zdb6RnslqLZigPcQRriAiwE4EvQIKoat4r3kA0F17zW+ERp/iQXL1l6XqaUY3Z5QLB8Z1Lu/1hb9Fqx6jh+ZRKVuerGtLf2+3mhCKMhgM=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass;
	spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
 permitted sender)
  smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
	dmarc=pass header.from=<peterx@redhat.com> (p=none dis=none)
Return-Path: <qemu-devel-bounces+importer=patchew.org@nongnu.org>
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by
 mx.zohomail.com
	with SMTPS id 1733446779218282.7807690623714;
 Thu, 5 Dec 2024 16:59:39 -0800 (PST)
Received: from localhost ([::1] helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <qemu-devel-bounces@nongnu.org>)
	id 1tJMgF-0003aS-O0; Thu, 05 Dec 2024 19:58:51 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1tJMgD-0003ZA-5D
 for qemu-devel@nongnu.org; Thu, 05 Dec 2024 19:58:49 -0500
Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1tJMg9-0005ki-T9
 for qemu-devel@nongnu.org; Thu, 05 Dec 2024 19:58:48 -0500
Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com
 [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS
 (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id
 us-mta-411-gVYBKsjnOAO6leXwWcEFDQ-1; Thu, 05 Dec 2024 19:58:43 -0500
Received: by mail-qv1-f70.google.com with SMTP id
 6a1803df08f44-6d88cde9cedso28575866d6.2
 for <qemu-devel@nongnu.org>; Thu, 05 Dec 2024 16:58:43 -0800 (PST)
Received: from x1n.redhat.com (pool-99-254-114-190.cpe.net.cable.rogers.com.
 [99.254.114.190]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6d8dac016cbsm12635226d6.117.2024.12.05.16.58.39
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 05 Dec 2024 16:58:40 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
 s=mimecast20190719; t=1733446725;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:
 content-transfer-encoding:content-transfer-encoding:
 in-reply-to:in-reply-to:references:references;
 bh=of03+bL8atAVQ8Sx7MKVcupnwu+B9QuMvu36EUykEjA=;
 b=NHOV1QF4BJsqGRRu1Z5+5vs11Lzxjjg8jSVL361T3IrIsldtisNt9xeWkpTfiD36XxzXbT
 k+hnnIlKjG968G4TdRac5EZY3ZLm/j9E8e7uC0OZpv6KVmLzQhHusnkTYA2JwY7LFEFLc8
 B100Hca/+6PWoc/QN9bH8OleU5cbeWQ=
X-MC-Unique: gVYBKsjnOAO6leXwWcEFDQ-1
X-Mimecast-MFC-AGG-ID: gVYBKsjnOAO6leXwWcEFDQ
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1733446722; x=1734051522;
 h=content-transfer-encoding:mime-version:references:in-reply-to
 :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=of03+bL8atAVQ8Sx7MKVcupnwu+B9QuMvu36EUykEjA=;
 b=AIYWUoIHyEvMwCcVc5D/uSWcaJhgW4oicaWBkQnchzFDB91BvgOerB4HXbRVYrqR2L
 mKYJfKxNDzXX6TNF8KxLBAMzfGNU0EDLutNWlHTXGC18kisFvY4VBbw6+xsXZMe2umGb
 x6qnsoltrudSlI5zpDlIePsjye/PQK8z8LjxFoZqTBQW+yOEjZz/YMwOMnmBicf6EEQh
 Ox/JGuiRwpCx5SB3Yzv0BifExmZl2mxeIjNFuTAtZFCw7OEId406o/Tg3ZC+qE5Zcgf0
 TxiuocmMbXStxvGz1dyAG3Yaar0BY0pZq7Ejl0g71zu+1R+aTbEvGudIYoB6Zb6keQUE
 3vsg==
X-Gm-Message-State: AOJu0YxDoT9ZeDibxCCvi2pnwmTdfdqQc3hHYotb62FfdG8zvGmQw6oR
 TpoqzMTFyDxUYWx0agTmQ/0IttywsD6bOZXx7Na0XWifhuLHkAN67j9rbEy+gJ/wcdd2kKWrO9H
 hbF4DTdStAtz7B6iS7WC2MTb7DIs111bmr6CcQ/oP3+3Gr9RyCCzaeWynvbymr+8M/7RMG2n+Po
 9gIWaxiE4v0y5LkQLch11N/tfwuSTX+GimmQ==
X-Gm-Gg: ASbGncvRCTWaufHCucyN+GJXfqzKOdD2DssNITbofsR2liIR6TJtY3Xs4QI6/PWcs0X
 h2qYeitCm6nLPegHO3dX0mqz0dmfj4gXHEJSZcNvWFs/sgt+YKwSER2/7t5lVN5WM0ryK89Acjr
 oo2vsEQmrSISHNjCyAkF7Y9DgDchaAmaoWOPOtY3/OpYypaWO4WqeJ0OcQyPS1bKouVNjaEvIwy
 XaX1GZZ7fdIL1rStBSvxdQfSpRC1tBWFuLkgCeMLcBIhlvUc2DpqoKk9+vPTre/0mzltBpd6fyU
 sApipkAXcNYyOBdkTSC/sAnM+A==
X-Received: by 2002:a05:6214:226b:b0:6d8:80e8:d567 with SMTP id
 6a1803df08f44-6d8e7171871mr19171456d6.18.1733446722285;
 Thu, 05 Dec 2024 16:58:42 -0800 (PST)
X-Google-Smtp-Source: 
 AGHT+IGHqSvKx0Fdgv1BuBiyzutcmeiZvriVu0GpdIByj+uT5tm34qvjtlNTSxZPTwVTAIRnWWSCGw==
X-Received: by 2002:a05:6214:226b:b0:6d8:80e8:d567 with SMTP id
 6a1803df08f44-6d8e7171871mr19171086d6.18.1733446721867;
 Thu, 05 Dec 2024 16:58:41 -0800 (PST)
From: Peter Xu <peterx@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Maciej S . Szmigiero" <mail@maciej.szmigiero.name>, peterx@redhat.com,
 =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= <clg@redhat.com>,
 Avihai Horon <avihaih@nvidia.com>,
 Alex Williamson <alex.williamson@redhat.com>,
 Fabiano Rosas <farosas@suse.de>, Prasad Pandit <ppandit@redhat.com>
Subject: [PATCH v2 2/7] migration/multifd: Allow to sync with sender threads
 only
Date: Thu,  5 Dec 2024 19:58:29 -0500
Message-ID: <20241206005834.1050905-3-peterx@redhat.com>
X-Mailer: git-send-email 2.47.0
In-Reply-To: <20241206005834.1050905-1-peterx@redhat.com>
References: <20241206005834.1050905-1-peterx@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17
 as permitted sender) client-ip=209.51.188.17;
 envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org;
 helo=lists.gnu.org;
Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com;
 helo=us-smtp-delivery-124.mimecast.com
X-Spam_score_int: -50
X-Spam_score: -5.1
X-Spam_bar: -----
X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.996,
 DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001,
 RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: qemu-devel@nongnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <https://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=subscribe>
Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org
Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org
X-ZohoMail-DKIM: pass (identity @redhat.com)
X-ZM-MESSAGEID: 1733446781732116600
Content-Type: text/plain; charset="utf-8"

Teach multifd_send_sync_main() to sync with threads only.

We already have such requests, which is when mapped-ram is enabled with
multifd.  In that case, no SYNC messages will be pushed to the stream when
multifd syncs the sender threads because there's no destination threads
waiting for that.  The whole point of the sync is to make sure all threads
flushed their jobs.

So fundamentally we have a request to do the sync in different ways:

  - Either to sync the threads only,
  - Or to sync the threads but also with the destination side.

Mapped-ram did it already because of the use_packet check in the sync
handler of the sender thread.  It works.

However it may stop working when e.g. VFIO may start to reuse multifd
channels to push device states.  In that case VFIO has similar request on
"thread-only sync" however we can't check a flag because such sync request
can still come from RAM which needs the on-wire notifications.

Paving way for that by allowing the multifd_send_sync_main() to specify
what kind of sync the caller needs.  We can use it for mapped-ram already.

No functional change intended.

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 migration/multifd.h        | 19 ++++++++++++++++---
 migration/multifd-nocomp.c |  7 ++++++-
 migration/multifd.c        | 15 +++++++++------
 3 files changed, 31 insertions(+), 10 deletions(-)

diff --git a/migration/multifd.h b/migration/multifd.h
index 50d58c0c9c..bd337631ec 100644
--- a/migration/multifd.h
+++ b/migration/multifd.h
@@ -19,6 +19,18 @@
 typedef struct MultiFDRecvData MultiFDRecvData;
 typedef struct MultiFDSendData MultiFDSendData;
=20
+typedef enum {
+    /* No sync request */
+    MULTIFD_SYNC_NONE =3D 0,
+    /* Sync locally on the sender threads without pushing messages */
+    MULTIFD_SYNC_LOCAL,
+    /*
+     * Sync not only on the sender threads, but also push "SYNC" message to
+     * the wire (which is for a remote sync).
+     */
+    MULTIFD_SYNC_ALL,
+} MultiFDSyncReq;
+
 bool multifd_send_setup(void);
 void multifd_send_shutdown(void);
 void multifd_send_channel_created(void);
@@ -28,7 +40,7 @@ void multifd_recv_shutdown(void);
 bool multifd_recv_all_channels_created(void);
 void multifd_recv_new_channel(QIOChannel *ioc, Error **errp);
 void multifd_recv_sync_main(void);
-int multifd_send_sync_main(void);
+int multifd_send_sync_main(MultiFDSyncReq req);
 bool multifd_queue_page(RAMBlock *block, ram_addr_t offset);
 bool multifd_recv(void);
 MultiFDRecvData *multifd_get_recv_data(void);
@@ -143,7 +155,7 @@ typedef struct {
     /* multifd flags for each packet */
     uint32_t flags;
     /*
-     * The sender thread has work to do if either of below boolean is set.
+     * The sender thread has work to do if either of below field is set.
      *
      * @pending_job:  a job is pending
      * @pending_sync: a sync request is pending
@@ -152,7 +164,8 @@ typedef struct {
      * cleared by the multifd sender threads.
      */
     bool pending_job;
-    bool pending_sync;
+    MultiFDSyncReq pending_sync;
+
     MultiFDSendData *data;
=20
     /* thread local variables. No locking required */
diff --git a/migration/multifd-nocomp.c b/migration/multifd-nocomp.c
index 55191152f9..219f9e58ef 100644
--- a/migration/multifd-nocomp.c
+++ b/migration/multifd-nocomp.c
@@ -345,6 +345,8 @@ retry:
=20
 int multifd_ram_flush_and_sync(void)
 {
+    MultiFDSyncReq req;
+
     if (!migrate_multifd()) {
         return 0;
     }
@@ -356,7 +358,10 @@ int multifd_ram_flush_and_sync(void)
         }
     }
=20
-    return multifd_send_sync_main();
+    /* File migrations only need to sync with threads */
+    req =3D migrate_mapped_ram() ? MULTIFD_SYNC_LOCAL : MULTIFD_SYNC_ALL;
+
+    return multifd_send_sync_main(req);
 }
=20
 bool multifd_send_prepare_common(MultiFDSendParams *p)
diff --git a/migration/multifd.c b/migration/multifd.c
index 498e71fd10..2248bd2d46 100644
--- a/migration/multifd.c
+++ b/migration/multifd.c
@@ -523,7 +523,7 @@ static int multifd_zero_copy_flush(QIOChannel *c)
     return ret;
 }
=20
-int multifd_send_sync_main(void)
+int multifd_send_sync_main(MultiFDSyncReq req)
 {
     int i;
     bool flush_zero_copy;
@@ -543,8 +543,8 @@ int multifd_send_sync_main(void)
          * We should be the only user so far, so not possible to be set by
          * others concurrently.
          */
-        assert(qatomic_read(&p->pending_sync) =3D=3D false);
-        qatomic_set(&p->pending_sync, true);
+        assert(qatomic_read(&p->pending_sync) =3D=3D MULTIFD_SYNC_NONE);
+        qatomic_set(&p->pending_sync, req);
         qemu_sem_post(&p->sem);
     }
     for (i =3D 0; i < migrate_multifd_channels(); i++) {
@@ -635,14 +635,17 @@ static void *multifd_send_thread(void *opaque)
              */
             qatomic_store_release(&p->pending_job, false);
         } else {
+            MultiFDSyncReq req =3D qatomic_read(&p->pending_sync);
+
             /*
              * If not a normal job, must be a sync request.  Note that
              * pending_sync is a standalone flag (unlike pending_job), so
              * it doesn't require explicit memory barriers.
              */
-            assert(qatomic_read(&p->pending_sync));
+            assert(req !=3D MULTIFD_SYNC_NONE);
=20
-            if (use_packets) {
+            /* Only push the SYNC message if it involves a remote sync */
+            if (req =3D=3D MULTIFD_SYNC_ALL) {
                 p->flags =3D MULTIFD_FLAG_SYNC;
                 multifd_send_fill_packet(p);
                 ret =3D qio_channel_write_all(p->c, (void *)p->packet,
@@ -654,7 +657,7 @@ static void *multifd_send_thread(void *opaque)
                 stat64_add(&mig_stats.multifd_bytes, p->packet_len);
             }
=20
-            qatomic_set(&p->pending_sync, false);
+            qatomic_set(&p->pending_sync, MULTIFD_SYNC_NONE);
             qemu_sem_post(&p->sem_sync);
         }
     }
--=20
2.47.0
From nobody Wed May  7 17:41:29 2025
Delivered-To: importer@patchew.org
Authentication-Results: mx.zohomail.com;
	dkim=pass;
	spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
 permitted sender)
  smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
	dmarc=pass(p=none dis=none)  header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1733446819; cv=none;
	d=zohomail.com; s=zohoarc;
	b=H3zZVAplSh8dTD380daM9XhTMuWKSbJ/3Tc6HZ3+OFQbZhr3bLKOVQiGpdICepLqGoaV6X41xTkOeKYO7iVdNnwHi04e4qrVMCBhePEOVuksUlw2U4JDn295IXPGwR79suF9L47MPe6kmAHPN1Dv0jOlZhsLBXWbogtzw/aOjJE=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com;
 s=zohoarc;
	t=1733446819;
 h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To;
	bh=eNEQZ+vPm4wSXR/zObKahxluS5nzwSvUVA2E+LmCCII=;
	b=AlGSeIi1NhSfUvl5qPLy+M0YT9q9G06ZIyi118eaCWVmZ1gpJVCMCmcaVvgCBuym6wVQOPkPA2ZdZplktdt+BfdHZKJ8vUgaY21ueOeSgPMQsNWriwjkvenWhvYRpb0OLTnpwfULk36FatrSi22HLSBF/xlEVSM+xfz2tMA9Wb4=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass;
	spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
 permitted sender)
  smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
	dmarc=pass header.from=<peterx@redhat.com> (p=none dis=none)
Return-Path: <qemu-devel-bounces+importer=patchew.org@nongnu.org>
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by
 mx.zohomail.com
	with SMTPS id 1733446819346995.8550630771593;
 Thu, 5 Dec 2024 17:00:19 -0800 (PST)
Received: from localhost ([::1] helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <qemu-devel-bounces@nongnu.org>)
	id 1tJMgE-0003a6-Nw; Thu, 05 Dec 2024 19:58:50 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1tJMgD-0003ZC-5h
 for qemu-devel@nongnu.org; Thu, 05 Dec 2024 19:58:49 -0500
Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1tJMgB-0005ks-1K
 for qemu-devel@nongnu.org; Thu, 05 Dec 2024 19:58:48 -0500
Received: from mail-vk1-f200.google.com (mail-vk1-f200.google.com
 [209.85.221.200]) by relay.mimecast.com with ESMTP with STARTTLS
 (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id
 us-mta-83-RkZwR9mPMTOSVnF-Mnb7Uw-1; Thu, 05 Dec 2024 19:58:45 -0500
Received: by mail-vk1-f200.google.com with SMTP id
 71dfb90a1353d-515cf20f2c7so376676e0c.0
 for <qemu-devel@nongnu.org>; Thu, 05 Dec 2024 16:58:45 -0800 (PST)
Received: from x1n.redhat.com (pool-99-254-114-190.cpe.net.cable.rogers.com.
 [99.254.114.190]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6d8dac016cbsm12635226d6.117.2024.12.05.16.58.42
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 05 Dec 2024 16:58:43 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
 s=mimecast20190719; t=1733446726;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:
 content-transfer-encoding:content-transfer-encoding:
 in-reply-to:in-reply-to:references:references;
 bh=eNEQZ+vPm4wSXR/zObKahxluS5nzwSvUVA2E+LmCCII=;
 b=ObLgKXlBCcauAmbFid098fLBTPLc98NoOFTnUVb7HgpmtJJM1dKeofkS26TbocgjtTX1pd
 tx/dZ3gedGWqlcVI0wRVHIdOWajL4GYUFnWZOt7/tPDW/n7VOynFICIc9AtY33qbnPdPbH
 U1flHyF5ffFkgtUBZbtpTrCgNH6Kspo=
X-MC-Unique: RkZwR9mPMTOSVnF-Mnb7Uw-1
X-Mimecast-MFC-AGG-ID: RkZwR9mPMTOSVnF-Mnb7Uw
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1733446724; x=1734051524;
 h=content-transfer-encoding:mime-version:references:in-reply-to
 :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=eNEQZ+vPm4wSXR/zObKahxluS5nzwSvUVA2E+LmCCII=;
 b=kzSemO+NoDlPzSJlVHz8oQcvNTm6DdKYOhLByR+LlpDUEKQBDj8DmzEGDPZTxjiNNh
 4AtsEn107FpF+mGY3qEf63MmdWlUbsWLxGyEZfAX7PfKH2b8y5XaU5aL0IQPmtbVo7Fb
 RfvRvnU2253dftvN3VTBVPQvVyIXyc2xaPy1Pg/aEYjY5n2HIWz+1lm05+lIk9d1FaRP
 lwNB18XoLa9OOA6Jy+qg4gJYlE9+qEiuhwoMygaFXffXO3SmbjSdsBdWJFjKW/p+/EeL
 itPz6zPDzDb0mqRwSyMoCNMdJX8LDl3Son3XkxkNyqGzW76fE7LPUJ+mZiaRj+o9MCl/
 ro4g==
X-Gm-Message-State: AOJu0Yzoud3N/ad/3Uq70Tan/V4R0FHm4UttfG7zAit9SCmlWrUCMH1Z
 W+xfvqjQvL4guJhzDiymt+cAcwrxFgiFkazH/w+uLqV7jUx3aO1qsfXdrDSGsYxEOZS2Bd025jD
 mKpLqB+RU0DWMAuuGEn0zNEi6VfPMktBO4ig91KoyvbQV2BV5ybBp4rSt9xlGKlLqPi0aGx0Pro
 g7ivVMvixOG5L93SOQH3Af02ZH7cT1tFw34g==
X-Gm-Gg: ASbGncu6Ull6mN2opODZg7i4r3p5B0yauOlziSM1LXpnhqjDXlA38qt1oACINrBnZXU
 uqDC8uf30Am/y+32BB1qD6ruPqU9VhbQUbWIy0r16v59yPl4KoX4VksUXmeLXAVC8BNkm5+Zqo0
 vbPrjyGbOJlbW3B20cGg6lPUAo7GcRwDrpDFHp1twJY9HQ5lmvO2V3yJVYoKouIOdGU1dFFl4OQ
 GpUheaUv/aoDl1G1m/Ulmt/wD4WFn2daeenY32alCRSi06rhKLuwRTVFTE2Rpo3qOTYNkMmSNCE
 5ewep+9JdSwAkEtWQrB8/2B2UA==
X-Received: by 2002:a05:6122:320b:b0:50d:83e1:94fb with SMTP id
 71dfb90a1353d-515fcb17e77mr2258133e0c.12.1733446724507;
 Thu, 05 Dec 2024 16:58:44 -0800 (PST)
X-Google-Smtp-Source: 
 AGHT+IHVeQ36fQkMVtmBstad6PzSsgH1UitLG9zUgtKUvTEJ/gJIZmBcg3Str9mpOaZcVRjl7pN6YA==
X-Received: by 2002:a05:6122:320b:b0:50d:83e1:94fb with SMTP id
 71dfb90a1353d-515fcb17e77mr2258098e0c.12.1733446724038;
 Thu, 05 Dec 2024 16:58:44 -0800 (PST)
From: Peter Xu <peterx@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Maciej S . Szmigiero" <mail@maciej.szmigiero.name>, peterx@redhat.com,
 =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= <clg@redhat.com>,
 Avihai Horon <avihaih@nvidia.com>,
 Alex Williamson <alex.williamson@redhat.com>,
 Fabiano Rosas <farosas@suse.de>, Prasad Pandit <ppandit@redhat.com>
Subject: [PATCH v2 3/7] migration/ram: Move RAM_SAVE_FLAG* into ram.h
Date: Thu,  5 Dec 2024 19:58:30 -0500
Message-ID: <20241206005834.1050905-4-peterx@redhat.com>
X-Mailer: git-send-email 2.47.0
In-Reply-To: <20241206005834.1050905-1-peterx@redhat.com>
References: <20241206005834.1050905-1-peterx@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17
 as permitted sender) client-ip=209.51.188.17;
 envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org;
 helo=lists.gnu.org;
Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com;
 helo=us-smtp-delivery-124.mimecast.com
X-Spam_score_int: -50
X-Spam_score: -5.1
X-Spam_bar: -----
X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.996,
 DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001,
 RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: qemu-devel@nongnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <https://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=subscribe>
Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org
Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org
X-ZohoMail-DKIM: pass (identity @redhat.com)
X-ZM-MESSAGEID: 1733446821281116600
Content-Type: text/plain; charset="utf-8"

Firstly, we're going to use the multifd flag soon in multifd code, so ram.c
isn't gonna work.

Secondly, we have a separate RDMA flag dangling around, which is definitely
not obvious.  There's one comment that helps, but not too much.

We should just put it altogether, so nothing will get overlooked.

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
---
 migration/ram.h  | 25 +++++++++++++++++++++++++
 migration/rdma.h |  7 -------
 migration/ram.c  | 21 ---------------------
 3 files changed, 25 insertions(+), 28 deletions(-)

diff --git a/migration/ram.h b/migration/ram.h
index 0d1981f888..cfdcccd266 100644
--- a/migration/ram.h
+++ b/migration/ram.h
@@ -33,6 +33,31 @@
 #include "exec/cpu-common.h"
 #include "io/channel.h"
=20
+/*
+ * RAM_SAVE_FLAG_ZERO used to be named RAM_SAVE_FLAG_COMPRESS, it
+ * worked for pages that were filled with the same char.  We switched
+ * it to only search for the zero value.  And to avoid confusion with
+ * RAM_SAVE_FLAG_COMPRESS_PAGE just rename it.
+ *
+ * RAM_SAVE_FLAG_FULL was obsoleted in 2009.
+ *
+ * RAM_SAVE_FLAG_COMPRESS_PAGE (0x100) was removed in QEMU 9.1.
+ */
+#define RAM_SAVE_FLAG_FULL     0x01
+#define RAM_SAVE_FLAG_ZERO     0x02
+#define RAM_SAVE_FLAG_MEM_SIZE 0x04
+#define RAM_SAVE_FLAG_PAGE     0x08
+#define RAM_SAVE_FLAG_EOS      0x10
+#define RAM_SAVE_FLAG_CONTINUE 0x20
+#define RAM_SAVE_FLAG_XBZRLE   0x40
+/*
+ * ONLY USED IN RDMA: Whenever this is found in the data stream, the flags
+ * will be passed to rdma functions in the incoming-migration side.
+ */
+#define RAM_SAVE_FLAG_HOOK     0x80
+#define RAM_SAVE_FLAG_MULTIFD_FLUSH    0x200
+/* We can't use any flag that is bigger than 0x200 */
+
 extern XBZRLECacheStats xbzrle_counters;
=20
 /* Should be holding either ram_list.mutex, or the RCU lock. */
diff --git a/migration/rdma.h b/migration/rdma.h
index a8d27f33b8..f55f28bbed 100644
--- a/migration/rdma.h
+++ b/migration/rdma.h
@@ -33,13 +33,6 @@ void rdma_start_incoming_migration(InetSocketAddress *ho=
st_port, Error **errp);
 #define RAM_CONTROL_ROUND     1
 #define RAM_CONTROL_FINISH    3
=20
-/*
- * Whenever this is found in the data stream, the flags
- * will be passed to rdma functions in the incoming-migration
- * side.
- */
-#define RAM_SAVE_FLAG_HOOK     0x80
-
 #define RAM_SAVE_CONTROL_NOT_SUPP -1000
 #define RAM_SAVE_CONTROL_DELAYED  -2000
=20
diff --git a/migration/ram.c b/migration/ram.c
index 7284c34bd8..44010ff325 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -71,27 +71,6 @@
 /***********************************************************/
 /* ram save/restore */
=20
-/*
- * RAM_SAVE_FLAG_ZERO used to be named RAM_SAVE_FLAG_COMPRESS, it
- * worked for pages that were filled with the same char.  We switched
- * it to only search for the zero value.  And to avoid confusion with
- * RAM_SAVE_FLAG_COMPRESS_PAGE just rename it.
- *
- * RAM_SAVE_FLAG_FULL was obsoleted in 2009.
- *
- * RAM_SAVE_FLAG_COMPRESS_PAGE (0x100) was removed in QEMU 9.1.
- */
-#define RAM_SAVE_FLAG_FULL     0x01
-#define RAM_SAVE_FLAG_ZERO     0x02
-#define RAM_SAVE_FLAG_MEM_SIZE 0x04
-#define RAM_SAVE_FLAG_PAGE     0x08
-#define RAM_SAVE_FLAG_EOS      0x10
-#define RAM_SAVE_FLAG_CONTINUE 0x20
-#define RAM_SAVE_FLAG_XBZRLE   0x40
-/* 0x80 is reserved in rdma.h for RAM_SAVE_FLAG_HOOK */
-#define RAM_SAVE_FLAG_MULTIFD_FLUSH    0x200
-/* We can't use any flag that is bigger than 0x200 */
-
 /*
  * mapped-ram migration supports O_DIRECT, so we need to make sure the
  * userspace buffer, the IO operation size and the file offset are
--=20
2.47.0
From nobody Wed May  7 17:41:29 2025
Delivered-To: importer@patchew.org
Authentication-Results: mx.zohomail.com;
	dkim=pass;
	spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
 permitted sender)
  smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
	dmarc=pass(p=none dis=none)  header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1733446779; cv=none;
	d=zohomail.com; s=zohoarc;
	b=n8U13ETyORs+G0Z+BDNCsTZF7L+m0KaMOcGqdJiOqqyVTlN6IS2/m75U/+Hh86+xa5lTB0yujxd4BDpy/qxO6x2gZNOPOVy4s1FeblX8iI8UFTz/8pkkkkWtseySImY2yit3ih0ptxs9Zyahx692dcU4/OM5UVBRTNbQdKrdSbw=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com;
 s=zohoarc;
	t=1733446779;
 h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To;
	bh=Jq9ieJQ9//qq2auSKoavGM6aHp5t8bjDLhHqkdKpCKw=;
	b=NTvJScLPpoe2szqcZ1QoNZbttZrH3/knChIfaZ27sEVkCcHNUTTKjrvlwH9gWr5LpjW7VmcEAthdgoAuleKNi5CnefMf8Aca88C2RZAw3LeatCNJppt3XzgB1xxklWHcI05ESgrkoF/QntQhlyOTEfEpGn2CFNPcEE+gV7Id1Ws=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass;
	spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
 permitted sender)
  smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
	dmarc=pass header.from=<peterx@redhat.com> (p=none dis=none)
Return-Path: <qemu-devel-bounces+importer=patchew.org@nongnu.org>
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by
 mx.zohomail.com
	with SMTPS id 173344677900014.421622668966165;
 Thu, 5 Dec 2024 16:59:39 -0800 (PST)
Received: from localhost ([::1] helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <qemu-devel-bounces@nongnu.org>)
	id 1tJMgJ-0003bN-0f; Thu, 05 Dec 2024 19:58:55 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1tJMgG-0003ah-9O
 for qemu-devel@nongnu.org; Thu, 05 Dec 2024 19:58:52 -0500
Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1tJMgE-0005lQ-Ow
 for qemu-devel@nongnu.org; Thu, 05 Dec 2024 19:58:52 -0500
Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com
 [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS
 (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id
 us-mta-96-FLqV5ex0N4SCPKB4n1bYbA-1; Thu, 05 Dec 2024 19:58:47 -0500
Received: by mail-qv1-f71.google.com with SMTP id
 6a1803df08f44-6d8860ab00dso27077736d6.0
 for <qemu-devel@nongnu.org>; Thu, 05 Dec 2024 16:58:47 -0800 (PST)
Received: from x1n.redhat.com (pool-99-254-114-190.cpe.net.cable.rogers.com.
 [99.254.114.190]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6d8dac016cbsm12635226d6.117.2024.12.05.16.58.44
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 05 Dec 2024 16:58:45 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
 s=mimecast20190719; t=1733446729;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:
 content-transfer-encoding:content-transfer-encoding:
 in-reply-to:in-reply-to:references:references;
 bh=Jq9ieJQ9//qq2auSKoavGM6aHp5t8bjDLhHqkdKpCKw=;
 b=CDjghp9fRAHrEKbPQdu37wARtijK88aRV1yFHpge5Aaf5XKhs1kGYijhrGMvy/aSEIXwJ7
 JMV9snlAmnzbXHihvLoEwI74Nf7Jm18MGja606flne5Gl6mmvRrFokExK2dgA7clCfK9to
 +3p69dnmC64nzso7KdTQj6+0s4VM8+w=
X-MC-Unique: FLqV5ex0N4SCPKB4n1bYbA-1
X-Mimecast-MFC-AGG-ID: FLqV5ex0N4SCPKB4n1bYbA
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1733446726; x=1734051526;
 h=content-transfer-encoding:mime-version:references:in-reply-to
 :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=Jq9ieJQ9//qq2auSKoavGM6aHp5t8bjDLhHqkdKpCKw=;
 b=nnSqCxGXjmxtczW796EPrz6TPdZVV1OuQB9Z21tPjryj6XocYMU1vZAGKqG1vh7a3e
 hib+wnoHVbc7C1CCm3El5B8HB994smDSG9PVo4QpBylYpF1h1piQ9zH+pnpIAasmvtCt
 WDVCDb/rcsls+BcJeXvlDv2cpyo5SYmzK0GYaHGssMVT4MJFTtZ60pVWStsRL3T2uMPK
 3N/tQDLJWMn26T5144xxCtfGbprn/YbynFmcvAR8AQ+MQuZPjAGBe25gqgzTd1Zb/WrM
 2lFRkhD5P3ttMS6aEOgV7Blpz4m6tokiqqJq0zCuIF2joKzI5WYHnejS4WYkLxpxx0RR
 D1yg==
X-Gm-Message-State: AOJu0YwsatRlIYfq+T9xUhFEWK9EqDqQuZkPSl/VlHO0AynB+w094nmi
 dENtjL3X7hXXJQPz/i38Yqp71od6xu1cWEFYLZ7u4nKexhArOXeQVH7lo9jv4FI/kttwkW4zBjG
 axA/CrLJcAoAzCVfJGJipnsfpDQpCJ/I+l2veDBOThTeaAqIZ0MqvhryAitcXGQKl2ro9mnc4Ov
 e1RmSR/jlWvj8OBMxhEkhRh0cNCx7SeF7QMA==
X-Gm-Gg: ASbGncugPWdKy8yPulj/vxxC+AXzTyX21tojODe0xZqvMNHuJL28t5/Pg9qzf9rSsL6
 eljDAEgZ6MmvD8AK0YRiDaWpwgFbGjOi0+V+YapIILtCJ/dEh3rmf1TYusZwVqhKxJjiY2yQPaY
 cXR9iy6qZzvdneSUEEusTz5zJZ9MtWb+Ng1niUpYmm/d85kK9AUbWrHZE6ioL5yziaVBJIsZk3S
 wtOxu/6VqQ9V1OMW4qikezMQrsB1UbYHw/70c5LRIuPeHcRGS4iOyimskL3FCMfzRIyN/aCr1ij
 TOopT45r+aN3tkjGFq3K6sRYzA==
X-Received: by 2002:a05:6214:400b:b0:6d8:8b9d:1502 with SMTP id
 6a1803df08f44-6d8e71ebf0dmr17251036d6.30.1733446726556;
 Thu, 05 Dec 2024 16:58:46 -0800 (PST)
X-Google-Smtp-Source: 
 AGHT+IHXPWxDLHx/ej3m1LYVm/1cVuhXlhPgz4QiSfuXcagOoJw1NDxUyU37WNo3iFCTdLuErwJyhQ==
X-Received: by 2002:a05:6214:400b:b0:6d8:8b9d:1502 with SMTP id
 6a1803df08f44-6d8e71ebf0dmr17250716d6.30.1733446726181;
 Thu, 05 Dec 2024 16:58:46 -0800 (PST)
From: Peter Xu <peterx@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Maciej S . Szmigiero" <mail@maciej.szmigiero.name>, peterx@redhat.com,
 =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= <clg@redhat.com>,
 Avihai Horon <avihaih@nvidia.com>,
 Alex Williamson <alex.williamson@redhat.com>,
 Fabiano Rosas <farosas@suse.de>, Prasad Pandit <ppandit@redhat.com>
Subject: [PATCH v2 4/7] migration/multifd: Unify RAM_SAVE_FLAG_MULTIFD_FLUSH
 messages
Date: Thu,  5 Dec 2024 19:58:31 -0500
Message-ID: <20241206005834.1050905-5-peterx@redhat.com>
X-Mailer: git-send-email 2.47.0
In-Reply-To: <20241206005834.1050905-1-peterx@redhat.com>
References: <20241206005834.1050905-1-peterx@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17
 as permitted sender) client-ip=209.51.188.17;
 envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org;
 helo=lists.gnu.org;
Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com;
 helo=us-smtp-delivery-124.mimecast.com
X-Spam_score_int: -50
X-Spam_score: -5.1
X-Spam_bar: -----
X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.996,
 DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001,
 RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: qemu-devel@nongnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <https://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=subscribe>
Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org
Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org
X-ZohoMail-DKIM: pass (identity @redhat.com)
X-ZM-MESSAGEID: 1733446781680116600
Content-Type: text/plain; charset="utf-8"

RAM_SAVE_FLAG_MULTIFD_FLUSH message should always be correlated to a sync
request on src.  Unify such message into one place, and conditionally send
the message only if necessary.

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
---
 migration/multifd.h        |  2 +-
 migration/multifd-nocomp.c | 27 +++++++++++++++++++++++++--
 migration/ram.c            | 18 ++++--------------
 3 files changed, 30 insertions(+), 17 deletions(-)

diff --git a/migration/multifd.h b/migration/multifd.h
index bd337631ec..c9ae57ea02 100644
--- a/migration/multifd.h
+++ b/migration/multifd.h
@@ -350,7 +350,7 @@ static inline uint32_t multifd_ram_page_count(void)
=20
 void multifd_ram_save_setup(void);
 void multifd_ram_save_cleanup(void);
-int multifd_ram_flush_and_sync(void);
+int multifd_ram_flush_and_sync(QEMUFile *f);
 size_t multifd_ram_payload_size(void);
 void multifd_ram_fill_packet(MultiFDSendParams *p);
 int multifd_ram_unfill_packet(MultiFDRecvParams *p, Error **errp);
diff --git a/migration/multifd-nocomp.c b/migration/multifd-nocomp.c
index 219f9e58ef..58372db0f4 100644
--- a/migration/multifd-nocomp.c
+++ b/migration/multifd-nocomp.c
@@ -20,6 +20,7 @@
 #include "qemu/cutils.h"
 #include "qemu/error-report.h"
 #include "trace.h"
+#include "qemu-file.h"
=20
 static MultiFDSendData *multifd_ram_send;
=20
@@ -343,9 +344,10 @@ retry:
     return true;
 }
=20
-int multifd_ram_flush_and_sync(void)
+int multifd_ram_flush_and_sync(QEMUFile *f)
 {
     MultiFDSyncReq req;
+    int ret;
=20
     if (!migrate_multifd()) {
         return 0;
@@ -361,7 +363,28 @@ int multifd_ram_flush_and_sync(void)
     /* File migrations only need to sync with threads */
     req =3D migrate_mapped_ram() ? MULTIFD_SYNC_LOCAL : MULTIFD_SYNC_ALL;
=20
-    return multifd_send_sync_main(req);
+    ret =3D multifd_send_sync_main(req);
+    if (ret) {
+        return ret;
+    }
+
+    /* If we don't need to sync with remote at all, nothing else to do */
+    if (req =3D=3D MULTIFD_SYNC_LOCAL) {
+        return 0;
+    }
+
+    /*
+     * Old QEMUs don't understand RAM_SAVE_FLAG_MULTIFD_FLUSH, it relies
+     * on RAM_SAVE_FLAG_EOS instead.
+     */
+    if (migrate_multifd_flush_after_each_section()) {
+        return 0;
+    }
+
+    qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_FLUSH);
+    qemu_fflush(f);
+
+    return 0;
 }
=20
 bool multifd_send_prepare_common(MultiFDSendParams *p)
diff --git a/migration/ram.c b/migration/ram.c
index 44010ff325..90811aabd4 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1306,15 +1306,10 @@ static int find_dirty_block(RAMState *rs, PageSearc=
hStatus *pss)
                 (!migrate_multifd_flush_after_each_section() ||
                  migrate_mapped_ram())) {
                 QEMUFile *f =3D rs->pss[RAM_CHANNEL_PRECOPY].pss_channel;
-                int ret =3D multifd_ram_flush_and_sync();
+                int ret =3D multifd_ram_flush_and_sync(f);
                 if (ret < 0) {
                     return ret;
                 }
-
-                if (!migrate_mapped_ram()) {
-                    qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_FLUSH);
-                    qemu_fflush(f);
-                }
             }
=20
             /* Hit the end of the list */
@@ -3044,18 +3039,13 @@ static int ram_save_setup(QEMUFile *f, void *opaque=
, Error **errp)
     }
=20
     bql_unlock();
-    ret =3D multifd_ram_flush_and_sync();
+    ret =3D multifd_ram_flush_and_sync(f);
     bql_lock();
     if (ret < 0) {
         error_setg(errp, "%s: multifd synchronization failed", __func__);
         return ret;
     }
=20
-    if (migrate_multifd() && !migrate_multifd_flush_after_each_section()
-        && !migrate_mapped_ram()) {
-        qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_FLUSH);
-    }
-
     qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
     ret =3D qemu_fflush(f);
     if (ret < 0) {
@@ -3190,7 +3180,7 @@ out:
     if (ret >=3D 0 && migration_is_running()) {
         if (migrate_multifd() && migrate_multifd_flush_after_each_section(=
) &&
             !migrate_mapped_ram()) {
-            ret =3D multifd_ram_flush_and_sync();
+            ret =3D multifd_ram_flush_and_sync(f);
             if (ret < 0) {
                 return ret;
             }
@@ -3268,7 +3258,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu=
e)
          * Only the old dest QEMU will need this sync, because each EOS
          * will require one SYNC message on each channel.
          */
-        ret =3D multifd_ram_flush_and_sync();
+        ret =3D multifd_ram_flush_and_sync(f);
         if (ret < 0) {
             return ret;
         }
--=20
2.47.0
From nobody Wed May  7 17:41:29 2025
Delivered-To: importer@patchew.org
Authentication-Results: mx.zohomail.com;
	dkim=pass;
	spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
 permitted sender)
  smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
	dmarc=pass(p=none dis=none)  header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1733446812; cv=none;
	d=zohomail.com; s=zohoarc;
	b=Q1v82CuCgAAG0tj5tDhHRXd837qMXgfp/nmkpJvyAlrmU8d7laLZZZhJ0kaCS0GOssDcLzaplpLFeaxXyPhSa3axdT0cB9Gehutseg/QDANht0Ghn7NJPJp9pZ+Z8/SCdcdfh9l+Ipd38KEbLn0K/CnPadRWcPFGCrORl+QXb5w=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com;
 s=zohoarc;
	t=1733446812;
 h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To;
	bh=/u1/cF9PD85EfXyRSPB3h8kEgVgxDKJ3kxfHiutEnP8=;
	b=VGGxxA8zgSxEG4W9PjUwfZ3VYCuk4LCDmMyc5RGPj01vrodniUKv2ER2ol06dMPNxt349TKTSj7HiKhTD4RX5/q2apVSwZA9TUtFgXg6LlLybtALuaMvJhzaNSMms46NGklApLTjsFIacszW0vg9sl0yWAQDI1KHYBzaG2CoD3c=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass;
	spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
 permitted sender)
  smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
	dmarc=pass header.from=<peterx@redhat.com> (p=none dis=none)
Return-Path: <qemu-devel-bounces+importer=patchew.org@nongnu.org>
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by
 mx.zohomail.com
	with SMTPS id 1733446812931278.1953260021644;
 Thu, 5 Dec 2024 17:00:12 -0800 (PST)
Received: from localhost ([::1] helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <qemu-devel-bounces@nongnu.org>)
	id 1tJMgI-0003bF-Cd; Thu, 05 Dec 2024 19:58:54 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1tJMgH-0003b5-JQ
 for qemu-devel@nongnu.org; Thu, 05 Dec 2024 19:58:53 -0500
Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1tJMgG-0005lm-8l
 for qemu-devel@nongnu.org; Thu, 05 Dec 2024 19:58:53 -0500
Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com
 [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS
 (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id
 us-mta-500-XaftsRzGMCqkM14pD0zE5g-1; Thu, 05 Dec 2024 19:58:49 -0500
Received: by mail-qv1-f70.google.com with SMTP id
 6a1803df08f44-6d8eb5ea994so573116d6.1
 for <qemu-devel@nongnu.org>; Thu, 05 Dec 2024 16:58:49 -0800 (PST)
Received: from x1n.redhat.com (pool-99-254-114-190.cpe.net.cable.rogers.com.
 [99.254.114.190]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6d8dac016cbsm12635226d6.117.2024.12.05.16.58.46
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 05 Dec 2024 16:58:47 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
 s=mimecast20190719; t=1733446731;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:
 content-transfer-encoding:content-transfer-encoding:
 in-reply-to:in-reply-to:references:references;
 bh=/u1/cF9PD85EfXyRSPB3h8kEgVgxDKJ3kxfHiutEnP8=;
 b=MdlNGzvjZJ7HbHE6DJihTVBDLyMqWp86iyeHxTt4u6rrAiY/iXtDjkzI8TFNU7R5B+5/uX
 vkFdb7z9q3gRxGuJXwC3w73ax1+m959i3bGV91C90RK4mpcWJsq1UM+yh2RrX5/LRKMEf0
 83CcB2QgiJlM0L6sKZjchH1I+TZn540=
X-MC-Unique: XaftsRzGMCqkM14pD0zE5g-1
X-Mimecast-MFC-AGG-ID: XaftsRzGMCqkM14pD0zE5g
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1733446728; x=1734051528;
 h=content-transfer-encoding:mime-version:references:in-reply-to
 :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=/u1/cF9PD85EfXyRSPB3h8kEgVgxDKJ3kxfHiutEnP8=;
 b=PqQ02f9uYSalALG7Rrla1BOTo+2l+mbgcpRQhlM09m8YUIkVlP6mKFKNyxshJl4W3F
 Q8SLut3/MJQUHJoRzdX7Wn+h20Eo2IN8iKSwamI2cDj1hXqLbtRaP0PBZaysyAhSHEf3
 DStWNymYvNzpRahUQQyQpQxToT67IGutKlbQvOJEGO0MsVZSsL2be7OrI2I5WzHxlF4x
 jmHGaKIxXGptNncOf27zdRmmxVDHGtMurBlkpGaoyF09uTOjG0MbsuDFI1F9FNaTLkI7
 TWz/JmLCazkD8I8Tvxho8zR3Pk30StT3FqFGph7zJ25qCC7T6ndcg5D6uNgRZEoLPcyC
 q2mA==
X-Gm-Message-State: AOJu0YyoqZZvHFuuaZDSMtFAs3DIRcpFKTcYHaCBh6Dnw+hTCQXAzeqa
 qqfUrWw5+t4gXfAi7VoCIrllMwofTl6cRtATTl3udN6z2+SGx7acyRgm2GgxKSiQjkgHN8nJGRy
 GA/nGpo3Rt1uk20b9c6qQSfNGbFACIj7xLgeMadGCVQoSfiAyOzAf3TX0Rq4HXVOt4Ilm734h0O
 pvaB8PUw3WcVqDj8Ux+fMLXd3U5M64mDvjbA==
X-Gm-Gg: ASbGncuxrXpNddNrDzuuX6SaD+IDNi3u8gkWlrXSgTn4T1yQmnNwQAmAYhL2d2ZSoEe
 33V4Av1cmnG6kNKCu8tgRTp8zyaECoRx5DnE/weHIPlycXdu7aWTJM4scOuJBbe1Tr3NgbLtDfv
 L3zhqZICDbJqbopEEUzF3G1kCQs/ObjT4x8f2k80cr3u6YJKDWL8lw4HM/EzmQGJwvlabQUASZp
 IV0fe0aFMFbXw2WW8e7khhlzp5J4Gc4c9TUSRtJZN/uWlQ7ex7sZ1rF1UqtcxrjQVq906y3tgm3
 BiaNecbGj6W62pWfCyWYpMTp/Q==
X-Received: by 2002:a05:6214:2684:b0:6d8:9d81:2107 with SMTP id
 6a1803df08f44-6d8e7132dcfmr14903966d6.20.1733446728565;
 Thu, 05 Dec 2024 16:58:48 -0800 (PST)
X-Google-Smtp-Source: 
 AGHT+IEldja1CzrluAZz2bZtPWJ4lXVZKhQwebwnK0Z8A1dFoCgwYTCE/VVZSxC0eIrsP49jtVxyNQ==
X-Received: by 2002:a05:6214:2684:b0:6d8:9d81:2107 with SMTP id
 6a1803df08f44-6d8e7132dcfmr14903706d6.20.1733446728142;
 Thu, 05 Dec 2024 16:58:48 -0800 (PST)
From: Peter Xu <peterx@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Maciej S . Szmigiero" <mail@maciej.szmigiero.name>, peterx@redhat.com,
 =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= <clg@redhat.com>,
 Avihai Horon <avihaih@nvidia.com>,
 Alex Williamson <alex.williamson@redhat.com>,
 Fabiano Rosas <farosas@suse.de>, Prasad Pandit <ppandit@redhat.com>
Subject: [PATCH v2 5/7] migration/multifd: Remove sync processing on postcopy
Date: Thu,  5 Dec 2024 19:58:32 -0500
Message-ID: <20241206005834.1050905-6-peterx@redhat.com>
X-Mailer: git-send-email 2.47.0
In-Reply-To: <20241206005834.1050905-1-peterx@redhat.com>
References: <20241206005834.1050905-1-peterx@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17
 as permitted sender) client-ip=209.51.188.17;
 envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org;
 helo=lists.gnu.org;
Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com;
 helo=us-smtp-delivery-124.mimecast.com
X-Spam_score_int: -50
X-Spam_score: -5.1
X-Spam_bar: -----
X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.996,
 DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001,
 RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: qemu-devel@nongnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <https://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=subscribe>
Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org
Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org
X-ZohoMail-DKIM: pass (identity @redhat.com)
X-ZM-MESSAGEID: 1733446815176116600
Content-Type: text/plain; charset="utf-8"

Multifd never worked with postcopy, at least yet so far.

Remove the sync processing there, because it's confusing, and they should
never appear.  Now if RAM_SAVE_FLAG_MULTIFD_FLUSH is observed, we fail hard
instead of trying to invoke multifd code.

Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
---
 migration/ram.c | 8 --------
 1 file changed, 8 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 90811aabd4..154ff5abd4 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -3772,15 +3772,7 @@ int ram_load_postcopy(QEMUFile *f, int channel)
                                          TARGET_PAGE_SIZE);
             }
             break;
-        case RAM_SAVE_FLAG_MULTIFD_FLUSH:
-            multifd_recv_sync_main();
-            break;
         case RAM_SAVE_FLAG_EOS:
-            /* normal exit */
-            if (migrate_multifd() &&
-                migrate_multifd_flush_after_each_section()) {
-                multifd_recv_sync_main();
-            }
             break;
         default:
             error_report("Unknown combination of migration flags: 0x%x"
--=20
2.47.0
From nobody Wed May  7 17:41:29 2025
Delivered-To: importer@patchew.org
Authentication-Results: mx.zohomail.com;
	dkim=pass;
	spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
 permitted sender)
  smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
	dmarc=pass(p=none dis=none)  header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1733446779; cv=none;
	d=zohomail.com; s=zohoarc;
	b=nP6rpWQRSZSgDrSMm7GK/BzIJVvRpNcd90/4j1nU+qU+s2LI9ItWX+9ScvrKA0P8T9BrC0niEn/DwOv1tXH9siREMh/h/GDYfDf1MPp667NaZIBLYOectmhHL+t7piuh5Z4mJZQgAR7rcSbGufrpBffzfwJUNEOSrg+I5WKIU8E=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com;
 s=zohoarc;
	t=1733446779;
 h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To;
	bh=drjbKA8KuZzlVuJlQXBYiI10JQger84ZUXuJUp7t2qg=;
	b=F+zQRA4zl1RqxaMXpKC+msbLUQvTd5hP5BFuxjyP/FbveeSqO00rC9yKDEPzI1xYCqumQiiK0E4NShEPE//PV2kxqHX+1E7/oTXSuT7TTwRIeb2Nn2y18CENvvIRy8Oe2fN7iIABhZIWbl7DfMESAkcSQzMm+xQz5W2zIukNuzc=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass;
	spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
 permitted sender)
  smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
	dmarc=pass header.from=<peterx@redhat.com> (p=none dis=none)
Return-Path: <qemu-devel-bounces+importer=patchew.org@nongnu.org>
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by
 mx.zohomail.com
	with SMTPS id 1733446779007531.1076079280007;
 Thu, 5 Dec 2024 16:59:39 -0800 (PST)
Received: from localhost ([::1] helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <qemu-devel-bounces@nongnu.org>)
	id 1tJMgK-0003cD-Pj; Thu, 05 Dec 2024 19:58:56 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1tJMgI-0003bG-M1
 for qemu-devel@nongnu.org; Thu, 05 Dec 2024 19:58:54 -0500
Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1tJMgG-0005lw-TJ
 for qemu-devel@nongnu.org; Thu, 05 Dec 2024 19:58:54 -0500
Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com
 [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS
 (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id
 us-mta-180-izdX7eyUPyeu5W1XxmY0_Q-1; Thu, 05 Dec 2024 19:58:51 -0500
Received: by mail-qv1-f71.google.com with SMTP id
 6a1803df08f44-6d87c55ca85so40361286d6.1
 for <qemu-devel@nongnu.org>; Thu, 05 Dec 2024 16:58:51 -0800 (PST)
Received: from x1n.redhat.com (pool-99-254-114-190.cpe.net.cable.rogers.com.
 [99.254.114.190]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6d8dac016cbsm12635226d6.117.2024.12.05.16.58.48
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 05 Dec 2024 16:58:49 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
 s=mimecast20190719; t=1733446732;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:
 content-transfer-encoding:content-transfer-encoding:
 in-reply-to:in-reply-to:references:references;
 bh=drjbKA8KuZzlVuJlQXBYiI10JQger84ZUXuJUp7t2qg=;
 b=Wqei99SUNCGVFa9KdDqNsWCPlLvOa+ZPyuBhzLQ8ZSQnsY5fwbtzi8GFrUzeFikwjSOSut
 Grgpg0/ZeaqgIdKR+sHCGGm96gpX6vDbO3kbzxxd8vrYcCCew0a/VNdlmofv4n+vMpdS8o
 QhgdxSdiFhwN2e9AWyfijXJ507dVGkA=
X-MC-Unique: izdX7eyUPyeu5W1XxmY0_Q-1
X-Mimecast-MFC-AGG-ID: izdX7eyUPyeu5W1XxmY0_Q
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1733446730; x=1734051530;
 h=content-transfer-encoding:mime-version:references:in-reply-to
 :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=drjbKA8KuZzlVuJlQXBYiI10JQger84ZUXuJUp7t2qg=;
 b=bnp2k23FslkSetNSBcwllOb6ViCALGtcb/l/1DfbqrKeXsioU/s7rU74YysXuhSJse
 fiGPjd3tUWt4/XMnSG1Dc5cZ7qcS5oirGRfMHGAh10yLwu4rkgEWVK8kBRwx7pybCe4y
 u+C9BUTXWDrK2oFmkT9fhmxfNnvt3kjw2tz3mrGTzk/ceZwbfiTuChRoZP1DhGeZQzet
 MQq1oZUU8XMx30RjmJeKEKM8yr8w1tZBM2jV4IXTjKIU/GKZgpxZGypDtAlY9aZHj3lA
 AhMapzA4HZ4LlPPcY/+qe2EyF0LOYfaLXSGu19FOiIJdVZ3oPKfjCMeZITt2hdP2TX56
 buMw==
X-Gm-Message-State: AOJu0Yw2JAe5ncSpkVqWkDsSNagoqBbbLMk493UmCUNHutrcnVO0T366
 9/6znFmyPdmyUiddoCPFSA8VOeCZNwWTxz7oy9xN5iv0413NHwWOZQrTIfrJxzuMOvh4ulEhABV
 QEK7IfDFdpr2Ynj95XVyFQ53nkmuJXpYr7EMbHHDKGc+Hp225ZegCoY4d6rQAoBCqjDxZO5iDZH
 WHex45AM79HirE3Nme+zQX5RM6h/jnsWHs9w==
X-Gm-Gg: ASbGncv14r6sBAVIVus9SChLo0ssP9rMR6IcnIBzmPktOy4TQvgACsRpnBlg1USzGGE
 4GiOtPtx9+4cYdrH4zVWE5MqIJvy6iYi3YgyBUY/BHs1BDRa9GpbT8q4LE9XMvOwVsiUkpuJl1/
 D/tuNYPVhlwEtRlzFVv8A2+8m1otniE6JGAGbQFpE+ihNImdTHnj1HzH9O2HRa7GqXXPt+CdDQf
 2QVkZKOetI6v4UE6NpeR2rDD+Y2cupJjrIQ05xIvRFUmACiGJlbAj86sxK/naESBZku7GGOTxqL
 fErH+m3ebn3kfdKuKFxNFwJQNw==
X-Received: by 2002:ad4:5b84:0:b0:6d4:1f6d:695f with SMTP id
 6a1803df08f44-6d8e70a4561mr17971796d6.2.1733446730702;
 Thu, 05 Dec 2024 16:58:50 -0800 (PST)
X-Google-Smtp-Source: 
 AGHT+IHEncKgYAyVwJxQJ2sxQIizv5+JUBBLheVgwUBG7yK1do59UnqfWleTbHlzw2eCjiKn29CnQQ==
X-Received: by 2002:ad4:5b84:0:b0:6d4:1f6d:695f with SMTP id
 6a1803df08f44-6d8e70a4561mr17971506d6.2.1733446730295;
 Thu, 05 Dec 2024 16:58:50 -0800 (PST)
From: Peter Xu <peterx@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Maciej S . Szmigiero" <mail@maciej.szmigiero.name>, peterx@redhat.com,
 =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= <clg@redhat.com>,
 Avihai Horon <avihaih@nvidia.com>,
 Alex Williamson <alex.williamson@redhat.com>,
 Fabiano Rosas <farosas@suse.de>, Prasad Pandit <ppandit@redhat.com>
Subject: [PATCH v2 6/7] migration/multifd: Cleanup src flushes on condition
 check
Date: Thu,  5 Dec 2024 19:58:33 -0500
Message-ID: <20241206005834.1050905-7-peterx@redhat.com>
X-Mailer: git-send-email 2.47.0
In-Reply-To: <20241206005834.1050905-1-peterx@redhat.com>
References: <20241206005834.1050905-1-peterx@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17
 as permitted sender) client-ip=209.51.188.17;
 envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org;
 helo=lists.gnu.org;
Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com;
 helo=us-smtp-delivery-124.mimecast.com
X-Spam_score_int: -50
X-Spam_score: -5.1
X-Spam_bar: -----
X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.996,
 DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001,
 RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: qemu-devel@nongnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <https://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=subscribe>
Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org
Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org
X-ZohoMail-DKIM: pass (identity @redhat.com)
X-ZM-MESSAGEID: 1733446781675116600
Content-Type: text/plain; charset="utf-8"

The src flush condition check is over complicated, and it's getting more
out of control if postcopy will be involved.

In general, we have two modes to do the sync: legacy or modern ways.
Legacy uses per-section flush, modern uses per-round flush.

Mapped-ram always uses the modern, which is per-round.

Introduce two helpers, which can greatly simplify the code, and hopefully
make it readable again.

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 migration/multifd.h        |  2 ++
 migration/multifd-nocomp.c | 42 ++++++++++++++++++++++++++++++++++++++
 migration/ram.c            | 10 +++------
 3 files changed, 47 insertions(+), 7 deletions(-)

diff --git a/migration/multifd.h b/migration/multifd.h
index c9ae57ea02..582040922f 100644
--- a/migration/multifd.h
+++ b/migration/multifd.h
@@ -351,6 +351,8 @@ static inline uint32_t multifd_ram_page_count(void)
 void multifd_ram_save_setup(void);
 void multifd_ram_save_cleanup(void);
 int multifd_ram_flush_and_sync(QEMUFile *f);
+bool multifd_ram_sync_per_round(void);
+bool multifd_ram_sync_per_section(void);
 size_t multifd_ram_payload_size(void);
 void multifd_ram_fill_packet(MultiFDSendParams *p);
 int multifd_ram_unfill_packet(MultiFDRecvParams *p, Error **errp);
diff --git a/migration/multifd-nocomp.c b/migration/multifd-nocomp.c
index 58372db0f4..c1f686c0ce 100644
--- a/migration/multifd-nocomp.c
+++ b/migration/multifd-nocomp.c
@@ -344,6 +344,48 @@ retry:
     return true;
 }
=20
+/*
+ * We have two modes for multifd flushes:
+ *
+ * - Per-section mode: this is the legacy way to flush, it requires one
+ *   MULTIFD_FLAG_SYNC message for each RAM_SAVE_FLAG_EOS.
+ *
+ * - Per-round mode: this is the modern way to flush, it requires one
+ *   MULTIFD_FLAG_SYNC message only for each round of RAM scan.  Normally
+ *   it's paired with a new RAM_SAVE_FLAG_MULTIFD_FLUSH message in network
+ *   based migrations.
+ *
+ * One thing to mention is mapped-ram always use the modern way to sync.
+ */
+
+/* Do we need a per-section multifd flush (legacy way)? */
+bool multifd_ram_sync_per_section(void)
+{
+    if (!migrate_multifd()) {
+        return false;
+    }
+
+    if (migrate_mapped_ram()) {
+        return false;
+    }
+
+    return migrate_multifd_flush_after_each_section();
+}
+
+/* Do we need a per-round multifd flush (modern way)? */
+bool multifd_ram_sync_per_round(void)
+{
+    if (!migrate_multifd()) {
+        return false;
+    }
+
+    if (migrate_mapped_ram()) {
+        return true;
+    }
+
+    return !migrate_multifd_flush_after_each_section();
+}
+
 int multifd_ram_flush_and_sync(QEMUFile *f)
 {
     MultiFDSyncReq req;
diff --git a/migration/ram.c b/migration/ram.c
index 154ff5abd4..5d4bdefe69 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1302,9 +1302,7 @@ static int find_dirty_block(RAMState *rs, PageSearchS=
tatus *pss)
         pss->page =3D 0;
         pss->block =3D QLIST_NEXT_RCU(pss->block, next);
         if (!pss->block) {
-            if (migrate_multifd() &&
-                (!migrate_multifd_flush_after_each_section() ||
-                 migrate_mapped_ram())) {
+            if (multifd_ram_sync_per_round()) {
                 QEMUFile *f =3D rs->pss[RAM_CHANNEL_PRECOPY].pss_channel;
                 int ret =3D multifd_ram_flush_and_sync(f);
                 if (ret < 0) {
@@ -3178,8 +3176,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
=20
 out:
     if (ret >=3D 0 && migration_is_running()) {
-        if (migrate_multifd() && migrate_multifd_flush_after_each_section(=
) &&
-            !migrate_mapped_ram()) {
+        if (multifd_ram_sync_per_section()) {
             ret =3D multifd_ram_flush_and_sync(f);
             if (ret < 0) {
                 return ret;
@@ -3252,8 +3249,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu=
e)
         }
     }
=20
-    if (migrate_multifd() &&
-        migrate_multifd_flush_after_each_section()) {
+    if (multifd_ram_sync_per_section()) {
         /*
          * Only the old dest QEMU will need this sync, because each EOS
          * will require one SYNC message on each channel.
--=20
2.47.0
From nobody Wed May  7 17:41:29 2025
Delivered-To: importer@patchew.org
Authentication-Results: mx.zohomail.com;
	dkim=pass;
	spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
 permitted sender)
  smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
	dmarc=pass(p=none dis=none)  header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1733446812; cv=none;
	d=zohomail.com; s=zohoarc;
	b=ezKrM2CdtJDEnuQpuG7KyQlyak85yEy2TRTudu1DFweuW+ONiGWUyd4qwUIkXtJzyLFaPt4WoiGTNF0aTD9jDfuWVwJUQCJJU8+04Ak/dbDBEzpYjhLJQmKaNwWkFXIhfjj/si5doGPGo2OpYsPPKibif4gZPV98+7RI6zPA1NE=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com;
 s=zohoarc;
	t=1733446812;
 h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To;
	bh=rlpFg1t4p+x/wsjA8Y1ehZs0v0oTzhln8SN7DYaHVqQ=;
	b=JY3W4En8eS+C5gfDnaijx7pwtHFLKfOZ+CY0mOM9zIBxxRX57E4+xL+rAZLvsCdyFSC7LKQ+ukfnfa6boWBmPWJpXtWxiQ6Adz/SsYRA3HM8/f+MZxLaSaj/mwbt2sB+NUgchjgbuR06HzHs0LhPsTy0JWXElydyTfPin4OhbPw=
ARC-Authentication-Results: i=1; mx.zohomail.com;
	dkim=pass;
	spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
 permitted sender)
  smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
	dmarc=pass header.from=<peterx@redhat.com> (p=none dis=none)
Return-Path: <qemu-devel-bounces+importer=patchew.org@nongnu.org>
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by
 mx.zohomail.com
	with SMTPS id 173344681291955.96194162028678;
 Thu, 5 Dec 2024 17:00:12 -0800 (PST)
Received: from localhost ([::1] helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <qemu-devel-bounces@nongnu.org>)
	id 1tJMgN-0003cR-RS; Thu, 05 Dec 2024 19:58:59 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1tJMgM-0003cG-0c
 for qemu-devel@nongnu.org; Thu, 05 Dec 2024 19:58:58 -0500
Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <peterx@redhat.com>) id 1tJMgK-0005md-JJ
 for qemu-devel@nongnu.org; Thu, 05 Dec 2024 19:58:57 -0500
Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com
 [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS
 (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id
 us-mta-318-frkpDalANPSdz5MD5Y2AkA-1; Thu, 05 Dec 2024 19:58:53 -0500
Received: by mail-qv1-f69.google.com with SMTP id
 6a1803df08f44-6d8860ab00dso27078726d6.0
 for <qemu-devel@nongnu.org>; Thu, 05 Dec 2024 16:58:53 -0800 (PST)
Received: from x1n.redhat.com (pool-99-254-114-190.cpe.net.cable.rogers.com.
 [99.254.114.190]) by smtp.gmail.com with ESMTPSA id
 6a1803df08f44-6d8dac016cbsm12635226d6.117.2024.12.05.16.58.50
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 05 Dec 2024 16:58:51 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
 s=mimecast20190719; t=1733446734;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:
 content-transfer-encoding:content-transfer-encoding:
 in-reply-to:in-reply-to:references:references;
 bh=rlpFg1t4p+x/wsjA8Y1ehZs0v0oTzhln8SN7DYaHVqQ=;
 b=bX0J4/XWx8zaeASCxCzz0aAkb7K4G0WRkzzRVVZQ8Hi52RJ2dbw28uQFFOZkVtdZ1oehZe
 biTEGe3+hiDkpfvsdbzifGxtBdiGXQySXtE6aLS1yUavOcHiPB6vQKlPUzN2ZPit1SnfAN
 rBmVhmxPTq3viMVBk+0hnuKjpN2MH7g=
X-MC-Unique: frkpDalANPSdz5MD5Y2AkA-1
X-Mimecast-MFC-AGG-ID: frkpDalANPSdz5MD5Y2AkA
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1733446732; x=1734051532;
 h=content-transfer-encoding:mime-version:references:in-reply-to
 :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=rlpFg1t4p+x/wsjA8Y1ehZs0v0oTzhln8SN7DYaHVqQ=;
 b=gSKkHhJi7cDbbhnRmWHMflARIADtBPBGo+0iXsmTFtXI43AvXmMRbGJCeW2c5p+Ukg
 7S4R5XnHXEBUqjMgL8GT50JDPc7pXi0HkNz4syHSGgabPVLClk6XN3teXDgUVJYa/uwj
 48vHlGUurXJXDuFmXlkiPv2HO0mboqVcrhHvbgDFx0C7NhWFYNTl1Q+LObwGfCZb8X1N
 Xc6mRN979Jt+pnfMJB/baqhnBglP2QEsYuNhsXiRb6z8K5xwH5Hi6YnhjhgmXt/Gjrc0
 R+aDpIVbDW72bQ10zRoqnjynuqOwkhAp/9LVCh/6ntXwhKofrl9NTrpHNl4Dn/UeMO/t
 ySFA==
X-Gm-Message-State: AOJu0YxgQBNgO0GXA2l5K0W6pifDhoPukAEurlYeeehFG3dy6YBINZAq
 X0W8A0eAx0FabrXk1QIWZ9+oVubaahwpSs6trvkjaHZ7YY7e0TuWfuDr5z3IDQxvcPJjTRhigfh
 qJnKYqmVxR9Rs2wWm0gqgTbW5UC3f/LO46JUNZu4anwaV82KDqFJxPqK1myzQvmiIzmZ9nJOTqT
 C8TeTqquSgaWXXcBlIpwubcZlBlMnCXMowqA==
X-Gm-Gg: ASbGncsghqfdP4Y6mmj5+vt0O436ZW400vd3fvRfziBlZNNE2bdtnFea7mXaSlUzitw
 U936/bJQ14HFw7WshP3EDxJV5yDMOK4eQd49K3x/i9qMqQgO8r5172UGqDoeitg3MDW+OQ4Q7pP
 AU6NyuFTYTVAVMTR4+Hu9w5yM7S6ZXz1g+Vq2fvM6cpIN/PVC3illcPpQEZEGDXg5+uODJkQZ1Q
 usBdp1TAp1g6IUxU1mSOI9mAlg3UipDfs9brEWj56h1tvLQGBKlfpfsPI7JnA+WXCE4TkeyoN2h
 Rc6C8Dsq/J0TpoxhmtJDxXRyYg==
X-Received: by 2002:ad4:5745:0:b0:6d8:8a8f:75b0 with SMTP id
 6a1803df08f44-6d8e7128d28mr20532646d6.14.1733446732512;
 Thu, 05 Dec 2024 16:58:52 -0800 (PST)
X-Google-Smtp-Source: 
 AGHT+IEzrpn8ChXzP6fq6lz8dwmTdEkw/KsgeF2Fo83weEfj60YxFPRjaAk05xlnPU2UU1EVdJnDgg==
X-Received: by 2002:ad4:5745:0:b0:6d8:8a8f:75b0 with SMTP id
 6a1803df08f44-6d8e7128d28mr20532266d6.14.1733446732154;
 Thu, 05 Dec 2024 16:58:52 -0800 (PST)
From: Peter Xu <peterx@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Maciej S . Szmigiero" <mail@maciej.szmigiero.name>, peterx@redhat.com,
 =?UTF-8?q?C=C3=A9dric=20Le=20Goater?= <clg@redhat.com>,
 Avihai Horon <avihaih@nvidia.com>,
 Alex Williamson <alex.williamson@redhat.com>,
 Fabiano Rosas <farosas@suse.de>, Prasad Pandit <ppandit@redhat.com>
Subject: [PATCH v2 7/7] migration/multifd: Document the reason to sync for
 save_setup()
Date: Thu,  5 Dec 2024 19:58:34 -0500
Message-ID: <20241206005834.1050905-8-peterx@redhat.com>
X-Mailer: git-send-email 2.47.0
In-Reply-To: <20241206005834.1050905-1-peterx@redhat.com>
References: <20241206005834.1050905-1-peterx@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17
 as permitted sender) client-ip=209.51.188.17;
 envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org;
 helo=lists.gnu.org;
Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com;
 helo=us-smtp-delivery-124.mimecast.com
X-Spam_score_int: -50
X-Spam_score: -5.1
X-Spam_bar: -----
X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.996,
 DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001,
 RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: qemu-devel@nongnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <https://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=subscribe>
Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org
Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org
X-ZohoMail-DKIM: pass (identity @redhat.com)
X-ZM-MESSAGEID: 1733446815229116600
Content-Type: text/plain; charset="utf-8"

It's not straightforward to see why src QEMU needs to sync multifd during
setup() phase.  After all, there's no page queued at that point.

For old QEMUs, there's a solid reason: EOS requires it to work.  While it's
clueless on the new QEMUs which do not take EOS message as sync requests.

One will figure that out only when this is conditionally removed.  In fact,
the author did try it out.  Logically we could still avoid doing this on
new machine types, however that needs a separate compat field and that can
be an overkill in some trivial overhead in setup() phase.

Let's instead document it completely, to avoid someone else tries this
again and do the debug one more time, or anyone confused on why this ever
existed.

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 migration/ram.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/migration/ram.c b/migration/ram.c
index 5d4bdefe69..ddee703585 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -3036,6 +3036,33 @@ static int ram_save_setup(QEMUFile *f, void *opaque,=
 Error **errp)
         migration_ops->ram_save_target_page =3D ram_save_target_page_legac=
y;
     }
=20
+    /*
+     * This operation is unfortunate..
+     *
+     * For legacy QEMUs using per-section sync
+     * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
+     *
+     * This must exist because the EOS below requires the SYNC messages
+     * per-channel to work.
+     *
+     * For modern QEMUs using per-round sync
+     * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
+     *
+     * Logically this sync is not needed (because we know there's nothing
+     * in the multifd queue yet!).  However as a side effect, this makes
+     * sure the dest side won't receive any data before it properly reaches
+     * ram_load_precopy().
+     *
+     * Without this sync, src QEMU can send data too soon so that dest may
+     * not have been ready to receive it (e.g., rb->receivedmap may be
+     * uninitialized, for example).
+     *
+     * Logically "wait for recv setup ready" shouldn't need to involve src
+     * QEMU at all, however to be compatible with old QEMUs, let's stick
+     * with this.  Fortunately the overhead is low to sync during setup
+     * because the VM is running, so at least it's not accounted as part of
+     * downtime.
+     */
     bql_unlock();
     ret =3D multifd_ram_flush_and_sync(f);
     bql_lock();
--=20
2.47.0