From nobody Fri May 17 10:34:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1676279816; cv=none; d=zohomail.com; s=zohoarc; b=NBe/7kLR8Lbzv06BbWfqCRA+8Z0KY5DkwGv5lvY1XR3PluYvtL6e7BtRauIwrRlVKH33DGJAWqJ6TBRXEHIqmhSCvTWUDjTgrsg7GIT2C6/vWY5tE/u1bIpBN6dCJE6K+u4wsdqEfO0uTT0t+TZQO1jrRfPwD5wgFNkWFofj95g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1676279816; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=9biWkcJ8ni5cexIWnW8T5++bGq3zkW5jWZHX1F+Gu7I=; b=dQjv6451ic0ywH4cvhR+Zc/mf7pGJ97s0XEHGyq2zpN8ACpg5omXrJ2JL1FHy/MIbMWI7gf3aggtTIziM6E/UKwMOnejyQLkFL3dHXh7R0VerPyQlMZYkTpFWLKmh//5apeyov20lDjC0zKYCz901Qw7eMvagVmm7MHi1o7oR7k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1676279816658875.0576540266036; Mon, 13 Feb 2023 01:16:56 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pRUwO-0000TL-GV; Mon, 13 Feb 2023 04:16:04 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pRUwM-0000T4-Mu for qemu-devel@nongnu.org; Mon, 13 Feb 2023 04:16:02 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pRUwJ-0001EO-Ev for qemu-devel@nongnu.org; Mon, 13 Feb 2023 04:16:00 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-164-OzZFiekwN7ab8x3q_6JxMA-1; Mon, 13 Feb 2023 04:15:55 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0558129A9CA2; Mon, 13 Feb 2023 09:15:55 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 23E0C140EBF4; Mon, 13 Feb 2023 09:15:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676279758; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9biWkcJ8ni5cexIWnW8T5++bGq3zkW5jWZHX1F+Gu7I=; b=YxWxzt7WpX1PbMmz06vyam2SQO9P+C0+nSl4E1S+zunzSqtxZ+QRBk5tzZh8V3kUslXhfG zyBlgxYnqd3mqBuHhDKLQvDtbF8fexqfE85GlTj35DhqFBh705XWFD55hmZQrd/p6U5Vt7 uOFiYMJ3h8BIjVh3DbicTE4lWTuN910= X-MC-Unique: OzZFiekwN7ab8x3q_6JxMA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , "Dr. David Alan Gilbert" , Eduardo Habkost , Eric Blake , Juan Quintela , Yanan Wang , Marcel Apfelbaum Subject: [PATCH v5 1/3] multifd: Create property multifd-sync-after-each-section Date: Mon, 13 Feb 2023 10:15:46 +0100 Message-Id: <20230213091548.76444-2-quintela@redhat.com> In-Reply-To: <20230213091548.76444-1-quintela@redhat.com> References: <20230213091548.76444-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1676279818802100003 Content-Type: text/plain; charset="utf-8" We used to synchronize all channels at the end of each RAM section sent. That is not needed, so preparing to only synchronize once every full round in latests patches. Notice that we initialize the property as true. We will change the default when we introduce the new mechanism. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Juan Quintela --- Rename each-iteration to after-each-section Signed-off-by: Juan Quintela --- qapi/migration.json | 10 +++++++++- migration/migration.h | 1 + hw/core/machine.c | 1 + migration/migration.c | 15 +++++++++++++-- 4 files changed, 24 insertions(+), 3 deletions(-) diff --git a/qapi/migration.json b/qapi/migration.json index c84fa10e86..2907241b9c 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -478,6 +478,13 @@ # should not affect the correctness of postcopy migrati= on. # (since 7.1) # +# @multifd-sync-after-each-section: Synchronize channels after each +# section is sent. We used to do +# that in the past, but it is +# suboptimal. +# Default value is true until all code i= s in. +# (since 8.0) +# # Features: # @unstable: Members @x-colo and @x-ignore-shared are experimental. # @@ -492,7 +499,8 @@ 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, 'validate-uuid', 'background-snapshot', - 'zero-copy-send', 'postcopy-preempt'] } + 'zero-copy-send', 'postcopy-preempt', + 'multifd-sync-after-each-section'] } =20 ## # @MigrationCapabilityStatus: diff --git a/migration/migration.h b/migration/migration.h index 2da2f8a164..cf84520196 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -424,6 +424,7 @@ int migrate_multifd_channels(void); MultiFDCompression migrate_multifd_compression(void); int migrate_multifd_zlib_level(void); int migrate_multifd_zstd_level(void); +bool migrate_multifd_sync_after_each_section(void); =20 #ifdef CONFIG_LINUX bool migrate_use_zero_copy_send(void); diff --git a/hw/core/machine.c b/hw/core/machine.c index f73fc4c45c..dc86849402 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -54,6 +54,7 @@ const size_t hw_compat_7_1_len =3D G_N_ELEMENTS(hw_compat= _7_1); GlobalProperty hw_compat_7_0[] =3D { { "arm-gicv3-common", "force-8-bit-prio", "on" }, { "nvme-ns", "eui64-default", "on"}, + { "migration", "multifd-sync-after-each-section", "on"}, }; const size_t hw_compat_7_0_len =3D G_N_ELEMENTS(hw_compat_7_0); =20 diff --git a/migration/migration.c b/migration/migration.c index 90fca70cb7..406c27bc82 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -167,7 +167,8 @@ INITIALIZE_MIGRATE_CAPS_SET(check_caps_background_snaps= hot, MIGRATION_CAPABILITY_XBZRLE, MIGRATION_CAPABILITY_X_COLO, MIGRATION_CAPABILITY_VALIDATE_UUID, - MIGRATION_CAPABILITY_ZERO_COPY_SEND); + MIGRATION_CAPABILITY_ZERO_COPY_SEND, + MIGRATION_CAPABILITY_MULTIFD_SYNC_AFTER_EACH_SECTION); =20 /* When we add fault tolerance, we could have several migrations at once. For now we don't need to add @@ -2701,6 +2702,15 @@ bool migrate_use_multifd(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_MULTIFD]; } =20 +bool migrate_multifd_sync_after_each_section(void) +{ + MigrationState *s =3D migrate_get_current(); + + return true; + // We will change this when code gets in. + return s->enabled_capabilities[MIGRATION_CAPABILITY_MULTIFD_SYNC_AFTER= _EACH_SECTION]; +} + bool migrate_pause_before_switchover(void) { MigrationState *s; @@ -4535,7 +4545,8 @@ static Property migration_properties[] =3D { DEFINE_PROP_MIG_CAP("x-zero-copy-send", MIGRATION_CAPABILITY_ZERO_COPY_SEND), #endif - + DEFINE_PROP_MIG_CAP("multifd-sync-after-each-section", + MIGRATION_CAPABILITY_MULTIFD_SYNC_AFTER_EACH_SECTI= ON), DEFINE_PROP_END_OF_LIST(), }; =20 --=20 2.39.1 From nobody Fri May 17 10:34:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1676279800; cv=none; d=zohomail.com; s=zohoarc; b=h/SX4z9w0q5JQ47+HJXXl3z+pVanmWOARio26eIdDvNuLyzOOL5bIrgoiQ/H5mJDP13ntgMqPk7PM9qCNA9FDrOBD/cppdMved56T1i+abUa6EKR5RRgq3ywL5oksV+seL1R4wSTgWYPhxAK6cY29n/gTxupRAu7BWGU6kplvew= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1676279800; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=EgagvUh3aXzsz8z6Rb++uamrbO9P3nso1XrEgp0OBrI=; b=cu5EwOeu2Czvt+KSccuOHFZTBbR/NAKv3CrTtEPSF9sVpwIZkiuwxM808LdH7t/JV1nskk6ZOTTr+Xp0rqLZhRKEVdgk3YXlhcH2BJUnFCtEiFAAL+x0iQ5OJpe9DaEtmrpIPAhnp3ZYr8Vtd//pfgOpkl17j1rF/1r0o+wDsAM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16762798002461013.574417750543; Mon, 13 Feb 2023 01:16:40 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pRUwR-0000U0-H2; Mon, 13 Feb 2023 04:16:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pRUwO-0000Th-MR for qemu-devel@nongnu.org; Mon, 13 Feb 2023 04:16:04 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pRUwM-0001FD-EN for qemu-devel@nongnu.org; Mon, 13 Feb 2023 04:16:04 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-574-QPniTfAWO9qYtuetNlEUXQ-1; Mon, 13 Feb 2023 04:15:57 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2E47385A5A3; Mon, 13 Feb 2023 09:15:57 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 495A1140EBF4; Mon, 13 Feb 2023 09:15:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676279761; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EgagvUh3aXzsz8z6Rb++uamrbO9P3nso1XrEgp0OBrI=; b=FrACLbctGqa7ebnUczarBQgaXuQ0MymeJioS/YyNUbzZci/YzJNBIJdGza3yllFvUZYhlq xv65rHhaI4+Gg1iNmsJHkfVSiT9z/PL0tUb1aKIMJEtIvTSC7+Lyzux5Z2wnky4csfWtZV FE1RJyK8SU5aHX/ljvpEshaftfDYiR8= X-MC-Unique: QPniTfAWO9qYtuetNlEUXQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , "Dr. David Alan Gilbert" , Eduardo Habkost , Eric Blake , Juan Quintela , Yanan Wang , Marcel Apfelbaum Subject: [PATCH v5 2/3] multifd: Protect multifd_send_sync_main() calls Date: Mon, 13 Feb 2023 10:15:47 +0100 Message-Id: <20230213091548.76444-3-quintela@redhat.com> In-Reply-To: <20230213091548.76444-1-quintela@redhat.com> References: <20230213091548.76444-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1676279801046100001 Content-Type: text/plain; charset="utf-8" We only need to do that on the ram_save_iterate() call on sending and on destination when we get a RAM_SAVE_FLAG_EOS. In setup() and complete() we need to synch in both new and old cases, so don't add a check there. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Juan Quintela --- Remove the wrappers that we take out on patch 5. --- migration/ram.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 521912385d..b954a24381 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3408,9 +3408,11 @@ static int ram_save_iterate(QEMUFile *f, void *opaqu= e) out: if (ret >=3D 0 && migration_is_setup_or_active(migrate_get_current()->state)) { - ret =3D multifd_send_sync_main(rs->pss[RAM_CHANNEL_PRECOPY].pss_ch= annel); - if (ret < 0) { - return ret; + if (migrate_multifd_sync_after_each_section()) { + ret =3D multifd_send_sync_main(rs->pss[RAM_CHANNEL_PRECOPY].ps= s_channel); + if (ret < 0) { + return ret; + } } =20 qemu_put_be64(f, RAM_SAVE_FLAG_EOS); @@ -4170,7 +4172,9 @@ int ram_load_postcopy(QEMUFile *f, int channel) =20 case RAM_SAVE_FLAG_EOS: /* normal exit */ - multifd_recv_sync_main(); + if (migrate_multifd_sync_after_each_section()) { + multifd_recv_sync_main(); + } break; default: error_report("Unknown combination of migration flags: 0x%x" @@ -4441,7 +4445,9 @@ static int ram_load_precopy(QEMUFile *f) break; case RAM_SAVE_FLAG_EOS: /* normal exit */ - multifd_recv_sync_main(); + if (migrate_multifd_sync_after_each_section()) { + multifd_recv_sync_main(); + } break; default: if (flags & RAM_SAVE_FLAG_HOOK) { --=20 2.39.1 From nobody Fri May 17 10:34:10 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1676279836; cv=none; d=zohomail.com; s=zohoarc; b=D6fp2tJjzzkMKQvtQOIYhvsALleul6vKsDNvrpyORscMsR6l6mraZ+59n/ESdKAlqqNsBX4oAOFg7/AuZk2bSB/b6Rhy4iBreOasXDjReVEfqAU3VYk14L120LAuctD6pZg5a1QI8V/ySQyPpJtPn0eAO+/ieAzvA6zmuwYc3EA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1676279836; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fWAUq9yJ0goOdL7y7Ca0Sie0/wgT2Fg2qUmspIWAWc0=; b=ck4suEYOmakQHVwn51IDYvF8LpnIwU1UBpmlI4l0E8e4A0dZSr1D64h5INcnXLWgw0eGmNCYEOLKA9PUVKG4hCP4opOsGRwWPMNIa+Yzg9pKcK6+1Mr8F2gx4cVwDDF4T7htiapsdACyhH4i9p+bQGE4KNuqDD4Hj6HIIAcITts= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1676279836749872.0028367281034; Mon, 13 Feb 2023 01:17:16 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pRUwt-0000aL-LP; Mon, 13 Feb 2023 04:16:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pRUwQ-0000U6-5R for qemu-devel@nongnu.org; Mon, 13 Feb 2023 04:16:06 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pRUwO-0001GJ-E9 for qemu-devel@nongnu.org; Mon, 13 Feb 2023 04:16:05 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-347-wfGnDqI0OCaKRMFSXbEvdw-1; Mon, 13 Feb 2023 04:16:00 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 76962101A55E; Mon, 13 Feb 2023 09:15:59 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 74B60140EBF4; Mon, 13 Feb 2023 09:15:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676279763; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fWAUq9yJ0goOdL7y7Ca0Sie0/wgT2Fg2qUmspIWAWc0=; b=Ykcp/hAqEmMt+S1opdiY/7gokivMEXu+b/2h9GrSE/Of5CubEmnbhJaZ14tX2V+kru7S6f o3h1T4+LCmhm2vYbsBeEnBZCKHIesCDcdFa3XmqVXaOoHk/6e/fmxB09TxZPhekoUptlAz 65NcMv2z7bZK5SQ3vPSQcIkUclne3xI= X-MC-Unique: wfGnDqI0OCaKRMFSXbEvdw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , "Dr. David Alan Gilbert" , Eduardo Habkost , Eric Blake , Juan Quintela , Yanan Wang , Marcel Apfelbaum Subject: [PATCH v5 3/3] multifd: Only sync once each full round of memory Date: Mon, 13 Feb 2023 10:15:48 +0100 Message-Id: <20230213091548.76444-4-quintela@redhat.com> In-Reply-To: <20230213091548.76444-1-quintela@redhat.com> References: <20230213091548.76444-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1676279838953100003 Content-Type: text/plain; charset="utf-8" We need to add a new flag to mean to sync at that point. Notice that we still synchronize at the end of setup and at the end of complete stages. Signed-off-by: Juan Quintela --- Add missing qemu_fflush(), now it passes all tests always. --- qapi/migration.json | 2 +- migration/migration.c | 2 -- migration/ram.c | 28 +++++++++++++++++++++++++++- 3 files changed, 28 insertions(+), 4 deletions(-) diff --git a/qapi/migration.json b/qapi/migration.json index 2907241b9c..5d0efa4590 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -482,7 +482,7 @@ # section is sent. We used to do # that in the past, but it is # suboptimal. -# Default value is true until all code i= s in. +# Default value is false. # (since 8.0) # # Features: diff --git a/migration/migration.c b/migration/migration.c index 406c27bc82..e4a66d67d7 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2706,8 +2706,6 @@ bool migrate_multifd_sync_after_each_section(void) { MigrationState *s =3D migrate_get_current(); =20 - return true; - // We will change this when code gets in. return s->enabled_capabilities[MIGRATION_CAPABILITY_MULTIFD_SYNC_AFTER= _EACH_SECTION]; } =20 diff --git a/migration/ram.c b/migration/ram.c index b954a24381..47dab5e1c3 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -85,6 +85,7 @@ #define RAM_SAVE_FLAG_XBZRLE 0x40 /* 0x80 is reserved in qemu-file.h for RAM_SAVE_FLAG_HOOK */ #define RAM_SAVE_FLAG_COMPRESS_PAGE 0x100 +#define RAM_SAVE_FLAG_MULTIFD_SYNC 0x200 /* We can't use any flag that is bigger than 0x200 */ =20 int (*xbzrle_encode_buffer_func)(uint8_t *, uint8_t *, int, @@ -1595,6 +1596,7 @@ retry: * associated with the search process. * * Returns: + * <0: An error happened * PAGE_ALL_CLEAN: no dirty page found, give up * PAGE_TRY_AGAIN: no dirty page found, retry for next block * PAGE_DIRTY_FOUND: dirty page found @@ -1622,6 +1624,15 @@ static int find_dirty_block(RAMState *rs, PageSearch= Status *pss) pss->page =3D 0; pss->block =3D QLIST_NEXT_RCU(pss->block, next); if (!pss->block) { + if (!migrate_multifd_sync_after_each_section()) { + QEMUFile *f =3D rs->pss[RAM_CHANNEL_PRECOPY].pss_channel; + int ret =3D multifd_send_sync_main(f); + if (ret < 0) { + return ret; + } + qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_SYNC); + qemu_fflush(f); + } /* * If memory migration starts over, we will meet a dirtied page * which may still exists in compression threads's ring, so we @@ -2614,6 +2625,9 @@ static int ram_find_and_save_block(RAMState *rs) break; } else if (res =3D=3D PAGE_TRY_AGAIN) { continue; + } else if (res < 0) { + pages =3D res; + break; } } } @@ -3300,6 +3314,10 @@ static int ram_save_setup(QEMUFile *f, void *opaque) return ret; } =20 + if (!migrate_multifd_sync_after_each_section()) { + qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_SYNC); + } + qemu_put_be64(f, RAM_SAVE_FLAG_EOS); qemu_fflush(f); =20 @@ -3485,6 +3503,9 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) return ret; } =20 + if (!migrate_multifd_sync_after_each_section()) { + qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_SYNC); + } qemu_put_be64(f, RAM_SAVE_FLAG_EOS); qemu_fflush(f); =20 @@ -4169,7 +4190,9 @@ int ram_load_postcopy(QEMUFile *f, int channel) } decompress_data_with_multi_threads(f, page_buffer, len); break; - + case RAM_SAVE_FLAG_MULTIFD_SYNC: + multifd_recv_sync_main(); + break; case RAM_SAVE_FLAG_EOS: /* normal exit */ if (migrate_multifd_sync_after_each_section()) { @@ -4443,6 +4466,9 @@ static int ram_load_precopy(QEMUFile *f) break; } break; + case RAM_SAVE_FLAG_MULTIFD_SYNC: + multifd_recv_sync_main(); + break; case RAM_SAVE_FLAG_EOS: /* normal exit */ if (migrate_multifd_sync_after_each_section()) { --=20 2.39.1