From nobody Sun May 19 22:21:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675985885; cv=none; d=zohomail.com; s=zohoarc; b=C6BvireuKliOEYd5kugHrj4Rk9Ik8gkeKIU7GhO3htNoJyf5smS2UWtuOCga1d21615SXN45wPprPXOs535jvMaQTLPg7ziCUxskVLsHmSssUGYMooz9jd90y7W7ssVI7VQJKdoI6xGP/ARXMV+8+UrOWleoR5Rl8sOf4BFILYU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675985885; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Ua8cMzmpaJOPuVqB5nzSuOThYS8MZapnmkQKX8gU5Es=; b=KslckI4HMVgpnDUWOc8r/kGem8a4R9q8Oqrt6osAmJSRCVs1iNGiKy1+xCfMLLVwEODUiyuBKOB2oyIN7vz1lOb3BycPs50okNnLGY+y+6f6VFnQSXWMRG6jw83L5+VFeAmVW9kYZCqm98tdiDx489oq1v91mYhrk3TAQuV/eLQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675985885023868.9208562025887; Thu, 9 Feb 2023 15:38:05 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pQGU5-0002HD-7r; Thu, 09 Feb 2023 18:37:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pQGU0-0002GK-SK for qemu-devel@nongnu.org; Thu, 09 Feb 2023 18:37:43 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pQGTz-0007e0-61 for qemu-devel@nongnu.org; Thu, 09 Feb 2023 18:37:40 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-49-BEBAeClPP_q-NOhgrQlVjQ-1; Thu, 09 Feb 2023 18:37:35 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DCB763C0D864; Thu, 9 Feb 2023 23:37:34 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6CD4D140EBF6; Thu, 9 Feb 2023 23:37:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675985858; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ua8cMzmpaJOPuVqB5nzSuOThYS8MZapnmkQKX8gU5Es=; b=b58DbmMWZUh01IHGY27PLvxT7yeAk6Ia/ZbPzR8ma1255LA+jLaVcQcIsBDtiwWSMwv7fe SaosUjIGMYioaRroQLK8RtGsHYzPErnTAvtrsVRviBnexlbIK4e3jqf1BTMhIe/ekmN0LT xf2TKO6WjNaoWZeBHDcEnEw0HOM+xqc= X-MC-Unique: BEBAeClPP_q-NOhgrQlVjQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Eduardo Habkost , Juan Quintela , Yanan Wang , Markus Armbruster , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Eric Blake , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PATCH v4 1/4] multifd: Create property multifd-sync-after-each-section Date: Fri, 10 Feb 2023 00:37:27 +0100 Message-Id: <20230209233730.38288-2-quintela@redhat.com> In-Reply-To: <20230209233730.38288-1-quintela@redhat.com> References: <20230209233730.38288-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675985886782100003 Content-Type: text/plain; charset="utf-8" We used to synchronize all channels at the end of each RAM section sent. That is not needed, so preparing to only synchronize once every full round in latests patches. Notice that we initialize the property as true. We will change the default when we introduce the new mechanism. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Juan Quintela --- Rename each-iteration to after-each-section Signed-off-by: Juan Quintela --- qapi/migration.json | 10 +++++++++- migration/migration.h | 1 + hw/core/machine.c | 1 + migration/migration.c | 15 +++++++++++++-- 4 files changed, 24 insertions(+), 3 deletions(-) diff --git a/qapi/migration.json b/qapi/migration.json index c84fa10e86..2907241b9c 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -478,6 +478,13 @@ # should not affect the correctness of postcopy migrati= on. # (since 7.1) # +# @multifd-sync-after-each-section: Synchronize channels after each +# section is sent. We used to do +# that in the past, but it is +# suboptimal. +# Default value is true until all code i= s in. +# (since 8.0) +# # Features: # @unstable: Members @x-colo and @x-ignore-shared are experimental. # @@ -492,7 +499,8 @@ 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, 'validate-uuid', 'background-snapshot', - 'zero-copy-send', 'postcopy-preempt'] } + 'zero-copy-send', 'postcopy-preempt', + 'multifd-sync-after-each-section'] } =20 ## # @MigrationCapabilityStatus: diff --git a/migration/migration.h b/migration/migration.h index 2da2f8a164..cf84520196 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -424,6 +424,7 @@ int migrate_multifd_channels(void); MultiFDCompression migrate_multifd_compression(void); int migrate_multifd_zlib_level(void); int migrate_multifd_zstd_level(void); +bool migrate_multifd_sync_after_each_section(void); =20 #ifdef CONFIG_LINUX bool migrate_use_zero_copy_send(void); diff --git a/hw/core/machine.c b/hw/core/machine.c index f73fc4c45c..dc86849402 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -54,6 +54,7 @@ const size_t hw_compat_7_1_len =3D G_N_ELEMENTS(hw_compat= _7_1); GlobalProperty hw_compat_7_0[] =3D { { "arm-gicv3-common", "force-8-bit-prio", "on" }, { "nvme-ns", "eui64-default", "on"}, + { "migration", "multifd-sync-after-each-section", "on"}, }; const size_t hw_compat_7_0_len =3D G_N_ELEMENTS(hw_compat_7_0); =20 diff --git a/migration/migration.c b/migration/migration.c index a5c22e327d..b2844d374f 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -167,7 +167,8 @@ INITIALIZE_MIGRATE_CAPS_SET(check_caps_background_snaps= hot, MIGRATION_CAPABILITY_XBZRLE, MIGRATION_CAPABILITY_X_COLO, MIGRATION_CAPABILITY_VALIDATE_UUID, - MIGRATION_CAPABILITY_ZERO_COPY_SEND); + MIGRATION_CAPABILITY_ZERO_COPY_SEND, + MIGRATION_CAPABILITY_MULTIFD_SYNC_AFTER_EACH_SECTION); =20 /* When we add fault tolerance, we could have several migrations at once. For now we don't need to add @@ -2705,6 +2706,15 @@ bool migrate_use_multifd(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_MULTIFD]; } =20 +bool migrate_multifd_sync_after_each_section(void) +{ + MigrationState *s =3D migrate_get_current(); + + return true; + // We will change this when code gets in. + return s->enabled_capabilities[MIGRATION_CAPABILITY_MULTIFD_SYNC_AFTER= _EACH_SECTION]; +} + bool migrate_pause_before_switchover(void) { MigrationState *s; @@ -4539,7 +4549,8 @@ static Property migration_properties[] =3D { DEFINE_PROP_MIG_CAP("x-zero-copy-send", MIGRATION_CAPABILITY_ZERO_COPY_SEND), #endif - + DEFINE_PROP_MIG_CAP("multifd-sync-after-each-section", + MIGRATION_CAPABILITY_MULTIFD_SYNC_AFTER_EACH_SECTI= ON), DEFINE_PROP_END_OF_LIST(), }; =20 --=20 2.39.1 From nobody Sun May 19 22:21:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675985899; cv=none; d=zohomail.com; s=zohoarc; b=edG/1Kd20KL9xPkKx2Nafd+++qy7+m0Jm7zHJ7wdKhwmHX/KA5CHbM3HiJonO5RW1LWwToaHToc3mcdgfY/tGqXXpi1kHF9oNInx5LLyFma5yOQ6Y6ZhEPdFe690NhRcKEqGzsLhXEUsyqIXADQOnZsVkAtZpnoWkPFVZW5sVB0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675985899; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=OzgODHyOzQWAbk4AHk7gxv4NiJacsm4N2/IzV+CEIKw=; b=Zp/bdmVYynDYCmukTpdiv+HRkl99wmy8CfN1YgD6ae5rny31vo38xMUNaC1syQB9FzizsBC/kisdfB69fXJhKGtnZtHtvqGmWItNSoIJpgHTnkRW0bnSD3pflMD2cYUFdwLAJhE93bMkftN/PjRzOWUKWoibynGuLgWsVb74UIk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675985899481249.8871829524843; Thu, 9 Feb 2023 15:38:19 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pQGU6-0002Hy-LU; Thu, 09 Feb 2023 18:37:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pQGU5-0002HB-3u for qemu-devel@nongnu.org; Thu, 09 Feb 2023 18:37:45 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pQGU3-0007ej-GV for qemu-devel@nongnu.org; Thu, 09 Feb 2023 18:37:44 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-531-62vO5Jn0MjC-t7BJp60MBA-1; Thu, 09 Feb 2023 18:37:37 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9F83D3C0D864; Thu, 9 Feb 2023 23:37:36 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3050E140EBF6; Thu, 9 Feb 2023 23:37:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675985860; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OzgODHyOzQWAbk4AHk7gxv4NiJacsm4N2/IzV+CEIKw=; b=gD53FK2nlpCiokF9QiJijdUKLMzO2U+VuIuUxpFq4hH6hH/95I429YTJFEtfRdkBkR3pHE BneznwQWxEKJgOC/Vw0D5weY+p40A+3pgRL0fNCxwSluIphxLzkSix0Yqo6IiTmqpb3YPZ iAiQzSxqmFg1UcHjAokmhKuR6H+wWRQ= X-MC-Unique: 62vO5Jn0MjC-t7BJp60MBA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Eduardo Habkost , Juan Quintela , Yanan Wang , Markus Armbruster , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Eric Blake , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PATCH v4 2/4] multifd: Protect multifd_send_sync_main() calls Date: Fri, 10 Feb 2023 00:37:28 +0100 Message-Id: <20230209233730.38288-3-quintela@redhat.com> In-Reply-To: <20230209233730.38288-1-quintela@redhat.com> References: <20230209233730.38288-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675985900849100001 Content-Type: text/plain; charset="utf-8" We only need to do that on the ram_save_iterate() call on sending and on destination when we get a RAM_SAVE_FLAG_EOS. In setup() and complete() we need to synch in both new and old cases, so don't add a check there. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Juan Quintela --- Remove the wrappers that we take out on patch 5. --- migration/ram.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 8d114afd4b..899e2cd8af 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3406,9 +3406,11 @@ static int ram_save_iterate(QEMUFile *f, void *opaqu= e) out: if (ret >=3D 0 && migration_is_setup_or_active(migrate_get_current()->state)) { - ret =3D multifd_send_sync_main(rs->pss[RAM_CHANNEL_PRECOPY].pss_ch= annel); - if (ret < 0) { - return ret; + if (migrate_multifd_sync_after_each_section()) { + ret =3D multifd_send_sync_main(rs->pss[RAM_CHANNEL_PRECOPY].ps= s_channel); + if (ret < 0) { + return ret; + } } =20 qemu_put_be64(f, RAM_SAVE_FLAG_EOS); @@ -4168,7 +4170,9 @@ int ram_load_postcopy(QEMUFile *f, int channel) =20 case RAM_SAVE_FLAG_EOS: /* normal exit */ - multifd_recv_sync_main(); + if (migrate_multifd_sync_after_each_section()) { + multifd_recv_sync_main(); + } break; default: error_report("Unknown combination of migration flags: 0x%x" @@ -4439,7 +4443,9 @@ static int ram_load_precopy(QEMUFile *f) break; case RAM_SAVE_FLAG_EOS: /* normal exit */ - multifd_recv_sync_main(); + if (migrate_multifd_sync_after_each_section()) { + multifd_recv_sync_main(); + } break; default: if (flags & RAM_SAVE_FLAG_HOOK) { --=20 2.39.1 From nobody Sun May 19 22:21:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675985878; cv=none; d=zohomail.com; s=zohoarc; b=de8w+QUcjkCjg4KGGfjR6FjdJSJ9ioTaicWHcd35SlU5qVlNvDa77/u8FpGwj30cxy0XwMnO9knAZqkGbi/y9L06Gt/EQa64mWqq6dAq05weR/WKAsXDhCSWTAXWD45x5R6E5trIPZIEU4lED4lJn4Yx9j7S7qRIawsjj/7Qv6o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675985878; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=VtYgSFYuL65BZjhUmpjkj4gvldPqTJ/e4USBQlgQAt0=; b=N6Nyn9u6QzD+NQioxChBNpaFJOvK0a66hwU/rYVpnYBMVXRw112UuDiK7c4A+0KYz849hMoFjQ/QXkZ8vjldlF5MiNUbRdVeHJUCBuUwBQsn29/Bkm1NskKKjdpJMdysnvZ1qToc6Y5f3Rw9ox5xH37rOcwoutmoEDuzFuQJiIk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675985878052214.24413286101344; Thu, 9 Feb 2023 15:37:58 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pQGU8-0002Le-Qi; Thu, 09 Feb 2023 18:37:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pQGU6-0002Hv-HP for qemu-devel@nongnu.org; Thu, 09 Feb 2023 18:37:46 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pQGU3-0007fR-KO for qemu-devel@nongnu.org; Thu, 09 Feb 2023 18:37:46 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-147-7L226PW_No2Iq1O-yAe9eQ-1; Thu, 09 Feb 2023 18:37:38 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5F8261871D94; Thu, 9 Feb 2023 23:37:38 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id E4581140EBF6; Thu, 9 Feb 2023 23:37:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675985862; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VtYgSFYuL65BZjhUmpjkj4gvldPqTJ/e4USBQlgQAt0=; b=byfydeambhNWgLtg37aFzdBom4+7T8FwZpBVUE6mAKTsQWruFxruhDPmEs94fAUxRgiTd3 9FQqbrcToyeD5e5ozkZE0WQqcpVRWEMSb0w0DXh15Myexp4Q9W7LuxOw4stY5elj7k14U3 1exQgVC/lVjM9pCMDfm6dvypn4WZNm8= X-MC-Unique: 7L226PW_No2Iq1O-yAe9eQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Eduardo Habkost , Juan Quintela , Yanan Wang , Markus Armbruster , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Eric Blake , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PATCH v4 3/4] multifd: Only sync once each full round of memory Date: Fri, 10 Feb 2023 00:37:29 +0100 Message-Id: <20230209233730.38288-4-quintela@redhat.com> In-Reply-To: <20230209233730.38288-1-quintela@redhat.com> References: <20230209233730.38288-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675985878788100001 Content-Type: text/plain; charset="utf-8" We need to add a new flag to mean to sync at that point. Notice that we still synchronize at the end of setup and at the end of complete stages. Signed-off-by: Juan Quintela --- Add missing qemu_fflush(), now it passes all tests always. --- qapi/migration.json | 2 +- migration/migration.c | 2 -- migration/ram.c | 28 +++++++++++++++++++++++++++- 3 files changed, 28 insertions(+), 4 deletions(-) diff --git a/qapi/migration.json b/qapi/migration.json index 2907241b9c..5d0efa4590 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -482,7 +482,7 @@ # section is sent. We used to do # that in the past, but it is # suboptimal. -# Default value is true until all code i= s in. +# Default value is false. # (since 8.0) # # Features: diff --git a/migration/migration.c b/migration/migration.c index b2844d374f..9eb061319d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2710,8 +2710,6 @@ bool migrate_multifd_sync_after_each_section(void) { MigrationState *s =3D migrate_get_current(); =20 - return true; - // We will change this when code gets in. return s->enabled_capabilities[MIGRATION_CAPABILITY_MULTIFD_SYNC_AFTER= _EACH_SECTION]; } =20 diff --git a/migration/ram.c b/migration/ram.c index 899e2cd8af..32fab7b5ee 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -82,6 +82,7 @@ #define RAM_SAVE_FLAG_XBZRLE 0x40 /* 0x80 is reserved in migration.h start with 0x100 next */ #define RAM_SAVE_FLAG_COMPRESS_PAGE 0x100 +#define RAM_SAVE_FLAG_MULTIFD_SYNC 0x200 =20 int (*xbzrle_encode_buffer_func)(uint8_t *, uint8_t *, int, uint8_t *, int) =3D xbzrle_encode_buffer; @@ -1593,6 +1594,7 @@ retry: * associated with the search process. * * Returns: + * <0: An error happened * PAGE_ALL_CLEAN: no dirty page found, give up * PAGE_TRY_AGAIN: no dirty page found, retry for next block * PAGE_DIRTY_FOUND: dirty page found @@ -1620,6 +1622,15 @@ static int find_dirty_block(RAMState *rs, PageSearch= Status *pss) pss->page =3D 0; pss->block =3D QLIST_NEXT_RCU(pss->block, next); if (!pss->block) { + if (!migrate_multifd_sync_after_each_section()) { + QEMUFile *f =3D rs->pss[RAM_CHANNEL_PRECOPY].pss_channel; + int ret =3D multifd_send_sync_main(f); + if (ret < 0) { + return ret; + } + qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_SYNC); + qemu_fflush(f); + } /* * If memory migration starts over, we will meet a dirtied page * which may still exists in compression threads's ring, so we @@ -2612,6 +2623,9 @@ static int ram_find_and_save_block(RAMState *rs) break; } else if (res =3D=3D PAGE_TRY_AGAIN) { continue; + } else if (res < 0) { + pages =3D res; + break; } } } @@ -3298,6 +3312,10 @@ static int ram_save_setup(QEMUFile *f, void *opaque) return ret; } =20 + if (!migrate_multifd_sync_after_each_section()) { + qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_SYNC); + } + qemu_put_be64(f, RAM_SAVE_FLAG_EOS); qemu_fflush(f); =20 @@ -3483,6 +3501,9 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) return ret; } =20 + if (!migrate_multifd_sync_after_each_section()) { + qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_SYNC); + } qemu_put_be64(f, RAM_SAVE_FLAG_EOS); qemu_fflush(f); =20 @@ -4167,7 +4188,9 @@ int ram_load_postcopy(QEMUFile *f, int channel) } decompress_data_with_multi_threads(f, page_buffer, len); break; - + case RAM_SAVE_FLAG_MULTIFD_SYNC: + multifd_recv_sync_main(); + break; case RAM_SAVE_FLAG_EOS: /* normal exit */ if (migrate_multifd_sync_after_each_section()) { @@ -4441,6 +4464,9 @@ static int ram_load_precopy(QEMUFile *f) break; } break; + case RAM_SAVE_FLAG_MULTIFD_SYNC: + multifd_recv_sync_main(); + break; case RAM_SAVE_FLAG_EOS: /* normal exit */ if (migrate_multifd_sync_after_each_section()) { --=20 2.39.1 From nobody Sun May 19 22:21:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1675985908; cv=none; d=zohomail.com; s=zohoarc; b=CmZM76E77lTQrcxkO2X0WtASbmPO627LRAz8fVx8HDt8gL0oyxncbEzrWrjnVvdtdHcWX89sedXU3serZI0PV5PyH4MjXVdnm66KWEbOkmydXzbhKsDB62IpwoJbtIayUZrtArmsUln7OfNa9WmTZ5U06/WuuhuS7vlWH1izEaM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1675985908; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=BOoM/jysyMMLXInKHjz8rdvH/wPBwEQg/f/iK2pEkbw=; b=EZQ85aoJdokTfyIzDzM8TiTgSi7k0Z4VW2UfeVRcnLHdmuFcAOuSHoVBQ4wJv8kuPtlBWrHVhUSXpeSJ7jTuu+tkS9qd5PKXqsOXOuddQeNqZ35tKH20cWBUkz77XzIjHigEje+hZsxDwJNQ+25XF3LP5WysNXWW64Eops7mWG0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1675985908201617.6303452431306; Thu, 9 Feb 2023 15:38:28 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pQGU9-0002Lv-E6; Thu, 09 Feb 2023 18:37:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pQGU7-0002Kw-Ra for qemu-devel@nongnu.org; Thu, 09 Feb 2023 18:37:47 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pQGU6-0007gE-8b for qemu-devel@nongnu.org; Thu, 09 Feb 2023 18:37:47 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-414-pCdLGbQNPNyJnD1-_BfgaA-1; Thu, 09 Feb 2023 18:37:40 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2056A3806102; Thu, 9 Feb 2023 23:37:40 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id A6938140EBF6; Thu, 9 Feb 2023 23:37:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675985865; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BOoM/jysyMMLXInKHjz8rdvH/wPBwEQg/f/iK2pEkbw=; b=TpzXHI9d41t0buCY9x9cy/q7gS1OsH0yZMBrYLwnH7M+9pJGM3c2Q/9F8x5mh7zypcD4Iw 2bQ+BSwsMI/VQCBb86jLDIlXT3WsP7U5b014HxSE5XMagRMeGQ/vcolqDdS4Iu0stURXoe cHUMcmyQxo6VRIsX1aW8ghZytVXgZxI= X-MC-Unique: pCdLGbQNPNyJnD1-_BfgaA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Eduardo Habkost , Juan Quintela , Yanan Wang , Markus Armbruster , "Dr. David Alan Gilbert" , Marcel Apfelbaum , Eric Blake , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PATCH v4 4/4] ram: Document migration ram flags Date: Fri, 10 Feb 2023 00:37:30 +0100 Message-Id: <20230209233730.38288-5-quintela@redhat.com> In-Reply-To: <20230209233730.38288-1-quintela@redhat.com> References: <20230209233730.38288-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1675985909127100001 Content-Type: text/plain; charset="utf-8" 0x80 is RAM_SAVE_FLAG_HOOK, it is in qemu-file now. Bigger usable flag is 0x200, noticing that. We can reuse RAM_SAVe_FLAG_FULL. Signed-off-by: Juan Quintela Reviewed-by: Eric Blake --- migration/ram.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 32fab7b5ee..3648cfc357 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -67,22 +67,26 @@ /***********************************************************/ /* ram save/restore */ =20 -/* RAM_SAVE_FLAG_ZERO used to be named RAM_SAVE_FLAG_COMPRESS, it +/* + * RAM_SAVE_FLAG_ZERO used to be named RAM_SAVE_FLAG_COMPRESS, it * worked for pages that where filled with the same char. We switched * it to only search for the zero value. And to avoid confusion with * RAM_SSAVE_FLAG_COMPRESS_PAGE just rename it. */ - -#define RAM_SAVE_FLAG_FULL 0x01 /* Obsolete, not used anymore */ +/* + * RAM_SAVE_FLAG_FULL was obsoleted in 2009, it can be reused now + */ +#define RAM_SAVE_FLAG_FULL 0x01 #define RAM_SAVE_FLAG_ZERO 0x02 #define RAM_SAVE_FLAG_MEM_SIZE 0x04 #define RAM_SAVE_FLAG_PAGE 0x08 #define RAM_SAVE_FLAG_EOS 0x10 #define RAM_SAVE_FLAG_CONTINUE 0x20 #define RAM_SAVE_FLAG_XBZRLE 0x40 -/* 0x80 is reserved in migration.h start with 0x100 next */ +/* 0x80 is reserved in qemu-file.h for RAM_SAVE_FLAG_HOOK */ #define RAM_SAVE_FLAG_COMPRESS_PAGE 0x100 #define RAM_SAVE_FLAG_MULTIFD_SYNC 0x200 +/* We can't use any flag that is bigger than 0x200 */ =20 int (*xbzrle_encode_buffer_func)(uint8_t *, uint8_t *, int, uint8_t *, int) =3D xbzrle_encode_buffer; --=20 2.39.1