From nobody Sat May 18 12:05:06 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1676484205; cv=none; d=zohomail.com; s=zohoarc; b=mSXGsoHsK+whm1BYDRJWkyLoScpwShFcKwkK/RxAlKFRRwNpt9rg7WqDXeAUKmuZUkrGa+SdCAWxveQYbFu/1iKFFmaukdomve+blxiNBpGrOSqkoNFfGyaTRcUyGR8hmQtKm55Rl5wRMILOWSeMoJTBEiWVAxP9ivrr6Yc/9EY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1676484205; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CAFtpDcdONtGky+j+/WmM/4GmehBKT7VfaWqWHIfzPU=; b=W0wKwOL+7MA/tvaOK2dFdAp6DtMpCtQbVWdXmeeyMAibfkh6Cq7HspIXRHe+yRpL7MgU8OezpjatxlFvIsP0BKTdjTq9FPMH37wjyZpD4kow9xz5u2Zev5P7QBGKhjkPkYr3n8j3M+IUiX4JWQZAYm6X57YeZ9MFULYZYO5rBhI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16764842059831005.8844894804835; Wed, 15 Feb 2023 10:03:25 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pSM7I-0000Rq-PL; Wed, 15 Feb 2023 13:02:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pSM78-0000PT-Qr for qemu-devel@nongnu.org; Wed, 15 Feb 2023 13:02:48 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pSM76-0004cd-SW for qemu-devel@nongnu.org; Wed, 15 Feb 2023 13:02:42 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-640-PCJoHOPWO3SVNmkLAR-Myg-1; Wed, 15 Feb 2023 13:02:36 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 016491019CBC; Wed, 15 Feb 2023 18:02:36 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.37]) by smtp.corp.redhat.com (Postfix) with ESMTP id 60D422026D4B; Wed, 15 Feb 2023 18:02:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676484159; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CAFtpDcdONtGky+j+/WmM/4GmehBKT7VfaWqWHIfzPU=; b=hxHs0Z+xFaLzV6GnQtiQ9uWM5tTmXQpRj44hqYGNER1SSBJUNWhQ9eg13ElsojNO3mu5k5 V8Jy7q5Lwl9sapFTvB2oy0S/C2GDQvbiTwFNHIuVgnsqWtAIjKY0BJrQlLQgSiCI9lmSjS gmSxdisa2TGETik+b6Vb82Y6YbZOskk= X-MC-Unique: PCJoHOPWO3SVNmkLAR-Myg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Eric Blake , "Dr. David Alan Gilbert" , Yanan Wang , Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Eduardo Habkost , Markus Armbruster , Juan Quintela Subject: [PATCH v6 1/3] multifd: Create property multifd-flush-after-each-section Date: Wed, 15 Feb 2023 19:02:29 +0100 Message-Id: <20230215180231.7644-2-quintela@redhat.com> In-Reply-To: <20230215180231.7644-1-quintela@redhat.com> References: <20230215180231.7644-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1676484206921100005 Content-Type: text/plain; charset="utf-8" We used to flush all channels at the end of each RAM section sent. That is not needed, so preparing to only flush after a full iteration through all the RAM. Default value of the property is false. But we return "true" in migrate_multifd_flush_after_each_section() until we implement the code in following patches. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Juan Quintela Acked-by: Peter Xu --- Rename each-iteration to after-each-section Rename multifd-sync-after-each-section to multifd-flush-after-each-section --- qapi/migration.json | 21 ++++++++++++++++++++- migration/migration.h | 1 + hw/core/machine.c | 1 + migration/migration.c | 17 +++++++++++++++-- 4 files changed, 37 insertions(+), 3 deletions(-) diff --git a/qapi/migration.json b/qapi/migration.json index c84fa10e86..3afd81174d 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -478,6 +478,24 @@ # should not affect the correctness of postcopy migrati= on. # (since 7.1) # +# @multifd-flush-after-each-section: flush every channel after each +# section sent. This assures that +# we can't mix pages from one +# iteration through ram pages with +# pages for the following +# iteration. We really only need +# to do this flush after we have go +# through all the dirty pages. +# For historical reasons, we do +# that after each section. This is +# suboptimal (we flush too many +# times). +# Default value is false. +# Setting this capability has no +# effect until the patch that +# removes this comment. +# (since 8.0) +# # Features: # @unstable: Members @x-colo and @x-ignore-shared are experimental. # @@ -492,7 +510,8 @@ 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, 'validate-uuid', 'background-snapshot', - 'zero-copy-send', 'postcopy-preempt'] } + 'zero-copy-send', 'postcopy-preempt', + 'multifd-flush-after-each-section'] } =20 ## # @MigrationCapabilityStatus: diff --git a/migration/migration.h b/migration/migration.h index 2da2f8a164..7f0f4260ba 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -424,6 +424,7 @@ int migrate_multifd_channels(void); MultiFDCompression migrate_multifd_compression(void); int migrate_multifd_zlib_level(void); int migrate_multifd_zstd_level(void); +bool migrate_multifd_flush_after_each_section(void); =20 #ifdef CONFIG_LINUX bool migrate_use_zero_copy_send(void); diff --git a/hw/core/machine.c b/hw/core/machine.c index f73fc4c45c..602e775f34 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -54,6 +54,7 @@ const size_t hw_compat_7_1_len =3D G_N_ELEMENTS(hw_compat= _7_1); GlobalProperty hw_compat_7_0[] =3D { { "arm-gicv3-common", "force-8-bit-prio", "on" }, { "nvme-ns", "eui64-default", "on"}, + { "migration", "multifd-flush-after-each-section", "on"}, }; const size_t hw_compat_7_0_len =3D G_N_ELEMENTS(hw_compat_7_0); =20 diff --git a/migration/migration.c b/migration/migration.c index 90fca70cb7..cfba0da005 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -167,7 +167,8 @@ INITIALIZE_MIGRATE_CAPS_SET(check_caps_background_snaps= hot, MIGRATION_CAPABILITY_XBZRLE, MIGRATION_CAPABILITY_X_COLO, MIGRATION_CAPABILITY_VALIDATE_UUID, - MIGRATION_CAPABILITY_ZERO_COPY_SEND); + MIGRATION_CAPABILITY_ZERO_COPY_SEND, + MIGRATION_CAPABILITY_MULTIFD_FLUSH_AFTER_EACH_SECTION); =20 /* When we add fault tolerance, we could have several migrations at once. For now we don't need to add @@ -2701,6 +2702,17 @@ bool migrate_use_multifd(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_MULTIFD]; } =20 +bool migrate_multifd_flush_after_each_section(void) +{ + MigrationState *s =3D migrate_get_current(); + + /* + * Until the patch that remove this comment, we always return that + * the capability is enabled. + */ + return true || s->enabled_capabilities[MIGRATION_CAPABILITY_MULTIFD_FL= USH_AFTER_EACH_SECTION]; +} + bool migrate_pause_before_switchover(void) { MigrationState *s; @@ -4535,7 +4547,8 @@ static Property migration_properties[] =3D { DEFINE_PROP_MIG_CAP("x-zero-copy-send", MIGRATION_CAPABILITY_ZERO_COPY_SEND), #endif - + DEFINE_PROP_MIG_CAP("multifd-flush-after-each-section", + MIGRATION_CAPABILITY_MULTIFD_FLUSH_AFTER_EACH_SECT= ION), DEFINE_PROP_END_OF_LIST(), }; =20 --=20 2.39.1 From nobody Sat May 18 12:05:06 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1676484205; cv=none; d=zohomail.com; s=zohoarc; b=W+3RkCLU9bF/LfIxVrnpMIYREM2KPxUP4gGymWM+I76jiMpvCz0Mtkb5Wj3BqVT45h8FX9yMNbJQDsUuM96IbTlCnvoxHnoJjkWKlYmX3yuNiMdcziPqv7cT0hG7YwdNGQome6azi0+rhCeF8VOcPm/XDpmRFDgnK7Pn1kZZTdU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1676484205; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=sCjEzCRxX0M0nk6I2m/GcDkpwt4qScXDPUqkDYQllqM=; b=aCqWa7qGz+40EUEdbrVIzjwJwvmy4zcwre8eTzuSKrVsYR2qw480yxKuC1XLYwzFqeo8Ra0FCd8WaJIph+Y8IhklaaUUrosfLyoVaC9n947rjAlL+K7fxy2DcGxRVJXNFE84r0lJv46dTQhlRuO9HSkZD8g95MIPqEX1iL+ipy4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1676484205893459.3125500544212; Wed, 15 Feb 2023 10:03:25 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pSM7Q-0000UN-DL; Wed, 15 Feb 2023 13:03:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pSM7F-0000QZ-HT for qemu-devel@nongnu.org; Wed, 15 Feb 2023 13:02:49 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pSM7D-0004dN-NB for qemu-devel@nongnu.org; Wed, 15 Feb 2023 13:02:49 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-287-rd4-2waCNhu6jB0xhjMx4g-1; Wed, 15 Feb 2023 13:02:38 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C12D385A588; Wed, 15 Feb 2023 18:02:37 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.37]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4918E2026D4B; Wed, 15 Feb 2023 18:02:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676484166; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sCjEzCRxX0M0nk6I2m/GcDkpwt4qScXDPUqkDYQllqM=; b=evH8cTB/tHusO1ODkoP4R+ENs05Mwwzu4wMOwnhjacWOk4FOoKn2avOHTRF1aKIbqqoDJV k/c4rEy0YmJgqOE6pEqB0c2/H38wVYqZB4OV/3S/ZZcMVJnwKMwmTtU4l+6cTtB0v05mnq LZH1lRqV0jNK8kwLWXZdMVi3q8bro68= X-MC-Unique: rd4-2waCNhu6jB0xhjMx4g-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Eric Blake , "Dr. David Alan Gilbert" , Yanan Wang , Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Eduardo Habkost , Markus Armbruster , Juan Quintela Subject: [PATCH v6 2/3] multifd: Protect multifd_send_sync_main() calls Date: Wed, 15 Feb 2023 19:02:30 +0100 Message-Id: <20230215180231.7644-3-quintela@redhat.com> In-Reply-To: <20230215180231.7644-1-quintela@redhat.com> References: <20230215180231.7644-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1676484206881100001 Content-Type: text/plain; charset="utf-8" We only need to do that on the ram_save_iterate() call on sending and on destination when we get a RAM_SAVE_FLAG_EOS. In setup() and complete() we need to synch in both new and old cases, so don't add a check there. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Juan Quintela Acked-by: Peter Xu --- Remove the wrappers that we take out on patch 5. --- migration/ram.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 521912385d..6191dac9af 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3408,9 +3408,11 @@ static int ram_save_iterate(QEMUFile *f, void *opaqu= e) out: if (ret >=3D 0 && migration_is_setup_or_active(migrate_get_current()->state)) { - ret =3D multifd_send_sync_main(rs->pss[RAM_CHANNEL_PRECOPY].pss_ch= annel); - if (ret < 0) { - return ret; + if (migrate_multifd_flush_after_each_section()) { + ret =3D multifd_send_sync_main(rs->pss[RAM_CHANNEL_PRECOPY].ps= s_channel); + if (ret < 0) { + return ret; + } } =20 qemu_put_be64(f, RAM_SAVE_FLAG_EOS); @@ -4170,7 +4172,9 @@ int ram_load_postcopy(QEMUFile *f, int channel) =20 case RAM_SAVE_FLAG_EOS: /* normal exit */ - multifd_recv_sync_main(); + if (migrate_multifd_flush_after_each_section()) { + multifd_recv_sync_main(); + } break; default: error_report("Unknown combination of migration flags: 0x%x" @@ -4441,7 +4445,9 @@ static int ram_load_precopy(QEMUFile *f) break; case RAM_SAVE_FLAG_EOS: /* normal exit */ - multifd_recv_sync_main(); + if (migrate_multifd_flush_after_each_section()) { + multifd_recv_sync_main(); + } break; default: if (flags & RAM_SAVE_FLAG_HOOK) { --=20 2.39.1 From nobody Sat May 18 12:05:06 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1676484205; cv=none; d=zohomail.com; s=zohoarc; b=JnBvF1T8CAQBoHFlUlsM3UtR+aH/WErEDKlYClo87UeNkHWi+iMxaYvTDADI0JrKywM1ZGM0OTzgbCHlwAnbMl59InXPyvJlSg3R9iYPhtz9KqzeRTQcUDxwwWbqIjN643f06V4X0DRBvlxAaNtayR2EWNPR7GrvZi7Wwq1cahs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1676484205; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=2WEhfUDKbOOyE0I63FWgVCyN6bwg5AA6M8XNoXnCvWs=; b=ACUqEYN99tXxtUtVHPGOal4ZePUxs0iCm41pTlIw6E+IPc0nIv67EJNW87SKdpWGKE0w1nypTCzO11ju8mPobQEDWyT/22+Z1pLTB0kjX620JGP+YnWeBc3Hz1VgLHDvks9sG7E6/MhMyIk+3gosM4tqfWviWmpc5mmnmx7kIiI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1676484205888283.4135391929583; Wed, 15 Feb 2023 10:03:25 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pSM7H-0000RP-B1; Wed, 15 Feb 2023 13:02:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pSM7C-0000Pe-JE for qemu-devel@nongnu.org; Wed, 15 Feb 2023 13:02:48 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pSM7A-0004d0-RD for qemu-devel@nongnu.org; Wed, 15 Feb 2023 13:02:46 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-82-KFQwiObjPRSkS4E-L7XFZA-1; Wed, 15 Feb 2023 13:02:40 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A83BC80D0E9; Wed, 15 Feb 2023 18:02:39 +0000 (UTC) Received: from secure.mitica (unknown [10.39.192.37]) by smtp.corp.redhat.com (Postfix) with ESMTP id 13A452026D4B; Wed, 15 Feb 2023 18:02:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676484164; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2WEhfUDKbOOyE0I63FWgVCyN6bwg5AA6M8XNoXnCvWs=; b=KdONC1g/GULnPX+QEDYy519GDKCt1ZLkMyFzRXpk6ZjDg1NA6Do1paZG9rgYPl9EwVOITr Ipy/P2bTKjywifIBmvtIwIKpH+WbgrRxiAKGwRur4FH68N3UlssP9uixkWyTCY2I/G9igY cB1vtXActxMDtQk4WbLE4n0fCrLwmsI= X-MC-Unique: KFQwiObjPRSkS4E-L7XFZA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Eric Blake , "Dr. David Alan Gilbert" , Yanan Wang , Marcel Apfelbaum , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Eduardo Habkost , Markus Armbruster , Juan Quintela Subject: [PATCH v6 3/3] multifd: Only flush once each full round of memory Date: Wed, 15 Feb 2023 19:02:31 +0100 Message-Id: <20230215180231.7644-4-quintela@redhat.com> In-Reply-To: <20230215180231.7644-1-quintela@redhat.com> References: <20230215180231.7644-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1676484206890100003 Content-Type: text/plain; charset="utf-8" We need to add a new flag to mean to flush at that point. Notice that we still flush at the end of setup and at the end of complete stages. Signed-off-by: Juan Quintela Acked-by: Peter Xu --- Add missing qemu_fflush(), now it passes all tests always. --- qapi/migration.json | 3 --- migration/migration.c | 6 +----- migration/ram.c | 28 +++++++++++++++++++++++++++- 3 files changed, 28 insertions(+), 9 deletions(-) diff --git a/qapi/migration.json b/qapi/migration.json index 3afd81174d..34e1657c4e 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -491,9 +491,6 @@ # suboptimal (we flush too many # times). # Default value is false. -# Setting this capability has no -# effect until the patch that -# removes this comment. # (since 8.0) # # Features: diff --git a/migration/migration.c b/migration/migration.c index cfba0da005..74bcc16848 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2706,11 +2706,7 @@ bool migrate_multifd_flush_after_each_section(void) { MigrationState *s =3D migrate_get_current(); =20 - /* - * Until the patch that remove this comment, we always return that - * the capability is enabled. - */ - return true || s->enabled_capabilities[MIGRATION_CAPABILITY_MULTIFD_FL= USH_AFTER_EACH_SECTION]; + return s->enabled_capabilities[MIGRATION_CAPABILITY_MULTIFD_FLUSH_AFTE= R_EACH_SECTION]; } =20 bool migrate_pause_before_switchover(void) diff --git a/migration/ram.c b/migration/ram.c index 6191dac9af..bc5eb1640b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -85,6 +85,7 @@ #define RAM_SAVE_FLAG_XBZRLE 0x40 /* 0x80 is reserved in qemu-file.h for RAM_SAVE_FLAG_HOOK */ #define RAM_SAVE_FLAG_COMPRESS_PAGE 0x100 +#define RAM_SAVE_FLAG_MULTIFD_FLUSH 0x200 /* We can't use any flag that is bigger than 0x200 */ =20 int (*xbzrle_encode_buffer_func)(uint8_t *, uint8_t *, int, @@ -1595,6 +1596,7 @@ retry: * associated with the search process. * * Returns: + * <0: An error happened * PAGE_ALL_CLEAN: no dirty page found, give up * PAGE_TRY_AGAIN: no dirty page found, retry for next block * PAGE_DIRTY_FOUND: dirty page found @@ -1622,6 +1624,15 @@ static int find_dirty_block(RAMState *rs, PageSearch= Status *pss) pss->page =3D 0; pss->block =3D QLIST_NEXT_RCU(pss->block, next); if (!pss->block) { + if (!migrate_multifd_flush_after_each_section()) { + QEMUFile *f =3D rs->pss[RAM_CHANNEL_PRECOPY].pss_channel; + int ret =3D multifd_send_sync_main(f); + if (ret < 0) { + return ret; + } + qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_FLUSH); + qemu_fflush(f); + } /* * If memory migration starts over, we will meet a dirtied page * which may still exists in compression threads's ring, so we @@ -2614,6 +2625,9 @@ static int ram_find_and_save_block(RAMState *rs) break; } else if (res =3D=3D PAGE_TRY_AGAIN) { continue; + } else if (res < 0) { + pages =3D res; + break; } } } @@ -3300,6 +3314,10 @@ static int ram_save_setup(QEMUFile *f, void *opaque) return ret; } =20 + if (!migrate_multifd_flush_after_each_section()) { + qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_FLUSH); + } + qemu_put_be64(f, RAM_SAVE_FLAG_EOS); qemu_fflush(f); =20 @@ -3485,6 +3503,9 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) return ret; } =20 + if (!migrate_multifd_flush_after_each_section()) { + qemu_put_be64(f, RAM_SAVE_FLAG_MULTIFD_FLUSH); + } qemu_put_be64(f, RAM_SAVE_FLAG_EOS); qemu_fflush(f); =20 @@ -4169,7 +4190,9 @@ int ram_load_postcopy(QEMUFile *f, int channel) } decompress_data_with_multi_threads(f, page_buffer, len); break; - + case RAM_SAVE_FLAG_MULTIFD_FLUSH: + multifd_recv_sync_main(); + break; case RAM_SAVE_FLAG_EOS: /* normal exit */ if (migrate_multifd_flush_after_each_section()) { @@ -4443,6 +4466,9 @@ static int ram_load_precopy(QEMUFile *f) break; } break; + case RAM_SAVE_FLAG_MULTIFD_FLUSH: + multifd_recv_sync_main(); + break; case RAM_SAVE_FLAG_EOS: /* normal exit */ if (migrate_multifd_flush_after_each_section()) { --=20 2.39.1