From nobody Tue Nov 26 06:29:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1709260295; cv=none; d=zohomail.com; s=zohoarc; b=Ft1MqEjOgKolovvZpKUWEwyN0OtXptGYMD8Q7ghvSoqiH5hjn5Yu9ka5WYRS+o15BrSjWExCSYwDNLhxrqSj2QlPlviY3CwKMvcxDmlIARcrdTKpR4i/cgYBmxo+Mp2mmoq4gTMSN+wZyx7Tn2XlRtU5frNq8XPpDTnF7v+rc8s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709260295; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Sb2DGHjJyDbpFDS1zLN6i0Hdzzb38omi0MKX/F3Yb7c=; b=FvnUjk4xPVEW28PjXeC+fqV94znsD0T0eJc55VQ1Nn43BmUWl6ksUvnWWSwRen3bzCaSjlji92dNzljE1n0zPyqqFzhfBDPhlSev9yFRUVX7DfsjySupwIAmWDMrcbHc8+ncuHlBtoDIShYUMNpHk5KDClukEKMjAEUAuf/qlXU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709260295252590.7046925368229; Thu, 29 Feb 2024 18:31:35 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rfseQ-0003xC-DF; Thu, 29 Feb 2024 21:29:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rfseO-0003x0-DL for qemu-devel@nongnu.org; Thu, 29 Feb 2024 21:29:28 -0500 Received: from mail-qv1-xf2f.google.com ([2607:f8b0:4864:20::f2f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rfseL-0005Ou-WF for qemu-devel@nongnu.org; Thu, 29 Feb 2024 21:29:28 -0500 Received: by mail-qv1-xf2f.google.com with SMTP id 6a1803df08f44-68f74fb38a8so6390486d6.3 for ; Thu, 29 Feb 2024 18:29:24 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.120]) by smtp.gmail.com with ESMTPSA id qo13-20020a056214590d00b0068d11cf887bsm1384107qvb.55.2024.02.29.18.29.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 18:29:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1709260164; x=1709864964; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Sb2DGHjJyDbpFDS1zLN6i0Hdzzb38omi0MKX/F3Yb7c=; b=X9ngEd2gGSkj/ra/NnNRx/L339X90jCFzGq3FbB5DQnPnzCiraU9puDLijqqSaKsdU cbz+Mgq1Z9YaXzx1Pj3Y6kipYzkg8bGRSeE6N1orb9oSxz+y7KeNyFAPZowBP04thksI 1bJWPphtQohaz1simmsgiiqiJsltrnRDbFKk/mFXsvE0BLVdX0L2snbvBfH4J/CAI6GP xQoCvD4A5oYBoGAjD6kXkBoRtIWU/+5BUabshfjjm8EOmnSDV38jJqJP6PmYyUPY9A9W sc2/PnxXfzdxoBepc7jM7r4foVPpS03TkjcuglztfngYihRQnIjWUPxSGj/+oIki8JEz gzfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709260164; x=1709864964; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Sb2DGHjJyDbpFDS1zLN6i0Hdzzb38omi0MKX/F3Yb7c=; b=ufix3o48bl+jeEGoLTtkYUsYbVgk1y4OyzmCgLJAt/9VGmjRl2pAi2KXg2zZHPUwyW uN+emdq+2vFel02fKzEKWveCRH/L9eErClZwbxwHw1bQYVKAZs5sYVfFzNRu60VL4Tf3 svPt09BJOw9xG3fLaDJkBOt1uYzHX1e+h3ixkDUPpYq7Jk0AU0SBCUbA62rPRXMg2rCv x/9fX5NmS2jdafj8hagg33b1TWmIjYkMwhmkn0DrEiRlkRvsyvpMa/qAg1EM70F8Mhe4 jTuF4JkGJAfCiwg1D7cmxa8oDWcDjAWAvnUFxysm17B/mJX7fpUmIocQeX1knkbouV3s oJjQ== X-Forwarded-Encrypted: i=1; AJvYcCUq7FenFjkbD++ymxsj8aXwK8MdUlRsTwebg36u6nyAOeZm7PQ58xe7OeprFTvcuR1i4KdubwMF/omsX19www/uyVTJmjo= X-Gm-Message-State: AOJu0YyZjHha2wAoNhPbJ1Io6PLEQBmPswQgq/FlO1XF1ARKunuakNNU CsPNjHMYec7ImOrF4i0cmrgjepv6XDk0ZhK8VzC07t/q4RUJXj9ti57bmIpAbbA= X-Google-Smtp-Source: AGHT+IHjDk2ySFQWdSO4XNU640EXZDOODKvRT6DYKhdMYBL/hdycJZSHH8FZztp5+lSx0bN2x/474A== X-Received: by 2002:a0c:9a81:0:b0:690:4917:6242 with SMTP id y1-20020a0c9a81000000b0069049176242mr484389qvd.7.1709260164345; Thu, 29 Feb 2024 18:29:24 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, jdenemar@redhat.com, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, qemu-devel@nongnu.org Cc: Hao Xiang Subject: [PATCH v4 1/7] migration/multifd: Add new migration option zero-page-detection. Date: Fri, 1 Mar 2024 02:28:23 +0000 Message-Id: <20240301022829.3390548-2-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240301022829.3390548-1-hao.xiang@bytedance.com> References: <20240301022829.3390548-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::f2f; envelope-from=hao.xiang@bytedance.com; helo=mail-qv1-xf2f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1709260296117100001 Content-Type: text/plain; charset="utf-8" This new parameter controls where the zero page checking is running. 1. If this parameter is set to 'legacy', zero page checking is done in the migration main thread. 2. If this parameter is set to 'none', zero page checking is disabled. Signed-off-by: Hao Xiang Acked-by: Markus Armbruster Reviewed-by: Peter Xu --- hw/core/qdev-properties-system.c | 10 +++++++++ include/hw/qdev-properties-system.h | 4 ++++ migration/migration-hmp-cmds.c | 9 ++++++++ migration/options.c | 21 ++++++++++++++++++ migration/options.h | 1 + migration/ram.c | 4 ++++ qapi/migration.json | 33 ++++++++++++++++++++++++++--- 7 files changed, 79 insertions(+), 3 deletions(-) diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-sys= tem.c index 1a396521d5..228e685f52 100644 --- a/hw/core/qdev-properties-system.c +++ b/hw/core/qdev-properties-system.c @@ -679,6 +679,16 @@ const PropertyInfo qdev_prop_mig_mode =3D { .set_default_value =3D qdev_propinfo_set_default_value_enum, }; =20 +const PropertyInfo qdev_prop_zero_page_detection =3D { + .name =3D "ZeroPageDetection", + .description =3D "zero_page_detection values, " + "none,legacy", + .enum_table =3D &ZeroPageDetection_lookup, + .get =3D qdev_propinfo_get_enum, + .set =3D qdev_propinfo_set_enum, + .set_default_value =3D qdev_propinfo_set_default_value_enum, +}; + /* --- Reserved Region --- */ =20 /* diff --git a/include/hw/qdev-properties-system.h b/include/hw/qdev-properti= es-system.h index 06c359c190..839b170235 100644 --- a/include/hw/qdev-properties-system.h +++ b/include/hw/qdev-properties-system.h @@ -8,6 +8,7 @@ extern const PropertyInfo qdev_prop_macaddr; extern const PropertyInfo qdev_prop_reserved_region; extern const PropertyInfo qdev_prop_multifd_compression; extern const PropertyInfo qdev_prop_mig_mode; +extern const PropertyInfo qdev_prop_zero_page_detection; extern const PropertyInfo qdev_prop_losttickpolicy; extern const PropertyInfo qdev_prop_blockdev_on_error; extern const PropertyInfo qdev_prop_bios_chs_trans; @@ -47,6 +48,9 @@ extern const PropertyInfo qdev_prop_iothread_vq_mapping_l= ist; #define DEFINE_PROP_MIG_MODE(_n, _s, _f, _d) \ DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_mig_mode, \ MigMode) +#define DEFINE_PROP_ZERO_PAGE_DETECTION(_n, _s, _f, _d) \ + DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_zero_page_detection, \ + ZeroPageDetection) #define DEFINE_PROP_LOSTTICKPOLICY(_n, _s, _f, _d) \ DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_losttickpolicy, \ LostTickPolicy) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 99b49df5dd..7e96ae6ffd 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -344,6 +344,11 @@ void hmp_info_migrate_parameters(Monitor *mon, const Q= Dict *qdict) monitor_printf(mon, "%s: %s\n", MigrationParameter_str(MIGRATION_PARAMETER_MULTIFD_COMPRESSION= ), MultiFDCompression_str(params->multifd_compression)); + assert(params->has_zero_page_detection); + monitor_printf(mon, "%s: %s\n", + MigrationParameter_str(MIGRATION_PARAMETER_ZERO_PAGE_DETECTION= ), + qapi_enum_lookup(&ZeroPageDetection_lookup, + params->zero_page_detection)); monitor_printf(mon, "%s: %" PRIu64 " bytes\n", MigrationParameter_str(MIGRATION_PARAMETER_XBZRLE_CACHE_SIZE), params->xbzrle_cache_size); @@ -634,6 +639,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDi= ct *qdict) p->has_multifd_zstd_level =3D true; visit_type_uint8(v, param, &p->multifd_zstd_level, &err); break; + case MIGRATION_PARAMETER_ZERO_PAGE_DETECTION: + p->has_zero_page_detection =3D true; + visit_type_ZeroPageDetection(v, param, &p->zero_page_detection, &e= rr); + break; case MIGRATION_PARAMETER_XBZRLE_CACHE_SIZE: p->has_xbzrle_cache_size =3D true; if (!visit_type_size(v, param, &cache_size, &err)) { diff --git a/migration/options.c b/migration/options.c index 3e3e0b93b4..3c603391b0 100644 --- a/migration/options.c +++ b/migration/options.c @@ -179,6 +179,9 @@ Property migration_properties[] =3D { DEFINE_PROP_MIG_MODE("mode", MigrationState, parameters.mode, MIG_MODE_NORMAL), + DEFINE_PROP_ZERO_PAGE_DETECTION("zero-page-detection", MigrationState, + parameters.zero_page_detection, + ZERO_PAGE_DETECTION_LEGACY), =20 /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), @@ -903,6 +906,13 @@ uint64_t migrate_xbzrle_cache_size(void) return s->parameters.xbzrle_cache_size; } =20 +ZeroPageDetection migrate_zero_page_detection(void) +{ + MigrationState *s =3D migrate_get_current(); + + return s->parameters.zero_page_detection; +} + /* parameter setters */ =20 void migrate_set_block_incremental(bool value) @@ -1013,6 +1023,8 @@ MigrationParameters *qmp_query_migrate_parameters(Err= or **errp) params->vcpu_dirty_limit =3D s->parameters.vcpu_dirty_limit; params->has_mode =3D true; params->mode =3D s->parameters.mode; + params->has_zero_page_detection =3D true; + params->zero_page_detection =3D s->parameters.zero_page_detection; =20 return params; } @@ -1049,6 +1061,7 @@ void migrate_params_init(MigrationParameters *params) params->has_x_vcpu_dirty_limit_period =3D true; params->has_vcpu_dirty_limit =3D true; params->has_mode =3D true; + params->has_zero_page_detection =3D true; } =20 /* @@ -1350,6 +1363,10 @@ static void migrate_params_test_apply(MigrateSetPara= meters *params, if (params->has_mode) { dest->mode =3D params->mode; } + + if (params->has_zero_page_detection) { + dest->zero_page_detection =3D params->zero_page_detection; + } } =20 static void migrate_params_apply(MigrateSetParameters *params, Error **err= p) @@ -1494,6 +1511,10 @@ static void migrate_params_apply(MigrateSetParameter= s *params, Error **errp) if (params->has_mode) { s->parameters.mode =3D params->mode; } + + if (params->has_zero_page_detection) { + s->parameters.zero_page_detection =3D params->zero_page_detection; + } } =20 void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) diff --git a/migration/options.h b/migration/options.h index 246c160aee..b7c4fb3861 100644 --- a/migration/options.h +++ b/migration/options.h @@ -93,6 +93,7 @@ const char *migrate_tls_authz(void); const char *migrate_tls_creds(void); const char *migrate_tls_hostname(void); uint64_t migrate_xbzrle_cache_size(void); +ZeroPageDetection migrate_zero_page_detection(void); =20 /* parameters setters */ =20 diff --git a/migration/ram.c b/migration/ram.c index 45a00b45ed..2581ee1e04 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1122,6 +1122,10 @@ static int save_zero_page(RAMState *rs, PageSearchSt= atus *pss, QEMUFile *file =3D pss->pss_channel; int len =3D 0; =20 + if (migrate_zero_page_detection() =3D=3D ZERO_PAGE_DETECTION_NONE) { + return 0; + } + if (!buffer_is_zero(p, TARGET_PAGE_SIZE)) { return 0; } diff --git a/qapi/migration.json b/qapi/migration.json index 0b33a71ab4..8da05dba47 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -664,6 +664,18 @@ { 'enum': 'MigMode', 'data': [ 'normal', 'cpr-reboot' ] } =20 +## +# @ZeroPageDetection: +# +# @none: Do not perform zero page checking. +# +# @legacy: Perform zero page checking in main migration thread. +# +# Since: 9.0 +## +{ 'enum': 'ZeroPageDetection', + 'data': [ 'none', 'legacy' ] } + ## # @BitmapMigrationBitmapAliasTransform: # @@ -885,6 +897,10 @@ # @mode: Migration mode. See description in @MigMode. Default is 'normal'. # (Since 8.2) # +# @zero-page-detection: Whether and how to detect zero pages. +# See description in @ZeroPageDetection. Default is 'legacy'. +# (since 9.0) +# # Features: # # @deprecated: Member @block-incremental is deprecated. Use @@ -918,7 +934,8 @@ 'block-bitmap-mapping', { 'name': 'x-vcpu-dirty-limit-period', 'features': ['unstable']= }, 'vcpu-dirty-limit', - 'mode'] } + 'mode', + 'zero-page-detection'] } =20 ## # @MigrateSetParameters: @@ -1077,6 +1094,10 @@ # @mode: Migration mode. See description in @MigMode. Default is 'normal'. # (Since 8.2) # +# @zero-page-detection: Whether and how to detect zero pages. +# See description in @ZeroPageDetection. Default is 'legacy'. +# (since 9.0) +# # Features: # # @deprecated: Member @block-incremental is deprecated. Use @@ -1130,7 +1151,8 @@ '*x-vcpu-dirty-limit-period': { 'type': 'uint64', 'features': [ 'unstable' ] }, '*vcpu-dirty-limit': 'uint64', - '*mode': 'MigMode'} } + '*mode': 'MigMode', + '*zero-page-detection': 'ZeroPageDetection'} } =20 ## # @migrate-set-parameters: @@ -1305,6 +1327,10 @@ # @mode: Migration mode. See description in @MigMode. Default is 'normal'. # (Since 8.2) # +# @zero-page-detection: Whether and how to detect zero pages. +# See description in @ZeroPageDetection. Default is 'legacy'. +# (since 9.0) +# # Features: # # @deprecated: Member @block-incremental is deprecated. Use @@ -1355,7 +1381,8 @@ '*x-vcpu-dirty-limit-period': { 'type': 'uint64', 'features': [ 'unstable' ] }, '*vcpu-dirty-limit': 'uint64', - '*mode': 'MigMode'} } + '*mode': 'MigMode', + '*zero-page-detection': 'ZeroPageDetection'} } =20 ## # @query-migrate-parameters: --=20 2.30.2 From nobody Tue Nov 26 06:29:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1709260251; cv=none; d=zohomail.com; s=zohoarc; b=ijywi/aNn7LkFb28LWUbTZPUwbw4rDCaA2YsMuFz4xFfjULVzH8O6i2mwkKwHKBheQB7/ONYIDPPNKTxJOww87z7oRrCOOYtDXX8f+agvSL7E+HlJuBReArg3pUo++qRSmCX8hqpNKqd5UCosvOJ2qB2YkUzXpksvMAUGBl3Zzw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709260251; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=nwduwwXtJOerHFO3VZjKLXU2ctuBeO4NUdSa+BwMg1E=; b=lKeGH0+0c/wLSf09qZhTueok+ZpM8InieMDapXCLrqTmjkMfirlgxmynbUlGmPV3yJvhGFtDVLGJnxP0a96n9VQ5tPKmQGl0IP33AuokpnazNB1Gdp840mdAqV6QA3FFrbpcYOOG5eK8FwV5WUHSTwXoIaeFuSpmsQkE+BzKhGo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709260251518477.40730747896896; Thu, 29 Feb 2024 18:30:51 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rfseT-0003yf-Gn; Thu, 29 Feb 2024 21:29:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rfseQ-0003xp-B3 for qemu-devel@nongnu.org; Thu, 29 Feb 2024 21:29:30 -0500 Received: from mail-qv1-xf30.google.com ([2607:f8b0:4864:20::f30]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rfseN-0005P0-8U for qemu-devel@nongnu.org; Thu, 29 Feb 2024 21:29:30 -0500 Received: by mail-qv1-xf30.google.com with SMTP id 6a1803df08f44-68f74fb38a8so6390536d6.3 for ; Thu, 29 Feb 2024 18:29:26 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.120]) by smtp.gmail.com with ESMTPSA id qo13-20020a056214590d00b0068d11cf887bsm1384107qvb.55.2024.02.29.18.29.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 18:29:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1709260166; x=1709864966; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nwduwwXtJOerHFO3VZjKLXU2ctuBeO4NUdSa+BwMg1E=; b=VGcYpN7i7ywodAsk5++3KKNv0E/32ZO8q9Uj9ec6lBJDYD7KiGKFl5CW0YTfziPERO OxbVaSZxBNX/sKYmVrjdNDy/x228vMgWMfIL+Y/J+05DMOXaUsxmceysn/7WdU+uucZS 8aFGZX/9gieA/1SvrLIDvuceL5/tXsXUexxJ7SrSIxobMeSD8tfKpl4dKYjijuz6rz+p kCIhy46oqgPcmfFaYXZwPUUoYCdCr69S/Og83rf7BuaY8//b7oC8syEkrKVg2iwIpK6U rPK14zrU9OHWlxmeGTSYUvaZF5nN6RVhHjXyEFG5fL32OfytNxa5EIP040iHqwuglOVj +dpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709260166; x=1709864966; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nwduwwXtJOerHFO3VZjKLXU2ctuBeO4NUdSa+BwMg1E=; b=uhoGBWInaNXUPTNLC8800j5TAAmownTNTLLRPeWT3ziaNs2OjISlZR0v1krgs4PpBD oSW2td6o/jk4lBsHasC4hCLLGovbognRhS+caLw1knxvXYPYbMN490z2Nqts+1bvuirW j8Zjtu+LPHYl2M5TWSfT3PAjVmahzvaoVZrZnBzXWEqJvZWLsLpyaw/eJ2njeLj/MMs/ u/PIuaBQ6FdU0yCR07lHqtZ7bwi1Q7ewkCTF9G2Q3/M+GrN3D5WKlE5UoSyd0sWFadMT 7qGE0CSwnRZDJv8zWm9FQrUMyG/ATX7FsRIKvhPrSbI8SgQnxWqZSjM4BHOdXg1zCKA0 U3KA== X-Forwarded-Encrypted: i=1; AJvYcCXx1b3z8mWdL+lTrGpavaNdktkbnZypyCFKd1yCjl/B12lO/8//KoUxQ4Rpm0yx1aurx/LZaAqUysQS7bAMthiNZniXUbA= X-Gm-Message-State: AOJu0Ywipu3etiH2f+YXFwWSsv7LTXXxpMoLZS4QTyCAJb13Ogmbyr6/ h3hKUcp712Mbs+2bj8GdDwG4YQJVW9Wgf5PI6JsKF4mdvRerrF7aw6PjOOEzTjU= X-Google-Smtp-Source: AGHT+IGX5bDwWReLznRrUH51WzjH+/HoFuLxVh4MpnVfCCU3HzJREWlw+QUJoYbrFaokdJoRt+yAcQ== X-Received: by 2002:a0c:c788:0:b0:68f:6be:9f94 with SMTP id k8-20020a0cc788000000b0068f06be9f94mr426050qvj.12.1709260166100; Thu, 29 Feb 2024 18:29:26 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, jdenemar@redhat.com, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, qemu-devel@nongnu.org Cc: Hao Xiang Subject: [PATCH v4 2/7] migration/multifd: Implement zero page transmission on the multifd thread. Date: Fri, 1 Mar 2024 02:28:24 +0000 Message-Id: <20240301022829.3390548-3-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240301022829.3390548-1-hao.xiang@bytedance.com> References: <20240301022829.3390548-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::f30; envelope-from=hao.xiang@bytedance.com; helo=mail-qv1-xf30.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1709260251994100009 Content-Type: text/plain; charset="utf-8" 1. Add zero_pages field in MultiFDPacket_t. 2. Implements the zero page detection and handling on the multifd threads for non-compression, zlib and zstd compression backends. 3. Added a new value 'multifd' in ZeroPageDetection enumeration. 4. Handle migration QEMU9.0 -> QEMU8.2 compatibility. 5. Adds zero page counters and updates multifd send/receive tracing format to track the newly added counters. Signed-off-by: Hao Xiang Acked-by: Markus Armbruster --- hw/core/machine.c | 4 +- hw/core/qdev-properties-system.c | 2 +- migration/meson.build | 1 + migration/multifd-zero-page.c | 92 ++++++++++++++++++++++++++++++++ migration/multifd-zlib.c | 21 ++++++-- migration/multifd-zstd.c | 20 +++++-- migration/multifd.c | 83 +++++++++++++++++++++++----- migration/multifd.h | 24 ++++++++- migration/ram.c | 1 - migration/trace-events | 8 +-- qapi/migration.json | 7 ++- 11 files changed, 230 insertions(+), 33 deletions(-) create mode 100644 migration/multifd-zero-page.c diff --git a/hw/core/machine.c b/hw/core/machine.c index 9ac5d5389a..0e9d646b61 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -32,7 +32,9 @@ #include "hw/virtio/virtio-net.h" #include "audio/audio.h" =20 -GlobalProperty hw_compat_8_2[] =3D {}; +GlobalProperty hw_compat_8_2[] =3D { + { "migration", "zero-page-detection", "legacy"}, +}; const size_t hw_compat_8_2_len =3D G_N_ELEMENTS(hw_compat_8_2); =20 GlobalProperty hw_compat_8_1[] =3D { diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-sys= tem.c index 228e685f52..6e6f68ae1b 100644 --- a/hw/core/qdev-properties-system.c +++ b/hw/core/qdev-properties-system.c @@ -682,7 +682,7 @@ const PropertyInfo qdev_prop_mig_mode =3D { const PropertyInfo qdev_prop_zero_page_detection =3D { .name =3D "ZeroPageDetection", .description =3D "zero_page_detection values, " - "none,legacy", + "none,legacy,multifd", .enum_table =3D &ZeroPageDetection_lookup, .get =3D qdev_propinfo_get_enum, .set =3D qdev_propinfo_set_enum, diff --git a/migration/meson.build b/migration/meson.build index 92b1cc4297..1eeb915ff6 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -22,6 +22,7 @@ system_ss.add(files( 'migration.c', 'multifd.c', 'multifd-zlib.c', + 'multifd-zero-page.c', 'ram-compress.c', 'options.c', 'postcopy-ram.c', diff --git a/migration/multifd-zero-page.c b/migration/multifd-zero-page.c new file mode 100644 index 0000000000..e695f6ff7d --- /dev/null +++ b/migration/multifd-zero-page.c @@ -0,0 +1,92 @@ +/* + * Multifd zero page detection implementation. + * + * Copyright (c) 2024 Bytedance Inc + * + * Authors: + * Hao Xiang + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qemu/cutils.h" +#include "exec/ramblock.h" +#include "migration.h" +#include "multifd.h" +#include "options.h" +#include "ram.h" + +static bool multifd_zero_page(void) +{ + return migrate_zero_page_detection() =3D=3D ZERO_PAGE_DETECTION_MULTIF= D; +} + +static void swap_page_offset(ram_addr_t *pages_offset, int a, int b) +{ + ram_addr_t temp; + + if (a =3D=3D b) { + return; + } + + temp =3D pages_offset[a]; + pages_offset[a] =3D pages_offset[b]; + pages_offset[b] =3D temp; +} + +/** + * multifd_send_zero_page_check: Perform zero page detection on all pages. + * + * Sorts normal pages before zero pages in p->pages->offset and updates + * p->pages->normal_num. + * + * @param p A pointer to the send params. + */ +void multifd_send_zero_page_check(MultiFDSendParams *p) +{ + MultiFDPages_t *pages =3D p->pages; + RAMBlock *rb =3D pages->block; + int i =3D 0; + int j =3D pages->num - 1; + + /* + * QEMU older than 9.0 don't understand zero page + * on multifd channel. This switch is required to + * maintain backward compatibility. + */ + if (multifd_zero_page()) { + pages->normal_num =3D pages->num; + return; + } + + /* + * Sort the page offset array by moving all normal pages to + * the left and all zero pages to the right of the array. + */ + while (i <=3D j) { + uint64_t offset =3D pages->offset[i]; + + if (!buffer_is_zero(rb->host + offset, p->page_size)) { + i++; + continue; + } + + swap_page_offset(pages->offset, i, j); + ram_release_page(rb->idstr, offset); + j--; + } + + pages->normal_num =3D i; +} + +void multifd_recv_zero_page_check(MultiFDRecvParams *p) +{ + for (int i =3D 0; i < p->zero_num; i++) { + void *page =3D p->host + p->zero[i]; + if (!buffer_is_zero(page, p->page_size)) { + memset(page, 0, p->page_size); + } + } +} diff --git a/migration/multifd-zlib.c b/migration/multifd-zlib.c index 012e3bdea1..f749e703a9 100644 --- a/migration/multifd-zlib.c +++ b/migration/multifd-zlib.c @@ -123,13 +123,15 @@ static int zlib_send_prepare(MultiFDSendParams *p, Er= ror **errp) int ret; uint32_t i; =20 - multifd_send_prepare_header(p); + if (!multifd_send_prepare_common(p)) { + goto out; + } =20 - for (i =3D 0; i < pages->num; i++) { + for (i =3D 0; i < pages->normal_num; i++) { uint32_t available =3D z->zbuff_len - out_size; int flush =3D Z_NO_FLUSH; =20 - if (i =3D=3D pages->num - 1) { + if (i =3D=3D pages->normal_num - 1) { flush =3D Z_SYNC_FLUSH; } =20 @@ -172,10 +174,10 @@ static int zlib_send_prepare(MultiFDSendParams *p, Er= ror **errp) p->iov[p->iovs_num].iov_len =3D out_size; p->iovs_num++; p->next_packet_size =3D out_size; - p->flags |=3D MULTIFD_FLAG_ZLIB; =20 +out: + p->flags |=3D MULTIFD_FLAG_ZLIB; multifd_send_fill_packet(p); - return 0; } =20 @@ -261,6 +263,14 @@ static int zlib_recv_pages(MultiFDRecvParams *p, Error= **errp) p->id, flags, MULTIFD_FLAG_ZLIB); return -1; } + + multifd_recv_zero_page_check(p); + + if (!p->normal_num) { + assert(in_size =3D=3D 0); + return 0; + } + ret =3D qio_channel_read_all(p->c, (void *)z->zbuff, in_size, errp); =20 if (ret !=3D 0) { @@ -310,6 +320,7 @@ static int zlib_recv_pages(MultiFDRecvParams *p, Error = **errp) p->id, out_size, expected_size); return -1; } + return 0; } =20 diff --git a/migration/multifd-zstd.c b/migration/multifd-zstd.c index dc8fe43e94..80635e19ef 100644 --- a/migration/multifd-zstd.c +++ b/migration/multifd-zstd.c @@ -118,16 +118,18 @@ static int zstd_send_prepare(MultiFDSendParams *p, Er= ror **errp) int ret; uint32_t i; =20 - multifd_send_prepare_header(p); + if (!multifd_send_prepare_common(p)) { + goto out; + } =20 z->out.dst =3D z->zbuff; z->out.size =3D z->zbuff_len; z->out.pos =3D 0; =20 - for (i =3D 0; i < pages->num; i++) { + for (i =3D 0; i < pages->normal_num; i++) { ZSTD_EndDirective flush =3D ZSTD_e_continue; =20 - if (i =3D=3D pages->num - 1) { + if (i =3D=3D pages->normal_num - 1) { flush =3D ZSTD_e_flush; } z->in.src =3D p->pages->block->host + pages->offset[i]; @@ -161,10 +163,10 @@ static int zstd_send_prepare(MultiFDSendParams *p, Er= ror **errp) p->iov[p->iovs_num].iov_len =3D z->out.pos; p->iovs_num++; p->next_packet_size =3D z->out.pos; - p->flags |=3D MULTIFD_FLAG_ZSTD; =20 +out: + p->flags |=3D MULTIFD_FLAG_ZSTD; multifd_send_fill_packet(p); - return 0; } =20 @@ -257,6 +259,14 @@ static int zstd_recv_pages(MultiFDRecvParams *p, Error= **errp) p->id, flags, MULTIFD_FLAG_ZSTD); return -1; } + + multifd_recv_zero_page_check(p); + + if (!p->normal_num) { + assert(in_size =3D=3D 0); + return 0; + } + ret =3D qio_channel_read_all(p->c, (void *)z->zbuff, in_size, errp); =20 if (ret !=3D 0) { diff --git a/migration/multifd.c b/migration/multifd.c index 6c07f19af1..a95c763df8 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -11,6 +11,7 @@ */ =20 #include "qemu/osdep.h" +#include "qemu/cutils.h" #include "qemu/rcu.h" #include "exec/target_page.h" #include "sysemu/sysemu.h" @@ -139,6 +140,8 @@ static int nocomp_send_prepare(MultiFDSendParams *p, Er= ror **errp) MultiFDPages_t *pages =3D p->pages; int ret; =20 + multifd_send_zero_page_check(p); + if (!use_zero_copy_send) { /* * Only !zerocopy needs the header in IOV; zerocopy will @@ -147,13 +150,13 @@ static int nocomp_send_prepare(MultiFDSendParams *p, = Error **errp) multifd_send_prepare_header(p); } =20 - for (int i =3D 0; i < pages->num; i++) { + for (int i =3D 0; i < pages->normal_num; i++) { p->iov[p->iovs_num].iov_base =3D pages->block->host + pages->offse= t[i]; p->iov[p->iovs_num].iov_len =3D p->page_size; p->iovs_num++; } =20 - p->next_packet_size =3D pages->num * p->page_size; + p->next_packet_size =3D pages->normal_num * p->page_size; p->flags |=3D MULTIFD_FLAG_NOCOMP; =20 multifd_send_fill_packet(p); @@ -215,6 +218,13 @@ static int nocomp_recv_pages(MultiFDRecvParams *p, Err= or **errp) p->id, flags, MULTIFD_FLAG_NOCOMP); return -1; } + + multifd_recv_zero_page_check(p); + + if (!p->normal_num) { + return 0; + } + for (int i =3D 0; i < p->normal_num; i++) { p->iov[i].iov_base =3D p->host + p->normal[i]; p->iov[i].iov_len =3D p->page_size; @@ -249,6 +259,7 @@ static void multifd_pages_reset(MultiFDPages_t *pages) * overwritten later when reused. */ pages->num =3D 0; + pages->normal_num =3D 0; pages->block =3D NULL; } =20 @@ -340,11 +351,13 @@ void multifd_send_fill_packet(MultiFDSendParams *p) MultiFDPacket_t *packet =3D p->packet; MultiFDPages_t *pages =3D p->pages; uint64_t packet_num; + uint32_t zero_num =3D pages->num - pages->normal_num; int i; =20 packet->flags =3D cpu_to_be32(p->flags); packet->pages_alloc =3D cpu_to_be32(p->pages->allocated); - packet->normal_pages =3D cpu_to_be32(pages->num); + packet->normal_pages =3D cpu_to_be32(pages->normal_num); + packet->zero_pages =3D cpu_to_be32(zero_num); packet->next_packet_size =3D cpu_to_be32(p->next_packet_size); =20 packet_num =3D qatomic_fetch_inc(&multifd_send_state->packet_num); @@ -362,10 +375,11 @@ void multifd_send_fill_packet(MultiFDSendParams *p) } =20 p->packets_sent++; - p->total_normal_pages +=3D pages->num; + p->total_normal_pages +=3D pages->normal_num; + p->total_zero_pages +=3D zero_num; =20 - trace_multifd_send(p->id, packet_num, pages->num, p->flags, - p->next_packet_size); + trace_multifd_send(p->id, packet_num, pages->normal_num, zero_num, + p->flags, p->next_packet_size); } =20 static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) @@ -406,20 +420,29 @@ static int multifd_recv_unfill_packet(MultiFDRecvPara= ms *p, Error **errp) p->normal_num =3D be32_to_cpu(packet->normal_pages); if (p->normal_num > packet->pages_alloc) { error_setg(errp, "multifd: received packet " - "with %u pages and expected maximum pages are %u", + "with %u normal pages and expected maximum pages are %u= ", p->normal_num, packet->pages_alloc) ; return -1; } =20 + p->zero_num =3D be32_to_cpu(packet->zero_pages); + if (p->zero_num > packet->pages_alloc - p->normal_num) { + error_setg(errp, "multifd: received packet " + "with %u zero pages and expected maximum zero pages are= %u", + p->zero_num, packet->pages_alloc - p->normal_num) ; + return -1; + } + p->next_packet_size =3D be32_to_cpu(packet->next_packet_size); p->packet_num =3D be64_to_cpu(packet->packet_num); p->packets_recved++; p->total_normal_pages +=3D p->normal_num; + p->total_zero_pages +=3D p->zero_num; =20 - trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->flags, - p->next_packet_size); + trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->zero_num, + p->flags, p->next_packet_size); =20 - if (p->normal_num =3D=3D 0) { + if (p->normal_num =3D=3D 0 && p->zero_num =3D=3D 0) { return 0; } =20 @@ -445,6 +468,18 @@ static int multifd_recv_unfill_packet(MultiFDRecvParam= s *p, Error **errp) p->normal[i] =3D offset; } =20 + for (i =3D 0; i < p->zero_num; i++) { + uint64_t offset =3D be64_to_cpu(packet->offset[p->normal_num + i]); + + if (offset > (p->block->used_length - p->page_size)) { + error_setg(errp, "multifd: offset too long %" PRIu64 + " (max " RAM_ADDR_FMT ")", + offset, p->block->used_length); + return -1; + } + p->zero[i] =3D offset; + } + return 0; } =20 @@ -837,6 +872,8 @@ static void *multifd_send_thread(void *opaque) =20 stat64_add(&mig_stats.multifd_bytes, p->next_packet_size + p->packet_len); + stat64_add(&mig_stats.normal_pages, pages->normal_num); + stat64_add(&mig_stats.zero_pages, pages->num - pages->normal_n= um); =20 multifd_pages_reset(p->pages); p->next_packet_size =3D 0; @@ -880,7 +917,8 @@ out: =20 rcu_unregister_thread(); migration_threads_remove(thread); - trace_multifd_send_thread_end(p->id, p->packets_sent, p->total_normal_= pages); + trace_multifd_send_thread_end(p->id, p->packets_sent, p->total_normal_= pages, + p->total_zero_pages); =20 return NULL; } @@ -1147,6 +1185,8 @@ static void multifd_recv_cleanup_channel(MultiFDRecvP= arams *p) p->iov =3D NULL; g_free(p->normal); p->normal =3D NULL; + g_free(p->zero); + p->zero =3D NULL; multifd_recv_state->ops->recv_cleanup(p); } =20 @@ -1241,7 +1281,7 @@ static void *multifd_recv_thread(void *opaque) p->flags &=3D ~MULTIFD_FLAG_SYNC; qemu_mutex_unlock(&p->mutex); =20 - if (p->normal_num) { + if (p->normal_num + p->zero_num) { ret =3D multifd_recv_state->ops->recv_pages(p, &local_err); if (ret !=3D 0) { break; @@ -1260,7 +1300,9 @@ static void *multifd_recv_thread(void *opaque) } =20 rcu_unregister_thread(); - trace_multifd_recv_thread_end(p->id, p->packets_recved, p->total_norma= l_pages); + trace_multifd_recv_thread_end(p->id, p->packets_recved, + p->total_normal_pages, + p->total_zero_pages); =20 return NULL; } @@ -1299,6 +1341,7 @@ int multifd_recv_setup(Error **errp) p->name =3D g_strdup_printf("multifdrecv_%d", i); p->iov =3D g_new0(struct iovec, page_count); p->normal =3D g_new0(ram_addr_t, page_count); + p->zero =3D g_new0(ram_addr_t, page_count); p->page_count =3D page_count; p->page_size =3D qemu_target_page_size(); } @@ -1368,3 +1411,17 @@ void multifd_recv_new_channel(QIOChannel *ioc, Error= **errp) QEMU_THREAD_JOINABLE); qatomic_inc(&multifd_recv_state->count); } + +bool multifd_send_prepare_common(MultiFDSendParams *p) +{ + multifd_send_zero_page_check(p); + + if (!p->pages->normal_num) { + p->next_packet_size =3D 0; + return false; + } + + multifd_send_prepare_header(p); + + return true; +} diff --git a/migration/multifd.h b/migration/multifd.h index b3fe27ae93..4b59d5569c 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -48,14 +48,24 @@ typedef struct { /* size of the next packet that contains pages */ uint32_t next_packet_size; uint64_t packet_num; - uint64_t unused[4]; /* Reserved for future use */ + /* zero pages */ + uint32_t zero_pages; + uint32_t unused32[1]; /* Reserved for future use */ + uint64_t unused64[3]; /* Reserved for future use */ char ramblock[256]; + /* + * This array contains the pointers to: + * - normal pages (initial normal_pages entries) + * - zero pages (following zero_pages entries) + */ uint64_t offset[]; } __attribute__((packed)) MultiFDPacket_t; =20 typedef struct { /* number of used pages */ uint32_t num; + /* number of normal pages */ + uint32_t normal_num; /* number of allocated pages */ uint32_t allocated; /* offset of each page */ @@ -122,6 +132,8 @@ typedef struct { uint64_t packets_sent; /* non zero pages sent through this channel */ uint64_t total_normal_pages; + /* zero pages sent through this channel */ + uint64_t total_zero_pages; /* buffers to send */ struct iovec *iov; /* number of iovs used */ @@ -176,12 +188,18 @@ typedef struct { uint8_t *host; /* non zero pages recv through this channel */ uint64_t total_normal_pages; + /* zero pages recv through this channel */ + uint64_t total_zero_pages; /* buffers to recv */ struct iovec *iov; /* Pages that are not zero */ ram_addr_t *normal; /* num of non zero pages */ uint32_t normal_num; + /* Pages that are zero */ + ram_addr_t *zero; + /* num of zero pages */ + uint32_t zero_num; /* used for de-compression methods */ void *data; } MultiFDRecvParams; @@ -203,6 +221,9 @@ typedef struct { =20 void multifd_register_ops(int method, MultiFDMethods *ops); void multifd_send_fill_packet(MultiFDSendParams *p); +bool multifd_send_prepare_common(MultiFDSendParams *p); +void multifd_send_zero_page_check(MultiFDSendParams *p); +void multifd_recv_zero_page_check(MultiFDRecvParams *p); =20 static inline void multifd_send_prepare_header(MultiFDSendParams *p) { @@ -211,5 +232,4 @@ static inline void multifd_send_prepare_header(MultiFDS= endParams *p) p->iovs_num++; } =20 - #endif diff --git a/migration/ram.c b/migration/ram.c index 2581ee1e04..e1fa229acf 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1258,7 +1258,6 @@ static int ram_save_multifd_page(RAMBlock *block, ram= _addr_t offset) if (!multifd_queue_page(block, offset)) { return -1; } - stat64_add(&mig_stats.normal_pages, 1); =20 return 1; } diff --git a/migration/trace-events b/migration/trace-events index 298ad2b0dd..9f1d7ae71a 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -128,21 +128,21 @@ postcopy_preempt_reset_channel(void) "" # multifd.c multifd_new_send_channel_async(uint8_t id) "channel %u" multifd_new_send_channel_async_error(uint8_t id, void *err) "channel=3D%u = err=3D%p" -multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flag= s, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " pages %u f= lags 0x%x next packet size %u" +multifd_recv(uint8_t id, uint64_t packet_num, uint32_t normal, uint32_t ze= ro, uint32_t flags, uint32_t next_packet_size) "channel %u packet_num %" PR= Iu64 " normal pages %u zero pages %u flags 0x%x next packet size %u" multifd_recv_new_channel(uint8_t id) "channel %u" multifd_recv_sync_main(long packet_num) "packet num %ld" multifd_recv_sync_main_signal(uint8_t id) "channel %u" multifd_recv_sync_main_wait(uint8_t id) "channel %u" multifd_recv_terminate_threads(bool error) "error %d" -multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "cha= nnel %u packets %" PRIu64 " pages %" PRIu64 +multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal pages %" PRI= u64 " zero pages %" PRIu64 multifd_recv_thread_start(uint8_t id) "%u" -multifd_send(uint8_t id, uint64_t packet_num, uint32_t normal, uint32_t fl= ags, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " normal p= ages %u flags 0x%x next packet size %u" +multifd_send(uint8_t id, uint64_t packet_num, uint32_t normal_pages, uint3= 2_t zero_pages, uint32_t flags, uint32_t next_packet_size) "channel %u pack= et_num %" PRIu64 " normal pages %u zero pages %u flags 0x%x next packet siz= e %u" multifd_send_error(uint8_t id) "channel %u" multifd_send_sync_main(long packet_num) "packet num %ld" multifd_send_sync_main_signal(uint8_t id) "channel %u" multifd_send_sync_main_wait(uint8_t id) "channel %u" multifd_send_terminate_threads(void) "" -multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s) "channel %u packets %" PRIu64 " normal pages %" PRIu64 +multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal pages %" PR= Iu64 " zero pages %" PRIu64 multifd_send_thread_start(uint8_t id) "%u" multifd_tls_outgoing_handshake_start(void *ioc, void *tioc, const char *ho= stname) "ioc=3D%p tioc=3D%p hostname=3D%s" multifd_tls_outgoing_handshake_error(void *ioc, const char *err) "ioc=3D%p= err=3D%s" diff --git a/qapi/migration.json b/qapi/migration.json index 8da05dba47..846d0411d5 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -671,10 +671,15 @@ # # @legacy: Perform zero page checking in main migration thread. # +# @multifd: Perform zero page checking in multifd sender thread. +# This option only takes effect if migration capability multifd +# is set. Otherwise, it will have the same effect as legacy. +# # Since: 9.0 +# ## { 'enum': 'ZeroPageDetection', - 'data': [ 'none', 'legacy' ] } + 'data': [ 'none', 'legacy', 'multifd' ] } =20 ## # @BitmapMigrationBitmapAliasTransform: --=20 2.30.2 From nobody Tue Nov 26 06:29:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1709260290; cv=none; d=zohomail.com; s=zohoarc; b=SUwyamU/8AFq3pJCYus+gAp2zWgIC0iT+vuY4liU1LZ1MSk+nA9aqaQRmVywO4JRZSjnPQOuyHCm1M6lbcX8NuZ3uWtRSJiFP28fH2THub2GAxK4DW+IJ49eu4s3My10zt6sDDqgW84zTZJOrynjTFgS0a9uxaIY2G0DydTtY3M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709260290; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=PxHV8roC2fjNMfvMvdxuHI9CffnQMqXm91qkh+DJCQw=; b=guvUgpWUEKZ1xKK24nEuuGZfrOcT5gUTptneLi5VOERFI75b/Q8cYoqDJxAlxYwbUGoUiYkcu6I5Oh7FcqUTjww1O0SGHpM/bh9xFmPEkC/yVwhk9XjTqTOz524Oy07RK1z913m19am/HpAdyJXXZkF7GjiGdC65unT5Rqp1ut4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709260290312590.9263953486504; Thu, 29 Feb 2024 18:31:30 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rfseR-0003y3-FI; Thu, 29 Feb 2024 21:29:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rfseQ-0003xk-5j for qemu-devel@nongnu.org; Thu, 29 Feb 2024 21:29:30 -0500 Received: from mail-qv1-xf31.google.com ([2607:f8b0:4864:20::f31]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rfseO-0005PA-K8 for qemu-devel@nongnu.org; Thu, 29 Feb 2024 21:29:29 -0500 Received: by mail-qv1-xf31.google.com with SMTP id 6a1803df08f44-68f571be9ddso12293036d6.0 for ; Thu, 29 Feb 2024 18:29:28 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.120]) by smtp.gmail.com with ESMTPSA id qo13-20020a056214590d00b0068d11cf887bsm1384107qvb.55.2024.02.29.18.29.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 18:29:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1709260167; x=1709864967; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PxHV8roC2fjNMfvMvdxuHI9CffnQMqXm91qkh+DJCQw=; b=UeTX4wiNdcfn8fEk80jfmdJhGNCH2jkbWNAVQld0oEUHscr3ykVSR41go+1A6usqAE yXA7mPrz96t/u4AR/VEPI+OAL2rG5e6aHQWRssRd0QHDf2mzHg/fcHwJ/1gb7Oj58Qlv 3mL0mV9E0viy3eUlCWl0R2p5oSVSkEjRqp9o5FFth/7rFg/ugTThWjzb5Au4I9mJMn12 y98EAFKTxHnzphGRJVOqfHoLwj/xkE/RQYnv9i7a2I/cG2b6Q2PkiqfRcxTbXxPjG5L5 Uk2W0yyzorTWrfutPxe28UH7f/Th0AtK1MyWTBmsZqPhAUMBoqgaL7d2Raj28/piRGKP K5YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709260167; x=1709864967; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PxHV8roC2fjNMfvMvdxuHI9CffnQMqXm91qkh+DJCQw=; b=ZtsR04A68xl22HXeRNfEN+q9ymOO/z1wyPvu49tOHHtyiCGxDR4W/kLOj27hgcyubv gWTaqD352HG3lC4wm7qaSJOr9Ur7FJLR1QUUk7pe6dtsOgOIQNp/MzEhHP8adkXYXQRk g5pvDP7zj3Zo2coP6WuQkOWUXswBC0KmNLAoE5O53lJAJbB8/hv7Armpz7KkyU9DJ4Kt wln9tEyTujTESOxEDvrP4tJQRz8j24rGzg9Z2+bdve8KHx7MZzpYnsr3Mi31YM2RjmY3 tx7sn3gYsyH7WXZo+FvGrXfrq2oXPJVJNmb3TmUsXHYvhPFUoA5LngXpcCtTsVz6+Cga JmtA== X-Forwarded-Encrypted: i=1; AJvYcCWfeS9Ck85ZcczGk+nAI3yFVDj9ft8CaxKFvTB0xOLK6KeDQbtCDsHHTy+sHZKLgeH1indkPCw10qE+UuzCKM2TgUVESg4= X-Gm-Message-State: AOJu0Yy09XusL8zhVUXAuEn9ZuEkOnKK0yCG2gOFiB9PMM2BRCrHzKGl 7hPqJ7vR3OjO25emmI1VX3jK2PUJILouWDfXGdvCz5bSSL6VBNoqNbUVDti4T3k= X-Google-Smtp-Source: AGHT+IEsvuwiDHIehYUztD4V9Y3AFzD+uUaCTByOxeGLhLo5VYfEnJbjvXFFqFAYYgVKvUkV2xUmtw== X-Received: by 2002:a0c:f50c:0:b0:690:50c5:4b72 with SMTP id j12-20020a0cf50c000000b0069050c54b72mr468754qvm.52.1709260167598; Thu, 29 Feb 2024 18:29:27 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, jdenemar@redhat.com, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, qemu-devel@nongnu.org Cc: Hao Xiang Subject: [PATCH v4 3/7] migration/multifd: Implement ram_save_target_page_multifd to handle multifd version of MigrationOps::ram_save_target_page. Date: Fri, 1 Mar 2024 02:28:25 +0000 Message-Id: <20240301022829.3390548-4-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240301022829.3390548-1-hao.xiang@bytedance.com> References: <20240301022829.3390548-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::f31; envelope-from=hao.xiang@bytedance.com; helo=mail-qv1-xf31.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1709260292053100001 Content-Type: text/plain; charset="utf-8" 1. Add a dedicated handler for MigrationOps::ram_save_target_page in multifd live migration. 2. Refactor ram_save_target_page_legacy so that the legacy and multifd handlers don't have internal functions calling into each other. Signed-off-by: Hao Xiang Reviewed-by: Fabiano Rosas Message-Id: <20240226195654.934709-4-hao.xiang@bytedance.com> --- migration/ram.c | 43 ++++++++++++++++++++++++++++++------------- 1 file changed, 30 insertions(+), 13 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index e1fa229acf..f9d6ea65cc 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1122,10 +1122,6 @@ static int save_zero_page(RAMState *rs, PageSearchSt= atus *pss, QEMUFile *file =3D pss->pss_channel; int len =3D 0; =20 - if (migrate_zero_page_detection() =3D=3D ZERO_PAGE_DETECTION_NONE) { - return 0; - } - if (!buffer_is_zero(p, TARGET_PAGE_SIZE)) { return 0; } @@ -2045,7 +2041,6 @@ static bool save_compress_page(RAMState *rs, PageSear= chStatus *pss, */ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss) { - RAMBlock *block =3D pss->block; ram_addr_t offset =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; int res; =20 @@ -2061,17 +2056,34 @@ static int ram_save_target_page_legacy(RAMState *rs= , PageSearchStatus *pss) return 1; } =20 + return ram_save_page(rs, pss); +} + +/** + * ram_save_target_page_multifd: send one target page to multifd workers + * + * Returns 1 if the page was queued, -1 otherwise. + * + * @rs: current RAM state + * @pss: data about the page we want to send + */ +static int ram_save_target_page_multifd(RAMState *rs, PageSearchStatus *ps= s) +{ + RAMBlock *block =3D pss->block; + ram_addr_t offset =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; + /* - * Do not use multifd in postcopy as one whole host page should be - * placed. Meanwhile postcopy requires atomic update of pages, so even - * if host page size =3D=3D guest page size the dest guest during run = may - * still see partially copied pages which is data corruption. + * Backward compatibility support. While using multifd live + * migration, we still need to handle zero page checking on the + * migration main thread. */ - if (migrate_multifd() && !migration_in_postcopy()) { - return ram_save_multifd_page(block, offset); + if (migrate_zero_page_detection() =3D=3D ZERO_PAGE_DETECTION_LEGACY) { + if (save_zero_page(rs, pss, offset)) { + return 1; + } } =20 - return ram_save_page(rs, pss); + return ram_save_multifd_page(block, offset); } =20 /* Should be called before sending a host page */ @@ -2983,7 +2995,12 @@ static int ram_save_setup(QEMUFile *f, void *opaque) } =20 migration_ops =3D g_malloc0(sizeof(MigrationOps)); - migration_ops->ram_save_target_page =3D ram_save_target_page_legacy; + + if (migrate_multifd()) { + migration_ops->ram_save_target_page =3D ram_save_target_page_multi= fd; + } else { + migration_ops->ram_save_target_page =3D ram_save_target_page_legac= y; + } =20 bql_unlock(); ret =3D multifd_send_sync_main(); --=20 2.30.2 From nobody Tue Nov 26 06:29:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1709260241; cv=none; d=zohomail.com; s=zohoarc; b=NOlCY9hOciPHqkbT+q0zKpQaYm3epGcKfF9Da6i9QnW9KI6iMArhNVEZi1aAQmDcrSLEiwdOU6nS6sMjfvqvJMlETJy7igoykwYDKMkJsqKxlS91FF3zfgpidFc3Tspnj4o+/TbegRmwbw4Fhn+G0FAmH1u+mVOwDIrUa0k9rc0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709260241; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9QASZ+kUUj+8Xo1I0M095jiimdEHlGS7QcRB7kGSAjM=; b=gXUW6jDIax1QvIvQOAn8AOTNUkgWrX2Oe/RX5iuMfL9BbFYNr4ZaiMtcKHlskZKb6+DDvmPv9jKMtRiNkeDHiF+wHqcCqjQCyQEiZ4dYzYJvoX4s2GoEVYZlRotxUNSKF/CFB5vN1Sv5Zcwnf6t961XVnSprUmQvhiyRMTxwwAQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17092602414011009.6002356560787; Thu, 29 Feb 2024 18:30:41 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rfseU-00040Q-9d; Thu, 29 Feb 2024 21:29:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rfseS-0003yO-OM for qemu-devel@nongnu.org; Thu, 29 Feb 2024 21:29:32 -0500 Received: from mail-oo1-xc2c.google.com ([2607:f8b0:4864:20::c2c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rfseR-0005PY-6b for qemu-devel@nongnu.org; Thu, 29 Feb 2024 21:29:32 -0500 Received: by mail-oo1-xc2c.google.com with SMTP id 006d021491bc7-59fdcf8ebbcso966597eaf.3 for ; Thu, 29 Feb 2024 18:29:30 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.120]) by smtp.gmail.com with ESMTPSA id qo13-20020a056214590d00b0068d11cf887bsm1384107qvb.55.2024.02.29.18.29.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 18:29:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1709260169; x=1709864969; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9QASZ+kUUj+8Xo1I0M095jiimdEHlGS7QcRB7kGSAjM=; b=At3xnI9CynzdhegHkjKGvWF2o2xTfim8N76glp0wRSYl0TsukYLIuMehXiICZUYODI X7VRGDT/lJ+i+97H75O8wlFAxHrH6WUxf2tHHFcovyG8YDbrD5gIUMXh20ZqeGl9iD1D 1IijNsLLDkrjQRRNY07F8A+LIZYXnE3A+EpBcpVqQsJnklutHnvGMkuljA+WxOBdKzYe D2GW919s93KnzxWB9F32yTcAKfmW7rS6l3YpcssjEfjNtl9g5c5hHg6RYdCJ+dUpC04U 6SDmcOpChxn+Xd7RTcNVts24Y5Xqc3HfXOr8PlxOPHcB2wX/ta1O7qaUjw8hI1Su7HT9 GcVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709260169; x=1709864969; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9QASZ+kUUj+8Xo1I0M095jiimdEHlGS7QcRB7kGSAjM=; b=pHs7H4m3ZaLjXDimcKJTRTTATEmc/jpRyF++0BLxsFB3AEi8wvEuEhRFN0TgnzT+VP EXiM/4nRQVh9r4g+iRrPzYHFhTXGp2cQ5vuFaNaCQDOC/LcQqRL93lKZp5CAegenxwa3 UVe/noHTDwvEHPFhIQyj0m+lsBV9k8Qp70/bAieiBq50SQNGpCa/zPgyG+wRazLg/ds9 ZwO1guLIgl4VCWD4cwZNuH0oEsNx/Qd6T4XooduZSSI3ezUiHOycRL+b3l26wrqURBkx DbMCURNE1KBBFozWLr0Vs8Gc3AUXV+Ck2kmUSCvYcmExIcpxnznFBydu1Al+D2/zeyST dYMw== X-Forwarded-Encrypted: i=1; AJvYcCXuLLF5VXzzKSKisvyrj5ENKnEoaNQe83j4B0Qe2XSJaxWPfg+41+eOuYfM38xco/85ButT73WVTcFAPKqRmG1PtXEfRgs= X-Gm-Message-State: AOJu0YxkaoTAkl0lBWrC3vdAflAeE5IMq5iis+lvRnVgtfgJDjS3usXS ljLjKj5q3dNQrM8PE+uosfSqegiOjakTvCXaZKI9ov1ypFegmrCQpZlZGahBBuY= X-Google-Smtp-Source: AGHT+IHCUMJESzCLFs5UxLwPYWIGP8OUZsg6oDToiPo7NJOAwQ4Jm0UUY3ZFp2VGaZ1igCC6/JhJuw== X-Received: by 2002:a05:6359:4117:b0:17b:ef19:ee7d with SMTP id kh23-20020a056359411700b0017bef19ee7dmr354422rwc.26.1709260169160; Thu, 29 Feb 2024 18:29:29 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, jdenemar@redhat.com, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, qemu-devel@nongnu.org Cc: Hao Xiang Subject: [PATCH v4 4/7] migration/multifd: Enable multifd zero page checking by default. Date: Fri, 1 Mar 2024 02:28:26 +0000 Message-Id: <20240301022829.3390548-5-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240301022829.3390548-1-hao.xiang@bytedance.com> References: <20240301022829.3390548-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::c2c; envelope-from=hao.xiang@bytedance.com; helo=mail-oo1-xc2c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1709260241901100001 Content-Type: text/plain; charset="utf-8" Set default "zero-page-detection" option to "multifd". Now zero page checking can be done in the multifd threads and this becomes the default configuration. We still provide backward compatibility where zero page checking is done from the migration main thread. Signed-off-by: Hao Xiang --- migration/options.c | 2 +- qapi/migration.json | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/migration/options.c b/migration/options.c index 3c603391b0..3c79b6ccd4 100644 --- a/migration/options.c +++ b/migration/options.c @@ -181,7 +181,7 @@ Property migration_properties[] =3D { MIG_MODE_NORMAL), DEFINE_PROP_ZERO_PAGE_DETECTION("zero-page-detection", MigrationState, parameters.zero_page_detection, - ZERO_PAGE_DETECTION_LEGACY), + ZERO_PAGE_DETECTION_MULTIFD), =20 /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), diff --git a/qapi/migration.json b/qapi/migration.json index 846d0411d5..ca9561fbf1 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -903,7 +903,7 @@ # (Since 8.2) # # @zero-page-detection: Whether and how to detect zero pages. -# See description in @ZeroPageDetection. Default is 'legacy'. +# See description in @ZeroPageDetection. Default is 'multifd'. # (since 9.0) # # Features: @@ -1100,7 +1100,7 @@ # (Since 8.2) # # @zero-page-detection: Whether and how to detect zero pages. -# See description in @ZeroPageDetection. Default is 'legacy'. +# See description in @ZeroPageDetection. Default is 'multifd'. # (since 9.0) # # Features: @@ -1333,7 +1333,7 @@ # (Since 8.2) # # @zero-page-detection: Whether and how to detect zero pages. -# See description in @ZeroPageDetection. Default is 'legacy'. +# See description in @ZeroPageDetection. Default is 'multifd'. # (since 9.0) # # Features: --=20 2.30.2 From nobody Tue Nov 26 06:29:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1709260234; cv=none; d=zohomail.com; s=zohoarc; b=fkziULGYFi2UZtJXPhrX4VD0liopk3wvKj924oOt7uuxWhemMO5D5yTEtf8s0Ly1yYfCzCJ+VqbPnac8KAql95LcV1hx/EKhyGartis/aJ3mWWojtLqNPZDcXuleDrsd4UX9alfzGcXvdi31ir9IWuzfi3vx60wcxwR+qEmoZB8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709260234; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=xMtilp2rV73+XgiHl3BV7scdj6VbpaZuDbPLXK6KWJY=; b=bDxyzuF84y92MjiX5fGKqTVl0bdiOoaNKMHYQufvSy9rqwI3ol/OdkyqJrFXWICbOrBbFQRImjqKwCZ22FRLuZQYXBLK+B5LofY1fn7KcGft/0OqTlMPUUnaBIe49myk81bI8yi8dw35UzrlDooONruEs1fgyIXhj1lTqx5X8rk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709260234959730.6392283153702; Thu, 29 Feb 2024 18:30:34 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rfseW-00041V-UC; Thu, 29 Feb 2024 21:29:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rfseV-00040t-AM for qemu-devel@nongnu.org; Thu, 29 Feb 2024 21:29:35 -0500 Received: from mail-oo1-xc2b.google.com ([2607:f8b0:4864:20::c2b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rfseS-0005Py-M3 for qemu-devel@nongnu.org; Thu, 29 Feb 2024 21:29:35 -0500 Received: by mail-oo1-xc2b.google.com with SMTP id 006d021491bc7-5a0deaf21efso913834eaf.0 for ; Thu, 29 Feb 2024 18:29:32 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.120]) by smtp.gmail.com with ESMTPSA id qo13-20020a056214590d00b0068d11cf887bsm1384107qvb.55.2024.02.29.18.29.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 18:29:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1709260171; x=1709864971; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xMtilp2rV73+XgiHl3BV7scdj6VbpaZuDbPLXK6KWJY=; b=E8NnPrC45Sg5DcH3s6I3tYlI65LFCLAOV/E6evkF4iCiERAJcbEDSpgZHKLeE6YsTh hVRHUru/Z2xxmoxLU2L4Kfg99sO03ei18rbA0zEH5mGJM9jTtytAxrAHRB3zuEI/zmsd 0zNV32oyQ3rL/6YAv/viCojCyAdi8bae/q0T8oilY+V0lF6l1xwjVo9zBZQJYUSKuljU 8bns89Oijfbskwy4S0roWw5EfZBD+fvZkABPqWiv7XP/ZYxC5nvK42p3G2xVcuDbtqXI nR2vYA3YpBfKc/FmMs6ALsH6opSuq+kMKFtVEKNaVxecbtRT1+wMbIGAMeba7vrpyQMM uHyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709260171; x=1709864971; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xMtilp2rV73+XgiHl3BV7scdj6VbpaZuDbPLXK6KWJY=; b=wieopX7cO8f9vK3PIlaja6Uh6PzXT7juQ7Kyl1wVskhLThY245KbV6XhYJHO3xTGAS YawXisYiAEFQTCof9XMsNEre/lZ0Y/yU/u6rVXA9OZcdpKpsIwgtGBaECJAkBttqfr8p dDY/Bqjsi0RcGURD2qEowWTtKM74pGUrjFt8tWS3Iy3qYxDqSJjDGD4p5Z6huLMyWXDl Ed3VJAo6jTip5Aqyh7AYGS67j5YPrmAGhQhMB/ZWD/5/4nE/QwwyxjNUACkqcgVaXvZe 8o7fb48AHQ7SV8gu1MC+lBjqs6KG4G3gdFTEimEBpqSYiFrpCPSq+lQFV4wdP4J/5u7A TnSQ== X-Forwarded-Encrypted: i=1; AJvYcCUtSwiOKPhS8+aaoCZ9/5IaA50wfbKgtPZECNpif/IZbbY//oed8POyqA8cojEaY7FAu8cKZClGC//I6C+45VQQviqb53Q= X-Gm-Message-State: AOJu0YwuSVY1g3SiX/fUs68cqHdCTQ1gcEp6qkFTslXKkqz9hH61Jh5q d8iEGMELNQk2yj1CzP4AJvXZN1VIDYep7vxvEl69ja4Y/SMLsIQIgC4b/+KuoL0= X-Google-Smtp-Source: AGHT+IGW3CviKShkGk7LY0aFrVPrjPSYmVm6r8+4vnqkx5REQcCSBztCIUk/qpWzmXFMXt7L3b43mQ== X-Received: by 2002:a05:6358:6383:b0:17b:f319:9449 with SMTP id k3-20020a056358638300b0017bf3199449mr331677rwh.7.1709260171451; Thu, 29 Feb 2024 18:29:31 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, jdenemar@redhat.com, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, qemu-devel@nongnu.org Cc: Hao Xiang Subject: [PATCH v4 5/7] migration/multifd: Add new migration test cases for legacy zero page checking. Date: Fri, 1 Mar 2024 02:28:27 +0000 Message-Id: <20240301022829.3390548-6-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240301022829.3390548-1-hao.xiang@bytedance.com> References: <20240301022829.3390548-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::c2b; envelope-from=hao.xiang@bytedance.com; helo=mail-oo1-xc2b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1709260236131100001 Content-Type: text/plain; charset="utf-8" Now that zero page checking is done on the multifd sender threads by default, we still provide an option for backward compatibility. This change adds a qtest migration test case to set the zero-page-detection option to "legacy" and run multifd migration with zero page checking on the migration main thread. Signed-off-by: Hao Xiang Reviewed-by: Peter Xu --- tests/qtest/migration-test.c | 52 ++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 83512bce85..8a966364b5 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -2660,6 +2660,24 @@ test_migrate_precopy_tcp_multifd_start(QTestState *f= rom, return test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); } =20 +static void * +test_migrate_precopy_tcp_multifd_start_zero_page_legacy(QTestState *from, + QTestState *to) +{ + test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); + migrate_set_parameter_str(from, "zero-page-detection", "legacy"); + return NULL; +} + +static void * +test_migration_precopy_tcp_multifd_start_no_zero_page(QTestState *from, + QTestState *to) +{ + test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); + migrate_set_parameter_str(from, "zero-page-detection", "none"); + return NULL; +} + static void * test_migrate_precopy_tcp_multifd_zlib_start(QTestState *from, QTestState *to) @@ -2691,6 +2709,36 @@ static void test_multifd_tcp_none(void) test_precopy_common(&args); } =20 +static void test_multifd_tcp_zero_page_legacy(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "defer", + .start_hook =3D test_migrate_precopy_tcp_multifd_start_zero_page_l= egacy, + /* + * Multifd is more complicated than most of the features, it + * directly takes guest page buffers when sending, make sure + * everything will work alright even if guest page is changing. + */ + .live =3D true, + }; + test_precopy_common(&args); +} + +static void test_multifd_tcp_no_zero_page(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "defer", + .start_hook =3D test_migration_precopy_tcp_multifd_start_no_zero_p= age, + /* + * Multifd is more complicated than most of the features, it + * directly takes guest page buffers when sending, make sure + * everything will work alright even if guest page is changing. + */ + .live =3D true, + }; + test_precopy_common(&args); +} + static void test_multifd_tcp_zlib(void) { MigrateCommon args =3D { @@ -3592,6 +3640,10 @@ int main(int argc, char **argv) } migration_test_add("/migration/multifd/tcp/plain/none", test_multifd_tcp_none); + migration_test_add("/migration/multifd/tcp/plain/zero-page/legacy", + test_multifd_tcp_zero_page_legacy); + migration_test_add("/migration/multifd/tcp/plain/zero-page/none", + test_multifd_tcp_no_zero_page); migration_test_add("/migration/multifd/tcp/plain/cancel", test_multifd_tcp_cancel); migration_test_add("/migration/multifd/tcp/plain/zlib", --=20 2.30.2 From nobody Tue Nov 26 06:29:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1709260246; cv=none; d=zohomail.com; s=zohoarc; b=C7TwrHn19zzezDfMCsKM8l8SiF8OfhrsjcOr5qoq4TFM7F2n8CJL/kAKMgtTR0bOcEgi0K4CJG8fFfjtq6xZ0X/put8kX/vPtQlXh6fXfaeqRVtZDgA2GpwNOF7656T11BqlhT5R6iTRuHzEgTGuE82S9g4dTLFXgT1R3NBFE+0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709260246; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9JKPdceht9UhXxil7+9Basmri/1zxv20YhwsApVqg0s=; b=V5Tr+lXTiE46aga74Bg4KKmNXD9NdUW4lYVQVMK5ru0PaGYtP/iSFfisRbffD1UTdxtFpL2tYEIs6cNin6/yqKL4+L3QdSMuYxqCe/a9Z4jlfn+jlxINeGmLMMC8+kPboxFdO3KwWFMEAyg5+4tyFnhvCpPNrGmgEosAlIVE6O4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709260246921777.6684975690127; Thu, 29 Feb 2024 18:30:46 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rfseb-00042S-Vw; Thu, 29 Feb 2024 21:29:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rfsea-00042I-Sh for qemu-devel@nongnu.org; Thu, 29 Feb 2024 21:29:40 -0500 Received: from mail-qv1-xf2e.google.com ([2607:f8b0:4864:20::f2e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rfseZ-0005Rm-Av for qemu-devel@nongnu.org; Thu, 29 Feb 2024 21:29:40 -0500 Received: by mail-qv1-xf2e.google.com with SMTP id 6a1803df08f44-6900f479e3cso14755466d6.0 for ; Thu, 29 Feb 2024 18:29:39 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.120]) by smtp.gmail.com with ESMTPSA id qo13-20020a056214590d00b0068d11cf887bsm1384107qvb.55.2024.02.29.18.29.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 18:29:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1709260178; x=1709864978; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9JKPdceht9UhXxil7+9Basmri/1zxv20YhwsApVqg0s=; b=k7YQqXy2yYDFvBAQ/zbONW60RC1flyTMWFarBXMaAugA1pe/17bxP+uwQXjqcUedHK 47SspwcUaEwLu2is3wIyH+CERA/XqDVcSx8RIclqI8v7P+L5hL/HK8b/nSK1olyVOogQ OdqyZWlh2s3EnBk5Seier0vrfmHQTooqzsU7rHdr3+umyuaGjq4++jppKq6dIfnNx8KM ZQ1nm6uiezvDuLH79Y18VMBPBTEUvwufDLkxt2xi4VGSo7l7JyHim64bmS1JgScbjByD u4lv++BW6yROGTTWtxagJTjQ8gOxGCUAr2JTV1lIC95k1hLe8sdhfibAbtJQe0QyTt8Y +tyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709260178; x=1709864978; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9JKPdceht9UhXxil7+9Basmri/1zxv20YhwsApVqg0s=; b=LtUCgIX2DCJFAAJeRz95KH9Hoasby9/TMW1wjvwD524lz6OZKz3qM8Z8UquaUOzyen x8l2Vc/24ZBx8NuR+Gu1jqt+9nW0GYlGwENKHrKS9E+5E5scWYl825RPNNRiBK6STbEY urYLjFYJBZos7o6f2V8WnKzT8VikOCGdg6W7TVobrySphG4y9KO32ysn60PH6dEzxrcv EXywbe8xNO/k+jzKE3iZNFY+BkC5+tH14Npji8sFXs97hNB61d46rbxgp3AzaYiVIA/1 bhXVlAqgzNWVUi9Fju1yhy+icill1RlOSfDGMXlfALsD1JKitsv7mBzvflVIlfSvBLFX qATA== X-Forwarded-Encrypted: i=1; AJvYcCVCP+lvkeZ/H6AlvathkB3zA7XZ6NO+Ig7MHpH7fDmO1r60262MVlT0X9bxwP/iyVtjDWDJAvMnUCwNqQFvWD40uXdnfy4= X-Gm-Message-State: AOJu0Yw4PZB3AZFipWy1MdCGtrxNGjTOVTtcUW1wrXXQGiFa6H9RV3T6 PVKa0dGj+bPswaBp5PZveS18sI1lSXuj9QtqfKmPehfDxuGIdV2gVj2GB39qmCU= X-Google-Smtp-Source: AGHT+IEFfwMhJqX/QVOdR18hRg4UO3YYfvB5bT7wdSIVj3+cOC7HtMkZKUtpNKux4uvI6EPgFebMXA== X-Received: by 2002:a0c:c603:0:b0:690:690:a5e2 with SMTP id v3-20020a0cc603000000b006900690a5e2mr472835qvi.10.1709260178358; Thu, 29 Feb 2024 18:29:38 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, jdenemar@redhat.com, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, qemu-devel@nongnu.org Cc: Hao Xiang Subject: [PATCH v4 6/7] migration/multifd: Add zero pages and zero bytes counter to migration status interface. Date: Fri, 1 Mar 2024 02:28:28 +0000 Message-Id: <20240301022829.3390548-7-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240301022829.3390548-1-hao.xiang@bytedance.com> References: <20240301022829.3390548-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::f2e; envelope-from=hao.xiang@bytedance.com; helo=mail-qv1-xf2e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1709260247968100001 Content-Type: text/plain; charset="utf-8" This change extends the MigrationStatus interface to track zero pages and zero bytes counter. Signed-off-by: Hao Xiang --- migration/migration-hmp-cmds.c | 4 ++++ migration/migration.c | 2 ++ qapi/migration.json | 15 ++++++++++++++- tests/migration/guestperf/engine.py | 2 ++ 4 files changed, 22 insertions(+), 1 deletion(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 7e96ae6ffd..a38ad0255d 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -111,6 +111,10 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) info->ram->normal); monitor_printf(mon, "normal bytes: %" PRIu64 " kbytes\n", info->ram->normal_bytes >> 10); + monitor_printf(mon, "zero pages: %" PRIu64 " pages\n", + info->ram->zero_pages); + monitor_printf(mon, "zero bytes: %" PRIu64 " kbytes\n", + info->ram->zero_bytes >> 10); monitor_printf(mon, "dirty sync count: %" PRIu64 "\n", info->ram->dirty_sync_count); monitor_printf(mon, "page size: %" PRIu64 " kbytes\n", diff --git a/migration/migration.c b/migration/migration.c index bab68bcbef..fc4e3ef52d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1126,6 +1126,8 @@ static void populate_ram_info(MigrationInfo *info, Mi= grationState *s) info->ram->skipped =3D 0; info->ram->normal =3D stat64_get(&mig_stats.normal_pages); info->ram->normal_bytes =3D info->ram->normal * page_size; + info->ram->zero_pages =3D stat64_get(&mig_stats.zero_pages); + info->ram->zero_bytes =3D info->ram->zero_pages * page_size; info->ram->mbps =3D s->mbps; info->ram->dirty_sync_count =3D stat64_get(&mig_stats.dirty_sync_count); diff --git a/qapi/migration.json b/qapi/migration.json index ca9561fbf1..03b850bab7 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -63,6 +63,10 @@ # between 0 and @dirty-sync-count * @multifd-channels. (since # 7.1) # +# @zero-pages: number of zero pages (since 9.0) +# +# @zero-bytes: number of zero bytes sent (since 9.0) +# # Features: # # @deprecated: Member @skipped is always zero since 1.5.3 @@ -81,7 +85,8 @@ 'multifd-bytes': 'uint64', 'pages-per-second': 'uint64', 'precopy-bytes': 'uint64', 'downtime-bytes': 'uint64', 'postcopy-bytes': 'uint64', - 'dirty-sync-missed-zero-copy': 'uint64' } } + 'dirty-sync-missed-zero-copy': 'uint64', + 'zero-pages': 'int', 'zero-bytes': 'size' } } =20 ## # @XBZRLECacheStats: @@ -332,6 +337,8 @@ # "duplicate":123, # "normal":123, # "normal-bytes":123456, +# "zero-pages":123, +# "zero-bytes":123456, # "dirty-sync-count":15 # } # } @@ -358,6 +365,8 @@ # "duplicate":123, # "normal":123, # "normal-bytes":123456, +# "zero-pages":123, +# "zero-bytes":123456, # "dirty-sync-count":15 # } # } @@ -379,6 +388,8 @@ # "duplicate":123, # "normal":123, # "normal-bytes":123456, +# "zero-pages":123, +# "zero-bytes":123456, # "dirty-sync-count":15 # }, # "disk":{ @@ -405,6 +416,8 @@ # "duplicate":10, # "normal":3333, # "normal-bytes":3412992, +# "zero-pages":3333, +# "zero-bytes":3412992, # "dirty-sync-count":15 # }, # "xbzrle-cache":{ diff --git a/tests/migration/guestperf/engine.py b/tests/migration/guestper= f/engine.py index 608d7270f6..693e07c227 100644 --- a/tests/migration/guestperf/engine.py +++ b/tests/migration/guestperf/engine.py @@ -92,6 +92,8 @@ def _migrate_progress(self, vm): info["ram"].get("skipped", 0), info["ram"].get("normal", 0), info["ram"].get("normal-bytes", 0), + info["ram"].get("zero-pages", 0); + info["ram"].get("zero-bytes", 0); info["ram"].get("dirty-pages-rate", 0), info["ram"].get("mbps", 0), info["ram"].get("dirty-sync-count", 0) --=20 2.30.2 From nobody Tue Nov 26 06:29:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1709260249; cv=none; d=zohomail.com; s=zohoarc; b=n5hPgkogNMSkvsCDKopn8PFuV/Bz5+LbMPgJ1LL50N+sgc6+1BbT0e1vPT6O1lzyC/dEhLUohMAtiUJbhV7je9N7ZOWCfpjxH6m0m7WVpGYI+UOzkc8msQ3vWRabXX1vfFbWV5VPQBd4vayNG3d6xFRETxW365J1btsw2MF9+xE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709260249; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=yw8iSnc3LXROkWLue6u8GAvL3L94OebgZkCfuGQmnSk=; b=YgjsR9wFK5Gsqb4042CL8f54fEhYl/tDtUMApBAAfv2ReLITKekgsdSWfISeGcTT4gLIr7F0Is3DrkUdreezWHOYhf/EZkCGLEiWoRJe1nvgSJi/RDu2D4UpsyjPKslUNsUPQSoKM8LsEqAbIHQmKnH7L62PeYqd+nhbNzZ9aGw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709260249820578.5259940707089; Thu, 29 Feb 2024 18:30:49 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rfsee-000439-Eh; Thu, 29 Feb 2024 21:29:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rfsec-00042Z-JB for qemu-devel@nongnu.org; Thu, 29 Feb 2024 21:29:42 -0500 Received: from mail-qv1-xf29.google.com ([2607:f8b0:4864:20::f29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rfseb-0005Rr-5e for qemu-devel@nongnu.org; Thu, 29 Feb 2024 21:29:42 -0500 Received: by mail-qv1-xf29.google.com with SMTP id 6a1803df08f44-68f51c5f9baso8402866d6.3 for ; Thu, 29 Feb 2024 18:29:40 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.120]) by smtp.gmail.com with ESMTPSA id qo13-20020a056214590d00b0068d11cf887bsm1384107qvb.55.2024.02.29.18.29.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 18:29:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1709260180; x=1709864980; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yw8iSnc3LXROkWLue6u8GAvL3L94OebgZkCfuGQmnSk=; b=DBy2vArY8UWVValpDPzz5bceCW6sMEE35XFTinIZRzSu2Y+91XelzMmelAbBCA1sBn aIVagQRF1DPO1yvvsgKIjxuOTeylyPZh8hZrZUALNQVncVTxBb2D5u/mY7wSuwgbxWvW 0ZgUHxS4m034A+Nb+5xXUKRYmf5C0kHeciCtcR89XDbecGgL9o1cOda5MpURhZLAuFpe dG6Bo+O02EuaTen92fpY8yTVVRAZ7SEAiylnuobp08lZTsgRRSIkNJaONpHla4oE1kIv 8SXOJQD0IhE95B/ntk3c74/SxFqal2XAFtdY1lngHmoXHLan3hiI5J48pG9Ag8IsYdro zTqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709260180; x=1709864980; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yw8iSnc3LXROkWLue6u8GAvL3L94OebgZkCfuGQmnSk=; b=h2QgGwhSut1f9+rNw/jX7Tc48wMRDypWjYW99coVbRnThNLsFyb6TZzolQYEDaT9ci XBN/zS7IAYjYMzJwVoItSJBiA8+kzWItaVjv9Ec6z0TxsuZeSnH/7m5LCqdkKOgly1+/ uymwjLHboBjxWshwZ822i1yeNKg3CnC3yQ+WiLvIN6x3KWazPMo1r/tcs9CKYPFf9Jzx PUJ9tIIsFk0yaNBucKsCWDFPSHptBjAzeF2BEyU4JB0fNYyHCTRV/YUTaxRdWPoZNV0b M3+d6+xI122ZMJk7uIQmlLTIni0cAsFklL91ReJ2wlwFzjXxhHikik68NKCtG+7F5Zaz 2N4g== X-Forwarded-Encrypted: i=1; AJvYcCUsMW6ytYF20ST05pzMKr4QmruPDnVQ6uF2PM3zKePGgJVnBuEsY5wskyuvxJfJMpe301FisyIMOLBwu90eX/e5hRxDEN8= X-Gm-Message-State: AOJu0YzKYfT2FRb0GAKns5PnKt98jJ9cxrjcVpNtEKBdO6r4N7S71Rp2 U0kUh+36lRbKUptMTol6sAEmzc1ycHx2mMIlm8UJQFOX6lN4Ig0dqEp34oCiwEw= X-Google-Smtp-Source: AGHT+IEA4Isy3CFGW10rXyr/7totJiQ81C8HldgJQ77k15ev9qC3TRz1lBLy6l6CvGWQF+VUPemdrA== X-Received: by 2002:a05:6214:180d:b0:690:4e5e:4351 with SMTP id o13-20020a056214180d00b006904e5e4351mr418876qvw.41.1709260180123; Thu, 29 Feb 2024 18:29:40 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, jdenemar@redhat.com, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, qemu-devel@nongnu.org Cc: Hao Xiang Subject: [PATCH v4 7/7] Update maintainer contact for migration multifd zero page checking acceleration. Date: Fri, 1 Mar 2024 02:28:29 +0000 Message-Id: <20240301022829.3390548-8-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240301022829.3390548-1-hao.xiang@bytedance.com> References: <20240301022829.3390548-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::f29; envelope-from=hao.xiang@bytedance.com; helo=mail-qv1-xf29.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1709260251952100007 Content-Type: text/plain; charset="utf-8" Add myself to maintain multifd zero page checking acceleration function. Signed-off-by: Hao Xiang --- MAINTAINERS | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 65dfdc9677..b547918e4d 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3414,6 +3414,11 @@ F: tests/migration/ F: util/userfaultfd.c X: migration/rdma* =20 +Migration multifd zero page checking acceleration +M: Hao Xiang +S: Maintained +F: migration/multifd-zero-page.c + RDMA Migration R: Li Zhijian R: Peter Xu --=20 2.30.2