From nobody Fri Nov 1 05:35:13 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1708123283; cv=none; d=zohomail.com; s=zohoarc; b=XntB0S11Y6eoh050e5iKk5VUnIU9/oYBhJosxpXvsWypx9vteDonvFXYMYIMKD3tKzMSqqqKlQ+JEvXZXjDb6nz7aKZYjXimplcE6vz5YvyEARypiAwknmWPfTq+UMG0j0cHWUR7+3m4VF6MY9PQAvipzdvI2WoAP+0bY6vQQsM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708123283; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9zRUWTv7OrSzBzMEMxbe1tPscL1aF0KUoTlz915XvUk=; b=WbTEqbeu6oo7z4R0+/b2fGTHazma2VzD6Ou6pdgZyCIIX1BFNtlsdd6NyahLzhRLx9XpTHEDpmQwkeGr4ptFbwVC4P100SoEpDe8z0GlMj0mLtPw7iYt0zZd1twjyX0EssW8F1qxHgO2fbuDo4e86/zkazJW2CLKEKvX5H9EaVk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708123283160953.3813748507113; Fri, 16 Feb 2024 14:41:23 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rb6tE-0004kY-Cc; Fri, 16 Feb 2024 17:41:04 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rb6tC-0004kG-28 for qemu-devel@nongnu.org; Fri, 16 Feb 2024 17:41:02 -0500 Received: from mail-ua1-x936.google.com ([2607:f8b0:4864:20::936]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rb6t9-0007Bg-Cv for qemu-devel@nongnu.org; Fri, 16 Feb 2024 17:41:01 -0500 Received: by mail-ua1-x936.google.com with SMTP id a1e0cc1a2514c-7d5fce59261so1550472241.3 for ; Fri, 16 Feb 2024 14:40:57 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.104]) by smtp.gmail.com with ESMTPSA id cz18-20020a056122449200b004c0a12c4d53sm120991vkb.51.2024.02.16.14.40.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 14:40:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1708123256; x=1708728056; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9zRUWTv7OrSzBzMEMxbe1tPscL1aF0KUoTlz915XvUk=; b=T9RVn0RxPPAWU3v16hZiXZSknYxwxtwwaUsP63Py1TctOfMOraz4evAjksEda6AB4l CIKLJEtyoACN3qDV7MHNqiEUa/IUhE1DJ3YbY185OmYGSSyVFpIERdGfIq2/rCOlXi79 mPpCHSODb6CefV/599qkjEANCyTZCNRD2WsKOhfyPmaonsn6S5M2/Em4Ll7E0zPf7C7E i3xDy4JkXdl06Du4EcNYc46tiGUMfzA3sjgOeCfh1EzhVCW+2SIUjG/DhgfombZBVU76 luFTayRhybbJTIwW/Wal/A8IagLXaOKcU7AHMVr8P5VqZYePzVrVf3K8aGMOrNrrfaMo qzcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708123256; x=1708728056; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9zRUWTv7OrSzBzMEMxbe1tPscL1aF0KUoTlz915XvUk=; b=f9ixsEIHK8bH9Q1cC6dm55XSblUHWyivGvJh6daKcTS1lQfCaM6KMiHciUwmvWaz80 RejrJjIb+DHj5ceSwHGiR/QUupCr618804FrnyeaHqGyFZbHiRAazmxHLCPAL4AMykkx iunN+vDAJJSTDavJgkmn77Q2hjMz8lihgM/nftlLWhF7ezy5aufcVZjBwT3XMj0B1rIn TtZv01Ssaao8/gsh7X0kKwFaf1Zi+A5Lmn+8E6Skn5/5Xj4rbLy+2pJ1AwCiWjQhh5y8 zLGI7aGkc2c+lO2cMWbaVpvnbRs1ev9lHp6AQxjaAJoemnKqQYotr8EjzyFYreQuhLV0 ru6Q== X-Forwarded-Encrypted: i=1; AJvYcCVATrYE7auAqbRDIumKFumFYMnetbjWAwIt4L62bli3yx34ZXMzkx50xJX9HfUHHY73ehiXM+AEyxVQqnKTN0VZ/TAYiMc= X-Gm-Message-State: AOJu0Ywl7lEKkXXQoJ53M8kHJsr3Ue4iBqYaf1e6rzlcCFboj64gITR7 UGmDTGGtwuT4jbnIQlv/Z3wknErmzjSIlkIQrmlLWEBVofmdmO1tiNAKWrstcMA= X-Google-Smtp-Source: AGHT+IGg5wgNexcUUkCZKLq+ugDcfN/mLZo6MQCsdp+XczmaNH66qr5BUkkclttb9tZIbGxpS4Vhaw== X-Received: by 2002:a1f:ea44:0:b0:4c0:3bf0:29fd with SMTP id i65-20020a1fea44000000b004c03bf029fdmr6251159vkh.0.1708123256312; Fri, 16 Feb 2024 14:40:56 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, qemu-devel@nongnu.org, jdenemar@redhat.com Cc: Hao Xiang Subject: [PATCH v2 1/7] migration/multifd: Add new migration option zero-page-detection. Date: Fri, 16 Feb 2024 22:39:56 +0000 Message-Id: <20240216224002.1476890-2-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240216224002.1476890-1-hao.xiang@bytedance.com> References: <20240216224002.1476890-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::936; envelope-from=hao.xiang@bytedance.com; helo=mail-ua1-x936.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1708123284918100001 Content-Type: text/plain; charset="utf-8" This new parameter controls where the zero page checking is running. 1. If this parameter is set to 'legacy', zero page checking is done in the migration main thread. 2. If this parameter is set to 'none', zero page checking is disabled. Signed-off-by: Hao Xiang --- hw/core/qdev-properties-system.c | 10 ++++++++++ include/hw/qdev-properties-system.h | 4 ++++ migration/migration-hmp-cmds.c | 9 +++++++++ migration/options.c | 21 ++++++++++++++++++++ migration/options.h | 1 + migration/ram.c | 4 ++++ qapi/migration.json | 30 ++++++++++++++++++++++++++--- 7 files changed, 76 insertions(+), 3 deletions(-) diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-sys= tem.c index 1a396521d5..63843f18b5 100644 --- a/hw/core/qdev-properties-system.c +++ b/hw/core/qdev-properties-system.c @@ -679,6 +679,16 @@ const PropertyInfo qdev_prop_mig_mode =3D { .set_default_value =3D qdev_propinfo_set_default_value_enum, }; =20 +const PropertyInfo qdev_prop_zero_page_detection =3D { + .name =3D "ZeroPageDetection", + .description =3D "zero_page_detection values, " + "multifd,legacy,none", + .enum_table =3D &ZeroPageDetection_lookup, + .get =3D qdev_propinfo_get_enum, + .set =3D qdev_propinfo_set_enum, + .set_default_value =3D qdev_propinfo_set_default_value_enum, +}; + /* --- Reserved Region --- */ =20 /* diff --git a/include/hw/qdev-properties-system.h b/include/hw/qdev-properti= es-system.h index 06c359c190..839b170235 100644 --- a/include/hw/qdev-properties-system.h +++ b/include/hw/qdev-properties-system.h @@ -8,6 +8,7 @@ extern const PropertyInfo qdev_prop_macaddr; extern const PropertyInfo qdev_prop_reserved_region; extern const PropertyInfo qdev_prop_multifd_compression; extern const PropertyInfo qdev_prop_mig_mode; +extern const PropertyInfo qdev_prop_zero_page_detection; extern const PropertyInfo qdev_prop_losttickpolicy; extern const PropertyInfo qdev_prop_blockdev_on_error; extern const PropertyInfo qdev_prop_bios_chs_trans; @@ -47,6 +48,9 @@ extern const PropertyInfo qdev_prop_iothread_vq_mapping_l= ist; #define DEFINE_PROP_MIG_MODE(_n, _s, _f, _d) \ DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_mig_mode, \ MigMode) +#define DEFINE_PROP_ZERO_PAGE_DETECTION(_n, _s, _f, _d) \ + DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_zero_page_detection, \ + ZeroPageDetection) #define DEFINE_PROP_LOSTTICKPOLICY(_n, _s, _f, _d) \ DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_losttickpolicy, \ LostTickPolicy) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 99b49df5dd..7e96ae6ffd 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -344,6 +344,11 @@ void hmp_info_migrate_parameters(Monitor *mon, const Q= Dict *qdict) monitor_printf(mon, "%s: %s\n", MigrationParameter_str(MIGRATION_PARAMETER_MULTIFD_COMPRESSION= ), MultiFDCompression_str(params->multifd_compression)); + assert(params->has_zero_page_detection); + monitor_printf(mon, "%s: %s\n", + MigrationParameter_str(MIGRATION_PARAMETER_ZERO_PAGE_DETECTION= ), + qapi_enum_lookup(&ZeroPageDetection_lookup, + params->zero_page_detection)); monitor_printf(mon, "%s: %" PRIu64 " bytes\n", MigrationParameter_str(MIGRATION_PARAMETER_XBZRLE_CACHE_SIZE), params->xbzrle_cache_size); @@ -634,6 +639,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDi= ct *qdict) p->has_multifd_zstd_level =3D true; visit_type_uint8(v, param, &p->multifd_zstd_level, &err); break; + case MIGRATION_PARAMETER_ZERO_PAGE_DETECTION: + p->has_zero_page_detection =3D true; + visit_type_ZeroPageDetection(v, param, &p->zero_page_detection, &e= rr); + break; case MIGRATION_PARAMETER_XBZRLE_CACHE_SIZE: p->has_xbzrle_cache_size =3D true; if (!visit_type_size(v, param, &cache_size, &err)) { diff --git a/migration/options.c b/migration/options.c index 3e3e0b93b4..3c603391b0 100644 --- a/migration/options.c +++ b/migration/options.c @@ -179,6 +179,9 @@ Property migration_properties[] =3D { DEFINE_PROP_MIG_MODE("mode", MigrationState, parameters.mode, MIG_MODE_NORMAL), + DEFINE_PROP_ZERO_PAGE_DETECTION("zero-page-detection", MigrationState, + parameters.zero_page_detection, + ZERO_PAGE_DETECTION_LEGACY), =20 /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), @@ -903,6 +906,13 @@ uint64_t migrate_xbzrle_cache_size(void) return s->parameters.xbzrle_cache_size; } =20 +ZeroPageDetection migrate_zero_page_detection(void) +{ + MigrationState *s =3D migrate_get_current(); + + return s->parameters.zero_page_detection; +} + /* parameter setters */ =20 void migrate_set_block_incremental(bool value) @@ -1013,6 +1023,8 @@ MigrationParameters *qmp_query_migrate_parameters(Err= or **errp) params->vcpu_dirty_limit =3D s->parameters.vcpu_dirty_limit; params->has_mode =3D true; params->mode =3D s->parameters.mode; + params->has_zero_page_detection =3D true; + params->zero_page_detection =3D s->parameters.zero_page_detection; =20 return params; } @@ -1049,6 +1061,7 @@ void migrate_params_init(MigrationParameters *params) params->has_x_vcpu_dirty_limit_period =3D true; params->has_vcpu_dirty_limit =3D true; params->has_mode =3D true; + params->has_zero_page_detection =3D true; } =20 /* @@ -1350,6 +1363,10 @@ static void migrate_params_test_apply(MigrateSetPara= meters *params, if (params->has_mode) { dest->mode =3D params->mode; } + + if (params->has_zero_page_detection) { + dest->zero_page_detection =3D params->zero_page_detection; + } } =20 static void migrate_params_apply(MigrateSetParameters *params, Error **err= p) @@ -1494,6 +1511,10 @@ static void migrate_params_apply(MigrateSetParameter= s *params, Error **errp) if (params->has_mode) { s->parameters.mode =3D params->mode; } + + if (params->has_zero_page_detection) { + s->parameters.zero_page_detection =3D params->zero_page_detection; + } } =20 void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) diff --git a/migration/options.h b/migration/options.h index 246c160aee..b7c4fb3861 100644 --- a/migration/options.h +++ b/migration/options.h @@ -93,6 +93,7 @@ const char *migrate_tls_authz(void); const char *migrate_tls_creds(void); const char *migrate_tls_hostname(void); uint64_t migrate_xbzrle_cache_size(void); +ZeroPageDetection migrate_zero_page_detection(void); =20 /* parameters setters */ =20 diff --git a/migration/ram.c b/migration/ram.c index 4649a81204..556725c30f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1123,6 +1123,10 @@ static int save_zero_page(RAMState *rs, PageSearchSt= atus *pss, QEMUFile *file =3D pss->pss_channel; int len =3D 0; =20 + if (migrate_zero_page_detection() !=3D ZERO_PAGE_DETECTION_LEGACY) { + return 0; + } + if (!buffer_is_zero(p, TARGET_PAGE_SIZE)) { return 0; } diff --git a/qapi/migration.json b/qapi/migration.json index 5a565d9b8d..99843a8e95 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -653,6 +653,17 @@ { 'enum': 'MigMode', 'data': [ 'normal', 'cpr-reboot' ] } =20 +## +# @ZeroPageDetection: +# +# @legacy: Perform zero page checking from main migration thread. (since 9= .0) +# +# @none: Do not perform zero page checking. +# +## +{ 'enum': 'ZeroPageDetection', + 'data': [ 'legacy', 'none' ] } + ## # @BitmapMigrationBitmapAliasTransform: # @@ -874,6 +885,9 @@ # @mode: Migration mode. See description in @MigMode. Default is 'normal'. # (Since 8.2) # +# @zero-page-detection: See description in @ZeroPageDetection. +# Default is 'legacy'. (Since 9.0) +# # Features: # # @deprecated: Member @block-incremental is deprecated. Use @@ -907,7 +921,8 @@ 'block-bitmap-mapping', { 'name': 'x-vcpu-dirty-limit-period', 'features': ['unstable']= }, 'vcpu-dirty-limit', - 'mode'] } + 'mode', + 'zero-page-detection'] } =20 ## # @MigrateSetParameters: @@ -1066,6 +1081,10 @@ # @mode: Migration mode. See description in @MigMode. Default is 'normal'. # (Since 8.2) # +# @zero-page-detection: See description in @ZeroPageDetection. +# Default is 'legacy'. (Since 9.0) +# +# # Features: # # @deprecated: Member @block-incremental is deprecated. Use @@ -1119,7 +1138,8 @@ '*x-vcpu-dirty-limit-period': { 'type': 'uint64', 'features': [ 'unstable' ] }, '*vcpu-dirty-limit': 'uint64', - '*mode': 'MigMode'} } + '*mode': 'MigMode', + '*zero-page-detection': 'ZeroPageDetection'} } =20 ## # @migrate-set-parameters: @@ -1294,6 +1314,9 @@ # @mode: Migration mode. See description in @MigMode. Default is 'normal'. # (Since 8.2) # +# @zero-page-detection: See description in @ZeroPageDetection. +# Default is 'legacy'. (Since 9.0) +# # Features: # # @deprecated: Member @block-incremental is deprecated. Use @@ -1344,7 +1367,8 @@ '*x-vcpu-dirty-limit-period': { 'type': 'uint64', 'features': [ 'unstable' ] }, '*vcpu-dirty-limit': 'uint64', - '*mode': 'MigMode'} } + '*mode': 'MigMode', + '*zero-page-detection': 'ZeroPageDetection'} } =20 ## # @query-migrate-parameters: --=20 2.30.2 From nobody Fri Nov 1 05:35:13 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1708123313; cv=none; d=zohomail.com; s=zohoarc; b=JsLgyoU6/Ytgv5dq97NAUM6VZ/WAFFhHeDOMw0y4oMnCgpNSx50mFb8dT/ga9AUlWuQxTM0STh7rCTy05+v4jkQRWyRO8ZFYcOgW7SIXUvBBqxL8aV412kKY41r6vUhIEo0rs3DtPjbBA+GIVYsNPKUmfSH8+jNVHelF4RxqRC4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708123313; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=MDa8kynJoLoRNVNC8BFkudqqHcHPgvw0b+BqMDcdRzY=; b=gbAVZsfgWbzfY3MIj179NRoTACU4zqCEFSjvpMszJs2mfXPyKMKflA1Z5IqyG9/sKd1IilYLz48WJvczifxDDPy1EENR+VRSoi/vlnHjLrxSK/31kb5jnYKuOUjGxLLTnFJj2Mpp7QbPTJ6kRewlr5E/ADfbFwh99yaDcvsdPUg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708123313219435.2995419584546; Fri, 16 Feb 2024 14:41:53 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rb6tK-0004uL-LL; Fri, 16 Feb 2024 17:41:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rb6tJ-0004tq-Ca for qemu-devel@nongnu.org; Fri, 16 Feb 2024 17:41:09 -0500 Received: from mail-vk1-xa2f.google.com ([2607:f8b0:4864:20::a2f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rb6tH-0007HE-85 for qemu-devel@nongnu.org; Fri, 16 Feb 2024 17:41:09 -0500 Received: by mail-vk1-xa2f.google.com with SMTP id 71dfb90a1353d-4c76b0962a2so205914e0c.1 for ; Fri, 16 Feb 2024 14:41:05 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.104]) by smtp.gmail.com with ESMTPSA id cz18-20020a056122449200b004c0a12c4d53sm120991vkb.51.2024.02.16.14.41.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 14:41:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1708123265; x=1708728065; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MDa8kynJoLoRNVNC8BFkudqqHcHPgvw0b+BqMDcdRzY=; b=Cmu38GUqOZEqCrVu0VEfJLLSglQIq5c3qBwoPNs4hOW2VSnKbsWTPPUV5w79jml8Cy LB+MskrGQ/KDEw6gMTXjOjsrSz+5FTZEO37qgEJGi75/Eww01UU31HrQyLiRGFTOk4e6 P0zFq1OMlc4jEvbUnjI/tVc28AHH3fWU60XwlUTBTiM1hRBtxVcA3Harso/eGMN3/KtU iAMSKB08JB7lrBQcJ9IzHze9Zmq2ZHmf5l2tqKyNXtWdvDuY6Du1UzraqAr+SqtonZIS g8xCvqbEP11dzg2WiBXFTzU1lkJt4M0cyNCPWUzrvB4/zfd/2Lorr/2D9jQK8E8zxk4A eovw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708123265; x=1708728065; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MDa8kynJoLoRNVNC8BFkudqqHcHPgvw0b+BqMDcdRzY=; b=T0ymr9hJnKeKNxW718tBd+S67dVPhqBDo23gOEdOY/1rUzEg2ahe4kbDCSyXn7LBqX Lgg2cRUSCj+WCKZLzBKilulxROu5XukfLYXQF8pHkJ527tksckPdkFhzXhhPhMAxKvrj /4MlSUU43j9PoDp/UffeUMYQLBP14Q2qEOjRmMr/ZagWU2IE27eixdcQpIVwv7vpNquA bxbOnMo9ZXfvtnsiCMK2XlY5uJRwqDJ6i4GVDZ6IqIb3V5Pw6ozlZalbcefqnB/uJeyT CreRBqcqA0MvhJFxYDo7e5ut2oQtsZMJYiOq9KFpr4oz1zYiR5sl7x01KS46pbLM2zwx c61g== X-Forwarded-Encrypted: i=1; AJvYcCWLRozhinldbLEjZEAQtBritjAcSNdASor9ghrHGeVgdpheheUo0tr8xHyX/Lu8XWCnC+ePnesI8NtU//tRFnb5sT+3CQQ= X-Gm-Message-State: AOJu0YxO+UJg0wyE8hVdQcMejyUe23mRqSNVoxJL2lAinHu9sG4vWc1X hYUrD/+d8fGSweVSiuq2HlKVkX3DSt9d/zgI71IBMu/CO38+uQKJn7R9ShyqrDA= X-Google-Smtp-Source: AGHT+IFtaoCia9TfxFEajv1LzQ17cfr4KrBmC81CTioC1tDcZR+61LgqGCY1CvlHpMAAaatuwM26YA== X-Received: by 2002:a1f:6dc3:0:b0:4bd:7a0f:960 with SMTP id i186-20020a1f6dc3000000b004bd7a0f0960mr5752799vkc.5.1708123265176; Fri, 16 Feb 2024 14:41:05 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, qemu-devel@nongnu.org, jdenemar@redhat.com Cc: Hao Xiang Subject: [PATCH v2 2/7] migration/multifd: Support for zero pages transmission in multifd format. Date: Fri, 16 Feb 2024 22:39:57 +0000 Message-Id: <20240216224002.1476890-3-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240216224002.1476890-1-hao.xiang@bytedance.com> References: <20240216224002.1476890-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::a2f; envelope-from=hao.xiang@bytedance.com; helo=mail-vk1-xa2f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1708123314763100002 Content-Type: text/plain; charset="utf-8" This change adds zero page counters and updates multifd send/receive tracing format to track the newly added counters. Signed-off-by: Hao Xiang --- migration/multifd.c | 43 ++++++++++++++++++++++++++++++++++-------- migration/multifd.h | 21 ++++++++++++++++++++- migration/ram.c | 1 - migration/trace-events | 8 ++++---- 4 files changed, 59 insertions(+), 14 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index adfe8c9a0a..a33dba40d9 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -236,6 +236,8 @@ static void multifd_pages_reset(MultiFDPages_t *pages) * overwritten later when reused. */ pages->num =3D 0; + pages->normal_num =3D 0; + pages->zero_num =3D 0; pages->block =3D NULL; } =20 @@ -309,6 +311,8 @@ static MultiFDPages_t *multifd_pages_init(uint32_t n) =20 pages->allocated =3D n; pages->offset =3D g_new0(ram_addr_t, n); + pages->normal =3D g_new0(ram_addr_t, n); + pages->zero =3D g_new0(ram_addr_t, n); =20 return pages; } @@ -319,6 +323,10 @@ static void multifd_pages_clear(MultiFDPages_t *pages) pages->allocated =3D 0; g_free(pages->offset); pages->offset =3D NULL; + g_free(pages->normal); + pages->normal =3D NULL; + g_free(pages->zero); + pages->zero =3D NULL; g_free(pages); } =20 @@ -332,6 +340,7 @@ void multifd_send_fill_packet(MultiFDSendParams *p) packet->flags =3D cpu_to_be32(p->flags); packet->pages_alloc =3D cpu_to_be32(p->pages->allocated); packet->normal_pages =3D cpu_to_be32(pages->num); + packet->zero_pages =3D cpu_to_be32(pages->zero_num); packet->next_packet_size =3D cpu_to_be32(p->next_packet_size); =20 packet_num =3D qatomic_fetch_inc(&multifd_send_state->packet_num); @@ -350,9 +359,10 @@ void multifd_send_fill_packet(MultiFDSendParams *p) =20 p->packets_sent++; p->total_normal_pages +=3D pages->num; + p->total_zero_pages +=3D pages->zero_num; =20 - trace_multifd_send(p->id, packet_num, pages->num, p->flags, - p->next_packet_size); + trace_multifd_send(p->id, packet_num, pages->num, pages->zero_num, + p->flags, p->next_packet_size); } =20 static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) @@ -393,20 +403,29 @@ static int multifd_recv_unfill_packet(MultiFDRecvPara= ms *p, Error **errp) p->normal_num =3D be32_to_cpu(packet->normal_pages); if (p->normal_num > packet->pages_alloc) { error_setg(errp, "multifd: received packet " - "with %u pages and expected maximum pages are %u", + "with %u normal pages and expected maximum pages are %u= ", p->normal_num, packet->pages_alloc) ; return -1; } =20 + p->zero_num =3D be32_to_cpu(packet->zero_pages); + if (p->zero_num > packet->pages_alloc - p->normal_num) { + error_setg(errp, "multifd: received packet " + "with %u zero pages and expected maximum zero pages are= %u", + p->zero_num, packet->pages_alloc - p->normal_num) ; + return -1; + } + p->next_packet_size =3D be32_to_cpu(packet->next_packet_size); p->packet_num =3D be64_to_cpu(packet->packet_num); p->packets_recved++; p->total_normal_pages +=3D p->normal_num; + p->total_zero_pages +=3D p->zero_num; =20 - trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->flags, - p->next_packet_size); + trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->zero_num, + p->flags, p->next_packet_size); =20 - if (p->normal_num =3D=3D 0) { + if (p->normal_num =3D=3D 0 && p->zero_num =3D=3D 0) { return 0; } =20 @@ -823,6 +842,8 @@ static void *multifd_send_thread(void *opaque) =20 stat64_add(&mig_stats.multifd_bytes, p->next_packet_size + p->packet_len); + stat64_add(&mig_stats.normal_pages, pages->num); + stat64_add(&mig_stats.zero_pages, pages->zero_num); =20 multifd_pages_reset(p->pages); p->next_packet_size =3D 0; @@ -866,7 +887,8 @@ out: =20 rcu_unregister_thread(); migration_threads_remove(thread); - trace_multifd_send_thread_end(p->id, p->packets_sent, p->total_normal_= pages); + trace_multifd_send_thread_end(p->id, p->packets_sent, p->total_normal_= pages, + p->total_zero_pages); =20 return NULL; } @@ -1132,6 +1154,8 @@ static void multifd_recv_cleanup_channel(MultiFDRecvP= arams *p) p->iov =3D NULL; g_free(p->normal); p->normal =3D NULL; + g_free(p->zero); + p->zero =3D NULL; multifd_recv_state->ops->recv_cleanup(p); } =20 @@ -1251,7 +1275,9 @@ static void *multifd_recv_thread(void *opaque) } =20 rcu_unregister_thread(); - trace_multifd_recv_thread_end(p->id, p->packets_recved, p->total_norma= l_pages); + trace_multifd_recv_thread_end(p->id, p->packets_recved, + p->total_normal_pages, + p->total_zero_pages); =20 return NULL; } @@ -1290,6 +1316,7 @@ int multifd_recv_setup(Error **errp) p->name =3D g_strdup_printf("multifdrecv_%d", i); p->iov =3D g_new0(struct iovec, page_count); p->normal =3D g_new0(ram_addr_t, page_count); + p->zero =3D g_new0(ram_addr_t, page_count); p->page_count =3D page_count; p->page_size =3D qemu_target_page_size(); } diff --git a/migration/multifd.h b/migration/multifd.h index 8a1cad0996..9822ff298a 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -48,7 +48,10 @@ typedef struct { /* size of the next packet that contains pages */ uint32_t next_packet_size; uint64_t packet_num; - uint64_t unused[4]; /* Reserved for future use */ + /* zero pages */ + uint32_t zero_pages; + uint32_t unused32[1]; /* Reserved for future use */ + uint64_t unused64[3]; /* Reserved for future use */ char ramblock[256]; uint64_t offset[]; } __attribute__((packed)) MultiFDPacket_t; @@ -56,10 +59,18 @@ typedef struct { typedef struct { /* number of used pages */ uint32_t num; + /* number of normal pages */ + uint32_t normal_num; + /* number of zero pages */ + uint32_t zero_num; /* number of allocated pages */ uint32_t allocated; /* offset of each page */ ram_addr_t *offset; + /* offset of normal page */ + ram_addr_t *normal; + /* offset of zero page */ + ram_addr_t *zero; RAMBlock *block; } MultiFDPages_t; =20 @@ -124,6 +135,8 @@ typedef struct { uint64_t packets_sent; /* non zero pages sent through this channel */ uint64_t total_normal_pages; + /* zero pages sent through this channel */ + uint64_t total_zero_pages; /* buffers to send */ struct iovec *iov; /* number of iovs used */ @@ -178,12 +191,18 @@ typedef struct { uint8_t *host; /* non zero pages recv through this channel */ uint64_t total_normal_pages; + /* zero pages recv through this channel */ + uint64_t total_zero_pages; /* buffers to recv */ struct iovec *iov; /* Pages that are not zero */ ram_addr_t *normal; /* num of non zero pages */ uint32_t normal_num; + /* Pages that are zero */ + ram_addr_t *zero; + /* num of zero pages */ + uint32_t zero_num; /* used for de-compression methods */ void *data; } MultiFDRecvParams; diff --git a/migration/ram.c b/migration/ram.c index 556725c30f..5ece9f042e 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1259,7 +1259,6 @@ static int ram_save_multifd_page(RAMBlock *block, ram= _addr_t offset) if (!multifd_queue_page(block, offset)) { return -1; } - stat64_add(&mig_stats.normal_pages, 1); =20 return 1; } diff --git a/migration/trace-events b/migration/trace-events index 298ad2b0dd..9f1d7ae71a 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -128,21 +128,21 @@ postcopy_preempt_reset_channel(void) "" # multifd.c multifd_new_send_channel_async(uint8_t id) "channel %u" multifd_new_send_channel_async_error(uint8_t id, void *err) "channel=3D%u = err=3D%p" -multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flag= s, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " pages %u f= lags 0x%x next packet size %u" +multifd_recv(uint8_t id, uint64_t packet_num, uint32_t normal, uint32_t ze= ro, uint32_t flags, uint32_t next_packet_size) "channel %u packet_num %" PR= Iu64 " normal pages %u zero pages %u flags 0x%x next packet size %u" multifd_recv_new_channel(uint8_t id) "channel %u" multifd_recv_sync_main(long packet_num) "packet num %ld" multifd_recv_sync_main_signal(uint8_t id) "channel %u" multifd_recv_sync_main_wait(uint8_t id) "channel %u" multifd_recv_terminate_threads(bool error) "error %d" -multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "cha= nnel %u packets %" PRIu64 " pages %" PRIu64 +multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal pages %" PRI= u64 " zero pages %" PRIu64 multifd_recv_thread_start(uint8_t id) "%u" -multifd_send(uint8_t id, uint64_t packet_num, uint32_t normal, uint32_t fl= ags, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " normal p= ages %u flags 0x%x next packet size %u" +multifd_send(uint8_t id, uint64_t packet_num, uint32_t normal_pages, uint3= 2_t zero_pages, uint32_t flags, uint32_t next_packet_size) "channel %u pack= et_num %" PRIu64 " normal pages %u zero pages %u flags 0x%x next packet siz= e %u" multifd_send_error(uint8_t id) "channel %u" multifd_send_sync_main(long packet_num) "packet num %ld" multifd_send_sync_main_signal(uint8_t id) "channel %u" multifd_send_sync_main_wait(uint8_t id) "channel %u" multifd_send_terminate_threads(void) "" -multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s) "channel %u packets %" PRIu64 " normal pages %" PRIu64 +multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal pages %" PR= Iu64 " zero pages %" PRIu64 multifd_send_thread_start(uint8_t id) "%u" multifd_tls_outgoing_handshake_start(void *ioc, void *tioc, const char *ho= stname) "ioc=3D%p tioc=3D%p hostname=3D%s" multifd_tls_outgoing_handshake_error(void *ioc, const char *err) "ioc=3D%p= err=3D%s" --=20 2.30.2 From nobody Fri Nov 1 05:35:13 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1708123316; cv=none; d=zohomail.com; s=zohoarc; b=A7ycBET2CkgTzpSV/o7y9JWXTlhrarDOFbUa4D/gN5Y5WgLwPRtcWGPMg21Urtb5b3WVBwlRN/mutt8C3vgUHwgUdkbY6hVdDRzkr52r6OYXOAFqog4Z3b73MjbP1n8z27ae8kIYUK3JCItzS+NrMzkOvQT9iZIvCVeWJ0GwwPg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708123316; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=v3VZ+i6lWDPBJl2XFGqb5oFW0n4zsCAj1svqfubVQtg=; b=TeRmMrNVC9mVKYAI5PEyKJDHBhCaJFe3kjHm1CQtcodzv06aEajBvaFwzZ9u/xJ0WZPWWZcqYdVxM7kPV417m9aGzhT1x+UzeThwKNzFpHjPJHXn8sOtCVas1u377MqRE36/BRHDkNFUlZpd30zQP8ZvWudMFO+O86ppbP6l0/U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708123316174838.438774127727; Fri, 16 Feb 2024 14:41:56 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rb6tO-0004vz-4J; Fri, 16 Feb 2024 17:41:14 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rb6tM-0004v2-8S for qemu-devel@nongnu.org; Fri, 16 Feb 2024 17:41:12 -0500 Received: from mail-vk1-xa2c.google.com ([2607:f8b0:4864:20::a2c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rb6tI-0007I9-UY for qemu-devel@nongnu.org; Fri, 16 Feb 2024 17:41:11 -0500 Received: by mail-vk1-xa2c.google.com with SMTP id 71dfb90a1353d-4c0245cba99so664660e0c.0 for ; Fri, 16 Feb 2024 14:41:08 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.104]) by smtp.gmail.com with ESMTPSA id cz18-20020a056122449200b004c0a12c4d53sm120991vkb.51.2024.02.16.14.41.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 14:41:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1708123268; x=1708728068; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=v3VZ+i6lWDPBJl2XFGqb5oFW0n4zsCAj1svqfubVQtg=; b=MtaFq7Fb9velGcm+mNKmmwcPQDnQtYyIACeC0rP+k2/y8cbLzFqssa80p/PX3/Rc0z Degq+ZlxOrcSIpGajlVXsYIBKPpPyChJ6+94Gv3FpTFGgIzCDRzf50T//gJ5Hl6P7GJw jZBiMIYsmrkAeKgnq9b7uiF/mW3YIyt/TU20aWL/pwOAqc9z0HYd6UUXxIyU9P9Nodly sxyMNZTFe3oJZWw2cJtl4EhG025j/aA4rHHybgcBsB/oYzRGQnQyeEkH4eMBGEDwR4IO frKLM4wKp/tIhZSZti2Fa4OXrb3sSd7vdLIHovJOLKYDhcIN2mdgfl3Maj21/fBPpijK IYLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708123268; x=1708728068; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v3VZ+i6lWDPBJl2XFGqb5oFW0n4zsCAj1svqfubVQtg=; b=Eir/0cvpGswe7QqKa7OH9B5k/w0gjjy0fo2rl5VXZcmmZ+KwQWFnoBgak7GhnmlUoH f49LN/O6xSVSiU6QN3Lzj58eh0M7BO+zMX1dK94oEqgLrnmQB8VlH6E/fhb8S6KrPSc6 9MjTn6hn+vmv21hpq2TYn+tzU/0lARl8ZsawMwfGTr0YBPckiW7ge7sONwfMfaItOUXZ fsNwGLNmDsTtnLFrtb8Ab17wxDq/sQrtQZ1AejXSedYOclzZkeYlt1/obuKp2PcRRmzf TxZmVS/3tvY0eV8jOFhkVwV7U5aI+EaZblKK8ZnrdRMSZUsOLJaZ5p9vDXqw47ObRbAh DozQ== X-Forwarded-Encrypted: i=1; AJvYcCV+n0cH0a1AvrGJbwAyZQDqpU0C1ClsIBbtWAnqeRkyl66eD7CL1myYRIWzRT5wWlVA9oOWhBRHgh7PhYM2TBlNsVzZ+GY= X-Gm-Message-State: AOJu0Yy8/OwuBEVhNa62/YGap2Dh20TW4RVILB6zDg309MWCOUCF2l1S broj+saytOAqG3RQIbDXT0HV0QXHsrok80eMKyMot70CFv1yb42bKYYZ7/TjCzI= X-Google-Smtp-Source: AGHT+IFSbay2OENf55AzM/kqmPauD8uxw2NBdT4Eqt+99bj0GNRyrLfGbsx38ajA8HD+YXVh5Y4oAA== X-Received: by 2002:a1f:62c3:0:b0:4c0:3621:7ab0 with SMTP id w186-20020a1f62c3000000b004c036217ab0mr5632423vkb.15.1708123267284; Fri, 16 Feb 2024 14:41:07 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, qemu-devel@nongnu.org, jdenemar@redhat.com Cc: Hao Xiang Subject: [PATCH v2 3/7] migration/multifd: Zero page transmission on the multifd thread. Date: Fri, 16 Feb 2024 22:39:58 +0000 Message-Id: <20240216224002.1476890-4-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240216224002.1476890-1-hao.xiang@bytedance.com> References: <20240216224002.1476890-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::a2c; envelope-from=hao.xiang@bytedance.com; helo=mail-vk1-xa2c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1708123316757100006 Content-Type: text/plain; charset="utf-8" 1. Implements the zero page detection and handling on the multifd threads for non-compression, zlib and zstd compression backends. 2. Added a new value 'multifd' in ZeroPageDetection enumeration. 3. Add proper asserts to ensure pages->normal are used for normal pages in all scenarios. Signed-off-by: Hao Xiang --- migration/meson.build | 1 + migration/multifd-zero-page.c | 59 +++++++++++++++++++++++++++++++++++ migration/multifd-zlib.c | 26 ++++++++++++--- migration/multifd-zstd.c | 25 ++++++++++++--- migration/multifd.c | 50 +++++++++++++++++++++++------ migration/multifd.h | 7 +++++ qapi/migration.json | 4 ++- 7 files changed, 151 insertions(+), 21 deletions(-) create mode 100644 migration/multifd-zero-page.c diff --git a/migration/meson.build b/migration/meson.build index 92b1cc4297..1eeb915ff6 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -22,6 +22,7 @@ system_ss.add(files( 'migration.c', 'multifd.c', 'multifd-zlib.c', + 'multifd-zero-page.c', 'ram-compress.c', 'options.c', 'postcopy-ram.c', diff --git a/migration/multifd-zero-page.c b/migration/multifd-zero-page.c new file mode 100644 index 0000000000..f0cd8e2c53 --- /dev/null +++ b/migration/multifd-zero-page.c @@ -0,0 +1,59 @@ +/* + * Multifd zero page detection implementation. + * + * Copyright (c) 2024 Bytedance Inc + * + * Authors: + * Hao Xiang + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qemu/cutils.h" +#include "exec/ramblock.h" +#include "migration.h" +#include "multifd.h" +#include "options.h" +#include "ram.h" + +void multifd_zero_page_check_send(MultiFDSendParams *p) +{ + /* + * QEMU older than 9.0 don't understand zero page + * on multifd channel. This switch is required to + * maintain backward compatibility. + */ + bool use_multifd_zero_page =3D + (migrate_zero_page_detection() =3D=3D ZERO_PAGE_DETECTION_MULTIFD); + MultiFDPages_t *pages =3D p->pages; + RAMBlock *rb =3D pages->block; + + assert(pages->num !=3D 0); + assert(pages->normal_num =3D=3D 0); + assert(pages->zero_num =3D=3D 0); + + for (int i =3D 0; i < pages->num; i++) { + uint64_t offset =3D pages->offset[i]; + if (use_multifd_zero_page && + buffer_is_zero(rb->host + offset, p->page_size)) { + pages->zero[pages->zero_num] =3D offset; + pages->zero_num++; + ram_release_page(rb->idstr, offset); + } else { + pages->normal[pages->normal_num] =3D offset; + pages->normal_num++; + } + } +} + +void multifd_zero_page_check_recv(MultiFDRecvParams *p) +{ + for (int i =3D 0; i < p->zero_num; i++) { + void *page =3D p->host + p->zero[i]; + if (!buffer_is_zero(page, p->page_size)) { + memset(page, 0, p->page_size); + } + } +} diff --git a/migration/multifd-zlib.c b/migration/multifd-zlib.c index 012e3bdea1..cdfe0fa70e 100644 --- a/migration/multifd-zlib.c +++ b/migration/multifd-zlib.c @@ -123,13 +123,20 @@ static int zlib_send_prepare(MultiFDSendParams *p, Er= ror **errp) int ret; uint32_t i; =20 + multifd_zero_page_check_send(p); + + if (!pages->normal_num) { + p->next_packet_size =3D 0; + goto out; + } + multifd_send_prepare_header(p); =20 - for (i =3D 0; i < pages->num; i++) { + for (i =3D 0; i < pages->normal_num; i++) { uint32_t available =3D z->zbuff_len - out_size; int flush =3D Z_NO_FLUSH; =20 - if (i =3D=3D pages->num - 1) { + if (i =3D=3D pages->normal_num - 1) { flush =3D Z_SYNC_FLUSH; } =20 @@ -138,7 +145,7 @@ static int zlib_send_prepare(MultiFDSendParams *p, Erro= r **errp) * with compression. zlib does not guarantee that this is safe, * therefore copy the page before calling deflate(). */ - memcpy(z->buf, p->pages->block->host + pages->offset[i], p->page_s= ize); + memcpy(z->buf, p->pages->block->host + pages->normal[i], p->page_s= ize); zs->avail_in =3D p->page_size; zs->next_in =3D z->buf; =20 @@ -172,10 +179,10 @@ static int zlib_send_prepare(MultiFDSendParams *p, Er= ror **errp) p->iov[p->iovs_num].iov_len =3D out_size; p->iovs_num++; p->next_packet_size =3D out_size; - p->flags |=3D MULTIFD_FLAG_ZLIB; =20 +out: + p->flags |=3D MULTIFD_FLAG_ZLIB; multifd_send_fill_packet(p); - return 0; } =20 @@ -261,6 +268,14 @@ static int zlib_recv_pages(MultiFDRecvParams *p, Error= **errp) p->id, flags, MULTIFD_FLAG_ZLIB); return -1; } + + multifd_zero_page_check_recv(p); + + if (!p->normal_num) { + assert(in_size =3D=3D 0); + return 0; + } + ret =3D qio_channel_read_all(p->c, (void *)z->zbuff, in_size, errp); =20 if (ret !=3D 0) { @@ -310,6 +325,7 @@ static int zlib_recv_pages(MultiFDRecvParams *p, Error = **errp) p->id, out_size, expected_size); return -1; } + return 0; } =20 diff --git a/migration/multifd-zstd.c b/migration/multifd-zstd.c index dc8fe43e94..27a1eba075 100644 --- a/migration/multifd-zstd.c +++ b/migration/multifd-zstd.c @@ -118,19 +118,26 @@ static int zstd_send_prepare(MultiFDSendParams *p, Er= ror **errp) int ret; uint32_t i; =20 + multifd_zero_page_check_send(p); + + if (!pages->normal_num) { + p->next_packet_size =3D 0; + goto out; + } + multifd_send_prepare_header(p); =20 z->out.dst =3D z->zbuff; z->out.size =3D z->zbuff_len; z->out.pos =3D 0; =20 - for (i =3D 0; i < pages->num; i++) { + for (i =3D 0; i < pages->normal_num; i++) { ZSTD_EndDirective flush =3D ZSTD_e_continue; =20 - if (i =3D=3D pages->num - 1) { + if (i =3D=3D pages->normal_num - 1) { flush =3D ZSTD_e_flush; } - z->in.src =3D p->pages->block->host + pages->offset[i]; + z->in.src =3D p->pages->block->host + pages->normal[i]; z->in.size =3D p->page_size; z->in.pos =3D 0; =20 @@ -161,10 +168,10 @@ static int zstd_send_prepare(MultiFDSendParams *p, Er= ror **errp) p->iov[p->iovs_num].iov_len =3D z->out.pos; p->iovs_num++; p->next_packet_size =3D z->out.pos; - p->flags |=3D MULTIFD_FLAG_ZSTD; =20 +out: + p->flags |=3D MULTIFD_FLAG_ZSTD; multifd_send_fill_packet(p); - return 0; } =20 @@ -257,6 +264,14 @@ static int zstd_recv_pages(MultiFDRecvParams *p, Error= **errp) p->id, flags, MULTIFD_FLAG_ZSTD); return -1; } + + multifd_zero_page_check_recv(p); + + if (!p->normal_num) { + assert(in_size =3D=3D 0); + return 0; + } + ret =3D qio_channel_read_all(p->c, (void *)z->zbuff, in_size, errp); =20 if (ret !=3D 0) { diff --git a/migration/multifd.c b/migration/multifd.c index a33dba40d9..fbb40ea10b 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -11,6 +11,7 @@ */ =20 #include "qemu/osdep.h" +#include "qemu/cutils.h" #include "qemu/rcu.h" #include "exec/target_page.h" #include "sysemu/sysemu.h" @@ -126,6 +127,8 @@ static int nocomp_send_prepare(MultiFDSendParams *p, Er= ror **errp) MultiFDPages_t *pages =3D p->pages; int ret; =20 + multifd_zero_page_check_send(p); + if (!use_zero_copy_send) { /* * Only !zerocopy needs the header in IOV; zerocopy will @@ -134,13 +137,13 @@ static int nocomp_send_prepare(MultiFDSendParams *p, = Error **errp) multifd_send_prepare_header(p); } =20 - for (int i =3D 0; i < pages->num; i++) { - p->iov[p->iovs_num].iov_base =3D pages->block->host + pages->offse= t[i]; + for (int i =3D 0; i < pages->normal_num; i++) { + p->iov[p->iovs_num].iov_base =3D pages->block->host + pages->norma= l[i]; p->iov[p->iovs_num].iov_len =3D p->page_size; p->iovs_num++; } =20 - p->next_packet_size =3D pages->num * p->page_size; + p->next_packet_size =3D pages->normal_num * p->page_size; p->flags |=3D MULTIFD_FLAG_NOCOMP; =20 multifd_send_fill_packet(p); @@ -202,6 +205,13 @@ static int nocomp_recv_pages(MultiFDRecvParams *p, Err= or **errp) p->id, flags, MULTIFD_FLAG_NOCOMP); return -1; } + + multifd_zero_page_check_recv(p); + + if (!p->normal_num) { + return 0; + } + for (int i =3D 0; i < p->normal_num; i++) { p->iov[i].iov_base =3D p->host + p->normal[i]; p->iov[i].iov_len =3D p->page_size; @@ -339,7 +349,7 @@ void multifd_send_fill_packet(MultiFDSendParams *p) =20 packet->flags =3D cpu_to_be32(p->flags); packet->pages_alloc =3D cpu_to_be32(p->pages->allocated); - packet->normal_pages =3D cpu_to_be32(pages->num); + packet->normal_pages =3D cpu_to_be32(pages->normal_num); packet->zero_pages =3D cpu_to_be32(pages->zero_num); packet->next_packet_size =3D cpu_to_be32(p->next_packet_size); =20 @@ -350,18 +360,25 @@ void multifd_send_fill_packet(MultiFDSendParams *p) strncpy(packet->ramblock, pages->block->idstr, 256); } =20 - for (i =3D 0; i < pages->num; i++) { + for (i =3D 0; i < pages->normal_num; i++) { /* there are architectures where ram_addr_t is 32 bit */ - uint64_t temp =3D pages->offset[i]; + uint64_t temp =3D pages->normal[i]; =20 packet->offset[i] =3D cpu_to_be64(temp); } =20 + for (i =3D 0; i < pages->zero_num; i++) { + /* there are architectures where ram_addr_t is 32 bit */ + uint64_t temp =3D pages->zero[i]; + + packet->offset[pages->normal_num + i] =3D cpu_to_be64(temp); + } + p->packets_sent++; - p->total_normal_pages +=3D pages->num; + p->total_normal_pages +=3D pages->normal_num; p->total_zero_pages +=3D pages->zero_num; =20 - trace_multifd_send(p->id, packet_num, pages->num, pages->zero_num, + trace_multifd_send(p->id, packet_num, pages->normal_num, pages->zero_n= um, p->flags, p->next_packet_size); } =20 @@ -451,6 +468,18 @@ static int multifd_recv_unfill_packet(MultiFDRecvParam= s *p, Error **errp) p->normal[i] =3D offset; } =20 + for (i =3D 0; i < p->zero_num; i++) { + uint64_t offset =3D be64_to_cpu(packet->offset[p->normal_num + i]); + + if (offset > (p->block->used_length - p->page_size)) { + error_setg(errp, "multifd: offset too long %" PRIu64 + " (max " RAM_ADDR_FMT ")", + offset, p->block->used_length); + return -1; + } + p->zero[i] =3D offset; + } + return 0; } =20 @@ -842,7 +871,7 @@ static void *multifd_send_thread(void *opaque) =20 stat64_add(&mig_stats.multifd_bytes, p->next_packet_size + p->packet_len); - stat64_add(&mig_stats.normal_pages, pages->num); + stat64_add(&mig_stats.normal_pages, pages->normal_num); stat64_add(&mig_stats.zero_pages, pages->zero_num); =20 multifd_pages_reset(p->pages); @@ -1256,7 +1285,8 @@ static void *multifd_recv_thread(void *opaque) p->flags &=3D ~MULTIFD_FLAG_SYNC; qemu_mutex_unlock(&p->mutex); =20 - if (p->normal_num) { + if (p->normal_num + p->zero_num) { + assert(!(flags & MULTIFD_FLAG_SYNC)); ret =3D multifd_recv_state->ops->recv_pages(p, &local_err); if (ret !=3D 0) { break; diff --git a/migration/multifd.h b/migration/multifd.h index 9822ff298a..125f0bbe60 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -53,6 +53,11 @@ typedef struct { uint32_t unused32[1]; /* Reserved for future use */ uint64_t unused64[3]; /* Reserved for future use */ char ramblock[256]; + /* + * This array contains the pointers to: + * - normal pages (initial normal_pages entries) + * - zero pages (following zero_pages entries) + */ uint64_t offset[]; } __attribute__((packed)) MultiFDPacket_t; =20 @@ -224,6 +229,8 @@ typedef struct { =20 void multifd_register_ops(int method, MultiFDMethods *ops); void multifd_send_fill_packet(MultiFDSendParams *p); +void multifd_zero_page_check_send(MultiFDSendParams *p); +void multifd_zero_page_check_recv(MultiFDRecvParams *p); =20 static inline void multifd_send_prepare_header(MultiFDSendParams *p) { diff --git a/qapi/migration.json b/qapi/migration.json index 99843a8e95..e2450b92d4 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -660,9 +660,11 @@ # # @none: Do not perform zero page checking. # +# @multifd: Perform zero page checking on the multifd sender thread. (sinc= e 9.0) +# ## { 'enum': 'ZeroPageDetection', - 'data': [ 'legacy', 'none' ] } + 'data': [ 'legacy', 'none', 'multifd' ] } =20 ## # @BitmapMigrationBitmapAliasTransform: --=20 2.30.2 From nobody Fri Nov 1 05:35:13 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1708123331; cv=none; d=zohomail.com; s=zohoarc; b=LPDAS5KkZ657+NJLvA7dmBZBM76v4+L1BB+NCcBhs/kWAa1pUkmo2eslenbCq/yFbpW7cJtZHtJ5LA2caA2FZaPo6V5WCwDHLyD0gxvRC+Y754aQyNktjQ9DKQWE8cj5kdxAMg/hxAQjqcuUc3vWOElijGH0GGqsY58ebf04Sx8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708123331; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fvpt2MlZ0tbwr8cgiAzgK+uI+W/ZNpZ/DpKFqWpBVDc=; b=UEPY0ZjBDlVF62gm0RKQFSakBScUaIGRe1o4fSm0dA6MNF42Mzc80UwsiYtriJDZp488dfNzDiDJNMZfgk/vSp740eVzazycfPpVR2SURafEg8oG6SQgZHJ4TbJ7pokPVZaIPc9VF/ekLEohLlFafhQyRlritngatFC95dRiIbw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708123331361405.9417073834751; Fri, 16 Feb 2024 14:42:11 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rb6tO-0004yM-P3; Fri, 16 Feb 2024 17:41:14 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rb6tM-0004v1-68 for qemu-devel@nongnu.org; Fri, 16 Feb 2024 17:41:12 -0500 Received: from mail-vk1-xa34.google.com ([2607:f8b0:4864:20::a34]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rb6tK-0007Ia-Bu for qemu-devel@nongnu.org; Fri, 16 Feb 2024 17:41:11 -0500 Received: by mail-vk1-xa34.google.com with SMTP id 71dfb90a1353d-4c015883a13so395642e0c.1 for ; Fri, 16 Feb 2024 14:41:10 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.104]) by smtp.gmail.com with ESMTPSA id cz18-20020a056122449200b004c0a12c4d53sm120991vkb.51.2024.02.16.14.41.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 14:41:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1708123269; x=1708728069; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fvpt2MlZ0tbwr8cgiAzgK+uI+W/ZNpZ/DpKFqWpBVDc=; b=JTmKksnSPm7Iqb/aBS8xgpxpZlPzVT+zNQM/OKmAQGGjJsdUIxN5xyaaiGQnexiJ7d ElxxlyKpy01J+JD9ZgF4STkHvNMGl9B5lblJ1haSMSIiDSAmT97FA2wy/7tvjcDWV0hC h2fj8+KSiZuf4JF9xrJho5LlkswSdg3Xi5pAKu9fE0Ytl6q3yjUIxCLhpfWXqI2lBNvB fzqlILwYf3Lve+xBo9aV8r4yMy3U7oj5WHKKMCQXNChkwwm3Yr/+2sVEWP2z4t/AtuX6 eDyhCM0FMFhf0+imxkhRiQKO8DPCg0WSYBCap2UwEbtR06nuu1wjJH/uLqWBKO6Vuz49 q6ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708123269; x=1708728069; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fvpt2MlZ0tbwr8cgiAzgK+uI+W/ZNpZ/DpKFqWpBVDc=; b=uVAQafr8kuaemPc9HjwfX3X24TZvCU4qGNmkQIf77bQ3D0iUczl7krWX1yhOf/GPFd SbDGP4rmjZYHz5aFPyg0WMqwZYM3jiZg6tU74gJzoPRVSSVLbfr0Sdo7ZMXVnDNuoXac aqllV9XBI6M36fXVbh6hD8TwF0I5lA5qvb/xZq5F0rTJVT4JRhYf58BSyXEbHUGbIPd+ 4WTIrTsjp5/4FCspTdfzbrQErOgT7v9Ws1yjfb60RZAsuqXXrGFSR9qG43V2g/PO1iZT A5tOaoj6XXR63qGYpDfKzT5n8RJTCgnjRsQB5FrSBkMt16tLk1PoRDX4NWZPpIIFwuUz 5TSg== X-Forwarded-Encrypted: i=1; AJvYcCUuO1o6zuNyd7bhWmwPi/lp1xzNDCSY9KvXhk4bSkJ6bqQxYn4GGEA/tR/x1x6/9zwrwd5v52Fv1JeyxgJkC0sEh30i+jI= X-Gm-Message-State: AOJu0YxIo+5TbFPQ3NuT5qEUoK7YSDTU41a/A0VJVzEzhbD2Z+7Ml8yT 0fngRhUoN9K0CZW/BK3ulMfjM32ZhI3IRCh/GSnNnp9TJX48Hmc63RB8CV6bzRI= X-Google-Smtp-Source: AGHT+IELGioLAf8VHpiOIIlebMrU2v8lgXxhXi3dhvFb4yrLU01i8xLFBhL6J9DXMdmzTvB992SdbQ== X-Received: by 2002:a1f:4804:0:b0:4bd:32c9:acb with SMTP id v4-20020a1f4804000000b004bd32c90acbmr5291648vka.7.1708123269125; Fri, 16 Feb 2024 14:41:09 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, qemu-devel@nongnu.org, jdenemar@redhat.com Cc: Hao Xiang Subject: [PATCH v2 4/7] migration/multifd: Enable zero page checking from multifd threads. Date: Fri, 16 Feb 2024 22:39:59 +0000 Message-Id: <20240216224002.1476890-5-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240216224002.1476890-1-hao.xiang@bytedance.com> References: <20240216224002.1476890-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::a34; envelope-from=hao.xiang@bytedance.com; helo=mail-vk1-xa34.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1708123332895100003 Content-Type: text/plain; charset="utf-8" This change adds a dedicated handler for MigrationOps::ram_save_target_page= in multifd live migration. Now zero page checking can be done in the multifd t= hreads and this becomes the default configuration. We still provide backward compa= tibility where zero page checking is done from the migration main thread. Signed-off-by: Hao Xiang --- migration/multifd.c | 1 + migration/options.c | 2 +- migration/ram.c | 53 ++++++++++++++++++++++++++++++++++----------- 3 files changed, 42 insertions(+), 14 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index fbb40ea10b..ef5dad1019 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -13,6 +13,7 @@ #include "qemu/osdep.h" #include "qemu/cutils.h" #include "qemu/rcu.h" +#include "qemu/cutils.h" #include "exec/target_page.h" #include "sysemu/sysemu.h" #include "exec/ramblock.h" diff --git a/migration/options.c b/migration/options.c index 3c603391b0..3c79b6ccd4 100644 --- a/migration/options.c +++ b/migration/options.c @@ -181,7 +181,7 @@ Property migration_properties[] =3D { MIG_MODE_NORMAL), DEFINE_PROP_ZERO_PAGE_DETECTION("zero-page-detection", MigrationState, parameters.zero_page_detection, - ZERO_PAGE_DETECTION_LEGACY), + ZERO_PAGE_DETECTION_MULTIFD), =20 /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), diff --git a/migration/ram.c b/migration/ram.c index 5ece9f042e..b088c5a98c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1123,10 +1123,6 @@ static int save_zero_page(RAMState *rs, PageSearchSt= atus *pss, QEMUFile *file =3D pss->pss_channel; int len =3D 0; =20 - if (migrate_zero_page_detection() !=3D ZERO_PAGE_DETECTION_LEGACY) { - return 0; - } - if (!buffer_is_zero(p, TARGET_PAGE_SIZE)) { return 0; } @@ -1256,6 +1252,10 @@ static int ram_save_page(RAMState *rs, PageSearchSta= tus *pss) =20 static int ram_save_multifd_page(RAMBlock *block, ram_addr_t offset) { + assert(migrate_multifd()); + assert(!migrate_compress()); + assert(!migration_in_postcopy()); + if (!multifd_queue_page(block, offset)) { return -1; } @@ -2046,7 +2046,6 @@ static bool save_compress_page(RAMState *rs, PageSear= chStatus *pss, */ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss) { - RAMBlock *block =3D pss->block; ram_addr_t offset =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; int res; =20 @@ -2062,17 +2061,40 @@ static int ram_save_target_page_legacy(RAMState *rs= , PageSearchStatus *pss) return 1; } =20 + return ram_save_page(rs, pss); +} + +/** + * ram_save_target_page_multifd: save one target page + * + * Returns the number of pages written + * + * @rs: current RAM state + * @pss: data about the page we want to send + */ +static int ram_save_target_page_multifd(RAMState *rs, PageSearchStatus *ps= s) +{ + RAMBlock *block =3D pss->block; + ram_addr_t offset =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; + + /* Multifd is not compatible with old compression. */ + assert(!migrate_compress()); + + /* Multifd is not compabible with postcopy. */ + assert(!migration_in_postcopy()); + /* - * Do not use multifd in postcopy as one whole host page should be - * placed. Meanwhile postcopy requires atomic update of pages, so even - * if host page size =3D=3D guest page size the dest guest during run = may - * still see partially copied pages which is data corruption. + * Backward compatibility support. While using multifd live + * migration, we still need to handle zero page checking on the + * migration main thread. */ - if (migrate_multifd() && !migration_in_postcopy()) { - return ram_save_multifd_page(block, offset); + if (migrate_zero_page_detection() =3D=3D ZERO_PAGE_DETECTION_LEGACY) { + if (save_zero_page(rs, pss, offset)) { + return 1; + } } =20 - return ram_save_page(rs, pss); + return ram_save_multifd_page(block, offset); } =20 /* Should be called before sending a host page */ @@ -2984,7 +3006,12 @@ static int ram_save_setup(QEMUFile *f, void *opaque) } =20 migration_ops =3D g_malloc0(sizeof(MigrationOps)); - migration_ops->ram_save_target_page =3D ram_save_target_page_legacy; + + if (migrate_multifd()) { + migration_ops->ram_save_target_page =3D ram_save_target_page_multi= fd; + } else { + migration_ops->ram_save_target_page =3D ram_save_target_page_legac= y; + } =20 bql_unlock(); ret =3D multifd_send_sync_main(); --=20 2.30.2 From nobody Fri Nov 1 05:35:13 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1708123341; cv=none; d=zohomail.com; s=zohoarc; b=Nij7w3rpcHlG0Ruc9wZo9zQcCU8ZpKjvlkCYRKTlaZvmYApU+TgwY2KjrH/BOMjXyhQDooTZz3rwFCTSiDJNNXbN/mZDy6j+eT6JkIWQLpYlxbkgLC99W4gbTPKQZxx9s7zMNW8EJpjtHrV1d7Rngo9IWcZoXVnrBKKdnRcPPO0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708123341; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=YwX0gK/0yCJN5W3TVY6lm3Na3nSQLdooSPqXCUs7/Kk=; b=YoIxgE1uwDWPEVkHZUFIv9Wr/o0x+8QCERCiGeZiSgSlec6WzYX3b5PXXV+VKYTy5cAsCRnckvFnbPu+irwg9fQZw3tsynizdKrpmRhPZXnuFih+UdEhkdJdA+kKR91QLrfzsHxNB8nvnyBGofCFVs007WGNaRct8W2BeIE5kbc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708123341943534.5920610773619; Fri, 16 Feb 2024 14:42:21 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rb6tQ-000544-3d; Fri, 16 Feb 2024 17:41:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rb6tN-0004vQ-Mv for qemu-devel@nongnu.org; Fri, 16 Feb 2024 17:41:13 -0500 Received: from mail-vk1-xa36.google.com ([2607:f8b0:4864:20::a36]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rb6tM-0007K6-0k for qemu-devel@nongnu.org; Fri, 16 Feb 2024 17:41:13 -0500 Received: by mail-vk1-xa36.google.com with SMTP id 71dfb90a1353d-4c0375d438eso973150e0c.0 for ; Fri, 16 Feb 2024 14:41:11 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.104]) by smtp.gmail.com with ESMTPSA id cz18-20020a056122449200b004c0a12c4d53sm120991vkb.51.2024.02.16.14.41.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 14:41:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1708123271; x=1708728071; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YwX0gK/0yCJN5W3TVY6lm3Na3nSQLdooSPqXCUs7/Kk=; b=b7tFT5hgGieqV1KXZtA/UJanWiYIjN4nMc/QtT70k0zZSYNYQ4FZ93t3Ersse1ghZQ AUDcvHj349z2pTzgukv2Osa7LbvcNsY2fMTOxcBI/5TVdQIJuTgAu+obAmTS2Ps3ty+i uaGHrbRnaQ3ywjTi1cfakdE52/P8sMDeVIsSeyHe2XXZZokYENBJ7KFkidWkoAtPLTb1 xL81AL48Jv3iAPBuPglIrKXC8ZjsRaqxjrfDUC0z+7VkruUs0yP95Lwgx0UYqpHVJQfa aw3tmDm6ds6ULok5WbOegtxP+fwAkDi1FWsugLXThauHOB1W0gGiU9U4gyZmKZaCn164 nfvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708123271; x=1708728071; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YwX0gK/0yCJN5W3TVY6lm3Na3nSQLdooSPqXCUs7/Kk=; b=QCiZgtL7P9+rERTsNyslsQlIkk85pcZLCO8z7sAktF4FwJPnyS8H1tZTxELdEiuDoq MrSl+lbbG2yqek27VWtpnapM/LWegWcH/YXtww5Us36CJwBWb7drohfxfMmfzzdpGS/a iMTik0ry3DtQk01Gnum75eToGSC4N2j1LGYwboMBIN8UWxPqo/DwMuuQWSbfiyt8qCXO Vzua1ThbI4utBAPmDC5uyAtmPsOM0gV7YI4xWmqp8o0MYDXmkxjIYV0BBAp168TMUKVT njutF2HeyHio6B1Sp/H7LFNsBm15bacwx2XPpYbVy1HFksGGCc6dWnbgvF2Xew8VB2Ii HMKA== X-Forwarded-Encrypted: i=1; AJvYcCXatWiLqZ5hxpeUKRK3gs4/NiEpkddg/8EacYtv8zmV5W43MfIlZJnOI/K9NKcC+dGJyreh45unDTjh3Gge9yNMEdNzimc= X-Gm-Message-State: AOJu0YzTQtC8V0ccB8nBLbqhGOTy68cCOVYaafQipKFs75ZeH/NsU4zH kOhXJSZoBKX3s2jd6SxoCL/d3VleVVpJAh5s75Zd4uk6zm6KbONOKUvR3PYGguQ= X-Google-Smtp-Source: AGHT+IHFI/KudF8hUc+21IzrFrXYJ/N3FkmT7UjkcCVElptK5eXuuHo+YTx5537KczEFSg077WT9ew== X-Received: by 2002:a1f:cc86:0:b0:4c0:e29:c438 with SMTP id c128-20020a1fcc86000000b004c00e29c438mr7308132vkg.8.1708123270926; Fri, 16 Feb 2024 14:41:10 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, qemu-devel@nongnu.org, jdenemar@redhat.com Cc: Hao Xiang Subject: [PATCH v2 5/7] migration/multifd: Add new migration test cases for legacy zero page checking. Date: Fri, 16 Feb 2024 22:40:00 +0000 Message-Id: <20240216224002.1476890-6-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240216224002.1476890-1-hao.xiang@bytedance.com> References: <20240216224002.1476890-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::a36; envelope-from=hao.xiang@bytedance.com; helo=mail-vk1-xa36.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1708123342994100003 Content-Type: text/plain; charset="utf-8" Now that zero page checking is done on the multifd sender threads by default, we still provide an option for backward compatibility. This change adds a qtest migration test case to set the zero-page-detection option to "legacy" and run multifd migration with zero page checking on the migration main thread. Signed-off-by: Hao Xiang --- tests/qtest/migration-test.c | 52 ++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 8a5bb1752e..c27083110a 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -2621,6 +2621,24 @@ test_migrate_precopy_tcp_multifd_start(QTestState *f= rom, return test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); } =20 +static void * +test_migrate_precopy_tcp_multifd_start_zero_page_legacy(QTestState *from, + QTestState *to) +{ + test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); + migrate_set_parameter_str(from, "zero-page-detection", "legacy"); + return NULL; +} + +static void * +test_migration_precopy_tcp_multifd_start_no_zero_page(QTestState *from, + QTestState *to) +{ + test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); + migrate_set_parameter_str(from, "zero-page-detection", "none"); + return NULL; +} + static void * test_migrate_precopy_tcp_multifd_zlib_start(QTestState *from, QTestState *to) @@ -2652,6 +2670,36 @@ static void test_multifd_tcp_none(void) test_precopy_common(&args); } =20 +static void test_multifd_tcp_zero_page_legacy(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "defer", + .start_hook =3D test_migrate_precopy_tcp_multifd_start_zero_page_l= egacy, + /* + * Multifd is more complicated than most of the features, it + * directly takes guest page buffers when sending, make sure + * everything will work alright even if guest page is changing. + */ + .live =3D true, + }; + test_precopy_common(&args); +} + +static void test_multifd_tcp_no_zero_page(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "defer", + .start_hook =3D test_migration_precopy_tcp_multifd_start_no_zero_p= age, + /* + * Multifd is more complicated than most of the features, it + * directly takes guest page buffers when sending, make sure + * everything will work alright even if guest page is changing. + */ + .live =3D true, + }; + test_precopy_common(&args); +} + static void test_multifd_tcp_zlib(void) { MigrateCommon args =3D { @@ -3550,6 +3598,10 @@ int main(int argc, char **argv) } migration_test_add("/migration/multifd/tcp/plain/none", test_multifd_tcp_none); + migration_test_add("/migration/multifd/tcp/plain/zero_page_legacy", + test_multifd_tcp_zero_page_legacy); + migration_test_add("/migration/multifd/tcp/plain/no_zero_page", + test_multifd_tcp_no_zero_page); migration_test_add("/migration/multifd/tcp/plain/cancel", test_multifd_tcp_cancel); migration_test_add("/migration/multifd/tcp/plain/zlib", --=20 2.30.2 From nobody Fri Nov 1 05:35:13 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1708123338; cv=none; d=zohomail.com; s=zohoarc; b=hl9MNqbg2KIDINQRX2sfgL5+AEBTlqr5yPHbCNx1IpAi7p79HPAyyQ0318TtpWYxu9pBzQ4bWmK68b252qTcOrf/naMo8u9TbXBaj1YnMpSxkelnH/g28jiM4pkr+aC7w4I4SRG9GQRAINUWDq5EGsGRuY7o1a5CCi/224lSG5Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708123338; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=r/OfLhO0CA6m0WDBPYJTqQUVPYsQDRD36SN09xd28yU=; b=aHroY5hLsZuHiRDMg1nN1bc55sX5bhcRx3IeA4NU4MB3hx37+0x5gC8zhJkz/kEmUO0cqzOkFbmz9ve79xxY+hWAqFI+m+2qz7hUTwjr0NU9VZsC4SJZ/lSL8xL14J3EzwQTeCS7ItY1XH9u0+GR/3wrS6caEz3bnR5ZoHJhih8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708123338064313.6741679170892; Fri, 16 Feb 2024 14:42:18 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rb6tQ-00057D-Mi; Fri, 16 Feb 2024 17:41:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rb6tP-000513-Fm for qemu-devel@nongnu.org; Fri, 16 Feb 2024 17:41:15 -0500 Received: from mail-vk1-xa2c.google.com ([2607:f8b0:4864:20::a2c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rb6tN-0007KU-MW for qemu-devel@nongnu.org; Fri, 16 Feb 2024 17:41:15 -0500 Received: by mail-vk1-xa2c.google.com with SMTP id 71dfb90a1353d-4c01a5e85e8so699104e0c.1 for ; Fri, 16 Feb 2024 14:41:13 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.104]) by smtp.gmail.com with ESMTPSA id cz18-20020a056122449200b004c0a12c4d53sm120991vkb.51.2024.02.16.14.41.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 14:41:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1708123273; x=1708728073; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=r/OfLhO0CA6m0WDBPYJTqQUVPYsQDRD36SN09xd28yU=; b=NozMc7+GQFb5iur7RwiYsaYUX9fUhMml3ZhOLnmK72zUDcitP+MEKTsrkCLaIRKU+s DfJfTQDwFYqw+vTAbrdS7DP3A7s+YiyclzvlQJhPQATl2hkMQ1U+5TmWPg1M+Q1mEeEi +F/6yeXDxvDg5Vqo7lZou940kF9e2cu6ZiUqG1SpIAzp6fl4FiOuMYLG6I4GWU5LQhZ2 C54MXPiBY2S8uQgjYpkehL4kMGmjwxbpzdvRSwxUM5XaWBXmJKOhehNOvtba0yhzEi8P vPonJIPzW/kagce2RLWLjl0IDL2jsPiRlRqSikJuO4OalVmEPlENl4jxSGrMIs71XKMI HW8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708123273; x=1708728073; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=r/OfLhO0CA6m0WDBPYJTqQUVPYsQDRD36SN09xd28yU=; b=fOk27XMUzWzO5zPYh38+ijjAQmhrYiUYw2CowV8AK3HiWZrUa+B8rkWc8NYzSvBTXs dYOLxWKZVOhwi098JK5dmPKk57vUZDh7576ylsasc1es4+HnybcfugirFFLEecjEFnGr 7gElokSjoxWjRS8Jg9gnXJXTouZLyir2WmCEEKudrciGsg0MFnpWRKc/EFdQGpKu6p7C 8VkeWgHFoDPbOvY4f63jNUnKvrkWDUVthoZ4DMMEY41PNA19e8PhI3B/d7pditko3g6D B1GLffnvgdpzGaGKXGiwZOCPg4pvXjJEPdhAy+7uk6NmHClxyal2Pbar/WagcREh6Wf6 hFNA== X-Forwarded-Encrypted: i=1; AJvYcCVDz45my7b2kjETxkalusXA6NpqqubETVMRuYQEEZ5txg/5kDyvlI1YNeuKZQ+36Vwqlguc+jfDkABeop1L0p1Yhhtqjao= X-Gm-Message-State: AOJu0YzCm8LmgiDxSUPjdSYmNnuUoXZ3FPH44D+2yKmy6/sATUQCpARl 8NqVsP9m0dVCgo1H8YB898+VacaSS/tm1uigzg4CB1uauoDnArn3j7mXK4GysMw= X-Google-Smtp-Source: AGHT+IH2r+7Tp4EotXB4AUc4JLvq9c/++hrwcItqmtZKi1Kc2tpGnm+BH+MP+zmM6aDFJ2/fVLAdng== X-Received: by 2002:a1f:eb03:0:b0:4b6:cc19:42dc with SMTP id j3-20020a1feb03000000b004b6cc1942dcmr6329585vkh.11.1708123272767; Fri, 16 Feb 2024 14:41:12 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, qemu-devel@nongnu.org, jdenemar@redhat.com Cc: Hao Xiang Subject: [PATCH v2 6/7] migration/multifd: Add zero pages and zero bytes counter to migration status interface. Date: Fri, 16 Feb 2024 22:40:01 +0000 Message-Id: <20240216224002.1476890-7-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240216224002.1476890-1-hao.xiang@bytedance.com> References: <20240216224002.1476890-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::a2c; envelope-from=hao.xiang@bytedance.com; helo=mail-vk1-xa2c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1708123338817100002 Content-Type: text/plain; charset="utf-8" This change extends the MigrationStatus interface to track zero pages and zero bytes counter. Signed-off-by: Hao Xiang --- migration/migration-hmp-cmds.c | 4 ++++ migration/migration.c | 2 ++ qapi/migration.json | 15 ++++++++++++++- tests/migration/guestperf/engine.py | 2 ++ 4 files changed, 22 insertions(+), 1 deletion(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 7e96ae6ffd..abe035c9f2 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -111,6 +111,10 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) info->ram->normal); monitor_printf(mon, "normal bytes: %" PRIu64 " kbytes\n", info->ram->normal_bytes >> 10); + monitor_printf(mon, "zero: %" PRIu64 " pages\n", + info->ram->zero); + monitor_printf(mon, "zero bytes: %" PRIu64 " kbytes\n", + info->ram->zero_bytes >> 10); monitor_printf(mon, "dirty sync count: %" PRIu64 "\n", info->ram->dirty_sync_count); monitor_printf(mon, "page size: %" PRIu64 " kbytes\n", diff --git a/migration/migration.c b/migration/migration.c index ab21de2cad..1968ea7075 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1112,6 +1112,8 @@ static void populate_ram_info(MigrationInfo *info, Mi= grationState *s) info->ram->skipped =3D 0; info->ram->normal =3D stat64_get(&mig_stats.normal_pages); info->ram->normal_bytes =3D info->ram->normal * page_size; + info->ram->zero =3D stat64_get(&mig_stats.zero_pages); + info->ram->zero_bytes =3D info->ram->zero * page_size; info->ram->mbps =3D s->mbps; info->ram->dirty_sync_count =3D stat64_get(&mig_stats.dirty_sync_count); diff --git a/qapi/migration.json b/qapi/migration.json index e2450b92d4..892875da18 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -63,6 +63,10 @@ # between 0 and @dirty-sync-count * @multifd-channels. (since # 7.1) # +# @zero: number of zero pages (since 9.0) +# +# @zero-bytes: number of zero bytes sent (since 9.0) +# # Features: # # @deprecated: Member @skipped is always zero since 1.5.3 @@ -81,7 +85,8 @@ 'multifd-bytes': 'uint64', 'pages-per-second': 'uint64', 'precopy-bytes': 'uint64', 'downtime-bytes': 'uint64', 'postcopy-bytes': 'uint64', - 'dirty-sync-missed-zero-copy': 'uint64' } } + 'dirty-sync-missed-zero-copy': 'uint64', + 'zero': 'int', 'zero-bytes': 'int' } } =20 ## # @XBZRLECacheStats: @@ -332,6 +337,8 @@ # "duplicate":123, # "normal":123, # "normal-bytes":123456, +# "zero":123, +# "zero-bytes":123456, # "dirty-sync-count":15 # } # } @@ -358,6 +365,8 @@ # "duplicate":123, # "normal":123, # "normal-bytes":123456, +# "zero":123, +# "zero-bytes":123456, # "dirty-sync-count":15 # } # } @@ -379,6 +388,8 @@ # "duplicate":123, # "normal":123, # "normal-bytes":123456, +# "zero":123, +# "zero-bytes":123456, # "dirty-sync-count":15 # }, # "disk":{ @@ -405,6 +416,8 @@ # "duplicate":10, # "normal":3333, # "normal-bytes":3412992, +# "zero":3333, +# "zero-bytes":3412992, # "dirty-sync-count":15 # }, # "xbzrle-cache":{ diff --git a/tests/migration/guestperf/engine.py b/tests/migration/guestper= f/engine.py index 608d7270f6..75315b99b7 100644 --- a/tests/migration/guestperf/engine.py +++ b/tests/migration/guestperf/engine.py @@ -92,6 +92,8 @@ def _migrate_progress(self, vm): info["ram"].get("skipped", 0), info["ram"].get("normal", 0), info["ram"].get("normal-bytes", 0), + info["ram"].get("zero", 0); + info["ram"].get("zero-bytes", 0); info["ram"].get("dirty-pages-rate", 0), info["ram"].get("mbps", 0), info["ram"].get("dirty-sync-count", 0) --=20 2.30.2 From nobody Fri Nov 1 05:35:13 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1708123320; cv=none; d=zohomail.com; s=zohoarc; b=A5YchXfBuqtvB/rzxJ+ouJc5b5iKeYIGA7nsrxDdmD35Jr2riWMxbukduJW0HKWPQg39uGQIpYLRImbEY1xFPOq+aWVfNjGUvNRmiW+LoDjnpT+LLFEmkW4FCBK2KBCLtmza0o8DMEgoNYVMLiol2e4lVuMgsZqNJIEMails4oc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708123320; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=PsYydAOoJWHHnnEKgrZhgm6MgPanWUyFgyt4kohdSeI=; b=cL0SDF3/dp/Tv5P1Gb4SwQ/2rtl0WorCrKt96AD0ry9QRMzXQalbhvhic49hbncBrZUOVsYpiNUJViJLceN1FF/QmyU4AFmv9URwoIHyXP/ovOICxp4ht0pSr+a2mjHImDlwyKxs8wNymIMfSgSICjXNMxFm/EOyi03nksCyoyk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17081233207331005.4527741910807; Fri, 16 Feb 2024 14:42:00 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rb6tT-0005Du-89; Fri, 16 Feb 2024 17:41:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rb6tR-0005Bh-Aa for qemu-devel@nongnu.org; Fri, 16 Feb 2024 17:41:17 -0500 Received: from mail-ua1-x92c.google.com ([2607:f8b0:4864:20::92c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rb6tP-0007Kq-S9 for qemu-devel@nongnu.org; Fri, 16 Feb 2024 17:41:17 -0500 Received: by mail-ua1-x92c.google.com with SMTP id a1e0cc1a2514c-7d5c25267deso1110262241.3 for ; Fri, 16 Feb 2024 14:41:15 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.104]) by smtp.gmail.com with ESMTPSA id cz18-20020a056122449200b004c0a12c4d53sm120991vkb.51.2024.02.16.14.41.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 14:41:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1708123275; x=1708728075; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PsYydAOoJWHHnnEKgrZhgm6MgPanWUyFgyt4kohdSeI=; b=kTFONq3bwhvjHSUD6H3GHAy8Z4BOVew8VTkAVbr/18MMVv9MQsiXpas12WbjLaSpwh u6nQvuPds0KJbFnIQaNjPCuC4d6Xc5q9hlS6mCSNZZenkltk4tXGJlkE4CNCJIzNfHOL GupzOHQvRhPr/hz4/+EGcsqfTixoRuCY9SXG/r6Xe+k2TkelWDON+Nh5AZP6Lu2woag9 sKwOiTDehWd3HbpZX5SRhg5/6ado0dOvv+e6KGg0vnOVgG2wnXpnWXtyLmMWE+uqy6w5 T5GF6PTQxEAS6xghBmDXl9GqPKOs7m92MJL3u73LuNhyCRuyvch63R5MBUtcQSIvgDI0 F3GQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708123275; x=1708728075; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PsYydAOoJWHHnnEKgrZhgm6MgPanWUyFgyt4kohdSeI=; b=jTi2CliFt6KF/UusBK5f9rU7olZuHRGrcwZjhhyJMRREvK1yZ4s8bPpWfVk1FvJZat 2UAnLtEZ76/myQt8/neX0ZO6NLMmVZZqwm88anUJTW/hdbf5hWd8pGDQ/vtentP3IjVb Xm40PtHL8r+MFhSMhZTYT4qHdt0BstwJF7sw7h3CNk5e/EzNlx4G+qV51ipn0vJx/GAy WE6bcaUprX9xHYrlFWxn78DTlZ5uFhgOf4ezbq/YF3NqdDQG8sWbDl/gRS68i9MjIJtq k8eawoRY0TMxP3Ps6VGpZCHe4s0ZpiQz+K/wsRpP7LFPWlh62dNh+rfIH2NOgGJZZZh+ 6BKQ== X-Forwarded-Encrypted: i=1; AJvYcCVPw/p9qP1/D/IpJAfq8nQqoKA6xyKEuRdq+7tMLGMwkwYJzZe1DEFlX93tyh7CNrMc0kG8Fq9vtccw9Fmudw4/cJk0/pw= X-Gm-Message-State: AOJu0YyKGKqfKXxeHbn4REVOQ+akfOOJ581NtM+fGC6c2VqlCXexmIyP tyKQWNIuzysUQJGJarc3aDej62KD+Qfv4VnmIqJx8YikMAJpr678osn0fIateos= X-Google-Smtp-Source: AGHT+IFgQ4dxzpTeD9lxmwHvMchPMHL0MhqGhTVtdiUJSghu6+bQTyXTvylsMCb5cxdCcvaULTyKxA== X-Received: by 2002:a1f:ec83:0:b0:4c0:2cff:9d7a with SMTP id k125-20020a1fec83000000b004c02cff9d7amr6631435vkh.5.1708123274843; Fri, 16 Feb 2024 14:41:14 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, qemu-devel@nongnu.org, jdenemar@redhat.com Cc: Hao Xiang Subject: [PATCH v2 7/7] Update maintainer contact for migration multifd zero page checking acceleration. Date: Fri, 16 Feb 2024 22:40:02 +0000 Message-Id: <20240216224002.1476890-8-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240216224002.1476890-1-hao.xiang@bytedance.com> References: <20240216224002.1476890-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::92c; envelope-from=hao.xiang@bytedance.com; helo=mail-ua1-x92c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1708123322733100002 Content-Type: text/plain; charset="utf-8" Add myself to maintain multifd zero page checking acceleration function. Signed-off-by: Hao Xiang --- MAINTAINERS | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index a24c2b51b6..3ca407cb58 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3403,6 +3403,11 @@ F: tests/migration/ F: util/userfaultfd.c X: migration/rdma* =20 +Migration multifd zero page checking acceleration +M: Hao Xiang +S: Maintained +F: migration/multifd-zero-page.c + RDMA Migration R: Li Zhijian R: Peter Xu --=20 2.30.2