From nobody Tue Nov 26 06:50:20 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1708977572; cv=none; d=zohomail.com; s=zohoarc; b=CIpud8MWfloY8zohQO7b4C5Rsy19sGy47ryiI/xmBM6/d7WCaz/Cu4lAfUN+Fbbac0wkmk5yqqcmHBdAqnpNn2p7v2D2SWEXeUdko7sM84cZ81lB0zNZNYCTpXuxle7udg7GYfB+yIjpffZ1W/XBMIsUj835GDZMGk6jC3AkQG0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708977572; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ZgDjuALvbSevFWEfp0Xrz/jY7qt4RLoP+bPDhImpo/k=; b=a7saYEioyUnTbwdrtfHIpSqK2V5nCYi91LklSgD7gifShGT6F5KKaGrl+95eyvUjHSoQxyAWwHyV1rROcZc+SuevGYpCh7srntH2D3xsdpPqzRtwNtD446oyyvx0NP+yYWvqv1fVOoDN263EGAJKe6p45IF8Mz+P+UzWEyODdm8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708977572368905.6322417702992; Mon, 26 Feb 2024 11:59:32 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1reh6u-00012i-Et; Mon, 26 Feb 2024 14:58:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1reh6m-0000yj-Tm for qemu-devel@nongnu.org; Mon, 26 Feb 2024 14:57:53 -0500 Received: from mail-vk1-xa30.google.com ([2607:f8b0:4864:20::a30]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1reh6k-0004tk-L0 for qemu-devel@nongnu.org; Mon, 26 Feb 2024 14:57:52 -0500 Received: by mail-vk1-xa30.google.com with SMTP id 71dfb90a1353d-4cb26623d0aso809017e0c.1 for ; Mon, 26 Feb 2024 11:57:49 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.95]) by smtp.gmail.com with ESMTPSA id ej5-20020a056122270500b004ca3dc45886sm712738vkb.47.2024.02.26.11.57.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 11:57:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1708977469; x=1709582269; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZgDjuALvbSevFWEfp0Xrz/jY7qt4RLoP+bPDhImpo/k=; b=VsbAA6lYtbS7rugZBrrV+RWXL+77F4qc8Fs1eMiazc1Vs0jh/6R9nLhKXg6exjeZg5 M/DMxQBaqNsXyDs94hIIToOzvjoRvNC5NG5hhDJo3GwDSEMqCvHhy72NH0msB62SGi9I PYDFi9Q4AJtOMgVfK1DjKV++cniLR3HRrvCR59IXAPEFU3YwXdoGeIYT6BZ+GKjRqDJg C9WK5Dy/NOTrqnS2MHSajQQGUkWis5/ZEvyXzg7AI9Y19CaTmi5aq639mlOg28j8ZW3K 6SOK8bldE7GVPX6KQN34+ZYgtFmlBvO4yBbEzW+y9Ez+ugdtDmhC7d9arUSe1A89TdyE kt+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708977469; x=1709582269; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZgDjuALvbSevFWEfp0Xrz/jY7qt4RLoP+bPDhImpo/k=; b=U0I0y5BhKHW2eMe2ysLbVvOFwFGDFJUCZnEw9TQtwWsQbTBWXm7Ihbd2MH790Yj/vP fnrBeQYGPN6JqJEMKFB0ChwGfBqhjthyz2A7ECEFSF3GSA5/7hTNNMCCzUJxghfTyoZc KsKfA3LL/+krFKS6Gpum1YRfm1WXF69uNYLntKzxF/+z2eJZWGblF9pa1Qr4qKi/y2J7 ELlQ7g2iscem/vh+h5AG5/HBPHMqYlQOOsKpCy8GAAjKJjgA5ehYpK57XTDKZzucdYBe t9PUBnQRSY712+tp4RfbDgWnwymDnh1KhpnI9WYHoWMnkxZ+Pd0mEcZ4/rf+i71dsvit NypQ== X-Forwarded-Encrypted: i=1; AJvYcCUfQuiJ1NDoyYIFz8GWyUj3sweUSCRbTkBDJgqA60Dgc49IT6VdPr2if6EbL4JU/fA0Cltf4Ud5GoUwm7554HGOrad6/hk= X-Gm-Message-State: AOJu0YzNcHCHVCFIyfzmH9deB89E5KvQlkNjN2FOt4Q4kL8RXGnt1P/O a/nUVTLNmy4wa95BNe3jca+wzuPt3xthO6UrIc31HsoT8BSXBjgXkxBu3Sqreqo= X-Google-Smtp-Source: AGHT+IF96s81tLbr+OkLEguOKeJ2SwaPlPN5oDO+/nB5jtBRiMfnlJsTGNwBA86UIrNMiPycO6u+KQ== X-Received: by 2002:a1f:48c3:0:b0:4c9:98f8:83d7 with SMTP id v186-20020a1f48c3000000b004c998f883d7mr4354080vka.0.1708977469029; Mon, 26 Feb 2024 11:57:49 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, jdenemar@redhat.com, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, qemu-devel@nongnu.org Cc: Hao Xiang Subject: [PATCH v3 1/7] migration/multifd: Add new migration option zero-page-detection. Date: Mon, 26 Feb 2024 19:56:48 +0000 Message-Id: <20240226195654.934709-2-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240226195654.934709-1-hao.xiang@bytedance.com> References: <20240226195654.934709-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::a30; envelope-from=hao.xiang@bytedance.com; helo=mail-vk1-xa30.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1708977573653100002 Content-Type: text/plain; charset="utf-8" This new parameter controls where the zero page checking is running. 1. If this parameter is set to 'legacy', zero page checking is done in the migration main thread. 2. If this parameter is set to 'none', zero page checking is disabled. Signed-off-by: Hao Xiang --- hw/core/qdev-properties-system.c | 10 ++++++++++ include/hw/qdev-properties-system.h | 4 ++++ migration/migration-hmp-cmds.c | 9 +++++++++ migration/options.c | 21 ++++++++++++++++++++ migration/options.h | 1 + migration/ram.c | 4 ++++ qapi/migration.json | 30 ++++++++++++++++++++++++++--- 7 files changed, 76 insertions(+), 3 deletions(-) diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-sys= tem.c index 1a396521d5..228e685f52 100644 --- a/hw/core/qdev-properties-system.c +++ b/hw/core/qdev-properties-system.c @@ -679,6 +679,16 @@ const PropertyInfo qdev_prop_mig_mode =3D { .set_default_value =3D qdev_propinfo_set_default_value_enum, }; =20 +const PropertyInfo qdev_prop_zero_page_detection =3D { + .name =3D "ZeroPageDetection", + .description =3D "zero_page_detection values, " + "none,legacy", + .enum_table =3D &ZeroPageDetection_lookup, + .get =3D qdev_propinfo_get_enum, + .set =3D qdev_propinfo_set_enum, + .set_default_value =3D qdev_propinfo_set_default_value_enum, +}; + /* --- Reserved Region --- */ =20 /* diff --git a/include/hw/qdev-properties-system.h b/include/hw/qdev-properti= es-system.h index 06c359c190..839b170235 100644 --- a/include/hw/qdev-properties-system.h +++ b/include/hw/qdev-properties-system.h @@ -8,6 +8,7 @@ extern const PropertyInfo qdev_prop_macaddr; extern const PropertyInfo qdev_prop_reserved_region; extern const PropertyInfo qdev_prop_multifd_compression; extern const PropertyInfo qdev_prop_mig_mode; +extern const PropertyInfo qdev_prop_zero_page_detection; extern const PropertyInfo qdev_prop_losttickpolicy; extern const PropertyInfo qdev_prop_blockdev_on_error; extern const PropertyInfo qdev_prop_bios_chs_trans; @@ -47,6 +48,9 @@ extern const PropertyInfo qdev_prop_iothread_vq_mapping_l= ist; #define DEFINE_PROP_MIG_MODE(_n, _s, _f, _d) \ DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_mig_mode, \ MigMode) +#define DEFINE_PROP_ZERO_PAGE_DETECTION(_n, _s, _f, _d) \ + DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_zero_page_detection, \ + ZeroPageDetection) #define DEFINE_PROP_LOSTTICKPOLICY(_n, _s, _f, _d) \ DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_losttickpolicy, \ LostTickPolicy) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 99b49df5dd..7e96ae6ffd 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -344,6 +344,11 @@ void hmp_info_migrate_parameters(Monitor *mon, const Q= Dict *qdict) monitor_printf(mon, "%s: %s\n", MigrationParameter_str(MIGRATION_PARAMETER_MULTIFD_COMPRESSION= ), MultiFDCompression_str(params->multifd_compression)); + assert(params->has_zero_page_detection); + monitor_printf(mon, "%s: %s\n", + MigrationParameter_str(MIGRATION_PARAMETER_ZERO_PAGE_DETECTION= ), + qapi_enum_lookup(&ZeroPageDetection_lookup, + params->zero_page_detection)); monitor_printf(mon, "%s: %" PRIu64 " bytes\n", MigrationParameter_str(MIGRATION_PARAMETER_XBZRLE_CACHE_SIZE), params->xbzrle_cache_size); @@ -634,6 +639,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDi= ct *qdict) p->has_multifd_zstd_level =3D true; visit_type_uint8(v, param, &p->multifd_zstd_level, &err); break; + case MIGRATION_PARAMETER_ZERO_PAGE_DETECTION: + p->has_zero_page_detection =3D true; + visit_type_ZeroPageDetection(v, param, &p->zero_page_detection, &e= rr); + break; case MIGRATION_PARAMETER_XBZRLE_CACHE_SIZE: p->has_xbzrle_cache_size =3D true; if (!visit_type_size(v, param, &cache_size, &err)) { diff --git a/migration/options.c b/migration/options.c index 3e3e0b93b4..3c603391b0 100644 --- a/migration/options.c +++ b/migration/options.c @@ -179,6 +179,9 @@ Property migration_properties[] =3D { DEFINE_PROP_MIG_MODE("mode", MigrationState, parameters.mode, MIG_MODE_NORMAL), + DEFINE_PROP_ZERO_PAGE_DETECTION("zero-page-detection", MigrationState, + parameters.zero_page_detection, + ZERO_PAGE_DETECTION_LEGACY), =20 /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), @@ -903,6 +906,13 @@ uint64_t migrate_xbzrle_cache_size(void) return s->parameters.xbzrle_cache_size; } =20 +ZeroPageDetection migrate_zero_page_detection(void) +{ + MigrationState *s =3D migrate_get_current(); + + return s->parameters.zero_page_detection; +} + /* parameter setters */ =20 void migrate_set_block_incremental(bool value) @@ -1013,6 +1023,8 @@ MigrationParameters *qmp_query_migrate_parameters(Err= or **errp) params->vcpu_dirty_limit =3D s->parameters.vcpu_dirty_limit; params->has_mode =3D true; params->mode =3D s->parameters.mode; + params->has_zero_page_detection =3D true; + params->zero_page_detection =3D s->parameters.zero_page_detection; =20 return params; } @@ -1049,6 +1061,7 @@ void migrate_params_init(MigrationParameters *params) params->has_x_vcpu_dirty_limit_period =3D true; params->has_vcpu_dirty_limit =3D true; params->has_mode =3D true; + params->has_zero_page_detection =3D true; } =20 /* @@ -1350,6 +1363,10 @@ static void migrate_params_test_apply(MigrateSetPara= meters *params, if (params->has_mode) { dest->mode =3D params->mode; } + + if (params->has_zero_page_detection) { + dest->zero_page_detection =3D params->zero_page_detection; + } } =20 static void migrate_params_apply(MigrateSetParameters *params, Error **err= p) @@ -1494,6 +1511,10 @@ static void migrate_params_apply(MigrateSetParameter= s *params, Error **errp) if (params->has_mode) { s->parameters.mode =3D params->mode; } + + if (params->has_zero_page_detection) { + s->parameters.zero_page_detection =3D params->zero_page_detection; + } } =20 void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) diff --git a/migration/options.h b/migration/options.h index 246c160aee..b7c4fb3861 100644 --- a/migration/options.h +++ b/migration/options.h @@ -93,6 +93,7 @@ const char *migrate_tls_authz(void); const char *migrate_tls_creds(void); const char *migrate_tls_hostname(void); uint64_t migrate_xbzrle_cache_size(void); +ZeroPageDetection migrate_zero_page_detection(void); =20 /* parameters setters */ =20 diff --git a/migration/ram.c b/migration/ram.c index 4649a81204..67035fb4b1 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1123,6 +1123,10 @@ static int save_zero_page(RAMState *rs, PageSearchSt= atus *pss, QEMUFile *file =3D pss->pss_channel; int len =3D 0; =20 + if (migrate_zero_page_detection() =3D=3D ZERO_PAGE_DETECTION_NONE) { + return 0; + } + if (!buffer_is_zero(p, TARGET_PAGE_SIZE)) { return 0; } diff --git a/qapi/migration.json b/qapi/migration.json index 5a565d9b8d..1e66272f8f 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -653,6 +653,18 @@ { 'enum': 'MigMode', 'data': [ 'normal', 'cpr-reboot' ] } =20 +## +# @ZeroPageDetection: +# +# @none: Do not perform zero page checking. +# +# @legacy: Perform zero page checking from main migration thread. +# +# Since: 9.0 +## +{ 'enum': 'ZeroPageDetection', + 'data': [ 'none', 'legacy' ] } + ## # @BitmapMigrationBitmapAliasTransform: # @@ -874,6 +886,9 @@ # @mode: Migration mode. See description in @MigMode. Default is 'normal'. # (Since 8.2) # +# @zero-page-detection: Whether and how to detect zero pages. More details +# see description in @ZeroPageDetection. Default is 'legacy'. (since = 9.0) +# # Features: # # @deprecated: Member @block-incremental is deprecated. Use @@ -907,7 +922,8 @@ 'block-bitmap-mapping', { 'name': 'x-vcpu-dirty-limit-period', 'features': ['unstable']= }, 'vcpu-dirty-limit', - 'mode'] } + 'mode', + 'zero-page-detection'] } =20 ## # @MigrateSetParameters: @@ -1066,6 +1082,9 @@ # @mode: Migration mode. See description in @MigMode. Default is 'normal'. # (Since 8.2) # +# @zero-page-detection: Whether and how to detect zero pages. More details +# see description in @ZeroPageDetection. Default is 'legacy'. (since = 9.0) +# # Features: # # @deprecated: Member @block-incremental is deprecated. Use @@ -1119,7 +1138,8 @@ '*x-vcpu-dirty-limit-period': { 'type': 'uint64', 'features': [ 'unstable' ] }, '*vcpu-dirty-limit': 'uint64', - '*mode': 'MigMode'} } + '*mode': 'MigMode', + '*zero-page-detection': 'ZeroPageDetection'} } =20 ## # @migrate-set-parameters: @@ -1294,6 +1314,9 @@ # @mode: Migration mode. See description in @MigMode. Default is 'normal'. # (Since 8.2) # +# @zero-page-detection: Whether and how to detect zero pages. More details +# see description in @ZeroPageDetection. Default is 'legacy'. (since = 9.0) +# # Features: # # @deprecated: Member @block-incremental is deprecated. Use @@ -1344,7 +1367,8 @@ '*x-vcpu-dirty-limit-period': { 'type': 'uint64', 'features': [ 'unstable' ] }, '*vcpu-dirty-limit': 'uint64', - '*mode': 'MigMode'} } + '*mode': 'MigMode', + '*zero-page-detection': 'ZeroPageDetection'} } =20 ## # @query-migrate-parameters: --=20 2.30.2 From nobody Tue Nov 26 06:50:20 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1708977522; cv=none; d=zohomail.com; s=zohoarc; b=HyDANHjg9XbjeYC2Evxr/OOofg9c6dXet8mDjReRw/uW91s1rgdBnP7zzM1QZJJ+t631S0hcX4GI4rleD9QpEFhY8ZTJbsRJUZJqq+2YuokbNqRYTtbvaXM7Wl1c9zHf5XQsnKMDkwFSuAWM4Xcba0hqwSo+67ZbpRGRFplWDqo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708977522; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=eRAn6tam9k5Jq9Tx+2Gp3ggI2rPf83QN99x9AxDRzi4=; b=KctJgv+m5LEVthaq707lNfds7NkMYRcb/vT76HzAKE5LYjjZf1RRKgw+P5JKjRukpuYI8RT3+90GEHIa2jJgn+54mciL/tWaXpPLJhXsUZlyENnfVcn5FxT/t+b7T2qlMeOpxKMpz+lSgCud/Fx31N21pygHuNZPj0vfLv+JU4o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708977522721181.03680807713306; Mon, 26 Feb 2024 11:58:42 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1reh6v-00014l-5t; Mon, 26 Feb 2024 14:58:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1reh6p-0000zL-AU for qemu-devel@nongnu.org; Mon, 26 Feb 2024 14:57:55 -0500 Received: from mail-vk1-xa2b.google.com ([2607:f8b0:4864:20::a2b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1reh6m-0004tx-Mt for qemu-devel@nongnu.org; Mon, 26 Feb 2024 14:57:55 -0500 Received: by mail-vk1-xa2b.google.com with SMTP id 71dfb90a1353d-4ccce61a0e8so497961e0c.0 for ; Mon, 26 Feb 2024 11:57:51 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.95]) by smtp.gmail.com with ESMTPSA id ej5-20020a056122270500b004ca3dc45886sm712738vkb.47.2024.02.26.11.57.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 11:57:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1708977471; x=1709582271; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eRAn6tam9k5Jq9Tx+2Gp3ggI2rPf83QN99x9AxDRzi4=; b=TuFCSiomBHv/WZCF8ciTUXZylvMVO5v1+dqNUQKkML1itkzPOwPsJNh7nB78p27sQy WrdkjN6LfP02QZFdxucvencZQoUysul+FjCF466YZpaKcVdEEuNo6iaOp6MZOm2nL74F oN0TIqi4Vf9HhJsPJdC4YZyf98sL5tO5I4ynIvlVFnW5aRnO02OnRtdNhHDUL2YC6QDK i1bZr1UFf89RdT8Huu1e64WUBSU6EvzdiGYWcD7ydZSlN0zAeJD2uEdcS4bkpQDr+4// xS1R7GGTR2iqCFr1kUs3J/n5jND1JdMjruEC5UgvSLME0CFk+OQDSnZdJ9wnZVotLO0l VW6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708977471; x=1709582271; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eRAn6tam9k5Jq9Tx+2Gp3ggI2rPf83QN99x9AxDRzi4=; b=SPfHhicBauSLm5ZsSdry4U42rQZCzlI9rS6YawZluyXY7Mc1VIvVTqiVKnXwn72Dkq q2pSMbTyl3zwYnQlDNFcpPGGlTN8BvjTsd+oDmebK1EnYu9giCPC8nCb7lh+GYRszauR DtVzvde4h4fqMBlUOrH1w0frjz1QceYA6deE8EnzVTSZOtQc6vMnVY4pQ5qvD+gJhVOb c4Pfk0mFN9uwQsuWqlupS+8nE3AhcjVEbBNq+cb3gc0R8782zNNKX79CIqP4n1a4z9uS XaMNh7sA+rkMiTmqrAJUUmGHfxxQg/vnXwGdeLRgtjxz5dlx+QAa5HQBa7CDclyy4vSw 2uYQ== X-Forwarded-Encrypted: i=1; AJvYcCX2txjNV+IlzvPoKPXzWa56qYP/bSDtATeEG6yxY2dgOqaajQn2z/EsqUrkBbsbsyY8HNEBRDb0wk1pYvE+++Bi5thPbYg= X-Gm-Message-State: AOJu0YziLt4ualL62aLtrLyQuCMq0j+XiyRO1k2lwsE+Cs/fRA/RjXY9 +eQNjPpnMeA0LWiS9X5IWjSw+yyD7BOzcDVLEPeis+zTE+eeTF03umLNdMhnCf0= X-Google-Smtp-Source: AGHT+IES4+FaGf+Kn7MFfRyqybSQ+zgsoxucvX1fFqMIJjjG2Eo5tHK0Bj3XB8JhhZhMgOAoYyk9/g== X-Received: by 2002:a1f:c844:0:b0:4cb:56c5:5816 with SMTP id y65-20020a1fc844000000b004cb56c55816mr4394716vkf.12.1708977471009; Mon, 26 Feb 2024 11:57:51 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, jdenemar@redhat.com, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, qemu-devel@nongnu.org Cc: Hao Xiang Subject: [PATCH v3 2/7] migration/multifd: Implement zero page transmission on the multifd thread. Date: Mon, 26 Feb 2024 19:56:49 +0000 Message-Id: <20240226195654.934709-3-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240226195654.934709-1-hao.xiang@bytedance.com> References: <20240226195654.934709-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::a2b; envelope-from=hao.xiang@bytedance.com; helo=mail-vk1-xa2b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1708977523489100001 Content-Type: text/plain; charset="utf-8" 1. Add zero_pages field in MultiFDPacket_t. 2. Implements the zero page detection and handling on the multifd threads for non-compression, zlib and zstd compression backends. 3. Added a new value 'multifd' in ZeroPageDetection enumeration. 4. Handle migration QEMU9.0 -> QEMU8.2 compatibility. 5. Adds zero page counters and updates multifd send/receive tracing format to track the newly added counters. Signed-off-by: Hao Xiang --- hw/core/machine.c | 4 +- hw/core/qdev-properties-system.c | 2 +- migration/meson.build | 1 + migration/multifd-zero-page.c | 78 ++++++++++++++++++++++++++++++ migration/multifd-zlib.c | 21 ++++++-- migration/multifd-zstd.c | 20 ++++++-- migration/multifd.c | 83 +++++++++++++++++++++++++++----- migration/multifd.h | 24 ++++++++- migration/ram.c | 1 - migration/trace-events | 8 +-- qapi/migration.json | 5 +- 11 files changed, 214 insertions(+), 33 deletions(-) create mode 100644 migration/multifd-zero-page.c diff --git a/hw/core/machine.c b/hw/core/machine.c index fb5afdcae4..746da219a4 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -32,7 +32,9 @@ #include "hw/virtio/virtio-net.h" #include "audio/audio.h" =20 -GlobalProperty hw_compat_8_2[] =3D {}; +GlobalProperty hw_compat_8_2[] =3D { + { "migration", "zero-page-detection", "legacy"}, +}; const size_t hw_compat_8_2_len =3D G_N_ELEMENTS(hw_compat_8_2); =20 GlobalProperty hw_compat_8_1[] =3D { diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-sys= tem.c index 228e685f52..6e6f68ae1b 100644 --- a/hw/core/qdev-properties-system.c +++ b/hw/core/qdev-properties-system.c @@ -682,7 +682,7 @@ const PropertyInfo qdev_prop_mig_mode =3D { const PropertyInfo qdev_prop_zero_page_detection =3D { .name =3D "ZeroPageDetection", .description =3D "zero_page_detection values, " - "none,legacy", + "none,legacy,multifd", .enum_table =3D &ZeroPageDetection_lookup, .get =3D qdev_propinfo_get_enum, .set =3D qdev_propinfo_set_enum, diff --git a/migration/meson.build b/migration/meson.build index 92b1cc4297..1eeb915ff6 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -22,6 +22,7 @@ system_ss.add(files( 'migration.c', 'multifd.c', 'multifd-zlib.c', + 'multifd-zero-page.c', 'ram-compress.c', 'options.c', 'postcopy-ram.c', diff --git a/migration/multifd-zero-page.c b/migration/multifd-zero-page.c new file mode 100644 index 0000000000..1650c41b26 --- /dev/null +++ b/migration/multifd-zero-page.c @@ -0,0 +1,78 @@ +/* + * Multifd zero page detection implementation. + * + * Copyright (c) 2024 Bytedance Inc + * + * Authors: + * Hao Xiang + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qemu/cutils.h" +#include "exec/ramblock.h" +#include "migration.h" +#include "multifd.h" +#include "options.h" +#include "ram.h" + +static void swap_page_offset(ram_addr_t *pages_offset, int a, int b) +{ + ram_addr_t temp; + + if (a =3D=3D b) { + return; + } + + temp =3D pages_offset[a]; + pages_offset[a] =3D pages_offset[b]; + pages_offset[b] =3D temp; +} + +/** + * multifd_zero_page_check_send: Perform zero page detection on all pages. + * + * Sort the page offset array by moving all normal pages to + * the left and all zero pages to the right of the array. + * + * @param p A pointer to the send params. + */ +void multifd_zero_page_check_send(MultiFDSendParams *p) +{ + /* + * QEMU older than 9.0 don't understand zero page + * on multifd channel. This switch is required to + * maintain backward compatibility. + */ + bool use_multifd_zero_page =3D + (migrate_zero_page_detection() =3D=3D ZERO_PAGE_DETECTION_MULTIFD); + MultiFDPages_t *pages =3D p->pages; + RAMBlock *rb =3D pages->block; + int index_normal =3D 0; + int index_zero =3D pages->num - 1; + + while (index_normal <=3D index_zero) { + uint64_t offset =3D pages->offset[index_normal]; + if (use_multifd_zero_page && + buffer_is_zero(rb->host + offset, p->page_size)) { + swap_page_offset(pages->offset, index_normal, index_zero); + index_zero--; + ram_release_page(rb->idstr, offset); + } else { + index_normal++; + pages->normal_num++; + } + } +} + +void multifd_zero_page_check_recv(MultiFDRecvParams *p) +{ + for (int i =3D 0; i < p->zero_num; i++) { + void *page =3D p->host + p->zero[i]; + if (!buffer_is_zero(page, p->page_size)) { + memset(page, 0, p->page_size); + } + } +} diff --git a/migration/multifd-zlib.c b/migration/multifd-zlib.c index 012e3bdea1..a8b26bc5e4 100644 --- a/migration/multifd-zlib.c +++ b/migration/multifd-zlib.c @@ -123,13 +123,15 @@ static int zlib_send_prepare(MultiFDSendParams *p, Er= ror **errp) int ret; uint32_t i; =20 - multifd_send_prepare_header(p); + if (!multifd_send_prepare_common(p)) { + goto out; + } =20 - for (i =3D 0; i < pages->num; i++) { + for (i =3D 0; i < pages->normal_num; i++) { uint32_t available =3D z->zbuff_len - out_size; int flush =3D Z_NO_FLUSH; =20 - if (i =3D=3D pages->num - 1) { + if (i =3D=3D pages->normal_num - 1) { flush =3D Z_SYNC_FLUSH; } =20 @@ -172,10 +174,10 @@ static int zlib_send_prepare(MultiFDSendParams *p, Er= ror **errp) p->iov[p->iovs_num].iov_len =3D out_size; p->iovs_num++; p->next_packet_size =3D out_size; - p->flags |=3D MULTIFD_FLAG_ZLIB; =20 +out: + p->flags |=3D MULTIFD_FLAG_ZLIB; multifd_send_fill_packet(p); - return 0; } =20 @@ -261,6 +263,14 @@ static int zlib_recv_pages(MultiFDRecvParams *p, Error= **errp) p->id, flags, MULTIFD_FLAG_ZLIB); return -1; } + + multifd_zero_page_check_recv(p); + + if (!p->normal_num) { + assert(in_size =3D=3D 0); + return 0; + } + ret =3D qio_channel_read_all(p->c, (void *)z->zbuff, in_size, errp); =20 if (ret !=3D 0) { @@ -310,6 +320,7 @@ static int zlib_recv_pages(MultiFDRecvParams *p, Error = **errp) p->id, out_size, expected_size); return -1; } + return 0; } =20 diff --git a/migration/multifd-zstd.c b/migration/multifd-zstd.c index dc8fe43e94..7768040124 100644 --- a/migration/multifd-zstd.c +++ b/migration/multifd-zstd.c @@ -118,16 +118,18 @@ static int zstd_send_prepare(MultiFDSendParams *p, Er= ror **errp) int ret; uint32_t i; =20 - multifd_send_prepare_header(p); + if (!multifd_send_prepare_common(p)) { + goto out; + } =20 z->out.dst =3D z->zbuff; z->out.size =3D z->zbuff_len; z->out.pos =3D 0; =20 - for (i =3D 0; i < pages->num; i++) { + for (i =3D 0; i < pages->normal_num; i++) { ZSTD_EndDirective flush =3D ZSTD_e_continue; =20 - if (i =3D=3D pages->num - 1) { + if (i =3D=3D pages->normal_num - 1) { flush =3D ZSTD_e_flush; } z->in.src =3D p->pages->block->host + pages->offset[i]; @@ -161,10 +163,10 @@ static int zstd_send_prepare(MultiFDSendParams *p, Er= ror **errp) p->iov[p->iovs_num].iov_len =3D z->out.pos; p->iovs_num++; p->next_packet_size =3D z->out.pos; - p->flags |=3D MULTIFD_FLAG_ZSTD; =20 +out: + p->flags |=3D MULTIFD_FLAG_ZSTD; multifd_send_fill_packet(p); - return 0; } =20 @@ -257,6 +259,14 @@ static int zstd_recv_pages(MultiFDRecvParams *p, Error= **errp) p->id, flags, MULTIFD_FLAG_ZSTD); return -1; } + + multifd_zero_page_check_recv(p); + + if (!p->normal_num) { + assert(in_size =3D=3D 0); + return 0; + } + ret =3D qio_channel_read_all(p->c, (void *)z->zbuff, in_size, errp); =20 if (ret !=3D 0) { diff --git a/migration/multifd.c b/migration/multifd.c index adfe8c9a0a..eace0278af 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -11,6 +11,7 @@ */ =20 #include "qemu/osdep.h" +#include "qemu/cutils.h" #include "qemu/rcu.h" #include "exec/target_page.h" #include "sysemu/sysemu.h" @@ -126,6 +127,8 @@ static int nocomp_send_prepare(MultiFDSendParams *p, Er= ror **errp) MultiFDPages_t *pages =3D p->pages; int ret; =20 + multifd_zero_page_check_send(p); + if (!use_zero_copy_send) { /* * Only !zerocopy needs the header in IOV; zerocopy will @@ -134,13 +137,13 @@ static int nocomp_send_prepare(MultiFDSendParams *p, = Error **errp) multifd_send_prepare_header(p); } =20 - for (int i =3D 0; i < pages->num; i++) { + for (int i =3D 0; i < pages->normal_num; i++) { p->iov[p->iovs_num].iov_base =3D pages->block->host + pages->offse= t[i]; p->iov[p->iovs_num].iov_len =3D p->page_size; p->iovs_num++; } =20 - p->next_packet_size =3D pages->num * p->page_size; + p->next_packet_size =3D pages->normal_num * p->page_size; p->flags |=3D MULTIFD_FLAG_NOCOMP; =20 multifd_send_fill_packet(p); @@ -202,6 +205,13 @@ static int nocomp_recv_pages(MultiFDRecvParams *p, Err= or **errp) p->id, flags, MULTIFD_FLAG_NOCOMP); return -1; } + + multifd_zero_page_check_recv(p); + + if (!p->normal_num) { + return 0; + } + for (int i =3D 0; i < p->normal_num; i++) { p->iov[i].iov_base =3D p->host + p->normal[i]; p->iov[i].iov_len =3D p->page_size; @@ -236,6 +246,7 @@ static void multifd_pages_reset(MultiFDPages_t *pages) * overwritten later when reused. */ pages->num =3D 0; + pages->normal_num =3D 0; pages->block =3D NULL; } =20 @@ -327,11 +338,13 @@ void multifd_send_fill_packet(MultiFDSendParams *p) MultiFDPacket_t *packet =3D p->packet; MultiFDPages_t *pages =3D p->pages; uint64_t packet_num; + uint32_t zero_num =3D pages->num - pages->normal_num; int i; =20 packet->flags =3D cpu_to_be32(p->flags); packet->pages_alloc =3D cpu_to_be32(p->pages->allocated); - packet->normal_pages =3D cpu_to_be32(pages->num); + packet->normal_pages =3D cpu_to_be32(pages->normal_num); + packet->zero_pages =3D cpu_to_be32(zero_num); packet->next_packet_size =3D cpu_to_be32(p->next_packet_size); =20 packet_num =3D qatomic_fetch_inc(&multifd_send_state->packet_num); @@ -349,10 +362,11 @@ void multifd_send_fill_packet(MultiFDSendParams *p) } =20 p->packets_sent++; - p->total_normal_pages +=3D pages->num; + p->total_normal_pages +=3D pages->normal_num; + p->total_zero_pages +=3D zero_num; =20 - trace_multifd_send(p->id, packet_num, pages->num, p->flags, - p->next_packet_size); + trace_multifd_send(p->id, packet_num, pages->normal_num, zero_num, + p->flags, p->next_packet_size); } =20 static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) @@ -393,20 +407,29 @@ static int multifd_recv_unfill_packet(MultiFDRecvPara= ms *p, Error **errp) p->normal_num =3D be32_to_cpu(packet->normal_pages); if (p->normal_num > packet->pages_alloc) { error_setg(errp, "multifd: received packet " - "with %u pages and expected maximum pages are %u", + "with %u normal pages and expected maximum pages are %u= ", p->normal_num, packet->pages_alloc) ; return -1; } =20 + p->zero_num =3D be32_to_cpu(packet->zero_pages); + if (p->zero_num > packet->pages_alloc - p->normal_num) { + error_setg(errp, "multifd: received packet " + "with %u zero pages and expected maximum zero pages are= %u", + p->zero_num, packet->pages_alloc - p->normal_num) ; + return -1; + } + p->next_packet_size =3D be32_to_cpu(packet->next_packet_size); p->packet_num =3D be64_to_cpu(packet->packet_num); p->packets_recved++; p->total_normal_pages +=3D p->normal_num; + p->total_zero_pages +=3D p->zero_num; =20 - trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->flags, - p->next_packet_size); + trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->zero_num, + p->flags, p->next_packet_size); =20 - if (p->normal_num =3D=3D 0) { + if (p->normal_num =3D=3D 0 && p->zero_num =3D=3D 0) { return 0; } =20 @@ -432,6 +455,18 @@ static int multifd_recv_unfill_packet(MultiFDRecvParam= s *p, Error **errp) p->normal[i] =3D offset; } =20 + for (i =3D 0; i < p->zero_num; i++) { + uint64_t offset =3D be64_to_cpu(packet->offset[p->normal_num + i]); + + if (offset > (p->block->used_length - p->page_size)) { + error_setg(errp, "multifd: offset too long %" PRIu64 + " (max " RAM_ADDR_FMT ")", + offset, p->block->used_length); + return -1; + } + p->zero[i] =3D offset; + } + return 0; } =20 @@ -823,6 +858,8 @@ static void *multifd_send_thread(void *opaque) =20 stat64_add(&mig_stats.multifd_bytes, p->next_packet_size + p->packet_len); + stat64_add(&mig_stats.normal_pages, pages->normal_num); + stat64_add(&mig_stats.zero_pages, pages->num - pages->normal_n= um); =20 multifd_pages_reset(p->pages); p->next_packet_size =3D 0; @@ -866,7 +903,8 @@ out: =20 rcu_unregister_thread(); migration_threads_remove(thread); - trace_multifd_send_thread_end(p->id, p->packets_sent, p->total_normal_= pages); + trace_multifd_send_thread_end(p->id, p->packets_sent, p->total_normal_= pages, + p->total_zero_pages); =20 return NULL; } @@ -1132,6 +1170,8 @@ static void multifd_recv_cleanup_channel(MultiFDRecvP= arams *p) p->iov =3D NULL; g_free(p->normal); p->normal =3D NULL; + g_free(p->zero); + p->zero =3D NULL; multifd_recv_state->ops->recv_cleanup(p); } =20 @@ -1232,7 +1272,7 @@ static void *multifd_recv_thread(void *opaque) p->flags &=3D ~MULTIFD_FLAG_SYNC; qemu_mutex_unlock(&p->mutex); =20 - if (p->normal_num) { + if (p->normal_num + p->zero_num) { ret =3D multifd_recv_state->ops->recv_pages(p, &local_err); if (ret !=3D 0) { break; @@ -1251,7 +1291,9 @@ static void *multifd_recv_thread(void *opaque) } =20 rcu_unregister_thread(); - trace_multifd_recv_thread_end(p->id, p->packets_recved, p->total_norma= l_pages); + trace_multifd_recv_thread_end(p->id, p->packets_recved, + p->total_normal_pages, + p->total_zero_pages); =20 return NULL; } @@ -1290,6 +1332,7 @@ int multifd_recv_setup(Error **errp) p->name =3D g_strdup_printf("multifdrecv_%d", i); p->iov =3D g_new0(struct iovec, page_count); p->normal =3D g_new0(ram_addr_t, page_count); + p->zero =3D g_new0(ram_addr_t, page_count); p->page_count =3D page_count; p->page_size =3D qemu_target_page_size(); } @@ -1359,3 +1402,17 @@ void multifd_recv_new_channel(QIOChannel *ioc, Error= **errp) QEMU_THREAD_JOINABLE); qatomic_inc(&multifd_recv_state->count); } + +bool multifd_send_prepare_common(MultiFDSendParams *p) +{ + multifd_zero_page_check_send(p); + + if (!p->pages->normal_num) { + p->next_packet_size =3D 0; + return false; + } + + multifd_send_prepare_header(p); + + return true; +} diff --git a/migration/multifd.h b/migration/multifd.h index 8a1cad0996..e18ffdf7cc 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -48,14 +48,24 @@ typedef struct { /* size of the next packet that contains pages */ uint32_t next_packet_size; uint64_t packet_num; - uint64_t unused[4]; /* Reserved for future use */ + /* zero pages */ + uint32_t zero_pages; + uint32_t unused32[1]; /* Reserved for future use */ + uint64_t unused64[3]; /* Reserved for future use */ char ramblock[256]; + /* + * This array contains the pointers to: + * - normal pages (initial normal_pages entries) + * - zero pages (following zero_pages entries) + */ uint64_t offset[]; } __attribute__((packed)) MultiFDPacket_t; =20 typedef struct { /* number of used pages */ uint32_t num; + /* number of normal pages */ + uint32_t normal_num; /* number of allocated pages */ uint32_t allocated; /* offset of each page */ @@ -124,6 +134,8 @@ typedef struct { uint64_t packets_sent; /* non zero pages sent through this channel */ uint64_t total_normal_pages; + /* zero pages sent through this channel */ + uint64_t total_zero_pages; /* buffers to send */ struct iovec *iov; /* number of iovs used */ @@ -178,12 +190,18 @@ typedef struct { uint8_t *host; /* non zero pages recv through this channel */ uint64_t total_normal_pages; + /* zero pages recv through this channel */ + uint64_t total_zero_pages; /* buffers to recv */ struct iovec *iov; /* Pages that are not zero */ ram_addr_t *normal; /* num of non zero pages */ uint32_t normal_num; + /* Pages that are zero */ + ram_addr_t *zero; + /* num of zero pages */ + uint32_t zero_num; /* used for de-compression methods */ void *data; } MultiFDRecvParams; @@ -205,6 +223,9 @@ typedef struct { =20 void multifd_register_ops(int method, MultiFDMethods *ops); void multifd_send_fill_packet(MultiFDSendParams *p); +bool multifd_send_prepare_common(MultiFDSendParams *p); +void multifd_zero_page_check_send(MultiFDSendParams *p); +void multifd_zero_page_check_recv(MultiFDRecvParams *p); =20 static inline void multifd_send_prepare_header(MultiFDSendParams *p) { @@ -213,5 +234,4 @@ static inline void multifd_send_prepare_header(MultiFDS= endParams *p) p->iovs_num++; } =20 - #endif diff --git a/migration/ram.c b/migration/ram.c index 67035fb4b1..414cd0d753 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1259,7 +1259,6 @@ static int ram_save_multifd_page(RAMBlock *block, ram= _addr_t offset) if (!multifd_queue_page(block, offset)) { return -1; } - stat64_add(&mig_stats.normal_pages, 1); =20 return 1; } diff --git a/migration/trace-events b/migration/trace-events index 298ad2b0dd..9f1d7ae71a 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -128,21 +128,21 @@ postcopy_preempt_reset_channel(void) "" # multifd.c multifd_new_send_channel_async(uint8_t id) "channel %u" multifd_new_send_channel_async_error(uint8_t id, void *err) "channel=3D%u = err=3D%p" -multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flag= s, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " pages %u f= lags 0x%x next packet size %u" +multifd_recv(uint8_t id, uint64_t packet_num, uint32_t normal, uint32_t ze= ro, uint32_t flags, uint32_t next_packet_size) "channel %u packet_num %" PR= Iu64 " normal pages %u zero pages %u flags 0x%x next packet size %u" multifd_recv_new_channel(uint8_t id) "channel %u" multifd_recv_sync_main(long packet_num) "packet num %ld" multifd_recv_sync_main_signal(uint8_t id) "channel %u" multifd_recv_sync_main_wait(uint8_t id) "channel %u" multifd_recv_terminate_threads(bool error) "error %d" -multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "cha= nnel %u packets %" PRIu64 " pages %" PRIu64 +multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal pages %" PRI= u64 " zero pages %" PRIu64 multifd_recv_thread_start(uint8_t id) "%u" -multifd_send(uint8_t id, uint64_t packet_num, uint32_t normal, uint32_t fl= ags, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " normal p= ages %u flags 0x%x next packet size %u" +multifd_send(uint8_t id, uint64_t packet_num, uint32_t normal_pages, uint3= 2_t zero_pages, uint32_t flags, uint32_t next_packet_size) "channel %u pack= et_num %" PRIu64 " normal pages %u zero pages %u flags 0x%x next packet siz= e %u" multifd_send_error(uint8_t id) "channel %u" multifd_send_sync_main(long packet_num) "packet num %ld" multifd_send_sync_main_signal(uint8_t id) "channel %u" multifd_send_sync_main_wait(uint8_t id) "channel %u" multifd_send_terminate_threads(void) "" -multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s) "channel %u packets %" PRIu64 " normal pages %" PRIu64 +multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal pages %" PR= Iu64 " zero pages %" PRIu64 multifd_send_thread_start(uint8_t id) "%u" multifd_tls_outgoing_handshake_start(void *ioc, void *tioc, const char *ho= stname) "ioc=3D%p tioc=3D%p hostname=3D%s" multifd_tls_outgoing_handshake_error(void *ioc, const char *err) "ioc=3D%p= err=3D%s" diff --git a/qapi/migration.json b/qapi/migration.json index 1e66272f8f..5a1bb8ad62 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -660,10 +660,13 @@ # # @legacy: Perform zero page checking from main migration thread. # +# @multifd: Perform zero page checking from multifd sender thread. +# # Since: 9.0 +# ## { 'enum': 'ZeroPageDetection', - 'data': [ 'none', 'legacy' ] } + 'data': [ 'none', 'legacy', 'multifd' ] } =20 ## # @BitmapMigrationBitmapAliasTransform: --=20 2.30.2 From nobody Tue Nov 26 06:50:20 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1708977538; cv=none; d=zohomail.com; s=zohoarc; b=NwAnf4kyjYJ4yOIvCsTrKfSvL51bx43cx1eJyagIfxJQcAk3JyAj4DKeaGSwCHDsj7N0/477huMIbxgv4fmhgxw9K32fRFe8M9faZ0Scjkb8K9mHgdWx7rLRPuHit3RWh4opncX3mCTtqfMzTmsPMQ/yOmUXSZt/zEFMsyPAxTg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708977538; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=kj4WrPMcdHH42vMb8MjOr0OwgYlZTTIRTtBw9DfaH6Y=; b=GKEDq7xu4VjzBfQKp5STB1oMBuYmA1lf7xnSJw+3FvGdj6/zzsstADzJURbyvPJxdMqU8RQKvLck2b1A0ZVwKYVd09CLpvdeVZR8u3nqePl7jtPqvNP1thh9LZHiPvuAGXBSGxMg6MXVFjnEcAor/Z46GidPG8VJI2TAlvbfBI8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708977538410315.6752619747019; Mon, 26 Feb 2024 11:58:58 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1reh6v-000145-SZ; Mon, 26 Feb 2024 14:58:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1reh6p-0000zM-Ke for qemu-devel@nongnu.org; Mon, 26 Feb 2024 14:57:55 -0500 Received: from mail-vk1-xa31.google.com ([2607:f8b0:4864:20::a31]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1reh6o-0004uH-1V for qemu-devel@nongnu.org; Mon, 26 Feb 2024 14:57:55 -0500 Received: by mail-vk1-xa31.google.com with SMTP id 71dfb90a1353d-4c78b074fe0so661235e0c.3 for ; Mon, 26 Feb 2024 11:57:53 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.95]) by smtp.gmail.com with ESMTPSA id ej5-20020a056122270500b004ca3dc45886sm712738vkb.47.2024.02.26.11.57.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 11:57:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1708977473; x=1709582273; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kj4WrPMcdHH42vMb8MjOr0OwgYlZTTIRTtBw9DfaH6Y=; b=jHVShwJ//PFpnFsa7/dukPCV31olDZI1J3Ca44zY3SJeiP+Z65I61JjzVOkU6YH9ig 9lizHAPyKnDAtrn4KiHI6I6lphATbr6mKD5rEWmzsk9vQwr2dHtzgqlrfd2wVS3Z1mCo zMqp9t9TNoFiswHS2H3AyHPypJEGyqdzlqEARUIbged+ITcK8ldCbEvBE5fwf1Y3gb3o K49wOcHnPWP1LmPn1B0xdJuyfIrrVy14HWk4ht1h2pFjEXq26XJfLunlwmIl88S4VuF5 gz1x9pru91zLStBFPRd+4oxMCpb1gh0Ix+GuzxXJj+UkiYRHJo4SpJzYxJBbutPDgVUB Z5+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708977473; x=1709582273; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kj4WrPMcdHH42vMb8MjOr0OwgYlZTTIRTtBw9DfaH6Y=; b=W9nD6a4evsEEFStgdTS8BR8WTKmrJvena+WeTR0FXQNjEtpG2O4XwnbB0TIfwPK2i+ eMQJkqRJswIUc659BLsVzfeUV7rXSGo7WxY3Br7YUXihjAnKGfQTUT3kcCgfog4wCj11 yilYMIa/ZpWkmJlp84oSorJ4LVZZutGz+xthQpGHpMqtC2CYRwavwNmVYp0JohcYYfeL BayiIDXc+IDmBq3GZcJ4OUlu5G5kjZA+Njs/F+tYoCzA/RrsdljtvN6GK4O6n1ukxfNk lPm8mJX8jA4/TY/+Nm25966Tk3uuRW7Bb+hzWi7GJBD3lRBVKdT4ixVpxR+NTP72ivAS WH7Q== X-Forwarded-Encrypted: i=1; AJvYcCXiRTdjD7h90AShRCYeYn+d6VFyRZolAwKXIh91kXmlORe8TPwaqgrF1LTI5LuFFG4nQPtsopozw8swLg7D/8998ZfOXS4= X-Gm-Message-State: AOJu0YybzHbqjJRkPKq+plnPuEcTsFaY3bL4HFdkFFAfndyXWtMTB2or ascknQKyA7lv/16jekk2huuqSlVokG9CaD9RW8HEsHPrwrrPA2uztbTblV0osmY= X-Google-Smtp-Source: AGHT+IHs5SZT+dUlpowdSkaj3k9ZtOTzNMnYhg42kJfwVzviTdj/xLLDjYoHfQcQFZ3nfZlQInIvRQ== X-Received: by 2002:a1f:c706:0:b0:4cd:20ea:35aa with SMTP id x6-20020a1fc706000000b004cd20ea35aamr4316988vkf.8.1708977473118; Mon, 26 Feb 2024 11:57:53 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, jdenemar@redhat.com, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, qemu-devel@nongnu.org Cc: Hao Xiang Subject: [PATCH v3 3/7] migration/multifd: Implement ram_save_target_page_multifd to handle multifd version of MigrationOps::ram_save_target_page. Date: Mon, 26 Feb 2024 19:56:50 +0000 Message-Id: <20240226195654.934709-4-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240226195654.934709-1-hao.xiang@bytedance.com> References: <20240226195654.934709-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::a31; envelope-from=hao.xiang@bytedance.com; helo=mail-vk1-xa31.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1708977539520100006 Content-Type: text/plain; charset="utf-8" 1. Add a dedicated handler for MigrationOps::ram_save_target_page in multifd live migration. 2. Refactor ram_save_target_page_legacy so that the legacy and multifd handlers don't have internal functions calling into each other. Signed-off-by: Hao Xiang Reviewed-by: Fabiano Rosas --- migration/ram.c | 43 ++++++++++++++++++++++++++++++------------- 1 file changed, 30 insertions(+), 13 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 414cd0d753..f60627e11a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1123,10 +1123,6 @@ static int save_zero_page(RAMState *rs, PageSearchSt= atus *pss, QEMUFile *file =3D pss->pss_channel; int len =3D 0; =20 - if (migrate_zero_page_detection() =3D=3D ZERO_PAGE_DETECTION_NONE) { - return 0; - } - if (!buffer_is_zero(p, TARGET_PAGE_SIZE)) { return 0; } @@ -2046,7 +2042,6 @@ static bool save_compress_page(RAMState *rs, PageSear= chStatus *pss, */ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss) { - RAMBlock *block =3D pss->block; ram_addr_t offset =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; int res; =20 @@ -2062,17 +2057,34 @@ static int ram_save_target_page_legacy(RAMState *rs= , PageSearchStatus *pss) return 1; } =20 + return ram_save_page(rs, pss); +} + +/** + * ram_save_target_page_multifd: send one target page to multifd workers + * + * Returns 1 if the page was queued, -1 otherwise. + * + * @rs: current RAM state + * @pss: data about the page we want to send + */ +static int ram_save_target_page_multifd(RAMState *rs, PageSearchStatus *ps= s) +{ + RAMBlock *block =3D pss->block; + ram_addr_t offset =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; + /* - * Do not use multifd in postcopy as one whole host page should be - * placed. Meanwhile postcopy requires atomic update of pages, so even - * if host page size =3D=3D guest page size the dest guest during run = may - * still see partially copied pages which is data corruption. + * Backward compatibility support. While using multifd live + * migration, we still need to handle zero page checking on the + * migration main thread. */ - if (migrate_multifd() && !migration_in_postcopy()) { - return ram_save_multifd_page(block, offset); + if (migrate_zero_page_detection() =3D=3D ZERO_PAGE_DETECTION_LEGACY) { + if (save_zero_page(rs, pss, offset)) { + return 1; + } } =20 - return ram_save_page(rs, pss); + return ram_save_multifd_page(block, offset); } =20 /* Should be called before sending a host page */ @@ -2984,7 +2996,12 @@ static int ram_save_setup(QEMUFile *f, void *opaque) } =20 migration_ops =3D g_malloc0(sizeof(MigrationOps)); - migration_ops->ram_save_target_page =3D ram_save_target_page_legacy; + + if (migrate_multifd()) { + migration_ops->ram_save_target_page =3D ram_save_target_page_multi= fd; + } else { + migration_ops->ram_save_target_page =3D ram_save_target_page_legac= y; + } =20 bql_unlock(); ret =3D multifd_send_sync_main(); --=20 2.30.2 From nobody Tue Nov 26 06:50:20 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1708977577; cv=none; d=zohomail.com; s=zohoarc; b=Eq13sg27D2whSuLebZCJf8Wsu/PIcimaxL+bJn5fc1sNZhl23lMBKFZq4D2kF00plEJ8L4nHaLgtFxK+kWENRdVBPecoyz6Yl+fPm5grFBXZjKdXUIvfdP+BHY1INbZ8CSvQ6gs8LZmxuvxSn74lNZBtjvldscDSSHPeM1jD5HU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708977577; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=VUMxmmc1oEZ/KucWH0CWtubboX9PrDgWV4N0uAYEoGg=; b=ao8v/TG1E4UgMQ6BlyrKlIJK7DnGnVjrNyO57X/PnZIbfp0XFzn0RYaDmYcOQBw4rWX9wqLAVcEqh1AbK7TfSp6etu70seBGOdAQw74mDB9B8EY0aw3XFAac2TFLXtTrEHt06dEBYfT2yewj9GAXPe6e5taYAGqwPRZS/RgYY6Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708977577513970.4223169403372; Mon, 26 Feb 2024 11:59:37 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1reh6w-00017b-N9; Mon, 26 Feb 2024 14:58:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1reh6r-000108-PS for qemu-devel@nongnu.org; Mon, 26 Feb 2024 14:57:59 -0500 Received: from mail-ua1-x92f.google.com ([2607:f8b0:4864:20::92f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1reh6q-0004ud-6x for qemu-devel@nongnu.org; Mon, 26 Feb 2024 14:57:57 -0500 Received: by mail-ua1-x92f.google.com with SMTP id a1e0cc1a2514c-7d5bbbe592dso1778231241.0 for ; Mon, 26 Feb 2024 11:57:55 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.95]) by smtp.gmail.com with ESMTPSA id ej5-20020a056122270500b004ca3dc45886sm712738vkb.47.2024.02.26.11.57.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 11:57:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1708977475; x=1709582275; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VUMxmmc1oEZ/KucWH0CWtubboX9PrDgWV4N0uAYEoGg=; b=HVtSYtchGyq4O6gUwgITjQkcH+ReblbqRiYhTuc0o20lDL6bKcIgA5mnSuJusMptTt WynHrRH8pq7abWHCiCCNM+tW+TaDaPTvvq7Zpf9KsAxDnHXjGpkd3ApXw9QcAbSCTJDK C+CrpEodkI6bmEOLzsEGj/BGssKwn7qYKMXIy0WHabt2bXWKb+MwtJlv5MUPubxmv5G5 iDH/yruO6dscVBLf2BXxrS32SoYiO0Qg+u3xuMPPBasNoInxUPWcJVXCoSC8xR5DVf8j rqz6EJZoeBx1wMtdx4MtsMAKgfopIrRYwQArmmr29h9BquOzCEyEVlP+fHaUG3k3wPbd 4gbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708977475; x=1709582275; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VUMxmmc1oEZ/KucWH0CWtubboX9PrDgWV4N0uAYEoGg=; b=eGWzsA6Vmw43TKqnvSriyxyxaTYZ9zH+0+kIfgFdpq8BmyLx4jrWZ+qXDPvJX/qAXo Ekt73mphJvZcZYoQpTbT0PwPdy8uhuevxbFDOKhnFyOsVLyZK/ZlIjYE5qOlII3Nryfs /isDMHRof1tnUh0IWi+Q5wvnLOyjpjTtrRLfP8WdwhCKz8eeOoPP2aOPggeVtHTDOGuR UzohsQstbge04nx6kX9b/gtL0+AfaKOVnWJ2VUoBP7tIqINCcs0a042AKnAZ6Dqr0IO6 QS5KbH+l6cpzpMv4E+sxv+r6bMkt/17k4FRAy35c4jiVnNBSVnj3lPPyApuzZhLzeSC/ OFLQ== X-Forwarded-Encrypted: i=1; AJvYcCWo6TZ0blxq3IoiOl735Cy98X2YD5xQ3aRerMzoj1wokup7PA3rQxZwkiBZz14en4ECOy+GxbjbtUdWyWsBPdDkB/2jZ/I= X-Gm-Message-State: AOJu0Yz2UeM1R5JnQbFMM9+PL/ukV4NAegXf8cxAITpc4KU+5AcWnUum Mrncj1dSOGPVRFRmyRaARtXe0hnOlettBbNh8CHfZ+A9yOQ/b3lHzlrL3dTtwm8= X-Google-Smtp-Source: AGHT+IH+8rm2SL6IP+KLTyud81Z8gEdlCxyY3qFbwCoO7e7ZB83fzdGVEXhgg9WFroVvvt2tNDolfg== X-Received: by 2002:a1f:e701:0:b0:4c9:c252:6afb with SMTP id e1-20020a1fe701000000b004c9c2526afbmr4506813vkh.10.1708977474987; Mon, 26 Feb 2024 11:57:54 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, jdenemar@redhat.com, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, qemu-devel@nongnu.org Cc: Hao Xiang Subject: [PATCH v3 4/7] migration/multifd: Enable multifd zero page checking by default. Date: Mon, 26 Feb 2024 19:56:51 +0000 Message-Id: <20240226195654.934709-5-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240226195654.934709-1-hao.xiang@bytedance.com> References: <20240226195654.934709-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::92f; envelope-from=hao.xiang@bytedance.com; helo=mail-ua1-x92f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1708977579609100001 Content-Type: text/plain; charset="utf-8" Set default "zero-page-detection" option to "multifd". Now zero page checking can be done in the multifd threads and this becomes the default configuration. We still provide backward compatibility where zero page checking is done from the migration main thread. Signed-off-by: Hao Xiang --- migration/options.c | 2 +- qapi/migration.json | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/migration/options.c b/migration/options.c index 3c603391b0..3c79b6ccd4 100644 --- a/migration/options.c +++ b/migration/options.c @@ -181,7 +181,7 @@ Property migration_properties[] =3D { MIG_MODE_NORMAL), DEFINE_PROP_ZERO_PAGE_DETECTION("zero-page-detection", MigrationState, parameters.zero_page_detection, - ZERO_PAGE_DETECTION_LEGACY), + ZERO_PAGE_DETECTION_MULTIFD), =20 /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), diff --git a/qapi/migration.json b/qapi/migration.json index 5a1bb8ad62..a0a85a0312 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -890,7 +890,7 @@ # (Since 8.2) # # @zero-page-detection: Whether and how to detect zero pages. More details -# see description in @ZeroPageDetection. Default is 'legacy'. (since = 9.0) +# see description in @ZeroPageDetection. Default is 'multifd'. (since= 9.0) # # Features: # @@ -1086,7 +1086,7 @@ # (Since 8.2) # # @zero-page-detection: Whether and how to detect zero pages. More details -# see description in @ZeroPageDetection. Default is 'legacy'. (since = 9.0) +# see description in @ZeroPageDetection. Default is 'multifd'. (since= 9.0) # # Features: # @@ -1318,7 +1318,7 @@ # (Since 8.2) # # @zero-page-detection: Whether and how to detect zero pages. More details -# see description in @ZeroPageDetection. Default is 'legacy'. (since = 9.0) +# see description in @ZeroPageDetection. Default is 'multifd'. (since= 9.0) # # Features: # --=20 2.30.2 From nobody Tue Nov 26 06:50:20 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1708977571; cv=none; d=zohomail.com; s=zohoarc; b=WgU8Qgg5iWxZMeC0YvAsI6R1LoSpd4jfy+Om/MJOEuLnBohXhAHrQ1sVK6LAlImiPXRQIp/djtPopMzwQ6DYa15QEiHLTezzmfuGvel5/YwESWSrp7AGNjVFpsreI0n3OJ/n+lTKrSnxsmJv1Xd6nMB7I31iCWWHasAeV9eMSfI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708977571; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=hq1QawsT8MllTo+xYm9OfKbj2az/Aa/YOrPNVEv9ZhE=; b=kzHwejt4jq3fT2f3TCtrIQFhuCajBjmkx9AZL9DGEBI5w3b1gvEvCfZ3+dciKJybApFBMlLh5NsV43/0tSkDh+dWBOybMQzSUwSaEEBuaEatwDEvYoDl38pVxg6HOgpD+TyRno1FaHXrPjvhgE4O7thhcLgDMdi4/v1Lsun5KIc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708977571749400.43857930089825; Mon, 26 Feb 2024 11:59:31 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1reh75-0001Jf-Jr; Mon, 26 Feb 2024 14:58:11 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1reh74-0001J9-VY for qemu-devel@nongnu.org; Mon, 26 Feb 2024 14:58:10 -0500 Received: from mail-ua1-x930.google.com ([2607:f8b0:4864:20::930]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1reh6r-0004ur-Nc for qemu-devel@nongnu.org; Mon, 26 Feb 2024 14:58:10 -0500 Received: by mail-ua1-x930.google.com with SMTP id a1e0cc1a2514c-7d5bbbe592dso1778250241.0 for ; Mon, 26 Feb 2024 11:57:57 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.95]) by smtp.gmail.com with ESMTPSA id ej5-20020a056122270500b004ca3dc45886sm712738vkb.47.2024.02.26.11.57.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 11:57:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1708977477; x=1709582277; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hq1QawsT8MllTo+xYm9OfKbj2az/Aa/YOrPNVEv9ZhE=; b=RBAUA095LBaDjfyIwaS7KyXQGvtrZLmHb2F5ZKnbGt1qUOdwwIumJao6JWiOIDX1I1 J/sMmZB8x4PgFGonLWE+j0pRku3Hp+yGkwXsNIWELfN/jkp1ZCt4VgiMzIXZ/+KtmrEh H5cjAMG/bke6GqaUFDbPVrI4F+fcX6rfHgOhXZbKXyXpwDsJUydnBSxZn30LXdUt+BZF yUX2GnLtXt25I+KnLG6jnXmaHJK7IZNdWP+7FYoR6mr6ER7ZjAIKrcsKZu2IYP5fvUO7 lYsdEtiOa6hDxG8o/6gsKKirpdmsLCpabP7g+HSy3y2ijClI6x0IwVC57c0cUFrPCl2t LHMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708977477; x=1709582277; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hq1QawsT8MllTo+xYm9OfKbj2az/Aa/YOrPNVEv9ZhE=; b=E0lIpGLa0EPZAfRgV71wVV2R+TNQmFPPpH4DAiGGIqx9P3syDxhcO62FKjxq5UPoKv ANF3tTafqZpbq75tIlHfyPnUjN3pWUMQzJasGCOM6KDPrlXOrby27cQYIvy36ImSMIOh i/nHbzjxz33Jbd6lgco7ECO0GBllqera8MVRDvIXPjeB8Iz1LhBkmVyTEq6s1n1vROJj rHi2PgYCsNN+VlvGiq2U4r/wQdoGFTAbZz1sSFyymTUici6XGvf6deLmH0Q+vRAQcgi6 3CN4eFhCL7Oh6vdNkSAq7bvIfObqkZZXWNM4zh33RF1cwZPIsHJpC8gKUE9LomL7wb33 MRRw== X-Forwarded-Encrypted: i=1; AJvYcCWBor/661B7NXlGhyB2O3hSrjFmZ4oT7O6FgYfD4juhSkx4fh1Y3V2/eosmhgmJ+RACZ3RcnnafXo8niNih4hIx5T13Lic= X-Gm-Message-State: AOJu0YzYwnVPGzzd39mNcKNw99jXtRrl5SL125zpD9xuZDwBdN0/szrB JMH2dwMPuqA8Y/KxXQ+DYsYMmMY08wU5h/2rLrpVyISqsFzIKhEB5S7Ag0Q1UFM= X-Google-Smtp-Source: AGHT+IHwXlE6BMlZKOQaMqVrymYi8lbN7eUUWEWsl89k9I0iGew2eAnv5xZ4bqamtlTUD3A/ev728w== X-Received: by 2002:a1f:e701:0:b0:4c9:c252:6afb with SMTP id e1-20020a1fe701000000b004c9c2526afbmr4506850vkh.10.1708977476783; Mon, 26 Feb 2024 11:57:56 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, jdenemar@redhat.com, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, qemu-devel@nongnu.org Cc: Hao Xiang Subject: [PATCH v3 5/7] migration/multifd: Add new migration test cases for legacy zero page checking. Date: Mon, 26 Feb 2024 19:56:52 +0000 Message-Id: <20240226195654.934709-6-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240226195654.934709-1-hao.xiang@bytedance.com> References: <20240226195654.934709-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::930; envelope-from=hao.xiang@bytedance.com; helo=mail-ua1-x930.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1708977573595100001 Content-Type: text/plain; charset="utf-8" Now that zero page checking is done on the multifd sender threads by default, we still provide an option for backward compatibility. This change adds a qtest migration test case to set the zero-page-detection option to "legacy" and run multifd migration with zero page checking on the migration main thread. Signed-off-by: Hao Xiang --- tests/qtest/migration-test.c | 52 ++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 8a5bb1752e..65b531d871 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -2621,6 +2621,24 @@ test_migrate_precopy_tcp_multifd_start(QTestState *f= rom, return test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); } =20 +static void * +test_migrate_precopy_tcp_multifd_start_zero_page_legacy(QTestState *from, + QTestState *to) +{ + test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); + migrate_set_parameter_str(from, "zero-page-detection", "legacy"); + return NULL; +} + +static void * +test_migration_precopy_tcp_multifd_start_no_zero_page(QTestState *from, + QTestState *to) +{ + test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); + migrate_set_parameter_str(from, "zero-page-detection", "none"); + return NULL; +} + static void * test_migrate_precopy_tcp_multifd_zlib_start(QTestState *from, QTestState *to) @@ -2652,6 +2670,36 @@ static void test_multifd_tcp_none(void) test_precopy_common(&args); } =20 +static void test_multifd_tcp_zero_page_legacy(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "defer", + .start_hook =3D test_migrate_precopy_tcp_multifd_start_zero_page_l= egacy, + /* + * Multifd is more complicated than most of the features, it + * directly takes guest page buffers when sending, make sure + * everything will work alright even if guest page is changing. + */ + .live =3D true, + }; + test_precopy_common(&args); +} + +static void test_multifd_tcp_no_zero_page(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "defer", + .start_hook =3D test_migration_precopy_tcp_multifd_start_no_zero_p= age, + /* + * Multifd is more complicated than most of the features, it + * directly takes guest page buffers when sending, make sure + * everything will work alright even if guest page is changing. + */ + .live =3D true, + }; + test_precopy_common(&args); +} + static void test_multifd_tcp_zlib(void) { MigrateCommon args =3D { @@ -3550,6 +3598,10 @@ int main(int argc, char **argv) } migration_test_add("/migration/multifd/tcp/plain/none", test_multifd_tcp_none); + migration_test_add("/migration/multifd/tcp/plain/zero-page/legacy", + test_multifd_tcp_zero_page_legacy); + migration_test_add("/migration/multifd/tcp/plain/zero-page/none", + test_multifd_tcp_no_zero_page); migration_test_add("/migration/multifd/tcp/plain/cancel", test_multifd_tcp_cancel); migration_test_add("/migration/multifd/tcp/plain/zlib", --=20 2.30.2 From nobody Tue Nov 26 06:50:20 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1708977541; cv=none; d=zohomail.com; s=zohoarc; b=Xd2Ii4/lu/zPJ+JIVcwtoAwVG5E9sXaHuj4XBwF5hCZGw9lDGukZRUCLuSLU2J+6CkQo1qbuT31ZWsUoqxtP5AwiCQgDgNpTf0oWyb/Idb9NnlDFLr8ySj7gQlgELLXIgvnahUkbl5CVYHHSQGmzX42sdkSihzp1kuxVaGLzejY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708977541; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=T5vJ7Fapjt2fLK2rGlFf/dXYV4K/HjL/h5AzC5P5oqU=; b=Wnnk1wmm+dSPFXuytcra4hgEqoByEGLYK5oNJxaBvjLEdMvXyo7lKFCN5OmrxdpwU9Q1sZmOQ6tbBV95zEKqRxFQXhKSWX0cc43A1MxaoYF1gR3S03ZPrh8mX3SxsVpb9+xMkjOErkhxh4SUSB2RY71vryI9uxny2+D49ZZiBK4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708977541232577.7885763081517; Mon, 26 Feb 2024 11:59:01 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1reh6y-00017n-1J; Mon, 26 Feb 2024 14:58:04 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1reh6v-00015S-Ia for qemu-devel@nongnu.org; Mon, 26 Feb 2024 14:58:01 -0500 Received: from mail-ua1-x92f.google.com ([2607:f8b0:4864:20::92f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1reh6t-0004v5-Vk for qemu-devel@nongnu.org; Mon, 26 Feb 2024 14:58:01 -0500 Received: by mail-ua1-x92f.google.com with SMTP id a1e0cc1a2514c-7d6a772e08dso983497241.2 for ; Mon, 26 Feb 2024 11:57:59 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.95]) by smtp.gmail.com with ESMTPSA id ej5-20020a056122270500b004ca3dc45886sm712738vkb.47.2024.02.26.11.57.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 11:57:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1708977479; x=1709582279; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=T5vJ7Fapjt2fLK2rGlFf/dXYV4K/HjL/h5AzC5P5oqU=; b=cB74g3Oa8gMqGLedsAmUo86VW8hLb8iEX0I0LLC0ccc3wV3F5cxdE/X6nB9MifMdoi Gte9gHJ7rYIZerH4IB/LbzHZNl3azaoj6etxZ1o1h0u4CTf/taXpSwryfcLdjg3kIdCj Z6wXybRC73jt2k1zUbq2wJ3lcHZ8QKTAUJAYUzX6/UCXP7q7ZPw+ltUObrkIVe8+2IjG I2zDQY6M21+An1Dm7aQBTikeE7RKn7k80aZNeTU7levhNMSieioHUQHcdUwwy/z5YNsj T1gYXIJtB7t4NVVUF9seJcrl38RzrkBK1xUH+u4JvfNWoDCSfvQAM2SummGu8xBGAkpI vBeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708977479; x=1709582279; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=T5vJ7Fapjt2fLK2rGlFf/dXYV4K/HjL/h5AzC5P5oqU=; b=sglkfmICNnOYE8iLYrCbg22BB6FAyWbFXNPQCyUx8fOSXSTBkoPwZPGNsUctl/0eri LQqSrRMZrGUGuSk2NoKoAi/52B5UV/tYW/GAJpwcTDjjSb5gX6tkto8WQN8fwLFXsfFc iuM8+72I0DAZVfoFmpCRYjrVw/9uLbBPDzOxtnyDVW3DJ5Iuq2GjmMmfZrKIYWuK9m1B +jry04lgLH/eF4nD9CbSsci/6CWkikcjE2ccPkgoX1i6a7m1PIU31PPLT6ddjAPutBH2 metucDAmv2QNId5g5r4PugSBJNEgYmqZGpA4OUKcX8zfZGi7nhvTmhIcnplH1gttVg7+ PTIQ== X-Forwarded-Encrypted: i=1; AJvYcCWEKVVzjpXEPVZxV8kXif3BpsKDzF3VW3lLOdHCVNnFOnX/gUR0tPppIXnWP8IvhtGvQ2VG/VZRdUx5eRloFODxTtiSMx4= X-Gm-Message-State: AOJu0YxX5bTLuQdjYuIPae0XHM2ClEUT/eYJGoMPaTV3mEU5GLp6P3+U NlINwqHHgKaY5muKs1QQCiTsE2cojdBvREfVMDuf2mGUOY7ucCV0q/9oseOPveo= X-Google-Smtp-Source: AGHT+IF/gC96i+aIgOoScT2eiHvHRV4EG5SBgLHNY4+yGTH+RYZLKbxzxg2f2jaYGZOIkEqHmo1UzQ== X-Received: by 2002:a1f:ebc2:0:b0:4c0:37a4:dbbd with SMTP id j185-20020a1febc2000000b004c037a4dbbdmr4086711vkh.13.1708977478633; Mon, 26 Feb 2024 11:57:58 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, jdenemar@redhat.com, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, qemu-devel@nongnu.org Cc: Hao Xiang Subject: [PATCH v3 6/7] migration/multifd: Add zero pages and zero bytes counter to migration status interface. Date: Mon, 26 Feb 2024 19:56:53 +0000 Message-Id: <20240226195654.934709-7-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240226195654.934709-1-hao.xiang@bytedance.com> References: <20240226195654.934709-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::92f; envelope-from=hao.xiang@bytedance.com; helo=mail-ua1-x92f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1708977541542100009 Content-Type: text/plain; charset="utf-8" This change extends the MigrationStatus interface to track zero pages and zero bytes counter. Signed-off-by: Hao Xiang --- migration/migration-hmp-cmds.c | 4 ++++ migration/migration.c | 2 ++ qapi/migration.json | 15 ++++++++++++++- tests/migration/guestperf/engine.py | 2 ++ 4 files changed, 22 insertions(+), 1 deletion(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 7e96ae6ffd..a38ad0255d 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -111,6 +111,10 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) info->ram->normal); monitor_printf(mon, "normal bytes: %" PRIu64 " kbytes\n", info->ram->normal_bytes >> 10); + monitor_printf(mon, "zero pages: %" PRIu64 " pages\n", + info->ram->zero_pages); + monitor_printf(mon, "zero bytes: %" PRIu64 " kbytes\n", + info->ram->zero_bytes >> 10); monitor_printf(mon, "dirty sync count: %" PRIu64 "\n", info->ram->dirty_sync_count); monitor_printf(mon, "page size: %" PRIu64 " kbytes\n", diff --git a/migration/migration.c b/migration/migration.c index ab21de2cad..a99f86f273 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1112,6 +1112,8 @@ static void populate_ram_info(MigrationInfo *info, Mi= grationState *s) info->ram->skipped =3D 0; info->ram->normal =3D stat64_get(&mig_stats.normal_pages); info->ram->normal_bytes =3D info->ram->normal * page_size; + info->ram->zero_pages =3D stat64_get(&mig_stats.zero_pages); + info->ram->zero_bytes =3D info->ram->zero_pages * page_size; info->ram->mbps =3D s->mbps; info->ram->dirty_sync_count =3D stat64_get(&mig_stats.dirty_sync_count); diff --git a/qapi/migration.json b/qapi/migration.json index a0a85a0312..171734c07e 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -63,6 +63,10 @@ # between 0 and @dirty-sync-count * @multifd-channels. (since # 7.1) # +# @zero-pages: number of zero pages (since 9.0) +# +# @zero-bytes: number of zero bytes sent (since 9.0) +# # Features: # # @deprecated: Member @skipped is always zero since 1.5.3 @@ -81,7 +85,8 @@ 'multifd-bytes': 'uint64', 'pages-per-second': 'uint64', 'precopy-bytes': 'uint64', 'downtime-bytes': 'uint64', 'postcopy-bytes': 'uint64', - 'dirty-sync-missed-zero-copy': 'uint64' } } + 'dirty-sync-missed-zero-copy': 'uint64', + 'zero-pages': 'int', 'zero-bytes': 'size' } } =20 ## # @XBZRLECacheStats: @@ -332,6 +337,8 @@ # "duplicate":123, # "normal":123, # "normal-bytes":123456, +# "zero-pages":123, +# "zero-bytes":123456, # "dirty-sync-count":15 # } # } @@ -358,6 +365,8 @@ # "duplicate":123, # "normal":123, # "normal-bytes":123456, +# "zero-pages":123, +# "zero-bytes":123456, # "dirty-sync-count":15 # } # } @@ -379,6 +388,8 @@ # "duplicate":123, # "normal":123, # "normal-bytes":123456, +# "zero-pages":123, +# "zero-bytes":123456, # "dirty-sync-count":15 # }, # "disk":{ @@ -405,6 +416,8 @@ # "duplicate":10, # "normal":3333, # "normal-bytes":3412992, +# "zero-pages":3333, +# "zero-bytes":3412992, # "dirty-sync-count":15 # }, # "xbzrle-cache":{ diff --git a/tests/migration/guestperf/engine.py b/tests/migration/guestper= f/engine.py index 608d7270f6..693e07c227 100644 --- a/tests/migration/guestperf/engine.py +++ b/tests/migration/guestperf/engine.py @@ -92,6 +92,8 @@ def _migrate_progress(self, vm): info["ram"].get("skipped", 0), info["ram"].get("normal", 0), info["ram"].get("normal-bytes", 0), + info["ram"].get("zero-pages", 0); + info["ram"].get("zero-bytes", 0); info["ram"].get("dirty-pages-rate", 0), info["ram"].get("mbps", 0), info["ram"].get("dirty-sync-count", 0) --=20 2.30.2 From nobody Tue Nov 26 06:50:20 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1708977537; cv=none; d=zohomail.com; s=zohoarc; b=eTAZM4m/AGbxFG2x6zUnXkTjefDF53D7wvZAA4IkuRAfHarM0nfQPwTVplBbbMdKM81m+eYoYoLp++Z/NQpnsraaMbd52N+A7voZg7km2OBtJmSbwGyw+htc+SCkTNu7ZpDrNz4dyUdTnzZRDHzUNqZnF5kJPqZ/ZNfA7Tz0ih4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1708977537; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Q+PTBS3KKeQY3OK8H3H0sBlsb0C06FUYzlAVrOFK09Y=; b=PLNlJhGDsGUcZ1CWOKU5DeEFt1hyDbeVM1P+KR/9Yglz9C/U0vMCvZlOkeQayOgxeUiFVcQufFxHSsUCLEZzjJ+e2wTxyEV/QkZKTRm5qCTzuRM3toub7zkEq2RriQTt4RKcuHmXOD0U1NgBANCOKPyq1mDyL+9kiHmF6JkHuRY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1708977537271818.4259943000327; Mon, 26 Feb 2024 11:58:57 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1reh76-0001KA-OI; Mon, 26 Feb 2024 14:58:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1reh74-0001J3-UD for qemu-devel@nongnu.org; Mon, 26 Feb 2024 14:58:10 -0500 Received: from mail-vk1-xa31.google.com ([2607:f8b0:4864:20::a31]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1reh6x-0004vN-JP for qemu-devel@nongnu.org; Mon, 26 Feb 2024 14:58:10 -0500 Received: by mail-vk1-xa31.google.com with SMTP id 71dfb90a1353d-4cbc49dacc2so1893375e0c.1 for ; Mon, 26 Feb 2024 11:58:02 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.212.95]) by smtp.gmail.com with ESMTPSA id ej5-20020a056122270500b004ca3dc45886sm712738vkb.47.2024.02.26.11.58.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 11:58:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1708977481; x=1709582281; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Q+PTBS3KKeQY3OK8H3H0sBlsb0C06FUYzlAVrOFK09Y=; b=jwxwwBFPxwHnukQ2veUsNHzbBtBEG6F7OFiEXAIM8IHG0MVGEC7PzwakYOByt/rEdM ysUDtyD4gq0gv8MP/bc/1gFXIDWWELC0dkVId9GCg8DN7ddxXf0b9lc7kVMCh7otUVXz JfCnKP2fAijiB9DUxYPVlCtnY4KYFDnCT5LwGLi5XPp7Q9ivG4o+tPcKyAnGwFlKMzJ+ rWZqbHhajLPhKTlcp0FK8eu9EtIcpyjajDl3h3xru9n/K8fw7H6BY33deIWQEsH647tS grisAM4y76WflhqrX1wWS9z3iK4ihp3YaFdrGMT8Hldoz9Y4BaOgQnRVrEzNpUPl03FQ QhhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708977481; x=1709582281; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Q+PTBS3KKeQY3OK8H3H0sBlsb0C06FUYzlAVrOFK09Y=; b=hhUuREJmSpGjyPnjMTm0F5hSG+QGMKaIi+Wj0YYZOk1wxWysrhYvRMhq5CmoHHQLSa zOl6H2zOmCbn+PRq1YuRNoVfBNIqviJYyiw5XZdBJhCxIlgSwneQK1pcn6YngnHjM0lY /hVctnjbLQmT2F2vgQyKORmVcygoebWg15efZcLsydMFh3dZMjway67z+BlQJsLIiE4j qkPPC/2nfXjV22opgTjYDP4OF8mhDw36wXJyaTynGc73KEmtw8qkpZaVmVsNf+1yWXfZ FUn0wSVK/B+UeLqXCRy0rzNDRR8LOFXjrhJu9mo+IIjyO4OlnhC7puKXz93xRGYMNf5I UkrQ== X-Forwarded-Encrypted: i=1; AJvYcCWwQ8GPthiBFvUXY+K4HFSJFwQOd9PyrgNEC/5KxyQ9kPhWwxQTmKOr67hbsvoZwAHyhsrnYUTVhAr3omlL/taEEd16HNo= X-Gm-Message-State: AOJu0YycI3D0Kj9KN8FzYrp8/qtVG9iOdua/6xlQ5aKYS/Dfdm9QjjhQ LTnst0QLkF4PVi2XJE60ktJVr3tyGlBWKRbH5bd5QSMF1iOtvXMr4ASrpDQZmMI= X-Google-Smtp-Source: AGHT+IHtxxekvVzB/trJeF2/orsW8jpD2AX84jZC+UwDYMCwJW7WI/gNPwWleeX6kf6ZaQhlWChq+g== X-Received: by 2002:a1f:dac3:0:b0:4cc:b98:43d2 with SMTP id r186-20020a1fdac3000000b004cc0b9843d2mr5944422vkg.4.1708977481510; Mon, 26 Feb 2024 11:58:01 -0800 (PST) From: Hao Xiang To: pbonzini@redhat.com, berrange@redhat.com, eduardo@habkost.net, peterx@redhat.com, farosas@suse.de, eblake@redhat.com, armbru@redhat.com, thuth@redhat.com, lvivier@redhat.com, jdenemar@redhat.com, marcel.apfelbaum@gmail.com, philmd@linaro.org, wangyanan55@huawei.com, qemu-devel@nongnu.org Cc: Hao Xiang Subject: [PATCH v3 7/7] Update maintainer contact for migration multifd zero page checking acceleration. Date: Mon, 26 Feb 2024 19:56:54 +0000 Message-Id: <20240226195654.934709-8-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240226195654.934709-1-hao.xiang@bytedance.com> References: <20240226195654.934709-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::a31; envelope-from=hao.xiang@bytedance.com; helo=mail-vk1-xa31.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1708977537478100001 Content-Type: text/plain; charset="utf-8" Add myself to maintain multifd zero page checking acceleration function. Signed-off-by: Hao Xiang --- MAINTAINERS | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 992799171f..4a4f8f24e0 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3413,6 +3413,11 @@ F: tests/migration/ F: util/userfaultfd.c X: migration/rdma* =20 +Migration multifd zero page checking acceleration +M: Hao Xiang +S: Maintained +F: migration/multifd-zero-page.c + RDMA Migration R: Li Zhijian R: Peter Xu --=20 2.30.2