From nobody Tue Nov 26 13:24:07 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1707261667; cv=none; d=zohomail.com; s=zohoarc; b=PMHDN6CYv9p7ELZKl6lY3fjr5fWwzUpeDtF52S3/jaoSBG3HiowZVSEb3/WHawgigFEAoBiueOFdLaS0bIb1qLW/JP9X4cRgs12W6a0XQUAYZJbQSOb3xkqIFNmWDjimQyrBmdXggeoD9XE2o9DUZ6xXlYcKw1Cdss1bLtosSDU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1707261667; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=GOyUBhprnkXUNH3q7XOwDPa0Q/3zpIV90CrsK+ksq4M=; b=apN0akf9KhB5fND19PcgPo7SugyQLBWRfCnOei5dwI9MA6CZedyNnoFmKSuDoIMzXEd9ZLSZJ3X5UCBcwwHqSN0wlZIEngFm2o0bRTSBtLKzoHZ4N0FMXsWeU8/JBjkwUplhPOa9nYRH6n6ilGWffLaySySYTPKFE15FRmUFt6k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1707261667302114.92955129481459; Tue, 6 Feb 2024 15:21:07 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rXUjx-0006Dm-NN; Tue, 06 Feb 2024 18:20:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rXUjw-0006DO-Ld for qemu-devel@nongnu.org; Tue, 06 Feb 2024 18:20:32 -0500 Received: from mail-vs1-xe29.google.com ([2607:f8b0:4864:20::e29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rXUjh-0005iG-OV for qemu-devel@nongnu.org; Tue, 06 Feb 2024 18:20:32 -0500 Received: by mail-vs1-xe29.google.com with SMTP id ada2fe7eead31-46d2c1077easo956176137.1 for ; Tue, 06 Feb 2024 15:20:13 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.215.112]) by smtp.gmail.com with ESMTPSA id d11-20020a67b60b000000b0046d20ce8886sm2352vsm.1.2024.02.06.15.20.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Feb 2024 15:20:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1707261612; x=1707866412; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GOyUBhprnkXUNH3q7XOwDPa0Q/3zpIV90CrsK+ksq4M=; b=lUCAda4gaHomk38YZqhwubNdhhxPXn1YiP4w3IgZ5+Fz5GfxvMEQASuqfWjN3KqbqT Mc1gLMAOF3lFSc3lWHjiiLWIjjzLGt1UY92EDqNNyC+upO6tpsexzsH7t1OzG+xVlFVw J96oNXM/2M8cOTi8orXaC50tlcwTNvlyhpqvgslvudcgI7/byIQo8MlHakxUtrwfzwzM 23Rrjq6qmP+IyaLvQnP9uAfr17CnvhFKBci0Zo3xXJYim56Cmi3iIpKHiCaf0K2QXzRj Nbi9e4pdCouf/ewEiZVOXSA0iAza7rJixsRnDebGXxdp2HauOEt19t82KeNeLe3IcdTu Vepg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707261612; x=1707866412; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GOyUBhprnkXUNH3q7XOwDPa0Q/3zpIV90CrsK+ksq4M=; b=QqwJAecLcK8pCVVL0OQdACvvhjtvzwsL4qHm1GyM47WCA1RoZgoQuO2g6MfUtFDwUI 0K50RMog2YWOiqzbSREG8xzuF5wPJABP4/WxQkcaWfP8L4U1jos2hHnNZNFjSAgba41J 6VSRxAtYNA6N7+jR/7SYTwYkTC7pz+tR2JIdwffef4Eq+8o3T+sC3TT65Dyicn3r4wx2 itC+xaqsV/eH6bWJhIMIBdkjOt3VRu9jIJN7ld/i10+tljwMUpNmGSQCZSVpZfM4m+Vc 8iIj00Jf/PN2ZYZxevnmoTwcbeoPLBao5CRX2qXDHNeRukZ7TgQ2iM6VFYtvQqhwJsTx Yz9w== X-Gm-Message-State: AOJu0YyHNvu71uRmeC21mVlKQoAJtlwf/0+Hso57VoGyhVvmH31CPptX 5uWcTqmaUq9g3Ka9/91tNCKhdTgv3o7wBCbmS5wQCfV6Mkn5eW4h3WyWHN5fDdG6igiPdXOHd29 07Xs= X-Google-Smtp-Source: AGHT+IFQKKlKdB6Ds43ZQ7GJK7ykk4LqNjvNJhTqIrBeQXqtD+bWy5e0boDJJjYuRlvKyqJAepKztg== X-Received: by 2002:a05:6102:34d5:b0:46d:1043:fe65 with SMTP id a21-20020a05610234d500b0046d1043fe65mr1113085vst.26.1707261612354; Tue, 06 Feb 2024 15:20:12 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCXHNS8gDsLf0J52zqZtp8RjTGFYKSXyxXs94Czhe+zj0oD3QaZhqzYteRO5FjqwhHZXhhIGUmlu/MVCEhsugCo6CJX3vJGi0k41NOLktexfF9f4 From: Hao Xiang To: qemu-devel@nongnu.org, farosas@suse.de, peterx@redhat.com Cc: Hao Xiang Subject: [PATCH 1/6] migration/multifd: Add new migration option multifd-zero-page. Date: Tue, 6 Feb 2024 23:19:03 +0000 Message-Id: <20240206231908.1792529-2-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240206231908.1792529-1-hao.xiang@bytedance.com> References: <20240206231908.1792529-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::e29; envelope-from=hao.xiang@bytedance.com; helo=mail-vs1-xe29.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, T_SPF_HELO_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1707261670134100006 Content-Type: text/plain; charset="utf-8" This new parameter controls where the zero page checking is running. If this parameter is set to true, zero page checking is done in the multifd sender threads. If this parameter is set to false, zero page checking is done in the migration main thread. Signed-off-by: Hao Xiang --- migration/migration-hmp-cmds.c | 7 +++++++ migration/options.c | 20 ++++++++++++++++++++ migration/options.h | 1 + qapi/migration.json | 24 +++++++++++++++++++++--- 4 files changed, 49 insertions(+), 3 deletions(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 99b49df5dd..8b0c205a41 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -344,6 +344,9 @@ void hmp_info_migrate_parameters(Monitor *mon, const QD= ict *qdict) monitor_printf(mon, "%s: %s\n", MigrationParameter_str(MIGRATION_PARAMETER_MULTIFD_COMPRESSION= ), MultiFDCompression_str(params->multifd_compression)); + monitor_printf(mon, "%s: %s\n", + MigrationParameter_str(MIGRATION_PARAMETER_MULTIFD_ZERO_PAGE), + params->multifd_zero_page ? "on" : "off"); monitor_printf(mon, "%s: %" PRIu64 " bytes\n", MigrationParameter_str(MIGRATION_PARAMETER_XBZRLE_CACHE_SIZE), params->xbzrle_cache_size); @@ -634,6 +637,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDi= ct *qdict) p->has_multifd_zstd_level =3D true; visit_type_uint8(v, param, &p->multifd_zstd_level, &err); break; + case MIGRATION_PARAMETER_MULTIFD_ZERO_PAGE: + p->has_multifd_zero_page =3D true; + visit_type_bool(v, param, &p->multifd_zero_page, &err); + break; case MIGRATION_PARAMETER_XBZRLE_CACHE_SIZE: p->has_xbzrle_cache_size =3D true; if (!visit_type_size(v, param, &cache_size, &err)) { diff --git a/migration/options.c b/migration/options.c index 3e3e0b93b4..cb18a41267 100644 --- a/migration/options.c +++ b/migration/options.c @@ -179,6 +179,8 @@ Property migration_properties[] =3D { DEFINE_PROP_MIG_MODE("mode", MigrationState, parameters.mode, MIG_MODE_NORMAL), + DEFINE_PROP_BOOL("multifd-zero-page", MigrationState, + parameters.multifd_zero_page, true), =20 /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), @@ -903,6 +905,13 @@ uint64_t migrate_xbzrle_cache_size(void) return s->parameters.xbzrle_cache_size; } =20 +bool migrate_multifd_zero_page(void) +{ + MigrationState *s =3D migrate_get_current(); + + return s->parameters.multifd_zero_page; +} + /* parameter setters */ =20 void migrate_set_block_incremental(bool value) @@ -1013,6 +1022,8 @@ MigrationParameters *qmp_query_migrate_parameters(Err= or **errp) params->vcpu_dirty_limit =3D s->parameters.vcpu_dirty_limit; params->has_mode =3D true; params->mode =3D s->parameters.mode; + params->has_multifd_zero_page =3D true; + params->multifd_zero_page =3D s->parameters.multifd_zero_page; =20 return params; } @@ -1049,6 +1060,7 @@ void migrate_params_init(MigrationParameters *params) params->has_x_vcpu_dirty_limit_period =3D true; params->has_vcpu_dirty_limit =3D true; params->has_mode =3D true; + params->has_multifd_zero_page =3D true; } =20 /* @@ -1350,6 +1362,10 @@ static void migrate_params_test_apply(MigrateSetPara= meters *params, if (params->has_mode) { dest->mode =3D params->mode; } + + if (params->has_multifd_zero_page) { + dest->multifd_zero_page =3D params->multifd_zero_page; + } } =20 static void migrate_params_apply(MigrateSetParameters *params, Error **err= p) @@ -1494,6 +1510,10 @@ static void migrate_params_apply(MigrateSetParameter= s *params, Error **errp) if (params->has_mode) { s->parameters.mode =3D params->mode; } + + if (params->has_multifd_zero_page) { + s->parameters.multifd_zero_page =3D params->multifd_zero_page; + } } =20 void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) diff --git a/migration/options.h b/migration/options.h index 246c160aee..c080a6ba18 100644 --- a/migration/options.h +++ b/migration/options.h @@ -93,6 +93,7 @@ const char *migrate_tls_authz(void); const char *migrate_tls_creds(void); const char *migrate_tls_hostname(void); uint64_t migrate_xbzrle_cache_size(void); +bool migrate_multifd_zero_page(void); =20 /* parameters setters */ =20 diff --git a/qapi/migration.json b/qapi/migration.json index 819708321d..ff033a0344 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -874,6 +874,11 @@ # @mode: Migration mode. See description in @MigMode. Default is 'normal'. # (Since 8.2) # +# @multifd-zero-page: Multifd zero page checking. If the parameter is true, +# zero page checking is done on the multifd sender thread. If the para= meter +# is false, zero page checking is done on the migration main thread. D= efault +# is set to true. (Since 9.0) +# # Features: # # @deprecated: Member @block-incremental is deprecated. Use @@ -907,7 +912,8 @@ 'block-bitmap-mapping', { 'name': 'x-vcpu-dirty-limit-period', 'features': ['unstable']= }, 'vcpu-dirty-limit', - 'mode'] } + 'mode', + 'multifd-zero-page'] } =20 ## # @MigrateSetParameters: @@ -1062,6 +1068,11 @@ # @mode: Migration mode. See description in @MigMode. Default is 'normal'. # (Since 8.2) # +# @multifd-zero-page: Multifd zero page checking. If the parameter is true, +# zero page checking is done on the multifd sender thread. If the para= meter +# is false, zero page checking is done on the migration main thread. D= efault +# is set to true. (Since 9.0) +# # Features: # # @deprecated: Member @block-incremental is deprecated. Use @@ -1115,7 +1126,8 @@ '*x-vcpu-dirty-limit-period': { 'type': 'uint64', 'features': [ 'unstable' ] }, '*vcpu-dirty-limit': 'uint64', - '*mode': 'MigMode'} } + '*mode': 'MigMode', + '*multifd-zero-page': 'bool'} } =20 ## # @migrate-set-parameters: @@ -1290,6 +1302,11 @@ # @mode: Migration mode. See description in @MigMode. Default is 'normal'. # (Since 8.2) # +# @multifd-zero-page: Multifd zero page checking. If the parameter is true, +# zero page checking is done on the multifd sender thread. If the para= meter +# is false, zero page checking is done on the migration main thread. D= efault +# is set to true. (Since 9.0) +# # Features: # # @deprecated: Member @block-incremental is deprecated. Use @@ -1340,7 +1357,8 @@ '*x-vcpu-dirty-limit-period': { 'type': 'uint64', 'features': [ 'unstable' ] }, '*vcpu-dirty-limit': 'uint64', - '*mode': 'MigMode'} } + '*mode': 'MigMode', + '*multifd-zero-page': 'bool'} } =20 ## # @query-migrate-parameters: --=20 2.30.2 From nobody Tue Nov 26 13:24:07 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1707261680; cv=none; d=zohomail.com; s=zohoarc; b=oEtpNYV5tkSIVIVZ4tetKAS2KRf1gWYUP3+ZEt2csXe9EMviy6BzqEVba4ds0HbPCHBkKCx6t8sMDb/J9CL35vcHiJ5LH4D1+/5UFCJBEnH0EiU+4bbmkfoKxvKyVuoiCEoCUpn2yiBf/cHOY+3gm/Vp+fvLyihwhNVVQJc+G60= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1707261680; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=DrAT5QHTNwGXU+/ed1LMaREV4iyygAJT3lqN0TCwnd0=; b=e2roPNiQgUp5rkHO1SLoIC2z2k1qxLNMLdpnTJfwytPCH0oxuw0ZcpGtIBnfD2JxCWfgTa0mZrbN4hh2rBEzc3pDQIaCE2BW+7OlZGmwiME5BHRm4q8mGxB0aYVGr/h2r2T0uw7yq8CjjriG1T0peQx5eB9sh4SGNF1ZWRpD9ds= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1707261680311955.2926897874911; Tue, 6 Feb 2024 15:21:20 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rXUjs-0006CD-Nd; Tue, 06 Feb 2024 18:20:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rXUjr-0006Bg-Sj for qemu-devel@nongnu.org; Tue, 06 Feb 2024 18:20:27 -0500 Received: from mail-vs1-xe36.google.com ([2607:f8b0:4864:20::e36]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rXUji-0005jX-0a for qemu-devel@nongnu.org; Tue, 06 Feb 2024 18:20:26 -0500 Received: by mail-vs1-xe36.google.com with SMTP id ada2fe7eead31-46d2085eeebso31929137.1 for ; Tue, 06 Feb 2024 15:20:15 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.215.112]) by smtp.gmail.com with ESMTPSA id d11-20020a67b60b000000b0046d20ce8886sm2352vsm.1.2024.02.06.15.20.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Feb 2024 15:20:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1707261614; x=1707866414; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DrAT5QHTNwGXU+/ed1LMaREV4iyygAJT3lqN0TCwnd0=; b=Qjrme7xm7Ify6xqaVdc18c0Vuvcl49GWbdwJfCXlBiCXbAFQluU5ibMU+1jE4WnVUj 0eafi0lOeAb8Dz8J5/QGElLs/CCEdyJ2tVuMeV46qlXhfGFqdECyQ08AgvZiYxPY9Ht5 /tdCN8piEHGNRX6My0j3RXF4Dg8+8SOs+6XfxIt1UX4rpwWn9eVrOGCYz1m+DH+XKJW8 IopZVlHgJmEuK8XAuay5Am07KLzpjm/rGXyFuoF6nJKplb2bA413iOiouBUSS9drfjdG 45gGlYUj6idUeSwjcBH4I7fj9MWphgUo2jtgHspKF6QZZ1i7r8MGHe0AWR8CGr0fWJlM 9MTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707261614; x=1707866414; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DrAT5QHTNwGXU+/ed1LMaREV4iyygAJT3lqN0TCwnd0=; b=Oopf04zKzan7RsgU7hJIGQ1TPJP5VnclbLycLYNs/UcTf3HOyzOdIQjaPTUkISwlFy nvPDkLNTpYcIGtQlfNZ47NTXu2AC0K0OONVZA6AlOTrp5LDViR91nUlGwS0av+2lK2Vl dVdHg1VumrERi2sYKFaePzkMEHGvHhW38/hNeOg4VPjN3X0tCdqEg0uBLK39tUyquI5e uc5cYKngVgACyMGkVkplihVCWwkjmrkRFoeBQirnz77y0pNjT1HEFl4LQ2FfSg3WiTLv JJNbpwcUvGB0yxv15+LTmDswecOCdY0Q05JKsGaLNWlCVEFnV1KjVJK7a3cluX3NfCY8 R0wQ== X-Gm-Message-State: AOJu0YxTQRAuK9y4nWLkzuOg+b+P5GQ/TQVlu6BcPi3EgGHzo+12bOoO EQBJSZg/UU5OIFBiof+94LtsXa7r5sJWozL60YvnvtdXKj5xKQYlqFE7CE4G6i7b1/35GVLp67m O30s= X-Google-Smtp-Source: AGHT+IEYKbYmrDerRepghez/tsmqx8LZ8LICBCgYDAoqBFh8RN/pOstJoSSeYMrU0zuiX8enD0ogSg== X-Received: by 2002:a05:6102:3706:b0:46d:1ffd:d87d with SMTP id s6-20020a056102370600b0046d1ffdd87dmr981364vst.17.1707261614260; Tue, 06 Feb 2024 15:20:14 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCUb2H7+cC7SgzsDo5R674bYxAQR2vIK5PRl2ku2EQla+6MGSesIuPeVZ0ZwrqwbLqMa6OgBprANwDES0x60uN6fb6ejaIuFY2W4pHye/y5xs+/O From: Hao Xiang To: qemu-devel@nongnu.org, farosas@suse.de, peterx@redhat.com Cc: Hao Xiang Subject: [PATCH 2/6] migration/multifd: Add zero pages and zero bytes counter to migration status interface. Date: Tue, 6 Feb 2024 23:19:04 +0000 Message-Id: <20240206231908.1792529-3-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240206231908.1792529-1-hao.xiang@bytedance.com> References: <20240206231908.1792529-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::e36; envelope-from=hao.xiang@bytedance.com; helo=mail-vs1-xe36.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1707261682301100001 Content-Type: text/plain; charset="utf-8" This change extends the MigrationStatus interface to track zero pages and zero bytes counter. Signed-off-by: Hao Xiang Reviewed-by: Peter Xu --- qapi/migration.json | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/qapi/migration.json b/qapi/migration.json index ff033a0344..69366fe3f4 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -63,6 +63,10 @@ # between 0 and @dirty-sync-count * @multifd-channels. (since # 7.1) # +# @zero: number of zero pages (since 9.0) +# +# @zero-bytes: number of zero bytes sent (since 9.0) +# # Features: # # @deprecated: Member @skipped is always zero since 1.5.3 @@ -81,7 +85,8 @@ 'multifd-bytes': 'uint64', 'pages-per-second': 'uint64', 'precopy-bytes': 'uint64', 'downtime-bytes': 'uint64', 'postcopy-bytes': 'uint64', - 'dirty-sync-missed-zero-copy': 'uint64' } } + 'dirty-sync-missed-zero-copy': 'uint64', + 'zero': 'int', 'zero-bytes': 'int' } } =20 ## # @XBZRLECacheStats: @@ -332,6 +337,8 @@ # "duplicate":123, # "normal":123, # "normal-bytes":123456, +# "zero":123, +# "zero-bytes":123456, # "dirty-sync-count":15 # } # } @@ -358,6 +365,8 @@ # "duplicate":123, # "normal":123, # "normal-bytes":123456, +# "zero":123, +# "zero-bytes":123456, # "dirty-sync-count":15 # } # } @@ -379,6 +388,8 @@ # "duplicate":123, # "normal":123, # "normal-bytes":123456, +# "zero":123, +# "zero-bytes":123456, # "dirty-sync-count":15 # }, # "disk":{ @@ -405,6 +416,8 @@ # "duplicate":10, # "normal":3333, # "normal-bytes":3412992, +# "zero":3333, +# "zero-bytes":3412992, # "dirty-sync-count":15 # }, # "xbzrle-cache":{ --=20 2.30.2 From nobody Tue Nov 26 13:24:07 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1707261698; cv=none; d=zohomail.com; s=zohoarc; b=fUZ37BSRhVffBh7NwU6AmFZEl2LJU6Yn1hm/OmGrHG1VpT9Ghum1Su3QpFAjjTidaVueDOTVGAzf0XMQXjBrPHbvRaAhdm2PUHDCkYppPAjI25hJdfnzVE6NiFJfY+2sV5pRXGDY4OTVc0dr5G67hQs3iXwG36H30sVJItfaWN4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1707261698; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Hql99z+y7jRDtiY0A8dhT8uNlKVIz08nW2SMNrsn4vg=; b=H/dY06+9rfgi3u71iI1rWhR7O1LnvCbwcDG71dLM0QEA+GRxxIXkIrl9+b243+b07BDwEs8+O8ro7+ZhcrkXgrPcezPcmsAbmDTF7j/gIb4N4IJKw3phvS9VTocmo1WId6zFWgMR2ssIihpQbHBRkTOus+K1BZNequkVQj+l9oY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1707261698727313.1145268980097; Tue, 6 Feb 2024 15:21:38 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rXUjs-0006C2-3N; Tue, 06 Feb 2024 18:20:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rXUjq-0006BX-FJ for qemu-devel@nongnu.org; Tue, 06 Feb 2024 18:20:26 -0500 Received: from mail-vs1-xe31.google.com ([2607:f8b0:4864:20::e31]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rXUji-0005kf-SI for qemu-devel@nongnu.org; Tue, 06 Feb 2024 18:20:26 -0500 Received: by mail-vs1-xe31.google.com with SMTP id ada2fe7eead31-46d1bebe946so946571137.2 for ; Tue, 06 Feb 2024 15:20:17 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.215.112]) by smtp.gmail.com with ESMTPSA id d11-20020a67b60b000000b0046d20ce8886sm2352vsm.1.2024.02.06.15.20.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Feb 2024 15:20:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1707261616; x=1707866416; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Hql99z+y7jRDtiY0A8dhT8uNlKVIz08nW2SMNrsn4vg=; b=ZR7mxwCK2JtAFEbhBU/64mPwwihRNZUirhSwkkMoXstX69dXbsqk4mkCdyWVdaH1M7 hcbXzyKRFE1f3teQvnzbmZEh7Nx4cFhMAoiyocxKiiBPci4TYCeG+7ZgndDxTDcXsosY MA/3bJ6CWG3Pldy6yB/G61zP5R5Pvv4almUDZ/d+PYg6GVGgbNSDZKo0nZnuturDYT70 +C2uDs5xX+pa5fzZ0CdYs9a9/dtu65v7Nn7tFiNDgIJ21reE0LSdvp/Ab6w7C9JWKxna DMHBBpfN9o8RkORSlIGIXEndTwVTTTymcXqV/nPHF7tD4z97Cg5RvxG+CEkzP0l0lsFp N2Ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707261616; x=1707866416; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Hql99z+y7jRDtiY0A8dhT8uNlKVIz08nW2SMNrsn4vg=; b=EoFOUvawuxFENBxprRY1iXsD5kJWOBGGUZx/Kat5V1h20ZB1f0zq1WxdhYEtPRnSV1 Hno5HsANYidt/LAo5pJ6+I/k/f64npNvhzfcgLXCcq1WIWcl07csHJrofROh9E/jZJ09 0opLoeJJseS9QO9xJ5yvEzUA/R640piG+SbhALzGZQRlxEUITUU5P2OchBfbgybGuTtn EK8jRMNxKvS9KqmU205FCBI7w6XzNykUKSIUGEwnDNLctiPQAzvB/oCD1H+ULXKsCsV7 Sqvp8ifmu4M1SDtKGLC4QA8pjFClfhpOYJWDF6Jn8QgXlNU3SIjbJgleYy+kT3LjTxPE l8nQ== X-Gm-Message-State: AOJu0YynnL5dFes/sHMP5A2A5KnnVvi+FGcsxklFOj2r9zlglbPOeDLE sPRPaWNr9fl3vGpkPWUGgqD0Wkg8CVZNh8DFoZBm0rwfX+cBAeq8YtVs571bHobcKJezmdpS0tK bUHI= X-Google-Smtp-Source: AGHT+IHXAiTF72wiy35ZvVV3r2q25C//x/eHRNML9F2n9XkBb2YWH9m2kan8GjqXd8Fkbxlbgv/qIw== X-Received: by 2002:a67:fd8c:0:b0:46d:3c23:a26 with SMTP id k12-20020a67fd8c000000b0046d3c230a26mr1037609vsq.22.1707261616147; Tue, 06 Feb 2024 15:20:16 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCUlnbhe3r2KWe1hPn8fXB6mfhpC19fSqbNrAQi44u0CeRBgKbesIT0XeTfrCRT6kNnaRzJxueSObYew2oEC+NO3HIif0h7izbuXo821OjRcdNNp From: Hao Xiang To: qemu-devel@nongnu.org, farosas@suse.de, peterx@redhat.com Cc: Hao Xiang Subject: [PATCH 3/6] migration/multifd: Support for zero pages transmission in multifd format. Date: Tue, 6 Feb 2024 23:19:05 +0000 Message-Id: <20240206231908.1792529-4-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240206231908.1792529-1-hao.xiang@bytedance.com> References: <20240206231908.1792529-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::e31; envelope-from=hao.xiang@bytedance.com; helo=mail-vs1-xe31.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1707261700238100005 Content-Type: text/plain; charset="utf-8" This change adds zero page counters and updates multifd send/receive tracing format to track the newly added counters. Signed-off-by: Hao Xiang --- migration/migration-hmp-cmds.c | 4 ++++ migration/multifd.c | 43 ++++++++++++++++++++++++++-------- migration/multifd.h | 17 +++++++++++++- migration/trace-events | 8 +++---- 4 files changed, 57 insertions(+), 15 deletions(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 8b0c205a41..2dd99b0509 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -111,6 +111,10 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict) info->ram->normal); monitor_printf(mon, "normal bytes: %" PRIu64 " kbytes\n", info->ram->normal_bytes >> 10); + monitor_printf(mon, "zero: %" PRIu64 " pages\n", + info->ram->zero); + monitor_printf(mon, "zero bytes: %" PRIu64 " kbytes\n", + info->ram->zero_bytes >> 10); monitor_printf(mon, "dirty sync count: %" PRIu64 "\n", info->ram->dirty_sync_count); monitor_printf(mon, "page size: %" PRIu64 " kbytes\n", diff --git a/migration/multifd.c b/migration/multifd.c index 25cbc6dc6b..a20d0ed10e 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -264,6 +264,7 @@ static void multifd_send_fill_packet(MultiFDSendParams = *p) packet->flags =3D cpu_to_be32(p->flags); packet->pages_alloc =3D cpu_to_be32(p->pages->allocated); packet->normal_pages =3D cpu_to_be32(p->normal_num); + packet->zero_pages =3D cpu_to_be32(p->zero_num); packet->next_packet_size =3D cpu_to_be32(p->next_packet_size); packet->packet_num =3D cpu_to_be64(p->packet_num); =20 @@ -317,18 +318,26 @@ static int multifd_recv_unfill_packet(MultiFDRecvPara= ms *p, Error **errp) p->normal_num =3D be32_to_cpu(packet->normal_pages); if (p->normal_num > packet->pages_alloc) { error_setg(errp, "multifd: received packet " - "with %u pages and expected maximum pages are %u", + "with %u normal pages and expected maximum pages are %u= ", p->normal_num, packet->pages_alloc) ; return -1; } =20 - p->next_packet_size =3D be32_to_cpu(packet->next_packet_size); - p->packet_num =3D be64_to_cpu(packet->packet_num); + p->zero_num =3D be32_to_cpu(packet->zero_pages); + if (p->zero_num > packet->pages_alloc - p->normal_num) { + error_setg(errp, "multifd: received packet " + "with %u zero pages and expected maximum zero pages are= %u", + p->zero_num, packet->pages_alloc - p->normal_num) ; + return -1; + } =20 - if (p->normal_num =3D=3D 0) { + if (p->normal_num =3D=3D 0 && p->zero_num =3D=3D 0) { return 0; } =20 + p->next_packet_size =3D be32_to_cpu(packet->next_packet_size); + p->packet_num =3D be64_to_cpu(packet->packet_num); + /* make sure that ramblock is 0 terminated */ packet->ramblock[255] =3D 0; p->block =3D qemu_ram_block_by_name(packet->ramblock); @@ -430,6 +439,7 @@ static int multifd_send_pages(void) p->packet_num =3D multifd_send_state->packet_num++; multifd_send_state->pages =3D p->pages; p->pages =3D pages; + qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); =20 @@ -551,6 +561,8 @@ void multifd_save_cleanup(void) p->iov =3D NULL; g_free(p->normal); p->normal =3D NULL; + g_free(p->zero); + p->zero =3D NULL; multifd_send_state->ops->send_cleanup(p, &local_err); if (local_err) { migrate_set_error(migrate_get_current(), local_err); @@ -679,6 +691,7 @@ static void *multifd_send_thread(void *opaque) uint64_t packet_num =3D p->packet_num; uint32_t flags; p->normal_num =3D 0; + p->zero_num =3D 0; =20 if (use_zero_copy_send) { p->iovs_num =3D 0; @@ -703,12 +716,13 @@ static void *multifd_send_thread(void *opaque) p->flags =3D 0; p->num_packets++; p->total_normal_pages +=3D p->normal_num; + p->total_zero_pages +=3D p->zero_num; p->pages->num =3D 0; p->pages->block =3D NULL; qemu_mutex_unlock(&p->mutex); =20 - trace_multifd_send(p->id, packet_num, p->normal_num, flags, - p->next_packet_size); + trace_multifd_send(p->id, packet_num, p->normal_num, p->zero_n= um, + flags, p->next_packet_size); =20 if (use_zero_copy_send) { /* Send header first, without zerocopy */ @@ -731,6 +745,8 @@ static void *multifd_send_thread(void *opaque) =20 stat64_add(&mig_stats.multifd_bytes, p->next_packet_size + p->packet_len); + stat64_add(&mig_stats.normal_pages, p->normal_num); + stat64_add(&mig_stats.zero_pages, p->zero_num); p->next_packet_size =3D 0; qemu_mutex_lock(&p->mutex); p->pending_job--; @@ -761,7 +777,8 @@ out: =20 rcu_unregister_thread(); migration_threads_remove(thread); - trace_multifd_send_thread_end(p->id, p->num_packets, p->total_normal_p= ages); + trace_multifd_send_thread_end(p->id, p->num_packets, p->total_normal_p= ages, + p->total_zero_pages); =20 return NULL; } @@ -936,6 +953,7 @@ int multifd_save_setup(Error **errp) /* We need one extra place for the packet header */ p->iov =3D g_new0(struct iovec, page_count + 1); p->normal =3D g_new0(ram_addr_t, page_count); + p->zero =3D g_new0(ram_addr_t, page_count); p->page_size =3D qemu_target_page_size(); p->page_count =3D page_count; =20 @@ -1051,6 +1069,8 @@ void multifd_load_cleanup(void) p->iov =3D NULL; g_free(p->normal); p->normal =3D NULL; + g_free(p->zero); + p->zero =3D NULL; multifd_recv_state->ops->recv_cleanup(p); } qemu_sem_destroy(&multifd_recv_state->sem_sync); @@ -1119,10 +1139,11 @@ static void *multifd_recv_thread(void *opaque) flags =3D p->flags; /* recv methods don't know how to handle the SYNC flag */ p->flags &=3D ~MULTIFD_FLAG_SYNC; - trace_multifd_recv(p->id, p->packet_num, p->normal_num, flags, - p->next_packet_size); + trace_multifd_recv(p->id, p->packet_num, p->normal_num, p->zero_nu= m, + flags, p->next_packet_size); p->num_packets++; p->total_normal_pages +=3D p->normal_num; + p->total_zero_pages +=3D p->zero_num; qemu_mutex_unlock(&p->mutex); =20 if (p->normal_num) { @@ -1147,7 +1168,8 @@ static void *multifd_recv_thread(void *opaque) qemu_mutex_unlock(&p->mutex); =20 rcu_unregister_thread(); - trace_multifd_recv_thread_end(p->id, p->num_packets, p->total_normal_p= ages); + trace_multifd_recv_thread_end(p->id, p->num_packets, p->total_normal_p= ages, + p->total_zero_pages); =20 return NULL; } @@ -1186,6 +1208,7 @@ int multifd_load_setup(Error **errp) p->name =3D g_strdup_printf("multifdrecv_%d", i); p->iov =3D g_new0(struct iovec, page_count); p->normal =3D g_new0(ram_addr_t, page_count); + p->zero =3D g_new0(ram_addr_t, page_count); p->page_count =3D page_count; p->page_size =3D qemu_target_page_size(); } diff --git a/migration/multifd.h b/migration/multifd.h index 35d11f103c..6be9b2f6c1 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -48,7 +48,10 @@ typedef struct { /* size of the next packet that contains pages */ uint32_t next_packet_size; uint64_t packet_num; - uint64_t unused[4]; /* Reserved for future use */ + /* zero pages */ + uint32_t zero_pages; + uint32_t unused32[1]; /* Reserved for future use */ + uint64_t unused64[3]; /* Reserved for future use */ char ramblock[256]; uint64_t offset[]; } __attribute__((packed)) MultiFDPacket_t; @@ -120,6 +123,8 @@ typedef struct { uint64_t num_packets; /* non zero pages sent through this channel */ uint64_t total_normal_pages; + /* zero pages sent through this channel */ + uint64_t total_zero_pages; /* buffers to send */ struct iovec *iov; /* number of iovs used */ @@ -128,6 +133,10 @@ typedef struct { ram_addr_t *normal; /* num of non zero pages */ uint32_t normal_num; + /* Pages that are zero */ + ram_addr_t *zero; + /* num of zero pages */ + uint32_t zero_num; /* used for compression methods */ void *data; } MultiFDSendParams; @@ -179,12 +188,18 @@ typedef struct { uint8_t *host; /* non zero pages recv through this channel */ uint64_t total_normal_pages; + /* zero pages recv through this channel */ + uint64_t total_zero_pages; /* buffers to recv */ struct iovec *iov; /* Pages that are not zero */ ram_addr_t *normal; /* num of non zero pages */ uint32_t normal_num; + /* Pages that are zero */ + ram_addr_t *zero; + /* num of zero pages */ + uint32_t zero_num; /* used for de-compression methods */ void *data; } MultiFDRecvParams; diff --git a/migration/trace-events b/migration/trace-events index de4a743c8a..c0a758db9d 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -128,21 +128,21 @@ postcopy_preempt_reset_channel(void) "" # multifd.c multifd_new_send_channel_async(uint8_t id) "channel %u" multifd_new_send_channel_async_error(uint8_t id, void *err) "channel=3D%u = err=3D%p" -multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flag= s, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " pages %u f= lags 0x%x next packet size %u" +multifd_recv(uint8_t id, uint64_t packet_num, uint32_t normal, uint32_t ze= ro, uint32_t flags, uint32_t next_packet_size) "channel %u packet_num %" PR= Iu64 " normal pages %u zero pages %u flags 0x%x next packet size %u" multifd_recv_new_channel(uint8_t id) "channel %u" multifd_recv_sync_main(long packet_num) "packet num %ld" multifd_recv_sync_main_signal(uint8_t id) "channel %u" multifd_recv_sync_main_wait(uint8_t id) "channel %u" multifd_recv_terminate_threads(bool error) "error %d" -multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t pages) "cha= nnel %u packets %" PRIu64 " pages %" PRIu64 +multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal pages %" PRI= u64 " zero pages %" PRIu64 multifd_recv_thread_start(uint8_t id) "%u" -multifd_send(uint8_t id, uint64_t packet_num, uint32_t normal, uint32_t fl= ags, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " normal p= ages %u flags 0x%x next packet size %u" +multifd_send(uint8_t id, uint64_t packet_num, uint32_t normalpages, uint32= _t zero_pages, uint32_t flags, uint32_t next_packet_size) "channel %u packe= t_num %" PRIu64 " normal pages %u zero pages %u flags 0x%x next packet size= %u" multifd_send_error(uint8_t id) "channel %u" multifd_send_sync_main(long packet_num) "packet num %ld" multifd_send_sync_main_signal(uint8_t id) "channel %u" multifd_send_sync_main_wait(uint8_t id) "channel %u" multifd_send_terminate_threads(bool error) "error %d" -multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s) "channel %u packets %" PRIu64 " normal pages %" PRIu64 +multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t normal_page= s, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal pages %" PR= Iu64 " zero pages %" PRIu64 multifd_send_thread_start(uint8_t id) "%u" multifd_tls_outgoing_handshake_start(void *ioc, void *tioc, const char *ho= stname) "ioc=3D%p tioc=3D%p hostname=3D%s" multifd_tls_outgoing_handshake_error(void *ioc, const char *err) "ioc=3D%p= err=3D%s" --=20 2.30.2 From nobody Tue Nov 26 13:24:07 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1707261673; cv=none; d=zohomail.com; s=zohoarc; b=K0euNuyvnpe8tQzJetNrKNqTCm5vJBr++O+zDVx0FbXkuABP5ocOdlUt/AeTTyQqlJaPWdhS5QFVfy27ob/LPFv4ayEjc5eMu5ktNRsVhiN6xt9PRDeTcnpC/R1WU1e8jL2SzhdiS7La5gepPH+wKdIr2wL2NGz1jzfBelM5B6c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1707261673; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=leJ95zOxaZBV0dR1eA9xS1nffMJu6UFCj/XCjWoCKsQ=; b=nOVnSZrB2I+h3kSJRs6pHpxltGkaVbvctY3FBewXEb6tCz+L6ZDMQavQs4L0dx3SvvRsm0bMQrUz9Pwt0Akt1VIxyhy6XYJjtjqDH75N1T3gzE3jsguPu3JQDMyU3SboghtTniiHIzzSdw5l5dnT5QV6MbGWi04nTa7uwlQxBh8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1707261673471457.79910642310085; Tue, 6 Feb 2024 15:21:13 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rXUjt-0006CS-9D; Tue, 06 Feb 2024 18:20:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rXUjr-0006Bx-Tn for qemu-devel@nongnu.org; Tue, 06 Feb 2024 18:20:27 -0500 Received: from mail-vs1-xe2c.google.com ([2607:f8b0:4864:20::e2c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rXUjk-0005mP-Qo for qemu-devel@nongnu.org; Tue, 06 Feb 2024 18:20:27 -0500 Received: by mail-vs1-xe2c.google.com with SMTP id ada2fe7eead31-46d3b3e0671so594295137.1 for ; Tue, 06 Feb 2024 15:20:19 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.215.112]) by smtp.gmail.com with ESMTPSA id d11-20020a67b60b000000b0046d20ce8886sm2352vsm.1.2024.02.06.15.20.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Feb 2024 15:20:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1707261617; x=1707866417; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=leJ95zOxaZBV0dR1eA9xS1nffMJu6UFCj/XCjWoCKsQ=; b=JFsKLcQRGb1RpTfnJ3rShMCcw1csajRjY4TPGfaVmJgaQmtGylulC8odKUiHv/g0Ex z6N05cM2j1SSixnUv8uyv3D+HX8emJo6QQpJxG19ypjELCKU6Kly9TPXTyAhRm+wefhW AT1nwBJhdYrPhvAVLdNd5DLfi9Ftv8NWcxTs2co3PggqMBNEK+8uQAPHcVkycGMuak1X VbrdkDyKfb0Zhxld+tlWG7i5AYlW7Y5Smk7V6tdIrEr3xfi/B04LcW4fHEDydMQ5an1p JfOpxFFgMHytw/1cnG8PWxi0QVc4VJwx3TROl4aYbocHPin40TSPm4cuqqglMbRYxNXJ FlQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707261617; x=1707866417; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=leJ95zOxaZBV0dR1eA9xS1nffMJu6UFCj/XCjWoCKsQ=; b=mTyMcgbKCCndZmXWa1WxVavlDWaNC+2GQ/c8XCaBlMyZg4LhMHm/ikqqVHWzjd6B5c UXeWxEoXSWat1NnJ7a4BbE0UeGN60d+dIAFqHRHx+hGyrx/VZm8yyzCc4Oa0UK/9196M 42mj+57EfUc1+a0sQtWs6ZhGVcpZA2Zz2VT9UA0g8SCOpoFNSJmRsli+R0NLDKlPgZog z7JoePLf5WAkVNXDpX8usg02Jj/SjI+y0me9rKg8FnBgru2X0mLczPa1MUmF+xp0e/mL +1o8ja+SAaRNXJMuSOYDQGJYg0sA6D2zYdPvY+MtH9vcoSfMaxcB5JSdczCQOfCq/rwP vYEA== X-Gm-Message-State: AOJu0YyW45U4yhNi68L+1PK93YzJme+uBXBqJ5Fo+1A6jsDGSB21zz5F SSR1zxFD715c+CB0ppYJiJIzslkaeoODUKOL9QpMrNgJQjVqRVl5nI0Covf8+URunalJjH/O7sQ XxLo= X-Google-Smtp-Source: AGHT+IEP1+8U0KedU2ehuuQBQhmBQyACXw8o+IFCofqxVbAMWadc7ZCtzV6sNtW5k5wB2HzEBE+9eQ== X-Received: by 2002:a05:6102:408:b0:46c:f977:4f9f with SMTP id d8-20020a056102040800b0046cf9774f9fmr999583vsq.0.1707261617743; Tue, 06 Feb 2024 15:20:17 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCVYOK1ZYEhlaL+Zu4x9ak6NOXgumqrLo5h2thw1yEh09d+ogGEKZ2NvllIctKVNqGdyT2pjtqyIugtZwYpLB34Wf++JWqYP95cubUFO/6kZ0hXa From: Hao Xiang To: qemu-devel@nongnu.org, farosas@suse.de, peterx@redhat.com Cc: Hao Xiang Subject: [PATCH 4/6] migration/multifd: Zero page transmission on the multifd thread. Date: Tue, 6 Feb 2024 23:19:06 +0000 Message-Id: <20240206231908.1792529-5-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240206231908.1792529-1-hao.xiang@bytedance.com> References: <20240206231908.1792529-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::e2c; envelope-from=hao.xiang@bytedance.com; helo=mail-vs1-xe2c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1707261674212100001 Content-Type: text/plain; charset="utf-8" This implements the zero page detection and handling on the multifd threads. Signed-off-by: Hao Xiang --- migration/multifd.c | 62 +++++++++++++++++++++++++++++++++++++++++---- migration/multifd.h | 5 ++++ 2 files changed, 62 insertions(+), 5 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index a20d0ed10e..c031f947c7 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -11,6 +11,7 @@ */ =20 #include "qemu/osdep.h" +#include "qemu/cutils.h" #include "qemu/rcu.h" #include "exec/target_page.h" #include "sysemu/sysemu.h" @@ -278,6 +279,12 @@ static void multifd_send_fill_packet(MultiFDSendParams= *p) =20 packet->offset[i] =3D cpu_to_be64(temp); } + for (i =3D 0; i < p->zero_num; i++) { + /* there are architectures where ram_addr_t is 32 bit */ + uint64_t temp =3D p->zero[i]; + + packet->offset[p->normal_num + i] =3D cpu_to_be64(temp); + } } =20 static int multifd_recv_unfill_packet(MultiFDRecvParams *p, Error **errp) @@ -360,6 +367,18 @@ static int multifd_recv_unfill_packet(MultiFDRecvParam= s *p, Error **errp) p->normal[i] =3D offset; } =20 + for (i =3D 0; i < p->zero_num; i++) { + uint64_t offset =3D be64_to_cpu(packet->offset[p->normal_num + i]); + + if (offset > (p->block->used_length - p->page_size)) { + error_setg(errp, "multifd: offset too long %" PRIu64 + " (max " RAM_ADDR_FMT ")", + offset, p->block->used_length); + return -1; + } + p->zero[i] =3D offset; + } + return 0; } =20 @@ -658,13 +677,37 @@ int multifd_send_sync_main(void) return 0; } =20 +static void zero_page_check_send(MultiFDSendParams *p) +{ + /* + * QEMU older than 9.0 don't understand zero page + * on multifd channel. This switch is required to + * maintain backward compatibility. + */ + bool use_multifd_zero_page =3D migrate_multifd_zero_page(); + RAMBlock *rb =3D p->pages->block; + + for (int i =3D 0; i < p->pages->num; i++) { + uint64_t offset =3D p->pages->offset[i]; + if (use_multifd_zero_page && + buffer_is_zero(rb->host + offset, p->page_size)) { + p->zero[p->zero_num] =3D offset; + p->zero_num++; + ram_release_page(rb->idstr, offset); + } else { + p->normal[p->normal_num] =3D offset; + p->normal_num++; + } + } +} + static void *multifd_send_thread(void *opaque) { MultiFDSendParams *p =3D opaque; MigrationThread *thread =3D NULL; Error *local_err =3D NULL; - int ret =3D 0; bool use_zero_copy_send =3D migrate_zero_copy_send(); + int ret =3D 0; =20 thread =3D migration_threads_add(p->name, qemu_get_thread_id()); =20 @@ -699,10 +742,7 @@ static void *multifd_send_thread(void *opaque) p->iovs_num =3D 1; } =20 - for (int i =3D 0; i < p->pages->num; i++) { - p->normal[p->normal_num] =3D p->pages->offset[i]; - p->normal_num++; - } + zero_page_check_send(p); =20 if (p->normal_num) { ret =3D multifd_send_state->ops->send_prepare(p, &local_er= r); @@ -1107,6 +1147,16 @@ void multifd_recv_sync_main(void) trace_multifd_recv_sync_main(multifd_recv_state->packet_num); } =20 +static void zero_page_check_recv(MultiFDRecvParams *p) +{ + for (int i =3D 0; i < p->zero_num; i++) { + void *page =3D p->host + p->zero[i]; + if (!buffer_is_zero(page, p->page_size)) { + memset(page, 0, p->page_size); + } + } +} + static void *multifd_recv_thread(void *opaque) { MultiFDRecvParams *p =3D opaque; @@ -1153,6 +1203,8 @@ static void *multifd_recv_thread(void *opaque) } } =20 + zero_page_check_recv(p); + if (flags & MULTIFD_FLAG_SYNC) { qemu_sem_post(&multifd_recv_state->sem_sync); qemu_sem_wait(&p->sem_sync); diff --git a/migration/multifd.h b/migration/multifd.h index 6be9b2f6c1..7448cb1aa9 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -53,6 +53,11 @@ typedef struct { uint32_t unused32[1]; /* Reserved for future use */ uint64_t unused64[3]; /* Reserved for future use */ char ramblock[256]; + /* + * This array contains the pointers to: + * - normal pages (initial normal_pages entries) + * - zero pages (following zero_pages entries) + */ uint64_t offset[]; } __attribute__((packed)) MultiFDPacket_t; =20 --=20 2.30.2 From nobody Tue Nov 26 13:24:07 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1707261667; cv=none; d=zohomail.com; s=zohoarc; b=LLkrdty5fJzOJeClm+M/iBR84pD+FXa26CQmR43/qZrWWNAxSqK5I65vPQ1oxjux0VW2oawHAQiPoPUuvKBGqNdyths4uMlHk4P18FLeGVQVU+BLfhCZiaqBRSf2Ks3e8wx0iQQllHrI6B99rsYiJQCOk/ksHOt2P2DCiiumRus= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1707261667; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=RlfCHJI8BMYL5xByh2htwg446RRZbeVbn6VxSty6oBo=; b=SqwIO4839Oqq+i+L1TgkmBDmjAS+hjnVrEWy/0nwiae3DCtOgEqvyi6a3Da+cOIwfjbkNtmifuikeMdpISkS3qv5Neh+HNXB5Gk9gtMRKdcLiizEiUTJM3X7c99SyJG8Q6oy5YTsJ65vtf0qi15yT81n2YMhWjr7DGLc3jiSLFY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1707261667265127.43755804470106; Tue, 6 Feb 2024 15:21:07 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rXUk2-0006Ec-2A; Tue, 06 Feb 2024 18:20:38 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rXUjz-0006EF-U0 for qemu-devel@nongnu.org; Tue, 06 Feb 2024 18:20:35 -0500 Received: from mail-vs1-xe2d.google.com ([2607:f8b0:4864:20::e2d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rXUjk-0005n5-Sc for qemu-devel@nongnu.org; Tue, 06 Feb 2024 18:20:35 -0500 Received: by mail-vs1-xe2d.google.com with SMTP id ada2fe7eead31-46d49b31e76so330025137.0 for ; Tue, 06 Feb 2024 15:20:20 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.215.112]) by smtp.gmail.com with ESMTPSA id d11-20020a67b60b000000b0046d20ce8886sm2352vsm.1.2024.02.06.15.20.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Feb 2024 15:20:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1707261619; x=1707866419; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RlfCHJI8BMYL5xByh2htwg446RRZbeVbn6VxSty6oBo=; b=EY/lQetEeYfkoO6PrHKm+JJkmpv7HQ46F/X4Jtf0e3SRWv00MLlE122QbmfGnvVUPP 1g4nucOVTHHMYOg172AJnuOIABlhCkgS3vEs6q4CeHFqHLUhN06P2M4i6OJhZLg65Zw6 dWRepjF01CVnSxZi2EQYFNQ1FdgVfXrjYR0XFaUcQmnWXGoU7eXVT36L6oOalG27k4r9 42hn+BujkmWd3z8yFtN8hHhztC7KaVaUdvA6LtbI/eFjL3g5RpDSgMw/2AscYPQVgYrJ r60YMZ/KkbDyVTLAe2TQuaL+UlUgXDQgFS4vDND3OPJ7Cmvq63Fq6beMlJhhS24Yut95 p/OA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707261619; x=1707866419; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RlfCHJI8BMYL5xByh2htwg446RRZbeVbn6VxSty6oBo=; b=qGHFkx9t0nVNZj1t5BnLuWNuiCx9WvDD3cU3sfRV9FsAK/XzF9zMthtZpNZwpKhIfk SC52K7xIQ6yCGdaV0zgNr0Xg22C2/spXhCojOAFHCC+7smwnEZHspzsPKw35rNRzc5QS ZWqtdrGqF6UvqwpPeGAGqJv6nii331PMGAHqcHJNcmp76zxf+JdUnZLcmBNtasZqZWwj AdUftDf+iPQyGRf/SgvCXiWfcy84VczZq/d1YxMeClcwYvmchXqBf9uDVuMMzbzgNJHT p7FBcBKq5359onaTle/aiJMRBxiJ70en8SVUGv00nZhURWFkb/1ln/dIFjL2izK7pOiI rwpA== X-Gm-Message-State: AOJu0YztXAbby2auAAImMC4w96VKWLVXwGEDosshrhbWYyzvtxdkzMo1 +QIngkgD/Wj9T26vjvHOcNw5lYPTncbUsIsIE9ERprBO7OV+rtGndKsfO6GnDFqgb96BcY10hoO 5oHs= X-Google-Smtp-Source: AGHT+IGgnPZosJsxkryP7wd6hhRRfBOsvVRx0ahS3t6fNKP3NUw7efGHioXEx/i6cy0Ids1IjhOgAA== X-Received: by 2002:a05:6102:21ba:b0:46d:372a:d114 with SMTP id i26-20020a05610221ba00b0046d372ad114mr683387vsb.16.1707261619181; Tue, 06 Feb 2024 15:20:19 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCXnZ8LT2OnrNb13AB1D0zIynOW9qnioXJFSJT76IUOjlxBnPXn/NvL1FVyd66yZpoe5+pGoGOH+p/JsLCcc4LI4xgmvcD3tZycVEx/wAKl8dDOg From: Hao Xiang To: qemu-devel@nongnu.org, farosas@suse.de, peterx@redhat.com Cc: Hao Xiang Subject: [PATCH 5/6] migration/multifd: Enable zero page checking from multifd threads. Date: Tue, 6 Feb 2024 23:19:07 +0000 Message-Id: <20240206231908.1792529-6-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240206231908.1792529-1-hao.xiang@bytedance.com> References: <20240206231908.1792529-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::e2d; envelope-from=hao.xiang@bytedance.com; helo=mail-vs1-xe2d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1707261670083100003 Content-Type: text/plain; charset="utf-8" This change adds a dedicated handler for MigrationOps::ram_save_target_page= in multifd live migration. Now zero page checking can be done in the multifd t= hreads and this becomes the default configuration. We still provide backward compa= tibility where zero page checking is done from the migration main thread. Signed-off-by: Hao Xiang --- migration/multifd.c | 3 ++- migration/ram.c | 49 ++++++++++++++++++++++++++++++++++++--------- 2 files changed, 42 insertions(+), 10 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index c031f947c7..c6833ccb07 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -13,6 +13,7 @@ #include "qemu/osdep.h" #include "qemu/cutils.h" #include "qemu/rcu.h" +#include "qemu/cutils.h" #include "exec/target_page.h" #include "sysemu/sysemu.h" #include "exec/ramblock.h" @@ -458,7 +459,6 @@ static int multifd_send_pages(void) p->packet_num =3D multifd_send_state->packet_num++; multifd_send_state->pages =3D p->pages; p->pages =3D pages; - qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); =20 @@ -733,6 +733,7 @@ static void *multifd_send_thread(void *opaque) if (p->pending_job) { uint64_t packet_num =3D p->packet_num; uint32_t flags; + p->normal_num =3D 0; p->zero_num =3D 0; =20 diff --git a/migration/ram.c b/migration/ram.c index d5b7cd5ac2..e6742c9593 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1252,6 +1252,10 @@ static int ram_save_page(RAMState *rs, PageSearchSta= tus *pss) =20 static int ram_save_multifd_page(RAMBlock *block, ram_addr_t offset) { + assert(migrate_multifd()); + assert(!migrate_compress()); + assert(!migration_in_postcopy()); + if (multifd_queue_page(block, offset) < 0) { return -1; } @@ -2043,7 +2047,6 @@ static bool save_compress_page(RAMState *rs, PageSear= chStatus *pss, */ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss) { - RAMBlock *block =3D pss->block; ram_addr_t offset =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; int res; =20 @@ -2059,17 +2062,40 @@ static int ram_save_target_page_legacy(RAMState *rs= , PageSearchStatus *pss) return 1; } =20 + return ram_save_page(rs, pss); +} + +/** + * ram_save_target_page_multifd: save one target page + * + * Returns the number of pages written + * + * @rs: current RAM state + * @pss: data about the page we want to send + */ +static int ram_save_target_page_multifd(RAMState *rs, PageSearchStatus *ps= s) +{ + RAMBlock *block =3D pss->block; + ram_addr_t offset =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; + + /* Multifd is not compatible with old compression. */ + assert(!migrate_compress()); + + /* Multifd is not compabible with postcopy. */ + assert(!migration_in_postcopy()); + /* - * Do not use multifd in postcopy as one whole host page should be - * placed. Meanwhile postcopy requires atomic update of pages, so even - * if host page size =3D=3D guest page size the dest guest during run = may - * still see partially copied pages which is data corruption. + * Backward compatibility support. While using multifd live + * migration, we still need to handle zero page checking on the + * migration main thread. */ - if (migrate_multifd() && !migration_in_postcopy()) { - return ram_save_multifd_page(block, offset); + if (!migrate_multifd_zero_page()) { + if (save_zero_page(rs, pss, offset)) { + return 1; + } } =20 - return ram_save_page(rs, pss); + return ram_save_multifd_page(block, offset); } =20 /* Should be called before sending a host page */ @@ -2981,7 +3007,12 @@ static int ram_save_setup(QEMUFile *f, void *opaque) } =20 migration_ops =3D g_malloc0(sizeof(MigrationOps)); - migration_ops->ram_save_target_page =3D ram_save_target_page_legacy; + + if (migrate_multifd()) { + migration_ops->ram_save_target_page =3D ram_save_target_page_multi= fd; + } else { + migration_ops->ram_save_target_page =3D ram_save_target_page_legac= y; + } =20 bql_unlock(); ret =3D multifd_send_sync_main(); --=20 2.30.2 From nobody Tue Nov 26 13:24:07 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1707261696; cv=none; d=zohomail.com; s=zohoarc; b=f0l9DuvC1Z4f+CTdNjfzXghstDbMjvA4/iTnQrhIQwnyYVeNgzpL15B5Xm2hLRy14gN2q0AANDAfiqJ4c+1n5lE9MJXx9NjYINxWW/ySvnD+ZmyKMELtJS1Nf8iLw/pGTz+mW6ay+lTWLsEPz48p18UrcEG2w0vXKn0jB1tz8uE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1707261696; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fpkOjImJK+FSoyIVEziU9Gl/0F3Bqogyd5w1Dik58Qw=; b=MEyHJuNSAA2yIKN3eoAau3qhzw2EKntGyLWRnBH2pi8Y+pENty0ObwQPXJ40VD9epP8nyqPpHx8HAfPYCsb+GYBKz//hcBBpz5wfHm2mUJvXF35kri+hb5KaUkkUk/wVRj5mKMDk8Yq1JhglKn+Cza+wMUGfZNzShYDqQgPMERY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1707261696563282.07041241942534; Tue, 6 Feb 2024 15:21:36 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rXUjv-0006DL-DY; Tue, 06 Feb 2024 18:20:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rXUjt-0006CF-0h for qemu-devel@nongnu.org; Tue, 06 Feb 2024 18:20:29 -0500 Received: from mail-ua1-x92e.google.com ([2607:f8b0:4864:20::92e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rXUjm-0005nf-Nj for qemu-devel@nongnu.org; Tue, 06 Feb 2024 18:20:28 -0500 Received: by mail-ua1-x92e.google.com with SMTP id a1e0cc1a2514c-7d5bddb0f4cso2286941241.2 for ; Tue, 06 Feb 2024 15:20:22 -0800 (PST) Received: from n231-230-216.byted.org ([130.44.215.112]) by smtp.gmail.com with ESMTPSA id d11-20020a67b60b000000b0046d20ce8886sm2352vsm.1.2024.02.06.15.20.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Feb 2024 15:20:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1707261621; x=1707866421; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fpkOjImJK+FSoyIVEziU9Gl/0F3Bqogyd5w1Dik58Qw=; b=hPC2a57WoIZ1Z9vtAcwWSvzVvdosl0KNvpPV8NV+jShrEFi1R+ARl3VVnN6CleDcI4 ZA1oJy8fmIeEzeQtUr8EMkb60bJVSUD2Om9lwT/4NFgOLhnAJ4Y2xzFxc506n9zQdeht jtRjyuPF0eCZJm8BoGMFIHbVAvMgBEO4s1NGDzX1QaPF5vxdxRYReEeGpVf15GsNhLcE vsinzgagxnoGIfZNLozIt2lbLMfvhmlv+qRNs6GW3VnG4mjXIeempW3Jxxwb2BGA/x+U +FciptfHzF/qc/Pz2TnMEvm9vF5ctINWXx0J5oRcS759Rxx4Qvd/IDMrQXbqK4E9VbES kcpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707261621; x=1707866421; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fpkOjImJK+FSoyIVEziU9Gl/0F3Bqogyd5w1Dik58Qw=; b=MgKhDMhpMG5EO6SPV45ZoJSMZfSzh0MqRaQb75kn9XJ3qSVyPw1bzsAJPn1e/9vDm0 ptzjbBUNPqg7tV8zo6OsmWvuLW5kgQcMMejKDYVWlHb3eaHchuot+dhCgN6mTOFk7m0l CcIdpxQshpNwUEOkh5nIVY5OOp6l6UkR8BmDHph3b2sJJ902zbWgaFYOqzXGKcJBJZh4 9dwhA5W1vZ8oiJ/GhsHD7LrGYgAoq4KYtd+uIMAUURxLAwUouuj/eEusucpsrujF6qmK 438aQ9rmouoVvkN9CkUqs5o3TdQXin/89bt6/0L9dGFL5tX1dIuUJvuUySP6hFrzCez9 vvcQ== X-Gm-Message-State: AOJu0YzPLMDInP+/3XwRbpysuZMnkZsk1dhQYd1VcBqxMb6Ro61bNSUh 4JQXgIFsE1oCVt0+KxWYf9MSpEpS0o0VPO5/rIeEYC017hlcdBRBTIgkVy56tqnv3GlJxS9Ax59 iErg= X-Google-Smtp-Source: AGHT+IGK6v7quUmYPNzLqEgzOsHH+O6f8yujk+X1Lu/Lk3iHOgStbsXDt4H1srKfgcLSJymCPFH14g== X-Received: by 2002:a05:6102:5f5a:b0:46d:1785:426c with SMTP id il26-20020a0561025f5a00b0046d1785426cmr1177438vsb.0.1707261620891; Tue, 06 Feb 2024 15:20:20 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCWzKqLpi2f3ZEutclyNlE0g23N/egQMDhhywflO2Ht6Nv0s2HiyK5SfnRXykAwQ7Q7CInDnvCQxprJQZJJ9+JaQ2bDJ2edPLHpbe0DZcrYUHJjr From: Hao Xiang To: qemu-devel@nongnu.org, farosas@suse.de, peterx@redhat.com Cc: Hao Xiang Subject: [PATCH 6/6] migration/multifd: Add a new migration test case for legacy zero page checking. Date: Tue, 6 Feb 2024 23:19:08 +0000 Message-Id: <20240206231908.1792529-7-hao.xiang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240206231908.1792529-1-hao.xiang@bytedance.com> References: <20240206231908.1792529-1-hao.xiang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::92e; envelope-from=hao.xiang@bytedance.com; helo=mail-ua1-x92e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1707261698192100003 Content-Type: text/plain; charset="utf-8" Now that zero page checking is done on the multifd sender threads by default, we still provide an option for backward compatibility. This change adds a qtest migration test case to set the multifd-zero-page option to false and run multifd migration with zero page checking on the migration main thread. Signed-off-by: Hao Xiang --- tests/qtest/migration-test.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 7675519cfa..2c13df04c3 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -2621,6 +2621,15 @@ test_migrate_precopy_tcp_multifd_start(QTestState *f= rom, return test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); } =20 +static void * +test_migrate_precopy_tcp_multifd_start_zero_page_legacy(QTestState *from, + QTestState *to) +{ + test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); + migrate_set_parameter_bool(from, "multifd-zero-page", false); + return NULL; +} + static void * test_migrate_precopy_tcp_multifd_zlib_start(QTestState *from, QTestState *to) @@ -2652,6 +2661,21 @@ static void test_multifd_tcp_none(void) test_precopy_common(&args); } =20 +static void test_multifd_tcp_zero_page_legacy(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "defer", + .start_hook =3D test_migrate_precopy_tcp_multifd_start_zero_page_l= egacy, + /* + * Multifd is more complicated than most of the features, it + * directly takes guest page buffers when sending, make sure + * everything will work alright even if guest page is changing. + */ + .live =3D true, + }; + test_precopy_common(&args); +} + static void test_multifd_tcp_zlib(void) { MigrateCommon args =3D { @@ -3550,6 +3574,8 @@ int main(int argc, char **argv) } migration_test_add("/migration/multifd/tcp/plain/none", test_multifd_tcp_none); + migration_test_add("/migration/multifd/tcp/plain/zero_page_legacy", + test_multifd_tcp_zero_page_legacy); migration_test_add("/migration/multifd/tcp/plain/cancel", test_multifd_tcp_cancel); migration_test_add("/migration/multifd/tcp/plain/zlib", --=20 2.30.2