From nobody Wed May 15 18:34:49 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1681739250; cv=none; d=zohomail.com; s=zohoarc; b=Vig91Awwy1YllOS2diToj9fycrzhC5fG/nlLpYtEXoO08qSo4YoqHkW2zqT2gfpioPSK1nDro4IXY2RzldnITBVQPdD50M8r1OehGVnZGqg4LLU555dVX+HeKZTv8EVA7Ij0thqp4/e5pVpfiB52O6phA/18t3JstdDxlfZBcJo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1681739250; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5aTPS6VfM9zbhLN8RqFqqE60eJjlf5FABMs+Ccxoxlc=; b=adNircWYG7uGLq8hAlm1Jlez+NiG0Uq5LYYCIZaNO7sp9g4mMhGYy/mBePRh2V+JsSumVESi7bcLn4zJj02DgXpDyPI+QCQzrDMJx+cIPOrea8AUC9ExfTrMgmFAFRgO+TTarEoD3Etzqhd5FMZp1ys8xijQp3EyT7VOuXRPcGY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1681739250520811.9210063818018; Mon, 17 Apr 2023 06:47:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1poPCL-0006Hh-DO; Mon, 17 Apr 2023 09:47:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1poKPX-0001Bz-Ru for qemu-devel@nongnu.org; Mon, 17 Apr 2023 04:40:31 -0400 Received: from mga14.intel.com ([192.55.52.115]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1poKPV-0003jJ-6F for qemu-devel@nongnu.org; Mon, 17 Apr 2023 04:40:31 -0400 Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Apr 2023 01:40:24 -0700 Received: from unknown (HELO inspur05.sh.intel.com) ([10.112.227.25]) by orsmga007.jf.intel.com with ESMTP; 17 Apr 2023 01:40:22 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681720829; x=1713256829; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dMNAyUtvCP4aVKY23naukDfU+KXRXMSByO/BAwzB2eY=; b=GsqDK10nLfLm+D8w+of6AgQGHVUijIr3mOxhpB+O7e/4l2WgU6J6wD7c mr0Ff1OVnjxtcFqq/iVU5mfKPd6EbHN3YByjbtDOL9z75jZ44p1m+QwrI PIhxKltOzAJwj+YMYc1/KxcIUs9/DUS4MgS/MVg0lqt13QVamKW2UScI3 3Ue7LBJ9S6slV3lywfZB1+tnjNKgRqRI84F79M7utTOwDq1MWhSMKQPlz 1Emu//Dgmug55E61u9eoELOwyK7J1TO6x3QwCk2zH3vWOjnI8DtMpeAk5 45rK9Qr6dSezEL8qsqM8GDsMnHgNHOu1s1SIn3vhDdKVnPv5Ygy4VsY1K A==; X-IronPort-AV: E=McAfee;i="6600,9927,10682"; a="344838012" X-IronPort-AV: E=Sophos;i="5.99,203,1677571200"; d="scan'208";a="344838012" X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10682"; a="684086674" X-IronPort-AV: E=Sophos;i="5.99,203,1677571200"; d="scan'208";a="684086674" From: "you.chen" To: qemu-devel@nongnu.org Cc: "you.chen" , "dennis . wu" , Paolo Bonzini , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PATCH v2 1/2] migration: add build option to support qemu build with qatzip Date: Mon, 17 Apr 2023 16:39:34 +0800 Message-Id: <20230417083935.415782-2-you.chen@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230417083935.415782-1-you.chen@intel.com> References: <20230417083935.415782-1-you.chen@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=192.55.52.115; envelope-from=you.chen@intel.com; helo=mga14.intel.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Mon, 17 Apr 2023 09:47:05 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @intel.com) X-ZM-MESSAGEID: 1681739251001100001 Content-Type: text/plain; charset="utf-8" In this patch, we added qatzip build option in the meson_options.txt and me= son-buildoptons.sh to support qemu build with qatzip. If you installed qatzip and would like to use it for live migration, you co= uld use "--enable-qatzip" during configure, it will check qatzip availablil= ity from the pkg-config list (Please make sure you correctly set PKG_CONFIG= _PATH to include qatzip.pc). If you simply use "configure" or use "configure --disable-qatzip", qatzip w= ill not be enabled. Co-developed-by: dennis.wu Signed-off-by: you.chen --- meson.build | 11 ++++++++++- meson_options.txt | 2 ++ scripts/meson-buildoptions.sh | 3 +++ 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a/meson.build b/meson.build index 29f8644d6d..aa7445e29e 100644 --- a/meson.build +++ b/meson.build @@ -553,6 +553,13 @@ if have_system or have_tools endif zlib =3D dependency('zlib', required: true, kwargs: static_kwargs) =20 +qatzip =3D not_found +if not get_option('qatzip').auto() + qatzip =3D dependency('qatzip', required: get_option('qatzip'), + method: 'pkg-config', + static: enable_static) +endif + libaio =3D not_found if not get_option('linux_aio').auto() or have_block libaio =3D cc.find_library('aio', has_headers: ['libaio.h'], @@ -1863,6 +1870,7 @@ config_host_data.set('CONFIG_LIBISCSI', libiscsi.foun= d()) config_host_data.set('CONFIG_LIBNFS', libnfs.found()) config_host_data.set('CONFIG_LIBSSH', libssh.found()) config_host_data.set('CONFIG_LINUX_AIO', libaio.found()) +config_host_data.set('CONFIG_QATZIP', qatzip.found()) config_host_data.set('CONFIG_LINUX_IO_URING', linux_io_uring.found()) config_host_data.set('CONFIG_LIBPMEM', libpmem.found()) config_host_data.set('CONFIG_NUMA', numa.found()) @@ -3339,7 +3347,7 @@ libmigration =3D static_library('migration', sources:= migration_files + genh, name_suffix: 'fa', build_by_default: false) migration =3D declare_dependency(link_with: libmigration, - dependencies: [zlib, qom, io]) + dependencies: [qatzip, zlib, qom, io]) softmmu_ss.add(migration) =20 block_ss =3D block_ss.apply(config_host, strict: false) @@ -3986,6 +3994,7 @@ summary_info +=3D {'vde support': vde} summary_info +=3D {'netmap support': have_netmap} summary_info +=3D {'l2tpv3 support': have_l2tpv3} summary_info +=3D {'Linux AIO support': libaio} +summary_info +=3D {'qatzip compress support': qatzip} summary_info +=3D {'Linux io_uring support': linux_io_uring} summary_info +=3D {'ATTR/XATTR support': libattr} summary_info +=3D {'RDMA support': rdma} diff --git a/meson_options.txt b/meson_options.txt index fc9447d267..ef6d639876 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -191,6 +191,8 @@ option('smartcard', type : 'feature', value : 'auto', description: 'CA smartcard emulation support') option('snappy', type : 'feature', value : 'auto', description: 'snappy compression support') +option('qatzip', type : 'feature', value : 'auto', + description: 'qatzip compress support') option('spice', type : 'feature', value : 'auto', description: 'Spice server support') option('spice_protocol', type : 'feature', value : 'auto', diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index 009fab1515..84d110197d 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -152,6 +152,7 @@ meson_options_help() { printf "%s\n" ' slirp-smbd use smbd (at path --smbd=3D*) in slirp = networking' printf "%s\n" ' smartcard CA smartcard emulation support' printf "%s\n" ' snappy snappy compression support' + printf "%s\n" ' qatzip qatzip compression support' printf "%s\n" ' sndio sndio sound support' printf "%s\n" ' sparse sparse checker' printf "%s\n" ' spice Spice server support' @@ -332,6 +333,8 @@ _meson_option_parse() { --disable-libvduse) printf "%s" -Dlibvduse=3Ddisabled ;; --enable-linux-aio) printf "%s" -Dlinux_aio=3Denabled ;; --disable-linux-aio) printf "%s" -Dlinux_aio=3Ddisabled ;; + --enable-qatzip) printf "%s" -Dqatzip=3Denabled ;; + --disable-qatzip) printf "%s" -Dqatzip=3Ddisabled ;; --enable-linux-io-uring) printf "%s" -Dlinux_io_uring=3Denabled ;; --disable-linux-io-uring) printf "%s" -Dlinux_io_uring=3Ddisabled ;; --enable-live-block-migration) printf "%s" -Dlive_block_migration=3Den= abled ;; --=20 2.27.0 From nobody Wed May 15 18:34:49 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1681739261; cv=none; d=zohomail.com; s=zohoarc; b=RPZ2hDjcZ93/Uv0x4qdLVqpLCrCFQIzQOP+HI70ES33zhEa1VViK6czfhDH2poJcTs4sRlVAoPYWwIhM00Rbv1t4gBwATJibGS0mlwb5cTjRwt/FC/Q/sqlH8QZierzYqffBkP9tr2D4B7n29qzkjuLc/0yHmVkDE4ycTfxKNK4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1681739261; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=EcxwC1M15hV89TDq9xt4W+3BSaTPUu2xTHgDtg+jdsY=; b=NC6iKD4fD9zQwJr5UvN6K5zotG6h6X7krNhQ73F3eQ00a6kT6oOCwmmw+vWu0I4wvH/Xe6ctAx2IhKdFqnLrMBkPFBBFaa/V/VVOvSlb3zifU46YliolVox8R+kTIVzy6WRjoNI4TUmdwa73zcdk0hbusMjIEgXt/M+jhN0FSUA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1681739261662287.6473047205136; Mon, 17 Apr 2023 06:47:41 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1poPCP-0006Ln-M3; Mon, 17 Apr 2023 09:47:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1poKPb-0001CM-86 for qemu-devel@nongnu.org; Mon, 17 Apr 2023 04:40:35 -0400 Received: from mga14.intel.com ([192.55.52.115]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1poKPX-0003jU-HY for qemu-devel@nongnu.org; Mon, 17 Apr 2023 04:40:35 -0400 Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Apr 2023 01:40:28 -0700 Received: from unknown (HELO inspur05.sh.intel.com) ([10.112.227.25]) by orsmga007.jf.intel.com with ESMTP; 17 Apr 2023 01:40:25 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681720831; x=1713256831; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=XJT7PrES2py6TL0IgVV6+zDrhUZ3bUm6yHq68nPuY+c=; b=V9VZ3WW+B4VQu8mJqmS1po+Ln/lnlvJJO4yCbZrSyv3SufR6c0j+5JmW 6MCBuXsQJNtrzrXwzDZWmeKrE8gw0bgeqXAlPCdjyAeoWrRgXh2tZB6iF DHLJCFjOe/XhRiQmgIU2Jg1jsuoQvIxdrTFTOoQzs3+RzpnMjghHLLg7z BJf+wgMWUC5Ac+0OsVeAcJOz/oSBH7JaYeQNb6yhuFu9H9mq9yFruF1cP SaQj0vniOa8n60hsf7EOC+ZOhmmhH9PrRP1Ag0+K3/F2nMBBy0vhUrbyr SDvAlQfvF7C/r+uHzdg8cMQBFBQfy/6jfVmMdsFLqVrKHo+h/NWWxboSz g==; X-IronPort-AV: E=McAfee;i="6600,9927,10682"; a="344838021" X-IronPort-AV: E=Sophos;i="5.99,203,1677571200"; d="scan'208";a="344838021" X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10682"; a="684086683" X-IronPort-AV: E=Sophos;i="5.99,203,1677571200"; d="scan'208";a="684086683" From: "you.chen" To: qemu-devel@nongnu.org Cc: "you.chen" , "dennis . wu" , Juan Quintela , "Dr. David Alan Gilbert" , Eric Blake , Markus Armbruster Subject: [PATCH v2 2/2] migration: add support for qatzip compression when doing live migration Date: Mon, 17 Apr 2023 16:39:35 +0800 Message-Id: <20230417083935.415782-3-you.chen@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230417083935.415782-1-you.chen@intel.com> References: <20230417083935.415782-1-you.chen@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=192.55.52.115; envelope-from=you.chen@intel.com; helo=mga14.intel.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Mon, 17 Apr 2023 09:47:05 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @intel.com) X-ZM-MESSAGEID: 1681739263255100002 Content-Type: text/plain; charset="utf-8" Add config and logics to use qatzip for page compression, in order to suppo= rt qatzip compression better, we collect multipe pages together to do qatzi= p compression for best performance. And we use compile option CONFIG_QATZIP to determine whether should qatzip = related code be compiled or not. Co-developed-by: dennis.wu Signed-off-by: you.chen --- migration/migration-hmp-cmds.c | 4 + migration/migration.c | 22 ++ migration/migration.h | 1 + migration/qemu-file.c | 15 +- migration/qemu-file.h | 5 + migration/ram.c | 675 +++++++++++++++++++++++++++++---- qapi/migration.json | 8 +- 7 files changed, 658 insertions(+), 72 deletions(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 72519ea99f..2608f67469 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -479,6 +479,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDi= ct *qdict) p->has_compress_threads =3D true; visit_type_uint8(v, param, &p->compress_threads, &err); break; + case MIGRATION_PARAMETER_COMPRESS_WITH_QAT: + p->has_compress_with_qat =3D true; + visit_type_bool(v, param, &p->compress_with_qat, &err); + break; case MIGRATION_PARAMETER_COMPRESS_WAIT_THREAD: p->has_compress_wait_thread =3D true; visit_type_bool(v, param, &p->compress_wait_thread, &err); diff --git a/migration/migration.c b/migration/migration.c index ae2025d9d8..15ab6ecf57 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -962,6 +962,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error= **errp) params->compress_level =3D s->parameters.compress_level; params->has_compress_threads =3D true; params->compress_threads =3D s->parameters.compress_threads; + params->has_compress_with_qat =3D true; + params->compress_with_qat =3D s->parameters.compress_with_qat; params->has_compress_wait_thread =3D true; params->compress_wait_thread =3D s->parameters.compress_wait_thread; params->has_decompress_threads =3D true; @@ -1678,6 +1680,10 @@ static void migrate_params_test_apply(MigrateSetPara= meters *params, dest->compress_threads =3D params->compress_threads; } =20 + if (params->has_compress_with_qat) { + dest->compress_with_qat =3D params->compress_with_qat; + } + if (params->has_compress_wait_thread) { dest->compress_wait_thread =3D params->compress_wait_thread; } @@ -1775,6 +1781,10 @@ static void migrate_params_apply(MigrateSetParameter= s *params, Error **errp) s->parameters.compress_threads =3D params->compress_threads; } =20 + if (params->has_compress_with_qat) { + s->parameters.compress_with_qat =3D params->compress_with_qat; + } + if (params->has_compress_wait_thread) { s->parameters.compress_wait_thread =3D params->compress_wait_threa= d; } @@ -2620,6 +2630,15 @@ bool migrate_use_compression(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_COMPRESS]; } =20 +bool migrate_compress_with_qat(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return migrate_use_compression() && s->parameters.compress_with_qat; +} + int migrate_compress_level(void) { MigrationState *s; @@ -4451,6 +4470,8 @@ static Property migration_properties[] =3D { DEFINE_PROP_UINT8("x-compress-threads", MigrationState, parameters.compress_threads, DEFAULT_MIGRATE_COMPRESS_THREAD_COUNT), + DEFINE_PROP_BOOL("x-compress-with-qat", MigrationState, + parameters.compress_with_qat, false), DEFINE_PROP_BOOL("x-compress-wait-thread", MigrationState, parameters.compress_wait_thread, true), DEFINE_PROP_UINT8("x-decompress-threads", MigrationState, @@ -4580,6 +4601,7 @@ static void migration_instance_init(Object *obj) params->has_compress_level =3D true; params->has_compress_threads =3D true; params->has_compress_wait_thread =3D true; + params->has_compress_with_qat =3D true; params->has_decompress_threads =3D true; params->has_throttle_trigger_threshold =3D true; params->has_cpu_throttle_initial =3D true; diff --git a/migration/migration.h b/migration/migration.h index 2da2f8a164..fcb5a2f670 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -443,6 +443,7 @@ bool migrate_use_return_path(void); uint64_t ram_get_total_transferred_pages(void); =20 bool migrate_use_compression(void); +bool migrate_compress_with_qat(void); int migrate_compress_level(void); int migrate_compress_threads(void); int migrate_compress_wait_thread(void); diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 102ab3b439..29d288e82e 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -26,12 +26,18 @@ #include "qemu/madvise.h" #include "qemu/error-report.h" #include "qemu/iov.h" +#include "qemu/units.h" #include "migration.h" #include "qemu-file.h" #include "trace.h" #include "qapi/error.h" =20 -#define IO_BUF_SIZE 32768 +#ifdef CONFIG_QATZIP +#define IO_BUF_SIZE (512 * KiB) +#else +#define IO_BUF_SIZE (32 * KiB) +#endif + #define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 64) =20 struct QEMUFile { @@ -514,7 +520,7 @@ static int add_to_iovec(QEMUFile *f, const uint8_t *buf= , size_t size, return 0; } =20 -static void add_buf_to_iovec(QEMUFile *f, size_t len) +void add_buf_to_iovec(QEMUFile *f, size_t len) { if (!add_to_iovec(f, f->buf + f->buf_index, len, false)) { f->buf_index +=3D len; @@ -524,6 +530,11 @@ static void add_buf_to_iovec(QEMUFile *f, size_t len) } } =20 +uint8_t *qemu_get_pos(QEMUFile *f) +{ + return f->buf + f->buf_index; +} + void qemu_put_buffer_async(QEMUFile *f, const uint8_t *buf, size_t size, bool may_free) { diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 9d0155a2a1..391e02f197 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -25,6 +25,9 @@ #ifndef MIGRATION_QEMU_FILE_H #define MIGRATION_QEMU_FILE_H =20 +#ifdef CONFIG_QATZIP +#include +#endif #include #include "exec/cpu-common.h" #include "io/channel.h" @@ -108,6 +111,8 @@ bool qemu_file_is_writable(QEMUFile *f); =20 #include "migration/qemu-file-types.h" =20 +void add_buf_to_iovec(QEMUFile *f, size_t len); +uint8_t *qemu_get_pos(QEMUFile *f); size_t qemu_peek_buffer(QEMUFile *f, uint8_t **buf, size_t size, size_t of= fset); size_t qemu_get_buffer_in_place(QEMUFile *f, uint8_t **buf, size_t size); ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream, diff --git a/migration/ram.c b/migration/ram.c index 96e8a19a58..a9dfca748b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -117,6 +117,20 @@ static void __attribute__((constructor)) init_cpu_flag= (void) =20 XBZRLECacheStats xbzrle_counters; =20 +/* define the max page number to compress together */ +#define MULTI_PAGE_NUM 64 +#define COMP_BUF_SIZE (TARGET_PAGE_SIZE * MULTI_PAGE_NUM * 2) +#define DECOMP_BUF_SIZE (TARGET_PAGE_SIZE * MULTI_PAGE_NUM) + +typedef struct MultiPageAddr { + /* real pages that will compress together */ + uint64_t pages; + /* the last index of the addr*/ + uint64_t last_idx; + /* each address might contain contineous pages*/ + uint64_t addr[MULTI_PAGE_NUM]; +} MultiPageAddr; + /* used by the search for pages to send */ struct PageSearchStatus { /* The migration channel used for a specific host page */ @@ -127,6 +141,12 @@ struct PageSearchStatus { RAMBlock *block; /* Current page to search from */ unsigned long page; + /* + * multi page search from current page + * for compress together with qatzip + * stream APIs + */ + MultiPageAddr mpa; /* Set once we wrap around */ bool complete_round; /* Whether we're sending a host page */ @@ -506,6 +526,15 @@ struct CompressParam { /* internally used fields */ z_stream stream; uint8_t *originbuf; + +#ifdef CONFIG_QATZIP + /*multi page address for compression*/ + MultiPageAddr mpa; + QzSession_T qzsess; + uint8_t *decompbuf; + uint8_t *compbuf; + /* QzStream_T qzstream; */ +#endif }; typedef struct CompressParam CompressParam; =20 @@ -518,6 +547,15 @@ struct DecompressParam { uint8_t *compbuf; int len; z_stream stream; + RAMBlock *block; + +#ifdef CONFIG_QATZIP + /* decompress multi pages with qzlib*/ + QzSession_T qzsess; + /* QzStream_T qzstream; */ + uint8_t *decompbuf; /* buffer after decompress */ + MultiPageAddr mpa; /* destination */ +#endif }; typedef struct DecompressParam DecompressParam; =20 @@ -541,6 +579,33 @@ static int ram_save_host_page_urgent(PageSearchStatus = *pss); static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, ram_addr_t offset, uint8_t *source_buf); =20 +#ifdef CONFIG_QATZIP +static inline uint64_t multi_page_addr_get_one(MultiPageAddr *mpa, uint64_= t idx); +static inline void multi_page_addr_put_one(MultiPageAddr *mpa, + uint64_t offset, + uint64_t pages); + +static void pss_find_next_multiple_dirty(PageSearchStatus *pss); +static bool migration_bitmap_clear_dirty_multiple(RAMState *rs, + RAMBlock *rb, + MultiPageAddr *mpa); +static size_t save_page_header_multiple(RAMBlock *block, QEMUFile *f, + MultiPageAddr *mpa); +static inline void set_compress_params_multiple(CompressParam *param, RAMB= lock *block, + MultiPageAddr *pmpa); +static bool save_compress_page_multiple(RAMState *rs, RAMBlock *block, + MultiPageAddr *mpa); +static int ram_save_target_page_multiple(RAMState *rs, PageSearchStatus *p= ss); +static int ram_save_host_page_multiple(RAMState *rs, PageSearchStatus *pss= ); +static void do_compress_ram_page_multiple(QEMUFile *f, QzSession_T *qzsess, + uint8_t *decompbuf, uint8_t *compbuf, + RAMBlock *block, MultiPageAddr *mpa); +static bool do_decompress_ram_page_multiple(QzSession_T *qzsess, uint8_t *= compbuf, + uint8_t *decompbuf, RAMBlock *block, int len, MultiPageAddr *mpa); +static int qemu_get_multiple(QEMUFile *f, MultiPageAddr *mpa, + uint8_t *compbuf, int *pbytes); +#endif + /* NOTE: page is the PFN not real ram_addr_t. */ static void pss_init(PageSearchStatus *pss, RAMBlock *rb, ram_addr_t page) { @@ -574,12 +639,21 @@ static void *do_data_compress(void *opaque) param->block =3D NULL; qemu_mutex_unlock(¶m->mutex); =20 - zero_page =3D do_compress_ram_page(param->file, ¶m->stream, - block, offset, param->originb= uf); +#ifdef CONFIG_QATZIP + if (migrate_compress_with_qat()) { + do_compress_ram_page_multiple(param->file, ¶m->qzsess, + param->decompbuf, param->compbuf, block, ¶m->mpa); + } else { +#endif + zero_page =3D do_compress_ram_page(param->file, + ¶m->stream, block, offset, param->originbuf); + param->zero_page =3D zero_page; +#ifdef CONFIG_QATZIP + } +#endif =20 qemu_mutex_lock(&comp_done_lock); param->done =3D true; - param->zero_page =3D zero_page; qemu_cond_signal(&comp_done_cond); qemu_mutex_unlock(&comp_done_lock); =20 @@ -619,8 +693,19 @@ static void compress_threads_save_cleanup(void) qemu_thread_join(compress_threads + i); qemu_mutex_destroy(&comp_param[i].mutex); qemu_cond_destroy(&comp_param[i].cond); - deflateEnd(&comp_param[i].stream); - g_free(comp_param[i].originbuf); +#ifdef CONFIG_QATZIP + if (migrate_compress_with_qat()) { + qzTeardownSession(&comp_param[i].qzsess); + qzClose(&comp_param[i].qzsess); + qzFree(comp_param[i].compbuf); + qzFree(comp_param[i].decompbuf); + } else { +#endif + deflateEnd(&comp_param[i].stream); + g_free(comp_param[i].originbuf); +#ifdef CONFIG_QATZIP + } +#endif qemu_fclose(comp_param[i].file); comp_param[i].file =3D NULL; } @@ -645,6 +730,9 @@ static int compress_threads_save_setup(void) qemu_cond_init(&comp_done_cond); qemu_mutex_init(&comp_done_lock); for (i =3D 0; i < thread_count; i++) { +#ifdef CONFIG_QATZIP + if(!migrate_compress_with_qat()) { +#endif comp_param[i].originbuf =3D g_try_malloc(TARGET_PAGE_SIZE); if (!comp_param[i].originbuf) { goto exit; @@ -655,7 +743,35 @@ static int compress_threads_save_setup(void) g_free(comp_param[i].originbuf); goto exit; } - +#ifdef CONFIG_QATZIP + } else { + int ret; + QzSessionParams_T sessParam; + ret =3D qzInit(&comp_param[i].qzsess, 1); + if (ret !=3D QZ_OK && ret !=3D QZ_DUPLICATE) { + error_report("qzInit on comp thread %d failed %d!", i, ret); + goto exit; + } + ret =3D qzGetDefaults(&sessParam); + if (ret !=3D QZ_OK) { + error_report("qzGetDefaults, thread %d failed %d!", i, ret); + goto exit; + } + sessParam.comp_lvl =3D migrate_compress_level(); + sessParam.direction =3D QZ_DIR_COMPRESS; + ret =3D qzSetupSession(&comp_param[i].qzsess, &sessParam); + if (ret !=3D QZ_OK) { + error_report("qzSetupSession, thread %d failed %d!", i, ret); + goto exit; + } + comp_param[i].compbuf =3D qzMalloc(COMP_BUF_SIZE, 0, PINNED_MEM); + comp_param[i].decompbuf =3D qzMalloc(DECOMP_BUF_SIZE, 0, PINNED_ME= M); + if (!comp_param[i].compbuf || !comp_param->decompbuf) { + error_report("can't allocate from the PINNED memory!"); + goto exit; + } + } +#endif /* comp_param[i].file is just used as a dummy buffer to save data, * set its ops to empty. */ @@ -1550,7 +1666,7 @@ static inline void set_compress_params(CompressParam = *param, RAMBlock *block, param->offset =3D offset; } =20 -static int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t off= set) +static int compress_page_with_multi_thread(RAMBlock *block, MultiPageAddr = *mpa, ram_addr_t offset) { int idx, thread_count, bytes_xmit =3D -1, pages =3D -1; bool wait =3D migrate_compress_wait_thread(); @@ -1565,10 +1681,19 @@ retry: bytes_xmit =3D qemu_put_qemu_file(ms->to_dst_file, comp_param[idx].file); qemu_mutex_lock(&comp_param[idx].mutex); +#ifdef CONFIG_QATZIP + if (migrate_compress_with_qat()) { + set_compress_params_multiple(&comp_param[idx], block, mpa); + pages =3D mpa->pages; + } else { +#endif set_compress_params(&comp_param[idx], block, offset); + pages =3D 1; +#ifdef CONFIG_QATZIP + } +#endif qemu_cond_signal(&comp_param[idx].cond); qemu_mutex_unlock(&comp_param[idx].mutex); - pages =3D 1; update_compress_thread_counts(&comp_param[idx], bytes_xmit); break; } @@ -1605,8 +1730,20 @@ retry: */ static int find_dirty_block(RAMState *rs, PageSearchStatus *pss) { - /* Update pss->page for the next dirty bit in ramblock */ - pss_find_next_dirty(pss); +#ifdef CONFIG_QATZIP + if (migrate_compress_with_qat()) { + if(pss->block =3D=3D pss->last_sent_block) { + pss_find_next_multiple_dirty(pss); + } else { + pss_find_next_dirty(pss); + } + } else { +#endif + /* Update pss->page for the next dirty bit in ramblock */ + pss_find_next_dirty(pss); +#ifdef CONFIG_QATZIP + } +#endif =20 if (pss->complete_round && pss->block =3D=3D rs->last_seen_block && pss->page >=3D rs->last_page) { @@ -2325,7 +2462,7 @@ static bool save_compress_page(RAMState *rs, PageSear= chStatus *pss, return false; } =20 - if (compress_page_with_multi_thread(block, offset) > 0) { + if (compress_page_with_multi_thread(block, NULL, offset) > 0) { return true; } =20 @@ -2377,7 +2514,6 @@ static int ram_save_target_page_legacy(RAMState *rs, = PageSearchStatus *pss) if (migrate_use_multifd() && !migration_in_postcopy()) { return ram_save_multifd_page(pss->pss_channel, block, offset); } - return ram_save_page(rs, pss); } =20 @@ -2485,28 +2621,7 @@ out: return ret; } =20 -/** - * ram_save_host_page: save a whole host page - * - * Starting at *offset send pages up to the end of the current host - * page. It's valid for the initial offset to point into the middle of - * a host page in which case the remainder of the hostpage is sent. - * Only dirty target pages are sent. Note that the host page size may - * be a huge page for this block. - * - * The saving stops at the boundary of the used_length of the block - * if the RAMBlock isn't a multiple of the host page size. - * - * The caller must be with ram_state.bitmap_mutex held to call this - * function. Note that this function can temporarily release the lock, but - * when the function is returned it'll make sure the lock is still held. - * - * Returns the number of pages written or negative on error - * - * @rs: current RAM state - * @pss: data about the page we want to send - */ -static int ram_save_host_page(RAMState *rs, PageSearchStatus *pss) +static int ram_save_host_page_single(RAMState *rs, PageSearchStatus *pss) { bool page_dirty, preempt_active =3D postcopy_preempt_active(); int tmppages, pages =3D 0; @@ -2568,6 +2683,44 @@ static int ram_save_host_page(RAMState *rs, PageSear= chStatus *pss) return (res < 0 ? res : pages); } =20 +/** + * ram_save_host_page: save a whole host page + * + * Starting at *offset send pages up to the end of the current host + * page. It's valid for the initial offset to point into the middle of + * a host page in which case the remainder of the hostpage is sent. + * Only dirty target pages are sent. Note that the host page size may + * be a huge page for this block. + * + * The saving stops at the boundary of the used_length of the block + * if the RAMBlock isn't a multiple of the host page size. + * + * The caller must be with ram_state.bitmap_mutex held to call this + * function. Note that this function can temporarily release the lock, but + * when the function is returned it'll make sure the lock is still held. + * + * Returns the number of pages written or negative on error + * + * @rs: current RAM state + * @pss: data about the page we want to send + */ +static int ram_save_host_page(RAMState *rs, PageSearchStatus *pss) +{ +#ifdef CONFIG_QATZIP + if (migrate_compress_with_qat()) { + if(pss->block =3D=3D pss->last_sent_block) { + return ram_save_host_page_multiple(rs, pss); + } else { + return ram_save_host_page_single(rs, pss); + } + } else { +#endif + return ram_save_host_page_single(rs, pss); +#ifdef CONFIG_QATZIP + } +#endif +} + /** * ram_find_and_save_block: finds a dirty page and sends it to f * @@ -3275,7 +3428,6 @@ static int ram_save_setup(QEMUFile *f, void *opaque) WITH_RCU_READ_LOCK_GUARD() { qemu_put_be64(f, ram_bytes_total_with_ignored() | RAM_SAVE_FLAG_MEM_SIZE); - RAMBLOCK_FOREACH_MIGRATABLE(block) { qemu_put_byte(f, strlen(block->idstr)); qemu_put_buffer(f, (uint8_t *)block->idstr, strlen(block->idst= r)); @@ -3404,7 +3556,6 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) * because of RDMA protocol. */ ram_control_after_iterate(f, RAM_CONTROL_ROUND); - out: if (ret >=3D 0 && migration_is_setup_or_active(migrate_get_current()->state)) { @@ -3711,21 +3862,27 @@ static void *do_data_decompress(void *opaque) =20 qemu_mutex_lock(¶m->mutex); while (!param->quit) { - if (param->des) { + if (param->block) { des =3D param->des; len =3D param->len; - param->des =3D 0; qemu_mutex_unlock(¶m->mutex); - +#ifdef CONFIG_QATZIP + if (migrate_compress_with_qat()) { + do_decompress_ram_page_multiple(¶m->qzsess, param->compbuf= , param->decompbuf, + param->block, len, ¶m->mp= a); + } else { +#endif pagesize =3D TARGET_PAGE_SIZE; - ret =3D qemu_uncompress_data(¶m->stream, des, pagesize, param->compbuf, len); if (ret < 0 && migrate_get_current()->decompress_error_check) { error_report("decompress data failed"); qemu_file_set_error(decomp_file, ret); } - +#ifdef CONFIG_QATZIP + } +#endif + param->block =3D NULL; qemu_mutex_lock(&decomp_done_lock); param->done =3D true; qemu_cond_signal(&decomp_done_cond); @@ -3790,9 +3947,20 @@ static void compress_threads_load_cleanup(void) qemu_thread_join(decompress_threads + i); qemu_mutex_destroy(&decomp_param[i].mutex); qemu_cond_destroy(&decomp_param[i].cond); - inflateEnd(&decomp_param[i].stream); - g_free(decomp_param[i].compbuf); - decomp_param[i].compbuf =3D NULL; +#ifdef CONFIG_QATZIP + if (migrate_compress_with_qat()) { + qzTeardownSession(&decomp_param[i].qzsess); + qzClose(&decomp_param[i].qzsess); + qzFree(decomp_param[i].compbuf); + qzFree(decomp_param[i].decompbuf); + } else { +#endif + inflateEnd(&decomp_param[i].stream); + g_free(decomp_param[i].compbuf); + decomp_param[i].compbuf =3D NULL; +#ifdef CONFIG_QATZIP + } +#endif } g_free(decompress_threads); g_free(decomp_param); @@ -3816,11 +3984,43 @@ static int compress_threads_load_setup(QEMUFile *f) qemu_cond_init(&decomp_done_cond); decomp_file =3D f; for (i =3D 0; i < thread_count; i++) { - if (inflateInit(&decomp_param[i].stream) !=3D Z_OK) { - goto exit; +#ifdef CONFIG_QATZIP + if (!migrate_compress_with_qat()) { +#endif + if (inflateInit(&decomp_param[i].stream) !=3D Z_OK) { + goto exit; + } + decomp_param[i].compbuf =3D g_malloc0(compressBound(TARGET_PAG= E_SIZE)); +#ifdef CONFIG_QATZIP + } else { + int ret; + QzSessionParams_T sessParam; + /* call inflateInit or qzInit */ + ret =3D qzInit(&decomp_param[i].qzsess, 1); + if (ret !=3D QZ_OK && ret !=3D QZ_DUPLICATE) { + error_report("qzInit, thread %d failed %d!", i, ret); + goto exit; + } + ret =3D qzGetDefaults(&sessParam); + if (ret !=3D QZ_OK) { + error_report("qzGetDefaults, thread %d failed %d!", i, ret= ); + goto exit; + } + sessParam.direction =3D QZ_DIR_DECOMPRESS; + ret =3D qzSetupSession(&decomp_param[i].qzsess, &sessParam); + if (ret !=3D QZ_OK) { + error_report("qzSetupSession, thread %d failed %d!", i, re= t); + goto exit; + } + decomp_param[i].compbuf =3D qzMalloc(COMP_BUF_SIZE, 0, PINNED_= MEM); + decomp_param[i].decompbuf =3D + qzMalloc(DECOMP_BUF_SIZE, 0, PINNED_ME= M); + if (!decomp_param[i].compbuf || !decomp_param[i].decompbuf) { + error_report("can't allocate the decompress related buffer= !"); + goto exit; + } } - - decomp_param[i].compbuf =3D g_malloc0(compressBound(TARGET_PAGE_SI= ZE)); +#endif qemu_mutex_init(&decomp_param[i].mutex); qemu_cond_init(&decomp_param[i].cond); decomp_param[i].done =3D true; @@ -3835,7 +4035,7 @@ exit: return -1; } =20 -static void decompress_data_with_multi_threads(QEMUFile *f, +static void decompress_data_with_multi_threads(QEMUFile *f, RAMBlock *bloc= k, void *host, int len) { int idx, thread_count; @@ -3847,9 +4047,21 @@ static void decompress_data_with_multi_threads(QEMUF= ile *f, if (decomp_param[idx].done) { decomp_param[idx].done =3D false; qemu_mutex_lock(&decomp_param[idx].mutex); - qemu_get_buffer(f, decomp_param[idx].compbuf, len); - decomp_param[idx].des =3D host; - decomp_param[idx].len =3D len; +#ifdef CONFIG_QATZIP + if (migrate_compress_with_qat()) { + qemu_get_multiple(f, + &decomp_param[idx].mpa, + decomp_param[idx].compbuf, + &decomp_param[idx].len); + decomp_param[idx].block =3D block; + } else { +#endif + qemu_get_buffer(f, decomp_param[idx].compbuf, len); + decomp_param[idx].des =3D host; + decomp_param[idx].len =3D len; +#ifdef CONFIG_QATZIP + } +#endif qemu_cond_signal(&decomp_param[idx].cond); qemu_mutex_unlock(&decomp_param[idx].mutex); break; @@ -4042,7 +4254,7 @@ int ram_load_postcopy(QEMUFile *f, int channel) void *place_source =3D NULL; RAMBlock *block =3D NULL; uint8_t ch; - int len; + int len =3D 0; =20 addr =3D qemu_get_be64(f); =20 @@ -4154,14 +4366,16 @@ int ram_load_postcopy(QEMUFile *f, int channel) } break; case RAM_SAVE_FLAG_COMPRESS_PAGE: - tmp_page->all_zero =3D false; - len =3D qemu_get_be32(f); - if (len < 0 || len > compressBound(TARGET_PAGE_SIZE)) { - error_report("Invalid compressed data length: %d", len); - ret =3D -EINVAL; - break; + if(!migrate_compress_with_qat()) { + tmp_page->all_zero =3D false; + len =3D qemu_get_be32(f); + if (len < 0 || len > compressBound(TARGET_PAGE_SIZE)) { + error_report("Invalid compressed data length: %d", len= ); + ret =3D -EINVAL; + break; + } } - decompress_data_with_multi_threads(f, page_buffer, len); + decompress_data_with_multi_threads(f, block, page_buffer, len); break; =20 case RAM_SAVE_FLAG_EOS: @@ -4303,10 +4517,10 @@ static int ram_load_precopy(QEMUFile *f) ret =3D -EINVAL; break; } - + RAMBlock *block =3D NULL; if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_XBZRLE)) { - RAMBlock *block =3D ram_block_from_stream(mis, f, flags, + block =3D ram_block_from_stream(mis, f, flags, RAM_CHANNEL_PRECOPY); =20 host =3D host_from_ram_block_offset(block, addr); @@ -4350,7 +4564,6 @@ static int ram_load_precopy(QEMUFile *f) /* Synchronize RAM block list */ total_ram_bytes =3D addr; while (!ret && total_ram_bytes) { - RAMBlock *block; char id[256]; ram_addr_t length; =20 @@ -4418,13 +4631,15 @@ static int ram_load_precopy(QEMUFile *f) break; =20 case RAM_SAVE_FLAG_COMPRESS_PAGE: - len =3D qemu_get_be32(f); - if (len < 0 || len > compressBound(TARGET_PAGE_SIZE)) { - error_report("Invalid compressed data length: %d", len); - ret =3D -EINVAL; - break; + if (!migrate_compress_with_qat()) { + len =3D qemu_get_be32(f); + if (len < 0 || len > compressBound(TARGET_PAGE_SIZE)) { + error_report("Invalid compressed data length: %d", len= ); + ret =3D -EINVAL; + break; + } } - decompress_data_with_multi_threads(f, host, len); + decompress_data_with_multi_threads(f, block, host, len); break; =20 case RAM_SAVE_FLAG_XBZRLE: @@ -4737,3 +4952,325 @@ void ram_mig_init(void) register_savevm_live("ram", 0, 4, &savevm_ram_handlers, &ram_state); ram_block_notifier_add(&ram_mig_ram_notifier); } + +#ifdef CONFIG_QATZIP +static inline uint64_t +multi_page_addr_get_one(MultiPageAddr *mpa, uint64_t idx) +{ + return mpa->addr[idx]; +} + +static inline void multi_page_addr_put_one(MultiPageAddr *mpa, + uint64_t offset, + uint64_t pages) +{ + uint64_t idx =3D mpa->last_idx; + uint64_t *addr =3D mpa->addr; + + /* The lower TARGET_PAGE_BITS:continuous pages number */ + addr[idx] =3D (offset << TARGET_PAGE_BITS) | pages; + mpa->last_idx =3D idx + 1; + mpa->pages +=3D pages; +} + +/** + * pss_find_next_multiple_dirty: find next multiple pages of current rambl= ock + * + * This function is similar to pss_find_next_dirty, and will update pss->p= age + * to point to the next MULTI_PAGE_NUM dirty page index within the rambloc= k to migrate, + * or the end of ramblock when nothing found. + * + * @pss: the current page search status + */ +static void pss_find_next_multiple_dirty(PageSearchStatus *pss) +{ + RAMBlock *rb =3D pss->block; + unsigned long size =3D rb->used_length >> TARGET_PAGE_BITS; + unsigned long *bitmap =3D rb->bmap; + uint64_t end, pages =3D 0; + + pss->mpa.last_idx =3D 0; + pss->mpa.pages =3D 0; + uint64_t start =3D pss->page; + if (ramblock_is_ignored(rb)) { + /* Points directly to the end, so we know no dirty page */ + pss->page =3D size; + return; + } + + /* + * If during sending a host page, only look for dirty pages within the + * current host page being send. + */ + if (pss->host_page_sending) { + assert(pss->host_page_end); + size =3D MIN(size, pss->host_page_end); + } + + if (start >=3D size) { + pss->page =3D size; + return; + } + + /* from the start pos to search the dirty bitmap*/ + while ((pss->mpa.pages < MULTI_PAGE_NUM)) { + start =3D find_next_bit(bitmap, size, start); + /* if start>=3D size mean can't find any more*/ + if (start >=3D size) { + pss->page =3D size; + return; + } + + /* + * find the next clean page, + * end-start =3D contineous dirty pages + */ + end =3D find_next_zero_bit(bitmap, size, start); + pages =3D end - start; + /* + * if total pages over MULTI_PAGE_NUM + * keep last mpa entry pages is: + * MULTI_PAGE_NUM - mpa->pages + */ + if ((pss->mpa.pages + pages) > MULTI_PAGE_NUM) { + pages =3D MULTI_PAGE_NUM - pss->mpa.pages; + } + multi_page_addr_put_one(&pss->mpa, start, pages); + start +=3D pages; + } + pss->page =3D start; +} + +static bool +migration_bitmap_clear_dirty_multiple(RAMState *rs, + RAMBlock *rb, + MultiPageAddr *mpa) +{ + uint64_t start, multi_pages, page, i, j; + bool ret =3D 0; + + for (i =3D 0; i < mpa->last_idx; i++) { + start =3D multi_page_addr_get_one(mpa, i); + + /* pages number in the lower TARGET_PAGE_BITS bit*/ + multi_pages =3D start & (~TARGET_PAGE_MASK); + start =3D start >> TARGET_PAGE_BITS; + for (j =3D 0; j < multi_pages; j++) { + page =3D start + j; + migration_clear_memory_region_dirty_bitmap(rb, page); + + ret =3D test_and_clear_bit(page, rb->bmap); + if (ret) { + rs->migration_dirty_pages--; + } + } + } + return ret; +} + +/* save page header for multiple pages */ +static size_t +save_page_header_multiple(RAMBlock *block, QEMUFile *f, + MultiPageAddr *mpa) +{ + int i, header_bytes; + ram_addr_t offset =3D multi_page_addr_get_one(mpa, 0) & TARGET_PAGE_MA= SK; + + offset |=3D (RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_CONTINUE); + + qemu_put_be64(f, offset); + qemu_put_be64(f, mpa->pages); + qemu_put_be64(f, mpa->last_idx); + for (i =3D 0; i < mpa->last_idx; i++) { + qemu_put_be64(f, mpa->addr[i]); + } + header_bytes =3D sizeof(offset) + sizeof(mpa->pages) + + sizeof(mpa->last_idx) + mpa->last_idx * sizeof(mpa->addr[0]); + return header_bytes; +} + +/* save compress paramters for mpa*/ +static inline void +set_compress_params_multiple(CompressParam *param, RAMBlock *block, + MultiPageAddr *pmpa) +{ + param->block =3D block; + memcpy(¶m->mpa, pmpa, sizeof(MultiPageAddr)); +} + +/* + * try to compress the page before posting it out, return true if the page + * has been properly handled by compression, otherwise needs other + * paths to handle it + */ +static bool +save_compress_page_multiple(RAMState *rs, RAMBlock *block, + MultiPageAddr *mpa) +{ + if (compress_page_with_multi_thread(block, mpa, 0) > 0) { + return true; + } + + compression_counters.busy++; + return false; +} + +/** + * ram_save_target_page: save one target page + * + * Returns the number of pages written + * + * @rs: current RAM state + * @pss: data about the page we want to send + */ +static int +ram_save_target_page_multiple(RAMState *rs, PageSearchStatus *pss) +{ + RAMBlock *block =3D pss->block; + MultiPageAddr *mpa =3D &pss->mpa; + + if (save_compress_page_multiple(rs, block, mpa)) { + return mpa->pages; + } + + return -1; +} + +static int +ram_save_host_page_multiple(RAMState *rs, PageSearchStatus *pss) +{ + int tmppages, pages =3D 0; + size_t pagesize_bits =3D + qemu_ram_pagesize(pss->block) >> TARGET_PAGE_BITS; + uint64_t start_page =3D pss->page; + int res; + + if (ramblock_is_ignored(pss->block)) { + error_report("block %s should not be migrated !", pss->block->idst= r); + return 0; + } + + /* Check the pages is dirty and if it is send it */ + if (migration_bitmap_clear_dirty_multiple(rs, pss->block, &pss->mpa)) { + tmppages =3D ram_save_target_page_multiple(rs, pss); + if (tmppages < 0) { + return tmppages; + } + + pages +=3D tmppages; + /* + * Allow rate limiting to happen in the middle of huge pages if + * something is sent in the current iteration. + */ + if (pagesize_bits > 1 && tmppages > 0) { + migration_rate_limit(); + } + } + + res =3D ram_save_release_protection(rs, pss, start_page); + return (res < 0 ? res : pages); +} + +static void +do_compress_ram_page_multiple(QEMUFile *f, QzSession_T *qzsess, + uint8_t *decompbuf, uint8_t *compbuf, + RAMBlock *block, MultiPageAddr *mpa) +{ + uint64_t start, multi_pages, i; + int rc; + uint32_t origin_size =3D 0, src_size =3D 0, dest_size =3D COMP_BUF_SIZ= E; + ram_addr_t offset; + uint8_t *base_addr; + uint8_t *origbuf =3D decompbuf; + + save_page_header_multiple(block, f, mpa); + /* memory copy to the Pinned memory for the qatzip compress API */ + for (i =3D 0; i < mpa->last_idx; i++) { + start =3D multi_page_addr_get_one(mpa, i); + + /* pages number in the lower TARGET_PAGE_BITS bit*/ + multi_pages =3D start & (~TARGET_PAGE_MASK); + offset =3D start & TARGET_PAGE_MASK; + base_addr =3D (uint8_t *) host_from_ram_block_offset(block, offset= ); + if (base_addr =3D=3D NULL) { + qemu_file_set_error(migrate_get_current()->to_dst_file, -1); + error_report("multi compressed data failed!"); + return; + } + + origin_size =3D multi_pages * TARGET_PAGE_SIZE; + memcpy(origbuf, base_addr, origin_size); + origbuf +=3D origin_size; + src_size +=3D origin_size; + } + + rc =3D qzCompress(qzsess, decompbuf, &src_size, compbuf, &dest_size, 1= ); + if (rc !=3D QZ_OK) { + qemu_file_set_error(migrate_get_current()->to_dst_file, rc); + error_report("multi compressed data failed!"); + return; + } + + qemu_put_be64(f, dest_size); + /* memory copy to the IO buffer */ + qemu_put_buffer(f, compbuf, dest_size); + + ram_counters.transferred +=3D dest_size; + compression_counters.compressed_size +=3D dest_size; + compression_counters.pages +=3D mpa->pages; +} + +static bool +do_decompress_ram_page_multiple(QzSession_T *qzsess, + uint8_t *compbuf, uint8_t *decompbuf, + RAMBlock *block, int len, MultiPageAddr *mpa) +{ + uint64_t start, multi_pages, i; + int rc; + ram_addr_t offset; + uint8_t *base_addr; + int decomp_size =3D 0; + uint32_t src_size =3D len, dest_size =3D DECOMP_BUF_SIZE; + + rc =3D qzDecompress(qzsess, compbuf, &src_size, decompbuf, &dest_size); + if (rc !=3D QZ_OK) { + error_report("ERROR: Decompress FAILED: %d!", rc); + return false; + } + + for (i =3D 0; i < mpa->last_idx; i++) { + start =3D multi_page_addr_get_one(mpa, i); + multi_pages =3D start & (~TARGET_PAGE_MASK); + offset =3D start & TARGET_PAGE_MASK; + base_addr =3D (uint8_t *)host_from_ram_block_offset(block, offset); + if (base_addr =3D=3D NULL) { + return false; + } + + decomp_size =3D multi_pages * TARGET_PAGE_SIZE; + memcpy(base_addr, decompbuf, decomp_size); + decompbuf +=3D decomp_size; + } + + return true; +} + +static int +qemu_get_multiple(QEMUFile *f, MultiPageAddr *mpa, + uint8_t *compbuf, int *pbytes) +{ + uint64_t bytes; + uint64_t i; + + mpa->pages =3D qemu_get_be64(f); + mpa->last_idx =3D qemu_get_be64(f); + mpa->addr[0] =3D 0; + for (i =3D 0; i < mpa->last_idx; i++) { + mpa->addr[i] =3D qemu_get_be64(f); + } + bytes =3D qemu_get_be64(f); + qemu_get_buffer(f, compbuf, bytes); + *pbytes =3D bytes; + return 0; +} +#endif diff --git a/qapi/migration.json b/qapi/migration.json index c84fa10e86..6459927c7a 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -644,6 +644,8 @@ # no compression, 1 means the best compression speed, and= 9 means best # compression ratio which will consume more CPU. # +# @compress-with-qat: compress with qat on and off. (Since 8.1) +# # @compress-threads: Set compression thread count to be used in live migra= tion, # the compression thread count is an integer between 1 = and 255. # @@ -784,7 +786,7 @@ { 'enum': 'MigrationParameter', 'data': ['announce-initial', 'announce-max', 'announce-rounds', 'announce-step', - 'compress-level', 'compress-threads', 'decompress-threads', + 'compress-level', 'compress-with-qat', 'compress-threads', 'dec= ompress-threads', 'compress-wait-thread', 'throttle-trigger-threshold', 'cpu-throttle-initial', 'cpu-throttle-increment', 'cpu-throttle-tailslow', @@ -815,6 +817,8 @@ # # @compress-level: compression level # +# @compress-with-qat: compression with qat (Since 8.1) +# # @compress-threads: compression thread count # # @compress-wait-thread: Controls behavior when all compression threads are @@ -954,6 +958,7 @@ '*announce-rounds': 'size', '*announce-step': 'size', '*compress-level': 'uint8', + '*compress-with-qat': 'bool', '*compress-threads': 'uint8', '*compress-wait-thread': 'bool', '*decompress-threads': 'uint8', @@ -1152,6 +1157,7 @@ '*announce-rounds': 'size', '*announce-step': 'size', '*compress-level': 'uint8', + '*compress-with-qat': 'bool', '*compress-threads': 'uint8', '*compress-wait-thread': 'bool', '*decompress-threads': 'uint8', --=20 2.27.0