From nobody Tue May 21 21:06:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1711493112; cv=none; d=zohomail.com; s=zohoarc; b=UZdKj50vxOH68+T0xV81PLUB2PiuD3JlB4gu4gNq3V9sm4hgjxduvTJHwfe8wq1hiPWunh+/8wWQaFk4uFuqCmS9YneFbkr4CA9CaeR8n5PVdrDabsMrWH2+apM+orzhjLOiDwK+Zgo8uPH0V3TvNser3R7SpRw2wXFzrNg6a9U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1711493112; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=hlzr1/mxNhrro+EzJZYCMXcsfNMOc4LNKlEsuKNRLok=; b=iFcW/SmKpxrEoZWegmrKsUsrgbKcDsiikFbtCLLusI11dLdiKmghhWs5KnC/1HYdM6IRyEhBUsdMLge5qALqu+Zrm+7pRjrNxJNnscg/VKkuJBqUhIFBNCwxfi5DBEGIsfJ3HUMc89igZCwYw5jMlYeWAn6s94KvPWoxEYWSXhA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1711493112165928.0881212707392; Tue, 26 Mar 2024 15:45:12 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rpFWK-00069i-64; Tue, 26 Mar 2024 18:43:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rpFWH-00068i-S8 for qemu-devel@nongnu.org; Tue, 26 Mar 2024 18:43:49 -0400 Received: from mail-qk1-x736.google.com ([2607:f8b0:4864:20::736]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rpFWG-0007aU-40 for qemu-devel@nongnu.org; Tue, 26 Mar 2024 18:43:49 -0400 Received: by mail-qk1-x736.google.com with SMTP id af79cd13be357-78a5ec6a17bso27575185a.0 for ; Tue, 26 Mar 2024 15:43:46 -0700 (PDT) Received: from n36-186-108.byted.org. ([147.160.184.146]) by smtp.gmail.com with ESMTPSA id x15-20020a05620a0ecf00b007887d30dbb7sm3397816qkm.60.2024.03.26.15.43.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 15:43:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1711493025; x=1712097825; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hlzr1/mxNhrro+EzJZYCMXcsfNMOc4LNKlEsuKNRLok=; b=ONeEQRULWzMVdI2UgW5ZtsrLUEaw9CLJ6N7B3BK8euHcOCUzye1MHjGcAfxiZ7+b1j zQet8TkxyRQ1wasutQu6H73xy4EQwhVcHr2nykPZoh4qess12nC72i/EEqjQDbJxgSAr QflqQ+JhCf8B2Y9dZZxmQvxsjYTSAk7Cn8J7AzweH6x6syF1iMiss/a/KZj0cl1uLcg0 J8X+qpDgkjjJvy17aa5Wfg/Vr1bsnH6ZPnQBwp43W5mAJTUgm7sPtBWe0a2Wf2PQRiZV 2iQkHoVnOAipw/O+Z4b5h3In+d67/89CnNfDidMOeLl+KVwN3cZuyTKFgKIYcjHqPGe9 1Tdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711493025; x=1712097825; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hlzr1/mxNhrro+EzJZYCMXcsfNMOc4LNKlEsuKNRLok=; b=Tlh7qanidCQHkrewWc7tsp4TxxsR8Zs561z6X7kcuiQpmx05KssbuMWaBMZdaA4fVL BGt9rjtcHcmoRAzaQt21h74Og5bqjctWY2YNWc++PnOOyqEw3zim4+kM3VfSh1Pffk+e UV74xVKnvxJto9u0YSMUiFyjXKpxhKIhfZ5YRXWcbH3Cso1wPzvNn55OpxBmNrDg6qXH jnktB9iC5tCIFh/yKrnH4+6fWr+dly+ENdCneKZkFSGg4jMAKwfhFSycGeYU49s8v1HH 1dgw3v7cz+mZ52/ClJM4fCFiIkH4neov0yl4Le+CPMrrXNat4tyaCarJwY0mAfQ1wpV4 isDg== X-Gm-Message-State: AOJu0YyUnEqIxWGXfsEFWNNDXRD8L1gSUx2ntxz7RmkAOEkwdG7+4Uy+ mj2Lg8kb+Hdp2BrZep3Zz1zqZHHKwYN8aPK0L65SZfLPrMmTjkmie9DivyMn1rVLHJ683LtT9hC Z X-Google-Smtp-Source: AGHT+IHuobs477Ig/8Nu/ySQ9JRQLFJwG0R5pUAEl+U7DutnMEFNvrIW2kuX7TAAFoQCG6fwB5RMZA== X-Received: by 2002:ae9:c304:0:b0:78a:32db:9df4 with SMTP id n4-20020ae9c304000000b0078a32db9df4mr4370763qkg.9.1711493025538; Tue, 26 Mar 2024 15:43:45 -0700 (PDT) From: Bryan Zhang To: qemu-devel@nongnu.org Cc: peterx@redhat.com, farosas@suse.de, yuan1.liu@intel.com, berrange@redhat.com, nanhai.zou@intel.com, hao.xiang@linux.dev, Bryan Zhang Subject: [PATCH v2 1/5] meson: Introduce 'qatzip' feature to the build system Date: Tue, 26 Mar 2024 22:42:17 +0000 Message-Id: <20240326224221.3623014-2-bryan.zhang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240326224221.3623014-1-bryan.zhang@bytedance.com> References: <20240326224221.3623014-1-bryan.zhang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::736; envelope-from=bryan.zhang@bytedance.com; helo=mail-qk1-x736.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1711493113056100003 Content-Type: text/plain; charset="utf-8" Add a 'qatzip' feature, which is automatically disabled, and which depends on the QATzip library if enabled. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang --- meson.build | 10 ++++++++++ meson_options.txt | 2 ++ scripts/meson-buildoptions.sh | 3 +++ 3 files changed, 15 insertions(+) diff --git a/meson.build b/meson.build index 0ef1654e86..aa79ee85bd 100644 --- a/meson.build +++ b/meson.build @@ -1198,6 +1198,14 @@ if not get_option('zstd').auto() or have_block required: get_option('zstd'), method: 'pkg-config') endif + +qatzip =3D not_found +if get_option('qatzip').enabled() + qatzip =3D dependency('qatzip', version: '>=3D1.1.2', + required: get_option('qatzip'), + method: 'pkg-config') +endif + virgl =3D not_found =20 have_vhost_user_gpu =3D have_tools and host_os =3D=3D 'linux' and pixman.f= ound() @@ -2299,6 +2307,7 @@ config_host_data.set('CONFIG_MALLOC_TRIM', has_malloc= _trim) config_host_data.set('CONFIG_STATX', has_statx) config_host_data.set('CONFIG_STATX_MNT_ID', has_statx_mnt_id) config_host_data.set('CONFIG_ZSTD', zstd.found()) +config_host_data.set('CONFIG_QATZIP', qatzip.found()) config_host_data.set('CONFIG_FUSE', fuse.found()) config_host_data.set('CONFIG_FUSE_LSEEK', fuse_lseek.found()) config_host_data.set('CONFIG_SPICE_PROTOCOL', spice_protocol.found()) @@ -4439,6 +4448,7 @@ summary_info +=3D {'snappy support': snappy} summary_info +=3D {'bzip2 support': libbzip2} summary_info +=3D {'lzfse support': liblzfse} summary_info +=3D {'zstd support': zstd} +summary_info +=3D {'qatzip support': qatzip} summary_info +=3D {'NUMA host support': numa} summary_info +=3D {'capstone': capstone} summary_info +=3D {'libpmem support': libpmem} diff --git a/meson_options.txt b/meson_options.txt index 0a99a059ec..106c540ce7 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -259,6 +259,8 @@ option('xkbcommon', type : 'feature', value : 'auto', description: 'xkbcommon support') option('zstd', type : 'feature', value : 'auto', description: 'zstd compression support') +option('qatzip', type: 'feature', value: 'disabled', + description: 'QATzip compression support') option('fuse', type: 'feature', value: 'auto', description: 'FUSE block device export') option('fuse_lseek', type : 'feature', value : 'auto', diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index 680fa3f581..1afd373606 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -164,6 +164,7 @@ meson_options_help() { printf "%s\n" ' plugins TCG plugins via shared library loading' printf "%s\n" ' png PNG support with libpng' printf "%s\n" ' pvrdma Enable PVRDMA support' + printf "%s\n" ' qatzip QATzip compression support' printf "%s\n" ' qcow1 qcow1 image format support' printf "%s\n" ' qed qed image format support' printf "%s\n" ' qga-vss build QGA VSS support (broken with MinG= W)' @@ -430,6 +431,8 @@ _meson_option_parse() { --prefix=3D*) quote_sh "-Dprefix=3D$2" ;; --enable-pvrdma) printf "%s" -Dpvrdma=3Denabled ;; --disable-pvrdma) printf "%s" -Dpvrdma=3Ddisabled ;; + --enable-qatzip) printf "%s" -Dqatzip=3Denabled ;; + --disable-qatzip) printf "%s" -Dqatzip=3Ddisabled ;; --enable-qcow1) printf "%s" -Dqcow1=3Denabled ;; --disable-qcow1) printf "%s" -Dqcow1=3Ddisabled ;; --enable-qed) printf "%s" -Dqed=3Denabled ;; --=20 2.30.2 From nobody Tue May 21 21:06:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1711493123; cv=none; d=zohomail.com; s=zohoarc; b=C4hJNKfXm5VntRBg7QGTTFKGe25zpSO9JqHAdVyNw/WqPDkyvCTVvslec/m82EX49HKTRyKZIi5uscpnOhyjTfimD97j7R/6qnBJcZmOaiOQB0+tXbzjBLT2t5Eu/DFYUMfITQGsbRoJD58Ye96jWiYXDxiA9XIFPabE31zqKwo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1711493123; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=QnG6eRKzWdMMY6J8vB7INPN8ux+bZX8gkkiNJp6OwR0=; b=kOfPDSTyH0w8oWA2igvto2mioDCddr36v1EPgtjXxq2yBeNtXkYOG7xQUmCyNPTJWx39X9YneTeF06pBoItHf2ROAPbNFHo7/x1P8rYX9QY1VCKIGLxFhoNOIOfozIulXRz4cZZDV9gB4xLO2Eb2TnZuuuiNlpR2wHMd95TAStg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1711493123433663.9802925554148; Tue, 26 Mar 2024 15:45:23 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rpFWK-0006A1-9E; Tue, 26 Mar 2024 18:43:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rpFWI-00069D-IE for qemu-devel@nongnu.org; Tue, 26 Mar 2024 18:43:50 -0400 Received: from mail-qk1-x72b.google.com ([2607:f8b0:4864:20::72b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rpFWG-0007b3-CK for qemu-devel@nongnu.org; Tue, 26 Mar 2024 18:43:50 -0400 Received: by mail-qk1-x72b.google.com with SMTP id af79cd13be357-789e6f7f748so352077385a.3 for ; Tue, 26 Mar 2024 15:43:48 -0700 (PDT) Received: from n36-186-108.byted.org. ([147.160.184.146]) by smtp.gmail.com with ESMTPSA id x15-20020a05620a0ecf00b007887d30dbb7sm3397816qkm.60.2024.03.26.15.43.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 15:43:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1711493027; x=1712097827; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QnG6eRKzWdMMY6J8vB7INPN8ux+bZX8gkkiNJp6OwR0=; b=WSYjkq87So8LvF1/mU+pPTMJqn+0mkEr0Iy9f00Hte+0tKX2bUpFh0o7UAd3ykZm3r w8USULIo8YRiRQpInQ1YzGWaf0yHN5E1430JwW0vRcOyFVkz4qPhR6W+3z3PrLPhjJDL FifhSaYyNE39PmLAXmOLZ4Hm5eXnhaiE/TRh2EA9RrwrdYeokWzMoD2UPlN8vVC87oPz atUB3e6luwQajR1SjsNQz/747Gd3pdcyQ+g5Cog2OWVZzniT2Wx7zjS6C1iNSFjL4EsW 8lHrgHlCCDw4I+naLuW2YXVi295W3dHWHxaRuPhe0NLMsSUuDTwqtenRtKPSx9to7gGk cYSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711493027; x=1712097827; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QnG6eRKzWdMMY6J8vB7INPN8ux+bZX8gkkiNJp6OwR0=; b=KuSg+Z8vy2l1mjlLlTq406tByA+lz0K9TAI7rmSWij+JaEDqMecvD2eaOIFY3GS/Ij SETlHX6BMxANSp6N0rACo+CKXXqnTu7qBUL9fShcbyEuutrEQRwSXw2xJq2vbcuNP8LZ O0pWvCEzBKdU3TL4uBoLe/PXr2hYl0wk+obXL6z17+1vAyGFlbqBJ9mwBTfZmsgFtSkC znNOdK2oYrw/pKhLZ+bEdsKyOyJBkq+JscYyGwucnPyxMnrGHeUT/TYGnJSqPksk/l2R 1tmNDCIk3VfCYd2vgdadiDy6MAB1K5YT39UuU7vU541qyQBQ/y2/eN+cUX2hQFrqTuaR GMbA== X-Gm-Message-State: AOJu0YxLbsLcV4KF3XL9uhuk7UBtcYYGs6JAQOUVWV9E84R+jPF/Kn39 cmnnwASlMx4Z3zwe0osQuLBOU70c1oFCewleqSrDuuJQxVD7Y1KfbpMYXgip3IbH8jPp+PawGM8 z X-Google-Smtp-Source: AGHT+IHqJyVNPKlq2fcCaqqWUKO5DO2lWK9t67D8Y8osn48gUEBRki3rOodRcwBrT7Ovj2K/HUTr5Q== X-Received: by 2002:a05:620a:44c4:b0:789:e524:933b with SMTP id y4-20020a05620a44c400b00789e524933bmr2950044qkp.31.1711493026968; Tue, 26 Mar 2024 15:43:46 -0700 (PDT) From: Bryan Zhang To: qemu-devel@nongnu.org Cc: peterx@redhat.com, farosas@suse.de, yuan1.liu@intel.com, berrange@redhat.com, nanhai.zou@intel.com, hao.xiang@linux.dev, Bryan Zhang Subject: [PATCH v2 2/5] migration: Add migration parameters for QATzip Date: Tue, 26 Mar 2024 22:42:18 +0000 Message-Id: <20240326224221.3623014-3-bryan.zhang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240326224221.3623014-1-bryan.zhang@bytedance.com> References: <20240326224221.3623014-1-bryan.zhang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::72b; envelope-from=bryan.zhang@bytedance.com; helo=mail-qk1-x72b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1711493124989100003 Content-Type: text/plain; charset="utf-8" Adds support for migration parameters to control QATzip compression level and to enable/disable software fallback when QAT hardware is unavailable. This is a preparatory commit for a subsequent commit that will actually use QATzip compression. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang --- Revision: This commit now includes a parameter for controlling software fallback. Fallback is generally intended to be disabled, but having this option available enables using software fallback for testing. This commit also now has some glue code to properly set parameters. migration/migration-hmp-cmds.c | 8 +++++ migration/options.c | 57 ++++++++++++++++++++++++++++++++++ migration/options.h | 2 ++ qapi/migration.json | 35 +++++++++++++++++++++ 4 files changed, 102 insertions(+) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 99b49df5dd..4bd23ba14d 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -630,6 +630,14 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDi= ct *qdict) p->has_multifd_zlib_level =3D true; visit_type_uint8(v, param, &p->multifd_zlib_level, &err); break; + case MIGRATION_PARAMETER_MULTIFD_QATZIP_LEVEL: + p->has_multifd_qatzip_level =3D true; + visit_type_uint8(v, param, &p->multifd_qatzip_level, &err); + break; + case MIGRATION_PARAMETER_MULTIFD_QATZIP_SW_FALLBACK: + p->has_multifd_qatzip_sw_fallback =3D true; + visit_type_bool(v, param, &p->multifd_qatzip_sw_fallback, &err); + break; case MIGRATION_PARAMETER_MULTIFD_ZSTD_LEVEL: p->has_multifd_zstd_level =3D true; visit_type_uint8(v, param, &p->multifd_zstd_level, &err); diff --git a/migration/options.c b/migration/options.c index 3e3e0b93b4..1316ea605a 100644 --- a/migration/options.c +++ b/migration/options.c @@ -62,6 +62,15 @@ #define DEFAULT_MIGRATE_MULTIFD_COMPRESSION MULTIFD_COMPRESSION_NONE /* 0: means nocompress, 1: best speed, ... 9: best compress ratio */ #define DEFAULT_MIGRATE_MULTIFD_ZLIB_LEVEL 1 +/* + * 1: best speed, ... 9: best compress ratio + * There is some nuance here. Refer to QATzip documentation to understand + * the mapping of QATzip levels to standard deflate levels. + */ +#define DEFAULT_MIGRATE_MULTIFD_QATZIP_LEVEL 1 +/* QATzip's SW fallback implementation is extremely slow, so avoid fallbac= k */ +#define DEFAULT_MIGRATE_MULTIFD_QATZIP_SW_FALLBACK false + /* 0: means nocompress, 1: best speed, ... 20: best compress ratio */ #define DEFAULT_MIGRATE_MULTIFD_ZSTD_LEVEL 1 =20 @@ -143,6 +152,12 @@ Property migration_properties[] =3D { DEFINE_PROP_UINT8("multifd-zlib-level", MigrationState, parameters.multifd_zlib_level, DEFAULT_MIGRATE_MULTIFD_ZLIB_LEVEL), + DEFINE_PROP_UINT8("multifd-qatzip-level", MigrationState, + parameters.multifd_qatzip_level, + DEFAULT_MIGRATE_MULTIFD_QATZIP_LEVEL), + DEFINE_PROP_BOOL("multifd-qatzip-sw-fallback", MigrationState, + parameters.multifd_qatzip_sw_fallback, + DEFAULT_MIGRATE_MULTIFD_QATZIP_SW_FALLBACK), DEFINE_PROP_UINT8("multifd-zstd-level", MigrationState, parameters.multifd_zstd_level, DEFAULT_MIGRATE_MULTIFD_ZSTD_LEVEL), @@ -861,6 +876,20 @@ int migrate_multifd_zlib_level(void) return s->parameters.multifd_zlib_level; } =20 +int migrate_multifd_qatzip_level(void) +{ + MigrationState *s =3D migrate_get_current(); + + return s->parameters.multifd_qatzip_level; +} + +bool migrate_multifd_qatzip_sw_fallback(void) +{ + MigrationState *s =3D migrate_get_current(); + + return s->parameters.multifd_qatzip_sw_fallback; +} + int migrate_multifd_zstd_level(void) { MigrationState *s =3D migrate_get_current(); @@ -983,6 +1012,11 @@ MigrationParameters *qmp_query_migrate_parameters(Err= or **errp) params->multifd_compression =3D s->parameters.multifd_compression; params->has_multifd_zlib_level =3D true; params->multifd_zlib_level =3D s->parameters.multifd_zlib_level; + params->has_multifd_qatzip_level =3D true; + params->multifd_qatzip_level =3D s->parameters.multifd_qatzip_level; + params->has_multifd_qatzip_sw_fallback =3D true; + params->multifd_qatzip_sw_fallback =3D + s->parameters.multifd_qatzip_sw_fallback; params->has_multifd_zstd_level =3D true; params->multifd_zstd_level =3D s->parameters.multifd_zstd_level; params->has_xbzrle_cache_size =3D true; @@ -1038,6 +1072,8 @@ void migrate_params_init(MigrationParameters *params) params->has_multifd_channels =3D true; params->has_multifd_compression =3D true; params->has_multifd_zlib_level =3D true; + params->has_multifd_qatzip_level =3D true; + params->has_multifd_qatzip_sw_fallback =3D true; params->has_multifd_zstd_level =3D true; params->has_xbzrle_cache_size =3D true; params->has_max_postcopy_bandwidth =3D true; @@ -1147,6 +1183,14 @@ bool migrate_params_check(MigrationParameters *param= s, Error **errp) return false; } =20 + if (params->has_multifd_qatzip_level && + ((params->multifd_qatzip_level > 9) || + (params->multifd_qatzip_level < 1))) { + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "multifd_qatzip_lev= el", + "a value between 1 and 9"); + return false; + } + if (params->has_multifd_zstd_level && (params->multifd_zstd_level > 20)) { error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "multifd_zstd_level= ", @@ -1312,6 +1356,12 @@ static void migrate_params_test_apply(MigrateSetPara= meters *params, if (params->has_multifd_compression) { dest->multifd_compression =3D params->multifd_compression; } + if (params->has_multifd_qatzip_level) { + dest->multifd_qatzip_level =3D params->multifd_qatzip_level; + } + if (params->has_multifd_qatzip_sw_fallback) { + dest->multifd_qatzip_sw_fallback =3D params->multifd_qatzip_sw_fal= lback; + } if (params->has_xbzrle_cache_size) { dest->xbzrle_cache_size =3D params->xbzrle_cache_size; } @@ -1447,6 +1497,13 @@ static void migrate_params_apply(MigrateSetParameter= s *params, Error **errp) if (params->has_multifd_compression) { s->parameters.multifd_compression =3D params->multifd_compression; } + if (params->has_multifd_qatzip_level) { + s->parameters.multifd_qatzip_level =3D params->multifd_qatzip_leve= l; + } + if (params->has_multifd_qatzip_sw_fallback) { + s->parameters.multifd_qatzip_sw_fallback =3D + params->multifd_qatzip_sw_fallback; + } if (params->has_xbzrle_cache_size) { s->parameters.xbzrle_cache_size =3D params->xbzrle_cache_size; xbzrle_cache_resize(params->xbzrle_cache_size, errp); diff --git a/migration/options.h b/migration/options.h index 246c160aee..94aee24d97 100644 --- a/migration/options.h +++ b/migration/options.h @@ -87,6 +87,8 @@ MigMode migrate_mode(void); int migrate_multifd_channels(void); MultiFDCompression migrate_multifd_compression(void); int migrate_multifd_zlib_level(void); +int migrate_multifd_qatzip_level(void); +bool migrate_multifd_qatzip_sw_fallback(void); int migrate_multifd_zstd_level(void); uint8_t migrate_throttle_trigger_threshold(void); const char *migrate_tls_authz(void); diff --git a/qapi/migration.json b/qapi/migration.json index 0b33a71ab4..66ea6d32fc 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -853,6 +853,16 @@ # speed, and 9 means best compression ratio which will consume # more CPU. Defaults to 1. (Since 5.0) # +# @multifd-qatzip-level: Set the compression level to be used in live +# migration. The level is an integer between 1 and 9, where 1 means +# the best compression speed, and 9 means the best compression +# ratio which will consume more CPU. Defaults to 1. +# +# @multifd-qatzip-sw-fallback: Enable software fallback if QAT hardware +# is unavailable. Defaults to false. Software fallback performance +# is very poor compared to regular zlib, so be cautious about +# enabling this option. +# # @multifd-zstd-level: Set the compression level to be used in live # migration, the compression level is an integer between 0 and 20, # where 0 means no compression, 1 means the best compression @@ -915,6 +925,7 @@ 'xbzrle-cache-size', 'max-postcopy-bandwidth', 'max-cpu-throttle', 'multifd-compression', 'multifd-zlib-level', 'multifd-zstd-level', + 'multifd-qatzip-level', 'multifd-qatzip-sw-fallback', 'block-bitmap-mapping', { 'name': 'x-vcpu-dirty-limit-period', 'features': ['unstable']= }, 'vcpu-dirty-limit', @@ -1045,6 +1056,16 @@ # speed, and 9 means best compression ratio which will consume # more CPU. Defaults to 1. (Since 5.0) # +# @multifd-qatzip-level: Set the compression level to be used in live +# migration. The level is an integer between 1 and 9, where 1 means +# the best compression speed, and 9 means the best compression +# ratio which will consume more CPU. Defaults to 1. +# +# @multifd-qatzip-sw-fallback: Enable software fallback if QAT hardware +# is unavailable. Defaults to false. Software fallback performance +# is very poor compared to regular zlib, so be cautious about +# enabling this option. +# # @multifd-zstd-level: Set the compression level to be used in live # migration, the compression level is an integer between 0 and 20, # where 0 means no compression, 1 means the best compression @@ -1125,6 +1146,8 @@ '*max-cpu-throttle': 'uint8', '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', + '*multifd-qatzip-level': 'uint8', + '*multifd-qatzip-sw-fallback': 'bool', '*multifd-zstd-level': 'uint8', '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], '*x-vcpu-dirty-limit-period': { 'type': 'uint64', @@ -1273,6 +1296,16 @@ # speed, and 9 means best compression ratio which will consume # more CPU. Defaults to 1. (Since 5.0) # +# @multifd-qatzip-level: Set the compression level to be used in live +# migration. The level is an integer between 1 and 9, where 1 means +# the best compression speed, and 9 means the best compression +# ratio which will consume more CPU. Defaults to 1. +# +# @multifd-qatzip-sw-fallback: Enable software fallback if QAT hardware +# is unavailable. Defaults to false. Software fallback performance +# is very poor compared to regular zlib, so be cautious about +# enabling this option. +# # @multifd-zstd-level: Set the compression level to be used in live # migration, the compression level is an integer between 0 and 20, # where 0 means no compression, 1 means the best compression @@ -1350,6 +1383,8 @@ '*max-cpu-throttle': 'uint8', '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', + '*multifd-qatzip-level': 'uint8', + '*multifd-qatzip-sw-fallback': 'bool', '*multifd-zstd-level': 'uint8', '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], '*x-vcpu-dirty-limit-period': { 'type': 'uint64', --=20 2.30.2 From nobody Tue May 21 21:06:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1711493090; cv=none; d=zohomail.com; s=zohoarc; b=jIbhDACQHrWjfTTarx9EHGgrjG9y+pABcGIHOPKyaQ+Zz17GKshPNV1Eq2UU0zBghi38XoFjIXES2rddN+GM3qhQjwIe3TzAafK6UkVKuq5XhEABFiv4SpEUHDHK4nwF3gnbUKX4h9l+J4oNScuakwpBTZtKWvtxw8e6wbNA+8Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1711493090; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=E2+htT0Abm6Y4hPWnu3Iu305PtkzW44cwWugxR7RsCs=; b=Nsbr0jgBKhwbmDVEejuv8aq2cPeijrOyWPbp1Jj6rGF6gJMmO++RHHqg++GcKwaeybksveNc0ea+vQIRieVfuRT8EGSoXqCPRNow0CkVmn1nFcGOVQ3NOcibHAvW+9db8uEggHm6c1W+5eSyHAvJmXknH8xzkcdC+Fysi7fZC/M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1711493090963233.86367657064852; Tue, 26 Mar 2024 15:44:50 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rpFWL-0006AV-VW; Tue, 26 Mar 2024 18:43:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rpFWK-0006AF-LM for qemu-devel@nongnu.org; Tue, 26 Mar 2024 18:43:52 -0400 Received: from mail-ot1-x335.google.com ([2607:f8b0:4864:20::335]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rpFWI-0007bq-HS for qemu-devel@nongnu.org; Tue, 26 Mar 2024 18:43:52 -0400 Received: by mail-ot1-x335.google.com with SMTP id 46e09a7af769-6e6a5bd015dso2865306a34.2 for ; Tue, 26 Mar 2024 15:43:50 -0700 (PDT) Received: from n36-186-108.byted.org. ([147.160.184.146]) by smtp.gmail.com with ESMTPSA id x15-20020a05620a0ecf00b007887d30dbb7sm3397816qkm.60.2024.03.26.15.43.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 15:43:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1711493029; x=1712097829; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=E2+htT0Abm6Y4hPWnu3Iu305PtkzW44cwWugxR7RsCs=; b=UdDKIg84WjYOlF3vWLEjzoL8Tbxgo6KAWCB35Sxy3L5f7MO3vaVzqUGMeW3zQPlvHe otetl79JLx5zJ6xU+fv51LwpGjiaJlCaQJ9xsWlCmNNm6P2LK4uG0pIjd+BlIg89R8/K OizUGEd1rOpHC6BBdzt0b6dOTBSrPTg3YBDU4+0bJQU3J8Xw0Lw4s7YaRu9EZJBFj+Aj X10HzvwNdNdx7UlEOibyv7Lup0EiIFQyzPHNmozmESR9g78qpRJZLfunlHYy0DsWS1a1 lZJscgDuPYJnpFoV3S0CWy7DQgtYWnNuAlG9vcj2QViJQxi9fOqwhggPK1SrvwMYvzIu Pkiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711493029; x=1712097829; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=E2+htT0Abm6Y4hPWnu3Iu305PtkzW44cwWugxR7RsCs=; b=mwiDszCO2HzdgaKUOObDS5ephqW/B0lUEkruU6ntimbgH6mZWn5lW2APR2DMILqyAf TBJlPDDSYNK10kJWh5Y2JExqUq1J7R/BTkSvI5zQKObZh3yoYIWxyIncPf02UxiAOfQO gLKZUX0WAqN+g7G/7QsunYyvCabz75LXKAntil2B7U9Oo94wAiKcOpeq2CsBioDFKChG drF5ob+Vjo6iXkGtUfvowbb4DYeeRmxRAEcd43uUsBm038ShXYIt1kui4MXfdWAo/SdW V9DFkOit9eoMnYoGVCQr4iOXY7AMustLKz86f4CHW/06uWhkWVxseU+mb3bcn6gzVyEm u0Qw== X-Gm-Message-State: AOJu0YwxsVmFFV0Dnvz8RIJA5IlMUAMyNPrP+9KQXSLFkiUrRyC4vrE1 wb0VDr6bUxYOD5DLciujuZ9841T74ykxlUQynAWajeQORrZr2UNwUsPAKUKtAHQpNcvOGTz7xR2 z X-Google-Smtp-Source: AGHT+IGQcacerCylv8HxykbNRfarkzrWBGTEbP/GOyYQp2UVoKstb7ui+hN+lhYJVgDKGTx1tvyHCQ== X-Received: by 2002:a05:6830:10d0:b0:6e6:6c3b:58bd with SMTP id z16-20020a05683010d000b006e66c3b58bdmr916261oto.29.1711493029015; Tue, 26 Mar 2024 15:43:49 -0700 (PDT) From: Bryan Zhang To: qemu-devel@nongnu.org Cc: peterx@redhat.com, farosas@suse.de, yuan1.liu@intel.com, berrange@redhat.com, nanhai.zou@intel.com, hao.xiang@linux.dev, Bryan Zhang Subject: [PATCH v2 3/5] migration: Introduce unimplemented 'qatzip' compression method Date: Tue, 26 Mar 2024 22:42:19 +0000 Message-Id: <20240326224221.3623014-4-bryan.zhang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240326224221.3623014-1-bryan.zhang@bytedance.com> References: <20240326224221.3623014-1-bryan.zhang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::335; envelope-from=bryan.zhang@bytedance.com; helo=mail-ot1-x335.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1711493092908100003 Content-Type: text/plain; charset="utf-8" Adds support for 'qatzip' as an option for the multifd compression method parameter, but copy-pastes the no-op logic to leave the actual methods effectively unimplemented. This is in preparation of a subsequent commit that will implement actually using QAT for compression and decompression. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang --- hw/core/qdev-properties-system.c | 6 +- migration/meson.build | 1 + migration/multifd-qatzip.c | 117 +++++++++++++++++++++++++++++++ migration/multifd.h | 1 + qapi/migration.json | 5 +- tests/qtest/meson.build | 4 ++ 6 files changed, 132 insertions(+), 2 deletions(-) create mode 100644 migration/multifd-qatzip.c diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-sys= tem.c index 1a396521d5..d8e48dcb0e 100644 --- a/hw/core/qdev-properties-system.c +++ b/hw/core/qdev-properties-system.c @@ -658,7 +658,11 @@ const PropertyInfo qdev_prop_fdc_drive_type =3D { const PropertyInfo qdev_prop_multifd_compression =3D { .name =3D "MultiFDCompression", .description =3D "multifd_compression values, " - "none/zlib/zstd", + "none/zlib/zstd" +#ifdef CONFIG_QATZIP + "/qatzip" +#endif + , .enum_table =3D &MultiFDCompression_lookup, .get =3D qdev_propinfo_get_enum, .set =3D qdev_propinfo_set_enum, diff --git a/migration/meson.build b/migration/meson.build index 92b1cc4297..e20f318379 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -40,6 +40,7 @@ if get_option('live_block_migration').allowed() system_ss.add(files('block.c')) endif system_ss.add(when: zstd, if_true: files('multifd-zstd.c')) +system_ss.add(when: qatzip, if_true: files('multifd-qatzip.c')) =20 specific_ss.add(when: 'CONFIG_SYSTEM_ONLY', if_true: files('ram.c', diff --git a/migration/multifd-qatzip.c b/migration/multifd-qatzip.c new file mode 100644 index 0000000000..f66336a4a7 --- /dev/null +++ b/migration/multifd-qatzip.c @@ -0,0 +1,117 @@ +/* + * Multifd QATzip compression implementation + * + * Copyright (c) Bytedance + * + * Authors: + * Bryan Zhang + * Hao Xiang + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "exec/ramblock.h" +#include "exec/target_page.h" +#include "qapi/error.h" +#include "migration.h" +#include "options.h" +#include "multifd.h" + +/* + * This is an intermediary file to introduce 'qatzip' as an option for mul= tifd + * compression. The actual method implementations are no-ops. + */ + +static int qatzip_send_setup(MultiFDSendParams *p, Error **errp) +{ + if (migrate_zero_copy_send()) { + p->write_flags |=3D QIO_CHANNEL_WRITE_FLAG_ZERO_COPY; + } + + return 0; +} + +static void qatzip_send_cleanup(MultiFDSendParams *p, Error **errp) +{ + return; +} + +static int qatzip_send_prepare(MultiFDSendParams *p, Error **errp) +{ + bool use_zero_copy_send =3D migrate_zero_copy_send(); + MultiFDPages_t *pages =3D p->pages; + int ret; + + if (!use_zero_copy_send) { + /* + * Only !zerocopy needs the header in IOV; zerocopy will + * send it separately. + */ + multifd_send_prepare_header(p); + } + + for (int i =3D 0; i < pages->num; i++) { + p->iov[p->iovs_num].iov_base =3D pages->block->host + pages->offse= t[i]; + p->iov[p->iovs_num].iov_len =3D p->page_size; + p->iovs_num++; + } + + p->next_packet_size =3D pages->num * p->page_size; + p->flags |=3D MULTIFD_FLAG_NOCOMP; + + multifd_send_fill_packet(p); + + if (use_zero_copy_send) { + /* Send header first, without zerocopy */ + ret =3D qio_channel_write_all(p->c, (void *)p->packet, + p->packet_len, errp); + if (ret !=3D 0) { + return -1; + } + } + + return 0; +} + +static int qatzip_recv_setup(MultiFDRecvParams *p, Error **errp) +{ + return 0; +} + +static void qatzip_recv_cleanup(MultiFDRecvParams *p) +{ +} + +static int qatzip_recv_pages(MultiFDRecvParams *p, Error **errp) +{ + uint32_t flags =3D p->flags & MULTIFD_FLAG_COMPRESSION_MASK; + + if (flags !=3D MULTIFD_FLAG_NOCOMP) { + error_setg(errp, "multifd %u: flags received %x flags expected %x", + p->id, flags, MULTIFD_FLAG_NOCOMP); + return -1; + } + for (int i =3D 0; i < p->normal_num; i++) { + p->iov[i].iov_base =3D p->host + p->normal[i]; + p->iov[i].iov_len =3D p->page_size; + } + return qio_channel_readv_all(p->c, p->iov, p->normal_num, errp); +} + +static MultiFDMethods multifd_qatzip_ops =3D { + .send_setup =3D qatzip_send_setup, + .send_cleanup =3D qatzip_send_cleanup, + .send_prepare =3D qatzip_send_prepare, + .recv_setup =3D qatzip_recv_setup, + .recv_cleanup =3D qatzip_recv_cleanup, + .recv_pages =3D qatzip_recv_pages +}; + +static void multifd_qatzip_register(void) +{ + multifd_register_ops(MULTIFD_COMPRESSION_QATZIP, &multifd_qatzip_ops); +} + +migration_init(multifd_qatzip_register); diff --git a/migration/multifd.h b/migration/multifd.h index b3fe27ae93..ae73f1713c 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -33,6 +33,7 @@ bool multifd_queue_page(RAMBlock *block, ram_addr_t offse= t); #define MULTIFD_FLAG_NOCOMP (0 << 1) #define MULTIFD_FLAG_ZLIB (1 << 1) #define MULTIFD_FLAG_ZSTD (2 << 1) +#define MULTIFD_FLAG_QATZIP (3 << 1) =20 /* This value needs to be a multiple of qemu_target_page_size() */ #define MULTIFD_PACKET_SIZE (512 * 1024) diff --git a/qapi/migration.json b/qapi/migration.json index 66ea6d32fc..9018166ac8 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -625,11 +625,14 @@ # # @zstd: use zstd compression method. # +# @qatzip: use qatzip compression method. +# # Since: 5.0 ## { 'enum': 'MultiFDCompression', 'data': [ 'none', 'zlib', - { 'name': 'zstd', 'if': 'CONFIG_ZSTD' } ] } + { 'name': 'zstd', 'if': 'CONFIG_ZSTD' }, + { 'name': 'qatzip', 'if': 'CONFIG_QATZIP'} ] } =20 ## # @MigMode: diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index 6ea77893f5..539104c06d 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -315,6 +315,10 @@ if gnutls.found() endif endif =20 +if qatzip.found() + migration_files +=3D [qatzip] +endif + qtests =3D { 'bios-tables-test': [io, 'boot-sector.c', 'acpi-utils.c', 'tpm-emu.c'], 'cdrom-test': files('boot-sector.c'), --=20 2.30.2 From nobody Tue May 21 21:06:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1711493061; cv=none; d=zohomail.com; s=zohoarc; b=EPXZyC20JgMrvcjHztFtEnUf19C8pmKuLDY/iCIq5uRqIHBj7JTDRMJrpJEQVkuCvZpXQY37CHFbD7x5eMIE9Cj2VoT/t82cd/0FgcONN3+pXjFpzw1JWgtSR22yy+bh0c5TUf2y5wRmKixBiaGu0Vv6cJRgOTv6ybSnq7ixhjo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1711493061; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=GVEzA2HQOLvutvBo/VheWIkAEa4Jjb54H0+Qgm8hF1A=; b=M8oVsiMH0/OF9u68GySotvDhMsEJ6d4iPZoDtl0Xze1Y32E9iZYjrWiARcmmZpSMWNUQARDiWFAZGe74pcBatfnfweh84JbFM4cb5S98qsSj8EUy2R4HPf/a87mLHxv6rWGed4L0mWLGAz33gIGaQ/OxW3r/qpfCt/1nBUnmfPY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1711493061146624.5866849005298; Tue, 26 Mar 2024 15:44:21 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rpFWN-0006B1-JN; Tue, 26 Mar 2024 18:43:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rpFWM-0006Aj-DY for qemu-devel@nongnu.org; Tue, 26 Mar 2024 18:43:54 -0400 Received: from mail-qk1-x731.google.com ([2607:f8b0:4864:20::731]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rpFWJ-0007cB-DX for qemu-devel@nongnu.org; Tue, 26 Mar 2024 18:43:54 -0400 Received: by mail-qk1-x731.google.com with SMTP id af79cd13be357-789fb1f80f5so405917285a.3 for ; Tue, 26 Mar 2024 15:43:51 -0700 (PDT) Received: from n36-186-108.byted.org. ([147.160.184.146]) by smtp.gmail.com with ESMTPSA id x15-20020a05620a0ecf00b007887d30dbb7sm3397816qkm.60.2024.03.26.15.43.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 15:43:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1711493030; x=1712097830; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GVEzA2HQOLvutvBo/VheWIkAEa4Jjb54H0+Qgm8hF1A=; b=dWPrwruukYZbPu0dbg/zL9qWyqiBp+NRUu6ZDl5HVFIn4f9ragBkNY5V86MB8Mh5rm LLFWPFej38FSiP9apxTdx758YumuR8nna0r769/80OWgAJ/mCWazOPjpCFvnh0o+aMbz //eFVk1OqUtCk0n+jr1wtp2c6qG7aChYSN3iH/xqfdAuLT3c+E0PO79q38oL1meYFbUw 6d1JnPeRrQHkkzziMqXB/Q2K6fXCl2BgRGb2QVhmRZi1Utd/7WAv7AGoTl/+lqZGp0Sk ZobEhl2hrGzXUUE+8PTIE8T1IlGIoHo3HpTqJ803jvlEE4PhBtLIvjOcKNl2JadE1RRL pLUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711493030; x=1712097830; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GVEzA2HQOLvutvBo/VheWIkAEa4Jjb54H0+Qgm8hF1A=; b=Fc+785XLEJ7oKhBR6/6kvhDKHzW2O3z52oRsmHSuerlzq2NOeVs6OzwKtJb1+5GvDF EnpR0A6t2qkVo/2vkemiOJBOksTBAXRFAbAB52k1oNkhlL1FcqRlUOfXCcamNymAMbUu CsFxZr/HKkPqFEm82UNR4Myys6K/V961XvH/WTpheFPSR2gjVXN1WziUSUAB0GfoZUcT OtB5OgmqxPMYL3zN0GQYGyiwfFM7dMz+Wjd3kXx2ghWfc+DWtLzHTgwOcCWtYr/rkctG l9uC8b6vLSjVb6RTTWdtPNGKZg1BN7jwOY0lnkw/U8HqOryaIgoN3W5rQdu3axbiwz2Z M22w== X-Gm-Message-State: AOJu0YyQzYksNSiF232Rui5qZ2Ah1J/O3PqMdlfnMoMlgvB0oqWR/dQe ePmLf+UbOfyq/MrJ/K4zczCcIwXFt60da8IR75oFQ87haxEWvMqs7KUqPOrafsAtW+LMr3OnOOK 5 X-Google-Smtp-Source: AGHT+IFaGRnvBhZzGDKiaiooinL3258ntccbXK/LzfwnmLaO6BDU2D0xXANtPkck1hoef0nyjEAH1g== X-Received: by 2002:a05:620a:6207:b0:78a:3913:549 with SMTP id ou7-20020a05620a620700b0078a39130549mr10392570qkn.65.1711493030218; Tue, 26 Mar 2024 15:43:50 -0700 (PDT) From: Bryan Zhang To: qemu-devel@nongnu.org Cc: peterx@redhat.com, farosas@suse.de, yuan1.liu@intel.com, berrange@redhat.com, nanhai.zou@intel.com, hao.xiang@linux.dev, Bryan Zhang Subject: [PATCH v2 4/5] migration: Implement 'qatzip' methods using QAT Date: Tue, 26 Mar 2024 22:42:20 +0000 Message-Id: <20240326224221.3623014-5-bryan.zhang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240326224221.3623014-1-bryan.zhang@bytedance.com> References: <20240326224221.3623014-1-bryan.zhang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::731; envelope-from=bryan.zhang@bytedance.com; helo=mail-qk1-x731.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1711493062881100009 Content-Type: text/plain; charset="utf-8" Uses QAT to offload deflate compression and decompression in the 'qatzip' compression method for multifd migration. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang --- migration/multifd-qatzip.c | 331 +++++++++++++++++++++++++++++++++---- 1 file changed, 298 insertions(+), 33 deletions(-) diff --git a/migration/multifd-qatzip.c b/migration/multifd-qatzip.c index f66336a4a7..13835cf76f 100644 --- a/migration/multifd-qatzip.c +++ b/migration/multifd-qatzip.c @@ -18,86 +18,351 @@ #include "migration.h" #include "options.h" #include "multifd.h" +#include =20 -/* - * This is an intermediary file to introduce 'qatzip' as an option for mul= tifd - * compression. The actual method implementations are no-ops. - */ +struct qatzip_data { + /* + * Unique session for use with QATzip API + */ + QzSession_T sess; + + /* + * For compression: Buffer for pages to compress + * For decompression: Buffer for data to decompress + */ + uint8_t *in_buf; + uint32_t in_len; =20 + /* + * For compression: Output buffer of compressed data + * For decompression: Output buffer of decompressed data + */ + uint8_t *out_buf; + uint32_t out_len; +}; + +/** + * qatzip_send_setup: Set up QATzip session and private buffers. + * + * @param p Multifd channel params + * @param errp Pointer to error, which will be set in case of error + * @return 0 on success, -1 on error (and *errp will be set) + */ static int qatzip_send_setup(MultiFDSendParams *p, Error **errp) { - if (migrate_zero_copy_send()) { - p->write_flags |=3D QIO_CHANNEL_WRITE_FLAG_ZERO_COPY; + struct qatzip_data *q; + QzSessionParamsDeflate_T params; + const char *err_msg; + int ret; + int sw_fallback; + + q =3D g_new0(struct qatzip_data, 1); + p->data =3D q; + + sw_fallback =3D 0; + if (migrate_multifd_qatzip_sw_fallback()) { + sw_fallback =3D 1; + } + + ret =3D qzInit(&q->sess, sw_fallback); + if (ret !=3D QZ_OK && ret !=3D QZ_DUPLICATE) { + err_msg =3D "qzInit failed"; + goto err_free_q; + } + + ret =3D qzGetDefaultsDeflate(¶ms); + if (ret !=3D QZ_OK) { + err_msg =3D "qzGetDefaultsDeflate failed"; + goto err_close; + } + + /* Make sure to use configured QATzip compression level. */ + params.common_params.comp_lvl =3D migrate_multifd_qatzip_level(); + + ret =3D qzSetupSessionDeflate(&q->sess, ¶ms); + if (ret !=3D QZ_OK && ret !=3D QZ_DUPLICATE) { + err_msg =3D "qzSetupSessionDeflate failed"; + goto err_close; + } + + /* TODO Add support for larger packets. */ + if (MULTIFD_PACKET_SIZE > UINT32_MAX) { + err_msg =3D "packet size too large for QAT"; + goto err_close; + } + + q->in_len =3D MULTIFD_PACKET_SIZE; + q->in_buf =3D qzMalloc(q->in_len, 0, PINNED_MEM); + if (!q->in_buf) { + err_msg =3D "qzMalloc failed"; + goto err_close; + } + + q->out_len =3D qzMaxCompressedLength(MULTIFD_PACKET_SIZE, &q->sess); + q->out_buf =3D qzMalloc(q->out_len, 0, PINNED_MEM); + if (!q->out_buf) { + err_msg =3D "qzMalloc failed"; + goto err_free_inbuf; } =20 return 0; + +err_free_inbuf: + qzFree(q->in_buf); +err_close: + qzClose(&q->sess); +err_free_q: + g_free(q); + error_setg(errp, "multifd %u: %s", p->id, err_msg); + return -1; } =20 +/** + * qatzip_send_cleanup: Tear down QATzip session and release private buffe= rs. + * + * @param p Multifd channel params + * @param errp Pointer to error, which will be set in case of error + * @return None + */ static void qatzip_send_cleanup(MultiFDSendParams *p, Error **errp) { + struct qatzip_data *q =3D p->data; + const char *err_msg; + int ret; + + ret =3D qzTeardownSession(&q->sess); + if (ret !=3D QZ_OK) { + err_msg =3D "qzTeardownSession failed"; + goto err; + } + + ret =3D qzClose(&q->sess); + if (ret !=3D QZ_OK) { + err_msg =3D "qzClose failed"; + goto err; + } + + qzFree(q->in_buf); + q->in_buf =3D NULL; + qzFree(q->out_buf); + q->out_buf =3D NULL; + g_free(p->data); + p->data =3D NULL; return; + +err: + error_setg(errp, "multifd %u: %s", p->id, err_msg); } =20 +/** + * qatzip_send_prepare: Compress pages and update IO channel info. + * + * @param p Multifd channel params + * @param errp Pointer to error, which will be set in case of error + * @return 0 on success, -1 on error (and *errp will be set) + */ static int qatzip_send_prepare(MultiFDSendParams *p, Error **errp) { - bool use_zero_copy_send =3D migrate_zero_copy_send(); MultiFDPages_t *pages =3D p->pages; + struct qatzip_data *q =3D p->data; int ret; + unsigned int in_len, out_len; =20 - if (!use_zero_copy_send) { - /* - * Only !zerocopy needs the header in IOV; zerocopy will - * send it separately. - */ - multifd_send_prepare_header(p); - } + multifd_send_prepare_header(p); =20 + /* memcpy all the pages into one buffer. */ for (int i =3D 0; i < pages->num; i++) { - p->iov[p->iovs_num].iov_base =3D pages->block->host + pages->offse= t[i]; - p->iov[p->iovs_num].iov_len =3D p->page_size; - p->iovs_num++; + memcpy(q->in_buf + (i * p->page_size), + p->pages->block->host + pages->offset[i], + p->page_size); } =20 - p->next_packet_size =3D pages->num * p->page_size; - p->flags |=3D MULTIFD_FLAG_NOCOMP; - - multifd_send_fill_packet(p); + in_len =3D pages->num * p->page_size; + if (in_len > q->in_len) { + error_setg(errp, "multifd %u: unexpectedly large input", p->id); + return -1; + } + out_len =3D q->out_len; =20 - if (use_zero_copy_send) { - /* Send header first, without zerocopy */ - ret =3D qio_channel_write_all(p->c, (void *)p->packet, - p->packet_len, errp); - if (ret !=3D 0) { - return -1; - } + /* + * Unlike other multifd compression implementations, we use a non-stre= aming + * API and place all the data into one buffer, rather than sending eac= h page + * to the compression API at a time. Based on initial benchmarks, the + * non-streaming API outperforms the streaming API. Plus, the logic in= QEMU + * is friendly to using the non-streaming API anyway. If either of the= se + * statements becomes no longer true, we can revisit adding a streaming + * implementation. + */ + ret =3D qzCompress(&q->sess, q->in_buf, &in_len, q->out_buf, &out_len,= 1); + if (ret !=3D QZ_OK) { + error_setg(errp, "multifd %u: QATzip returned %d instead of QZ_OK", + p->id, ret); + return -1; + } + if (in_len !=3D pages->num * p->page_size) { + error_setg(errp, "multifd %u: QATzip failed to compress all input", + p->id); + return -1; } =20 + p->iov[p->iovs_num].iov_base =3D q->out_buf; + p->iov[p->iovs_num].iov_len =3D out_len; + p->iovs_num++; + p->next_packet_size =3D out_len; + p->flags |=3D MULTIFD_FLAG_QATZIP; + + multifd_send_fill_packet(p); + return 0; } =20 +/** + * qatzip_recv_setup: Set up QATzip session and allocate private buffers. + * + * @param p Multifd channel params + * @param errp Pointer to error, which will be set in case of error + * @return 0 on success, -1 on error (and *errp will be set) + */ static int qatzip_recv_setup(MultiFDRecvParams *p, Error **errp) { + struct qatzip_data *q; + QzSessionParamsDeflate_T params; + const char *err_msg; + int ret; + int sw_fallback; + + q =3D g_new0(struct qatzip_data, 1); + p->data =3D q; + + sw_fallback =3D 0; + if (migrate_multifd_qatzip_sw_fallback()) { + sw_fallback =3D 1; + } + + ret =3D qzInit(&q->sess, sw_fallback); + if (ret !=3D QZ_OK && ret !=3D QZ_DUPLICATE) { + err_msg =3D "qzInit failed"; + goto err_free_q; + } + + ret =3D qzGetDefaultsDeflate(¶ms); + if (ret !=3D QZ_OK) { + err_msg =3D "qzGetDefaultsDeflate failed"; + goto err_close; + } + + /* Make sure to use configured QATzip compression level. */ + params.common_params.comp_lvl =3D migrate_multifd_qatzip_level(); + + ret =3D qzSetupSessionDeflate(&q->sess, ¶ms); + if (ret !=3D QZ_OK && ret !=3D QZ_DUPLICATE) { + err_msg =3D "qzSetupSessionDeflate failed"; + goto err_close; + } + + /* + * Mimic multifd-zlib, which reserves extra space for the + * incoming packet. + */ + q->in_len =3D MULTIFD_PACKET_SIZE * 2; + q->in_buf =3D qzMalloc(q->in_len, 0, PINNED_MEM); + if (!q->in_buf) { + err_msg =3D "qzMalloc failed"; + goto err_close; + } + + q->out_len =3D MULTIFD_PACKET_SIZE; + q->out_buf =3D qzMalloc(q->out_len, 0, PINNED_MEM); + if (!q->out_buf) { + err_msg =3D "qzMalloc failed"; + goto err_free_inbuf; + } + return 0; + +err_free_inbuf: + qzFree(q->in_buf); +err_close: + qzClose(&q->sess); +err_free_q: + g_free(q); + error_setg(errp, "multifd %u: %s", p->id, err_msg); + return -1; } =20 +/** + * qatzip_recv_cleanup: Tear down QATzip session and release private buffe= rs. + * + * @param p Multifd channel params + * @return None + */ static void qatzip_recv_cleanup(MultiFDRecvParams *p) { + struct qatzip_data *q =3D p->data; + + /* Ignoring return values here due to function signature. */ + qzTeardownSession(&q->sess); + qzClose(&q->sess); + qzFree(q->in_buf); + qzFree(q->out_buf); + g_free(p->data); } =20 + +/** + * qatzip_recv_pages: Decompress pages and copy them to the appropriate + * locations. + * + * @param p Multifd channel params + * @param errp Pointer to error, which will be set in case of error + * @return 0 on success, -1 on error (and *errp will be set) + */ static int qatzip_recv_pages(MultiFDRecvParams *p, Error **errp) { + struct qatzip_data *q =3D p->data; + int ret; + unsigned int in_len, out_len; + uint32_t in_size =3D p->next_packet_size; + uint32_t expected_size =3D p->normal_num * p->page_size; uint32_t flags =3D p->flags & MULTIFD_FLAG_COMPRESSION_MASK; =20 - if (flags !=3D MULTIFD_FLAG_NOCOMP) { + if (in_size > q->in_len) { + error_setg(errp, "multifd %u: received unexpectedly large packet", + p->id); + return -1; + } + + if (flags !=3D MULTIFD_FLAG_QATZIP) { error_setg(errp, "multifd %u: flags received %x flags expected %x", - p->id, flags, MULTIFD_FLAG_NOCOMP); + p->id, flags, MULTIFD_FLAG_QATZIP); + return -1; + } + + ret =3D qio_channel_read_all(p->c, (void *)q->in_buf, in_size, errp); + if (ret !=3D 0) { + return ret; + } + + in_len =3D in_size; + out_len =3D q->out_len; + ret =3D qzDecompress(&q->sess, q->in_buf, &in_len, q->out_buf, &out_le= n); + if (ret !=3D QZ_OK) { + error_setg(errp, "multifd %u: qzDecompress failed", p->id); + return -1; + } + if (out_len !=3D expected_size) { + error_setg(errp, "multifd %u: packet size received %u size expecte= d %u", + p->id, out_len, expected_size); return -1; } + + /* Copy each page to its appropriate location. */ for (int i =3D 0; i < p->normal_num; i++) { - p->iov[i].iov_base =3D p->host + p->normal[i]; - p->iov[i].iov_len =3D p->page_size; + memcpy(p->host + p->normal[i], + q->out_buf + p->page_size * i, + p->page_size); } - return qio_channel_readv_all(p->c, p->iov, p->normal_num, errp); + return 0; } =20 static MultiFDMethods multifd_qatzip_ops =3D { --=20 2.30.2 From nobody Tue May 21 21:06:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1711493061; cv=none; d=zohomail.com; s=zohoarc; b=mJUtAwT2glq8eHv60I16mRKijZnYoWM3dPnCKMfRP5gq0RypRD+XzcgTVuKkkoR8pCUkePOOUreFi7uCb9Fv+Ib9ZphepSO7OnslmosTAFjG3ThJ+9hpMswkMN3daIt3L48qTfeeGx2G2zURNDhNSjfOi9ECz6Bs7meDDb0/ID8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1711493061; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=YvvN9bWVZWWSbtgiZ4urL26zjFTj2quV6f4+ZK80U+M=; b=h+jSFi1yrp08ULAwWnVEoei7JkyxqwQErGAei8dozXbfddHwhPbU+YTngKPLG/YN+luFLR6/QLvbzyRZzY0drTVdBgbUQaKFX8TNkMB0w9ckL6zvoBNX6SVsTZd07aUKrjwK/2rGi7motqfdzqIWQAp/r0mG0JdmxDzsHlVqt/Y= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1711493061139890.8084119773001; Tue, 26 Mar 2024 15:44:21 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rpFWN-0006B2-Ri; Tue, 26 Mar 2024 18:43:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rpFWM-0006At-Q3 for qemu-devel@nongnu.org; Tue, 26 Mar 2024 18:43:54 -0400 Received: from mail-yw1-x1131.google.com ([2607:f8b0:4864:20::1131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rpFWL-0007dK-4d for qemu-devel@nongnu.org; Tue, 26 Mar 2024 18:43:54 -0400 Received: by mail-yw1-x1131.google.com with SMTP id 00721157ae682-60a0599f6e5so54655377b3.3 for ; Tue, 26 Mar 2024 15:43:52 -0700 (PDT) Received: from n36-186-108.byted.org. ([147.160.184.146]) by smtp.gmail.com with ESMTPSA id x15-20020a05620a0ecf00b007887d30dbb7sm3397816qkm.60.2024.03.26.15.43.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 15:43:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1711493031; x=1712097831; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YvvN9bWVZWWSbtgiZ4urL26zjFTj2quV6f4+ZK80U+M=; b=Q7p0B/RZWDBWOEKBNZoxdP6IFrrF48xdjT9FsAWpd37PdChQjjNemDGf3WBDcmgw17 Ye2MNZ9zRgg+/JVIw82MFnjiy1pFteBNt3JDgtEFCJZZl0AHCLj/LlstIW1P+xXx18JK wNy8sPq2ZfzjDe4qFnlhOd0sI099Zomw4MOp55E8XH0SKzvd8g9Ceh2A1tEsPMG+gmsr 64reDAzstq7RokxQWTd+vV6raLvQm/UWNJg96pngpAup05/O0d0RGjUMnWzNzFBPHJdf BWupbpAnd36l0MOKyVLtK9DMxGyELzu2kMEpWUUiJb13lGlUOp54PK2Ghg2/yGoZ7t5j G2YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711493031; x=1712097831; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YvvN9bWVZWWSbtgiZ4urL26zjFTj2quV6f4+ZK80U+M=; b=oODE/Qs93SZJBb+RWwx6MOLFSASY1iJkXJkGTyCrPoegbk76PIOXcl0ngXlEc7WVtm QM19yY/UMN3zr3q5Oiq5vfVxzH7aNTI1egjvuE5RdccajjhqIYd5+yfKhN5GmtX/mV5b bMbmkyT922VISex8WmYekFZXLJCnwla3h/mYmUe1CpX5iPSe5HRIrX08WkhcUZJ8EVTw hvnGze+pL7j4rx0rRIJ/RRndIuwpjK1z1KDXmD+UMKeC0NQOEnYdYgTBA/FioRi5dpXB l+GGrW6qGiWFkk6uYRyK/Gw+1fEzlrkJQpNltMn9G+7h3L1fgzqdXOp7iDVCRfN0AA2k h+xw== X-Gm-Message-State: AOJu0YzhzbevKQxOQo+pxupO1ghUON1Y7qBAXpgHyJYTcmPYNFbdAKVK 4ZxeAl5Q92EOXNbRcyx5nZ9WX9Yjavets7QEy8Ac6UCfjvogslFCtjuKvqgKhWy5vWSSjG8jsBT r X-Google-Smtp-Source: AGHT+IHh1GOjlCNE6QSHQSYThThivrzCEZ3N2EXgN9Xc2LvH3iZonEYmqLMfJbwP3QpCwMPjLiWt0Q== X-Received: by 2002:a81:6c04:0:b0:60f:ff19:fde1 with SMTP id h4-20020a816c04000000b0060fff19fde1mr1103549ywc.36.1711493031410; Tue, 26 Mar 2024 15:43:51 -0700 (PDT) From: Bryan Zhang To: qemu-devel@nongnu.org Cc: peterx@redhat.com, farosas@suse.de, yuan1.liu@intel.com, berrange@redhat.com, nanhai.zou@intel.com, hao.xiang@linux.dev, Bryan Zhang Subject: [PATCH v2 5/5] tests/migration: Add integration test for 'qatzip' compression method Date: Tue, 26 Mar 2024 22:42:21 +0000 Message-Id: <20240326224221.3623014-6-bryan.zhang@bytedance.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240326224221.3623014-1-bryan.zhang@bytedance.com> References: <20240326224221.3623014-1-bryan.zhang@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::1131; envelope-from=bryan.zhang@bytedance.com; helo=mail-yw1-x1131.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1711493062857100007 Content-Type: text/plain; charset="utf-8" Adds an integration test for 'qatzip'. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Reviewed-by: Fabiano Rosas --- Revision: This commit now does some parameter setting to test that changing the 'multifd-qatzip-level' parameter works, and to enable software fallback so that the QATzip test can be run even if the test machine does not have QAT. tests/qtest/migration-test.c | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 83512bce85..997f0aa323 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -35,6 +35,10 @@ # endif /* CONFIG_TASN1 */ #endif /* CONFIG_GNUTLS */ =20 +#ifdef CONFIG_QATZIP +#include +#endif /* CONFIG_QATZIP */ + /* For dirty ring test; so far only x86_64 is supported */ #if defined(__linux__) && defined(HOST_X86_64) #include "linux/kvm.h" @@ -2676,6 +2680,22 @@ test_migrate_precopy_tcp_multifd_zstd_start(QTestSta= te *from, } #endif /* CONFIG_ZSTD */ =20 +#ifdef CONFIG_QATZIP +static void * +test_migrate_precopy_tcp_multifd_qatzip_start(QTestState *from, + QTestState *to) +{ + migrate_set_parameter_int(from, "multifd-qatzip-level", 2); + migrate_set_parameter_int(to, "multifd-qatzip-level", 2); + + /* SW fallback is disabled by default, so enable it for testing. */ + migrate_set_parameter_bool(from, "multifd-qatzip-sw-fallback", true); + migrate_set_parameter_bool(to, "multifd-qatzip-sw-fallback", true); + + return test_migrate_precopy_tcp_multifd_start_common(from, to, "qatzip= "); +} +#endif + static void test_multifd_tcp_none(void) { MigrateCommon args =3D { @@ -2711,6 +2731,17 @@ static void test_multifd_tcp_zstd(void) } #endif =20 +#ifdef CONFIG_QATZIP +static void test_multifd_tcp_qatzip(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "defer", + .start_hook =3D test_migrate_precopy_tcp_multifd_qatzip_start, + }; + test_precopy_common(&args); +} +#endif + #ifdef CONFIG_GNUTLS static void * test_migrate_multifd_tcp_tls_psk_start_match(QTestState *from, @@ -3600,6 +3631,10 @@ int main(int argc, char **argv) migration_test_add("/migration/multifd/tcp/plain/zstd", test_multifd_tcp_zstd); #endif +#ifdef CONFIG_QATZIP + migration_test_add("/migration/multifd/tcp/plain/qatzip", + test_multifd_tcp_qatzip); +#endif #ifdef CONFIG_GNUTLS migration_test_add("/migration/multifd/tcp/tls/psk/match", test_multifd_tcp_tls_psk_match); --=20 2.30.2