From nobody Fri May 17 05:50:05 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1668612595; cv=none; d=zohomail.com; s=zohoarc; b=mOGKSivuUv/m3bcpg5DwWr3gS1a0J+cVpn073lm9m/LvM52BtKTtfdzpVwJppnUOoXcM/7HpTpJg81VrYoY7Hy4lLls+JYM7ALG1dM+/HOfgEs21GLE39dIq9EKtkJ1oYrhrGbahv9fQCfXU8oYZJnNvwtSsGjJjHwhLZ0cn/4M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1668612595; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=FoHXvHjAic9DByW4LjXkWh02PAmGl4DEqi7ZopzDiaw=; b=FlvCr+gRCMV5R+HZ2+DgmytFWGbSWg8YZiB5/Dmt19tYYUL1KTEDtSmv1Rolq45h/7aQG+oLqQgXiQhU/HNW1j2ffOj+WZbsj+k44UHTnv0Og9vkwcBlhhBJlY6SgOF/izWyus72nCYJ/Ldzxq5myvVvXxfH8j7Y4TrwWV23Gkw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1668612595516502.13425045526265; Wed, 16 Nov 2022 07:29:55 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ovKMG-0007bt-Fb; Wed, 16 Nov 2022 10:29:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ovKM7-0007Rg-Pr for qemu-devel@nongnu.org; Wed, 16 Nov 2022 10:29:41 -0500 Received: from mga02.intel.com ([134.134.136.20]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ovKM5-0004Yx-Fp for qemu-devel@nongnu.org; Wed, 16 Nov 2022 10:29:39 -0500 Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Nov 2022 07:29:35 -0800 Received: from xuling-b360m-d3h.sh.intel.com ([10.239.82.110]) by fmsmga008.fm.intel.com with ESMTP; 16 Nov 2022 07:29:33 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668612577; x=1700148577; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2jkxMTO5oskWbhtjRHfFSMgVTEU/aQcKMIbl87aE2oU=; b=PobbRKEn3k63m0M7xUqZkC89qz4Q9AunzK9s23LDEROxbhrv4mCGYUnF /LEDz7B89WoEgvgRN2BfWt129ce/Nw5pXvsZdSukzd99mvNgrPcGklvqR 2GaeL5dStc262F73AHA12gG9Tt+AqlQIJYR3Cyh7DcUR9Ywwmn5O7DZId slIVNkMXAr4HNIZNGLQG6Hmqy0KzjRJDdyxZEbgZmD78Cug/iWiu2VAKQ kMNF86Spn9M1+SrRkLuMKYM/CEcvbaOAXsyfGDIZ4DqvizJeDz+Hvobpf c+iRgY+FXe/Z9yz8rPFYk8t4oMFA4eyjikTuo0CLUTv3YeT5D/GudBiYz g==; X-IronPort-AV: E=McAfee;i="6500,9779,10533"; a="300099493" X-IronPort-AV: E=Sophos;i="5.96,167,1665471600"; d="scan'208";a="300099493" X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10533"; a="702911314" X-IronPort-AV: E=Sophos;i="5.96,167,1665471600"; d="scan'208";a="702911314" From: ling xu To: qemu-devel@nongnu.org Cc: quintela@redhat.com, dgilbert@redhat.com, ling xu , Zhou Zhao , Jun Jin Subject: [PATCH v7 1/2] AVX512 support for xbzrle_encode_buffer Date: Wed, 16 Nov 2022 23:29:22 +0800 Message-Id: <20221116152923.1087185-2-ling1.xu@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221116152923.1087185-1-ling1.xu@intel.com> References: <20221116152923.1087185-1-ling1.xu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=134.134.136.20; envelope-from=ling1.xu@intel.com; helo=mga02.intel.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @intel.com) X-ZM-MESSAGEID: 1668612596515100001 Content-Type: text/plain; charset="utf-8" This commit is the same with [PATCH v6 1/2], and provides avx512 support fo= r xbzrle_encode_buffer function to accelerate xbzrle encoding speed. Runtime check of avx512 support and benchmark for this feature are added. Compared with C version of xbzrle_encode_buffer function, avx512 version can achieve 50%-70% performance improvement on benchmarking. In addition, if dirty data is randomly located in 4K page, the avx512 version can achieve almost 140% performance gain. Signed-off-by: ling xu Co-authored-by: Zhou Zhao Co-authored-by: Jun Jin Reviewed-by: Juan Quintela --- meson.build | 16 ++++++ meson_options.txt | 2 + migration/ram.c | 34 +++++++++++-- migration/xbzrle.c | 124 +++++++++++++++++++++++++++++++++++++++++++++ migration/xbzrle.h | 4 ++ 5 files changed, 177 insertions(+), 3 deletions(-) diff --git a/meson.build b/meson.build index cf3e517e56..d0d28f5c9e 100644 --- a/meson.build +++ b/meson.build @@ -2344,6 +2344,22 @@ config_host_data.set('CONFIG_AVX512F_OPT', get_optio= n('avx512f') \ int main(int argc, char *argv[]) { return bar(argv[argc - 1]); } '''), error_message: 'AVX512F not available').allowed()) =20 +config_host_data.set('CONFIG_AVX512BW_OPT', get_option('avx512bw') \ + .require(have_cpuid_h, error_message: 'cpuid.h not available, cannot ena= ble AVX512BW') \ + .require(cc.links(''' + #pragma GCC push_options + #pragma GCC target("avx512bw") + #include + #include + static int bar(void *a) { + + __m512i *x =3D a; + __m512i res=3D _mm512_abs_epi8(*x); + return res[1]; + } + int main(int argc, char *argv[]) { return bar(argv[0]); } + '''), error_message: 'AVX512BW not available').allowed()) + have_pvrdma =3D get_option('pvrdma') \ .require(rdma.found(), error_message: 'PVRDMA requires OpenFabrics libra= ries') \ .require(cc.compiles(gnu_source_prefix + ''' diff --git a/meson_options.txt b/meson_options.txt index 66128178bf..96814dd211 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -104,6 +104,8 @@ option('avx2', type: 'feature', value: 'auto', description: 'AVX2 optimizations') option('avx512f', type: 'feature', value: 'disabled', description: 'AVX512F optimizations') +option('avx512bw', type: 'feature', value: 'auto', + description: 'AVX512BW optimizations') option('keyring', type: 'feature', value: 'auto', description: 'Linux keyring support') =20 diff --git a/migration/ram.c b/migration/ram.c index dc1de9ddbc..ff4c15c9c3 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -83,6 +83,34 @@ /* 0x80 is reserved in migration.h start with 0x100 next */ #define RAM_SAVE_FLAG_COMPRESS_PAGE 0x100 =20 +int (*xbzrle_encode_buffer_func)(uint8_t *, uint8_t *, int, + uint8_t *, int) =3D xbzrle_encode_buffer; +#if defined(CONFIG_AVX512BW_OPT) +#include "qemu/cpuid.h" +static void __attribute__((constructor)) init_cpu_flag(void) +{ + unsigned max =3D __get_cpuid_max(0, NULL); + int a, b, c, d; + if (max >=3D 1) { + __cpuid(1, a, b, c, d); + /* We must check that AVX is not just available, but usable. */ + if ((c & bit_OSXSAVE) && (c & bit_AVX) && max >=3D 7) { + int bv; + __asm("xgetbv" : "=3Da"(bv), "=3Dd"(d) : "c"(0)); + __cpuid_count(7, 0, a, b, c, d); + /* 0xe6: + * XCR0[7:5] =3D 111b (OPMASK state, upper 256-bit of ZMM0-ZMM= 15 + * and ZMM16-ZMM31 state are enabled by OS) + * XCR0[2:1] =3D 11b (XMM state and YMM state are enabled by O= S) + */ + if ((bv & 0xe6) =3D=3D 0xe6 && (b & bit_AVX512BW)) { + xbzrle_encode_buffer_func =3D xbzrle_encode_buffer_avx512; + } + } + } +} +#endif + XBZRLECacheStats xbzrle_counters; =20 /* struct contains XBZRLE cache and a static page @@ -802,9 +830,9 @@ static int save_xbzrle_page(RAMState *rs, uint8_t **cur= rent_data, memcpy(XBZRLE.current_buf, *current_data, TARGET_PAGE_SIZE); =20 /* XBZRLE encoding (if there is no overflow) */ - encoded_len =3D xbzrle_encode_buffer(prev_cached_page, XBZRLE.current_= buf, - TARGET_PAGE_SIZE, XBZRLE.encoded_bu= f, - TARGET_PAGE_SIZE); + encoded_len =3D xbzrle_encode_buffer_func(prev_cached_page, XBZRLE.cur= rent_buf, + TARGET_PAGE_SIZE, XBZRLE.encod= ed_buf, + TARGET_PAGE_SIZE); =20 /* * Update the cache contents, so that it corresponds to the data diff --git a/migration/xbzrle.c b/migration/xbzrle.c index 1ba482ded9..05366e86c0 100644 --- a/migration/xbzrle.c +++ b/migration/xbzrle.c @@ -174,3 +174,127 @@ int xbzrle_decode_buffer(uint8_t *src, int slen, uint= 8_t *dst, int dlen) =20 return d; } + +#if defined(CONFIG_AVX512BW_OPT) +#pragma GCC push_options +#pragma GCC target("avx512bw") +#include +int xbzrle_encode_buffer_avx512(uint8_t *old_buf, uint8_t *new_buf, int sl= en, + uint8_t *dst, int dlen) +{ + uint32_t zrun_len =3D 0, nzrun_len =3D 0; + int d =3D 0, i =3D 0, num =3D 0; + uint8_t *nzrun_start =3D NULL; + /* add 1 to include residual part in main loop */ + uint32_t count512s =3D (slen >> 6) + 1; + /* countResidual is tail of data, i.e., countResidual =3D slen % 64 */ + uint32_t count_residual =3D slen & 0b111111; + bool never_same =3D true; + uint64_t mask_residual =3D 1; + mask_residual <<=3D count_residual; + mask_residual -=3D 1; + __m512i r =3D _mm512_set1_epi32(0); + + while (count512s) { + if (d + 2 > dlen) { + return -1; + } + + int bytes_to_check =3D 64; + uint64_t mask =3D 0xffffffffffffffff; + if (count512s =3D=3D 1) { + bytes_to_check =3D count_residual; + mask =3D mask_residual; + } + __m512i old_data =3D _mm512_mask_loadu_epi8(r, + mask, old_buf + i); + __m512i new_data =3D _mm512_mask_loadu_epi8(r, + mask, new_buf + i); + uint64_t comp =3D _mm512_cmpeq_epi8_mask(old_data, new_data); + count512s--; + + bool is_same =3D (comp & 0x1); + while (bytes_to_check) { + if (is_same) { + if (nzrun_len) { + d +=3D uleb128_encode_small(dst + d, nzrun_len); + if (d + nzrun_len > dlen) { + return -1; + } + nzrun_start =3D new_buf + i - nzrun_len; + memcpy(dst + d, nzrun_start, nzrun_len); + d +=3D nzrun_len; + nzrun_len =3D 0; + } + /* 64 data at a time for speed */ + if (count512s && (comp =3D=3D 0xffffffffffffffff)) { + i +=3D 64; + zrun_len +=3D 64; + break; + } + never_same =3D false; + num =3D __builtin_ctzll(~comp); + num =3D (num < bytes_to_check) ? num : bytes_to_check; + zrun_len +=3D num; + bytes_to_check -=3D num; + comp >>=3D num; + i +=3D num; + if (bytes_to_check) { + /* still has different data after same data */ + d +=3D uleb128_encode_small(dst + d, zrun_len); + zrun_len =3D 0; + } else { + break; + } + } + if (never_same || zrun_len) { + /* + * never_same only acts if + * data begins with diff in first count512s + */ + d +=3D uleb128_encode_small(dst + d, zrun_len); + zrun_len =3D 0; + never_same =3D false; + } + /* has diff, 64 data at a time for speed */ + if ((bytes_to_check =3D=3D 64) && (comp =3D=3D 0x0)) { + i +=3D 64; + nzrun_len +=3D 64; + break; + } + num =3D __builtin_ctzll(comp); + num =3D (num < bytes_to_check) ? num : bytes_to_check; + nzrun_len +=3D num; + bytes_to_check -=3D num; + comp >>=3D num; + i +=3D num; + if (bytes_to_check) { + /* mask like 111000 */ + d +=3D uleb128_encode_small(dst + d, nzrun_len); + /* overflow */ + if (d + nzrun_len > dlen) { + return -1; + } + nzrun_start =3D new_buf + i - nzrun_len; + memcpy(dst + d, nzrun_start, nzrun_len); + d +=3D nzrun_len; + nzrun_len =3D 0; + is_same =3D true; + } + } + } + + if (nzrun_len !=3D 0) { + d +=3D uleb128_encode_small(dst + d, nzrun_len); + /* overflow */ + if (d + nzrun_len > dlen) { + return -1; + } + nzrun_start =3D new_buf + i - nzrun_len; + memcpy(dst + d, nzrun_start, nzrun_len); + d +=3D nzrun_len; + } + return d; +} +#pragma GCC pop_options +#endif diff --git a/migration/xbzrle.h b/migration/xbzrle.h index a0db507b9c..6feb49160a 100644 --- a/migration/xbzrle.h +++ b/migration/xbzrle.h @@ -18,4 +18,8 @@ int xbzrle_encode_buffer(uint8_t *old_buf, uint8_t *new_b= uf, int slen, uint8_t *dst, int dlen); =20 int xbzrle_decode_buffer(uint8_t *src, int slen, uint8_t *dst, int dlen); +#if defined(CONFIG_AVX512BW_OPT) +int xbzrle_encode_buffer_avx512(uint8_t *old_buf, uint8_t *new_buf, int sl= en, + uint8_t *dst, int dlen); +#endif #endif --=20 2.25.1 From nobody Fri May 17 05:50:05 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1668612628; cv=none; d=zohomail.com; s=zohoarc; b=fo1k5gfKunMpbik21L2jl70Pb22vXkmrCA2bpEb4Y+F1mhUPW39A6ZaNp+eunwxtiFixkVd4QI/gi7p2C9+txHF0Tvm7FEmQPKNWLNdkhZNGvX32IvUpIQhYGcaB25mqVDx2kwDmrANRVrrleM7qmdjHpbivqygpV3yO8/J3Fsw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1668612628; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Zyz6SNpbtRnz7XjkKrHzUA2OEjyhlORFuofyHmXv76k=; b=YUFW2yi5lHxOz7t3KrDxP6EBE1IhDkslGDc7xNeeXww2O1C+LL5bcizL7hlsrG0ChR87OYa7+b3HEf7eUz5rJq9V4Tr3pMmw1vlE8VnGAy4NorQn5uU293jGsif/WiT2VBlujsWALab3agQZiBF0BBqflHjOYBJZZgrd8c/mDPc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1668612628003614.2408963261099; Wed, 16 Nov 2022 07:30:28 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ovKMI-0007eS-99; Wed, 16 Nov 2022 10:29:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ovKMD-0007aF-Q8 for qemu-devel@nongnu.org; Wed, 16 Nov 2022 10:29:45 -0500 Received: from mga02.intel.com ([134.134.136.20]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ovKM9-0004Zi-CQ for qemu-devel@nongnu.org; Wed, 16 Nov 2022 10:29:45 -0500 Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Nov 2022 07:29:39 -0800 Received: from xuling-b360m-d3h.sh.intel.com ([10.239.82.110]) by fmsmga008.fm.intel.com with ESMTP; 16 Nov 2022 07:29:36 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668612581; x=1700148581; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7KrXBN5sWAs3frq8JOBhiTlHylwOw6NwOwPzedh4LXA=; b=MKpai/fIpi6x/DlvgFa0hYCFPD1ox86u023YVnC+76tiNRUrdy07MM9E 3JAso8SXNM1hN2YDFfJzVxdppOHsMVIHkvQNIzVJp9ctoQf8E28GYI9d1 li0+A9Fn8tFES7BJ5QBmcIux3DPY5rjFhbOKOXkwqr5uvoYuqunP43vZ9 JV1/QuybYWIPysNzhgWPXSZisZIOgcHs8Jply9rmQHXnrnsONcjsmEVF6 ZG+lplWckRSy7+F/mToeKG1nSxDKIujJM7Yaqa/M/sCqUNJjWjkkSjDtK 4d8B5FiYvabUJ2Ui1qWOZScIlVm/MNbKKrEKBO73t3NrlFbAdEHBINh9r A==; X-IronPort-AV: E=McAfee;i="6500,9779,10533"; a="300099511" X-IronPort-AV: E=Sophos;i="5.96,167,1665471600"; d="scan'208";a="300099511" X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10533"; a="702911321" X-IronPort-AV: E=Sophos;i="5.96,167,1665471600"; d="scan'208";a="702911321" From: ling xu To: qemu-devel@nongnu.org Cc: quintela@redhat.com, dgilbert@redhat.com, ling xu , Zhou Zhao , Jun Jin Subject: [PATCH v7 2/2] Update bench-code for addressing CI problem Date: Wed, 16 Nov 2022 23:29:23 +0800 Message-Id: <20221116152923.1087185-3-ling1.xu@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221116152923.1087185-1-ling1.xu@intel.com> References: <20221116152923.1087185-1-ling1.xu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=134.134.136.20; envelope-from=ling1.xu@intel.com; helo=mga02.intel.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @intel.com) X-ZM-MESSAGEID: 1668612628501100001 Content-Type: text/plain; charset="utf-8" Unit test code is in test-xbzrle.c, and benchmark code is in xbzrle-bench.c for performance benchmarking. we have modified xbzrle-bench.c to address CI problem. Signed-off-by: ling xu Co-authored-by: Zhou Zhao Co-authored-by: Jun Jin Reviewed-by: Juan Quintela --- tests/bench/meson.build | 4 + tests/bench/xbzrle-bench.c | 469 +++++++++++++++++++++++++++++++++++++ tests/unit/test-xbzrle.c | 39 ++- 3 files changed, 507 insertions(+), 5 deletions(-) create mode 100644 tests/bench/xbzrle-bench.c diff --git a/tests/bench/meson.build b/tests/bench/meson.build index 279a8fcc33..daefead58d 100644 --- a/tests/bench/meson.build +++ b/tests/bench/meson.build @@ -3,6 +3,10 @@ qht_bench =3D executable('qht-bench', sources: 'qht-bench.c', dependencies: [qemuutil]) =20 +xbzrle_bench =3D executable('xbzrle-bench', + sources: 'xbzrle-bench.c', + dependencies: [qemuutil,migration]) + executable('atomic_add-bench', sources: files('atomic_add-bench.c'), dependencies: [qemuutil], diff --git a/tests/bench/xbzrle-bench.c b/tests/bench/xbzrle-bench.c new file mode 100644 index 0000000000..8848a3a32d --- /dev/null +++ b/tests/bench/xbzrle-bench.c @@ -0,0 +1,469 @@ +/* + * Xor Based Zero Run Length Encoding unit tests. + * + * Copyright 2013 Red Hat, Inc. and/or its affiliates + * + * Authors: + * Orit Wasserman + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + * + */ +#include "qemu/osdep.h" +#include "qemu/cutils.h" +#include "../migration/xbzrle.h" + +#if defined(CONFIG_AVX512BW_OPT) +#define XBZRLE_PAGE_SIZE 4096 +static bool is_cpu_support_avx512bw; +#include "qemu/cpuid.h" +static void __attribute__((constructor)) init_cpu_flag(void) +{ + unsigned max =3D __get_cpuid_max(0, NULL); + int a, b, c, d; + is_cpu_support_avx512bw =3D false; + if (max >=3D 1) { + __cpuid(1, a, b, c, d); + /* We must check that AVX is not just available, but usable. */ + if ((c & bit_OSXSAVE) && (c & bit_AVX) && max >=3D 7) { + int bv; + __asm("xgetbv" : "=3Da"(bv), "=3Dd"(d) : "c"(0)); + __cpuid_count(7, 0, a, b, c, d); + /* 0xe6: + * XCR0[7:5] =3D 111b (OPMASK state, upper 256-bit of ZMM0-ZMM= 15 + * and ZMM16-ZMM31 state are enabled by OS) + * XCR0[2:1] =3D 11b (XMM state and YMM state are enabled by O= S) + */ + if ((bv & 0xe6) =3D=3D 0xe6 && (b & bit_AVX512BW)) { + is_cpu_support_avx512bw =3D true; + } + } + } + return ; +} + +struct ResTime { + float t_raw; + float t_512; +}; + + +/* Function prototypes +int xbzrle_encode_buffer_avx512(uint8_t *old_buf, uint8_t *new_buf, int sl= en, + uint8_t *dst, int dlen); +*/ +static void encode_decode_zero(struct ResTime *res) +{ + uint8_t *buffer =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *compressed =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *buffer512 =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *compressed512 =3D g_malloc0(XBZRLE_PAGE_SIZE); + int i =3D 0; + int dlen =3D 0, dlen512 =3D 0; + int diff_len =3D g_test_rand_int_range(0, XBZRLE_PAGE_SIZE - 1006); + + for (i =3D diff_len; i > 0; i--) { + buffer[1000 + i] =3D i; + buffer512[1000 + i] =3D i; + } + + buffer[1000 + diff_len + 3] =3D 103; + buffer[1000 + diff_len + 5] =3D 105; + + buffer512[1000 + diff_len + 3] =3D 103; + buffer512[1000 + diff_len + 5] =3D 105; + + /* encode zero page */ + time_t t_start, t_end, t_start512, t_end512; + t_start =3D clock(); + dlen =3D xbzrle_encode_buffer(buffer, buffer, XBZRLE_PAGE_SIZE, compre= ssed, + XBZRLE_PAGE_SIZE); + t_end =3D clock(); + float time_val =3D difftime(t_end, t_start); + g_assert(dlen =3D=3D 0); + + t_start512 =3D clock(); + dlen512 =3D xbzrle_encode_buffer_avx512(buffer512, buffer512, XBZRLE_P= AGE_SIZE, + compressed512, XBZRLE_PAGE_SIZE); + t_end512 =3D clock(); + float time_val512 =3D difftime(t_end512, t_start512); + g_assert(dlen512 =3D=3D 0); + + res->t_raw =3D time_val; + res->t_512 =3D time_val512; + + g_free(buffer); + g_free(compressed); + g_free(buffer512); + g_free(compressed512); + +} + +static void test_encode_decode_zero_avx512(void) +{ + int i; + float time_raw =3D 0.0, time_512 =3D 0.0; + struct ResTime res; + for (i =3D 0; i < 10000; i++) { + encode_decode_zero(&res); + time_raw +=3D res.t_raw; + time_512 +=3D res.t_512; + } + printf("Zero test:\n"); + printf("Raw xbzrle_encode time is %f ms\n", time_raw); + printf("512 xbzrle_encode time is %f ms\n", time_512); +} + +static void encode_decode_unchanged(struct ResTime *res) +{ + uint8_t *compressed =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *test =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *compressed512 =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *test512 =3D g_malloc0(XBZRLE_PAGE_SIZE); + int i =3D 0; + int dlen =3D 0, dlen512 =3D 0; + int diff_len =3D g_test_rand_int_range(0, XBZRLE_PAGE_SIZE - 1006); + + for (i =3D diff_len; i > 0; i--) { + test[1000 + i] =3D i + 4; + test512[1000 + i] =3D i + 4; + } + + test[1000 + diff_len + 3] =3D 107; + test[1000 + diff_len + 5] =3D 109; + + test512[1000 + diff_len + 3] =3D 107; + test512[1000 + diff_len + 5] =3D 109; + + /* test unchanged buffer */ + time_t t_start, t_end, t_start512, t_end512; + t_start =3D clock(); + dlen =3D xbzrle_encode_buffer(test, test, XBZRLE_PAGE_SIZE, compressed, + XBZRLE_PAGE_SIZE); + t_end =3D clock(); + float time_val =3D difftime(t_end, t_start); + g_assert(dlen =3D=3D 0); + + t_start512 =3D clock(); + dlen512 =3D xbzrle_encode_buffer_avx512(test512, test512, XBZRLE_PAGE_= SIZE, + compressed512, XBZRLE_PAGE_SIZE); + t_end512 =3D clock(); + float time_val512 =3D difftime(t_end512, t_start512); + g_assert(dlen512 =3D=3D 0); + + res->t_raw =3D time_val; + res->t_512 =3D time_val512; + + g_free(test); + g_free(compressed); + g_free(test512); + g_free(compressed512); + +} + +static void test_encode_decode_unchanged_avx512(void) +{ + int i; + float time_raw =3D 0.0, time_512 =3D 0.0; + struct ResTime res; + for (i =3D 0; i < 10000; i++) { + encode_decode_unchanged(&res); + time_raw +=3D res.t_raw; + time_512 +=3D res.t_512; + } + printf("Unchanged test:\n"); + printf("Raw xbzrle_encode time is %f ms\n", time_raw); + printf("512 xbzrle_encode time is %f ms\n", time_512); +} + +static void encode_decode_1_byte(struct ResTime *res) +{ + uint8_t *buffer =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *test =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *compressed =3D g_malloc(XBZRLE_PAGE_SIZE); + uint8_t *buffer512 =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *test512 =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *compressed512 =3D g_malloc(XBZRLE_PAGE_SIZE); + int dlen =3D 0, rc =3D 0, dlen512 =3D 0, rc512 =3D 0; + uint8_t buf[2]; + uint8_t buf512[2]; + + test[XBZRLE_PAGE_SIZE - 1] =3D 1; + test512[XBZRLE_PAGE_SIZE - 1] =3D 1; + + time_t t_start, t_end, t_start512, t_end512; + t_start =3D clock(); + dlen =3D xbzrle_encode_buffer(buffer, test, XBZRLE_PAGE_SIZE, compress= ed, + XBZRLE_PAGE_SIZE); + t_end =3D clock(); + float time_val =3D difftime(t_end, t_start); + g_assert(dlen =3D=3D (uleb128_encode_small(&buf[0], 4095) + 2)); + + rc =3D xbzrle_decode_buffer(compressed, dlen, buffer, XBZRLE_PAGE_SIZE= ); + g_assert(rc =3D=3D XBZRLE_PAGE_SIZE); + g_assert(memcmp(test, buffer, XBZRLE_PAGE_SIZE) =3D=3D 0); + + t_start512 =3D clock(); + dlen512 =3D xbzrle_encode_buffer_avx512(buffer512, test512, XBZRLE_PAG= E_SIZE, + compressed512, XBZRLE_PAGE_SIZE); + t_end512 =3D clock(); + float time_val512 =3D difftime(t_end512, t_start512); + g_assert(dlen512 =3D=3D (uleb128_encode_small(&buf512[0], 4095) + 2)); + + rc512 =3D xbzrle_decode_buffer(compressed512, dlen512, buffer512, + XBZRLE_PAGE_SIZE); + g_assert(rc512 =3D=3D XBZRLE_PAGE_SIZE); + g_assert(memcmp(test512, buffer512, XBZRLE_PAGE_SIZE) =3D=3D 0); + + res->t_raw =3D time_val; + res->t_512 =3D time_val512; + + g_free(buffer); + g_free(compressed); + g_free(test); + g_free(buffer512); + g_free(compressed512); + g_free(test512); + +} + +static void test_encode_decode_1_byte_avx512(void) +{ + int i; + float time_raw =3D 0.0, time_512 =3D 0.0; + struct ResTime res; + for (i =3D 0; i < 10000; i++) { + encode_decode_1_byte(&res); + time_raw +=3D res.t_raw; + time_512 +=3D res.t_512; + } + printf("1 byte test:\n"); + printf("Raw xbzrle_encode time is %f ms\n", time_raw); + printf("512 xbzrle_encode time is %f ms\n", time_512); +} + +static void encode_decode_overflow(struct ResTime *res) +{ + uint8_t *compressed =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *test =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *buffer =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *compressed512 =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *test512 =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *buffer512 =3D g_malloc0(XBZRLE_PAGE_SIZE); + int i =3D 0, rc =3D 0, rc512 =3D 0; + + for (i =3D 0; i < XBZRLE_PAGE_SIZE / 2 - 1; i++) { + test[i * 2] =3D 1; + test512[i * 2] =3D 1; + } + + /* encode overflow */ + time_t t_start, t_end, t_start512, t_end512; + t_start =3D clock(); + rc =3D xbzrle_encode_buffer(buffer, test, XBZRLE_PAGE_SIZE, compressed, + XBZRLE_PAGE_SIZE); + t_end =3D clock(); + float time_val =3D difftime(t_end, t_start); + g_assert(rc =3D=3D -1); + + t_start512 =3D clock(); + rc512 =3D xbzrle_encode_buffer_avx512(buffer512, test512, XBZRLE_PAGE_= SIZE, + compressed512, XBZRLE_PAGE_SIZE); + t_end512 =3D clock(); + float time_val512 =3D difftime(t_end512, t_start512); + g_assert(rc512 =3D=3D -1); + + res->t_raw =3D time_val; + res->t_512 =3D time_val512; + + g_free(buffer); + g_free(compressed); + g_free(test); + g_free(buffer512); + g_free(compressed512); + g_free(test512); + +} + +static void test_encode_decode_overflow_avx512(void) +{ + int i; + float time_raw =3D 0.0, time_512 =3D 0.0; + struct ResTime res; + for (i =3D 0; i < 10000; i++) { + encode_decode_overflow(&res); + time_raw +=3D res.t_raw; + time_512 +=3D res.t_512; + } + printf("Overflow test:\n"); + printf("Raw xbzrle_encode time is %f ms\n", time_raw); + printf("512 xbzrle_encode time is %f ms\n", time_512); +} + +static void encode_decode_range_avx512(struct ResTime *res) +{ + uint8_t *buffer =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *compressed =3D g_malloc(XBZRLE_PAGE_SIZE); + uint8_t *test =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *buffer512 =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *compressed512 =3D g_malloc(XBZRLE_PAGE_SIZE); + uint8_t *test512 =3D g_malloc0(XBZRLE_PAGE_SIZE); + int i =3D 0, rc =3D 0, rc512 =3D 0; + int dlen =3D 0, dlen512 =3D 0; + + int diff_len =3D g_test_rand_int_range(0, XBZRLE_PAGE_SIZE - 1006); + + for (i =3D diff_len; i > 0; i--) { + buffer[1000 + i] =3D i; + test[1000 + i] =3D i + 4; + buffer512[1000 + i] =3D i; + test512[1000 + i] =3D i + 4; + } + + buffer[1000 + diff_len + 3] =3D 103; + test[1000 + diff_len + 3] =3D 107; + + buffer[1000 + diff_len + 5] =3D 105; + test[1000 + diff_len + 5] =3D 109; + + buffer512[1000 + diff_len + 3] =3D 103; + test512[1000 + diff_len + 3] =3D 107; + + buffer512[1000 + diff_len + 5] =3D 105; + test512[1000 + diff_len + 5] =3D 109; + + /* test encode/decode */ + time_t t_start, t_end, t_start512, t_end512; + t_start =3D clock(); + dlen =3D xbzrle_encode_buffer(test, buffer, XBZRLE_PAGE_SIZE, compress= ed, + XBZRLE_PAGE_SIZE); + t_end =3D clock(); + float time_val =3D difftime(t_end, t_start); + rc =3D xbzrle_decode_buffer(compressed, dlen, test, XBZRLE_PAGE_SIZE); + g_assert(rc < XBZRLE_PAGE_SIZE); + g_assert(memcmp(test, buffer, XBZRLE_PAGE_SIZE) =3D=3D 0); + + t_start512 =3D clock(); + dlen512 =3D xbzrle_encode_buffer_avx512(test512, buffer512, XBZRLE_PAG= E_SIZE, + compressed512, XBZRLE_PAGE_SIZE); + t_end512 =3D clock(); + float time_val512 =3D difftime(t_end512, t_start512); + rc512 =3D xbzrle_decode_buffer(compressed512, dlen512, test512, XBZRLE= _PAGE_SIZE); + g_assert(rc512 < XBZRLE_PAGE_SIZE); + g_assert(memcmp(test512, buffer512, XBZRLE_PAGE_SIZE) =3D=3D 0); + + res->t_raw =3D time_val; + res->t_512 =3D time_val512; + + g_free(buffer); + g_free(compressed); + g_free(test); + g_free(buffer512); + g_free(compressed512); + g_free(test512); + +} + +static void test_encode_decode_avx512(void) +{ + int i; + float time_raw =3D 0.0, time_512 =3D 0.0; + struct ResTime res; + for (i =3D 0; i < 10000; i++) { + encode_decode_range_avx512(&res); + time_raw +=3D res.t_raw; + time_512 +=3D res.t_512; + } + printf("Encode decode test:\n"); + printf("Raw xbzrle_encode time is %f ms\n", time_raw); + printf("512 xbzrle_encode time is %f ms\n", time_512); +} + +static void encode_decode_random(struct ResTime *res) +{ + uint8_t *buffer =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *compressed =3D g_malloc(XBZRLE_PAGE_SIZE); + uint8_t *test =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *buffer512 =3D g_malloc0(XBZRLE_PAGE_SIZE); + uint8_t *compressed512 =3D g_malloc(XBZRLE_PAGE_SIZE); + uint8_t *test512 =3D g_malloc0(XBZRLE_PAGE_SIZE); + int i =3D 0, rc =3D 0, rc512 =3D 0; + int dlen =3D 0, dlen512 =3D 0; + + int diff_len =3D g_test_rand_int_range(0, XBZRLE_PAGE_SIZE - 1); + /* store the index of diff */ + int dirty_index[diff_len]; + for (int j =3D 0; j < diff_len; j++) { + dirty_index[j] =3D g_test_rand_int_range(0, XBZRLE_PAGE_SIZE - 1); + } + for (i =3D diff_len - 1; i >=3D 0; i--) { + buffer[dirty_index[i]] =3D i; + test[dirty_index[i]] =3D i + 4; + buffer512[dirty_index[i]] =3D i; + test512[dirty_index[i]] =3D i + 4; + } + + time_t t_start, t_end, t_start512, t_end512; + t_start =3D clock(); + dlen =3D xbzrle_encode_buffer(test, buffer, XBZRLE_PAGE_SIZE, compress= ed, + XBZRLE_PAGE_SIZE); + t_end =3D clock(); + float time_val =3D difftime(t_end, t_start); + rc =3D xbzrle_decode_buffer(compressed, dlen, test, XBZRLE_PAGE_SIZE); + g_assert(rc < XBZRLE_PAGE_SIZE); + + t_start512 =3D clock(); + dlen512 =3D xbzrle_encode_buffer_avx512(test512, buffer512, XBZRLE_PAG= E_SIZE, + compressed512, XBZRLE_PAGE_SIZE); + t_end512 =3D clock(); + float time_val512 =3D difftime(t_end512, t_start512); + rc512 =3D xbzrle_decode_buffer(compressed512, dlen512, test512, XBZRLE= _PAGE_SIZE); + g_assert(rc512 < XBZRLE_PAGE_SIZE); + + res->t_raw =3D time_val; + res->t_512 =3D time_val512; + + g_free(buffer); + g_free(compressed); + g_free(test); + g_free(buffer512); + g_free(compressed512); + g_free(test512); + +} + +static void test_encode_decode_random_avx512(void) +{ + int i; + float time_raw =3D 0.0, time_512 =3D 0.0; + struct ResTime res; + for (i =3D 0; i < 10000; i++) { + encode_decode_random(&res); + time_raw +=3D res.t_raw; + time_512 +=3D res.t_512; + } + printf("Random test:\n"); + printf("Raw xbzrle_encode time is %f ms\n", time_raw); + printf("512 xbzrle_encode time is %f ms\n", time_512); +} +#endif + +int main(int argc, char **argv) +{ + g_test_init(&argc, &argv, NULL); + g_test_rand_int(); + #if defined(CONFIG_AVX512BW_OPT) + if (likely(is_cpu_support_avx512bw)) { + g_test_add_func("/xbzrle/encode_decode_zero", test_encode_decode_z= ero_avx512); + g_test_add_func("/xbzrle/encode_decode_unchanged", + test_encode_decode_unchanged_avx512); + g_test_add_func("/xbzrle/encode_decode_1_byte", test_encode_decode= _1_byte_avx512); + g_test_add_func("/xbzrle/encode_decode_overflow", + test_encode_decode_overflow_avx512); + g_test_add_func("/xbzrle/encode_decode", test_encode_decode_avx512= ); + g_test_add_func("/xbzrle/encode_decode_random", test_encode_decode= _random_avx512); + } + #endif + return g_test_run(); +} diff --git a/tests/unit/test-xbzrle.c b/tests/unit/test-xbzrle.c index ef951b6e54..547046d093 100644 --- a/tests/unit/test-xbzrle.c +++ b/tests/unit/test-xbzrle.c @@ -16,6 +16,35 @@ =20 #define XBZRLE_PAGE_SIZE 4096 =20 +int (*xbzrle_encode_buffer_func)(uint8_t *, uint8_t *, int, + uint8_t *, int) =3D xbzrle_encode_buffer; +#if defined(CONFIG_AVX512BW_OPT) +#include "qemu/cpuid.h" +static void __attribute__((constructor)) init_cpu_flag(void) +{ + unsigned max =3D __get_cpuid_max(0, NULL); + int a, b, c, d; + if (max >=3D 1) { + __cpuid(1, a, b, c, d); + /* We must check that AVX is not just available, but usable. */ + if ((c & bit_OSXSAVE) && (c & bit_AVX) && max >=3D 7) { + int bv; + __asm("xgetbv" : "=3Da"(bv), "=3Dd"(d) : "c"(0)); + __cpuid_count(7, 0, a, b, c, d); + /* 0xe6: + * XCR0[7:5] =3D 111b (OPMASK state, upper 256-bit of ZMM0-ZMM= 15 + * and ZMM16-ZMM31 state are enabled by OS) + * XCR0[2:1] =3D 11b (XMM state and YMM state are enabled by O= S) + */ + if ((bv & 0xe6) =3D=3D 0xe6 && (b & bit_AVX512BW)) { + xbzrle_encode_buffer_func =3D xbzrle_encode_buffer_avx512; + } + } + } + return ; +} +#endif + static void test_uleb(void) { uint32_t i, val; @@ -54,7 +83,7 @@ static void test_encode_decode_zero(void) buffer[1000 + diff_len + 5] =3D 105; =20 /* encode zero page */ - dlen =3D xbzrle_encode_buffer(buffer, buffer, XBZRLE_PAGE_SIZE, compre= ssed, + dlen =3D xbzrle_encode_buffer_func(buffer, buffer, XBZRLE_PAGE_SIZE, c= ompressed, XBZRLE_PAGE_SIZE); g_assert(dlen =3D=3D 0); =20 @@ -78,7 +107,7 @@ static void test_encode_decode_unchanged(void) test[1000 + diff_len + 5] =3D 109; =20 /* test unchanged buffer */ - dlen =3D xbzrle_encode_buffer(test, test, XBZRLE_PAGE_SIZE, compressed, + dlen =3D xbzrle_encode_buffer_func(test, test, XBZRLE_PAGE_SIZE, compr= essed, XBZRLE_PAGE_SIZE); g_assert(dlen =3D=3D 0); =20 @@ -96,7 +125,7 @@ static void test_encode_decode_1_byte(void) =20 test[XBZRLE_PAGE_SIZE - 1] =3D 1; =20 - dlen =3D xbzrle_encode_buffer(buffer, test, XBZRLE_PAGE_SIZE, compress= ed, + dlen =3D xbzrle_encode_buffer_func(buffer, test, XBZRLE_PAGE_SIZE, com= pressed, XBZRLE_PAGE_SIZE); g_assert(dlen =3D=3D (uleb128_encode_small(&buf[0], 4095) + 2)); =20 @@ -121,7 +150,7 @@ static void test_encode_decode_overflow(void) } =20 /* encode overflow */ - rc =3D xbzrle_encode_buffer(buffer, test, XBZRLE_PAGE_SIZE, compressed, + rc =3D xbzrle_encode_buffer_func(buffer, test, XBZRLE_PAGE_SIZE, compr= essed, XBZRLE_PAGE_SIZE); g_assert(rc =3D=3D -1); =20 @@ -152,7 +181,7 @@ static void encode_decode_range(void) test[1000 + diff_len + 5] =3D 109; =20 /* test encode/decode */ - dlen =3D xbzrle_encode_buffer(test, buffer, XBZRLE_PAGE_SIZE, compress= ed, + dlen =3D xbzrle_encode_buffer_func(test, buffer, XBZRLE_PAGE_SIZE, com= pressed, XBZRLE_PAGE_SIZE); =20 rc =3D xbzrle_decode_buffer(compressed, dlen, test, XBZRLE_PAGE_SIZE); --=20 2.25.1