From nobody Tue Feb 10 13:18:00 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1710923709; cv=none; d=zohomail.com; s=zohoarc; b=f7J/estHt7F2Yqc2mVUctE4oT8pRsNisL+PiEQOlpETcRT3WNuYhvKeTeGKoVDB70TaQSj1dAY49PMA/fQiZ5CmmyyY8tRbkPcMntStQYyBaGHvoe8+S2yAO/s13ouY29HsknmK98hlTe/4GwrKxfuCwhacUPaO7PN5e5XQeHE0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1710923709; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=C+t0MeO6bIcsJ36G7s5Q2c4Gqta4ZtZMxc7SIsjoK4I=; b=dWh9/EXMmy6nXMD4tCQ4GHAueb1trnidtzlH90EC+I2SPKV0B96XOkZAXafjqiaV8o7ewU8olyIb5Dg8x6PZUCwoUQYc1QfYc/UhI5oGqUen2HgwuKMcb82FuPzfpa1X8ZT2YvLZLr3JR47BkRBqlrmQX+bm6Z+Ud1u5ctJLmMQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1710923709811234.53221238391768; Wed, 20 Mar 2024 01:35:09 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rmrO3-0004XN-ST; Wed, 20 Mar 2024 04:33:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rmrNv-0004Wa-KR for qemu-devel@nongnu.org; Wed, 20 Mar 2024 04:33:20 -0400 Received: from mgamail.intel.com ([198.175.65.18]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rmrNs-0005Th-GD for qemu-devel@nongnu.org; Wed, 20 Mar 2024 04:33:18 -0400 Received: from orviesa010.jf.intel.com ([10.64.159.150]) by orvoesa110.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Mar 2024 01:33:16 -0700 Received: from sae-gw02.sh.intel.com (HELO localhost) ([10.239.45.110]) by orviesa010.jf.intel.com with ESMTP; 20 Mar 2024 01:33:14 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1710923597; x=1742459597; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Ejrg1rOtZ3/ZvVXzfZ2it1XhOgKXcEWIjQNpiJPefHc=; b=DQt/zLS3CIiTikCs4iy/DGRjSPKgs4NyQDux2QVBayIgmsVlsiQEeyYl XvElZ2j+X++i8K/eVUGq6PJO7tv+S+GR6RcoKNSzfZXrv5ZZI8LSi2sxV xm/1JdrOxU7k/5Go9FBFKdew/o/GZwBG+SR33wZHY1Cz6V/B03mTf4Hu3 OYjeAEF3BtnKcOnlW/hC4Sum6Ms98bjXRl+8A9cvfjsule9ZTsNABQb7F nafKxN2/oWtBot4vd4Zk/u8XHRzy3O2NsaMDcuGb1cn8/CJjYLhY7ATQJ 8rtH3NaR6ZqJ7iKxMuFd3rPm0hwzfWOCsKK+mVpfbUsskkOdmS5vUuJJX g==; X-IronPort-AV: E=McAfee;i="6600,9927,11018"; a="5952946" X-IronPort-AV: E=Sophos;i="6.07,139,1708416000"; d="scan'208";a="5952946" X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,139,1708416000"; d="scan'208";a="13986732" From: Yuan Liu To: peterx@redhat.com, farosas@suse.de Cc: qemu-devel@nongnu.org, hao.xiang@bytedance.com, bryan.zhang@bytedance.com, yuan1.liu@intel.com, nanhai.zou@intel.com Subject: [PATCH v5 5/7] migration/multifd: implement initialization of qpl compression Date: Wed, 20 Mar 2024 00:45:25 +0800 Message-Id: <20240319164527.1873891-6-yuan1.liu@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240319164527.1873891-1-yuan1.liu@intel.com> References: <20240319164527.1873891-1-yuan1.liu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=198.175.65.18; envelope-from=yuan1.liu@intel.com; helo=mgamail.intel.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DATE_IN_PAST_12_24=1.049, DKIMWL_WL_HIGH=-0.422, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @intel.com) X-ZM-MESSAGEID: 1710923710724100001 Content-Type: text/plain; charset="utf-8" the qpl initialization includes memory allocation for compressed data and the qpl job initialization. the qpl initialization will check whether the In-Memory Analytics Accelerator(IAA) hardware is available, if the platform does not have IAA hardware or the IAA hardware is not available, the QPL compression initialization will fail. Signed-off-by: Yuan Liu Reviewed-by: Nanhai Zou --- migration/multifd-qpl.c | 243 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 242 insertions(+), 1 deletion(-) diff --git a/migration/multifd-qpl.c b/migration/multifd-qpl.c index 056a68a060..6de65e9da7 100644 --- a/migration/multifd-qpl.c +++ b/migration/multifd-qpl.c @@ -9,12 +9,253 @@ * This work is licensed under the terms of the GNU GPL, version 2 or late= r. * See the COPYING file in the top-level directory. */ + #include "qemu/osdep.h" #include "qemu/module.h" +#include "qapi/error.h" +#include "migration.h" +#include "multifd.h" +#include "qpl/qpl.h" + +typedef struct { + qpl_job **job_array; + /* the number of allocated jobs */ + uint32_t job_num; + /* the size of data processed by a qpl job */ + uint32_t data_size; + /* compressed data buffer */ + uint8_t *zbuf; + /* the length of compressed data */ + uint32_t *zbuf_hdr; +} QplData; + +static void free_zbuf(QplData *qpl) +{ + if (qpl->zbuf !=3D NULL) { + munmap(qpl->zbuf, qpl->job_num * qpl->data_size); + qpl->zbuf =3D NULL; + } + if (qpl->zbuf_hdr !=3D NULL) { + g_free(qpl->zbuf_hdr); + qpl->zbuf_hdr =3D NULL; + } +} + +static int alloc_zbuf(QplData *qpl, uint8_t chan_id, Error **errp) +{ + int flags =3D MAP_PRIVATE | MAP_POPULATE | MAP_ANONYMOUS; + uint32_t size =3D qpl->job_num * qpl->data_size; + uint8_t *buf; + + buf =3D (uint8_t *) mmap(NULL, size, PROT_READ | PROT_WRITE, flags, -1= , 0); + if (buf =3D=3D MAP_FAILED) { + error_setg(errp, "multifd: %u: alloc_zbuf failed, job num %u, size= %u", + chan_id, qpl->job_num, qpl->data_size); + return -1; + } + qpl->zbuf =3D buf; + qpl->zbuf_hdr =3D g_new0(uint32_t, qpl->job_num); + return 0; +} + +static void free_jobs(QplData *qpl) +{ + for (int i =3D 0; i < qpl->job_num; i++) { + qpl_fini_job(qpl->job_array[i]); + g_free(qpl->job_array[i]); + qpl->job_array[i] =3D NULL; + } + g_free(qpl->job_array); + qpl->job_array =3D NULL; +} + +static int alloc_jobs(QplData *qpl, uint8_t chan_id, Error **errp) +{ + qpl_status status; + uint32_t job_size =3D 0; + qpl_job *job =3D NULL; + /* always use IAA hardware accelerator */ + qpl_path_t path =3D qpl_path_hardware; + + status =3D qpl_get_job_size(path, &job_size); + if (status !=3D QPL_STS_OK) { + error_setg(errp, "multifd: %u: qpl_get_job_size failed with error = %d", + chan_id, status); + return -1; + } + qpl->job_array =3D g_new0(qpl_job *, qpl->job_num); + for (int i =3D 0; i < qpl->job_num; i++) { + job =3D g_malloc0(job_size); + status =3D qpl_init_job(path, job); + if (status !=3D QPL_STS_OK) { + error_setg(errp, "multifd: %u: qpl_init_job failed with error = %d", + chan_id, status); + free_jobs(qpl); + return -1; + } + qpl->job_array[i] =3D job; + } + return 0; +} + +static int init_qpl(QplData *qpl, uint32_t job_num, uint32_t data_size, + uint8_t chan_id, Error **errp) +{ + qpl->job_num =3D job_num; + qpl->data_size =3D data_size; + if (alloc_zbuf(qpl, chan_id, errp) !=3D 0) { + return -1; + } + if (alloc_jobs(qpl, chan_id, errp) !=3D 0) { + free_zbuf(qpl); + return -1; + } + return 0; +} + +static void deinit_qpl(QplData *qpl) +{ + if (qpl !=3D NULL) { + free_jobs(qpl); + free_zbuf(qpl); + qpl->job_num =3D 0; + qpl->data_size =3D 0; + } +} + +/** + * qpl_send_setup: setup send side + * + * Setup each channel with QPL compression. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static int qpl_send_setup(MultiFDSendParams *p, Error **errp) +{ + QplData *qpl; + + qpl =3D g_new0(QplData, 1); + if (init_qpl(qpl, p->page_count, p->page_size, p->id, errp) !=3D 0) { + g_free(qpl); + return -1; + } + p->compress_data =3D qpl; + + assert(p->iov =3D=3D NULL); + /* + * Each page will be compressed independently and sent using an IOV. T= he + * additional two IOVs are used to store packet header and compressed = data + * length + */ + p->iov =3D g_new0(struct iovec, p->page_count + 2); + return 0; +} + +/** + * qpl_send_cleanup: cleanup send side + * + * Close the channel and return memory. + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static void qpl_send_cleanup(MultiFDSendParams *p, Error **errp) +{ + QplData *qpl =3D p->compress_data; + + deinit_qpl(qpl); + g_free(p->compress_data); + p->compress_data =3D NULL; +} + +/** + * qpl_send_prepare: prepare data to be able to send + * + * Create a compressed buffer with all the pages that we are going to + * send. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static int qpl_send_prepare(MultiFDSendParams *p, Error **errp) +{ + /* Implement in next patch */ + return -1; +} + +/** + * qpl_recv_setup: setup receive side + * + * Create the compressed channel and buffer. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static int qpl_recv_setup(MultiFDRecvParams *p, Error **errp) +{ + QplData *qpl; + + qpl =3D g_new0(QplData, 1); + if (init_qpl(qpl, p->page_count, p->page_size, p->id, errp) !=3D 0) { + g_free(qpl); + return -1; + } + p->compress_data =3D qpl; + return 0; +} + +/** + * qpl_recv_cleanup: setup receive side + * + * Close the channel and return memory. + * + * @p: Params for the channel that we are using + */ +static void qpl_recv_cleanup(MultiFDRecvParams *p) +{ + QplData *qpl =3D p->compress_data; + + deinit_qpl(qpl); + g_free(p->compress_data); + p->compress_data =3D NULL; +} + +/** + * qpl_recv: read the data from the channel into actual pages + * + * Read the compressed buffer, and uncompress it into the actual + * pages. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static int qpl_recv(MultiFDRecvParams *p, Error **errp) +{ + /* Implement in next patch */ + return -1; +} + +static MultiFDMethods multifd_qpl_ops =3D { + .send_setup =3D qpl_send_setup, + .send_cleanup =3D qpl_send_cleanup, + .send_prepare =3D qpl_send_prepare, + .recv_setup =3D qpl_recv_setup, + .recv_cleanup =3D qpl_recv_cleanup, + .recv =3D qpl_recv, +}; =20 static void multifd_qpl_register(void) { - /* noop */ + multifd_register_ops(MULTIFD_COMPRESSION_QPL, &multifd_qpl_ops); } =20 migration_init(multifd_qpl_register); --=20 2.39.3