From nobody Wed Nov 27 21:29:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1697722838; cv=none; d=zohomail.com; s=zohoarc; b=HJoWF+SuwcpmuTNMLNbyOsJiPbR5cK2BITqe8EUH5ar1iiAM4IiwQDLN+sJnWtxn61lrbu8PjviWd3A4F6TfeGjgQTIizx9uX5BXjRFda/KYU6Bq6KgaZy/Bq1kdKB1kwofFKZ9rqciDo70f9JlfGzh08qgp1NREKitOz2bHaQU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1697722838; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=RAQ8eX33T0C055i5QdgUszdtKPCz1DK+9mp5MQJu6to=; b=JnPygIO42X2/MDo2/SlCdaDfaxoXo5sfwWsQJt5++XLKtzABaUeA9Icu7sQz3Ex2JVT6xd44drv9OHJaix+1TlEEa5iFJ1fMyx4/TaIp7Mxfrj3wQ4bgG5IXkUvGcAg1M0vGRWSzA+7wWuilsRmqPJZM8+OCGbEKXbGj6iQPObg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=@intel.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 169772283870573.1914620008597; Thu, 19 Oct 2023 06:40:38 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtTEt-0000jB-Oj; Thu, 19 Oct 2023 09:39:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtNyo-0003BS-Qr for qemu-devel@nongnu.org; Thu, 19 Oct 2023 04:02:08 -0400 Received: from mgamail.intel.com ([192.55.52.93]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtNym-0006MN-Ip for qemu-devel@nongnu.org; Thu, 19 Oct 2023 04:02:06 -0400 Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2023 01:02:02 -0700 Received: from sae-gw02.sh.intel.com (HELO localhost) ([10.239.45.110]) by fmsmga006.fm.intel.com with ESMTP; 19 Oct 2023 01:02:00 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697702524; x=1729238524; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wZYxJmWSfUug2p3jTY5+EH28LA+GLCN7OAdwgFeYHKU=; b=Cu4D4HLpI/WkweN+SJCBOctGynhfnz9ndiKlCJjGCiGawAWA/5EKE7ug bqQ7bXQSeVMvP8Bhhqrul8ifiYjmg9Pte09L3gJcvF7TPa6fLFVocUM2u 49f5YxvWaIXhK1L5il6YUfT7pud3IffOKSNlruceYC3W1JQE4U+U092U7 5OpOIt3MxIJZERofoahF5g1xA0aY7myqsrfnQ5TWglOEb8cd/ZEHAVkkn h/DauI8nHSQvKX5W6KmIErIhZvbgI9LEznEaGVc5loqe/tOqWC+Jdk+QW /DTZWZdHDO94HG1RNYg9DI40voZLnzm9ljMmQQwTGz3IEq+50JPixwI9n g==; X-IronPort-AV: E=McAfee;i="6600,9927,10867"; a="383418488" X-IronPort-AV: E=Sophos;i="6.03,236,1694761200"; d="scan'208";a="383418488" X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10867"; a="1004134861" X-IronPort-AV: E=Sophos;i="6.03,236,1694761200"; d="scan'208";a="1004134861" From: Yuan Liu To: quintela@redhat.com, peterx@redhat.com, farosas@suse.de, leobras@redhat.com Cc: qemu-devel@nongnu.org, yuan1.liu@intel.com, nanhai.zou@intel.com Subject: [PATCH 4/5] migration iaa-compress: Add IAA initialization and deinitialization Date: Thu, 19 Oct 2023 06:12:23 +0800 Message-Id: <20231018221224.599065-5-yuan1.liu@intel.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20231018221224.599065-1-yuan1.liu@intel.com> References: <20231018221224.599065-1-yuan1.liu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=192.55.52.93; envelope-from=yuan1.liu@intel.com; helo=mgamail.intel.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DATE_IN_PAST_06_12=1.543, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Thu, 19 Oct 2023 09:38:37 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @intel.com) X-ZM-MESSAGEID: 1697722840996100024 Content-Type: text/plain; charset="utf-8" This patch defines the structure for IAA jobs related to data compression and decompression, as well as the initialization and deinitialization processes for IAA. Signed-off-by: Yuan Liu Reviewed-by: Nanhai Zou --- migration/iaa-ram-compress.c | 152 +++++++++++++++++++++++++++++++++++ migration/iaa-ram-compress.h | 20 +++++ migration/meson.build | 1 + migration/ram-compress.c | 21 +++-- 4 files changed, 189 insertions(+), 5 deletions(-) create mode 100644 migration/iaa-ram-compress.c create mode 100644 migration/iaa-ram-compress.h diff --git a/migration/iaa-ram-compress.c b/migration/iaa-ram-compress.c new file mode 100644 index 0000000000..da45952594 --- /dev/null +++ b/migration/iaa-ram-compress.c @@ -0,0 +1,152 @@ +/* + * QEMU IAA compression support + * + * Copyright (c) 2023 Intel Corporation + * Written by: + * Yuan Liu + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include "qemu/cutils.h" +#include "qemu/error-report.h" +#include "migration.h" +#include "options.h" +#include "io/channel-null.h" +#include "exec/target_page.h" +#include "exec/ramblock.h" +#include "iaa-ram-compress.h" +#include "qpl/qpl.h" + +/* The IAA work queue maximum depth */ +#define IAA_JOB_NUM (512) + +typedef struct { + CompressResult result; + ram_addr_t offset; /* The offset of the compressed page in the block */ + RAMBlock *block; /* The block of the compressed page */ +} iaa_comp_param; + +typedef struct { + uint8_t *host; /* Target address for decompression page */ +} iaa_decomp_param; + +typedef struct IaaJob { + QSIMPLEQ_ENTRY(IaaJob) entry; + bool is_compression; + uint32_t in_len; + uint32_t out_len; + uint8_t *in_buf; + uint8_t *out_buf; + qpl_job *qpl; /* It is used to submit (de)compression work to IAA */ + union { + iaa_comp_param comp; + iaa_decomp_param decomp; + } param; +} IaaJob; + +typedef struct IaaJobPool { + uint32_t pos; + uint32_t cnt; + IaaJob *jobs[IAA_JOB_NUM]; + uint8_t *job_in_buf; /* The IAA device input buffers for all IAA jobs = */ + uint8_t *job_out_buf; /* The IAA device output buffers for all IAA job= s */ + size_t buf_size; +} IaaJobPool; + +static IaaJobPool iaa_job_pool; +/* This is used to record jobs that have been submitted but not yet comple= ted */ +static QSIMPLEQ_HEAD(, IaaJob) polling_queue =3D + QSIMPLEQ_HEAD_INITIALIZER(polling_queue= ); + +void iaa_compress_deinit(void) +{ + for (int i =3D 0; i < IAA_JOB_NUM; i++) { + if (iaa_job_pool.jobs[i]) { + if (iaa_job_pool.jobs[i]->qpl) { + qpl_fini_job(iaa_job_pool.jobs[i]->qpl); + g_free(iaa_job_pool.jobs[i]->qpl); + } + g_free(iaa_job_pool.jobs[i]); + } + } + if (iaa_job_pool.job_in_buf) { + munmap(iaa_job_pool.job_in_buf, iaa_job_pool.buf_size); + iaa_job_pool.job_in_buf =3D NULL; + } + if (iaa_job_pool.job_out_buf) { + munmap(iaa_job_pool.job_out_buf, iaa_job_pool.buf_size); + iaa_job_pool.job_out_buf =3D NULL; + } +} + +int iaa_compress_init(bool is_decompression) +{ + qpl_status status; + IaaJob *job =3D NULL; + uint32_t qpl_hw_size =3D 0; + int flags =3D MAP_PRIVATE | MAP_POPULATE | MAP_ANONYMOUS; + size_t buf_size =3D IAA_JOB_NUM * qemu_target_page_size(); + + QSIMPLEQ_INIT(&polling_queue); + memset(&iaa_job_pool, 0, sizeof(IaaJobPool)); + iaa_job_pool.buf_size =3D buf_size; + iaa_job_pool.job_out_buf =3D mmap(NULL, buf_size, PROT_READ | PROT_WRI= TE, + flags, -1, 0); + if (iaa_job_pool.job_out_buf =3D=3D MAP_FAILED) { + error_report("Failed to allocate iaa output buffer, error %s", + strerror(errno)); + return -1; + } + /* + * There is no need to allocate an input buffer for the compression + * function, the IAA hardware can directly access the virtual machine + * memory through the host address through Share Virtual Memory(SVM) + */ + if (is_decompression) { + iaa_job_pool.job_in_buf =3D mmap(NULL, buf_size, PROT_READ | PROT_= WRITE, + flags, -1, 0); + if (iaa_job_pool.job_in_buf =3D=3D MAP_FAILED) { + error_report("Failed to allocate iaa input buffer, error %s", + strerror(errno)); + goto init_err; + } + } + status =3D qpl_get_job_size(qpl_path_hardware, &qpl_hw_size); + if (status !=3D QPL_STS_OK) { + error_report("Failed to initialize iaa hardware, error %d", status= ); + goto init_err; + } + for (int i =3D 0; i < IAA_JOB_NUM; i++) { + size_t buf_offset =3D qemu_target_page_size() * i; + job =3D g_try_malloc0(sizeof(IaaJob)); + if (!job) { + error_report("Failed to allocate iaa job memory, error %s", + strerror(errno)); + goto init_err; + } + iaa_job_pool.jobs[i] =3D job; + job->qpl =3D g_try_malloc0(qpl_hw_size); + if (!job->qpl) { + error_report("Failed to allocate iaa qpl memory, error %s", + strerror(errno)); + goto init_err; + } + if (is_decompression) { + job->in_buf =3D iaa_job_pool.job_in_buf + buf_offset; + } + job->out_buf =3D iaa_job_pool.job_out_buf + buf_offset; + status =3D qpl_init_job(qpl_path_hardware, job->qpl); + if (status !=3D QPL_STS_OK) { + error_report("Failed to initialize iaa qpl, error %d", status); + goto init_err; + } + } + return 0; +init_err: + iaa_compress_deinit(); + return -1; +} diff --git a/migration/iaa-ram-compress.h b/migration/iaa-ram-compress.h new file mode 100644 index 0000000000..27998b255b --- /dev/null +++ b/migration/iaa-ram-compress.h @@ -0,0 +1,20 @@ +/* + * QEMU IAA compression support + * + * Copyright (c) 2023 Intel Corporation + * Written by: + * Yuan Liu + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + * + */ + +#ifndef QEMU_MIGRATION_IAA_COMPRESS_H +#define QEMU_MIGRATION_IAA_COMPRESS_H +#include "qemu-file.h" +#include "ram-compress.h" + +int iaa_compress_init(bool is_decompression); +void iaa_compress_deinit(void); +#endif diff --git a/migration/meson.build b/migration/meson.build index 92b1cc4297..9131815420 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -40,6 +40,7 @@ if get_option('live_block_migration').allowed() system_ss.add(files('block.c')) endif system_ss.add(when: zstd, if_true: files('multifd-zstd.c')) +system_ss.add(when: qpl, if_true: files('iaa-ram-compress.c')) =20 specific_ss.add(when: 'CONFIG_SYSTEM_ONLY', if_true: files('ram.c', diff --git a/migration/ram-compress.c b/migration/ram-compress.c index 47357352f7..acc511ce57 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -30,6 +30,9 @@ #include "qemu/cutils.h" =20 #include "ram-compress.h" +#ifdef CONFIG_QPL +#include "iaa-ram-compress.h" +#endif =20 #include "qemu/error-report.h" #include "migration.h" @@ -484,10 +487,11 @@ int ram_compress_save_setup(void) if (!migrate_compress()) { return 0; } +#ifdef CONFIG_QPL if (migrate_compress_with_iaa()) { - /* Implement in next patch */ - return 0; + return iaa_compress_init(false); } +#endif return compress_threads_save_setup(); } =20 @@ -496,10 +500,12 @@ void ram_compress_save_cleanup(void) if (!migrate_compress()) { return; } +#ifdef CONFIG_QPL if (migrate_compress_with_iaa()) { - /* Implement in next patch */ + iaa_compress_deinit(); return; } +#endif compress_threads_save_cleanup(); } =20 @@ -516,9 +522,11 @@ int ram_compress_load_setup(QEMUFile *f) if (!migrate_compress()) { return 0; } +#ifdef CONFIG_QPL if (migrate_compress_with_iaa()) { - /* Implement in next patch */ + return iaa_compress_init(true); } +#endif return compress_threads_load_setup(f); } =20 @@ -527,8 +535,11 @@ void ram_compress_load_cleanup(void) if (!migrate_compress()) { return; } +#ifdef CONFIG_QPL if (migrate_compress_with_iaa()) { - /* Implement in next patch */ + iaa_compress_deinit(); + return; } +#endif compress_threads_load_cleanup(); } --=20 2.39.3