From nobody Mon Feb 9 09:33:20 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1604913197; cv=none; d=zohomail.com; s=zohoarc; b=hOYNF6ryoHXHuoIuQ66J2lG2TtRiPWs3cIUTA6YF5qJ7YUcuMxo2Rxpc84+3grMvCesaX+xtEVyaTWM+Yx44CMUmmdKemHcIdjbKdUCNG0jmQezwB8t+ODuDAL9tXpzcD8lIhOu8IuE+QbDjCg9jBucRITljV4g3iuCm3n9EVGo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1604913197; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=cZMyeTkSwr9QJ9kxn3bhi0nwvQTqVfBhuuhkAcTjyhk=; b=SxHILzVXz8I4aWCeAngd3LDQ6xZK+ZbxIjJ1WA0p/H0QD0BRgmUJlmtC8Bemv6M+Xuf6XvR7k7EwzL4nm+z0nl3PY9T36e3OvCy4CVvDzoaBQzbjwrBePlD5LueHt2OMAWvqdazyaisuZJ+mknJJrXQT/rsZ5+w6T1tqVUBRwz0= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1604913197596707.5128771148437; Mon, 9 Nov 2020 01:13:17 -0800 (PST) Received: from localhost ([::1]:52964 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kc3Ei-0008Li-Es for importer@patchew.org; Mon, 09 Nov 2020 04:13:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:53506) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kc3B0-0004Ut-HH for qemu-devel@nongnu.org; Mon, 09 Nov 2020 04:09:26 -0500 Received: from szxga06-in.huawei.com ([45.249.212.32]:2084) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kc3Aw-0006yv-U9 for qemu-devel@nongnu.org; Mon, 09 Nov 2020 04:09:26 -0500 Received: from DGGEMS409-HUB.china.huawei.com (unknown [172.30.72.60]) by szxga06-in.huawei.com (SkyGuard) with ESMTP id 4CV4tX5ZBWzhjCR; Mon, 9 Nov 2020 17:09:08 +0800 (CST) Received: from localhost (10.174.186.67) by DGGEMS409-HUB.china.huawei.com (10.3.19.209) with Microsoft SMTP Server id 14.3.487.0; Mon, 9 Nov 2020 17:09:02 +0800 From: Zeyu Jin To: , Subject: [RFC PATCH 2/6] migration: Refactoring multi-thread compress migration Date: Mon, 9 Nov 2020 17:08:46 +0800 Message-ID: <20201109090850.2424-3-jinzeyu@huawei.com> X-Mailer: git-send-email 2.28.0.windows.1 In-Reply-To: <20201109090850.2424-1-jinzeyu@huawei.com> References: <20201109090850.2424-1-jinzeyu@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.186.67] X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.32; envelope-from=jinzeyu@huawei.com; helo=szxga06-in.huawei.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/11/09 04:09:11 X-ACL-Warn: Detected OS = Linux 3.1-3.10 [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ying Fang , qemu-devel@nongnu.org, Zeyu Jin , zhang.zhanghailiang@huawei.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Code refactor for the compression procedure which includes: 1. Move qemu_compress_data and qemu_put_compression_data from qemu-file.c to ram.c, for the reason that most part of the code logical has nothing to do with qemu-file. Besides, the decompression code is located at ram.c only. 2. Simplify the function input arguments for compression and decompression. Wrap the input into the param structure which already exists. This change a= lso makes the function much more flexible for other compression methods. Signed-off-by: Zeyu Jin Signed-off-by: Ying Fang --- migration/qemu-file.c | 62 ++++++------------------------- migration/qemu-file.h | 4 +- migration/ram.c | 86 ++++++++++++++++++++++++++++++------------- 3 files changed, 75 insertions(+), 77 deletions(-) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index be21518c57..1efb667aa1 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -737,56 +737,6 @@ uint64_t qemu_get_be64(QEMUFile *f) return v; } =20 -/* return the size after compression, or negative value on error */ -static int qemu_compress_data(z_stream *stream, uint8_t *dest, size_t dest= _len, - const uint8_t *source, size_t source_len) -{ - int err; - - err =3D deflateReset(stream); - if (err !=3D Z_OK) { - return -1; - } - - stream->avail_in =3D source_len; - stream->next_in =3D (uint8_t *)source; - stream->avail_out =3D dest_len; - stream->next_out =3D dest; - - err =3D deflate(stream, Z_FINISH); - if (err !=3D Z_STREAM_END) { - return -1; - } - - return stream->next_out - dest; -} - -/* Compress size bytes of data start at p and store the compressed - * data to the buffer of f. - * - * Since the file is dummy file with empty_ops, return -1 if f has no spac= e to - * save the compressed data. - */ -ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream, - const uint8_t *p, size_t size) -{ - ssize_t blen =3D IO_BUF_SIZE - f->buf_index - sizeof(int32_t); - - if (blen < compressBound(size)) { - return -1; - } - - blen =3D qemu_compress_data(stream, f->buf + f->buf_index + sizeof(int= 32_t), - blen, p, size); - if (blen < 0) { - return -1; - } - - qemu_put_be32(f, blen); - add_buf_to_iovec(f, blen); - return blen + sizeof(int32_t); -} - /* Put the data in the buffer of f_src to the buffer of f_des, and * then reset the buf_index of f_src to 0. */ @@ -846,3 +796,15 @@ void qemu_file_set_blocking(QEMUFile *f, bool block) f->ops->set_blocking(f->opaque, block, NULL); } } + +ssize_t qemu_put_compress_start(QEMUFile *f, uint8_t **dest_ptr) +{ + *dest_ptr =3D f->buf + f->buf_index + sizeof(int32_t); + return IO_BUF_SIZE - f->buf_index - sizeof(int32_t); +} + +void qemu_put_compress_end(QEMUFile *f, unsigned int v) +{ + qemu_put_be32(f, v); + add_buf_to_iovec(f, v); +} diff --git a/migration/qemu-file.h b/migration/qemu-file.h index a9b6d6ccb7..1ac1566460 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -138,8 +138,6 @@ bool qemu_file_is_writable(QEMUFile *f); =20 size_t qemu_peek_buffer(QEMUFile *f, uint8_t **buf, size_t size, size_t of= fset); size_t qemu_get_buffer_in_place(QEMUFile *f, uint8_t **buf, size_t size); -ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream, - const uint8_t *p, size_t size); int qemu_put_qemu_file(QEMUFile *f_des, QEMUFile *f_src); =20 /* @@ -166,6 +164,8 @@ void ram_control_before_iterate(QEMUFile *f, uint64_t f= lags); void ram_control_after_iterate(QEMUFile *f, uint64_t flags); void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data); =20 +ssize_t qemu_put_compress_start(QEMUFile *f, uint8_t **dest_ptr); +void qemu_put_compress_end(QEMUFile *f, unsigned int v); /* Whenever this is found in the data stream, the flags * will be passed to ram_control_load_hook in the incoming-migration * side. This lets before_ram_iterate/after_ram_iterate add diff --git a/migration/ram.c b/migration/ram.c index 2da2b622ab..75504540c9 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -453,27 +453,22 @@ static QemuThread *decompress_threads; static QemuMutex decomp_done_lock; static QemuCond decomp_done_cond; =20 -static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, - ram_addr_t offset, uint8_t *source_buf); +static bool do_compress_ram_page(CompressParam *param, RAMBlock *block); =20 static void *do_data_compress(void *opaque) { CompressParam *param =3D opaque; RAMBlock *block; - ram_addr_t offset; bool zero_page; =20 qemu_mutex_lock(¶m->mutex); while (!param->quit) { if (param->block) { block =3D param->block; - offset =3D param->offset; param->block =3D NULL; qemu_mutex_unlock(¶m->mutex); =20 - zero_page =3D do_compress_ram_page(param->file, ¶m->stream, - block, offset, param->originb= uf); - + zero_page =3D do_compress_ram_page(param, block); qemu_mutex_lock(&comp_done_lock); param->done =3D true; param->zero_page =3D zero_page; @@ -1214,28 +1209,73 @@ static int ram_save_multifd_page(RAMState *rs, RAMB= lock *block, return 1; } =20 -static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, - ram_addr_t offset, uint8_t *source_buf) +/* + * Compress size bytes of data start at p and store the compressed + * data to the buffer of f. + * + * Since the file is dummy file with empty_ops, return -1 if f has no spac= e to + * save the compressed data. + */ +static ssize_t qemu_put_compression_data(CompressParam *param, size_t size) +{ + int err; + uint8_t *dest =3D NULL; + z_stream *stream =3D ¶m->stream; + uint8_t *p =3D param->originbuf; + QEMUFile *f =3D f =3D param->file; + ssize_t blen =3D qemu_put_compress_start(f, &dest); + + if (blen < compressBound(size)) { + return -1; + } + + err =3D deflateReset(stream); + if (err !=3D Z_OK) { + return -1; + } + + stream->avail_in =3D size; + stream->next_in =3D p; + stream->avail_out =3D blen; + stream->next_out =3D dest; + + err =3D deflate(stream, Z_FINISH); + if (err !=3D Z_STREAM_END) { + return -1; + } + + blen =3D stream->next_out - dest; + if (blen < 0) { + return -1; + } + + qemu_put_compress_end(f, blen); + return blen + sizeof(int32_t); +} + +static bool do_compress_ram_page(CompressParam *param, RAMBlock *block) { RAMState *rs =3D ram_state; + ram_addr_t offset =3D param->offset; uint8_t *p =3D block->host + (offset & TARGET_PAGE_MASK); bool zero_page =3D false; int ret; =20 - if (save_zero_page_to_file(rs, f, block, offset)) { + if (save_zero_page_to_file(rs, param->file, block, offset)) { zero_page =3D true; goto exit; } =20 - save_page_header(rs, f, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); + save_page_header(rs, param->file, block, + offset | RAM_SAVE_FLAG_COMPRESS_PAGE); =20 /* * copy it to a internal buffer to avoid it being modified by VM * so that we can catch up the error during compression and * decompression */ - memcpy(source_buf, p, TARGET_PAGE_SIZE); - ret =3D qemu_put_compression_data(f, stream, source_buf, TARGET_PAGE_S= IZE); + memcpy(param->originbuf, p, TARGET_PAGE_SIZE); + ret =3D qemu_put_compression_data(param, TARGET_PAGE_SIZE); if (ret < 0) { qemu_file_set_error(migrate_get_current()->to_dst_file, ret); error_report("compressed data failed!"); @@ -2826,19 +2866,20 @@ void ram_handle_compressed(void *host, uint8_t ch, = uint64_t size) =20 /* return the size after decompression, or negative value on error */ static int -qemu_uncompress_data(z_stream *stream, uint8_t *dest, size_t dest_len, - const uint8_t *source, size_t source_len) +qemu_uncompress_data(DecompressParam *param, uint8_t *dest, size_t pagesiz= e) { int err; =20 + z_stream *stream =3D ¶m->stream; + err =3D inflateReset(stream); if (err !=3D Z_OK) { return -1; } =20 - stream->avail_in =3D source_len; - stream->next_in =3D (uint8_t *)source; - stream->avail_out =3D dest_len; + stream->avail_in =3D param->len; + stream->next_in =3D param->compbuf; + stream->avail_out =3D pagesize; stream->next_out =3D dest; =20 err =3D inflate(stream, Z_NO_FLUSH); @@ -2852,22 +2893,17 @@ qemu_uncompress_data(z_stream *stream, uint8_t *des= t, size_t dest_len, static void *do_data_decompress(void *opaque) { DecompressParam *param =3D opaque; - unsigned long pagesize; uint8_t *des; - int len, ret; + int ret; =20 qemu_mutex_lock(¶m->mutex); while (!param->quit) { if (param->des) { des =3D param->des; - len =3D param->len; param->des =3D 0; qemu_mutex_unlock(¶m->mutex); =20 - pagesize =3D TARGET_PAGE_SIZE; - - ret =3D qemu_uncompress_data(¶m->stream, des, pagesize, - param->compbuf, len); + ret =3D qemu_uncompress_data(param, des, TARGET_PAGE_SIZE); if (ret < 0 && migrate_get_current()->decompress_error_check) { error_report("decompress data failed"); qemu_file_set_error(decomp_file, ret); --=20 2.23.0