From nobody Wed Oct 29 17:30:52 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 15246872047871018.901985350154; Wed, 25 Apr 2018 13:13:24 -0700 (PDT) Received: from localhost ([::1]:38827 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBQn5-00034K-W4 for importer@patchew.org; Wed, 25 Apr 2018 16:13:24 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45317) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fBQdj-0003bk-MO for qemu-devel@nongnu.org; Wed, 25 Apr 2018 16:03:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fBQdi-00033D-F9 for qemu-devel@nongnu.org; Wed, 25 Apr 2018 16:03:43 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:57522 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fBQdi-000330-AX for qemu-devel@nongnu.org; Wed, 25 Apr 2018 16:03:42 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DAD80406E973; Wed, 25 Apr 2018 20:03:41 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-116-232.ams2.redhat.com [10.36.116.232]) by smtp.corp.redhat.com (Postfix) with ESMTP id E9B19215CDC8; Wed, 25 Apr 2018 20:03:40 +0000 (UTC) From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, a.perevalov@samsung.com, xiaoguangrong@tencent.com Date: Wed, 25 Apr 2018 21:03:19 +0100 Message-Id: <20180425200326.72337-10-dgilbert@redhat.com> In-Reply-To: <20180425200326.72337-1-dgilbert@redhat.com> References: <20180425200326.72337-1-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 25 Apr 2018 20:03:41 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 25 Apr 2018 20:03:41 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'dgilbert@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 09/16] migration: stop decompression to allocate and free memory frequently X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong Current code uses uncompress() to decompress memory which manages memory internally, that causes huge memory is allocated and freed very frequently, more worse, frequently returning memory to kernel will flush TLBs So, we maintain the memory by ourselves and reuse it for each decompression Reviewed-by: Peter Xu Reviewed-by: Jiang Biao Signed-off-by: Xiao Guangrong Message-Id: <20180330075128.26919-4-xiaoguangrong@tencent.com> Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 112 +++++++++++++++++++++++++++++++++++------------- 1 file changed, 82 insertions(+), 30 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index a21514a469..fb24b2f32f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -281,6 +281,7 @@ struct DecompressParam { void *des; uint8_t *compbuf; int len; + z_stream stream; }; typedef struct DecompressParam DecompressParam; =20 @@ -2524,6 +2525,31 @@ void ram_handle_compressed(void *host, uint8_t ch, u= int64_t size) } } =20 +/* return the size after decompression, or negative value on error */ +static int +qemu_uncompress_data(z_stream *stream, uint8_t *dest, size_t dest_len, + const uint8_t *source, size_t source_len) +{ + int err; + + err =3D inflateReset(stream); + if (err !=3D Z_OK) { + return -1; + } + + stream->avail_in =3D source_len; + stream->next_in =3D (uint8_t *)source; + stream->avail_out =3D dest_len; + stream->next_out =3D dest; + + err =3D inflate(stream, Z_NO_FLUSH); + if (err !=3D Z_STREAM_END) { + return -1; + } + + return stream->total_out; +} + static void *do_data_decompress(void *opaque) { DecompressParam *param =3D opaque; @@ -2540,13 +2566,13 @@ static void *do_data_decompress(void *opaque) qemu_mutex_unlock(¶m->mutex); =20 pagesize =3D TARGET_PAGE_SIZE; - /* uncompress() will return failed in some case, especially - * when the page is dirted when doing the compression, it's - * not a problem because the dirty page will be retransferred + /* qemu_uncompress_data() will return failed in some case, + * especially when the page is dirtied when doing the compress= ion, + * it's not a problem because the dirty page will be retransfe= rred * and uncompress() won't break the data in other pages. */ - uncompress((Bytef *)des, &pagesize, - (const Bytef *)param->compbuf, len); + qemu_uncompress_data(¶m->stream, des, pagesize, param->com= pbuf, + len); =20 qemu_mutex_lock(&decomp_done_lock); param->done =3D true; @@ -2581,30 +2607,6 @@ static void wait_for_decompress_done(void) qemu_mutex_unlock(&decomp_done_lock); } =20 -static void compress_threads_load_setup(void) -{ - int i, thread_count; - - if (!migrate_use_compression()) { - return; - } - thread_count =3D migrate_decompress_threads(); - decompress_threads =3D g_new0(QemuThread, thread_count); - decomp_param =3D g_new0(DecompressParam, thread_count); - qemu_mutex_init(&decomp_done_lock); - qemu_cond_init(&decomp_done_cond); - for (i =3D 0; i < thread_count; i++) { - qemu_mutex_init(&decomp_param[i].mutex); - qemu_cond_init(&decomp_param[i].cond); - decomp_param[i].compbuf =3D g_malloc0(compressBound(TARGET_PAGE_SI= ZE)); - decomp_param[i].done =3D true; - decomp_param[i].quit =3D false; - qemu_thread_create(decompress_threads + i, "decompress", - do_data_decompress, decomp_param + i, - QEMU_THREAD_JOINABLE); - } -} - static void compress_threads_load_cleanup(void) { int i, thread_count; @@ -2614,16 +2616,30 @@ static void compress_threads_load_cleanup(void) } thread_count =3D migrate_decompress_threads(); for (i =3D 0; i < thread_count; i++) { + /* + * we use it as a indicator which shows if the thread is + * properly init'd or not + */ + if (!decomp_param[i].compbuf) { + break; + } + qemu_mutex_lock(&decomp_param[i].mutex); decomp_param[i].quit =3D true; qemu_cond_signal(&decomp_param[i].cond); qemu_mutex_unlock(&decomp_param[i].mutex); } for (i =3D 0; i < thread_count; i++) { + if (!decomp_param[i].compbuf) { + break; + } + qemu_thread_join(decompress_threads + i); qemu_mutex_destroy(&decomp_param[i].mutex); qemu_cond_destroy(&decomp_param[i].cond); + inflateEnd(&decomp_param[i].stream); g_free(decomp_param[i].compbuf); + decomp_param[i].compbuf =3D NULL; } g_free(decompress_threads); g_free(decomp_param); @@ -2631,6 +2647,39 @@ static void compress_threads_load_cleanup(void) decomp_param =3D NULL; } =20 +static int compress_threads_load_setup(void) +{ + int i, thread_count; + + if (!migrate_use_compression()) { + return 0; + } + + thread_count =3D migrate_decompress_threads(); + decompress_threads =3D g_new0(QemuThread, thread_count); + decomp_param =3D g_new0(DecompressParam, thread_count); + qemu_mutex_init(&decomp_done_lock); + qemu_cond_init(&decomp_done_cond); + for (i =3D 0; i < thread_count; i++) { + if (inflateInit(&decomp_param[i].stream) !=3D Z_OK) { + goto exit; + } + + decomp_param[i].compbuf =3D g_malloc0(compressBound(TARGET_PAGE_SI= ZE)); + qemu_mutex_init(&decomp_param[i].mutex); + qemu_cond_init(&decomp_param[i].cond); + decomp_param[i].done =3D true; + decomp_param[i].quit =3D false; + qemu_thread_create(decompress_threads + i, "decompress", + do_data_decompress, decomp_param + i, + QEMU_THREAD_JOINABLE); + } + return 0; +exit: + compress_threads_load_cleanup(); + return -1; +} + static void decompress_data_with_multi_threads(QEMUFile *f, void *host, int len) { @@ -2670,8 +2719,11 @@ static void decompress_data_with_multi_threads(QEMUF= ile *f, */ static int ram_load_setup(QEMUFile *f, void *opaque) { + if (compress_threads_load_setup()) { + return -1; + } + xbzrle_load_setup(); - compress_threads_load_setup(); ramblock_recv_map_init(); return 0; } --=20 2.17.0