From nobody Tue Feb 10 04:17:48 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1534940378515245.19421874554712; Wed, 22 Aug 2018 05:19:38 -0700 (PDT) Received: from localhost ([::1]:58756 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fsS6r-0007q4-EI for importer@patchew.org; Wed, 22 Aug 2018 08:19:37 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39115) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fsRpX-00070L-DX for qemu-devel@nongnu.org; Wed, 22 Aug 2018 08:01:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fsRpU-0000HN-6X for qemu-devel@nongnu.org; Wed, 22 Aug 2018 08:01:43 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:39656 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fsRpR-0000Fq-JH for qemu-devel@nongnu.org; Wed, 22 Aug 2018 08:01:38 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A60A140200A0; Wed, 22 Aug 2018 12:01:36 +0000 (UTC) Received: from secure.mitica (ovpn-116-35.ams2.redhat.com [10.36.116.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 65A97A9E6B; Wed, 22 Aug 2018 12:01:35 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 22 Aug 2018 14:00:49 +0200 Message-Id: <20180822120050.12694-20-quintela@redhat.com> In-Reply-To: <20180822120050.12694-1-quintela@redhat.com> References: <20180822120050.12694-1-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 22 Aug 2018 12:01:36 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 22 Aug 2018 12:01:36 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'quintela@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 19/20] migration: move handle of zero page to the thread X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, Xiao Guangrong , dgilbert@redhat.com, peterx@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RDMRC_1 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Xiao Guangrong Detecting zero page is not a light work, moving it to the thread to speed the main thread up, btw, handling ram_release_pages() for the zero page is moved to the thread as well Reviewed-by: Peter Xu Signed-off-by: Xiao Guangrong Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/ram.c | 96 +++++++++++++++++++++++++++++++++++-------------- 1 file changed, 70 insertions(+), 26 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 11066df881..40013e68a1 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -341,6 +341,7 @@ typedef struct PageSearchStatus PageSearchStatus; struct CompressParam { bool done; bool quit; + bool zero_page; QEMUFile *file; QemuMutex mutex; QemuCond cond; @@ -382,7 +383,7 @@ static QemuThread *decompress_threads; static QemuMutex decomp_done_lock; static QemuCond decomp_done_cond; =20 -static void do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, +static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, ram_addr_t offset, uint8_t *source_buf); =20 static void *do_data_compress(void *opaque) @@ -390,6 +391,7 @@ static void *do_data_compress(void *opaque) CompressParam *param =3D opaque; RAMBlock *block; ram_addr_t offset; + bool zero_page; =20 qemu_mutex_lock(¶m->mutex); while (!param->quit) { @@ -399,11 +401,12 @@ static void *do_data_compress(void *opaque) param->block =3D NULL; qemu_mutex_unlock(¶m->mutex); =20 - do_compress_ram_page(param->file, ¶m->stream, block, offse= t, - param->originbuf); + zero_page =3D do_compress_ram_page(param->file, ¶m->stream, + block, offset, param->originb= uf); =20 qemu_mutex_lock(&comp_done_lock); param->done =3D true; + param->zero_page =3D zero_page; qemu_cond_signal(&comp_done_cond); qemu_mutex_unlock(&comp_done_lock); =20 @@ -1849,13 +1852,19 @@ static int ram_save_multifd_page(RAMState *rs, RAMB= lock *block, return 1; } =20 -static void do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, +static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, ram_addr_t offset, uint8_t *source_buf) { RAMState *rs =3D ram_state; uint8_t *p =3D block->host + (offset & TARGET_PAGE_MASK); + bool zero_page =3D false; int ret; =20 + if (save_zero_page_to_file(rs, f, block, offset)) { + zero_page =3D true; + goto exit; + } + save_page_header(rs, f, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); =20 /* @@ -1868,10 +1877,21 @@ static void do_compress_ram_page(QEMUFile *f, z_str= eam *stream, RAMBlock *block, if (ret < 0) { qemu_file_set_error(migrate_get_current()->to_dst_file, ret); error_report("compressed data failed!"); - return; + return false; } =20 +exit: ram_release_pages(block->idstr, offset & TARGET_PAGE_MASK, 1); + return zero_page; +} + +static void +update_compress_thread_counts(const CompressParam *param, int bytes_xmit) +{ + if (param->zero_page) { + ram_counters.duplicate++; + } + ram_counters.transferred +=3D bytes_xmit; } =20 static void flush_compressed_data(RAMState *rs) @@ -1895,7 +1915,12 @@ static void flush_compressed_data(RAMState *rs) qemu_mutex_lock(&comp_param[idx].mutex); if (!comp_param[idx].quit) { len =3D qemu_put_qemu_file(rs->f, comp_param[idx].file); - ram_counters.transferred +=3D len; + /* + * it's safe to fetch zero_page without holding comp_done_lock + * as there is no further request submitted to the thread, + * i.e, the thread should be waiting for a request at this poi= nt. + */ + update_compress_thread_counts(&comp_param[idx], len); } qemu_mutex_unlock(&comp_param[idx].mutex); } @@ -1926,7 +1951,7 @@ retry: qemu_cond_signal(&comp_param[idx].cond); qemu_mutex_unlock(&comp_param[idx].mutex); pages =3D 1; - ram_counters.transferred +=3D bytes_xmit; + update_compress_thread_counts(&comp_param[idx], bytes_xmit); break; } } @@ -2200,6 +2225,39 @@ static bool save_page_use_compression(RAMState *rs) return false; } =20 +/* + * try to compress the page before posting it out, return true if the page + * has been properly handled by compression, otherwise needs other + * paths to handle it + */ +static bool save_compress_page(RAMState *rs, RAMBlock *block, ram_addr_t o= ffset) +{ + if (!save_page_use_compression(rs)) { + return false; + } + + /* + * When starting the process of a new block, the first page of + * the block should be sent out before other pages in the same + * block, and all the pages in last block should have been sent + * out, keeping this order is important, because the 'cont' flag + * is used to avoid resending the block name. + * + * We post the fist page as normal page as compression will take + * much CPU resource. + */ + if (block !=3D rs->last_sent_block) { + flush_compressed_data(rs); + return false; + } + + if (compress_page_with_multi_thread(rs, block, offset) > 0) { + return true; + } + + return false; +} + /** * ram_save_target_page: save one target page * @@ -2220,15 +2278,8 @@ static int ram_save_target_page(RAMState *rs, PageSe= archStatus *pss, return res; } =20 - /* - * When starting the process of a new block, the first page of - * the block should be sent out before other pages in the same - * block, and all the pages in last block should have been sent - * out, keeping this order is important, because the 'cont' flag - * is used to avoid resending the block name. - */ - if (block !=3D rs->last_sent_block && save_page_use_compression(rs)) { - flush_compressed_data(rs); + if (save_compress_page(rs, block, offset)) { + return 1; } =20 res =3D save_zero_page(rs, block, offset); @@ -2246,17 +2297,10 @@ static int ram_save_target_page(RAMState *rs, PageS= earchStatus *pss, } =20 /* - * Make sure the first page is sent out before other pages. - * - * we post it as normal page as compression will take much - * CPU resource. + * do not use multifd for compression as the first page in the new + * block should be posted out before sending the compressed page */ - if (block =3D=3D rs->last_sent_block && save_page_use_compression(rs))= { - res =3D compress_page_with_multi_thread(rs, block, offset); - if (res > 0) { - return res; - } - } else if (migrate_use_multifd()) { + if (!save_page_use_compression(rs) && migrate_use_multifd()) { return ram_save_multifd_page(rs, block, offset); } =20 --=20 2.17.1