From nobody Sun May 19 09:42:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1591895593; cv=none; d=zohomail.com; s=zohoarc; b=Ybh7FwSJk5gEcVaLiOQ5PCGxQwuO7jNYn8TbeHFd2pTdPp/ewconn+U3NDZyUfoB/lDgCwHVXctTjTjwhuhnKpaOQlUT9FkvvCXEV5cR0dCVF7++K58yR202I3LDvewhECPVTZZNaQXqMJfDu3o1gLUquxoiYEBCiSAiGgzSZ3M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591895593; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=NISZXn1sr/gwlVqxMv031+McV+QozaMzP5w7+ovJjQA=; b=PJF46wXPaxif990zjgm3my2NIZxWLc2NtHT6HGKHY7izK0s3QAImBTy7vK0cA+LZOj9cqvVSZZB5fc+V3jlxU1QpuVi1rehyLd/BdLFOGfecBiORtZcJzvfOcO3oaaz0ebis7cKOXXbKyQn4NLFTRM3FJWn73Ze9+xLUA38+ckw= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1591895593768273.86338677796164; Thu, 11 Jun 2020 10:13:13 -0700 (PDT) Received: from localhost ([::1]:34098 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jjQlL-0007fd-W6 for importer@patchew.org; Thu, 11 Jun 2020 13:13:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34032) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jjQk6-0005Sl-IF; Thu, 11 Jun 2020 13:11:54 -0400 Received: from relay.sw.ru ([185.231.240.75]:35191 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jjQk3-0000qk-6z; Thu, 11 Jun 2020 13:11:54 -0400 Received: from [192.168.15.81] (helo=iris.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jjQjs-0000BN-Ou; Thu, 11 Jun 2020 20:11:40 +0300 From: "Denis V. Lunev" To: qemu-block@nongnu.org, qemu-devel@nongnu.org Subject: [PATCH 1/4] migration/savevm: respect qemu_fclose() error code in save_snapshot() Date: Thu, 11 Jun 2020 20:11:40 +0300 Message-Id: <20200611171143.21589-2-den@openvz.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200611171143.21589-1-den@openvz.org> References: <20200611171143.21589-1-den@openvz.org> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=den@openvz.org; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/11 13:11:47 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Vladimir Sementsov-Ogievskiy , Juan Quintela , "Dr. David Alan Gilbert" , Max Reitz , Denis Plotnikov , Stefan Hajnoczi , "Denis V. Lunev" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" qemu_fclose() could return error, f.e. if bdrv_co_flush() will return the error. This validation will become more important once we will start waiting of asynchronous IO operations, started from bdrv_write_vmstate(), which are coming soon. Signed-off-by: Denis V. Lunev CC: Kevin Wolf CC: Max Reitz CC: Stefan Hajnoczi CC: Fam Zheng CC: Juan Quintela CC: "Dr. David Alan Gilbert" CC: Vladimir Sementsov-Ogievskiy CC: Denis Plotnikov Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Vladimir Sementsov-Ogievskiy --- migration/savevm.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/migration/savevm.c b/migration/savevm.c index c00a6807d9..0ff5bb40ed 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2628,7 +2628,7 @@ int save_snapshot(const char *name, Error **errp) { BlockDriverState *bs, *bs1; QEMUSnapshotInfo sn1, *sn =3D &sn1, old_sn1, *old_sn =3D &old_sn1; - int ret =3D -1; + int ret =3D -1, ret2; QEMUFile *f; int saved_vm_running; uint64_t vm_state_size; @@ -2712,10 +2712,14 @@ int save_snapshot(const char *name, Error **errp) } ret =3D qemu_savevm_state(f, errp); vm_state_size =3D qemu_ftell(f); - qemu_fclose(f); + ret2 =3D qemu_fclose(f); if (ret < 0) { goto the_end; } + if (ret2 < 0) { + ret =3D ret2; + goto the_end; + } =20 /* The bdrv_all_create_snapshot() call that follows acquires the AioCo= ntext * for itself. BDRV_POLL_WHILE() does not support nested locking beca= use --=20 2.17.1 From nobody Sun May 19 09:42:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1591895598; cv=none; d=zohomail.com; s=zohoarc; b=DIv+BV0nsf9aayCzNJZ8//Yjt6kDa0t/p0nPxfMCcX0UW2xyGzWqoupTBbmmrV3fViAU2+M0aGy09pUMYNuwxCXyIXGVjce73agNcITrobENAumgWBccsRMwXKJB9q1RYp/bkrhpUZQJ8ERALsHPNtFblGN7bFy7z/E5S07oOqU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591895598; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=56fwaQnF6c2ZMi+FkPy/fdc7bm+A0BcuYxoBo33t0u4=; b=a9cC11H7gGS+pYymHzbB+Yt/9a4ZEuyJb9zB1c54bu0Ppr02qfurr1+xbnZWqwbfjpz4oH3tQgfMKDvRTfapayOctTlpDRVMA8bWjdRnaQnZp44lV4BeHySxyVHtQCLTisKoBuhxSODXB2UGRX9WunaLnn3wENPh2ZHmFvsh0mw= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 159189559842617.088638448136635; Thu, 11 Jun 2020 10:13:18 -0700 (PDT) Received: from localhost ([::1]:34468 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jjQlQ-0007og-Sq for importer@patchew.org; Thu, 11 Jun 2020 13:13:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34002) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jjQk5-0005Qn-1B; Thu, 11 Jun 2020 13:11:53 -0400 Received: from relay.sw.ru ([185.231.240.75]:35190 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jjQk2-0000qg-OH; Thu, 11 Jun 2020 13:11:52 -0400 Received: from [192.168.15.81] (helo=iris.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jjQjs-0000BN-Sx; Thu, 11 Jun 2020 20:11:40 +0300 From: "Denis V. Lunev" To: qemu-block@nongnu.org, qemu-devel@nongnu.org Subject: [PATCH 2/4] block/aio_task: allow start/wait task from any coroutine Date: Thu, 11 Jun 2020 20:11:41 +0300 Message-Id: <20200611171143.21589-3-den@openvz.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200611171143.21589-1-den@openvz.org> References: <20200611171143.21589-1-den@openvz.org> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=den@openvz.org; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/11 13:11:47 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Vladimir Sementsov-Ogievskiy , Juan Quintela , "Dr. David Alan Gilbert" , Max Reitz , Denis Plotnikov , Stefan Hajnoczi , "Denis V . Lunev" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Vladimir Sementsov-Ogievskiy Currently, aio task pool assumes that there is a main coroutine, which creates tasks and wait for them. Let's remove the restriction by using CoQueue. Code becomes clearer, interface more obvious. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev CC: Kevin Wolf CC: Max Reitz CC: Stefan Hajnoczi CC: Fam Zheng CC: Juan Quintela CC: "Dr. David Alan Gilbert" CC: Vladimir Sementsov-Ogievskiy CC: Denis Plotnikov --- block/aio_task.c | 21 ++++++--------------- 1 file changed, 6 insertions(+), 15 deletions(-) diff --git a/block/aio_task.c b/block/aio_task.c index 88989fa248..cf62e5c58b 100644 --- a/block/aio_task.c +++ b/block/aio_task.c @@ -27,11 +27,10 @@ #include "block/aio_task.h" =20 struct AioTaskPool { - Coroutine *main_co; int status; int max_busy_tasks; int busy_tasks; - bool waiting; + CoQueue waiters; }; =20 static void coroutine_fn aio_task_co(void *opaque) @@ -52,31 +51,23 @@ static void coroutine_fn aio_task_co(void *opaque) =20 g_free(task); =20 - if (pool->waiting) { - pool->waiting =3D false; - aio_co_wake(pool->main_co); - } + qemu_co_queue_restart_all(&pool->waiters); } =20 void coroutine_fn aio_task_pool_wait_one(AioTaskPool *pool) { assert(pool->busy_tasks > 0); - assert(qemu_coroutine_self() =3D=3D pool->main_co); =20 - pool->waiting =3D true; - qemu_coroutine_yield(); + qemu_co_queue_wait(&pool->waiters, NULL); =20 - assert(!pool->waiting); assert(pool->busy_tasks < pool->max_busy_tasks); } =20 void coroutine_fn aio_task_pool_wait_slot(AioTaskPool *pool) { - if (pool->busy_tasks < pool->max_busy_tasks) { - return; + while (pool->busy_tasks >=3D pool->max_busy_tasks) { + aio_task_pool_wait_one(pool); } - - aio_task_pool_wait_one(pool); } =20 void coroutine_fn aio_task_pool_wait_all(AioTaskPool *pool) @@ -98,8 +89,8 @@ AioTaskPool *coroutine_fn aio_task_pool_new(int max_busy_= tasks) { AioTaskPool *pool =3D g_new0(AioTaskPool, 1); =20 - pool->main_co =3D qemu_coroutine_self(); pool->max_busy_tasks =3D max_busy_tasks; + qemu_co_queue_init(&pool->waiters); =20 return pool; } --=20 2.17.1 From nobody Sun May 19 09:42:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1591895747; cv=none; d=zohomail.com; s=zohoarc; b=OmVkzqEZeSK04/p8e0KYHa0/ymv/7dYjNjOnvFa5pm4OV59/RBrjb0YUKdcB/VObVPXU5pvHDRUvdqm5bVjkjKAcQwKfHdQilGDTbJKt+aV2hg5iR5CJFwXvF+iqA5fSUHuH9nOIFvvoZRGafmCjtDCe5iLUfj/seexO+7XOjXg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591895747; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=/rb/scnlD3vg1dsTfPnKkNpvKIo/KtSvb+abl63ynMg=; b=YMDmO2oieG3N8xOashsVe4j3Y47L/zF6brUDfw15bhlAcgjtbDUBOai7rpKj2LKi+c9u6uN0mFGXZYs1roPTS0rs+igABFKuMgA0XCgYfAb6NGqa9apCPbPwc9dspvR9Hq7hzavL0e1eDm4q0xCxkXwY7vj/L0DMk63mOaBZbok= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1591895747980809.3764206693667; Thu, 11 Jun 2020 10:15:47 -0700 (PDT) Received: from localhost ([::1]:43716 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jjQnq-0003JJ-MG for importer@patchew.org; Thu, 11 Jun 2020 13:15:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34044) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jjQk8-0005XS-2t; Thu, 11 Jun 2020 13:11:56 -0400 Received: from relay.sw.ru ([185.231.240.75]:35188 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jjQk2-0000qi-OE; Thu, 11 Jun 2020 13:11:55 -0400 Received: from [192.168.15.81] (helo=iris.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jjQjt-0000BN-0n; Thu, 11 Jun 2020 20:11:41 +0300 From: "Denis V. Lunev" To: qemu-block@nongnu.org, qemu-devel@nongnu.org Subject: [PATCH 3/4] block, migration: add bdrv_flush_vmstate helper Date: Thu, 11 Jun 2020 20:11:42 +0300 Message-Id: <20200611171143.21589-4-den@openvz.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200611171143.21589-1-den@openvz.org> References: <20200611171143.21589-1-den@openvz.org> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=den@openvz.org; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/11 13:11:47 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Vladimir Sementsov-Ogievskiy , Juan Quintela , "Dr. David Alan Gilbert" , Max Reitz , Denis Plotnikov , Stefan Hajnoczi , "Denis V. Lunev" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Right now bdrv_fclose() is just calling bdrv_flush(). The problem is that migration code is working inefficently from black layer terms and are frequently called for very small pieces of not properly aligned data. Block layer is capable to work this way, but this is very slow. This patch is a preparation for the introduction of the intermediate buffer at block driver state. It would be beneficial to separate conventional bdrv_flush() from closing QEMU file from migration code. The patch also forces bdrv_flush_vmstate() operation inside synchronous blk_save_vmstate() operation. This helper is used from qemu-io only. Signed-off-by: Denis V. Lunev CC: Kevin Wolf CC: Max Reitz CC: Stefan Hajnoczi CC: Fam Zheng CC: Juan Quintela CC: "Dr. David Alan Gilbert" CC: Vladimir Sementsov-Ogievskiy CC: Denis Plotnikov --- block/block-backend.c | 6 +++++- block/io.c | 39 +++++++++++++++++++++++++++++++++++++++ include/block/block.h | 1 + migration/savevm.c | 3 +++ 4 files changed, 48 insertions(+), 1 deletion(-) diff --git a/block/block-backend.c b/block/block-backend.c index 9342a475cb..2107ace699 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -2177,7 +2177,7 @@ int blk_truncate(BlockBackend *blk, int64_t offset, b= ool exact, int blk_save_vmstate(BlockBackend *blk, const uint8_t *buf, int64_t pos, int size) { - int ret; + int ret, ret2; =20 if (!blk_is_available(blk)) { return -ENOMEDIUM; @@ -2187,6 +2187,10 @@ int blk_save_vmstate(BlockBackend *blk, const uint8_= t *buf, if (ret < 0) { return ret; } + ret2 =3D bdrv_flush_vmstate(blk_bs(blk)); + if (ret2 < 0) { + return ret; + } =20 if (ret =3D=3D size && !blk->enable_write_cache) { ret =3D bdrv_flush(blk_bs(blk)); diff --git a/block/io.c b/block/io.c index 121ce17a49..fbf352f39d 100644 --- a/block/io.c +++ b/block/io.c @@ -2725,6 +2725,45 @@ int bdrv_readv_vmstate(BlockDriverState *bs, QEMUIOV= ector *qiov, int64_t pos) return bdrv_rw_vmstate(bs, qiov, pos, true); } =20 + +typedef struct FlushVMStateCo { + BlockDriverState *bs; + int ret; +} FlushVMStateCo; + +static int coroutine_fn bdrv_co_flush_vmstate(BlockDriverState *bs) +{ + return 0; +} + +static void coroutine_fn bdrv_flush_vmstate_co_entry(void *opaque) +{ + FlushVMStateCo *rwco =3D opaque; + + rwco->ret =3D bdrv_co_flush_vmstate(rwco->bs); + aio_wait_kick(); +} + +int bdrv_flush_vmstate(BlockDriverState *bs) +{ + Coroutine *co; + FlushVMStateCo flush_co =3D { + .bs =3D bs, + .ret =3D NOT_DONE, + }; + + if (qemu_in_coroutine()) { + /* Fast-path if already in coroutine context */ + bdrv_flush_vmstate_co_entry(&flush_co); + } else { + co =3D qemu_coroutine_create(bdrv_flush_vmstate_co_entry, &flush_c= o); + bdrv_coroutine_enter(bs, co); + BDRV_POLL_WHILE(bs, flush_co.ret =3D=3D NOT_DONE); + } + + return flush_co.ret; +} + /**************************************************************/ /* async I/Os */ =20 diff --git a/include/block/block.h b/include/block/block.h index 25e299605e..024525b87d 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -572,6 +572,7 @@ int bdrv_save_vmstate(BlockDriverState *bs, const uint8= _t *buf, =20 int bdrv_load_vmstate(BlockDriverState *bs, uint8_t *buf, int64_t pos, int size); +int bdrv_flush_vmstate(BlockDriverState *bs); =20 void bdrv_img_create(const char *filename, const char *fmt, const char *base_filename, const char *base_fmt, diff --git a/migration/savevm.c b/migration/savevm.c index 0ff5bb40ed..9698c909d7 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -150,6 +150,9 @@ static ssize_t block_get_buffer(void *opaque, uint8_t *= buf, int64_t pos, =20 static int bdrv_fclose(void *opaque, Error **errp) { + int err =3D bdrv_flush_vmstate(opaque); + if (err < 0) + return err; return bdrv_flush(opaque); } =20 --=20 2.17.1 From nobody Sun May 19 09:42:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1591895598; cv=none; d=zohomail.com; s=zohoarc; b=SqWfQpnMKrf5K2itHXoTAYTMY62hEbMgBji5TMe5MOXW7071w2QNRePv8C4klwCrkIJek/FVhbMDq6MRNolq/LbeS5MTa/Hs1f6fWPNDRUGP6jRjSXO9ay/DaY1jex+R/Q826vMokZbOyfJQpv7oRnBgVRCPcXIGXCmX8A5uQb0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591895598; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=UmgsMQhr3SPAoH1+FR3eXo8XOcmdWLRNwDKGjDfn5dg=; b=hp2jXOmejtYxKHYmff+QHi7UnYC8gcK3Z4/MmvjCsFZYxhSK+HWebnaipdPHQj+TRkCZx+ELXK8VFv4PtxjZPuXbULuIyJsxtm4LhdhM9YtfEyCZiVxIYPGSxhDruMrxSCvOk5SGSyMG9ofJRQaCm11bqYvdEdLCO7Ch5NEvktY= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1591895598235457.33192486438725; Thu, 11 Jun 2020 10:13:18 -0700 (PDT) Received: from localhost ([::1]:34446 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jjQlQ-0007o6-Ud for importer@patchew.org; Thu, 11 Jun 2020 13:13:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34004) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jjQk5-0005Qz-6d; Thu, 11 Jun 2020 13:11:53 -0400 Received: from relay.sw.ru ([185.231.240.75]:35194 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jjQk2-0000qj-O4; Thu, 11 Jun 2020 13:11:52 -0400 Received: from [192.168.15.81] (helo=iris.sw.ru) by relay3.sw.ru with esmtp (Exim 4.93) (envelope-from ) id 1jjQjt-0000BN-4x; Thu, 11 Jun 2020 20:11:41 +0300 From: "Denis V. Lunev" To: qemu-block@nongnu.org, qemu-devel@nongnu.org Subject: [PATCH 4/4] block/io: improve savevm performance Date: Thu, 11 Jun 2020 20:11:43 +0300 Message-Id: <20200611171143.21589-5-den@openvz.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200611171143.21589-1-den@openvz.org> References: <20200611171143.21589-1-den@openvz.org> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=den@openvz.org; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/11 13:11:47 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Vladimir Sementsov-Ogievskiy , Juan Quintela , "Dr. David Alan Gilbert" , Max Reitz , Denis Plotnikov , Stefan Hajnoczi , "Denis V. Lunev" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch does 2 standard basic things: - it creates intermediate buffer for all writes from QEMU migration code to block driver, - this buffer is sent to disk asynchronously, allowing several writes to run in parallel. Thus bdrv_vmstate_write() is becoming asynchronous. All pending operations completion are performed in newly invented bdrv_flush_vmstate(). In general, migration code is fantastically inefficent (by observation), buffers are not aligned and sent with arbitrary pieces, a lot of time less than 100 bytes at a chunk, which results in read-modify-write operations if target file descriptor is opened with O_DIRECT. It should also be noted that all operations are performed into unallocated image blocks, which also suffer due to partial writes to such new clusters even on cached file descriptors. Snapshot creation time (2 GB Fedora-31 VM running over NVME storage): original fixed cached: 1.79s 1.27s non-cached: 3.29s 0.81s The difference over HDD would be more significant :) Signed-off-by: Denis V. Lunev CC: Kevin Wolf CC: Max Reitz CC: Stefan Hajnoczi CC: Fam Zheng CC: Juan Quintela CC: "Dr. David Alan Gilbert" CC: Vladimir Sementsov-Ogievskiy CC: Denis Plotnikov --- block/io.c | 121 +++++++++++++++++++++++++++++++++++++- include/block/block_int.h | 8 +++ 2 files changed, 127 insertions(+), 2 deletions(-) diff --git a/block/io.c b/block/io.c index fbf352f39d..698f1eef76 100644 --- a/block/io.c +++ b/block/io.c @@ -26,6 +26,7 @@ #include "trace.h" #include "sysemu/block-backend.h" #include "block/aio-wait.h" +#include "block/aio_task.h" #include "block/blockjob.h" #include "block/blockjob_int.h" #include "block/block_int.h" @@ -2633,6 +2634,102 @@ typedef struct BdrvVmstateCo { int ret; } BdrvVmstateCo; =20 +typedef struct BdrvVMStateTask { + AioTask task; + + BlockDriverState *bs; + int64_t offset; + void *buf; + size_t bytes; +} BdrvVMStateTask; + +typedef struct BdrvSaveVMState { + AioTaskPool *pool; + BdrvVMStateTask *t; +} BdrvSaveVMState; + + +static coroutine_fn int bdrv_co_vmstate_save_task_entry(AioTask *task) +{ + int err =3D 0; + BdrvVMStateTask *t =3D container_of(task, BdrvVMStateTask, task); + + if (t->bytes !=3D 0) { + QEMUIOVector local_qiov; + qemu_iovec_init_buf(&local_qiov, t->buf, t->bytes); + + bdrv_inc_in_flight(t->bs); + err =3D t->bs->drv->bdrv_save_vmstate(t->bs, &local_qiov, t->offse= t); + bdrv_dec_in_flight(t->bs); + } + + qemu_vfree(t->buf); + return err; +} + +static BdrvVMStateTask *bdrv_vmstate_task_create(BlockDriverState *bs, + int64_t pos, size_t size) +{ + BdrvVMStateTask *t =3D g_new(BdrvVMStateTask, 1); + + *t =3D (BdrvVMStateTask) { + .task.func =3D bdrv_co_vmstate_save_task_entry, + .buf =3D qemu_blockalign(bs, size), + .offset =3D pos, + .bs =3D bs, + }; + + return t; +} + +static int bdrv_co_do_save_vmstate(BlockDriverState *bs, QEMUIOVector *qio= v, + int64_t pos) +{ + BdrvSaveVMState *state =3D bs->savevm_state; + BdrvVMStateTask *t; + size_t buf_size =3D MAX(bdrv_get_cluster_size(bs), 1 * MiB); + size_t to_copy; + size_t off; + + if (state =3D=3D NULL) { + state =3D g_new(BdrvSaveVMState, 1); + *state =3D (BdrvSaveVMState) { + .pool =3D aio_task_pool_new(BDRV_VMSTATE_WORKERS_MAX), + .t =3D bdrv_vmstate_task_create(bs, pos, buf_size), + }; + + bs->savevm_state =3D state; + } + + if (aio_task_pool_status(state->pool) < 0) { + return aio_task_pool_status(state->pool); + } + + t =3D state->t; + if (t->offset + t->bytes !=3D pos) { + /* Normally this branch is not reachable from migration */ + return bs->drv->bdrv_save_vmstate(bs, qiov, pos); + } + + off =3D 0; + while (1) { + to_copy =3D MIN(qiov->size - off, buf_size - t->bytes); + qemu_iovec_to_buf(qiov, off, t->buf + t->bytes, to_copy); + t->bytes +=3D to_copy; + if (t->bytes < buf_size) { + return qiov->size; + } + + aio_task_pool_start_task(state->pool, &t->task); + + pos +=3D to_copy; + off +=3D to_copy; + state->t =3D t =3D bdrv_vmstate_task_create(bs, pos, buf_size); + } + + return qiov->size; +} + static int coroutine_fn bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos, bool is_read) @@ -2648,7 +2745,7 @@ bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector= *qiov, int64_t pos, if (is_read) { ret =3D drv->bdrv_load_vmstate(bs, qiov, pos); } else { - ret =3D drv->bdrv_save_vmstate(bs, qiov, pos); + ret =3D bdrv_co_do_save_vmstate(bs, qiov, pos); } } else if (bs->file) { ret =3D bdrv_co_rw_vmstate(bs->file->bs, qiov, pos, is_read); @@ -2733,7 +2830,27 @@ typedef struct FlushVMStateCo { =20 static int coroutine_fn bdrv_co_flush_vmstate(BlockDriverState *bs) { - return 0; + int err; + BdrvSaveVMState *state =3D bs->savevm_state; + + if (bs->drv->bdrv_save_vmstate =3D=3D NULL && bs->file !=3D NULL) { + return bdrv_co_flush_vmstate(bs->file->bs); + } + if (state =3D=3D NULL) { + return 0; + } + + aio_task_pool_start_task(state->pool, &state->t->task); + + aio_task_pool_wait_all(state->pool); + err =3D aio_task_pool_status(state->pool); + + aio_task_pool_free(state->pool); + g_free(state); + + bs->savevm_state =3D NULL; + + return err; } =20 static void coroutine_fn bdrv_flush_vmstate_co_entry(void *opaque) diff --git a/include/block/block_int.h b/include/block/block_int.h index 791de6a59c..f90f0e8b6a 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -61,6 +61,8 @@ =20 #define BLOCK_PROBE_BUF_SIZE 512 =20 +#define BDRV_VMSTATE_WORKERS_MAX 8 + enum BdrvTrackedRequestType { BDRV_TRACKED_READ, BDRV_TRACKED_WRITE, @@ -784,6 +786,9 @@ struct BdrvChild { QLIST_ENTRY(BdrvChild) next_parent; }; =20 + +typedef struct BdrvSaveVMState BdrvSaveVMState; + /* * Note: the function bdrv_append() copies and swaps contents of * BlockDriverStates, so if you add new fields to this struct, please @@ -947,6 +952,9 @@ struct BlockDriverState { =20 /* BdrvChild links to this node may never be frozen */ bool never_freeze; + + /* Intermediate buffer for VM state saving from snapshot creation code= */ + BdrvSaveVMState *savevm_state; }; =20 struct BlockBackendRootState { --=20 2.17.1