From nobody Mon May 20 00:21:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562039888; cv=none; d=zoho.com; s=zohoarc; b=TYd/xaQxoFOAJNtVJlnyFkb6qQDuycfGB4GPI/61ZZKp4xruPpmL5pZWZhKOEfyDsWcVWL22T/KS7/f2Z27FMiGrfNj+KX7E62iw6w58OwTLi199Uy+6n3tsHa11Xb2+a+cZ6qVWcanRKQou+x42aLmJCKZ93Ue1uPw1WmZkEho= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562039888; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=SDhuTq4hpTcejS84E+f5QncCDok7L7NyYGmVAhWBse4=; b=OOoZtljZXvDzmKxl7fC2F+z3uGPonxiXd/cdQOnRtmiA+uGT908eel+f6qwBPoXG5IfA61y8feI6gS+aQtCQYaUcUyfYgqCOpQFMGGm7W3xnsZ/A8nX0s+sHo4VRBKEBQWG+/9d25xJEJWV5Pd9u55Wrcc+vY32aqvDhp8h3MY0= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562039888340940.3110445049966; Mon, 1 Jul 2019 20:58:08 -0700 (PDT) Received: from localhost ([::1]:48305 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hi9vh-00007e-5H for importer@patchew.org; Mon, 01 Jul 2019 23:58:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51610) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hi8hR-00047x-0P for qemu-devel@nongnu.org; Mon, 01 Jul 2019 22:39:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hi8hN-00070T-H7 for qemu-devel@nongnu.org; Mon, 01 Jul 2019 22:39:15 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46468) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hi8hK-0006wI-L7; Mon, 01 Jul 2019 22:39:10 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D5E6430872F3; Tue, 2 Jul 2019 02:39:09 +0000 (UTC) Received: from localhost (ovpn-204-243.brq.redhat.com [10.40.204.243]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6B6237E668; Tue, 2 Jul 2019 02:39:09 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 2 Jul 2019 04:39:02 +0200 Message-Id: <20190702023905.32470-2-mreitz@redhat.com> In-Reply-To: <20190702023905.32470-1-mreitz@redhat.com> References: <20190702023905.32470-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.47]); Tue, 02 Jul 2019 02:39:09 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 1/4] block/rbd: increase dynamically the image size X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: Stefano Garzarella RBD APIs don't allow us to write more than the size set with rbd_create() or rbd_resize(). In order to support growing images (eg. qcow2), we resize the image before write operations that exceed the current size. Signed-off-by: Stefano Garzarella Message-id: 20190509145927.293369-1-sgarzare@redhat.com Signed-off-by: Max Reitz --- block/rbd.c | 42 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 39 insertions(+), 3 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index f2ac2c06f4..59757b3120 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -103,6 +103,7 @@ typedef struct BDRVRBDState { rbd_image_t image; char *image_name; char *snap; + uint64_t image_size; } BDRVRBDState; =20 static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx, @@ -778,6 +779,14 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *= options, int flags, goto failed_open; } =20 + r =3D rbd_get_size(s->image, &s->image_size); + if (r < 0) { + error_setg_errno(errp, -r, "error getting image size from %s", + s->image_name); + rbd_close(s->image); + goto failed_open; + } + /* If we are using an rbd snapshot, we must be r/o, otherwise * leave as-is */ if (s->snap !=3D NULL) { @@ -834,6 +843,22 @@ static void qemu_rbd_close(BlockDriverState *bs) rados_shutdown(s->cluster); } =20 +/* Resize the RBD image and update the 'image_size' with the current size = */ +static int qemu_rbd_resize(BlockDriverState *bs, uint64_t size) +{ + BDRVRBDState *s =3D bs->opaque; + int r; + + r =3D rbd_resize(s->image, size); + if (r < 0) { + return r; + } + + s->image_size =3D size; + + return 0; +} + static const AIOCBInfo rbd_aiocb_info =3D { .aiocb_size =3D sizeof(RBDAIOCB), }; @@ -935,13 +960,25 @@ static BlockAIOCB *rbd_start_aio(BlockDriverState *bs, } =20 switch (cmd) { - case RBD_AIO_WRITE: + case RBD_AIO_WRITE: { + /* + * RBD APIs don't allow us to write more than actual size, so in o= rder + * to support growing images, we resize the image before write + * operations that exceed the current size. + */ + if (off + size > s->image_size) { + r =3D qemu_rbd_resize(bs, off + size); + if (r < 0) { + goto failed_completion; + } + } #ifdef LIBRBD_SUPPORTS_IOVEC r =3D rbd_aio_writev(s->image, qiov->iov, qiov->niov, off, c); #else r =3D rbd_aio_write(s->image, off, size, rcb->buf, c); #endif break; + } case RBD_AIO_READ: #ifdef LIBRBD_SUPPORTS_IOVEC r =3D rbd_aio_readv(s->image, qiov->iov, qiov->niov, off, c); @@ -1052,7 +1089,6 @@ static int coroutine_fn qemu_rbd_co_truncate(BlockDri= verState *bs, PreallocMode prealloc, Error **errp) { - BDRVRBDState *s =3D bs->opaque; int r; =20 if (prealloc !=3D PREALLOC_MODE_OFF) { @@ -1061,7 +1097,7 @@ static int coroutine_fn qemu_rbd_co_truncate(BlockDri= verState *bs, return -ENOTSUP; } =20 - r =3D rbd_resize(s->image, offset); + r =3D qemu_rbd_resize(bs, offset); if (r < 0) { error_setg_errno(errp, -r, "Failed to resize file"); return r; --=20 2.21.0 From nobody Mon May 20 00:21:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562040677; cv=none; d=zoho.com; s=zohoarc; b=kwHbDhmoLJEQWtF/txSdtAhGsLqaVP9kDjh5UWWlfk9byuj3etfboY4iYHrGk60phqWBBjuu1MtVDnYRZ/YP3zdi85GBrmEjY4Q9ucaYxcJTFqe2E5EjwDJfkdrceCSmio2AC8JFUZDhj+aHaU5VbYM5DxshTeHwO/kO/EBv5hs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562040677; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=VhvsDz3bTWXcO/PKxFKlJiK/pO8CMhVhnXU1xlERMT0=; b=Mzto8KfHPssy7yYaBSvu6N/+TutXn7HE7p628kfdrKw5IQCoSAnYUmoECWicyyPdWJxmgYd3VXaTA3fpA7UoQmm++nCRyrMTAfVzwXi4iSYbl6Syh8B1Gct+isoUXhrNN9/rZbKmUJ0cz60fI6/6sGPY10eqOFpr6PK7thOPMxA= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562040677489541.0387512513089; Mon, 1 Jul 2019 21:11:17 -0700 (PDT) Received: from localhost ([::1]:48494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hiA8P-0005jt-6i for importer@patchew.org; Tue, 02 Jul 2019 00:11:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51758) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hi8hd-0004Cs-2m for qemu-devel@nongnu.org; Mon, 01 Jul 2019 22:39:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hi8hZ-0007BL-Dx for qemu-devel@nongnu.org; Mon, 01 Jul 2019 22:39:27 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43144) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hi8hN-00070A-W2; Mon, 01 Jul 2019 22:39:14 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4779330C1205; Tue, 2 Jul 2019 02:39:12 +0000 (UTC) Received: from localhost (ovpn-204-243.brq.redhat.com [10.40.204.243]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B70AD5D71C; Tue, 2 Jul 2019 02:39:11 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 2 Jul 2019 04:39:03 +0200 Message-Id: <20190702023905.32470-3-mreitz@redhat.com> In-Reply-To: <20190702023905.32470-1-mreitz@redhat.com> References: <20190702023905.32470-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Tue, 02 Jul 2019 02:39:13 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 2/4] block: include base when checking image chain for block allocation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: Andrey Shinkevich This patch is used in the 'block/stream: introduce a bottom node' that is following. Instead of the base node, the caller may pass the node that has the base as its backing image to the function bdrv_is_allocated_above() with a new parameter include_base =3D true and get rid of the dependency on the base that may change during commit/stream parallel jobs. Now, if the specified base is not found in the backing image chain, the QEMU will abort. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Message-id: 1559152576-281803-2-git-send-email-andrey.shinkevich@virtuozzo.= com [mreitz: Squashed in the following as a rebase on conflicting patches:] Message-id: e3cf99ae-62e9-8b6e-5a06-d3c8b9363b85@redhat.com Signed-off-by: Max Reitz --- include/block/block.h | 3 ++- block/commit.c | 2 +- block/io.c | 21 +++++++++++++++------ block/mirror.c | 2 +- block/qcow2.c | 3 ++- block/replication.c | 2 +- block/stream.c | 2 +- qemu-img.c | 2 +- 8 files changed, 24 insertions(+), 13 deletions(-) diff --git a/include/block/block.h b/include/block/block.h index f9415ed740..734c9d2f76 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -449,7 +449,8 @@ int bdrv_block_status_above(BlockDriverState *bs, Block= DriverState *base, int bdrv_is_allocated(BlockDriverState *bs, int64_t offset, int64_t bytes, int64_t *pnum); int bdrv_is_allocated_above(BlockDriverState *top, BlockDriverState *base, - int64_t offset, int64_t bytes, int64_t *pnum); + bool include_base, int64_t offset, int64_t byt= es, + int64_t *pnum); =20 bool bdrv_is_read_only(BlockDriverState *bs); int bdrv_can_set_read_only(BlockDriverState *bs, bool read_only, diff --git a/block/commit.c b/block/commit.c index 212c6f639e..ca7e408b26 100644 --- a/block/commit.c +++ b/block/commit.c @@ -174,7 +174,7 @@ static int coroutine_fn commit_run(Job *job, Error **er= rp) break; } /* Copy if allocated above the base */ - ret =3D bdrv_is_allocated_above(blk_bs(s->top), blk_bs(s->base), + ret =3D bdrv_is_allocated_above(blk_bs(s->top), blk_bs(s->base), f= alse, offset, COMMIT_BUFFER_SIZE, &n); copy =3D (ret =3D=3D 1); trace_commit_one_iteration(s, offset, n, ret); diff --git a/block/io.c b/block/io.c index 9ba1bada36..24a18759fd 100644 --- a/block/io.c +++ b/block/io.c @@ -2295,10 +2295,11 @@ int coroutine_fn bdrv_is_allocated(BlockDriverState= *bs, int64_t offset, /* * Given an image chain: ... -> [BASE] -> [INTER1] -> [INTER2] -> [TOP] * - * Return true if (a prefix of) the given range is allocated in any image - * between BASE and TOP (inclusive). BASE can be NULL to check if the giv= en - * offset is allocated in any image of the chain. Return false otherwise, - * or negative errno on failure. + * Return 1 if (a prefix of) the given range is allocated in any image + * between BASE and TOP (BASE is only included if include_base is set). + * BASE can be NULL to check if the given offset is allocated in any + * image of the chain. Return 0 otherwise, or negative errno on + * failure. * * 'pnum' is set to the number of bytes (including and immediately * following the specified offset) that are known to be in the same @@ -2310,17 +2311,21 @@ int coroutine_fn bdrv_is_allocated(BlockDriverState= *bs, int64_t offset, */ int bdrv_is_allocated_above(BlockDriverState *top, BlockDriverState *base, - int64_t offset, int64_t bytes, int64_t *pnum) + bool include_base, int64_t offset, + int64_t bytes, int64_t *pnum) { BlockDriverState *intermediate; int ret; int64_t n =3D bytes; =20 + assert(base || !include_base); + intermediate =3D top; - while (intermediate && intermediate !=3D base) { + while (include_base || intermediate !=3D base) { int64_t pnum_inter; int64_t size_inter; =20 + assert(intermediate); ret =3D bdrv_is_allocated(intermediate, offset, bytes, &pnum_inter= ); if (ret < 0) { return ret; @@ -2339,6 +2344,10 @@ int bdrv_is_allocated_above(BlockDriverState *top, n =3D pnum_inter; } =20 + if (intermediate =3D=3D base) { + break; + } + intermediate =3D backing_bs(intermediate); } =20 diff --git a/block/mirror.c b/block/mirror.c index d17be4cdbc..2fcec70e35 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -808,7 +808,7 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJo= b *s) return 0; } =20 - ret =3D bdrv_is_allocated_above(bs, base, offset, bytes, &count); + ret =3D bdrv_is_allocated_above(bs, base, false, offset, bytes, &c= ount); if (ret < 0) { return ret; } diff --git a/block/qcow2.c b/block/qcow2.c index 9396d490d5..2a59eb27fe 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2148,7 +2148,8 @@ static bool is_unallocated(BlockDriverState *bs, int6= 4_t offset, int64_t bytes) { int64_t nr; return !bytes || - (!bdrv_is_allocated_above(bs, NULL, offset, bytes, &nr) && nr =3D= =3D bytes); + (!bdrv_is_allocated_above(bs, NULL, false, offset, bytes, &nr) && + nr =3D=3D bytes); } =20 static bool is_zero_cow(BlockDriverState *bs, QCowL2Meta *m) diff --git a/block/replication.c b/block/replication.c index b41bc507c0..23b2993d74 100644 --- a/block/replication.c +++ b/block/replication.c @@ -275,7 +275,7 @@ static coroutine_fn int replication_co_writev(BlockDriv= erState *bs, while (remaining_sectors > 0) { int64_t count; =20 - ret =3D bdrv_is_allocated_above(top->bs, base->bs, + ret =3D bdrv_is_allocated_above(top->bs, base->bs, false, sector_num * BDRV_SECTOR_SIZE, remaining_sectors * BDRV_SECTOR_SIZE, &count); diff --git a/block/stream.c b/block/stream.c index 1a906fd860..97fddb2608 100644 --- a/block/stream.c +++ b/block/stream.c @@ -160,7 +160,7 @@ static int coroutine_fn stream_run(Job *job, Error **er= rp) } else if (ret >=3D 0) { /* Copy if allocated in the intermediate images. Limit to the * known-unallocated area [offset, offset+n*BDRV_SECTOR_SIZE).= */ - ret =3D bdrv_is_allocated_above(backing_bs(bs), base, + ret =3D bdrv_is_allocated_above(backing_bs(bs), base, false, offset, n, &n); =20 /* Finish early if end of backing file has been reached */ diff --git a/qemu-img.c b/qemu-img.c index 158b3a505f..79983772de 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -3518,7 +3518,7 @@ static int img_rebase(int argc, char **argv) * to take action */ ret =3D bdrv_is_allocated_above(backing_bs(bs), prefix_cha= in_bs, - offset, n, &n); + false, offset, n, &n); if (ret < 0) { error_report("error while reading image metadata: %s", strerror(-ret)); --=20 2.21.0 From nobody Mon May 20 00:21:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562040141; cv=none; d=zoho.com; s=zohoarc; b=hmg7WGvYp0ySS9xvwAErwzaJTLkAev1LOO2Sq9JElXcsEfHMYufqsyZXDeqzucZlicLc/jFm0J0fhXtQtqpKTqDvxb7EJ98Yvz58peIucvvaod+uhXkjQDwYkvvSzta4uJInv5nzrhHNGohMVgiknKf48nXwAJlZrjLSSoWyIpU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562040141; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=1UAFk136ouuJNVr8wtsp64ILrXw1m6GAnJY6V/EykfQ=; b=YemPKLGyQVcVCsXUIN6PD5ShNbtEBmoa113VnnR10b+rm94uXJ5LUb5n6SbpocgkU70PKsKXR6qOrH9VuRzXEvogc3LnQ4/eKwkEZGkkrwrhHCDIP9Ns0DLwMZz4ePWeSNIlHvkG/BhnC21lmXzabncX4iA2VXSCm7oZTT1uQgM= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 156204014139081.15018619128716; Mon, 1 Jul 2019 21:02:21 -0700 (PDT) Received: from localhost ([::1]:48388 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hi9zo-0003vY-4H for importer@patchew.org; Tue, 02 Jul 2019 00:02:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51991) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hi8hn-0004KX-JN for qemu-devel@nongnu.org; Mon, 01 Jul 2019 22:39:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hi8hl-0007Qf-Kp for qemu-devel@nongnu.org; Mon, 01 Jul 2019 22:39:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46830) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hi8hb-00072F-2I; Mon, 01 Jul 2019 22:39:29 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 326ABC057F2E; Tue, 2 Jul 2019 02:39:15 +0000 (UTC) Received: from localhost (ovpn-204-243.brq.redhat.com [10.40.204.243]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C3FEC60C44; Tue, 2 Jul 2019 02:39:14 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 2 Jul 2019 04:39:04 +0200 Message-Id: <20190702023905.32470-4-mreitz@redhat.com> In-Reply-To: <20190702023905.32470-1-mreitz@redhat.com> References: <20190702023905.32470-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 02 Jul 2019 02:39:15 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 3/4] block/stream: refactor stream_run: drop goto X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: Andrey Shinkevich The goto is unnecessary in the stream_run() since the common exit code was removed in the commit eb23654dbe43b549ea2a9ebff9d8e: "jobs: utilize job_exit shim". Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz Message-id: 1559152576-281803-3-git-send-email-andrey.shinkevich@virtuozzo.= com Reviewed-by: Max Reitz Signed-off-by: Max Reitz --- block/stream.c | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/block/stream.c b/block/stream.c index 97fddb2608..65b13b27e0 100644 --- a/block/stream.c +++ b/block/stream.c @@ -120,13 +120,12 @@ static int coroutine_fn stream_run(Job *job, Error **= errp) void *buf; =20 if (!bs->backing) { - goto out; + return 0; } =20 len =3D bdrv_getlength(bs); if (len < 0) { - ret =3D len; - goto out; + return len; } job_progress_set_remaining(&s->common.job, len); =20 @@ -203,14 +202,10 @@ static int coroutine_fn stream_run(Job *job, Error **= errp) bdrv_disable_copy_on_read(bs); } =20 - /* Do not remove the backing file if an error was there but ignored. = */ - ret =3D error; - qemu_vfree(buf); =20 -out: - /* Modify backing chain and close BDSes in main loop */ - return ret; + /* Do not remove the backing file if an error was there but ignored. */ + return error; } =20 static const BlockJobDriver stream_job_driver =3D { --=20 2.21.0 From nobody Mon May 20 00:21:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1562041603; cv=none; d=zoho.com; s=zohoarc; b=J2wKMATY657ObM/s4A2oLQs2a2SMO6uwikiCjFAcnYH95t31gNasJrVMj72eIPVl/CkD//+qVnbJ6VV3MjDMrM4t93A6VO2Msgo+wOay8H0q2ziopOqchqG30gO/z0YSDv+UXkPWX0n2cVe8csxUm0zaTmJHpESW5yU2RPfki7k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562041603; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=vV8u1/cyuBUfWAj7Y1U0GJbHrMW4ffvav94sLZmaj2o=; b=UT1okTd8aiP3GdclNlXqiW3wxhUfvvvQTqCuBxjrMWZbZv/hi8jGVyUFR6quGFxFxQ6lUm6BkD741g4Gjd8isdUUgVq2oArzgvUSu7ques0HcFHohvkedpC5jkBl+ZZk+aT8lsgFBiHtrGr1dwEm9ECl3g/JXHO4BC0m/rpxO3U= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1562041603110571.7173116366727; Mon, 1 Jul 2019 21:26:43 -0700 (PDT) Received: from localhost ([::1]:48710 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hiANN-0004RX-Uq for importer@patchew.org; Tue, 02 Jul 2019 00:26:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51941) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hi8hl-0004HG-C8 for qemu-devel@nongnu.org; Mon, 01 Jul 2019 22:39:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hi8hi-0007MZ-Si for qemu-devel@nongnu.org; Mon, 01 Jul 2019 22:39:37 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60164) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hi8hV-00074T-CS; Mon, 01 Jul 2019 22:39:24 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A650B307D914; Tue, 2 Jul 2019 02:39:17 +0000 (UTC) Received: from localhost (ovpn-204-243.brq.redhat.com [10.40.204.243]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 13AB218386; Tue, 2 Jul 2019 02:39:16 +0000 (UTC) From: Max Reitz To: qemu-block@nongnu.org Date: Tue, 2 Jul 2019 04:39:05 +0200 Message-Id: <20190702023905.32470-5-mreitz@redhat.com> In-Reply-To: <20190702023905.32470-1-mreitz@redhat.com> References: <20190702023905.32470-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Tue, 02 Jul 2019 02:39:17 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 4/4] block/stream: introduce a bottom node X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , qemu-devel@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" From: Andrey Shinkevich The bottom node is the intermediate block device that has the base as its backing image. It is used instead of the base node while a block stream job is running to avoid dependency on the base that may change due to the parallel jobs. The change may take place due to a filter node as well that is inserted between the base and the intermediate bottom node. It occurs when the base node is the top one for another commit or stream job. After the introduction of the bottom node, don't freeze its backing child, that's the base, anymore. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Message-id: 1559152576-281803-4-git-send-email-andrey.shinkevich@virtuozzo.= com Reviewed-by: Max Reitz Signed-off-by: Max Reitz --- block/stream.c | 43 ++++++++++++++++++++++-------------------- tests/qemu-iotests/245 | 4 ++-- 2 files changed, 25 insertions(+), 22 deletions(-) diff --git a/block/stream.c b/block/stream.c index 65b13b27e0..cd5e2ba9b0 100644 --- a/block/stream.c +++ b/block/stream.c @@ -31,7 +31,7 @@ enum { =20 typedef struct StreamBlockJob { BlockJob common; - BlockDriverState *base; + BlockDriverState *bottom; BlockdevOnError on_error; char *backing_file_str; bool bs_read_only; @@ -54,7 +54,7 @@ static void stream_abort(Job *job) =20 if (s->chain_frozen) { BlockJob *bjob =3D &s->common; - bdrv_unfreeze_backing_chain(blk_bs(bjob->blk), s->base); + bdrv_unfreeze_backing_chain(blk_bs(bjob->blk), s->bottom); } } =20 @@ -63,11 +63,11 @@ static int stream_prepare(Job *job) StreamBlockJob *s =3D container_of(job, StreamBlockJob, common.job); BlockJob *bjob =3D &s->common; BlockDriverState *bs =3D blk_bs(bjob->blk); - BlockDriverState *base =3D s->base; + BlockDriverState *base =3D backing_bs(s->bottom); Error *local_err =3D NULL; int ret =3D 0; =20 - bdrv_unfreeze_backing_chain(bs, base); + bdrv_unfreeze_backing_chain(bs, s->bottom); s->chain_frozen =3D false; =20 if (bs->backing) { @@ -110,7 +110,7 @@ static int coroutine_fn stream_run(Job *job, Error **er= rp) StreamBlockJob *s =3D container_of(job, StreamBlockJob, common.job); BlockBackend *blk =3D s->common.blk; BlockDriverState *bs =3D blk_bs(blk); - BlockDriverState *base =3D s->base; + bool enable_cor =3D !backing_bs(s->bottom); int64_t len; int64_t offset =3D 0; uint64_t delay_ns =3D 0; @@ -119,7 +119,8 @@ static int coroutine_fn stream_run(Job *job, Error **er= rp) int64_t n =3D 0; /* bytes */ void *buf; =20 - if (!bs->backing) { + if (bs =3D=3D s->bottom) { + /* Nothing to stream */ return 0; } =20 @@ -136,7 +137,7 @@ static int coroutine_fn stream_run(Job *job, Error **er= rp) * backing chain since the copy-on-read operation does not take base i= nto * account. */ - if (!base) { + if (enable_cor) { bdrv_enable_copy_on_read(bs); } =20 @@ -159,9 +160,8 @@ static int coroutine_fn stream_run(Job *job, Error **er= rp) } else if (ret >=3D 0) { /* Copy if allocated in the intermediate images. Limit to the * known-unallocated area [offset, offset+n*BDRV_SECTOR_SIZE).= */ - ret =3D bdrv_is_allocated_above(backing_bs(bs), base, false, + ret =3D bdrv_is_allocated_above(backing_bs(bs), s->bottom, tru= e, offset, n, &n); - /* Finish early if end of backing file has been reached */ if (ret =3D=3D 0 && n =3D=3D 0) { n =3D len - offset; @@ -198,7 +198,7 @@ static int coroutine_fn stream_run(Job *job, Error **er= rp) } } =20 - if (!base) { + if (enable_cor) { bdrv_disable_copy_on_read(bs); } =20 @@ -230,8 +230,10 @@ void stream_start(const char *job_id, BlockDriverState= *bs, StreamBlockJob *s; BlockDriverState *iter; bool bs_read_only; + int basic_flags =3D BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGE= D; + BlockDriverState *bottom =3D bdrv_find_overlay(bs, base); =20 - if (bdrv_freeze_backing_chain(bs, base, errp) < 0) { + if (bdrv_freeze_backing_chain(bs, bottom, errp) < 0) { return; } =20 @@ -248,10 +250,8 @@ void stream_start(const char *job_id, BlockDriverState= *bs, * already have our own plans. Also don't allow resize as the image si= ze is * queried only at the job start and then cached. */ s =3D block_job_create(job_id, &stream_job_driver, NULL, bs, - BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANG= ED | - BLK_PERM_GRAPH_MOD, - BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANG= ED | - BLK_PERM_WRITE, + basic_flags | BLK_PERM_GRAPH_MOD, + basic_flags | BLK_PERM_WRITE, speed, creation_flags, NULL, NULL, errp); if (!s) { goto fail; @@ -259,15 +259,18 @@ void stream_start(const char *job_id, BlockDriverStat= e *bs, =20 /* Block all intermediate nodes between bs and base, because they will * disappear from the chain after this operation. The streaming job re= ads - * every block only once, assuming that it doesn't change, so block wr= ites - * and resizes. */ + * every block only once, assuming that it doesn't change, so forbid w= rites + * and resizes. Reassign the base node pointer because the backing BS = of the + * bottom node might change after the call to bdrv_reopen_set_read_onl= y() + * due to parallel block jobs running. + */ + base =3D backing_bs(bottom); for (iter =3D backing_bs(bs); iter && iter !=3D base; iter =3D backing= _bs(iter)) { block_job_add_bdrv(&s->common, "intermediate node", iter, 0, - BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHA= NGED, - &error_abort); + basic_flags, &error_abort); } =20 - s->base =3D base; + s->bottom =3D bottom; s->backing_file_str =3D g_strdup(backing_file_str); s->bs_read_only =3D bs_read_only; s->chain_frozen =3D true; diff --git a/tests/qemu-iotests/245 b/tests/qemu-iotests/245 index 349b94aace..bc1ceb9792 100644 --- a/tests/qemu-iotests/245 +++ b/tests/qemu-iotests/245 @@ -866,9 +866,9 @@ class TestBlockdevReopen(iotests.QMPTestCase): auto_finalize =3D False) self.assert_qmp(result, 'return', {}) =20 - # We can't remove hd2 while the stream job is ongoing + # We can remove hd2 while the stream job is ongoing opts['backing']['backing'] =3D None - self.reopen(opts, {}, "Cannot change 'backing' link from 'hd1' to = 'hd2'") + self.reopen(opts, {}) =20 # We can't remove hd1 while the stream job is ongoing opts['backing'] =3D None --=20 2.21.0