From nobody Sun Feb 8 20:28:24 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1530286390348850.505855652305; Fri, 29 Jun 2018 08:33:10 -0700 (PDT) Received: from localhost ([::1]:42981 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYvOX-0006v4-Hk for importer@patchew.org; Fri, 29 Jun 2018 11:33:09 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40082) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYv7U-0001G3-Q1 for qemu-devel@nongnu.org; Fri, 29 Jun 2018 11:15:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYv7R-0006Ay-IO for qemu-devel@nongnu.org; Fri, 29 Jun 2018 11:15:32 -0400 Received: from relay.sw.ru ([185.231.240.75]:44520) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fYv7R-00063B-9M; Fri, 29 Jun 2018 11:15:29 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1fYv7N-0003YF-6r; Fri, 29 Jun 2018 18:15:25 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Fri, 29 Jun 2018 18:15:22 +0300 Message-Id: <20180629151524.138542-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20180629151524.138542-1-vsementsov@virtuozzo.com> References: <20180629151524.138542-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v2 1/3] blockdev-backup: enable non-root nodes for backup source X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This is needed to implement image-fleecing scheme, when we create a temporary node, mark our active node to be backing for the temp, and start backup(sync=3Dnone) from active node to the temp node. Temp node then represents a kind of snapshot and may be used for external backup through NBD. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: John Snow --- blockdev.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/blockdev.c b/blockdev.c index 58d7570932..72f5347df5 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1859,7 +1859,7 @@ static void blockdev_backup_prepare(BlkActionState *c= ommon, Error **errp) assert(common->action->type =3D=3D TRANSACTION_ACTION_KIND_BLOCKDEV_BA= CKUP); backup =3D common->action->u.blockdev_backup.data; =20 - bs =3D qmp_get_root_bs(backup->device, errp); + bs =3D bdrv_lookup_bs(backup->device, backup->device, errp); if (!bs) { return; } @@ -3517,7 +3517,7 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, = JobTxn *txn, backup->compress =3D false; } =20 - bs =3D qmp_get_root_bs(backup->device, errp); + bs =3D bdrv_lookup_bs(backup->device, backup->device, errp); if (!bs) { return NULL; } --=20 2.11.1 From nobody Sun Feb 8 20:28:24 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1530288023244943.2705311099048; Fri, 29 Jun 2018 09:00:23 -0700 (PDT) Received: from localhost ([::1]:43140 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYvos-0002iU-EE for importer@patchew.org; Fri, 29 Jun 2018 12:00:22 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40083) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYv7U-0001G4-QB for qemu-devel@nongnu.org; Fri, 29 Jun 2018 11:15:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYv7R-0006Ac-Ep for qemu-devel@nongnu.org; Fri, 29 Jun 2018 11:15:32 -0400 Received: from relay.sw.ru ([185.231.240.75]:44518) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fYv7Q-000639-Uq; Fri, 29 Jun 2018 11:15:29 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1fYv7N-0003YF-Ba; Fri, 29 Jun 2018 18:15:25 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Fri, 29 Jun 2018 18:15:23 +0300 Message-Id: <20180629151524.138542-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20180629151524.138542-1-vsementsov@virtuozzo.com> References: <20180629151524.138542-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v2 2/3] block/fleecing-filter: new filter driver for fleecing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We need to synchronize backup job with reading from fleecing image like it was done in block/replication.c. Otherwise, the following situation is theoretically possible: 1. client start reading 2. client understand, that there is no corresponding cluster in fleecing image 3. client is going to read from backing file (i.e. active image) 4. guest writes to active image 5. this write is stopped by backup(sync=3Dnone) and cluster is copied to fleecing image 6. guest write continues... 7. and client reads _new_ (or partly new) date from active image So, this fleecing-filter should be above fleecing image, the whole picture of fleecing looks like this: +-------+ +------------+ | | | | | guest | | NBD client +<------+ | | | | | ++-----++ +------------+ |only read | ^ | | IO | | v | +-----+------+ ++-----+---------+ | | | | | internal | | active image +----+ | NBD server | | | | | | +-+--------------+ |backup +-+----------+ ^ |sync=3Dnone ^ |backing | |only read | | | +-+--------------+ | +------+----------+ | | | | | | fleecing image +<---+ | fleecing filter | | | | | +--------+-------+ +-----+-----------+ ^ | | | +--------------------------+ file Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 6 ++-- block/fleecing-filter.c | 80 +++++++++++++++++++++++++++++++++++++++++++++= ++++ block/Makefile.objs | 1 + 3 files changed, 85 insertions(+), 2 deletions(-) create mode 100644 block/fleecing-filter.c diff --git a/qapi/block-core.json b/qapi/block-core.json index 577ce5e999..43872c3d79 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2542,7 +2542,8 @@ 'host_device', 'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels', 'qcow', 'qcow2', '= qed', 'quorum', 'raw', 'rbd', 'replication', 'sheepdog', 'ssh', - 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat', 'vxhs' ] } + 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat', 'vxhs', + 'fleecing-filter' ] } =20 ## # @BlockdevOptionsFile: @@ -3594,7 +3595,8 @@ 'vmdk': 'BlockdevOptionsGenericCOWFormat', 'vpc': 'BlockdevOptionsGenericFormat', 'vvfat': 'BlockdevOptionsVVFAT', - 'vxhs': 'BlockdevOptionsVxHS' + 'vxhs': 'BlockdevOptionsVxHS', + 'fleecing-filter': 'BlockdevOptionsGenericFormat' } } =20 ## diff --git a/block/fleecing-filter.c b/block/fleecing-filter.c new file mode 100644 index 0000000000..b501887c10 --- /dev/null +++ b/block/fleecing-filter.c @@ -0,0 +1,80 @@ +#include "qemu/osdep.h" +#include "qemu-common.h" +#include "block/blockjob.h" +#include "block/block_int.h" +#include "block/block_backup.h" + +static int64_t fleecing_getlength(BlockDriverState *bs) +{ + return bdrv_getlength(bs->file->bs); +} + +static coroutine_fn int fleecing_co_preadv(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags) +{ + int ret; + BlockJob *job =3D bs->file->bs->backing->bs->job; + CowRequest req; + + backup_wait_for_overlapping_requests(job, offset, bytes); + backup_cow_request_begin(&req, job, offset, bytes); + + ret =3D bdrv_co_preadv(bs->file, offset, bytes, qiov, flags); + + backup_cow_request_end(&req); + + return ret; +} + +static coroutine_fn int fleecing_co_pwritev(BlockDriverState *bs, + uint64_t offset, uint64_t byte= s, + QEMUIOVector *qiov, int flags) +{ + return -EINVAL; +} + +static bool fleecing_recurse_is_first_non_filter(BlockDriverState *bs, + BlockDriverState *candida= te) +{ + return bdrv_recurse_is_first_non_filter(bs->file->bs, candidate); +} + +static int fleecing_open(BlockDriverState *bs, QDict *options, + int flags, Error **errp) +{ + bs->file =3D bdrv_open_child(NULL, options, "file", bs, &child_file, f= alse, + errp); + + return bs->file ? 0 : -EINVAL; +} + +static void fleecing_close(BlockDriverState *bs) +{ + /* Do nothing, we have to add .bdrv_close, because bdrv_close() don't = check + * it, just call. */ +} + +BlockDriver bdrv_fleecing_filter =3D { + .format_name =3D "fleecing-filter", + .protocol_name =3D "fleecing-filter", + .instance_size =3D 0, + + .bdrv_open =3D fleecing_open, + .bdrv_close =3D fleecing_close, + + .bdrv_getlength =3D fleecing_getlength, + .bdrv_co_preadv =3D fleecing_co_preadv, + .bdrv_co_pwritev =3D fleecing_co_pwritev, + + .is_filter =3D true, + .bdrv_recurse_is_first_non_filter =3D fleecing_recurse_is_first_non_fi= lter, + .bdrv_child_perm =3D bdrv_filter_default_perms, +}; + +static void bdrv_fleecing_init(void) +{ + bdrv_register(&bdrv_fleecing_filter); +} + +block_init(bdrv_fleecing_init); diff --git a/block/Makefile.objs b/block/Makefile.objs index 899bfb5e2c..aa0a6dd971 100644 --- a/block/Makefile.objs +++ b/block/Makefile.objs @@ -27,6 +27,7 @@ block-obj-y +=3D write-threshold.o block-obj-y +=3D backup.o block-obj-$(CONFIG_REPLICATION) +=3D replication.o block-obj-y +=3D throttle.o copy-on-read.o +block-obj-y +=3D fleecing-filter.o =20 block-obj-y +=3D crypto.o =20 --=20 2.11.1 From nobody Sun Feb 8 20:28:24 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 153028807203924.707284423571764; Fri, 29 Jun 2018 09:01:12 -0700 (PDT) Received: from localhost ([::1]:43163 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYvpf-0003Lo-6j for importer@patchew.org; Fri, 29 Jun 2018 12:01:11 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40088) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYv7U-0001G6-RD for qemu-devel@nongnu.org; Fri, 29 Jun 2018 11:15:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYv7R-0006AR-E9 for qemu-devel@nongnu.org; Fri, 29 Jun 2018 11:15:32 -0400 Received: from relay.sw.ru ([185.231.240.75]:44516) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fYv7Q-000638-Un; Fri, 29 Jun 2018 11:15:29 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1fYv7N-0003YF-DX; Fri, 29 Jun 2018 18:15:25 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Fri, 29 Jun 2018 18:15:24 +0300 Message-Id: <20180629151524.138542-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20180629151524.138542-1-vsementsov@virtuozzo.com> References: <20180629151524.138542-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v2 3/3] qemu-iotests: Image fleecing test case 222 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, famz@redhat.com, armbru@redhat.com, mreitz@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Fam Zheng This tests the workflow of creating a lightweight point-in-time snapshot with blockdev-backup command, and exporting it with built-in NBD server. It's tested that any post-snapshot writing to the original device doesn't change data seen in NBD target. Signed-off-by: Fam Zheng [vsementsov: add -f iotests.imgfmt to qemu_io, fix target_img to be always qcow2, add -r to qemu_io for nbd, add fleecing-filter layer, wrap long lines] Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/222 | 112 +++++++++++++++++++++++++++++++++++++++++= ++++ tests/qemu-iotests/222.out | 5 ++ tests/qemu-iotests/group | 1 + 3 files changed, 118 insertions(+) create mode 100755 tests/qemu-iotests/222 create mode 100644 tests/qemu-iotests/222.out diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222 new file mode 100755 index 0000000000..3bcf9505fd --- /dev/null +++ b/tests/qemu-iotests/222 @@ -0,0 +1,112 @@ +#!/usr/bin/env python +# +# Tests for image fleecing (point in time snapshot export to NBD) +# +# Copyright (C) 2014 Red Hat, Inc. +# +# Based on 055. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import time +import os +import iotests +from iotests import qemu_img, qemu_io + +test_img =3D os.path.join(iotests.test_dir, 'test.img') +target_img =3D os.path.join(iotests.test_dir, 'target.img') +nbd_sock =3D os.path.join(iotests.test_dir, 'nbd.sock') + +class TestImageFleecing(iotests.QMPTestCase): + image_len =3D 64 * 1024 * 1024 # MB + + def setUp(self): + # Write data to the image so we can compare later + qemu_img('create', '-f', iotests.imgfmt, test_img, + str(TestImageFleecing.image_len)) + self.patterns =3D [ + ("0x5d", "0", "64k"), + ("0xd5", "1M", "64k"), + ("0xdc", "32M", "64k"), + ("0xdc", "67043328", "64k")] + + for p in self.patterns: + qemu_io('-f', iotests.imgfmt, '-c', 'write -P%s %s %s' % p, + test_img) + + qemu_img('create', '-f', 'qcow2', target_img, + str(TestImageFleecing.image_len)) + + self.vm =3D iotests.VM().add_drive(test_img) + self.vm.launch() + + self.overwrite_patterns =3D [ + ("0xa0", "0", "64k"), + ("0x0a", "1M", "64k"), + ("0x55", "32M", "64k"), + ("0x56", "67043328", "64k")] + + self.nbd_uri =3D "nbd+unix:///drive1?socket=3D%s" % nbd_sock + + def tearDown(self): + self.vm.shutdown() + os.remove(test_img) + os.remove(target_img) + + def verify_patterns(self): + for p in self.patterns: + self.assertEqual( + -1, + qemu_io(self.nbd_uri, '-r', '-c', 'read -P%s %s %s' % = p) + .find("verification failed"), + "Failed to verify pattern: %s %s %s" % p) + + def test_image_fleecing(self): + result =3D self.vm.qmp("blockdev-add", **{ + "driver": "fleecing-filter", + "node-name": "drive1", + "file": { + "driver": "qcow2", + "file": { + "driver": "file", + "filename": target_img, + }, + "backing": "drive0", + } + }) + self.assert_qmp(result, 'return', {}) + + result =3D self.vm.qmp( + "nbd-server-start", + **{"addr": { "type": "unix", "data": { "path": nbd_sock } = } }) + self.assert_qmp(result, 'return', {}) + result =3D self.vm.qmp("blockdev-backup", device=3D"drive0", + target=3D"drive1", sync=3D"none") + self.assert_qmp(result, 'return', {}) + result =3D self.vm.qmp("nbd-server-add", device=3D"drive1") + self.assert_qmp(result, 'return', {}) + + self.verify_patterns() + + for p in self.overwrite_patterns: + self.vm.hmp_qemu_io("drive0", "write -P%s %s %s" % p) + + self.verify_patterns() + + self.cancel_and_wait(resume=3DTrue) + self.assert_no_active_block_jobs() + +if __name__ =3D=3D '__main__': + iotests.main(supported_fmts=3D['raw', 'qcow2']) diff --git a/tests/qemu-iotests/222.out b/tests/qemu-iotests/222.out new file mode 100644 index 0000000000..ae1213e6f8 --- /dev/null +++ b/tests/qemu-iotests/222.out @@ -0,0 +1,5 @@ +. +---------------------------------------------------------------------- +Ran 1 tests + +OK diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index eea75819d2..8019a9f721 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -220,3 +220,4 @@ 218 rw auto quick 219 rw auto 221 rw auto quick +222 rw auto quick --=20 2.11.1