From nobody Sat May 18 15:08:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1598633678; cv=none; d=zohomail.com; s=zohoarc; b=JS1RRJ1jWes5BUL1UHKkxODvVNtZm0LnqxiOnfgMaciB9XxCJ3XyfZbJp1DuTVgJfYpFDq8Ox14DmrEeNmlNfjt/kdwrpcMkZ3hPpv9kx0E528LRCLqZqObCLi/AryaBegFJhY2jblldMJO4OnZPUqXiBr7AB86xCmVcpwV7hJ0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598633678; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=4cyjY8bhDU3J/efikPmoxPXCE6OXQBbAEVHxw9HqXA8=; b=TyC6CQhA2qW5wd7R9Xysrqp2icF/kWpV7AGp+Qu2k167f3xWgZ+bjd31S5N4/vxkBTIE4/gZwSnJvXM+lI39ySyWq03eyMo/qmSYKxZHZ6FJvKEG0gbxkREdJn7Ebi1uisirBpY5WJw0zWA0oyIr4D61XvCG7dFsr+TJZL7SIIA= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598633678114122.96972540971524; Fri, 28 Aug 2020 09:54:38 -0700 (PDT) Received: from localhost ([::1]:37092 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kBhe9-0002gq-DY for importer@patchew.org; Fri, 28 Aug 2020 12:54:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33478) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kBhdQ-0001dq-CS; Fri, 28 Aug 2020 12:53:52 -0400 Received: from relay.sw.ru ([185.231.240.75]:37640 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kBhdO-0001IM-EL; Fri, 28 Aug 2020 12:53:51 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.94) (envelope-from ) id 1kBhcu-001XPR-7L; Fri, 28 Aug 2020 19:53:20 +0300 To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, kwolf@redhat.com, mreitz@redhat.com, armbru@redhat.com, jsnow@redhat.com, eblake@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, andrey.shinkevich@virtuozzo.com Subject: [PATCH v8 1/7] copy-on-read: Support preadv/pwritev_part functions Date: Fri, 28 Aug 2020 19:52:53 +0300 Message-Id: <1598633579-221780-2-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1598633579-221780-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1598633579-221780-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/28 12:53:26 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Reply-to: Andrey Shinkevich From: Andrey Shinkevich via Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add support for the recently introduced functions bdrv_co_preadv_part() and bdrv_co_pwritev_part() to the COR-filter driver. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/block/copy-on-read.c b/block/copy-on-read.c index 2816e61..cb03e0f 100644 --- a/block/copy-on-read.c +++ b/block/copy-on-read.c @@ -74,21 +74,25 @@ static int64_t cor_getlength(BlockDriverState *bs) } =20 =20 -static int coroutine_fn cor_co_preadv(BlockDriverState *bs, - uint64_t offset, uint64_t bytes, - QEMUIOVector *qiov, int flags) +static int coroutine_fn cor_co_preadv_part(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, + size_t qiov_offset, + int flags) { - return bdrv_co_preadv(bs->file, offset, bytes, qiov, - flags | BDRV_REQ_COPY_ON_READ); + return bdrv_co_preadv_part(bs->file, offset, bytes, qiov, qiov_offset, + flags | BDRV_REQ_COPY_ON_READ); } =20 =20 -static int coroutine_fn cor_co_pwritev(BlockDriverState *bs, - uint64_t offset, uint64_t bytes, - QEMUIOVector *qiov, int flags) +static int coroutine_fn cor_co_pwritev_part(BlockDriverState *bs, + uint64_t offset, + uint64_t bytes, + QEMUIOVector *qiov, + size_t qiov_offset, int flags) { - - return bdrv_co_pwritev(bs->file, offset, bytes, qiov, flags); + return bdrv_co_pwritev_part(bs->file, offset, bytes, qiov, qiov_offset, + flags); } =20 =20 @@ -137,8 +141,8 @@ static BlockDriver bdrv_copy_on_read =3D { =20 .bdrv_getlength =3D cor_getlength, =20 - .bdrv_co_preadv =3D cor_co_preadv, - .bdrv_co_pwritev =3D cor_co_pwritev, + .bdrv_co_preadv_part =3D cor_co_preadv_part, + .bdrv_co_pwritev_part =3D cor_co_pwritev_part, .bdrv_co_pwrite_zeroes =3D cor_co_pwrite_zeroes, .bdrv_co_pdiscard =3D cor_co_pdiscard, .bdrv_co_pwritev_compressed =3D cor_co_pwritev_compressed, --=20 1.8.3.1 From nobody Sat May 18 15:08:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1598633731; cv=none; d=zohomail.com; s=zohoarc; b=GI+qpW0KgQrTbDhzbHjyjUNFNXHKuCA8VPVEdLQ5S8Lyznfl3d4wepXRiwsQIAf8EZcRl+/4uvU5IxEMsVB+VVZ/Qd1G8cvx837FTsLkNjeb3TqSLXzx1k1RupLj6mlx6/wFezvtgm6htB/givRvAkghLc/YAA5JXzd9K3QfDoc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598633731; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=zXdcpkuc03svA1fy9Nm3UW/jxwokr/03RVeYDfhvyTE=; b=Kk7G1I9Flb4/5FOU+uKeCXOu4m4F07dAXHOaIihiNjZifJyLZuQ5OIwwmEFk1ABCsOnQrIClJhBGZClD/IbDxNqaJZ/1imNkSfO57i1Owea2SK8PVdPNW13AnzqfqU+ytJ0rmOmbYnxOV/J9lchvuH5hvHX8MVHnjio2iYOjoXI= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598633731984963.2374935533743; Fri, 28 Aug 2020 09:55:31 -0700 (PDT) Received: from localhost ([::1]:41302 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kBhf0-0004az-Tt for importer@patchew.org; Fri, 28 Aug 2020 12:55:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33534) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kBhdo-0002Kt-Kp; Fri, 28 Aug 2020 12:54:16 -0400 Received: from relay.sw.ru ([185.231.240.75]:37742 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kBhdm-0001OR-NZ; Fri, 28 Aug 2020 12:54:16 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.94) (envelope-from ) id 1kBhdH-001XPR-RR; Fri, 28 Aug 2020 19:53:43 +0300 To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, kwolf@redhat.com, mreitz@redhat.com, armbru@redhat.com, jsnow@redhat.com, eblake@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, andrey.shinkevich@virtuozzo.com Subject: [PATCH v8 2/7] copy-on-read: add filter append/drop functions Date: Fri, 28 Aug 2020 19:52:54 +0300 Message-Id: <1598633579-221780-3-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1598633579-221780-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1598633579-221780-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/28 12:53:26 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Reply-to: Andrey Shinkevich From: Andrey Shinkevich via Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Provide API for the COR-filter insertion/removal. Also, drop the filter child permissions for an inactive state when the filter node is being removed. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 104 +++++++++++++++++++++++++++++++++++++++++++++++= ++++ block/copy-on-read.h | 35 +++++++++++++++++ 2 files changed, 139 insertions(+) create mode 100644 block/copy-on-read.h diff --git a/block/copy-on-read.c b/block/copy-on-read.c index cb03e0f..0ede7aa 100644 --- a/block/copy-on-read.c +++ b/block/copy-on-read.c @@ -23,11 +23,21 @@ #include "qemu/osdep.h" #include "block/block_int.h" #include "qemu/module.h" +#include "qapi/error.h" +#include "qapi/qmp/qdict.h" +#include "block/copy-on-read.h" + + +typedef struct BDRVStateCOR { + bool active; +} BDRVStateCOR; =20 =20 static int cor_open(BlockDriverState *bs, QDict *options, int flags, Error **errp) { + BDRVStateCOR *state =3D bs->opaque; + bs->file =3D bdrv_open_child(NULL, options, "file", bs, &child_of_bds, BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, false, errp); @@ -42,6 +52,13 @@ static int cor_open(BlockDriverState *bs, QDict *options= , int flags, ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & bs->file->bs->supported_zero_flags); =20 + state->active =3D true; + + /* + * We don't need to call bdrv_child_refresh_perms() now as the permiss= ions + * will be updated later when the filter node gets its parent. + */ + return 0; } =20 @@ -57,6 +74,17 @@ static void cor_child_perm(BlockDriverState *bs, BdrvChi= ld *c, uint64_t perm, uint64_t shared, uint64_t *nperm, uint64_t *nshared) { + BDRVStateCOR *s =3D bs->opaque; + + if (!s->active) { + /* + * While the filter is being removed + */ + *nperm =3D 0; + *nshared =3D BLK_PERM_ALL; + return; + } + *nperm =3D perm & PERM_PASSTHROUGH; *nshared =3D (shared & PERM_PASSTHROUGH) | PERM_UNCHANGED; =20 @@ -135,6 +163,7 @@ static void cor_lock_medium(BlockDriverState *bs, bool = locked) =20 static BlockDriver bdrv_copy_on_read =3D { .format_name =3D "copy-on-read", + .instance_size =3D sizeof(BDRVStateCOR), =20 .bdrv_open =3D cor_open, .bdrv_child_perm =3D cor_child_perm, @@ -159,4 +188,79 @@ static void bdrv_copy_on_read_init(void) bdrv_register(&bdrv_copy_on_read); } =20 + +static BlockDriverState *create_filter_node(BlockDriverState *bs, + const char *filter_node_name, + Error **errp) +{ + QDict *opts =3D qdict_new(); + + qdict_put_str(opts, "driver", "copy-on-read"); + qdict_put_str(opts, "file", bdrv_get_node_name(bs)); + if (filter_node_name) { + qdict_put_str(opts, "node-name", filter_node_name); + } + + return bdrv_open(NULL, NULL, opts, BDRV_O_RDWR, errp); +} + + +BlockDriverState *bdrv_cor_filter_append(BlockDriverState *bs, + const char *filter_node_name, + Error **errp) +{ + BlockDriverState *cor_filter_bs; + Error *local_err =3D NULL; + + cor_filter_bs =3D create_filter_node(bs, filter_node_name, errp); + if (cor_filter_bs =3D=3D NULL) { + error_prepend(errp, "Could not create filter node: "); + return NULL; + } + + if (!filter_node_name) { + cor_filter_bs->implicit =3D true; + } + + bdrv_drained_begin(bs); + bdrv_replace_node(bs, cor_filter_bs, &local_err); + bdrv_drained_end(bs); + + if (local_err) { + bdrv_unref(cor_filter_bs); + error_propagate(errp, local_err); + return NULL; + } + + return cor_filter_bs; +} + + +void bdrv_cor_filter_drop(BlockDriverState *cor_filter_bs) +{ + BdrvChild *child; + BlockDriverState *bs; + BDRVStateCOR *s =3D cor_filter_bs->opaque; + + child =3D bdrv_filter_child(cor_filter_bs); + if (!child) { + return; + } + bs =3D child->bs; + + /* Retain the BDS until we complete the graph change. */ + bdrv_ref(bs); + /* Hold a guest back from writing while permissions are being reset. */ + bdrv_drained_begin(bs); + /* Drop permissions before the graph change. */ + s->active =3D false; + bdrv_child_refresh_perms(cor_filter_bs, child, &error_abort); + bdrv_replace_node(cor_filter_bs, bs, &error_abort); + + bdrv_drained_end(bs); + bdrv_unref(bs); + bdrv_unref(cor_filter_bs); +} + + block_init(bdrv_copy_on_read_init); diff --git a/block/copy-on-read.h b/block/copy-on-read.h new file mode 100644 index 0000000..1686e4e --- /dev/null +++ b/block/copy-on-read.h @@ -0,0 +1,35 @@ +/* + * Copy-on-read filter block driver + * + * The filter driver performs Copy-On-Read (COR) operations + * + * Copyright (c) 2018-2020 Virtuozzo International GmbH. + * + * Author: + * Andrey Shinkevich + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#ifndef COPY_ON_READ_FILTER +#define COPY_ON_READ_FILTER + +#include "block/block_int.h" + +BlockDriverState *bdrv_cor_filter_append(BlockDriverState *bs, + const char *filter_node_name, + Error **errp); +void bdrv_cor_filter_drop(BlockDriverState *cor_filter_bs); + +#endif /* COPY_ON_READ_FILTER */ --=20 1.8.3.1 From nobody Sat May 18 15:08:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1598633734; cv=none; d=zohomail.com; s=zohoarc; b=NaXrwa1mmASYTDmzFwwCJg6SkN0j9bMsczdI84lA5OMYfdkSbxELzcJlFlnPa4m4e5n+bzejU7+BHNP7AJeCfNgfgxgkivKN2fZ+SD8JdpKtax7KpA0Wo5nFbUIBrVgRP3zoeUBoP9J3JOZq2xunYzUfhoD5uvf3uqUohwmqINs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598633734; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=k1G+XO2TEboeuoT/2PwnMVHvJSyfavsujPHnRKrAeuQ=; b=bqkkaPI2Xdj27biEByh/wjXttikYOKboWli/rSrIPv1RrcAUjYPzqVgEWjiBD1m4Ew+mRSQU/mD7ZGHKZ/PrlojSb0m2zrOSafr8zFe4X9KtJGe7SHPfxnd3A++3iKqPzc5IWpsdKADZ32tXcliuSjHpGOtMpOLARDaet5i9Xj8= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598633734962268.7955994767185; Fri, 28 Aug 2020 09:55:34 -0700 (PDT) Received: from localhost ([::1]:41534 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kBhf4-0004iE-2S for importer@patchew.org; Fri, 28 Aug 2020 12:55:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33654) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kBheB-000397-Pq; Fri, 28 Aug 2020 12:54:39 -0400 Received: from relay.sw.ru ([185.231.240.75]:37850 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kBheA-0001Pk-0r; Fri, 28 Aug 2020 12:54:39 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.94) (envelope-from ) id 1kBhdf-001XPR-FC; Fri, 28 Aug 2020 19:54:07 +0300 To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, kwolf@redhat.com, mreitz@redhat.com, armbru@redhat.com, jsnow@redhat.com, eblake@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, andrey.shinkevich@virtuozzo.com Subject: [PATCH v8 3/7] qapi: add filter-node-name to block-stream Date: Fri, 28 Aug 2020 19:52:55 +0300 Message-Id: <1598633579-221780-4-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1598633579-221780-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1598633579-221780-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/28 12:53:26 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Reply-to: Andrey Shinkevich From: Andrey Shinkevich via Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Provide the possibility to pass the 'filter-node-name' parameter to the block-stream job as it is done for the commit block job. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/monitor/block-hmp-cmds.c | 4 ++-- block/stream.c | 4 +++- blockdev.c | 4 +++- include/block/block_int.h | 7 ++++++- qapi/block-core.json | 6 ++++++ 5 files changed, 20 insertions(+), 5 deletions(-) diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c index 4d3db5e..4e66775 100644 --- a/block/monitor/block-hmp-cmds.c +++ b/block/monitor/block-hmp-cmds.c @@ -507,8 +507,8 @@ void hmp_block_stream(Monitor *mon, const QDict *qdict) =20 qmp_block_stream(true, device, device, base !=3D NULL, base, false, NU= LL, false, NULL, qdict_haskey(qdict, "speed"), speed, tru= e, - BLOCKDEV_ON_ERROR_REPORT, false, false, false, false, - &error); + BLOCKDEV_ON_ERROR_REPORT, false, NULL, false, false, = false, + false, &error); =20 hmp_handle_error(mon, error); } diff --git a/block/stream.c b/block/stream.c index b9c1141..8bf6b6d 100644 --- a/block/stream.c +++ b/block/stream.c @@ -221,7 +221,9 @@ static const BlockJobDriver stream_job_driver =3D { void stream_start(const char *job_id, BlockDriverState *bs, BlockDriverState *base, const char *backing_file_str, int creation_flags, int64_t speed, - BlockdevOnError on_error, Error **errp) + BlockdevOnError on_error, + const char *filter_node_name, + Error **errp) { StreamBlockJob *s; BlockDriverState *iter; diff --git a/blockdev.c b/blockdev.c index 237fffb..cc531cb 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2476,6 +2476,7 @@ void qmp_block_stream(bool has_job_id, const char *jo= b_id, const char *device, bool has_backing_file, const char *backing_file, bool has_speed, int64_t speed, bool has_on_error, BlockdevOnError on_error, + bool has_filter_node_name, const char *filter_node_n= ame, bool has_auto_finalize, bool auto_finalize, bool has_auto_dismiss, bool auto_dismiss, Error **errp) @@ -2558,7 +2559,8 @@ void qmp_block_stream(bool has_job_id, const char *jo= b_id, const char *device, } =20 stream_start(has_job_id ? job_id : NULL, bs, base_bs, base_name, - job_flags, has_speed ? speed : 0, on_error, &local_err); + job_flags, has_speed ? speed : 0, on_error, + filter_node_name, &local_err); if (local_err) { error_propagate(errp, local_err); goto out; diff --git a/include/block/block_int.h b/include/block/block_int.h index 465a601..3efde33 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -1122,6 +1122,9 @@ int is_windows_drive(const char *filename); * See @BlockJobCreateFlags * @speed: The maximum speed, in bytes per second, or 0 for unlimited. * @on_error: The action to take upon error. + * @filter_node_name: The node name that should be assigned to the filter + * driver that the commit job inserts into the graph above @bs. NULL means + * that a node name should be autogenerated. * @errp: Error object. * * Start a streaming operation on @bs. Clusters that are unallocated @@ -1134,7 +1137,9 @@ int is_windows_drive(const char *filename); void stream_start(const char *job_id, BlockDriverState *bs, BlockDriverState *base, const char *backing_file_str, int creation_flags, int64_t speed, - BlockdevOnError on_error, Error **errp); + BlockdevOnError on_error, + const char *filter_node_name, + Error **errp); =20 /** * commit_start: diff --git a/qapi/block-core.json b/qapi/block-core.json index 0b8ccd3..e5ccf8a 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2524,6 +2524,11 @@ # 'stop' and 'enospc' can only be used if the block device # supports io-status (see BlockInfo). Since 1.3. # +# @filter-node-name: the node name that should be assigned to the +# filter driver that the stream job inserts into the gr= aph +# above @device. If this option is not given, a node na= me is +# autogenerated. (Since: 5.2) +# # @auto-finalize: When false, this job will wait in a PENDING state after = it has # finished its work, waiting for @block-job-finalize before # making any block graph changes. @@ -2554,6 +2559,7 @@ 'data': { '*job-id': 'str', 'device': 'str', '*base': 'str', '*base-node': 'str', '*backing-file': 'str', '*speed': 'int', '*on-error': 'BlockdevOnError', + '*filter-node-name': 'str', '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } } =20 ## --=20 1.8.3.1 From nobody Sat May 18 15:08:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1598633755; cv=none; d=zohomail.com; s=zohoarc; b=fVPB+9+ZBnJobGc6R1BV2xmpaXMcTTRcWijiI7FFys+3y9TJqvOw5FMSSx0jXb3VeB4BEYDg2Ispy1yYDpw8LfCkiGpUaP+rk8BoJ9Lqlo3Eeqq+CpMGFD6YHxXtqBl1C7pQAUsToSMArdlZGq4fgEIxaFpoC7AoBAvNrkx8m/4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598633755; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=yGtJZ9t7rmoRmRUq6TWaT8+OMjViEKj3mf4k9qVx9eI=; b=dBn7dDaP9WbcTTPo+wld383CzNECaD38ZrJWANx0i0CjmbSUy9ALAT3wCPtEDb1tcf59fGEbr3QQGDlMG/xTg1zl/sn64a/9rzd/7wu75K0ICYgtkXmQdkQO6cUatyLzdYYmuwVmXwK9RGuGwfEXu1/RhH3LXDufFgG1A63EvPY= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598633755227763.8958460479442; Fri, 28 Aug 2020 09:55:55 -0700 (PDT) Received: from localhost ([::1]:43506 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kBhfO-0005VK-BM for importer@patchew.org; Fri, 28 Aug 2020 12:55:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33722) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kBheZ-0003zJ-Fh; Fri, 28 Aug 2020 12:55:03 -0400 Received: from relay.sw.ru ([185.231.240.75]:37972 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kBheX-0001SE-NU; Fri, 28 Aug 2020 12:55:03 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.94) (envelope-from ) id 1kBhe3-001XPR-Ip; Fri, 28 Aug 2020 19:54:31 +0300 To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, kwolf@redhat.com, mreitz@redhat.com, armbru@redhat.com, jsnow@redhat.com, eblake@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, andrey.shinkevich@virtuozzo.com Subject: [PATCH v8 4/7] copy-on-read: pass base file name to COR driver Date: Fri, 28 Aug 2020 19:52:56 +0300 Message-Id: <1598633579-221780-5-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1598633579-221780-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1598633579-221780-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/28 12:53:26 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Reply-to: Andrey Shinkevich From: Andrey Shinkevich via Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" To limit the guest's COR operations by the base node in the backing chain during stream job, pass the base file name to the copy-on-read driver. The rest of the functionality will be implemented in the patch that follows. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 41 ++++++++++++++++++++++++++++++++++++++++- block/copy-on-read.h | 1 + 2 files changed, 41 insertions(+), 1 deletion(-) diff --git a/block/copy-on-read.c b/block/copy-on-read.c index 0ede7aa..1f858bb 100644 --- a/block/copy-on-read.c +++ b/block/copy-on-read.c @@ -24,19 +24,45 @@ #include "block/block_int.h" #include "qemu/module.h" #include "qapi/error.h" +#include "qapi/qmp/qerror.h" #include "qapi/qmp/qdict.h" #include "block/copy-on-read.h" =20 =20 typedef struct BDRVStateCOR { bool active; + BlockDriverState *base_bs; } BDRVStateCOR; =20 +/* + * Non-zero pointers are the caller's responsibility. + */ +static BlockDriverState *get_base_by_name(BlockDriverState *bs, + const char *base_name, Error **e= rrp) +{ + BlockDriverState *base_bs =3D NULL; + AioContext *aio_context; + + base_bs =3D bdrv_find_backing_image(bs, base_name); + if (base_bs =3D=3D NULL) { + error_setg(errp, QERR_BASE_NOT_FOUND, base_name); + return NULL; + } + + aio_context =3D bdrv_get_aio_context(bs); + aio_context_acquire(aio_context); + assert(bdrv_get_aio_context(base_bs) =3D=3D aio_context); + aio_context_release(aio_context); + + return base_bs; +} =20 static int cor_open(BlockDriverState *bs, QDict *options, int flags, Error **errp) { + BlockDriverState *base_bs =3D NULL; BDRVStateCOR *state =3D bs->opaque; + const char *base_name =3D qdict_get_try_str(options, "base"); =20 bs->file =3D bdrv_open_child(NULL, options, "file", bs, &child_of_bds, BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, @@ -52,7 +78,15 @@ static int cor_open(BlockDriverState *bs, QDict *options= , int flags, ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & bs->file->bs->supported_zero_flags); =20 + if (base_name) { + qdict_del(options, "base"); + base_bs =3D get_base_by_name(bs, base_name, errp); + if (!base_bs) { + return -EINVAL; + } + } state->active =3D true; + state->base_bs =3D base_bs; =20 /* * We don't need to call bdrv_child_refresh_perms() now as the permiss= ions @@ -190,6 +224,7 @@ static void bdrv_copy_on_read_init(void) =20 =20 static BlockDriverState *create_filter_node(BlockDriverState *bs, + const char *base_name, const char *filter_node_name, Error **errp) { @@ -197,6 +232,9 @@ static BlockDriverState *create_filter_node(BlockDriver= State *bs, =20 qdict_put_str(opts, "driver", "copy-on-read"); qdict_put_str(opts, "file", bdrv_get_node_name(bs)); + if (base_name) { + qdict_put_str(opts, "base", base_name); + } if (filter_node_name) { qdict_put_str(opts, "node-name", filter_node_name); } @@ -206,13 +244,14 @@ static BlockDriverState *create_filter_node(BlockDriv= erState *bs, =20 =20 BlockDriverState *bdrv_cor_filter_append(BlockDriverState *bs, + const char *base_name, const char *filter_node_name, Error **errp) { BlockDriverState *cor_filter_bs; Error *local_err =3D NULL; =20 - cor_filter_bs =3D create_filter_node(bs, filter_node_name, errp); + cor_filter_bs =3D create_filter_node(bs, base_name, filter_node_name, = errp); if (cor_filter_bs =3D=3D NULL) { error_prepend(errp, "Could not create filter node: "); return NULL; diff --git a/block/copy-on-read.h b/block/copy-on-read.h index 1686e4e..6a7c8bb 100644 --- a/block/copy-on-read.h +++ b/block/copy-on-read.h @@ -28,6 +28,7 @@ #include "block/block_int.h" =20 BlockDriverState *bdrv_cor_filter_append(BlockDriverState *bs, + const char *base_name, const char *filter_node_name, Error **errp); void bdrv_cor_filter_drop(BlockDriverState *cor_filter_bs); --=20 1.8.3.1 From nobody Sat May 18 15:08:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1598633785; cv=none; d=zohomail.com; s=zohoarc; b=BeJ+IXa+6BYC7Z0OSpgR2qkjv56BnR7ek8SKBeMVORHNyVrwbAFByfd1a8G8SozC/w8JX3aUim1xdyk4TNlLMVgBvpWCYwxJtsLHKxPGo41iuqSE/+PDd5p59fW/1cwd8ImSp1wt9ZN7vhWTU7tHTuevNGkcTAnzAr7Tbp5MDHM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598633785; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=UFkahTJilEJ+6Bj+5rZwwjimCkJSv/iU+min1P94cew=; b=PsfYZYhFxZZeYa+pGUu87UYSPzClPZiV0O9ldSP09d6N3El5+hVH+fJGy0lJrXQpDp3q0MzplXBcX57cZDVDlanLFvxK0wFfVRErKTU0tvLYrWum6NHZTM25aXiwXksg2CCWAFsxuGq2YZgXYKHytdCCfdoSqre8toAkXG1gqzM= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598633785919166.37652882101224; Fri, 28 Aug 2020 09:56:25 -0700 (PDT) Received: from localhost ([::1]:46364 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kBhft-0006fi-5i for importer@patchew.org; Fri, 28 Aug 2020 12:56:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33804) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kBhex-0004zQ-2v; Fri, 28 Aug 2020 12:55:27 -0400 Received: from relay.sw.ru ([185.231.240.75]:38086 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kBhev-0001TZ-GJ; Fri, 28 Aug 2020 12:55:26 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.94) (envelope-from ) id 1kBheR-001XPR-6Z; Fri, 28 Aug 2020 19:54:55 +0300 To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, kwolf@redhat.com, mreitz@redhat.com, armbru@redhat.com, jsnow@redhat.com, eblake@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, andrey.shinkevich@virtuozzo.com Subject: [PATCH v8 5/7] copy-on-read: limit guest writes to base in COR driver Date: Fri, 28 Aug 2020 19:52:57 +0300 Message-Id: <1598633579-221780-6-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1598633579-221780-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1598633579-221780-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/28 12:53:26 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Reply-to: Andrey Shinkevich From: Andrey Shinkevich via Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Limit the guest's COR operations by the base node in the backing chain during a stream job. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/block/copy-on-read.c b/block/copy-on-read.c index 1f858bb..ecbd1f8 100644 --- a/block/copy-on-read.c +++ b/block/copy-on-read.c @@ -57,6 +57,37 @@ static BlockDriverState *get_base_by_name(BlockDriverSta= te *bs, return base_bs; } =20 +/* + * Returns 1 if the block is allocated in a node between cor_filter_bs->fi= le->bs + * and the base_bs in the backing chain. Otherwise, returns 0. + * The COR operation is allowed if the base_bs is not specified - return 1. + * Returns negative errno on failure. + */ +static int node_above_base(BlockDriverState *cor_filter_bs, uint64_t offse= t, + uint64_t bytes) +{ + int ret; + int64_t dummy; + BlockDriverState *file =3D NULL; + BDRVStateCOR *state =3D cor_filter_bs->opaque; + + if (!state->base_bs) { + return 1; + } + + ret =3D bdrv_block_status_above(cor_filter_bs->file->bs, state->base_b= s, + offset, bytes, &dummy, NULL, &file); + if (ret < 0) { + return ret; + } + + if (file) { + return 1; + } + + return 0; +} + static int cor_open(BlockDriverState *bs, QDict *options, int flags, Error **errp) { @@ -153,6 +184,12 @@ static int coroutine_fn cor_co_pwritev_part(BlockDrive= rState *bs, QEMUIOVector *qiov, size_t qiov_offset, int flags) { + int ret =3D node_above_base(bs, offset, bytes); + + if (!ret || ret < 0) { + return ret; + } + return bdrv_co_pwritev_part(bs->file, offset, bytes, qiov, qiov_offset, flags); } @@ -162,6 +199,12 @@ static int coroutine_fn cor_co_pwrite_zeroes(BlockDriv= erState *bs, int64_t offset, int bytes, BdrvRequestFlags flags) { + int ret =3D node_above_base(bs, offset, bytes); + + if (!ret || ret < 0) { + return ret; + } + return bdrv_co_pwrite_zeroes(bs->file, offset, bytes, flags); } =20 @@ -178,6 +221,12 @@ static int coroutine_fn cor_co_pwritev_compressed(Bloc= kDriverState *bs, uint64_t bytes, QEMUIOVector *qiov) { + int ret =3D node_above_base(bs, offset, bytes); + + if (!ret || ret < 0) { + return ret; + } + return bdrv_co_pwritev(bs->file, offset, bytes, qiov, BDRV_REQ_WRITE_COMPRESSED); } --=20 1.8.3.1 From nobody Sat May 18 15:08:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1598633811; cv=none; d=zohomail.com; s=zohoarc; b=eyR4NyDZbXqqmxQV9OBx3puwcNlc4KN0yK8IpdcrvAkvSVPnwlo/IsIxX7u+4FwFxdD4dKKIecvhfMkMgVlpVUhYvjIWWc0AMOzskD2a/TI0uZEOZiVVVo+DSc0j/Mqg5QMoOqPioiZzBPEastc4Gk5mISFkdgLiUdpvdFcedHU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598633811; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=cJNrq+vOaJjiDnF+7/ARrn4PWgEVnPfRdbORL+CZzRQ=; b=f52XyAiRBV++0yU+IbkH6j5qYMPppLnzvqfDRbnNkuDJ/lsdZZXaAjfwNFNaWCh+uLP38gqV9Jo+ObB5NDnyV3dn2mxfrK5LRj6w0tP+uBmRXsfN4ZRpIJ6X75dsEQ4chsGrbB9NDTXuioyjO8b+FNNGwK2+SJtpVFyhYP4wXvY= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598633811560894.0689791895894; Fri, 28 Aug 2020 09:56:51 -0700 (PDT) Received: from localhost ([::1]:48832 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kBhgI-0007eo-PP for importer@patchew.org; Fri, 28 Aug 2020 12:56:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33890) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kBhfM-00064b-6r; Fri, 28 Aug 2020 12:55:52 -0400 Received: from relay.sw.ru ([185.231.240.75]:38202 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kBhfJ-0001fm-9l; Fri, 28 Aug 2020 12:55:51 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.94) (envelope-from ) id 1kBheo-001XPR-R2; Fri, 28 Aug 2020 19:55:18 +0300 To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, kwolf@redhat.com, mreitz@redhat.com, armbru@redhat.com, jsnow@redhat.com, eblake@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, andrey.shinkevich@virtuozzo.com Subject: [PATCH v8 6/7] block-stream: freeze link to base node during stream job Date: Fri, 28 Aug 2020 19:52:58 +0300 Message-Id: <1598633579-221780-7-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1598633579-221780-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1598633579-221780-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/28 12:53:26 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Reply-to: Andrey Shinkevich From: Andrey Shinkevich via Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" To keep the base node unchanged during the block-stream operation, freeze it as the other part of the backing chain with the intermediate nodes related to the job. This patch revers the change made with the commit c624b015bf as the correct base file name and its format have to be written down to the QCOW2 header on the disk when the backing file is being changed after the stream job completes. This reversion incurs changes in the tests 030, 245 and discards the test 258 where concurrent stream/commit jobs are tested. When the link to a base node is frozen, the concurrent job cannot change the common backing chain. Signed-off-by: Andrey Shinkevich --- block/stream.c | 29 ++------ tests/qemu-iotests/030 | 10 +-- tests/qemu-iotests/245 | 2 +- tests/qemu-iotests/258 | 161 -----------------------------------------= ---- tests/qemu-iotests/258.out | 33 ---------- 5 files changed, 14 insertions(+), 221 deletions(-) delete mode 100755 tests/qemu-iotests/258 delete mode 100644 tests/qemu-iotests/258.out diff --git a/block/stream.c b/block/stream.c index 8bf6b6d..fee4117 100644 --- a/block/stream.c +++ b/block/stream.c @@ -32,7 +32,7 @@ enum { typedef struct StreamBlockJob { BlockJob common; BlockDriverState *base_overlay; /* COW overlay (stream from this) */ - BlockDriverState *above_base; /* Node directly above the base */ + BlockDriverState *base; /* The base node */ BlockdevOnError on_error; char *backing_file_str; bool bs_read_only; @@ -54,7 +54,7 @@ static void stream_abort(Job *job) =20 if (s->chain_frozen) { BlockJob *bjob =3D &s->common; - bdrv_unfreeze_backing_chain(blk_bs(bjob->blk), s->above_base); + bdrv_unfreeze_backing_chain(blk_bs(bjob->blk), s->base); } } =20 @@ -64,11 +64,11 @@ static int stream_prepare(Job *job) BlockJob *bjob =3D &s->common; BlockDriverState *bs =3D blk_bs(bjob->blk); BlockDriverState *unfiltered_bs =3D bdrv_skip_filters(bs); - BlockDriverState *base =3D bdrv_filter_or_cow_bs(s->above_base); + BlockDriverState *base =3D s->base; Error *local_err =3D NULL; int ret =3D 0; =20 - bdrv_unfreeze_backing_chain(bs, s->above_base); + bdrv_unfreeze_backing_chain(bs, s->base); s->chain_frozen =3D false; =20 if (bdrv_cow_child(unfiltered_bs)) { @@ -230,7 +230,6 @@ void stream_start(const char *job_id, BlockDriverState = *bs, bool bs_read_only; int basic_flags =3D BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGE= D; BlockDriverState *base_overlay =3D bdrv_find_overlay(bs, base); - BlockDriverState *above_base; =20 if (!base_overlay) { error_setg(errp, "'%s' is not in the backing chain of '%s'", @@ -238,20 +237,7 @@ void stream_start(const char *job_id, BlockDriverState= *bs, return; } =20 - /* - * Find the node directly above @base. @base_overlay is a COW overlay= , so - * it must have a bdrv_cow_child(), but it is the immediate overlay of - * @base, so between the two there can only be filters. - */ - above_base =3D base_overlay; - if (bdrv_cow_bs(above_base) !=3D base) { - above_base =3D bdrv_cow_bs(above_base); - while (bdrv_filter_bs(above_base) !=3D base) { - above_base =3D bdrv_filter_bs(above_base); - } - } - - if (bdrv_freeze_backing_chain(bs, above_base, errp) < 0) { + if (bdrv_freeze_backing_chain(bs, base, errp) < 0) { return; } =20 @@ -284,7 +270,6 @@ void stream_start(const char *job_id, BlockDriverState = *bs, * above_base node might change after the call to * bdrv_reopen_set_read_only() due to parallel block jobs running. */ - base =3D bdrv_filter_or_cow_bs(above_base); for (iter =3D bdrv_filter_or_cow_bs(bs); iter !=3D base; iter =3D bdrv_filter_or_cow_bs(iter)) { @@ -293,7 +278,7 @@ void stream_start(const char *job_id, BlockDriverState = *bs, } =20 s->base_overlay =3D base_overlay; - s->above_base =3D above_base; + s->base =3D base; s->backing_file_str =3D g_strdup(backing_file_str); s->bs_read_only =3D bs_read_only; s->chain_frozen =3D true; @@ -307,5 +292,5 @@ fail: if (bs_read_only) { bdrv_reopen_set_read_only(bs, true, NULL); } - bdrv_unfreeze_backing_chain(bs, above_base); + bdrv_unfreeze_backing_chain(bs, base); } diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030 index 1cdd7e2..c565e76 100755 --- a/tests/qemu-iotests/030 +++ b/tests/qemu-iotests/030 @@ -382,7 +382,7 @@ class TestParallelOps(iotests.QMPTestCase): # Stream from node2 into node4 result =3D self.vm.qmp('block-stream', device=3D'node4', base_node= =3D'node2', job_id=3D'node4') self.assert_qmp(result, 'error/desc', - "Cannot freeze 'backing' link to 'commit-filter'") + "Cannot change 'backing' link from 'commit-filter' to 'node2'") =20 result =3D self.vm.qmp('block-job-set-speed', device=3D'drive0', s= peed=3D0) self.assert_qmp(result, 'return', {}) @@ -406,13 +406,15 @@ class TestParallelOps(iotests.QMPTestCase): # Stream from node2 into node4 result =3D self.vm.qmp('block-stream', device=3D'node4', base_node=3D'commit-filter', job_id=3D'node4') - self.assert_qmp(result, 'return', {}) + self.assert_qmp(result, 'error/desc', + "Cannot freeze 'backing' link to 'commit-filter'") =20 result =3D self.vm.qmp('block-job-set-speed', device=3D'drive0', s= peed=3D0) self.assert_qmp(result, 'return', {}) =20 - self.vm.run_job(job=3D'drive0', auto_dismiss=3DTrue) - self.vm.run_job(job=3D'node4', auto_dismiss=3DTrue) + self.wait_until_completed() + #self.vm.run_job(job=3D'drive0', auto_dismiss=3DTrue) + #self.vm.run_job(job=3D'node4', auto_dismiss=3DTrue) self.assert_no_active_block_jobs() =20 # Test a block-stream and a block-commit job in parallel diff --git a/tests/qemu-iotests/245 b/tests/qemu-iotests/245 index 5035763..b9399ef 100755 --- a/tests/qemu-iotests/245 +++ b/tests/qemu-iotests/245 @@ -872,7 +872,7 @@ class TestBlockdevReopen(iotests.QMPTestCase): =20 # We can remove hd2 while the stream job is ongoing opts['backing']['backing'] =3D None - self.reopen(opts, {}) + self.reopen(opts, {}, "Cannot change 'backing' link from 'hd1' to = 'hd2'") =20 # We can't remove hd1 while the stream job is ongoing opts['backing'] =3D None diff --git a/tests/qemu-iotests/258 b/tests/qemu-iotests/258 deleted file mode 100755 index e305a15..0000000 --- a/tests/qemu-iotests/258 +++ /dev/null @@ -1,161 +0,0 @@ -#!/usr/bin/env python3 -# -# Very specific tests for adjacent commit/stream block jobs -# -# Copyright (C) 2019 Red Hat, Inc. -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see . -# -# Creator/Owner: Max Reitz - -import iotests -from iotests import log, qemu_img, qemu_io_silent, \ - filter_qmp_testfiles, filter_qmp_imgfmt - -# Returns a node for blockdev-add -def node(node_name, path, backing=3DNone, fmt=3DNone, throttle=3DNone): - if fmt is None: - fmt =3D iotests.imgfmt - - res =3D { - 'node-name': node_name, - 'driver': fmt, - 'file': { - 'driver': 'file', - 'filename': path - } - } - - if backing is not None: - res['backing'] =3D backing - - if throttle: - res['file'] =3D { - 'driver': 'throttle', - 'throttle-group': throttle, - 'file': res['file'] - } - - return res - -# Finds a node in the debug block graph -def find_graph_node(graph, node_id): - return next(node for node in graph['nodes'] if node['id'] =3D=3D node_= id) - - -def test_concurrent_finish(write_to_stream_node): - log('') - log('=3D=3D=3D Commit and stream finish concurrently (letting %s write= ) =3D=3D=3D' % \ - ('stream' if write_to_stream_node else 'commit')) - log('') - - # All chosen in such a way that when the commit job wants to - # finish, it polls and thus makes stream finish concurrently -- - # and the other way around, depending on whether the commit job - # is finalized before stream completes or not. - - with iotests.FilePath('node4.img') as node4_path, \ - iotests.FilePath('node3.img') as node3_path, \ - iotests.FilePath('node2.img') as node2_path, \ - iotests.FilePath('node1.img') as node1_path, \ - iotests.FilePath('node0.img') as node0_path, \ - iotests.VM() as vm: - - # It is important to use raw for the base layer (so that - # permissions are just handed through to the protocol layer) - assert qemu_img('create', '-f', 'raw', node0_path, '64M') =3D=3D 0 - - stream_throttle=3DNone - commit_throttle=3DNone - - for path in [node1_path, node2_path, node3_path, node4_path]: - assert qemu_img('create', '-f', iotests.imgfmt, path, '64M') = =3D=3D 0 - - if write_to_stream_node: - # This is what (most of the time) makes commit finish - # earlier and then pull in stream - assert qemu_io_silent(node2_path, - '-c', 'write %iK 64K' % (65536 - 192), - '-c', 'write %iK 64K' % (65536 - 64)) = =3D=3D 0 - - stream_throttle=3D'tg' - else: - # And this makes stream finish earlier - assert qemu_io_silent(node1_path, - '-c', 'write %iK 64K' % (65536 - 64)) = =3D=3D 0 - - commit_throttle=3D'tg' - - vm.launch() - - vm.qmp_log('object-add', - qom_type=3D'throttle-group', - id=3D'tg', - props=3D{ - 'x-iops-write': 1, - 'x-iops-write-max': 1 - }) - - vm.qmp_log('blockdev-add', - filters=3D[filter_qmp_testfiles, filter_qmp_imgfmt], - **node('node4', node4_path, throttle=3Dstream_throttle, - backing=3Dnode('node3', node3_path, - backing=3Dnode('node2', node2_path, - backing=3Dnode('node1', node1_path, - backing=3Dnode('node0', node0_path, throttle=3Dcommit= _throttle, - fmt=3D'raw')))))) - - vm.qmp_log('block-commit', - job_id=3D'commit', - device=3D'node4', - filter_node_name=3D'commit-filter', - top_node=3D'node1', - base_node=3D'node0', - auto_finalize=3DFalse) - - vm.qmp_log('block-stream', - job_id=3D'stream', - device=3D'node3', - base_node=3D'commit-filter') - - if write_to_stream_node: - vm.run_job('commit', auto_finalize=3DFalse, auto_dismiss=3DTru= e) - vm.run_job('stream', auto_finalize=3DTrue, auto_dismiss=3DTrue) - else: - # No, the jobs do not really finish concurrently here, - # the stream job does complete strictly before commit. - # But still, this is close enough for what we want to - # test. - vm.run_job('stream', auto_finalize=3DTrue, auto_dismiss=3DTrue) - vm.run_job('commit', auto_finalize=3DFalse, auto_dismiss=3DTru= e) - - # Assert that the backing node of node3 is node 0 now - graph =3D vm.qmp('x-debug-query-block-graph')['return'] - for edge in graph['edges']: - if edge['name'] =3D=3D 'backing' and \ - find_graph_node(graph, edge['parent'])['name'] =3D=3D 'node= 3': - assert find_graph_node(graph, edge['child'])['name'] =3D= =3D 'node0' - break - - -def main(): - log('Running tests:') - test_concurrent_finish(True) - test_concurrent_finish(False) - -if __name__ =3D=3D '__main__': - # Need backing file and change-backing-file support - iotests.script_main(main, - supported_fmts=3D['qcow2', 'qed'], - supported_platforms=3D['linux']) diff --git a/tests/qemu-iotests/258.out b/tests/qemu-iotests/258.out deleted file mode 100644 index ce6e9ba..0000000 --- a/tests/qemu-iotests/258.out +++ /dev/null @@ -1,33 +0,0 @@ -Running tests: - -=3D=3D=3D Commit and stream finish concurrently (letting stream write) =3D= =3D=3D - -{"execute": "object-add", "arguments": {"id": "tg", "props": {"x-iops-writ= e": 1, "x-iops-write-max": 1}, "qom-type": "throttle-group"}} -{"return": {}} -{"execute": "blockdev-add", "arguments": {"backing": {"backing": {"backing= ": {"backing": {"driver": "raw", "file": {"driver": "file", "filename": "TE= ST_DIR/PID-node0.img"}, "node-name": "node0"}, "driver": "IMGFMT", "file": = {"driver": "file", "filename": "TEST_DIR/PID-node1.img"}, "node-name": "nod= e1"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/= PID-node2.img"}, "node-name": "node2"}, "driver": "IMGFMT", "file": {"drive= r": "file", "filename": "TEST_DIR/PID-node3.img"}, "node-name": "node3"}, "= driver": "IMGFMT", "file": {"driver": "throttle", "file": {"driver": "file"= , "filename": "TEST_DIR/PID-node4.img"}, "throttle-group": "tg"}, "node-nam= e": "node4"}} -{"return": {}} -{"execute": "block-commit", "arguments": {"auto-finalize": false, "base-no= de": "node0", "device": "node4", "filter-node-name": "commit-filter", "job-= id": "commit", "top-node": "node1"}} -{"return": {}} -{"execute": "block-stream", "arguments": {"base-node": "commit-filter", "d= evice": "node3", "job-id": "stream"}} -{"return": {}} -{"execute": "job-finalize", "arguments": {"id": "commit"}} -{"return": {}} -{"data": {"id": "commit", "type": "commit"}, "event": "BLOCK_JOB_PENDING",= "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} -{"data": {"device": "commit", "len": 67108864, "offset": 67108864, "speed"= : 0, "type": "commit"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"micr= oseconds": "USECS", "seconds": "SECS"}} -{"data": {"device": "stream", "len": 67108864, "offset": 67108864, "speed"= : 0, "type": "stream"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"micr= oseconds": "USECS", "seconds": "SECS"}} - -=3D=3D=3D Commit and stream finish concurrently (letting commit write) =3D= =3D=3D - -{"execute": "object-add", "arguments": {"id": "tg", "props": {"x-iops-writ= e": 1, "x-iops-write-max": 1}, "qom-type": "throttle-group"}} -{"return": {}} -{"execute": "blockdev-add", "arguments": {"backing": {"backing": {"backing= ": {"backing": {"driver": "raw", "file": {"driver": "throttle", "file": {"d= river": "file", "filename": "TEST_DIR/PID-node0.img"}, "throttle-group": "t= g"}, "node-name": "node0"}, "driver": "IMGFMT", "file": {"driver": "file", = "filename": "TEST_DIR/PID-node1.img"}, "node-name": "node1"}, "driver": "IM= GFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-node2.img"}, "n= ode-name": "node2"}, "driver": "IMGFMT", "file": {"driver": "file", "filena= me": "TEST_DIR/PID-node3.img"}, "node-name": "node3"}, "driver": "IMGFMT", = "file": {"driver": "file", "filename": "TEST_DIR/PID-node4.img"}, "node-nam= e": "node4"}} -{"return": {}} -{"execute": "block-commit", "arguments": {"auto-finalize": false, "base-no= de": "node0", "device": "node4", "filter-node-name": "commit-filter", "job-= id": "commit", "top-node": "node1"}} -{"return": {}} -{"execute": "block-stream", "arguments": {"base-node": "commit-filter", "d= evice": "node3", "job-id": "stream"}} -{"return": {}} -{"data": {"device": "stream", "len": 67108864, "offset": 67108864, "speed"= : 0, "type": "stream"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"micr= oseconds": "USECS", "seconds": "SECS"}} -{"execute": "job-finalize", "arguments": {"id": "commit"}} -{"return": {}} -{"data": {"id": "commit", "type": "commit"}, "event": "BLOCK_JOB_PENDING",= "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} -{"data": {"device": "commit", "len": 67108864, "offset": 67108864, "speed"= : 0, "type": "commit"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"micr= oseconds": "USECS", "seconds": "SECS"}} --=20 1.8.3.1 From nobody Sat May 18 15:08:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1598633886; cv=none; d=zohomail.com; s=zohoarc; b=feIAJuKZRAGEW9IUasnTSaFBTpmsFGuWcH72N0ki39lQjSBQj9vZWi+9xbwPc5vPz3+EebvSkx/CXnvCJZoix4up3uwJmy+S6BnxtjkgdBHJfo7VQXVsRViao7LV90fWtS5QhIoQDPD1oaM2ttrIbvsg/yiDrazsyxSawIvc0a8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1598633886; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=cChs+L6HLacaZ9DxVUNHLrQrC5+MOKcqnDjCC46cAko=; b=W8uFLgkUMQi5YF7aQWRRYm9m8TOEX3CJIsRNPYpyFWIhKZofyXN1JhBD9dVL8vgobRB1MitnmOfI9//DYeL731p2c0IhW9mECdCYEB47OM2QIymKzfR5oTpucemdxXAwOJgibtDJ7BjtlmI+gJhSL8U5NhmxWBtr28Q0drsZ3c4= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1598633886098677.631729992182; Fri, 28 Aug 2020 09:58:06 -0700 (PDT) Received: from localhost ([::1]:51852 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kBhhV-0000UB-1h for importer@patchew.org; Fri, 28 Aug 2020 12:58:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33988) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kBhfj-0006nv-J4; Fri, 28 Aug 2020 12:56:15 -0400 Received: from relay.sw.ru ([185.231.240.75]:38324 helo=relay3.sw.ru) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kBhfg-0001im-K2; Fri, 28 Aug 2020 12:56:15 -0400 Received: from [172.16.25.136] (helo=localhost.sw.ru) by relay3.sw.ru with esmtp (Exim 4.94) (envelope-from ) id 1kBhfC-001XPR-Er; Fri, 28 Aug 2020 19:55:42 +0300 To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, kwolf@redhat.com, mreitz@redhat.com, armbru@redhat.com, jsnow@redhat.com, eblake@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, andrey.shinkevich@virtuozzo.com Subject: [PATCH v8 7/7] block: apply COR-filter to block-stream jobs Date: Fri, 28 Aug 2020 19:52:59 +0300 Message-Id: <1598633579-221780-8-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1598633579-221780-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1598633579-221780-1-git-send-email-andrey.shinkevich@virtuozzo.com> Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.231.240.75; envelope-from=andrey.shinkevich@virtuozzo.com; helo=relay3.sw.ru X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/28 12:53:26 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Reply-to: Andrey Shinkevich From: Andrey Shinkevich via Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch completes the series with the COR-filter insertion for block-stream operations. Adding the filter makes it possible for copied regions to be discarded in backing files during the block-stream job, what will reduce the disk overuse. The COR-filter insertion incurs changes in the iotests case 245:test_block_stream_4 that reopens the backing chain during a block-stream job. There are changes in the iotests #030 as well. The iotests case 030:test_stream_parallel was deleted due to multiple conflicts between the concurrent job operations over the same backing chain. The base backing node for one job is the top node for another job. It may change due to the filter node inserted into the backing chain while both jobs are running. Another issue is that the parts of the backing chain are being frozen by the running job and may not be changed by the concurrent job when needed. The concept of the parallel jobs with common nodes is considered vital no more. Signed-off-by: Andrey Shinkevich --- block/stream.c | 78 +++++++++++++++++++++++++++---------------= ---- tests/qemu-iotests/030 | 50 +++-------------------------- tests/qemu-iotests/030.out | 4 +-- tests/qemu-iotests/245 | 19 ++++++++--- 4 files changed, 67 insertions(+), 84 deletions(-) diff --git a/block/stream.c b/block/stream.c index fee4117..ab8ba39 100644 --- a/block/stream.c +++ b/block/stream.c @@ -19,6 +19,7 @@ #include "qapi/qmp/qerror.h" #include "qemu/ratelimit.h" #include "sysemu/block-backend.h" +#include "block/copy-on-read.h" =20 enum { /* @@ -33,6 +34,8 @@ typedef struct StreamBlockJob { BlockJob common; BlockDriverState *base_overlay; /* COW overlay (stream from this) */ BlockDriverState *base; /* The base node */ + BlockDriverState *cor_filter_bs; + BlockDriverState *target_bs; BlockdevOnError on_error; char *backing_file_str; bool bs_read_only; @@ -53,22 +56,19 @@ static void stream_abort(Job *job) StreamBlockJob *s =3D container_of(job, StreamBlockJob, common.job); =20 if (s->chain_frozen) { - BlockJob *bjob =3D &s->common; - bdrv_unfreeze_backing_chain(blk_bs(bjob->blk), s->base); + bdrv_unfreeze_backing_chain(s->cor_filter_bs, s->base); } } =20 static int stream_prepare(Job *job) { StreamBlockJob *s =3D container_of(job, StreamBlockJob, common.job); - BlockJob *bjob =3D &s->common; - BlockDriverState *bs =3D blk_bs(bjob->blk); - BlockDriverState *unfiltered_bs =3D bdrv_skip_filters(bs); + BlockDriverState *unfiltered_bs =3D bdrv_skip_filters(s->target_bs); BlockDriverState *base =3D s->base; Error *local_err =3D NULL; int ret =3D 0; =20 - bdrv_unfreeze_backing_chain(bs, s->base); + bdrv_unfreeze_backing_chain(s->cor_filter_bs, s->base); s->chain_frozen =3D false; =20 if (bdrv_cow_child(unfiltered_bs)) { @@ -94,13 +94,14 @@ static void stream_clean(Job *job) { StreamBlockJob *s =3D container_of(job, StreamBlockJob, common.job); BlockJob *bjob =3D &s->common; - BlockDriverState *bs =3D blk_bs(bjob->blk); + + bdrv_cor_filter_drop(s->cor_filter_bs); =20 /* Reopen the image back in read-only mode if necessary */ if (s->bs_read_only) { /* Give up write permissions before making it read-only */ blk_set_perm(bjob->blk, 0, BLK_PERM_ALL, &error_abort); - bdrv_reopen_set_read_only(bs, true, NULL); + bdrv_reopen_set_read_only(s->target_bs, true, NULL); } =20 g_free(s->backing_file_str); @@ -110,9 +111,7 @@ static int coroutine_fn stream_run(Job *job, Error **er= rp) { StreamBlockJob *s =3D container_of(job, StreamBlockJob, common.job); BlockBackend *blk =3D s->common.blk; - BlockDriverState *bs =3D blk_bs(blk); - BlockDriverState *unfiltered_bs =3D bdrv_skip_filters(bs); - bool enable_cor =3D !bdrv_cow_child(s->base_overlay); + BlockDriverState *unfiltered_bs =3D bdrv_skip_filters(s->target_bs); int64_t len; int64_t offset =3D 0; uint64_t delay_ns =3D 0; @@ -124,21 +123,12 @@ static int coroutine_fn stream_run(Job *job, Error **= errp) return 0; } =20 - len =3D bdrv_getlength(bs); + len =3D bdrv_getlength(s->target_bs); if (len < 0) { return len; } job_progress_set_remaining(&s->common.job, len); =20 - /* Turn on copy-on-read for the whole block device so that guest read - * requests help us make progress. Only do this when copying the enti= re - * backing chain since the copy-on-read operation does not take base i= nto - * account. - */ - if (enable_cor) { - bdrv_enable_copy_on_read(bs); - } - for ( ; offset < len; offset +=3D n) { bool copy; int ret; @@ -197,10 +187,6 @@ static int coroutine_fn stream_run(Job *job, Error **e= rrp) } } =20 - if (enable_cor) { - bdrv_disable_copy_on_read(bs); - } - /* Do not remove the backing file if an error was there but ignored. */ return error; } @@ -230,6 +216,7 @@ void stream_start(const char *job_id, BlockDriverState = *bs, bool bs_read_only; int basic_flags =3D BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGE= D; BlockDriverState *base_overlay =3D bdrv_find_overlay(bs, base); + BlockDriverState *cor_filter_bs =3D NULL; =20 if (!base_overlay) { error_setg(errp, "'%s' is not in the backing chain of '%s'", @@ -250,17 +237,37 @@ void stream_start(const char *job_id, BlockDriverStat= e *bs, } } =20 - /* Prevent concurrent jobs trying to modify the graph structure here, = we - * already have our own plans. Also don't allow resize as the image si= ze is - * queried only at the job start and then cached. */ - s =3D block_job_create(job_id, &stream_job_driver, NULL, bs, - basic_flags | BLK_PERM_GRAPH_MOD, - basic_flags | BLK_PERM_WRITE, + cor_filter_bs =3D bdrv_cor_filter_append(bs, backing_file_str, + filter_node_name, errp); + if (cor_filter_bs =3D=3D NULL) { + goto fail; + } + + if (bdrv_freeze_backing_chain(cor_filter_bs, bs, errp) < 0) { + bdrv_cor_filter_drop(cor_filter_bs); + cor_filter_bs =3D NULL; + goto fail; + } + + s =3D block_job_create(job_id, &stream_job_driver, NULL, cor_filter_bs, + BLK_PERM_CONSISTENT_READ, + basic_flags | BLK_PERM_WRITE | BLK_PERM_GRAPH_MOD, speed, creation_flags, NULL, NULL, errp); if (!s) { goto fail; } =20 + /* + * Prevent concurrent jobs trying to modify the graph structure here, = we + * already have our own plans. Also don't allow resize as the image si= ze is + * queried only at the job start and then cached. + */ + if (block_job_add_bdrv(&s->common, "active node", bs, + basic_flags | BLK_PERM_GRAPH_MOD, + basic_flags | BLK_PERM_WRITE, &error_abort)) { + goto fail; + } + /* Block all intermediate nodes between bs and base, because they will * disappear from the chain after this operation. The streaming job re= ads * every block only once, assuming that it doesn't change, so forbid w= rites @@ -279,6 +286,8 @@ void stream_start(const char *job_id, BlockDriverState = *bs, =20 s->base_overlay =3D base_overlay; s->base =3D base; + s->cor_filter_bs =3D cor_filter_bs; + s->target_bs =3D bs; s->backing_file_str =3D g_strdup(backing_file_str); s->bs_read_only =3D bs_read_only; s->chain_frozen =3D true; @@ -292,5 +301,10 @@ fail: if (bs_read_only) { bdrv_reopen_set_read_only(bs, true, NULL); } - bdrv_unfreeze_backing_chain(bs, base); + if (cor_filter_bs) { + bdrv_unfreeze_backing_chain(cor_filter_bs, base); + bdrv_cor_filter_drop(cor_filter_bs); + } else { + bdrv_unfreeze_backing_chain(bs, base); + } } diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030 index c565e76..d651779 100755 --- a/tests/qemu-iotests/030 +++ b/tests/qemu-iotests/030 @@ -221,60 +221,20 @@ class TestParallelOps(iotests.QMPTestCase): for img in self.imgs: os.remove(img) =20 - # Test that it's possible to run several block-stream operations - # in parallel in the same snapshot chain - def test_stream_parallel(self): - self.assert_no_active_block_jobs() - - # Check that the maps don't match before the streaming operations - for i in range(2, self.num_imgs, 2): - self.assertNotEqual(qemu_io('-f', iotests.imgfmt, '-rU', '-c',= 'map', self.imgs[i]), - qemu_io('-f', iotests.imgfmt, '-rU', '-c',= 'map', self.imgs[i-1]), - 'image file map matches backing file befor= e streaming') - - # Create all streaming jobs - pending_jobs =3D [] - for i in range(2, self.num_imgs, 2): - node_name =3D 'node%d' % i - job_id =3D 'stream-%s' % node_name - pending_jobs.append(job_id) - result =3D self.vm.qmp('block-stream', device=3Dnode_name, job= _id=3Djob_id, base=3Dself.imgs[i-2], speed=3D512*1024) - self.assert_qmp(result, 'return', {}) - - for job in pending_jobs: - result =3D self.vm.qmp('block-job-set-speed', device=3Djob, sp= eed=3D0) - self.assert_qmp(result, 'return', {}) - - # Wait for all jobs to be finished. - while len(pending_jobs) > 0: - for event in self.vm.get_qmp_events(wait=3DTrue): - if event['event'] =3D=3D 'BLOCK_JOB_COMPLETED': - job_id =3D self.dictpath(event, 'data/device') - self.assertTrue(job_id in pending_jobs) - self.assert_qmp_absent(event, 'data/error') - pending_jobs.remove(job_id) - - self.assert_no_active_block_jobs() - self.vm.shutdown() - - # Check that all maps match now - for i in range(2, self.num_imgs, 2): - self.assertEqual(qemu_io('-f', iotests.imgfmt, '-c', 'map', se= lf.imgs[i]), - qemu_io('-f', iotests.imgfmt, '-c', 'map', se= lf.imgs[i-1]), - 'image file map does not match backing file a= fter streaming') - # Test that it's not possible to perform two block-stream # operations if there are nodes involved in both. def test_overlapping_1(self): self.assert_no_active_block_jobs() =20 # Set a speed limit to make sure that this job blocks the rest - result =3D self.vm.qmp('block-stream', device=3D'node4', job_id=3D= 'stream-node4', base=3Dself.imgs[1], speed=3D1024*1024) + result =3D self.vm.qmp('block-stream', device=3D'node4', + job_id=3D'stream-node4', base=3Dself.imgs[1], + filter_node_name=3D'stream-filter', speed=3D1= 024*1024) self.assert_qmp(result, 'return', {}) =20 result =3D self.vm.qmp('block-stream', device=3D'node5', job_id=3D= 'stream-node5', base=3Dself.imgs[2]) self.assert_qmp(result, 'error/desc', - "Node 'node4' is busy: block device is in use by block job: st= ream") + "Node 'stream-filter' is busy: block device is in use by block= job: stream") =20 result =3D self.vm.qmp('block-stream', device=3D'node3', job_id=3D= 'stream-node3', base=3Dself.imgs[2]) self.assert_qmp(result, 'error/desc', @@ -287,7 +247,7 @@ class TestParallelOps(iotests.QMPTestCase): # block-commit should also fail if it touches nodes used by the st= ream job result =3D self.vm.qmp('block-commit', device=3D'drive0', base=3Ds= elf.imgs[4], job_id=3D'commit-node4') self.assert_qmp(result, 'error/desc', - "Node 'node4' is busy: block device is in use by block job: st= ream") + "Node 'stream-filter' is busy: block device is in use by block= job: stream") =20 result =3D self.vm.qmp('block-commit', device=3D'drive0', base=3Ds= elf.imgs[1], top=3Dself.imgs[3], job_id=3D'commit-node1') self.assert_qmp(result, 'error/desc', diff --git a/tests/qemu-iotests/030.out b/tests/qemu-iotests/030.out index 6d9bee1..5eb508d 100644 --- a/tests/qemu-iotests/030.out +++ b/tests/qemu-iotests/030.out @@ -1,5 +1,5 @@ -........................... +.......................... ---------------------------------------------------------------------- -Ran 27 tests +Ran 26 tests =20 OK diff --git a/tests/qemu-iotests/245 b/tests/qemu-iotests/245 index b9399ef..c756cfe 100755 --- a/tests/qemu-iotests/245 +++ b/tests/qemu-iotests/245 @@ -898,17 +898,26 @@ class TestBlockdevReopen(iotests.QMPTestCase): # make hd1 read-only and block-stream requires it to be read-write # (Which error message appears depends on whether the stream job is # already done with copying at this point.) - self.reopen(opts, {}, + # As the COR-filter node is inserted into the backing chain with t= he + # 'block-stream' operation, we move the options to their proper no= des. + opts =3D hd_opts(1) + opts['backing'] =3D hd_opts(2) + opts['backing']['backing'] =3D None + self.reopen(opts, {'read-only': True}, ["Can't set node 'hd1' to r/o with copy-on-read enabled", "Cannot make block node read-only, there is a writer on it"]) =20 # We can't remove hd2 while the stream job is ongoing - opts['backing']['backing'] =3D None - self.reopen(opts, {'backing.read-only': False}, "Cannot change 'ba= cking' link from 'hd1' to 'hd2'") + opts['backing'] =3D None + self.reopen(opts, {'read-only': False}, + "Cannot change 'backing' link from 'hd1' to 'hd2'") =20 - # We can detach hd1 from hd0 because it doesn't affect the stream = job + # We can't detach hd1 from hd0 because there is the COR-filter imp= licit + # node in between. + opts =3D hd_opts(0) opts['backing'] =3D None - self.reopen(opts) + self.reopen(opts, {}, + "Cannot change backing link if 'hd0' has an implicit b= acking file") =20 self.vm.run_job('stream0', auto_finalize =3D False, auto_dismiss = =3D True) =20 --=20 1.8.3.1