From nobody Sun Feb 8 22:57:55 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=virtuozzo.com); dmarc=pass(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=2; a=rsa-sha256; t=1591035777; cv=pass; d=zohomail.com; s=zohoarc; b=iILJB22dM77Lznl/SWZeftldtwZFozmJpttOI4qhGfeukEKo08VnfI+fISyEqgyVibTTskgQI2g5PotYcZIg0jwq0tTd9ze7Bhv+LJFlNwT78TfNoZD4tfM4/sJTxNAok0ThZofFdLOxOKgFdyUMb1W2xIf9nthw4QB2LR2AvOE= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1591035777; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=iYkryG17z2qbi66UH5knwakQqhGFwVBWkY8Ljnt3fOQ=; b=T1ll1AVATNnusu7ZUlrhA24qk5LcpLS5SP+zFTcMiVgmOP66c8NKXLgC3rgN/8t2bQOzjiHjk7hd8C3ZgsQBVqe+pLjV6ZGelYxc9R1IOXG7m40jwJx60MjGv8g1oevlbCskOI11BqQ+4z423FqcWCJ+IvKWpVgs7q8NxV39yWQ= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=virtuozzo.com); dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1591035777809143.1524161476891; Mon, 1 Jun 2020 11:22:57 -0700 (PDT) Received: from localhost ([::1]:51082 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfp5M-0002HP-Ev for importer@patchew.org; Mon, 01 Jun 2020 14:22:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37716) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfoub-0005CA-Lu; Mon, 01 Jun 2020 14:11:49 -0400 Received: from mail-eopbgr70093.outbound.protection.outlook.com ([40.107.7.93]:39047 helo=EUR04-HE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfoua-00026c-Au; Mon, 01 Jun 2020 14:11:49 -0400 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM7PR08MB5317.eurprd08.prod.outlook.com (2603:10a6:20b:101::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19; Mon, 1 Jun 2020 18:11:41 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312%4]) with mapi id 15.20.3045.024; Mon, 1 Jun 2020 18:11:41 +0000 Received: from localhost.localdomain (185.215.60.170) by AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.18 via Frontend Transport; Mon, 1 Jun 2020 18:11:40 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KUtHHHgWe7wbYQckufP9irvLDMWUOIkYxj+qqdHaedF9NzYBueauLda22AjK39xX4OVFEahnTvittTO7P0nSwje8NBAjeN4MnnYbriNAzm2HOgddL/BL1TNJwrRyH1X6O5milcReO/Eyy5GxTggdoJfFfpNLbyoj/5fVu5iiz63RdicfSP0jbRsU1SJ4bfAV2r//vO6rVMKgV5Q3zew7vGqProSPqleBJ2yypHQuXP9+/qPLou8pldH9AXbl67vvGRfz6WR/hZx00uWsDiVqD49hPl+uTd7cjHHM8A6teD4u2fTyER7yQH6nID17TGgQD3lbSCiFmZ4EhkAWaYocJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iYkryG17z2qbi66UH5knwakQqhGFwVBWkY8Ljnt3fOQ=; b=haJHi3hFjl+b1wTnrEWzGFYAKCZfYJbZNHJBnNttYlNALvnoTWcv+kMOKlcdF3vX4L+75NGUEkcoP0QXOeYBEy4AHEnSsZfhRPehNaRlDZZ0UK8y2SvVeuRfr2kqYDl0m8Y1FxVvHGlT0laHy5+TalpeyX3EaWdTW73UjRLgsRNHtAm3RAWqXPb+ifRRfUCFLaHq/cMkVI4ipx98YYJUEJhURUYmfKOejEN/yXVdTPEseccUXJ5P9Z5UlC79I11RaL9KErfOWreVaqfow0yPW4bJg1RI/JTuWsW03XLLPnhmwrru/ULJUFLAbdORsjJBUr+YnQXJL89lxLIpeG5tUA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iYkryG17z2qbi66UH5knwakQqhGFwVBWkY8Ljnt3fOQ=; b=jkRzF8JrRmmMEgKwmgItseY4a9tvLdKO5XIxGEhJm0XZuBlWbnoMiBHBxILE909BbMJ4XrLXV+QUpCvbH82y5+BJfgXRyHb9sJvQ/0deCQHFBRMo17g4Tk7K/6khkqelThYBdA63vYC5ye08kd8xbZnvy8VfcgVIqoh2QRpc1fI= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=virtuozzo.com; From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v2 11/20] qapi: backup: add x-max-chunk and x-max-workers parameters Date: Mon, 1 Jun 2020 21:11:09 +0300 Message-Id: <20200601181118.579-12-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200601181118.579-1-vsementsov@virtuozzo.com> References: <20200601181118.579-1-vsementsov@virtuozzo.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-Mailer: git-send-email 2.21.0 X-Originating-IP: [185.215.60.170] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6a41eab4-ddbb-4391-858d-08d806573bc4 X-MS-TrafficTypeDiagnostic: AM7PR08MB5317: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:923; X-Forefront-PRVS: 0421BF7135 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: h36Xtz5eXxd7VQ57+trMrTmiCxacQrrvN/nvrxoUtSjcoLtEqLcRFd5+a9IfIWdhPU+6MsiJ5cGLr1k0/z1uZ0Vo/fyOdevVI/hkdqaeS3BuQ/HUD1V36oCZerDh2t75zwFVW15CwEuKhrDMmU2ViyLcCsVHfF7n51q/RXSpflekTLXUYTzBGijNvuFgRYbmsH9P2sfC7q6T0IpuZC3lYVvby0NRxk3K+fVmjl/CDTzSOBJNDdZoskxISLlikoKPdY08IAOe+ex50I8EbKPm2SPBcRP6KVYG5jx+FcHCwudADW/cftqRqVeWThU8UkDDEanw283GegoCp0l9XZKLpSiXW4FOT8sBys7CEkIyUPgrIv2BoskQHzz0p+HMSEkQ X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(376002)(39840400004)(346002)(366004)(186003)(16526019)(8936002)(2906002)(8676002)(26005)(107886003)(36756003)(316002)(6916009)(52116002)(2616005)(956004)(478600001)(4326008)(86362001)(6486002)(66946007)(66476007)(66556008)(6512007)(6666004)(1076003)(69590400007)(5660300002)(83380400001)(6506007); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: 8VOcVqeP9XlcCFqoVqJ6H44U04B1VlYlox+l05yL2nEeZBHUt/T8meQy9RpwN6D5c7Ldt43tE53Aa+tF2P587NVZAXotgKaBlt4M57IcCHUv6zDIEZfexeYMAuMAvUQ5LSgHryFbsH5PaRIIGWAqugnp3LBelsTG4eVkGAXRhuZFFFy+4l5NmMJbhcEhr90qDJFLsRPolIOpn5Dk7KVpSs7UZaW2HpWhJOqz9huLLJWYCmvfuZI5jnDbnfdnNsp/bzgOiazXUTpC9ljFSA0+KhthrURAOerrUbp506n4nMBoS33nuSsHf2rfzMm1GcvXMY1jrYpmKoPEaLPanJhOvI5enNaaNyOuWBMq2WXl5Ry7e+vHw31TGERoC4EOm2PQZNdbyWR6AEAjNI1F+bh+EVKaw3zPvrQG6Any0neTYRr9dSrHJPZSPYCH+kt67JQc5nr7DkDGalzgC+93OpOiG3Fq7r6tZ001mKj78XYdyYI= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6a41eab4-ddbb-4391-858d-08d806573bc4 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jun 2020 18:11:40.9065 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3s6M3gYW+sAh5ngMya3903RwwALQ95A3qeZ90+IliZ9D/CjpW7fGF1cXv08e5F0gsaojzgZMc0LeRMoI7eqF4fOInGJtE6JX6moENTobLBs= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5317 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=40.107.7.93; envelope-from=vsementsov@virtuozzo.com; helo=EUR04-HE1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 14:11:32 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @virtuozzo.com) Content-Type: text/plain; charset="utf-8" Add new parameters to configure future backup features. The patch doesn't introduce aio backup requests (so we actually have only one worker) neither requests larger than one cluster. Still, formally we satisfy these maximums anyway, so add the parameters now, to facilitate further patch which will really change backup job behavior. Options are added with x- prefixes, as the only use for them are some very conservative iotests which will be updated soon. Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 9 ++++++++- include/block/block_int.h | 7 +++++++ block/backup.c | 21 +++++++++++++++++++++ block/replication.c | 2 +- blockdev.c | 5 +++++ 5 files changed, 42 insertions(+), 2 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 0c7600e4ec..f4abcde34e 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1407,6 +1407,12 @@ # # @x-use-copy-range: use copy offloading if possible. Default true. (Since= 5.1) # +# @x-max-workers: maximum of parallel requests for static data backup. This +# doesn't influence copy-before-write operations. (Since: = 5.1) +# +# @x-max-chunk: maximum chunk length for static data backup. This doesn't +# influence copy-before-write operations. (Since: 5.1) +# # Note: @on-source-error and @on-target-error only affect background # I/O. If an error occurs during a guest write request, the device's # rerror/werror actions will be used. @@ -1421,7 +1427,8 @@ '*on-source-error': 'BlockdevOnError', '*on-target-error': 'BlockdevOnError', '*auto-finalize': 'bool', '*auto-dismiss': 'bool', - '*filter-node-name': 'str', '*x-use-copy-range': 'bool' } } + '*filter-node-name': 'str', '*x-use-copy-range': 'bool', + '*x-max-workers': 'int', '*x-max-chunk': 'int64' } } =20 ## # @DriveBackup: diff --git a/include/block/block_int.h b/include/block/block_int.h index 93b9b3bdc0..d93a170d37 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -1207,6 +1207,11 @@ void mirror_start(const char *job_id, BlockDriverSta= te *bs, * @sync_mode: What parts of the disk image should be copied to the destin= ation. * @sync_bitmap: The dirty bitmap if sync_mode is 'bitmap' or 'incremental' * @bitmap_mode: The bitmap synchronization policy to use. + * @max_workers: The limit for parallel requests for main backup loop. + * Must be >=3D 1. + * @max_chunk: The limit for one IO operation length in main backup loop. + * Must be not less than job cluster size or zero. Zero means = no + * specific limit. * @on_source_error: The action to take upon error reading from the source. * @on_target_error: The action to take upon error writing to the target. * @creation_flags: Flags that control the behavior of the Job lifetime. @@ -1226,6 +1231,8 @@ BlockJob *backup_job_create(const char *job_id, Block= DriverState *bs, bool compress, const char *filter_node_name, bool use_copy_range, + int max_workers, + int64_t max_chunk, BlockdevOnError on_source_error, BlockdevOnError on_target_error, int creation_flags, diff --git a/block/backup.c b/block/backup.c index 76847b4daf..ec2676abc2 100644 --- a/block/backup.c +++ b/block/backup.c @@ -46,6 +46,8 @@ typedef struct BackupBlockJob { uint64_t len; uint64_t bytes_read; int64_t cluster_size; + int max_workers; + int64_t max_chunk; =20 BlockCopyState *bcs; } BackupBlockJob; @@ -335,6 +337,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, bool compress, const char *filter_node_name, bool use_copy_range, + int max_workers, + int64_t max_chunk, BlockdevOnError on_source_error, BlockdevOnError on_target_error, int creation_flags, @@ -355,6 +359,16 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, assert(sync_mode !=3D MIRROR_SYNC_MODE_INCREMENTAL); assert(sync_bitmap || sync_mode !=3D MIRROR_SYNC_MODE_BITMAP); =20 + if (max_workers < 1) { + error_setg(errp, "At least one worker needed"); + return NULL; + } + + if (max_chunk < 0) { + error_setg(errp, "max-chunk is negative"); + return NULL; + } + if (bs =3D=3D target) { error_setg(errp, "Source and target cannot be the same"); return NULL; @@ -422,6 +436,11 @@ BlockJob *backup_job_create(const char *job_id, BlockD= riverState *bs, if (cluster_size < 0) { goto error; } + if (max_chunk && max_chunk < cluster_size) { + error_setg(errp, "Required max-chunk (%" PRIi64") is less than bac= kup " + "cluster size (%" PRIi64 ")", max_chunk, cluster_size); + return NULL; + } =20 /* * If source is in backing chain of target assume that target is going= to be @@ -465,6 +484,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDr= iverState *bs, job->bcs =3D bcs; job->cluster_size =3D cluster_size; job->len =3D len; + job->max_workers =3D max_workers; + job->max_chunk =3D max_chunk; =20 block_copy_set_progress_callback(bcs, backup_progress_bytes_callback, = job); block_copy_set_progress_meter(bcs, &job->common.job.progress); diff --git a/block/replication.c b/block/replication.c index 25987eab0f..a9ee82db80 100644 --- a/block/replication.c +++ b/block/replication.c @@ -563,7 +563,7 @@ static void replication_start(ReplicationState *rs, Rep= licationMode mode, s->backup_job =3D backup_job_create( NULL, s->secondary_disk->bs, s->hidden_dis= k->bs, 0, MIRROR_SYNC_MODE_NONE, NULL, 0, false, = NULL, - true, + true, 0, 0, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL, backup_job_completed, bs, NULL, &local_err= ); diff --git a/blockdev.c b/blockdev.c index 28145afe7d..cf068d20fa 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2743,6 +2743,9 @@ static BlockJob *do_backup_common(BackupCommon *backu= p, if (!backup->has_compress) { backup->compress =3D false; } + if (!backup->has_x_max_workers) { + backup->x_max_workers =3D 64; + } =20 if ((backup->sync =3D=3D MIRROR_SYNC_MODE_BITMAP) || (backup->sync =3D=3D MIRROR_SYNC_MODE_INCREMENTAL)) { @@ -2822,6 +2825,8 @@ static BlockJob *do_backup_common(BackupCommon *backu= p, backup->compress, backup->filter_node_name, backup->x_use_copy_range, + backup->x_max_workers, + backup->x_max_chunk, backup->on_source_error, backup->on_target_error, job_flags, NULL, NULL, txn, errp); --=20 2.21.0