From nobody Fri Apr 26 09:05:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1602999513; cv=none; d=zohomail.com; s=zohoarc; b=UPDjUWX4gvdHuikwrrvCUU8N8RJdcWytTvZXw56UFAoNQGRjS+z39L9DRVAs90T52hxzDpN1rU4FSGgzW0hQz6i91jNfZBIGPSku7eCmloUEX6/UPwlmTokYi5d3ULc2gA738UdOeXoNTHMGiNPRcvoSi2H9JH+mgMTbBfyoTZs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1602999513; h=Content-Type:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=P6URVbQiKmqU10k8mvjYLP1+F8TjAzsWp76XvhqsuEA=; b=PPcW1QXdN6qwaxpxWfrEoNQyndrlZnOJBDWWAACJ/RlZQPrSDvPsSnvf9WmNZnGMBxzW7gwY/AXrR+ldDkAuYsxpBIOuRoAIEcg+7tT2dyid4DuymMYuza/LBwxFANbzePvb8aDH8L0NonJIYmbhmVCZENV2cry8y5n/Ocb1fhw= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1602999513652486.5007587623612; Sat, 17 Oct 2020 22:38:33 -0700 (PDT) Received: from localhost ([::1]:56754 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kU1Oq-0003TM-5q for importer@patchew.org; Sun, 18 Oct 2020 01:38:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34786) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kU1NE-0002YW-D7; Sun, 18 Oct 2020 01:36:52 -0400 Received: from szxga04-in.huawei.com ([45.249.212.190]:5181 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kU1NC-0004FZ-6b; Sun, 18 Oct 2020 01:36:52 -0400 Received: from DGGEMS408-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id C8E32EFBE5B79B732E85; Sun, 18 Oct 2020 13:36:43 +0800 (CST) Received: from DESKTOP-80C7KIU.china.huawei.com (10.174.187.210) by DGGEMS408-HUB.china.huawei.com (10.3.19.208) with Microsoft SMTP Server id 14.3.487.0; Sun, 18 Oct 2020 13:36:33 +0800 From: Zhengui li To: , , , Subject: [PATCH] qemu-img: add support for rate limit in qemu-img convert Date: Sun, 18 Oct 2020 05:36:30 +0000 Message-ID: <1602999390-21324-1-git-send-email-lizhengui@huawei.com> X-Mailer: git-send-email 2.6.4.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.174.187.210] X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.190; envelope-from=lizhengui@huawei.com; helo=huawei.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/18 01:36:44 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: xieyingtai@huawei.com, lizhengui@huawei.com, qemu-devel@nongnu.org, qemu-block@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zhengui Currently, there is no rate limit for qemu-img convert. This may cause the task of qemu-img convert to consume all the bandwidth of the storage. This will affect the IO performance of other processes and virtual machines under shared storage. So we add support for offline rate limit in qemu-img convert to get better quality of sevice. Signed-off-by: Zhengui --- qemu-img-cmds.hx | 4 ++-- qemu-img.c | 35 ++++++++++++++++++++++++++++++++++- 2 files changed, 36 insertions(+), 3 deletions(-) diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx index ed55b76..70c0bf7 100644 --- a/qemu-img-cmds.hx +++ b/qemu-img-cmds.hx @@ -46,9 +46,9 @@ SRST ERST =20 DEF("convert", img_convert, - "convert [--object objectdef] [--image-opts] [--target-image-opts] [--= target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cach= e] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapsh= ot_param] [-S sparse_size] [-m num_coroutines] [-W] [--salvage] filename [f= ilename2 [...]] output_filename") + "convert [--object objectdef] [--image-opts] [--target-image-opts] [--= target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cach= e] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapsh= ot_param] [-S sparse_size] [-r rate_limit] [-m num_coroutines] [-W] [--salv= age] filename [filename2 [...]] output_filename") SRST -.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-op= ts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [= -t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l= SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] [--salvage] FILE= NAME [FILENAME2 [...]] OUTPUT_FILENAME +.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-op= ts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [= -t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l= SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-r RATE_LIMIT] [-m NUM_COROUTINES] [-W] = [--salvage] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME ERST =20 DEF("create", img_create, diff --git a/qemu-img.c b/qemu-img.c index 67c83b47..d27d8a6 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -50,6 +50,8 @@ #include "block/qapi.h" #include "crypto/init.h" #include "trace/control.h" +#include "qemu/throttle.h" +#include "block/throttle-groups.h" =20 #define QEMU_IMG_VERSION "qemu-img version " QEMU_FULL_VERSION \ "\n" QEMU_COPYRIGHT "\n" @@ -1672,6 +1674,7 @@ enum ImgConvertBlockStatus { }; =20 #define MAX_COROUTINES 16 +#define CONVERT_THROTTLE_GROUP "img_convert" =20 typedef struct ImgConvertState { BlockBackend **src; @@ -2187,6 +2190,17 @@ static int convert_copy_bitmaps(BlockDriverState *sr= c, BlockDriverState *dst) =20 #define MAX_BUF_SECTORS 32768 =20 +static void set_rate_limit(BlockBackend *blk, int64_t rate_limit) +{ + ThrottleConfig cfg; + + throttle_config_init(&cfg); + cfg.buckets[THROTTLE_BPS_WRITE].avg =3D rate_limit; + + blk_io_limits_enable(blk, CONVERT_THROTTLE_GROUP); + blk_set_io_limits(blk, &cfg); +} + static int img_convert(int argc, char **argv) { int c, bs_i, flags, src_flags =3D 0; @@ -2207,6 +2221,7 @@ static int img_convert(int argc, char **argv) bool force_share =3D false; bool explict_min_sparse =3D false; bool bitmaps =3D false; + int64_t rate_limit =3D 0; =20 ImgConvertState s =3D (ImgConvertState) { /* Need at least 4k of zeros for sparse detection */ @@ -2229,7 +2244,7 @@ static int img_convert(int argc, char **argv) {"bitmaps", no_argument, 0, OPTION_BITMAPS}, {0, 0, 0, 0} }; - c =3D getopt_long(argc, argv, ":hf:O:B:Cco:l:S:pt:T:qnm:WU", + c =3D getopt_long(argc, argv, ":hf:O:B:Cco:l:S:pt:T:qnm:WUr:", long_options, NULL); if (c =3D=3D -1) { break; @@ -2326,6 +2341,16 @@ static int img_convert(int argc, char **argv) case 'U': force_share =3D true; break; + case 'r': { + char *end; + unsigned long long sval =3D strtoull(optarg, &end, 10); + if (*end !=3D '\0' || sval > UINT_MAX) { + error_report("rate limit parse failed"); + ret =3D -1; + goto fail_getopt; + } + rate_limit =3D (int64_t)sval * 1024 * 1024; + } break; case OPTION_OBJECT: { QemuOpts *object_opts; object_opts =3D qemu_opts_parse_noisily(&qemu_object_opts, @@ -2715,6 +2740,10 @@ static int img_convert(int argc, char **argv) s.cluster_sectors =3D bdi.cluster_size / BDRV_SECTOR_SIZE; } =20 + if (rate_limit) { + set_rate_limit(s.target, rate_limit); + } + ret =3D convert_do_copy(&s); =20 /* Now copy the bitmaps */ @@ -2730,6 +2759,10 @@ out: qemu_opts_del(opts); qemu_opts_free(create_opts); qemu_opts_del(sn_opts); + if (s.target && rate_limit && + blk_get_public(s.target)->throttle_group_member.throttle_state) { + blk_io_limits_disable(s.target); + } qobject_unref(open_opts); blk_unref(s.target); if (s.src) { --=20 1.8.3.1