From nobody Wed Nov 5 05:21:42 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1532900037715867.4356725813636; Sun, 29 Jul 2018 14:33:57 -0700 (PDT) Received: from localhost ([::1]:49692 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fjtK8-000659-I2 for importer@patchew.org; Sun, 29 Jul 2018 17:33:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33883) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fjtF2-00020J-7k for qemu-devel@nongnu.org; Sun, 29 Jul 2018 17:28:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fjtF0-0000XD-D8 for qemu-devel@nongnu.org; Sun, 29 Jul 2018 17:28:40 -0400 Received: from mail-sn1nam02on0111.outbound.protection.outlook.com ([104.47.36.111]:45370 helo=NAM02-SN1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fjtF0-0000W0-7K; Sun, 29 Jul 2018 17:28:38 -0400 Received: from Jupiter.local (141.226.29.227) by CY1PR07MB2617.namprd07.prod.outlook.com (2a01:111:e400:c637::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.995.19; Sun, 29 Jul 2018 21:28:30 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janustech.onmicrosoft.com; s=selector1-janustech-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WRgFzJsxSh7tzuci/flKSI6BKbDynhTPfKlIQm3TPnk=; b=N27hqlXQUjvAQlDV6bLOKkRWVudgVAnqSZVW/fvTkgdco91mh3GkEJQ+4CwM3dDnlnIxxKUfRPRBtPLWI8d0ZCMJ7H3qVgYaU2QlON92x8Cmu8tTX+iGIrTrDTwpAQTIi3nMwegMCiSUcKTSzKHBUnC1iB2CCYQpY/qMz+1uU8k= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=lbloch@janustech.com; From: Leonid Bloch To: qemu-devel@nongnu.org Date: Mon, 30 Jul 2018 00:27:43 +0300 Message-Id: <20180729212744.23709-6-lbloch@janustech.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180729212744.23709-1-lbloch@janustech.com> References: <20180729212744.23709-1-lbloch@janustech.com> MIME-Version: 1.0 X-Originating-IP: [141.226.29.227] X-ClientProxiedBy: HE1PR06CA0138.eurprd06.prod.outlook.com (2603:10a6:7:16::25) To CY1PR07MB2617.namprd07.prod.outlook.com (2a01:111:e400:c637::15) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5fd2a9ee-7514-4bde-8616-08d5f59a3bf4 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(7021125)(8989117)(5600074)(711020)(4534165)(7022125)(4603075)(4627221)(201702281549075)(8990107)(7048125)(7024125)(7027125)(7028125)(7023125)(2017052603328)(7153060)(7193020); SRVR:CY1PR07MB2617; X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2617; 3:8fjqfGZ+P4H5G85gNRvJQDqgGHDXy9Jsqbk8p+9f5QZFYPKRFqvDCfhUo0Epaj46RLRasniUB1IRrr1RsryTsO833wuzo/H6PrI7+Hkr75+rMgTGDayOwakFO7rra41Two5N5iNsk79XBCGTX+cGJxdRfZ13mQ7H1ih/LygJ5/ngef33O1f3pmhuKDtWy8bytvwebbmUqKJq4HnbWpliGXj5w7x6Tg1hUu6uYvmnMD5WSpsXJZX6+DrrM8JS1hbS; 25:qah0qqvNVV+STCbeEpvxM54NIZoUe7XmmKjgJ3m0OAidHBcirgCMJin6Q9WIJxxseofs8tMle/oPj9vBkZzj7gUlH2Y7pCpYRrGKAKVOXqJbCKyYJsk3oR3HPXNdwWCyeajCF285KjaTwNW7Uk/WP4dms4DQgiFzsoaaPwPwo+CpWTo9VBqCtaW9LQqSNoohfdPI5ohfws0GYhU+metKKqqnEqzqBuQvVQC73YvJnuHb3z6KbDiPS4CvB110i3zceq4EBE58gmxxVrExRHNWUOwE1v3fbo6ixO5uaF+UOV97JVnpOwp19cLoLCHo3Y3xppKNJL+bT9eanbyHOFo/Gw==; 31:Nx/E/zD7yPuCuJlscyqeOXOCD/XZkP5o4ZPZzKWnu9mppuAVVnaVS0NEvefSHKmW2XgEBI1rE5ZO5PMyGW6QuRe7glC+JSrkE11el3NXBBH8yW8l2JSryiNtI0/SgxhIYRpQFA6kPsUO+3qk8dtE2M8k+29NFLhgV6uylg+R4mpd+9KxuZCheFzdphQILniNs5oBy8JTKqMqhYG8g63w6uw9GIxK08oqXHnISRVmrZc= X-MS-TrafficTypeDiagnostic: CY1PR07MB2617: X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(3231311)(944501410)(52105095)(3002001)(10201501046)(149027)(150027)(6041310)(2016111802025)(20161123560045)(20161123564045)(20161123562045)(20161123558120)(6072148)(6043046)(201708071742011)(7699016); SRVR:CY1PR07MB2617; BCL:0; PCL:0; RULEID:; SRVR:CY1PR07MB2617; X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2617; 4:/oq0oRGWIpR96CK7ritQ1m/7eHZyQXXdbDod5rJtMySEG9yLwR3VTG2VlG/WG2kBNCjfCeSW+mji1Y2ZJbrjzsIrn5rqXorHpYhWNdGUjQAtAKdux3IWPTdY0F6aQ1doYLtjYuIjgZyGxBajEepV0YogVtqYGcv7rdBBJbX0RK22Nj4pS36vZkBmLpZSYel7JPC6lAJQ4rJrF0abpc/31Q/KDd/GuecWFGYJK0ga2wgu3gHDY6IwqzGXenAeViEXn8RWiET2UzZNMYEbUD34QQ== X-Forefront-PRVS: 0748FF9A04 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(136003)(39830400003)(366004)(346002)(376002)(396003)(199004)(189003)(3846002)(6116002)(4326008)(53936002)(51416003)(316002)(107886003)(52116002)(14444005)(26005)(486006)(81156014)(81166006)(8936002)(956004)(8676002)(16526019)(7736002)(186003)(6486002)(105586002)(476003)(305945005)(2616005)(6512007)(25786009)(478600001)(16586007)(66066001)(68736007)(47776003)(11346002)(54906003)(2906002)(2351001)(36756003)(106356001)(1076002)(76176011)(50466002)(5660300001)(6666003)(6506007)(446003)(386003)(2361001)(48376002)(97736004)(50226002)(6916009)(86362001); DIR:OUT; SFP:1102; SCL:1; SRVR:CY1PR07MB2617; H:Jupiter.local; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: None (protection.outlook.com: janustech.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR07MB2617; 23:0/7yi+5U2lHnwNWLrVVEGInaNzU5JodHqNR62mZZQ?= =?us-ascii?Q?3pgl8wx4DLeqDm9ry0GMzvphtRmGnBr9bRWMR/YrbOFpmR4IWAnJcX4fIwTF?= =?us-ascii?Q?c9dtB2jFjtOEE23TdlydEBaW7ILRIv2/F9fp+e7p9AANLQtQIBesRHqKyBzo?= =?us-ascii?Q?GMA7iiqyIRnMMxZjZxWcW0INs+A48eYPaDi2OgUCpnwlJd96IJyOtfbpZqp0?= =?us-ascii?Q?o/FBLtKmss3QB8pOfyP+tGN4hFlGscJUGsm4erDizn35s7L96/SImEb0J9TP?= =?us-ascii?Q?IhHl7tLIDrsi0Isb7njM9JExdSOeakLDML2oNb2KuHyv7aQqChRq42yjlkhj?= =?us-ascii?Q?QwDsRlLSwuEoGT9OeSqt/Efq5knrjDyGdvYAZObL80DOma7Hi+r7iNHdHAE1?= =?us-ascii?Q?GcXHqTHxIEIQ3yVDCM0n37vcIWvBh5zUYFGpeA5dt/SdEeTllOqoML9i36w/?= =?us-ascii?Q?ot2QfevHSLfAKn1EaxCHmPHinKxSeTLWBfPUWCKl+8357acKhq1RtrupkOWV?= =?us-ascii?Q?07sa3Cgz4+kEu2htfB3D7Iyp3XuPru6bHwmQu7pf9J5QaQdzT8o/E7YrxIT/?= =?us-ascii?Q?LZg1yOgE+ZceJVEKjp9q7A2Snqb79UrS3HQcDiT8zVbuMj23K8rrrAq6QWxQ?= =?us-ascii?Q?saeOZtLafqHx1xAQ84YsPx8svSg0Mpe292kisfA9FeUJ4H8Kj9L/b5XGOl9R?= =?us-ascii?Q?DRfxT1alz2cdFYz6j6Ay42WGEahZLHiLCZWElWK2MhYPLPr30fcIK6ugruq1?= =?us-ascii?Q?nAt8X10tTnJDgB1zQRpP0B1093auk+yKRcGWFWT0HMXj2gzLnL9Pl7/XyXAH?= =?us-ascii?Q?Y8vO9OlVigc1hbUN5fzi2fPHW/5iTfe3/fOEAlYvlkRx3dgnMYQ3cYMJViLk?= =?us-ascii?Q?5RTAvVv1MZ0Q8PeD9B49ovBSCAy1S4n0PjPrA1QWplWDXP3jGQxmLpxt/XYz?= =?us-ascii?Q?rUbIDZiHUxWgqor6WapYtaOa0+DhQEaJqQc4pJDEZAWWAWYWVinkjvLUMqWq?= =?us-ascii?Q?ByheoS/XkCBXylFnJWthaRt7s/Datmpx+r/51EtZM6TIoOcdommiqWqqy/Pb?= =?us-ascii?Q?Sh5GUQ0lwV2Ckyj8E/kbArre1RcWj2eaWsX/+Z1QlmJHiPCn2SajTi+rSbMz?= =?us-ascii?Q?BOYnRqAe4egip1sqqrHgPXMZ11MVrBOVClqe3+o7J72GexXwzerAgNhnI2F2?= =?us-ascii?Q?ut4GJOrVFGWZOD9uj2wbLTqTzweeYi/5UMiJ/XHvN4rlg+bnp8829pSUwAyG?= =?us-ascii?Q?tLRzCumb+LIrv43CqibuBZBRhZqnNYJhDxHg9KN/Jki/iZZsW810MKgt3T8x?= =?us-ascii?B?dz09?= X-Microsoft-Antispam-Message-Info: A/fir5DV59tnkMJMq2PMKsh/DOp75/WUAAV35tx5dAUxVkGRTWS4s+ho7kJQU5qesiCijk59aRAfvB+6y510Uc6SetdO9kp52mA2J1XoPefQzdMggVF++naCCdgjcA2K0piUcX7lRCC804LQRDbd3l+FlcW2Cc6So99zLfg+x9XqMewEnTOm60+OmuULQ4ixaan5apZdIwAluvcNOjmovU4mZbBLB6BCz7DQfkf/ze9yJe7+Qnkp4PbHjFKXdrzcaKoE3lxXUWsA2+KWX7i4anj9WqQNq+4pq2q7VdfChPl26buAtQWcGl5X19xdebf5T+Zg1FAblsT7nik5w0Z3xDhjJtuvMh5nF7D+DKlrE6E= X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2617; 6:FyAWSOpTEGRMWlkxwjNMnNtj8f0V29prC+khH8oPk4RBwQJxIQU/xmJgpV2w9563x+1s1pDw24DAJXzAnBDCc6/LMOIGSgjuIc+rvN4KgHgM0umwrWz5zQaX7SXIHCwn1FRGv9+bUAIwzD9DR/0BMUHVCOQ7xBUyy/8+b+aVPEHNooZm+acciBXnbQroapREdulVJh36OuUhsJeBZpY0no/8yhdzpTzLEeBJ1wCC0ndeYm2lYhAVELp2gPcB1WdnAnTdi7J+IB2kyYyCdH80UdcqaAl9hiP/j6pl6R83Mz3Xt3iiKz/MvmmKK55AlAqTJbVxcCAS1Ht6aCFOdZlkqwgRWJ1dOUa9oa0j1PgLhWoZRVHKvdjs4FfKtYta/T308Cmdg2RvCoJhUo+hk4ZMT6Urh3GBCCvS2LceP1kOObp/YPTxMwskYHbf95mQaIvY8rJh17txndxbNnI+asqgJg==; 5:0EYiAamaDoKJAbW0Rk5TDuR4bzmhyUzGdYJZJvafXmGISzIUtBOuTrL0jtxh9gfcELkQW6u74XboJlvLB6+IvzE/t17ZYEksDyfTsLYrRXW4QIUUhNKMjOxjBxYrs1GDhbQ9VZ7o/ZBwCLpRqHRqR+ofran2D3NXfFGvt9jNMZQ=; 7:qo1fqnnKFFleHTbBAcVkRGLGncy3QmG29Ksk4f/UiBUv9U1mEV4xowCy+0tkfh0dc8/mk1o4nENzZ1Us42Lrau+S6SYpQFadD2r+7haF6z5Z3VNp6QZuiMeRk2kfPGlaBJtjSbkAD494aackrNQDF7eh7kr6uvuNtmWlpHZJ5jmTTfKrszfeT3t4uzP4k0aYoBnP0e2OWQySOPiMlsbjWVVPob/bBmJ6S7D5qvDSYH0JAzZKTvOoDrwL0xI3bge8 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: janustech.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Jul 2018 21:28:30.2437 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5fd2a9ee-7514-4bde-8616-08d5f59a3bf4 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 34eb020e-3cf3-43a1-afeb-f5cf1310628e X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR07MB2617 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 104.47.36.111 Subject: [Qemu-devel] [PATCH 5/6] qcow2: Make the default L2 cache sufficient to cover the entire image X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Leonid Bloch , qemu-block@nongnu.org, Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Sufficient L2 cache can noticeably improve the performance when using large images with frequent I/O. The memory overhead is not significant, as the cache size is only 1 MB for each 8 GB of virtual image size (with the default cluster size of 64 KB). On systems with limited memory, one can limit the cache size by the l2-cache-size and cache-size options. Signed-off-by: Leonid Bloch --- block/qcow2.c | 10 ++++------ block/qcow2.h | 4 ---- docs/qcow2-cache.txt | 21 +++++++++++---------- qemu-options.hx | 6 +++--- 4 files changed, 18 insertions(+), 23 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 223d351e40..74f2cb10a4 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -793,6 +793,9 @@ static void read_cache_sizes(BlockDriverState *bs, Qemu= Opts *opts, *l2_cache_entry_size =3D qemu_opt_get_size( opts, QCOW2_OPT_L2_CACHE_ENTRY_SIZE, s->cluster_size); =20 + uint64_t virtual_disk_size =3D bs->total_sectors * BDRV_SECTOR_SIZE; + uint64_t max_l2_cache =3D virtual_disk_size / (s->cluster_size / 8); + if (combined_cache_size_set) { if (l2_cache_size_set && refcount_cache_size_set) { error_setg(errp, QCOW2_OPT_CACHE_SIZE ", " QCOW2_OPT_L2_CACHE_= SIZE @@ -814,9 +817,6 @@ static void read_cache_sizes(BlockDriverState *bs, Qemu= Opts *opts, } else if (refcount_cache_size_set) { *l2_cache_size =3D combined_cache_size - *refcount_cache_size; } else { - uint64_t virtual_disk_size =3D bs->total_sectors * BDRV_SECTOR= _SIZE; - uint64_t max_l2_cache =3D virtual_disk_size / (s->cluster_size= / 8); - /* Assign as much memory as possible to the L2 cache, and * use the remainder for the refcount cache */ if (combined_cache_size >=3D max_l2_cache + min_refcount_cache= ) { @@ -830,9 +830,7 @@ static void read_cache_sizes(BlockDriverState *bs, Qemu= Opts *opts, } } else { if (!l2_cache_size_set) { - *l2_cache_size =3D MAX(DEFAULT_L2_CACHE_BYTE_SIZE, - (uint64_t)DEFAULT_L2_CACHE_CLUSTERS - * s->cluster_size); + *l2_cache_size =3D max_l2_cache; } if (!refcount_cache_size_set) { *refcount_cache_size =3D min_refcount_cache; diff --git a/block/qcow2.h b/block/qcow2.h index 81b844e936..1b9005e13c 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -73,10 +73,6 @@ /* Must be at least 4 to cover all cases of refcount table growth */ #define MIN_REFCOUNT_CACHE_SIZE 4 /* clusters */ =20 -/* Whichever is more */ -#define DEFAULT_L2_CACHE_CLUSTERS 8 /* clusters */ -#define DEFAULT_L2_CACHE_BYTE_SIZE 1048576 /* bytes */ - #define DEFAULT_CLUSTER_SIZE 65536 =20 =20 diff --git a/docs/qcow2-cache.txt b/docs/qcow2-cache.txt index fd9a6911cc..bcc03c8857 100644 --- a/docs/qcow2-cache.txt +++ b/docs/qcow2-cache.txt @@ -97,12 +97,14 @@ need: l2_cache_size =3D disk_size_GB * 131072 refcount_cache_size =3D disk_size_GB * 32768 =20 -QEMU has a default L2 cache of 1MB (1048576 bytes) and a refcount -cache of 256KB (262144 bytes), so using the formulas we've just seen -we have +QEMU will use a default L2 cache sufficient to cover the entire virtual +size of an image, which with the default cluster size will result in 1MB +of cache for every 8GB of virtual image size: =20 - 1048576 / 131072 =3D 8 GB of virtual disk covered by that cache - 262144 / 32768 =3D 8 GB + 65536 / 8 =3D 8192 =3D 8 GB / 1 MB + +A default refcount cache is 4 times the cluster size, which defaults to +256KB (262144 bytes). =20 =20 How to configure the cache sizes @@ -121,8 +123,8 @@ There are a few things that need to be taken into accou= nt: - Both caches must have a size that is a multiple of the cluster size (or the cache entry size: see "Using smaller cache sizes" below). =20 - - The default L2 cache size is 8 clusters or 1MB (whichever is more), - and the minimum is 2 clusters (or 2 cache entries, see below). + - The default L2 cache size is enough to cover the entire virtual size of + an image, and the minimum is 2 clusters (or 2 cache entries, see below). =20 - The default (and minimum) refcount cache size is 4 clusters. =20 @@ -180,9 +182,8 @@ Some things to take into account: always uses the cluster size as the entry size. =20 - If the L2 cache is big enough to hold all of the image's L2 tables - (as explained in the "Choosing the right cache sizes" section - earlier in this document) then none of this is necessary and you - can omit the "l2-cache-entry-size" parameter altogether. + (the default behavior) then none of this is necessary and you can + omit the "l2-cache-entry-size" parameter altogether. =20 =20 Reducing the memory usage diff --git a/qemu-options.hx b/qemu-options.hx index 18f3f87da5..c9e7764c13 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -756,9 +756,9 @@ The maximal total size of the L2 table and refcount blo= ck caches in bytes =20 @item l2-cache-size The maximal size of the L2 table cache in bytes -(default: if cache-size is not defined - 1048576 bytes or 8 clusters, whic= hever -is larger; otherwise, as large as possible or needed within the cache-size, -while permitting the requested or the minimal refcount cache size) +(default: if cache-size is not defined - enough to cover the entire image; +otherwise, as large as possible or needed within the cache-size, while +permitting the requested or the minimal refcount cache size) =20 @item refcount-cache-size The maximal size of the refcount block cache in bytes --=20 2.17.1