From nobody Sat Sep 26 21:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1787928468; cv=none; d=zohomail.com; s=zohoarc; b=LCAlS69ON+uZzOdP1MmZh1bNOKCKIoEhXG662UB05yuBb9Tcuo71m6obloPTG+BNVVJ73iD2n4JQUTaye0enYcycmC3gM0blyOgEfkctY8rqtcQChu4nKWOwZ3SuUWLoc2J9eptuJxdk8hoLn3Qx8/rDVL7dkx6r/U40Uq68gEc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1787928468; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=YEJDYv1NKzCxVU6hnQ6Czcdjfzbx6kLk6o9Vawi2MIY=; b=ntZo/FH4Z5UhHQ0W9/qQZUPaqWCzbTzBGm40z+S4VFm4PpreD/gF0BYXYjw9Cnc/a2O0bgJvCeUdKxMsYdWyCXZnsf7hkwziNlwi8JKbkz9wuD/DcM5f22p06Q5EIXhgQ4sw/t7ADEUJpBPrpZlHkdDX/7O8mUUKuzI1hWa16zo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1787928468613231.03858140703926; Fri, 28 Aug 2026 07:47:48 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wzxqI-0002fT-3u; Fri, 28 Aug 2026 10:46:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wzxqG-0002eD-FJ for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:04 -0400 Received: from mail-wm1-x329.google.com ([2a00:1450:4864:20::329]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wzxqD-0000dW-FH for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:04 -0400 Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-49b965570d7so7525985e9.0 for ; Fri, 28 Aug 2026 07:46:00 -0700 (PDT) Received: from athena.sw.ru ([2a06:5b06:b600:300:9458:da19:ab04:373a]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49b94dcaf61sm53730925e9.4.2026.08.28.07.45.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 07:45:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openvz.org; s=google; t=1787928359; x=1788533159; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=YEJDYv1NKzCxVU6hnQ6Czcdjfzbx6kLk6o9Vawi2MIY=; b=MUCFZ8kP0zNN93AlMdxDce6eHjSEbqbKyNOjAq1IYlNM9hTiWCADU6ROZr0gmwMMpe QiyvfgsH9hP+4JMf+heUVQPcd+37t4bmzij0g5vI5uBnoSe5ONSucXte7UudKgrlqXAN rO5/5uaG/vjX3/htRkjOKIyyD+ErA8Vnv3Ox+mEthHbQ7eJk5VyKOQtIWGQOOgJo5Idf RO9Qxi9tXuixdi9TOOfiVsWZqHmSZfXaMtUpmN5wZj7Yhex7ldlkAMVirMmf2eCo7j0T RBK/LID6ylQNH5JJj+aAVO1AB2US2H8T/g0j+PyjH6PXZBL5bC6OhJ1HEH8HGjmJ4wNn +mNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787928359; x=1788533159; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=YEJDYv1NKzCxVU6hnQ6Czcdjfzbx6kLk6o9Vawi2MIY=; b=AIR6WdlyJFLpOL7lWzNpQDMBWLnvzLXAMj4ZAmzyqL3KZ51F8aZxDRG9ma/YyMr3ew tnFLqR3JZ+jlUlj4/2vidzKRJYh5H0KfEHUrGad1lXcbKxMBXcuPc8LpVhMTLQD2oSmz aYktqBVyq2RsHCY1JJcWkmc5Aa++zqJE8pZiOeIY27F+haMbirucUPKClzLKfgRjqRjj vO0NuKjAo0cIYRvOijRkRD0Omq8J3AS3zdlkTZnLlj1cFTx2gwIXD7x0H9Smj7eMjvou i8Z0vgoYdwHatkTkuUfWfxTOPqrwRTS3+tACtNGl0VzMawsQxAToWhWXp0mfPxzkQEDf HfnQ== X-Gm-Message-State: AFuF++mawQ4IIVNx/y5bqzA2+ypA/5My3wkArvKxzvuCtDkh7xOirpGO hNbM6ippGiRU2tKTkMjc5qamgBtL/7KHyeY2/g5n7tNQ2Cv2kT3olKggvFcnn3Wd56Xlf46oHPI P/76Q X-Gm-Gg: AR+sD10H7L0BLYPH8lPdb2Xd8XQGXQ39E55fj7kuUF0YSIrdO6ubu+0rpjbS6WbB9cx qsbU6TtpR1A4TAvJQg2BaSuK99Gk60Cl0QgAeTSYFszDLOFhci2NKycAb740KOj1GLxccwY1DnM OIWLsErPZb213g966EQfrJqM95a5sxFye5sZNOpxnsi/eZM3dQpEKLvzrkcicn7PNS33f+7V9Ou WXWd+OdI7I8rERByZ/AganUohGbpBzB0Jw3njNz1rloz3+ygiBd2dK5HMxBdJJRPVjZMz+CiLT6 9kDy+ZRPCsexPOf8Edq7e5UtSwK2D6O4VzhiTiP2o84p7mB5MCpHIQuxFTDGuVmPhc7EzuIvI/M nFg8DDT5DDqnH01fpFA07QoxkQaOMfJz+1FoT3hfCjhGS3Kg1+E8F/Y2/fxL9WQqF+yqNb5ZfjV 6K8EdK4EBwGcmY/qI/s1Rv/ShDsm+bd9AxkbPaDdT313rpqznprcVXoaueouD5KRBD+KUJ X-Received: by 2002:a05:600c:3b28:b0:495:4749:16a7 with SMTP id 5b1f17b1804b1-49b91c56cdemr107416475e9.14.1787928359517; Fri, 28 Aug 2026 07:45:59 -0700 (PDT) From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Vladimir Sementsov-Ogievskiy , John Snow , Andrey Drobyshev , Eric Blake Subject: [PATCH 1/9] block/commit: pass BDRV_WANT_PRECISE to block-status Date: Fri, 28 Aug 2026 16:45:46 +0200 Message-ID: <20260828144554.1741910-2-den@openvz.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260828144554.1741910-1-den@openvz.org> References: <20260828144554.1741910-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::329; envelope-from=den@openvz.org; helo=mail-wm1-x329.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @openvz.org) X-ZM-MESSAGEID: 1787928471112154100 Content-Type: text/plain; charset="utf-8" From: Denis V. Lunev c33159dec790 replaced the want_zero bool with a bitmask of BDRV_WANT_* flags. block/commit.c was missed and still passes "true", which is BDRV_BLOCK_DATA, not BDRV_WANT_PRECISE. Convert it like the other callers. Fixes: c33159dec790 ("block: Expand block status mode from bool to flags") Signed-off-by: Denis V. Lunev CC: Vladimir Sementsov-Ogievskiy CC: John Snow CC: Andrey Drobyshev CC: Eric Blake Reviewed-by: Andrey Drobyshev --- block/commit.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/commit.c b/block/commit.c index 2d52c39594..4e0b0f9029 100644 --- a/block/commit.c +++ b/block/commit.c @@ -140,8 +140,8 @@ commit_iteration(CommitBlockJob *s, int64_t offset, /* Copy if allocated above the base */ WITH_GRAPH_RDLOCK_GUARD() { ret =3D bdrv_co_common_block_status_above(blk_bs(s->top), - s->base_overlay, true, true, offset, COMMIT_BUFFER_SIZE, - &bytes, NULL, NULL, NULL); + s->base_overlay, true, BDRV_WANT_PRECISE, offset, + COMMIT_BUFFER_SIZE, &bytes, NULL, NULL, NULL); } =20 trace_commit_one_iteration(s, offset, bytes, ret); --=20 2.53.0 From nobody Sat Sep 26 21:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1787928466; cv=none; d=zohomail.com; s=zohoarc; b=PcZ7jC021IHJhBxpJEIFyhCnFenSvVL0K+ZlbUPIECUQIE6ixiBWIAlpm06fBH8AiveirK+Tp1aVFKxBRtoADgk00wpmRh8zkCtxyPvTsKp416rrpxJ+irf+gcy4y5nAlRpjNlVr1ajLT+CQAE8cZ1TWCFCgvtmrvuwfbeFrzy8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1787928466; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=sudiIuAM/uHqM1+yV/O4MXxcGiv04xIH8Dio7xR7rac=; b=lPwztdB0oBDJ67L2Qs73/y494aTxnuj0JQKZpY3S+mIQlM4rOut7w46z10dasIU91hfyiMtWosVHcUHAr76iW4nSBC4A/o+1vwTrIrPkXplXImoKHFJznRXTsOIe3tDe4l2mylW69EWWuVbTUr6MyKgaRq63pEjJmFh7or8gAx8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1787928466853872.4141834506747; Fri, 28 Aug 2026 07:47:46 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wzxqI-0002g3-JJ; Fri, 28 Aug 2026 10:46:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wzxqG-0002eT-Sn for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:04 -0400 Received: from mail-wm1-x331.google.com ([2a00:1450:4864:20::331]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wzxqE-0000eA-3Y for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:04 -0400 Received: by mail-wm1-x331.google.com with SMTP id 5b1f17b1804b1-49b392ccaacso12863765e9.2 for ; Fri, 28 Aug 2026 07:46:01 -0700 (PDT) Received: from athena.sw.ru ([2a06:5b06:b600:300:9458:da19:ab04:373a]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49b94dcaf61sm53730925e9.4.2026.08.28.07.45.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 07:46:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openvz.org; s=google; t=1787928361; x=1788533161; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=sudiIuAM/uHqM1+yV/O4MXxcGiv04xIH8Dio7xR7rac=; b=LjSeOduGqsY5tUq107bV/WFnpaSa62ubi03DfzTPvu0/ycXbyN2HYXBKBAvzBSccKf JzcFjxtDoETtRWxit+croHx2rShomLMo3crwh6SGzXJwHo+jiZXzbTOytokJDrt0eg9N jtG9n7bNHubf6bHI/KHRcg0fmUUPfGFAC8DV4TPGzpZdzTP95AZnfLSIEQh2JNdF8OMo W3mhsaoiPmOboGxpmj4kocpope8pQAVJq6FND6p2dRJL1x7yE441OLA7FWR8wTSX37Po /5bdvJObxz9+md7C2Yp7hfJcEe1EGjj4r2R2bDTQ/wVqJC3UVgctM9IjPzs2q3ULdpUa BhhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787928361; x=1788533161; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=sudiIuAM/uHqM1+yV/O4MXxcGiv04xIH8Dio7xR7rac=; b=EsPPYkQ7W8kgv/KvEwKzsQBE7JzlzeQzOqXXf/HXK/qcuf9W51zqX2YEHa96CMMoFf I0QaPvn18aSPOSpKfn8Wcm/zBDTKmruHJSkKVeErcvbFzbNL5eap+WU79Ykxz8j/S7El YIUUv8+RytdtIQqTTAhhPVkj6rykzRj14AjTvdWELnj8v9Pgv6XarIGsbKoqXsF+O4cC D9pPKJM74bigFpzrk8PjgzDrbVZxkF+6OcHznJniHjRzYDjs5NPazQ5cRPJuhxbZiO5o OnrNyX2XDLrfA2CO/yqlHury0797ltZr51pGV4RfA+wl5rfXUPgb8SkPqrl6rPbjTkT7 RDLQ== X-Gm-Message-State: AFuF++mdvlDN6F/nE6Swd9aY6wod0RQgbb9YFfUYYlRvyZLNGTpIPLVb JaLdXDq9qSp0TrHDEZgq4WN/7TFF86vE9pYj1f1VD8wRa1ZJOX0qGDCLW5LbayZ+Yi5SH5yOWRD N2N9w X-Gm-Gg: AR+sD13fJ8YVLWe6/36dAY7OWKMHAl0TkxlsLtjKLKCIRPSb0FHHS42AJUUe1l3TL3J jWi6WNu8IaFfwM2cQfeVgoUq5wUmk2zA90SlhaTgU2RWYPXstSbUsrroY2cd1hia2ZtQJ3kP5Iz JIjUVrBbw9T2BNPbRnfI1OBeGyEuINYLFwm1aeWzRY929+Us/Z/ptJjRlEVTrJM2+o/UnxFs8Bd OyuFj3LVgeiHtihL60y3TlwNVzeIskWxkY8r8TjPvy5/Zd9S312fY8X0/VeDLQSfklLDvdiwRg8 TBczANe8PXRmv0wskwIvoIGZSriLG+wLoMiDsoX2GdB2da57Wj3x+LqoabZFDAxh9alFedhEqCz s7/HhL3vbnTJ+I4XnEeqFQFqljYRSiYxTZLAsvxd6nItu1JqdsDh0Ai9cj8aOx3XT+zOVHGFqHW 7qB3IrvgAsjzHs2hy13KwyzgFST8F870KL7fmsjLd6Q39QNM6nWO3wvWZBCw== X-Received: by 2002:a05:600c:354f:b0:493:c47f:3c55 with SMTP id 5b1f17b1804b1-49b91c1ece1mr123144035e9.5.1787928360582; Fri, 28 Aug 2026 07:46:00 -0700 (PDT) From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Vladimir Sementsov-Ogievskiy , John Snow , Andrey Drobyshev Subject: [PATCH 2/9] iotests/040: cover large and fragmented commit runs Date: Fri, 28 Aug 2026 16:45:47 +0200 Message-ID: <20260828144554.1741910-3-den@openvz.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260828144554.1741910-1-den@openvz.org> References: <20260828144554.1741910-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::331; envelope-from=den@openvz.org; helo=mail-wm1-x331.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @openvz.org) X-ZM-MESSAGEID: 1787928469400154100 Content-Type: text/plain; charset="utf-8" From: Denis V. Lunev 040 never commits runs long enough to span more than one block-status query, so the answer that commit_iteration() is about to start caching goes untested. Add two cases on a base <- mid <- active chain, committing mid so the job takes the regular commit path rather than active commit, and compare base against a snapshot of mid taken before the commit. One case mixes multi-megabyte data and hole runs, the other fragments them down to single clusters with every transition off the 512K boundary. Signed-off-by: Denis V. Lunev CC: Vladimir Sementsov-Ogievskiy CC: John Snow CC: Andrey Drobyshev --- tests/qemu-iotests/040 | 95 +++++++++++++++++++++++++++++++++++++- tests/qemu-iotests/040.out | 4 +- 2 files changed, 96 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 index 5c18e413ec..57e32b99c1 100755 --- a/tests/qemu-iotests/040 +++ b/tests/qemu-iotests/040 @@ -25,13 +25,14 @@ import time import os import iotests -from iotests import qemu_img, qemu_io +from iotests import qemu_img, qemu_img_create, qemu_io, compare_images import struct import errno =20 backing_img =3D os.path.join(iotests.test_dir, 'backing.img') mid_img =3D os.path.join(iotests.test_dir, 'mid.img') test_img =3D os.path.join(iotests.test_dir, 'test.img') +reference_img =3D os.path.join(iotests.test_dir, 'reference.img') =20 class ImageCommitTestCase(iotests.QMPTestCase): '''Abstract base class for image commit test cases''' @@ -951,6 +952,98 @@ class TestCommitWithOverriddenBacking(iotests.QMPTestC= ase): self.vm.qmp('block-job-complete', device=3D'commit') self.vm.event_wait('BLOCK_JOB_COMPLETED') =20 +class TestCommitLargeRuns(iotests.QMPTestCase): + """Commit runs long enough to cross commit_iteration()'s cached span."= "" + + MB =3D 1024 * 1024 + CLUSTER =3D 64 * 1024 + + # Runs several COMMIT_BUFFER_SIZE (512K) chunks long, both kinds. + SIZE =3D 32 * MB + LAYOUT =3D [ + (0, 4 * MB, 'data'), + (4 * MB, 6 * MB, 'hole'), + (10 * MB, 4 * MB, 'data'), + (14 * MB, 6 * MB, 'hole'), + (20 * MB, 4 * MB, 'data'), + ] + + # The same, with every transition off the 512K boundary. + SIZE_FRAGMENTED =3D 384 * CLUSTER # 24M + LAYOUT_FRAGMENTED =3D [ + (0, 45 * CLUSTER, 'data'), + (100 * CLUSTER, CLUSTER, 'data'), # data island in a hole + (150 * CLUSTER, 80 * CLUSTER, 'data'), + # [230C, 231C) is a hole island in a data run + (231 * CLUSTER, 69 * CLUSTER, 'data'), + ] + + def setUp(self): + self.vm =3D iotests.VM() + self.vm.launch() + + def tearDown(self): + self.vm.shutdown() + for img in (backing_img, mid_img, test_img, reference_img): + if os.path.exists(img): + os.remove(img) + + def build_images(self, layout, size): + # A pattern of its own in base, so a misplaced cluster shows up. + qemu_img_create('-f', iotests.imgfmt, backing_img, str(size)) + qemu_io('-c', f'write -P 0x11 0 {size}', backing_img) + + qemu_img_create('-f', iotests.imgfmt, '-b', backing_img, '-F', + iotests.imgfmt, mid_img, str(size)) + for offset, length, kind in layout: + if kind =3D=3D 'data': + qemu_io('-c', f'write -P 0x22 {offset} {length}', mid_img) + + # What base must equal once mid is committed into it. + qemu_img('convert', '-f', iotests.imgfmt, '-O', iotests.imgfmt, + mid_img, reference_img) + + # An empty layer above mid, so top_node=3Dmid is not the active on= e. + qemu_img_create('-f', iotests.imgfmt, '-b', mid_img, '-F', + iotests.imgfmt, test_img, str(size)) + + self.vm.cmd('blockdev-add', { + 'node-name': 'base', + 'driver': iotests.imgfmt, + 'file': {'driver': 'file', 'filename': backing_img}, + }) + self.vm.cmd('blockdev-add', { + 'node-name': 'mid', + 'driver': iotests.imgfmt, + 'file': {'driver': 'file', 'filename': mid_img}, + 'backing': 'base', + }) + self.vm.cmd('blockdev-add', { + 'node-name': 'active', + 'driver': iotests.imgfmt, + 'file': {'driver': 'file', 'filename': test_img}, + 'backing': 'mid', + }) + + def commit_and_verify(self): + self.vm.cmd('block-commit', job_id=3D'commit0', device=3D'active', + top_node=3D'mid', base_node=3D'base') + self.wait_until_completed(drive=3D'commit0') + + self.vm.cmd('blockdev-del', node_name=3D'active') + self.vm.cmd('blockdev-del', node_name=3D'mid') + self.vm.cmd('blockdev-del', node_name=3D'base') + self.assertTrue(compare_images(reference_img, backing_img)) + + def test_commit_large_runs(self): + self.build_images(self.LAYOUT, self.SIZE) + self.commit_and_verify() + + def test_commit_fragmented_runs(self): + self.build_images(self.LAYOUT_FRAGMENTED, self.SIZE_FRAGMENTED) + self.commit_and_verify() + + if __name__ =3D=3D '__main__': iotests.main(supported_fmts=3D['qcow2', 'qed'], supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/040.out b/tests/qemu-iotests/040.out index 1bb1dc5f0e..d2e2a2d98f 100644 --- a/tests/qemu-iotests/040.out +++ b/tests/qemu-iotests/040.out @@ -1,5 +1,5 @@ -................................................................. +................................................................... ---------------------------------------------------------------------- -Ran 65 tests +Ran 67 tests =20 OK --=20 2.53.0 From nobody Sat Sep 26 21:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1787928427; cv=none; d=zohomail.com; s=zohoarc; b=Z+fTQwf7nD1faB5VZpL/C5xPfHPPAaqs7USaKef+INUxDnh5Mlr+U36+jb919EvlS45wzfAbn53RqTp2d4ZlJOE5yCZCE0BzYVxQYSfFghVWhXX3wzQHrDGQMZOYx+e9jtlYpWoL6900r4QAbrOBwibFGJm7DxAv5klMfj4icWU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1787928427; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=JAXlrjXZ6Fm9mPDZhDxHdudSyY1vaCDPY6fIxYTWzCs=; b=ar46mHHwk9EdCYpbP5z0PbR7zyiQPpJNMURzB0AOSL3q4UpO4z6TGfm5oUqT1NjFlX6bXqhWmNQowl3Y0LCVewJcJkv3HCw7duLYnpbn8did65Ut9JHxrFuiLJtYd2lxsQaBXUACERg4IvHnHnk+j84pR1UC8xyaNxM5bMgc3dU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1787928427784327.17632462901577; Fri, 28 Aug 2026 07:47:07 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wzxqJ-0002gP-57; Fri, 28 Aug 2026 10:46:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wzxqH-0002ec-1d for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:05 -0400 Received: from mail-wm1-x334.google.com ([2a00:1450:4864:20::334]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wzxqF-0000eW-7h for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:04 -0400 Received: by mail-wm1-x334.google.com with SMTP id 5b1f17b1804b1-4995b0343c1so9526075e9.3 for ; Fri, 28 Aug 2026 07:46:02 -0700 (PDT) Received: from athena.sw.ru ([2a06:5b06:b600:300:9458:da19:ab04:373a]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49b94dcaf61sm53730925e9.4.2026.08.28.07.46.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 07:46:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openvz.org; s=google; t=1787928362; x=1788533162; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=JAXlrjXZ6Fm9mPDZhDxHdudSyY1vaCDPY6fIxYTWzCs=; b=ZOaLYiITwSh0X94h/00WWPm+F3MhEcLxsj1DpOlP+w/5typzvFJOnmbJbUo2SfXebr ZCN3fYy7EUzHIIhXcJzvcgAV5qPzhA1mv65U4AYA5HDZ0uW5t8+M5FHFwWGNDx5ei/iu gwzXJtV+NdLS3CUui9hIH5X9b6F5iY7Nyyc012X8hepsJYtp5lH5rCjlnGkX6YxF4pSw h/KDQ0ugc0dOmwLfT2AUyjF+UlwZfozwfgl+4BPg3QwqzZuhwAEAdLmJgOzy/WOmPaDA vbaW8WZgTsAQWCP4wPJVTgsijr9z+F74bi5rjIpu393Un7Dyh0XUFxxxnZtgWq+HU2Jf aODg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787928362; x=1788533162; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=JAXlrjXZ6Fm9mPDZhDxHdudSyY1vaCDPY6fIxYTWzCs=; b=SNlkv4QI08VHvoI879jHlj9gK87jYXIq7D5L2AvdM5Oeq3Q7Gt0GYbdiaZAZtnYvT+ uRSQ91sLvzXrvzjPbTYr0GXnUYMgF2ykQXTe0aRqHK8cEi5sa4CWxDB6jp+ub5Fxq56J kBc4eRWp30AnWeXHSIx7YrlTd0rhyW5jtal3G+VVZq/VbgeiIenZVeNpYbxF87AJBTkU OojMCUzV/j0uYllBcXIRe9AgqCj5lEaaiz85ni+qEnB0B+Di4fw0t23cPNe7NjKNxjJH 0frvD47qfFT1tA3ZK3+vECXtLQrkJ9BLoIYRddZpny5WIHZf5Ntzw6dAhu7d1z7zEzHE DV1g== X-Gm-Message-State: AFuF++nPicO+sEqCYOxAQ93jhFmaCXWzmvQ9Eic2SnHT+3tViit7rggv sVzKIlL4Oy0DOvxUtAmwdyiCWOPKkGDsJ6I+I+wR991G/o8BHDRUNCTYk103Nb36g5s3zRLky85 6JBkW X-Gm-Gg: AR+sD12qzny3yXKu7bD7vEPRjzEd6pCc63mnqyOk8/Z501THa6uaujUojsSSeay0jgD idnvUw8snt2h14TZHq68wVsFrTxB1ANHefSKBw/Ayp5fxG58M5J4XzQNXHPoMWxCMWdYNkyiSU2 bQ2fnCM+RobJ9BEvlngE0ukCFkqfYONdEeSt9BCGfsj/DXge8JphcCGbsjUXE9w7sVTFhquWx1j etVJhSBnIQwTzOqdMbwrzXR5Nhb3PhRWDaKmA5nFZajlY5ghK/iEKRp/B3Y9aEYAunjAXiZrMSQ odNblgNUxkrF8dUPJ0g8MbT77KygmEFmmVJb/myeyja4whQrzIg1cpnsVcFqm0WCL7Li8Uk59+x ivO6psl6v9kLBBhNx/A1TGRITQ3T9ezS1d2rqdqbeQs66MldyZWICjfXpUEctujpZUmI3byzp4Y xCtKtVaQd4l1JtscHIt09+cIjdhhcCIw9AbWBV7kseYCU4mfR4OxFpWwM2+A== X-Received: by 2002:a05:600c:a48:b0:499:9eb8:a1d7 with SMTP id 5b1f17b1804b1-49b91c3b7b6mr93831295e9.9.1787928361703; Fri, 28 Aug 2026 07:46:01 -0700 (PDT) From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Vladimir Sementsov-Ogievskiy , John Snow , Andrey Drobyshev Subject: [PATCH 3/9] block/commit: batch block-status queries Date: Fri, 28 Aug 2026 16:45:48 +0200 Message-ID: <20260828144554.1741910-4-den@openvz.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260828144554.1741910-1-den@openvz.org> References: <20260828144554.1741910-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::334; envelope-from=den@openvz.org; helo=mail-wm1-x334.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @openvz.org) X-ZM-MESSAGEID: 1787928429120158500 Content-Type: text/plain; charset="utf-8" From: Denis V. Lunev commit_iteration() asks for block status COMMIT_BUFFER_SIZE (512K) at a time, so a long run above base pays one query per 512K for an answer the whole run shares. Query the remainder of the image instead and keep the answer in a CommitStatus owned by commit_run(). Nothing can write top above base_overlay while the job runs: BLK_PERM_CONSISTENT_READ is shared only from filtered_base downwards, and the job writes below base_overlay. Copying is still bounded by the read buffer. Zeroes need no buffer, so COMMIT_ZERO_CHUNK bounds them, keeping a cancel from waiting on a huge write-zeroes, and an unallocated span is crossed in one step. Signed-off-by: Denis V. Lunev CC: Vladimir Sementsov-Ogievskiy CC: John Snow CC: Andrey Drobyshev --- block/commit.c | 50 +++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 39 insertions(+), 11 deletions(-) diff --git a/block/commit.c b/block/commit.c index 4e0b0f9029..5aa3df31e1 100644 --- a/block/commit.c +++ b/block/commit.c @@ -31,8 +31,17 @@ enum { * contiguous regions of the image is efficient. */ COMMIT_BUFFER_SIZE =3D 512 * 1024, /* in bytes */ + COMMIT_ZERO_CHUNK =3D 256 * 1024 * 1024, /* in bytes */ }; =20 +/* Last block-status answer, covering [offset, end). Empty when equal. */ +typedef struct CommitStatus { + int64_t len; + int64_t offset; + int64_t end; + int ret; +} CommitStatus; + typedef struct CommitBlockJob { BlockJob common; BlockDriverState *commit_top_bs; @@ -130,26 +139,43 @@ static void commit_clean(Job *job) =20 static int coroutine_fn commit_iteration(CommitBlockJob *s, int64_t offset, - int64_t *requested_bytes, void *buf) + int64_t *requested_bytes, void *buf, CommitStatus *st) { BlockErrorAction action; - int64_t bytes =3D *requested_bytes; + int64_t bytes; int ret =3D 0; bool error_in_source =3D true; =20 - /* Copy if allocated above the base */ - WITH_GRAPH_RDLOCK_GUARD() { - ret =3D bdrv_co_common_block_status_above(blk_bs(s->top), - s->base_overlay, true, BDRV_WANT_PRECISE, offset, - COMMIT_BUFFER_SIZE, &bytes, NULL, NULL, NULL); + if (offset < st->offset || offset >=3D st->end) { + /* Copy if allocated above the base */ + WITH_GRAPH_RDLOCK_GUARD() { + ret =3D bdrv_co_common_block_status_above(blk_bs(s->top), + s->base_overlay, true, BDRV_WANT_PRECISE, offset, + st->len - offset, &bytes, NULL, NULL, NULL); + } + + if (ret < 0) { + trace_commit_one_iteration(s, offset, 0, ret); + goto fail; + } + + st->offset =3D offset; + st->end =3D offset + bytes; + st->ret =3D ret; } =20 - trace_commit_one_iteration(s, offset, bytes, ret); + ret =3D st->ret; + bytes =3D st->end - offset; =20 - if (ret < 0) { - goto fail; + /* An unallocated span costs no I/O, so it is crossed in one step. */ + if (ret & BDRV_BLOCK_ZERO) { + bytes =3D MIN(bytes, COMMIT_ZERO_CHUNK); + } else if (ret & BDRV_BLOCK_ALLOCATED) { + bytes =3D MIN(bytes, COMMIT_BUFFER_SIZE); } =20 + trace_commit_one_iteration(s, offset, bytes, ret); + if (ret & BDRV_BLOCK_ALLOCATED) { if (ret & BDRV_BLOCK_ZERO) { /* @@ -215,6 +241,7 @@ static int coroutine_fn commit_run(Job *job, Error **er= rp) int64_t n =3D 0; /* bytes */ QEMU_AUTO_VFREE void *buf =3D NULL; int64_t len, base_len; + CommitStatus st =3D { 0 }; =20 len =3D blk_co_getlength(s->top); if (len < 0) { @@ -235,6 +262,7 @@ static int coroutine_fn commit_run(Job *job, Error **er= rp) } =20 buf =3D blk_blockalign(s->top, COMMIT_BUFFER_SIZE); + st.len =3D len; =20 for (offset =3D 0; offset < len; offset +=3D n) { /* Note that even when no rate limit is applied we need to yield @@ -245,7 +273,7 @@ static int coroutine_fn commit_run(Job *job, Error **er= rp) break; } =20 - ret =3D commit_iteration(s, offset, &n, buf); + ret =3D commit_iteration(s, offset, &n, buf, &st); =20 if (ret < 0) { return ret; --=20 2.53.0 From nobody Sat Sep 26 21:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1787928424; cv=none; d=zohomail.com; s=zohoarc; b=JEcruGm130etXa2F5OPYPAbTPWbvKQASFixw4HY3VjcV6yIGG7hbEuGNGEk6zVBlcbU7SHkDA+7dYwU8Xqwf3nbPMoF9EW/PnxkyTkhrh524XlbhaGVlFizWVYGMdlSyovvqqc2cWdZOOSgbF++Oqu0mCdevwmuVlbsKaWmsTIM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1787928424; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ic8YSpl0sDXU/HqDuSw78ZZ5FsWNoX87Js49G50yPhs=; b=B3lbpyLyFo1V/LLqn9ffKBwHh+jynDISVyiCC4gdOopHZxpgIntgH205T4HTWT+1VFJfm9fF5jLsX8CxyTMJM+PDZp38BEBT73JK/fcyOKuhiu6eTxFat85k8NzQfKGx2Ns0z1oQ+LaZ2iIG4pBWyOQVmehKLK6UwWQwm6VDK/E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1787928424216835.5185989519435; Fri, 28 Aug 2026 07:47:04 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wzxqL-0002iF-No; Fri, 28 Aug 2026 10:46:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wzxqJ-0002go-Lt for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:07 -0400 Received: from mail-wm1-x330.google.com ([2a00:1450:4864:20::330]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wzxqG-0000fG-Q5 for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:07 -0400 Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-49b8be0409fso5703725e9.2 for ; Fri, 28 Aug 2026 07:46:04 -0700 (PDT) Received: from athena.sw.ru ([2a06:5b06:b600:300:9458:da19:ab04:373a]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49b94dcaf61sm53730925e9.4.2026.08.28.07.46.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 07:46:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openvz.org; s=google; t=1787928363; x=1788533163; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ic8YSpl0sDXU/HqDuSw78ZZ5FsWNoX87Js49G50yPhs=; b=Q+6HcWmiy9o2st3Trq0lfSEPBWAQuB5uZUrq9Y70ikDCEO8g8EqkRW8UG+QRLrU7/r 6oi3CbY8o3dr35fpwfMd2JokWKt+wS1XNRib0j9i458ABXOeBQ9XhvfDOaNXiB5rdpim bax+TCnyxGH9JQea4jySw+LUv43lw+5ofl9coiXo2RSz9DWX/Cmygfc5AKdn6kjlrI9R jKKFLr5ONJ9DqWlZ+ZsKmGy/WEkU9hTA4b5DJ3TPnPBLZcggaIeue2bxwesDv3ZawgM2 NHZYh+CFe74NEHnavqqUqyweFxUbpjZSExycT4WQasH3B8Bz9xL5SyRYNmFzcR6cCBGy +89w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787928363; x=1788533163; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=ic8YSpl0sDXU/HqDuSw78ZZ5FsWNoX87Js49G50yPhs=; b=NeLylop67g8pfYbhAG/rN8ieFYRNPtBajwfD6TNUAQmMNMWWJ6l/HVnBK1CqzPLJN4 oWnAYy7ZbGu3w5F2GMlyolnSi7sKNG6W80wtMpyzVzDxncEj/lQNeZWDeKpTn/exOJAy L7WFXrt8jLjvmTH36y4/tBGdCiR8XXRRGHL38VIsA8Y+KGAV845Fs3Ab26XzFTRCJUyf mP2uelDqL9AYrwFCge9OlcnMNXpVXFD6Tr3MN5UkWlnbcAKcosaqqsMOGMFr3SMYC8zI FMM8LRhxpoUW9jSGPLb9+fQzuTtIqzaQB2ve/wzzTOwWMVBI5CA7BRj30KWoJlzlikNK GL7g== X-Gm-Message-State: AFuF++nhSGqjYcpn759XGAFZkwjlLWGxXYiGIUsiufzZ0UvH6XYeeST6 ysRZo4X1Pl1UPNV3IDlmPGMYxGR7sb5fIxBnmbixi7XcLZTvUW1xh/ecFegapBSX69L/6m66s/p lT28N X-Gm-Gg: AR+sD10U77rvq9arUAzILQmtb1FyT079Ot9CfUaJ7ebHxZMQP9v/sMZJrcm0eta37Wf i7c25XJMioffb07aXufU78WB3Uo3loe83+ALRslh3GHD6dQ5JhFIdhfaj7J05mhTe5c4O/G1a+G dMN7yg7JTsqok/J1yTxNBjHtQb3Mro66nUZeGXmn6cLSxBnEKP0e4n/cmnPpjbnEjIK7yDSllyG 7QHg6GQSgauJHjSsmHYGvGY1I//ire+hIQTrcRv6Qcex2XK5cC41KORxitUGNCoVYHJfDMuYUtc 0BPww3k85vFd4ZdCvq7V5sx+VaXMBCCnUXBNEzEOIHSAFZdaolpmAez/zRWi9dR1VhTr/elw8FW malQ7YUCps2af7Ug4+6cttymQjw8e9u/gSkVCT+yvGO20emLnwzPZ2llPbS9+jhYX5M5tx1jmC5 Frk9IpVB4NQejZrYLj9/XTAJbjdvbvTmEFXFrAehR/7QCZfYnlEkmdIFaBlg== X-Received: by 2002:a05:600c:5288:b0:499:84fe:5f3e with SMTP id 5b1f17b1804b1-49b91c41014mr122321075e9.9.1787928363074; Fri, 28 Aug 2026 07:46:03 -0700 (PDT) From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Vladimir Sementsov-Ogievskiy , John Snow , Andrey Drobyshev Subject: [PATCH 4/9] iotests/124: cover backup of zero clusters and holes Date: Fri, 28 Aug 2026 16:45:49 +0200 Message-ID: <20260828144554.1741910-5-den@openvz.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260828144554.1741910-1-den@openvz.org> References: <20260828144554.1741910-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::330; envelope-from=den@openvz.org; helo=mail-wm1-x330.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @openvz.org) X-ZM-MESSAGEID: 1787928427525154100 Content-Type: text/plain; charset="utf-8" From: Denis V. Lunev The backup tests compare content only, and their sources hold data alone, so nothing pins how backup treats zero clusters and holes. Back up a source mixing data, write-zero and holes with sync=3Dfull, sync=3Dtop and sync=3Dbitmap, and check the target with qemu-img map, which tells a copied cluster apart from a write-zero and from an untouched one. Signed-off-by: Denis V. Lunev CC: Vladimir Sementsov-Ogievskiy CC: John Snow CC: Andrey Drobyshev --- tests/qemu-iotests/124 | 402 ++++++++++++++++++++++++++++++++++++- tests/qemu-iotests/124.out | 4 +- 2 files changed, 403 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124 index b2f4328e34..eb5ab02904 100755 --- a/tests/qemu-iotests/124 +++ b/tests/qemu-iotests/124 @@ -22,8 +22,11 @@ # =20 import os +from collections import namedtuple + import iotests -from iotests import try_remove +from iotests import (compare_images, qemu_img_create, qemu_img_map, qemu_i= o, + try_remove) from qemu.qmp.qmp_client import ExecuteError =20 =20 @@ -748,6 +751,403 @@ class TestIncrementalBackupBlkdebug(TestIncrementalBa= ckupBase): self.check_backups() =20 =20 +# Backup of sources mixing data, zero clusters and holes. + +Extent =3D namedtuple('Extent', ['start', 'length', 'kind']) + +source_img =3D os.path.join(iotests.test_dir, 'source') +base_img =3D os.path.join(iotests.test_dir, 'base') +target_img =3D os.path.join(iotests.test_dir, 'target') + +SIZE =3D 64 * 1024 * 1024 +MB =3D 1024 * 1024 +CLUSTER =3D 64 * 1024 + +# Data, write-zero and holes, every combination at a cluster boundary. +LAYOUT =3D [ + (0, MB, 'data'), + (2 * MB, MB, 'zero'), + (5 * MB, 2 * MB, 'data'), + (8 * MB, MB, 'zero'), + (10 * MB, MB // 2, 'data'), + (20 * MB, 4 * MB, 'zero'), + (30 * MB, MB, 'data'), +] + +# Single-cluster runs, and a zero run ending exactly at EOF. +BOUNDARY_SIZE =3D 16 * MB +LAYOUT_BOUNDARY =3D [ + (0, CLUSTER, 'zero'), + (CLUSTER, CLUSTER, 'data'), + (2 * CLUSTER, CLUSTER, 'zero'), + (3 * CLUSTER, CLUSTER, 'data'), + (4 * CLUSTER, 8 * MB, 'zero'), + (4 * CLUSTER + 8 * MB, CLUSTER, 'data'), + (BOUNDARY_SIZE - CLUSTER, CLUSTER, 'zero'), +] + +# A zero run past the old block_copy_chunk_size() 16M cap. +LARGE_ZERO_SIZE =3D 48 * MB +LAYOUT_LARGE_ZERO =3D [ + (0, MB, 'data'), + (4 * MB, 32 * MB, 'zero'), + (40 * MB, MB, 'data'), +] + +# Image size not a multiple of the cluster size: a partial tail cluster. +TAIL_SIZE =3D 4 * MB + 4096 +LAYOUT_TAIL_ZERO =3D [ + (0, MB, 'data'), + (2 * MB, TAIL_SIZE - 2 * MB, 'zero'), +] +LAYOUT_TAIL_DATA =3D [ + (0, MB, 'data'), + (2 * MB, MB, 'zero'), + (4 * MB, TAIL_SIZE - 4 * MB, 'data'), +] + + +def coalesce(extents): + out =3D [] + for e in extents: + prev =3D out[-1] if out else None + adjacent =3D prev is not None and prev.start + prev.length =3D=3D = e.start + if adjacent and prev.kind =3D=3D e.kind: + out[-1] =3D prev._replace(length=3Dprev.length + e.length) + else: + out.append(e) + return out + + +def layout_to_extents(layout, size, gap=3D'hole'): + extents =3D [] + pos =3D 0 + for offset, length, kind in layout: + if offset > pos: + extents.append(Extent(pos, offset - pos, gap)) + extents.append(Extent(offset, length, kind)) + pos =3D offset + length + if pos < size: + extents.append(Extent(pos, size - pos, gap)) + return coalesce(extents) + + +class TestBackupZeroClusters(iotests.QMPTestCase): + def setUp(self): + self.vm =3D iotests.VM() + self.vm.launch() + + def tearDown(self): + self.vm.shutdown() + for img in (source_img, base_img, target_img): + if os.path.exists(img): + os.remove(img) + + def hmp_write(self, drive, cmd): + res =3D self.vm.hmp_qemu_io(drive, cmd) + assert 'error' not in res['return'].lower(), res + + def assert_map(self, img, layout, size, backing=3DFalse, gap=3D'hole'): + # Content is not enough, pin the data/zero/hole split as well. + def classify(e): + is_hole =3D e['depth'] > 0 if backing else not e['present'] + return 'hole' if is_hole else ('zero' if e['zero'] else 'data') + + actual =3D coalesce([Extent(e['start'], e['length'], classify(e)) + for e in qemu_img_map(img)]) + + self.assertEqual(actual, layout_to_extents(layout, size, gap)) + + def add_source(self, layout=3DNone, size=3DSIZE, backing=3DNone): + if layout is None: + layout =3D LAYOUT + + create_args =3D ['-f', iotests.imgfmt] + if backing: + create_args +=3D ['-b', backing, '-F', iotests.imgfmt] + create_args +=3D [source_img, str(size)] + qemu_img_create(*create_args) + + self.vm.cmd('blockdev-add', { + 'node-name': 'src', + 'driver': iotests.imgfmt, + 'file': {'driver': 'file', 'filename': source_img}, + }) + + # Write through the node, so an attached bitmap sees it. + for offset, length, kind in layout: + opt =3D '-z' if kind =3D=3D 'zero' else '-P 0x5a' + self.hmp_write('src', f'write {opt} {offset} {length}') + + def do_backup(self, sync, target_backing=3DNone, size=3DSIZE, prefill= =3DNone, + **kwargs): + create_args =3D ['-f', iotests.imgfmt] + if target_backing: + create_args +=3D ['-b', target_backing, '-F', iotests.imgfmt] + create_args +=3D [target_img, str(size)] + qemu_img_create(*create_args) + + if prefill is not None: + # Not zero, so a skipped cluster is provably untouched. + qemu_io('-c', f'write -P {prefill} 0 {size}', target_img) + + self.vm.cmd('blockdev-add', { + 'node-name': 'target', + 'driver': iotests.imgfmt, + 'file': {'driver': 'file', 'filename': target_img}, + }) + + self.vm.cmd('blockdev-backup', device=3D'src', target=3D'target', + job_id=3D'bk0', sync=3Dsync, **kwargs) + self.wait_until_completed(drive=3D'bk0') + + self.vm.cmd('blockdev-del', node_name=3D'target') + self.vm.cmd('blockdev-del', node_name=3D'src') + + def test_full(self): + self.add_source() + self.do_backup('full') + self.assertTrue(compare_images(source_img, target_img)) + self.assert_map(target_img, LAYOUT, SIZE, gap=3D'zero') + + def test_full_zero_overwrite(self): + # full skips holes, so only check that zero overwrites prefill. + self.add_source(layout=3D[(2 * MB, MB, 'zero')]) + self.do_backup('full', prefill=3D0xcc) + qemu_io('-c', f'read -P 0 {2 * MB} {MB}', target_img) + + def test_bitmap(self): + self.add_source() + + # A new bitmap tracks nothing yet: dirty all, then re-apply. + self.vm.cmd('block-dirty-bitmap-add', node=3D'src', name=3D'bm0') + self.hmp_write('src', f'write -z 0 {SIZE}') + for offset, length, kind in LAYOUT: + opt =3D '-z' if kind =3D=3D 'zero' else '-P 0x5a' + self.hmp_write('src', f'write {opt} {offset} {length}') + + self.do_backup('bitmap', bitmap=3D'bm0', bitmap_mode=3D'never') + self.assertTrue(compare_images(source_img, target_img)) + + # The blanket write-zero above turned LAYOUT's gaps into zero. + self.assert_map(target_img, LAYOUT, SIZE, gap=3D'zero') + + def test_top(self): + # Non-zero backing data, so a hole and an explicit zero differ. + qemu_img_create('-f', iotests.imgfmt, base_img, str(SIZE)) + qemu_io('-c', f'write -P 0x33 0 {SIZE}', base_img) + + self.add_source(backing=3Dbase_img) + + self.do_backup('top', target_backing=3Dbase_img) + self.assertTrue(compare_images(source_img, target_img)) + self.assert_map(target_img, LAYOUT, SIZE, backing=3DTrue) + + def test_boundary_full(self): + self.add_source(layout=3DLAYOUT_BOUNDARY, size=3DBOUNDARY_SIZE) + self.do_backup('full', size=3DBOUNDARY_SIZE) + self.assertTrue(compare_images(source_img, target_img)) + self.assert_map(target_img, LAYOUT_BOUNDARY, BOUNDARY_SIZE, + gap=3D'zero') + + def test_boundary_bitmap(self): + self.add_source(layout=3DLAYOUT_BOUNDARY, size=3DBOUNDARY_SIZE) + + self.vm.cmd('block-dirty-bitmap-add', node=3D'src', name=3D'bm0') + self.hmp_write('src', f'write -z 0 {BOUNDARY_SIZE}') + for offset, length, kind in LAYOUT_BOUNDARY: + opt =3D '-z' if kind =3D=3D 'zero' else '-P 0x5a' + self.hmp_write('src', f'write {opt} {offset} {length}') + + self.do_backup('bitmap', bitmap=3D'bm0', size=3DBOUNDARY_SIZE, + bitmap_mode=3D'never') + self.assertTrue(compare_images(source_img, target_img)) + self.assert_map(target_img, LAYOUT_BOUNDARY, BOUNDARY_SIZE, gap=3D= 'zero') + + def test_large_zero_full(self): + self.add_source(layout=3DLAYOUT_LARGE_ZERO, size=3DLARGE_ZERO_SIZE) + self.do_backup('full', size=3DLARGE_ZERO_SIZE) + self.assertTrue(compare_images(source_img, target_img)) + self.assert_map(target_img, LAYOUT_LARGE_ZERO, LARGE_ZERO_SIZE, + gap=3D'zero') + + def test_large_zero_bitmap(self): + self.add_source(layout=3DLAYOUT_LARGE_ZERO, size=3DLARGE_ZERO_SIZE) + + self.vm.cmd('block-dirty-bitmap-add', node=3D'src', name=3D'bm0') + self.hmp_write('src', f'write -z 0 {LARGE_ZERO_SIZE}') + for offset, length, kind in LAYOUT_LARGE_ZERO: + opt =3D '-z' if kind =3D=3D 'zero' else '-P 0x5a' + self.hmp_write('src', f'write {opt} {offset} {length}') + + self.do_backup('bitmap', bitmap=3D'bm0', size=3DLARGE_ZERO_SIZE, + bitmap_mode=3D'never') + self.assertTrue(compare_images(source_img, target_img)) + self.assert_map(target_img, LAYOUT_LARGE_ZERO, LARGE_ZERO_SIZE, + gap=3D'zero') + + def test_huge_zero(self): + # One run past BLOCK_COPY_MAX_ZERO_CHUNK (256M), so it splits. + size =3D 768 * MB + self.add_source(layout=3D[(0, size, 'zero')], size=3Dsize) + self.do_backup('full', size=3Dsize) + self.assertTrue(compare_images(source_img, target_img)) + self.assert_map(target_img, [(0, size, 'zero')], size) + + def test_tail_data(self): + self.add_source(layout=3DLAYOUT_TAIL_DATA, size=3DTAIL_SIZE) + self.do_backup('full', size=3DTAIL_SIZE) + self.assertTrue(compare_images(source_img, target_img)) + self.assert_map(target_img, LAYOUT_TAIL_DATA, TAIL_SIZE, gap=3D'ze= ro') + + def test_tail_zero(self): + # The partial tail cluster falls out of zero_bitmap, so it copies. + self.add_source(layout=3DLAYOUT_TAIL_ZERO, size=3DTAIL_SIZE) + self.do_backup('full', size=3DTAIL_SIZE) + self.assertTrue(compare_images(source_img, target_img)) + + tail_cluster =3D (TAIL_SIZE // CLUSTER) * CLUSTER + expected =3D [ + (0, MB, 'data'), + (2 * MB, tail_cluster - 2 * MB, 'zero'), + (tail_cluster, TAIL_SIZE - tail_cluster, 'data'), + ] + self.assert_map(target_img, expected, TAIL_SIZE, gap=3D'zero') + + def test_tail_zero_bitmap(self): + # Same tail rounding as test_tail_zero, via the bitmap scan. + self.add_source(layout=3DLAYOUT_TAIL_ZERO, size=3DTAIL_SIZE) + + self.vm.cmd('block-dirty-bitmap-add', node=3D'src', name=3D'bm0') + self.hmp_write('src', f'write -z 0 {TAIL_SIZE}') + for offset, length, kind in LAYOUT_TAIL_ZERO: + opt =3D '-z' if kind =3D=3D 'zero' else '-P 0x5a' + self.hmp_write('src', f'write {opt} {offset} {length}') + + self.do_backup('bitmap', bitmap=3D'bm0', size=3DTAIL_SIZE, + bitmap_mode=3D'never') + self.assertTrue(compare_images(source_img, target_img)) + + tail_cluster =3D (TAIL_SIZE // CLUSTER) * CLUSTER + expected =3D [ + (0, MB, 'data'), + (2 * MB, tail_cluster - 2 * MB, 'zero'), + (tail_cluster, TAIL_SIZE - tail_cluster, 'data'), + ] + self.assert_map(target_img, expected, TAIL_SIZE, gap=3D'zero') + + def test_mixed_cluster(self): + # 4K source clusters, so content varies inside one 64K cluster. + size =3D 2 * CLUSTER + qemu_img_create('-f', iotests.imgfmt, '-o', 'cluster_size=3D4k', + source_img, str(size)) + self.vm.cmd('blockdev-add', { + 'node-name': 'src', + 'driver': iotests.imgfmt, + 'file': {'driver': 'file', 'filename': source_img}, + }) + + # Zero, data and a hole inside the first cluster. + self.hmp_write('src', 'write -z 0 8k') + self.hmp_write('src', 'write -P 0x5a 8k 8k') + + # The second cluster is uniformly zero. + self.hmp_write('src', f'write -z {CLUSTER} {CLUSTER}') + + self.do_backup('full', size=3Dsize) + self.assertTrue(compare_images(source_img, target_img)) + + # A mixed cluster must be data, or the data at [8k, 16k) is lost. + self.assert_map(target_img, [(0, CLUSTER, 'data'), + (CLUSTER, CLUSTER, 'zero')], size) + + def test_top_zero_broken(self): + # An overlay hole before an explicit-zero run: the zero prefix + # must stop at the hole, or the backing data under it is lost. + size =3D 2 * CLUSTER + qemu_img_create('-f', iotests.imgfmt, '-o', 'cluster_size=3D4k', + base_img, str(size)) + qemu_io('-c', f'write -P 0x33 0 {size}', base_img) + + qemu_img_create('-f', iotests.imgfmt, '-o', 'cluster_size=3D4k', + '-b', base_img, '-F', iotests.imgfmt, + source_img, str(size)) + self.vm.cmd('blockdev-add', { + 'node-name': 'src', + 'driver': iotests.imgfmt, + 'file': {'driver': 'file', 'filename': source_img}, + }) + # [0, 4k) stays a hole, the zero run spills into cluster 1. + self.hmp_write('src', f'write -z 4096 {CLUSTER}') + + self.do_backup('top', target_backing=3Dbase_img, size=3Dsize) + self.assertTrue(compare_images(source_img, target_img)) + + def test_bitmap_straddle(self): + # One dirty run straddling a zero/data transition, both ways. + self.add_source(layout=3D[]) + self.vm.cmd('block-dirty-bitmap-add', node=3D'src', name=3D'bm0') + + # zero -> data straddle: one contiguous dirty run [7M, 9M) + self.hmp_write('src', f'write -z {7 * MB} {MB}') + self.hmp_write('src', f'write -P 0x5a {8 * MB} {MB}') + + # data -> zero straddle: one contiguous dirty run [15M, 17M) + self.hmp_write('src', f'write -P 0x5a {15 * MB} {MB}') + self.hmp_write('src', f'write -z {16 * MB} {MB}') + + self.do_backup('bitmap', bitmap=3D'bm0', bitmap_mode=3D'never') + self.assertTrue(compare_images(source_img, target_img)) + + def test_bitmap_fragmented(self): + # Isolated single-cluster dirty spots: nothing outside them + # may reach the target. + self.add_source(layout=3D[]) + + # Written before the bitmap exists, so untracked. + self.hmp_write('src', f'write -P 0x5a 0 {16 * MB}') + + self.vm.cmd('block-dirty-bitmap-add', node=3D'src', name=3D'bm0') + + # The only dirty bit in the data region. + data_island =3D 4 * MB + self.hmp_write('src', f'write -P 0x7b {data_island} {CLUSTER}') + + # The only dirty bit in the hole region. + zero_island =3D 24 * MB + self.hmp_write('src', f'write -z {zero_island} {CLUSTER}') + + self.do_backup('bitmap', bitmap=3D'bm0', bitmap_mode=3D'never', + prefill=3D0xcc) + + # Only the two isolated spots should have reached target ... + qemu_io('-c', f'read -P 0x7b {data_island} {CLUSTER}', target_img) + qemu_io('-c', f'read -P 0 {zero_island} {CLUSTER}', target_img) + + # ... everything else must still hold the prefill pattern. + qemu_io('-c', f'read -P 0xcc 0 {CLUSTER}', target_img) + qemu_io('-c', f'read -P 0xcc {8 * MB} {CLUSTER}', target_img) + + def test_bitmap_clean_zero_gap(self): + # A clean zero run between two dirty ones must stay prefilled. + self.add_source(layout=3D[], size=3D16 * MB) + self.hmp_write('src', f'write -z 0 {16 * MB}') + + self.vm.cmd('block-dirty-bitmap-add', node=3D'src', name=3D'bm0') + self.hmp_write('src', f'write -z {2 * MB} {MB}') + self.hmp_write('src', f'write -z {6 * MB} {MB}') + + self.do_backup('bitmap', bitmap=3D'bm0', bitmap_mode=3D'never', + size=3D16 * MB, prefill=3D0xcc) + + qemu_io('-c', f'read -P 0 {2 * MB} {MB}', target_img) + qemu_io('-c', f'read -P 0 {6 * MB} {MB}', target_img) + + qemu_io('-c', f'read -P 0xcc 0 {2 * MB}', target_img) + qemu_io('-c', f'read -P 0xcc {3 * MB} {3 * MB}', target_img) + qemu_io('-c', f'read -P 0xcc {7 * MB} {9 * MB}', target_img) + + if __name__ =3D=3D '__main__': iotests.main(supported_fmts=3D['qcow2'], supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/124.out b/tests/qemu-iotests/124.out index fa16b5ccef..5ce2f9a2ed 100644 --- a/tests/qemu-iotests/124.out +++ b/tests/qemu-iotests/124.out @@ -1,5 +1,5 @@ -............. +.............................. ---------------------------------------------------------------------- -Ran 13 tests +Ran 30 tests =20 OK --=20 2.53.0 From nobody Sat Sep 26 21:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1787928425; cv=none; d=zohomail.com; s=zohoarc; b=Z+rI/MqUkTuaeqAe3OuPkJJ5s1cVquXw06zBu+UOhFFtJ+V1HtOFSgCzbOr27wqsIz6MyiLVcdYMSlNpgfEkdtVda3LuWiEs4JsICU3oK1IjPKqxpmmJPD7athADIZ+e569olbGocR50C53X/SMMUdeWU+XNaKVlEiXMpmYzmqk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1787928425; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=SJFEI1lQ8SXWXqcCjUVxKX69y0HMHBq5YIZK1wiDeeQ=; b=SlxXpLlsJ8E61FwBTUYNMpFAATLaIQ4020BF8s/tOW9dB+UsXp/9zeZkdTFimcDwYOaQHvVO798XpOUrcNa1kHdoRAg8fx4poOLkUcYiXB57OVxmZ/wrsXkU6GS2VW8uygTL+MdwvpFDEOhOZuicBQIkvuHlsTB0sDJMcUaMTTI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1787928425251421.57615888552914; Fri, 28 Aug 2026 07:47:05 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wzxqM-0002iZ-Ip; Fri, 28 Aug 2026 10:46:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wzxqK-0002hc-Ln for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:08 -0400 Received: from mail-wm1-x32e.google.com ([2a00:1450:4864:20::32e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wzxqH-0000fy-Uj for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:08 -0400 Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-49b8687630fso7112925e9.3 for ; Fri, 28 Aug 2026 07:46:05 -0700 (PDT) Received: from athena.sw.ru ([2a06:5b06:b600:300:9458:da19:ab04:373a]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49b94dcaf61sm53730925e9.4.2026.08.28.07.46.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 07:46:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openvz.org; s=google; t=1787928364; x=1788533164; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=SJFEI1lQ8SXWXqcCjUVxKX69y0HMHBq5YIZK1wiDeeQ=; b=lNmrEW1U3fxUQ2CfZ3PLHqzniWwI6yQiE/6C+o3KjM26PCdrdmgeEhobZgOPhXh2vY NVZi0jbcPJnOvE8Se8QabcGNmykvLJX+NjyCFaQIxbmuBPHtn19L/FqTTMgfIZxOo6hR 0r/JoEI9xKbTQRe2UcWPlxde+7yFZKYfuWN0P8xJ/Yh6BlYX3bGqJIaVxc6wgIGsKsYq HPYeXPkd1SX/FY0Bz+4S6E/X+oXCnwAjSj0Aj/9ZOLGUTKABvuo++JeykyCfC15jGhpX vmNM8762pi37Thq8+lwoMKgjsalprnB5T4pkASS8XOyx2GwarA9p0ZukuUEvCuyKaV4f 2npQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787928364; x=1788533164; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=SJFEI1lQ8SXWXqcCjUVxKX69y0HMHBq5YIZK1wiDeeQ=; b=LG8xNNU/uWPRpfYWrrp2Qh9sOalDZd7s7vQRpqEC2xeqBe5Moifb44kLmkfgqLouCa tA2RcygTk67cofEFAhM/ZHPF/8gFswtb3i1jobh1jFP8JmSFJZ0N6j6YUzJN+gxShtHB 57ksyQyDJv7tlzCoRqCcE7wq2D4agTUnlB4Xhdamxy/wzGoBt2/gfT+W7vA9GGXXf604 f6Dttc94kSTrIIB55MQKrOF5tYvQ7XwefdaZly34bBwAGtSV3JgpXmLRuvHY0JGXyDrt HWN4cWBzfJGIJ3tjrv3oRMQEsQDQq+fQ1nc0+6GUo9IRF3xTw/51p39+LwluQwYivrd1 PQMQ== X-Gm-Message-State: AFuF++kxXf+kMdT4LlX6cHwjPpyvq7YmAPrDq2AJk6ulSW/o1ICMFEv6 RFlQzh/KZdKIEcZr8mjWtBN2Pmvhg53ow0iI4iQWl3WrQSn03gQHzLpgbWXlK1TqADt3sUp+kAD XtU6c X-Gm-Gg: AR+sD12KP52SFUPOLh2NbCib3sqnUYyA3YBWK/6WA3oh812gFUrET66z/1tGQFZy3oj N9ypRDTcqyCS+amblx62TzpenZijW9wNsH4txDCdTzJhtsub/brCee2LweekSJ2/JfoOKf6XbRl Z3V28Mtwl7rJOI8fLj1zzPKz5zZfl675qrBAHW3C2knd8SdDlcgGcx5FKbIrAdkBnd8GE3rV6e5 zTP54lIwNnXlc/nb9RtUBJMIkcYE3uJCyanYS6GG9poCMzKOlC6fift+qSJN5RGG8mfBGz51gME AUphtwJcmIhu28AxaFDXXpUDxhd1zSKvLzCuD1d0/kQODFSUhOMmfebZmBQ6QW2mWTlD6H/UlMS aABchp+X/5YQD2zOwpQMcnWB6D+GfFMhG47Hv5MG7eB5XrCekaxquOMbAmlYj6PuJmaSZt76yE+ MMxvDYvTPsE4XaQk1ANwGRUz7nbSZ/7ngIjVWnrbBx3dMwON/TzjYANMkVYg== X-Received: by 2002:a05:600c:6748:b0:499:b65d:124f with SMTP id 5b1f17b1804b1-49b91c57449mr123180105e9.11.1787928364171; Fri, 28 Aug 2026 07:46:04 -0700 (PDT) From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Andrey Drobyshev , Vladimir Sementsov-Ogievskiy , John Snow Subject: [PATCH 5/9] block/block-copy: don't reserve memory for zero tasks Date: Fri, 28 Aug 2026 16:45:50 +0200 Message-ID: <20260828144554.1741910-6-den@openvz.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260828144554.1741910-1-den@openvz.org> References: <20260828144554.1741910-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32e; envelope-from=den@openvz.org; helo=mail-wm1-x32e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @openvz.org) X-ZM-MESSAGEID: 1787928428981158500 Content-Type: text/plain; charset="utf-8" From: Denis V. Lunev block_copy_dirty_clusters() charges task->req.bytes to the shared memory pool for every task, but a COPY_WRITE_ZEROES task allocates no bounce buffer, so there is nothing to account for. Add block_copy_task_shres_bytes() and use it at all three call sites. Zero tasks lose the throttling the pool gave them incidentally and are bounded only by BLOCK_COPY_MAX_WORKERS (64). Reviewed-by: Andrey Drobyshev Signed-off-by: Denis V. Lunev CC: Vladimir Sementsov-Ogievskiy CC: John Snow CC: Andrey Drobyshev --- block/block-copy.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index 1826c2e1c7..21ebe8aec2 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -453,6 +453,11 @@ void block_copy_set_progress_meter(BlockCopyState *s, = ProgressMeter *pm) s->progress =3D pm; } =20 +static uint64_t block_copy_task_shres_bytes(BlockCopyTask *task) +{ + return task->method =3D=3D COPY_WRITE_ZEROES ? 0 : task->req.bytes; +} + /* * Takes ownership of @task * @@ -474,7 +479,7 @@ static coroutine_fn int block_copy_task_run(AioTaskPool= *pool, =20 aio_task_pool_wait_slot(pool); if (aio_task_pool_status(pool) < 0) { - co_put_to_shres(task->s->mem, task->req.bytes); + co_put_to_shres(task->s->mem, block_copy_task_shres_bytes(task)); block_copy_task_end(task, -ECANCELED); g_free(task); return -ECANCELED; @@ -605,7 +610,7 @@ static coroutine_fn int block_copy_task_entry(AioTask *= task) progress_work_done(s->progress, t->req.bytes); } } - co_put_to_shres(s->mem, t->req.bytes); + co_put_to_shres(s->mem, block_copy_task_shres_bytes(t)); block_copy_task_end(t, ret); =20 if (s->discard_source && ret =3D=3D 0) { @@ -816,7 +821,7 @@ block_copy_dirty_clusters(BlockCopyCallState *call_stat= e) =20 trace_block_copy_process(s, task->req.offset); =20 - co_get_from_shres(s->mem, task->req.bytes); + co_get_from_shres(s->mem, block_copy_task_shres_bytes(task)); =20 offset =3D task_end(task); bytes =3D end - offset; --=20 2.53.0 From nobody Sat Sep 26 21:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1787928423; cv=none; d=zohomail.com; s=zohoarc; b=KZR3iQyCpo/UxkUwVF4p1wj/BOCincBlBM+tT3X9aekDnjmz+O7V3In4amasNnjhWp67GaMAZ9n7bzfHjVm8Y4CHbjfDVgO4qvuDm6N4tgYbh1dvXp1NRFaBiVS2zRHuzhWq5xbHvGVt0JUUMqc9mEjHviDWTG/yqgYMcf2r9Ig= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1787928423; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=K0kBd+JRwRmYuSzj9Gu1odey8tK1kya/yjE5FN2wYZ8=; b=Kzq3MbVQlncftru1sCWueAcy9C5lTv3jVC676Sa7zpLNrsC/Lpqef4qB6mDesLpYX+teFJ7ax334Jga2vPzLWx+L2WYBnCqDi/U4zbudZ7suAUGYAgumizNrpDjr1g+KT8dRIrJzPZv3HCsGrjkHUpUjXGLtfYZcJ7L/swUByJA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1787928423675910.3433164544847; Fri, 28 Aug 2026 07:47:03 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wzxqN-0002jB-3h; Fri, 28 Aug 2026 10:46:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wzxqK-0002hv-UN for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:08 -0400 Received: from mail-wm1-x32c.google.com ([2a00:1450:4864:20::32c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wzxqJ-0000ga-27 for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:08 -0400 Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-49b8ce9b733so7781015e9.1 for ; Fri, 28 Aug 2026 07:46:06 -0700 (PDT) Received: from athena.sw.ru ([2a06:5b06:b600:300:9458:da19:ab04:373a]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49b94dcaf61sm53730925e9.4.2026.08.28.07.46.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 07:46:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openvz.org; s=google; t=1787928365; x=1788533165; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=K0kBd+JRwRmYuSzj9Gu1odey8tK1kya/yjE5FN2wYZ8=; b=JQiWw7NbiUearP3EXq27THpEbI7ce9HFIBKLl4oQS7BWLbCxx48vAfIVx7QmD05xL7 I45NaaCEwVJrdqLKe6U50maSBdp3c4eV6VlUGU+hP963TYtiAopBjZr7s7/2nn+a2QAL Se0PLX1XliVRd1mPdP8EabwFXVmtm5s69mUp2ezr6gfOI0e9bptQymao7F20+oMKZC1o 1I8cos8ogMJTCvBulluqfZn0aDcxQd/wKhReRUnsik0ZmbU/vfzmlVP6dwPHdyOF6rLB JEWHlGAqrEta6en+YEARmwzT7vGVIQ9oTgJ89rWV9wqo79cQY+Wn3SyWeM4w9EVlcmZV yH+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787928365; x=1788533165; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=K0kBd+JRwRmYuSzj9Gu1odey8tK1kya/yjE5FN2wYZ8=; b=gIfiHyxIByzp8AifPrxJHbld0KkGEV3YQgNwzIow23vLuyZG0jAacDHltvaeAFWW4J RtPhRzJJswSs1ofnlaEc/pmCaG5T7F6n+/kNhUnENRMuy1FanL8DHXfIQuF+1OWzjkYP tOWzHRBx6OEYKncnTQVQfPiJkQrQ9wFMOMSXEeGbqFiBD5bxvR+cQ9XdzmtE9G3cjrw7 n6XOdvoBnqyMA10DWwUKWA+VUNIWQSEKEvSjHLTpgUkkZPJ49OrQDWjv/WwZqJznH+cj gqqvYzjwucC5q0mo4evz3Zht9UzzgWgL/R3yZQjJMzILQrWNt7dMK+e4UKWiJCog/T12 HFOA== X-Gm-Message-State: AFuF++mgffIbIUqAHK/z04+NLxVEdDGkTnF3MJWJYSkrfc3AuPZwJ2mY bWccB5AYFPkCo6vAhfO2gXQ1UpNs33jclIM2kb1BXi5xPwvrFWeLXCFABYPe+Oqn6Z0GXLzYFNb Xo/fh X-Gm-Gg: AR+sD13pG377rYBIwKM9gGdoPz4FQp8HJ1oyENif5b90pRnAHRbkVT5QxkdA6Du5bw1 ntbbYoBykOe/RgnD3ZLPlcO6PGlWk3E1fFGvQZS8hX8XMqdhhbTicCXv7vUqSmMqrbafiespbd8 lyvIRAgcd5C5FQs1t06IJhbBh6v6M16s4GYyli0JGE2NhOVNgNXPt3xiYoy0qEiff0WlvfQKLgD HN6DlCWkGbpEUJuzjdkf67pKb/A5PClR1lm27e2dSwbZwPW2vQNL7w5aUNua31H30cwyOV4yA1D /KNMJ+zBobD/YMNXBSaqnNp48tr0nHiMdAxZBm7MBR3vhox16dN+SzEGaO4LRhIu0mWXoT8as0m y9BwBc2JeY627HPa5RS4tNoYm+R8uQfUguX34MMDeHWA/1uk76uA0Y45MYtl2tiYSVmusQMRZBi ceVD79niHX4zW3HJQf8kKOfBZ1vvbEeOT/DXIOoKaYDz22NysFWkt0i25VHQ== X-Received: by 2002:a05:600c:19c8:b0:49b:c8e:a211 with SMTP id 5b1f17b1804b1-49b91c20bfamr118453945e9.5.1787928365565; Fri, 28 Aug 2026 07:46:05 -0700 (PDT) From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Andrey Drobyshev , Vladimir Sementsov-Ogievskiy , John Snow Subject: [PATCH 6/9] block/block-copy: extract block_copy_set_task_method() Date: Fri, 28 Aug 2026 16:45:51 +0200 Message-ID: <20260828144554.1741910-7-den@openvz.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260828144554.1741910-1-den@openvz.org> References: <20260828144554.1741910-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32c; envelope-from=den@openvz.org; helo=mail-wm1-x32c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @openvz.org) X-ZM-MESSAGEID: 1787928427479154100 Content-Type: text/plain; charset="utf-8" From: Denis V. Lunev Move the per-task block-status query and its skip_unallocated and BDRV_BLOCK_ZERO handling out of block_copy_dirty_clusters() into block_copy_set_task_method(). No behavior change. The next patch changes that decision, and having it in one place keeps that change to just the new logic. The block-status result no longer lands in the caller's @ret, so drop the note saying @ret may be positive at the out: label. Reviewed-by: Andrey Drobyshev Signed-off-by: Denis V. Lunev CC: Vladimir Sementsov-Ogievskiy CC: John Snow CC: Andrey Drobyshev --- block/block-copy.c | 46 ++++++++++++++++++++++++++++++---------------- 1 file changed, 30 insertions(+), 16 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index 21ebe8aec2..d71d070dbd 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -741,6 +741,35 @@ int64_t coroutine_fn block_copy_reset_unallocated(Bloc= kCopyState *s, return ret; } =20 +/* + * Decide how @task is copied: COPY_WRITE_ZEROES if it reads as zero. May + * shrink @task. Returns false if @task is to be skipped (already ended, + * not freed). + */ +static bool coroutine_fn GRAPH_RDLOCK +block_copy_set_task_method(BlockCopyState *s, BlockCopyTask *task) +{ + int ret; + int64_t status_bytes; + + ret =3D block_copy_block_status(s, task->req.offset, task->req.bytes, + &status_bytes); + assert(ret >=3D 0); /* never fail */ + if (status_bytes < task->req.bytes) { + block_copy_task_shrink(task, status_bytes); + } + if (qatomic_read(&s->skip_unallocated) && !(ret & BDRV_BLOCK_ALLOCATED= )) { + block_copy_task_end(task, 0); + trace_block_copy_skip_range(s, task->req.offset, task->req.bytes); + return false; + } + if (ret & BDRV_BLOCK_ZERO) { + task->method =3D COPY_WRITE_ZEROES; + } + + return true; +} + /* * block_copy_dirty_clusters * @@ -773,7 +802,6 @@ block_copy_dirty_clusters(BlockCopyCallState *call_stat= e) while (bytes && aio_task_pool_status(aio) =3D=3D 0 && !qatomic_read(&call_state->cancelled)) { BlockCopyTask *task; - int64_t status_bytes; =20 task =3D block_copy_task_create(s, call_state, offset, bytes); if (!task) { @@ -787,24 +815,12 @@ block_copy_dirty_clusters(BlockCopyCallState *call_st= ate) =20 found_dirty =3D true; =20 - ret =3D block_copy_block_status(s, task->req.offset, task->req.byt= es, - &status_bytes); - assert(ret >=3D 0); /* never fail */ - if (status_bytes < task->req.bytes) { - block_copy_task_shrink(task, status_bytes); - } - if (qatomic_read(&s->skip_unallocated) && - !(ret & BDRV_BLOCK_ALLOCATED)) { - block_copy_task_end(task, 0); - trace_block_copy_skip_range(s, task->req.offset, task->req.byt= es); + if (!block_copy_set_task_method(s, task)) { offset =3D task_end(task); bytes =3D end - offset; g_free(task); continue; } - if (ret & BDRV_BLOCK_ZERO) { - task->method =3D COPY_WRITE_ZEROES; - } =20 if (!call_state->ignore_ratelimit) { uint64_t ns =3D ratelimit_calculate_delay(&s->rate_limit, 0); @@ -845,8 +861,6 @@ out: * block_copy_task_run. If it fails, it means some task already fa= iled * for real reason, let's return first failure. * Still, assert that we don't rewrite failure by success. - * - * Note: ret may be positive here because of block-status result. */ assert(ret >=3D 0 || aio_task_pool_status(aio) < 0); ret =3D aio_task_pool_status(aio); --=20 2.53.0 From nobody Sat Sep 26 21:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1787928450; cv=none; d=zohomail.com; s=zohoarc; b=iOocF4r2plyc/plT+CBgJQ52cN37MZGAtx3I9YRwvnSvrop6AVVJiTc2gg2aJkCOqNjNsyvaVxAGCiMHWboO3WnwrDDrjFr9Jl5wxqoI+oZgkkhjGDTll3lzhrAmqfaTL71ZpNkHYhgr/5zjmxOXr+ld/Nz7eGM7criXx1VLdMc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1787928450; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ZAQIX6p6+0pj7RUdjT3ml8ET9Xu5Dsntmwd7YXqtoog=; b=JvTz7yGFhwf2bGv/3mqXQ+0T+pBSUtQjxoyvdLtXfxhNJt1BSztTiXNe6hBOEGZQ5bEviVO/+PVqpyVcTdLr1y5z0wvsnht66mJXdcDeWrlTmTEn82y9XFcN8U4rygtjxMRvTf0SIkJYp+OEcNYFaAInPA10bB8WHB+zYY/Y3Ec= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1787928450959963.4070482800989; Fri, 28 Aug 2026 07:47:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wzxqP-0002kp-97; Fri, 28 Aug 2026 10:46:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wzxqN-0002j6-0R for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:11 -0400 Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wzxqK-0000hO-V2 for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:10 -0400 Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-49b8ce9b733so7781345e9.1 for ; Fri, 28 Aug 2026 07:46:08 -0700 (PDT) Received: from athena.sw.ru ([2a06:5b06:b600:300:9458:da19:ab04:373a]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49b94dcaf61sm53730925e9.4.2026.08.28.07.46.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 07:46:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openvz.org; s=google; t=1787928367; x=1788533167; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ZAQIX6p6+0pj7RUdjT3ml8ET9Xu5Dsntmwd7YXqtoog=; b=ef0bVd/dFXKU9GO/hrDpuqJlQ+lAg926sY/LEE5/fLi7toP/CPicSKFjmLMrcYONja hXVVOVFtGQ59lGlRwhscqW1AQ8u3LTHOlE7/rwOdoPNJrqHKsqbSRU/QHk+v+cnvtS2r yoohisuxfpQQ/EBjexlqArqkhTqnee/HKyEsKQoYZnom6L8pB2bDYjzqGqvk7TSvmZ5C I8vpA3vPqYeX0wPJncngXRnCb6ImzH9V8NQqUEFZLumE7pNHeBbBjEemYyZA9rGrZ//f gWjb3bqjDXst+Vhnz5gkX2fTBncebkL0nuIJtiI5BCw+Kwpn4NV5BsgnV/GFQ+PlefIP A3NA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787928367; x=1788533167; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=ZAQIX6p6+0pj7RUdjT3ml8ET9Xu5Dsntmwd7YXqtoog=; b=K49UKuv+DJpEuALMirW+S4z+5GLRQAJ978HoCBRd3zyhgSb0QWmvV+AbTOfi8zJf47 C0WgcqPlFnAIjRmxtvywZ35nAB/cOaJ6XfsQSniXxP+XBY3g2bs17qAleL3Nn8PaSF5s MJcz2whMNt57zBHoU2N2Ly3OZYzlD4SphyLYbpMsQoKCfYwzjKhRcM6v+d0pDPkZIRWF qyqzI7uWlBHAYLD51LExHZy51u2rG8PJraJwLlRFvODsEuqOJPlH1JePrlzfsZ7qgSbd OjbJ9nKrsYTQ1V2OAqcMMyxXcScFhou/y9IRyV/P9rDr8fsretsRb9BF2fIw9dDDWxBf Ayaw== X-Gm-Message-State: AFuF++nsOAJie0RLeflLaib5ynHK++eqbMENfnLWtmvUwCLnSCqg2qBw UqaEsHZpL+sQCfOpxQe83BC7JnnhAfRQqOVV3F7G1vGa39AZWQadWNjgCKHKRhRY+fKysHUYd1b Qi1y9 X-Gm-Gg: AR+sD12OlOpa7g9vFQ9KPC0ixPQNXZIxdKclkAYR2FimbarBmdP4auOXAjhY07ANlTS r+Y/Uqgd8gOjaMZHG9gLA43gphIubD0NLop4ZrfW6yb2aPKCMGw/2ep98NcK3nQVWcJ0/YwvqtM lLN8YcrSXiZmg1YaHa2RuopFmKLqaVwUKUqcVi7M/8DPX09rFULpNTBqRThDp4NcW/LGgwl6Vfs 9Xl1y1IcG3TPQBkKQNNaUH4GR9D8JR8Ml9wfgAgAoLjjMDsKL8CbHOxxQYhqznqvvEK5w4CUtxS mEguAOZyAXQK1vV1AW4QmHtt3mSmDQwjSleS+L+yPofqlYHbStzDZJGyWZ9TKhPm1MKnRoaLvqk YWyNLPCLfBsX8j22a/TQBx8PJL0pknOmkUn//QTDbztiwHIfbUUFeMMdo5BYNxVxKKHtHpO7W9T hdja6l2RnefuNEyeEiV2iarwfGTLqbfpdc5UwqBJN8MUTDJlKmVvOXU9Q8yQ== X-Received: by 2002:a05:600c:1d0c:b0:499:484a:81d0 with SMTP id 5b1f17b1804b1-49b91c3e67fmr96155895e9.9.1787928367326; Fri, 28 Aug 2026 07:46:07 -0700 (PDT) From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Vladimir Sementsov-Ogievskiy , John Snow , Andrey Drobyshev Subject: [PATCH 7/9] block/block-copy: track known-zero source clusters Date: Fri, 28 Aug 2026 16:45:52 +0200 Message-ID: <20260828144554.1741910-8-den@openvz.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260828144554.1741910-1-den@openvz.org> References: <20260828144554.1741910-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::333; envelope-from=den@openvz.org; helo=mail-wm1-x333.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @openvz.org) X-ZM-MESSAGEID: 1787928453423154100 Content-Type: text/plain; charset="utf-8" From: Denis V. Lunev block_copy_reset_unallocated()'s up-front scan already queries the source, and the same query reports zero-ness. The copy loop re-queries per task and, having no answer up front, can only size a task by the copy buffer, so a mostly-zero image is cut into thousands of pieces that each turn out to read as zero. Cache the answer in zero_bitmap, published by zero_bitmap_valid once the scan is done. block_copy_task_create() then picks the method from the bitmap and ends the task where the answer changes, so a zero task never reaches into data and a copy task never swallows a zero run. The per-task query stays as the fallback until the flag is set, since CBW intercepts guest writes while the scan is still running. The scan and the query it replaces must resolve BDRV_BLOCK_ZERO against the same part of the chain, so the scan moves from bdrv_co_is_allocated() to bdrv_co_block_status_above() and the base selection is factored out into block_copy_status_base(). zero_bitmap is a plain HBitmap, not a BdrvDirtyBitmap: an internal cache has no business in query-named-block-nodes. One writer and readers gated by the flag need no mutex, but the publish needs ordering, hence store-release and load-acquire. It is allocated on demand, so sync=3Dnone and a standalone copy-before-write filter do not pay for it. Signed-off-by: Denis V. Lunev CC: Vladimir Sementsov-Ogievskiy CC: John Snow CC: Andrey Drobyshev --- block/backup.c | 1 + block/block-copy.c | 108 ++++++++++++++++++++++++++++++++----- include/block/block-copy.h | 1 + 3 files changed, 98 insertions(+), 12 deletions(-) diff --git a/block/backup.c b/block/backup.c index d4713fa1cd..11d70243e2 100644 --- a/block/backup.c +++ b/block/backup.c @@ -280,6 +280,7 @@ static int coroutine_fn backup_run(Job *job, Error **er= rp) offset +=3D count; } block_copy_set_skip_unallocated(s->bcs, false); + block_copy_set_zero_bitmap_valid(s->bcs); } =20 if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_NONE) { diff --git a/block/block-copy.c b/block/block-copy.c index d71d070dbd..94d4f3dd69 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -20,6 +20,8 @@ #include "block/block_int-io.h" #include "block/dirty-bitmap.h" #include "block/reqlist.h" +#include "qemu/hbitmap.h" +#include "qemu/host-utils.h" #include "system/block-backend.h" #include "qemu/units.h" #include "qemu/co-shared-resource.h" @@ -157,6 +159,10 @@ typedef struct BlockCopyState { bool skip_unallocated; /* atomic */ /* State fields that use a thread-safe API */ BdrvDirtyBitmap *copy_bitmap; + /* Clusters reading as zero; allocated on demand, frozen once valid. */ + HBitmap *zero_bitmap; + /* Published only after the scan, with skip_unallocated already false.= */ + bool zero_bitmap_valid; /* atomic, store-release/load-acquire */ ProgressMeter *progress; SharedResource *mem; RateLimit rate_limit; @@ -190,6 +196,7 @@ block_copy_task_create(BlockCopyState *s, BlockCopyCall= State *call_state, int64_t offset, int64_t bytes) { BlockCopyTask *task; + BlockCopyMethod method; int64_t max_chunk; =20 QEMU_LOCK_GUARD(&s->lock); @@ -201,6 +208,27 @@ block_copy_task_create(BlockCopyState *s, BlockCopyCal= lState *call_state, return NULL; } =20 + method =3D s->method; + + /* + * The scan already knows how this range reads: pick the method here a= nd + * stop the task where the answer changes. + */ + if (qatomic_load_acquire(&s->zero_bitmap_valid)) { + int64_t boundary; + + if (hbitmap_get(s->zero_bitmap, offset)) { + method =3D COPY_WRITE_ZEROES; + boundary =3D hbitmap_next_zero(s->zero_bitmap, offset, bytes); + } else { + boundary =3D hbitmap_next_dirty(s->zero_bitmap, offset, bytes); + } + + if (boundary >=3D 0) { + bytes =3D boundary - offset; + } + } + assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); bytes =3D QEMU_ALIGN_UP(bytes, s->cluster_size); =20 @@ -215,7 +243,7 @@ block_copy_task_create(BlockCopyState *s, BlockCopyCall= State *call_state, .task.func =3D block_copy_task_entry, .s =3D s, .call_state =3D call_state, - .method =3D s->method, + .method =3D method, }; reqlist_init_req(&s->reqs, &task->req, offset, bytes); =20 @@ -271,6 +299,9 @@ void block_copy_state_free(BlockCopyState *s) =20 ratelimit_destroy(&s->rate_limit); bdrv_release_dirty_bitmap(s->copy_bitmap); + if (s->zero_bitmap) { + hbitmap_free(s->zero_bitmap); + } shres_destroy(s->mem); g_free(s); } @@ -624,20 +655,24 @@ static coroutine_fn int block_copy_task_entry(AioTask= *task) return ret; } =20 +/* The scan and the per-task query must resolve BDRV_BLOCK_ZERO alike. */ +static GRAPH_RDLOCK BlockDriverState *block_copy_status_base(BlockCopyStat= e *s) +{ + if (qatomic_read(&s->skip_unallocated)) { + return bdrv_backing_chain_next(s->source->bs); + } + + return NULL; +} + static coroutine_fn GRAPH_RDLOCK int block_copy_block_status(BlockCopyState *s, int64_t offset, int64_t byt= es, int64_t *pnum) { int64_t num; - BlockDriverState *base; + BlockDriverState *base =3D block_copy_status_base(s); int ret; =20 - if (qatomic_read(&s->skip_unallocated)) { - base =3D bdrv_backing_chain_next(s->source->bs); - } else { - base =3D NULL; - } - ret =3D bdrv_co_block_status_above(s->source->bs, base, offset, bytes,= &num, NULL, NULL); if (ret < 0 || num < s->cluster_size) { @@ -657,16 +692,41 @@ int block_copy_block_status(BlockCopyState *s, int64_= t offset, int64_t bytes, return ret; } =20 +/* Only the scan allocates, and it runs before zero_bitmap_valid. */ +static HBitmap *block_copy_zero_bitmap(BlockCopyState *s) +{ + if (!s->zero_bitmap) { + s->zero_bitmap =3D hbitmap_alloc(s->len, ctz32(s->cluster_size)); + } + + return s->zero_bitmap; +} + +static void block_copy_mark_zero_prefix(BlockCopyState *s, int64_t offset, + int64_t zero_count) +{ + int64_t zero_bytes =3D QEMU_ALIGN_DOWN(zero_count, s->cluster_size); + + if (zero_bytes > 0) { + hbitmap_set(block_copy_zero_bitmap(s), offset, zero_bytes); + } +} + /* * Check if the cluster starting at offset is allocated or not. * return via pnum the number of contiguous clusters sharing this allocati= on. + * Also marks the zero prefix of the range in zero_bitmap. */ static int coroutine_fn GRAPH_RDLOCK block_copy_is_cluster_allocated(BlockCopyState *s, int64_t offset, int64_t *pnum) { BlockDriverState *bs =3D s->source->bs; + BlockDriverState *base =3D block_copy_status_base(s); + int64_t orig_offset =3D offset; int64_t count, total_count =3D 0; + int64_t zero_count =3D 0; + bool zero_broken =3D false; int64_t bytes =3D s->len - offset; int ret; =20 @@ -674,25 +734,37 @@ block_copy_is_cluster_allocated(BlockCopyState *s, in= t64_t offset, =20 while (true) { /* protected in backup_run() */ - ret =3D bdrv_co_is_allocated(bs, offset, bytes, &count); + ret =3D bdrv_co_block_status_above(bs, base, offset, bytes, &count, + NULL, NULL); if (ret < 0) { return ret; } =20 + if (!zero_broken) { + if (ret & BDRV_BLOCK_ZERO) { + zero_count +=3D count; + } else { + zero_broken =3D true; + } + } + total_count +=3D count; =20 - if (ret || count =3D=3D 0) { + if ((ret & BDRV_BLOCK_ALLOCATED) || count =3D=3D 0) { /* - * ret: partial segment(s) are considered allocated. + * BDRV_BLOCK_ALLOCATED: partial segment(s) are considered + * allocated. * otherwise: unallocated tail is treated as an entire segment. */ *pnum =3D DIV_ROUND_UP(total_count, s->cluster_size); - return ret; + block_copy_mark_zero_prefix(s, orig_offset, zero_count); + return !!(ret & BDRV_BLOCK_ALLOCATED); } =20 /* Unallocated segment(s) with uncertain following segment(s) */ if (total_count >=3D s->cluster_size) { *pnum =3D total_count / s->cluster_size; + block_copy_mark_zero_prefix(s, orig_offset, zero_count); return 0; } =20 @@ -752,6 +824,12 @@ block_copy_set_task_method(BlockCopyState *s, BlockCop= yTask *task) int ret; int64_t status_bytes; =20 + /* block_copy_task_create() already decided, from zero_bitmap. */ + if (qatomic_load_acquire(&s->zero_bitmap_valid)) { + return true; + } + + /* CBW filter could call this early. */ ret =3D block_copy_block_status(s, task->req.offset, task->req.bytes, &status_bytes); assert(ret >=3D 0); /* never fail */ @@ -1081,6 +1159,12 @@ void block_copy_set_skip_unallocated(BlockCopyState = *s, bool skip) qatomic_set(&s->skip_unallocated, skip); } =20 +void block_copy_set_zero_bitmap_valid(BlockCopyState *s) +{ + block_copy_zero_bitmap(s); + qatomic_store_release(&s->zero_bitmap_valid, true); +} + void block_copy_set_speed(BlockCopyState *s, uint64_t speed) { ratelimit_set_speed(&s->rate_limit, speed, BLOCK_COPY_SLICE_TIME); diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 0df2771181..e4e5b56753 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -101,5 +101,6 @@ void block_copy_call_cancel(BlockCopyCallState *call_st= ate); BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s); int64_t block_copy_cluster_size(BlockCopyState *s); void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip); +void block_copy_set_zero_bitmap_valid(BlockCopyState *s); =20 #endif /* BLOCK_COPY_H */ --=20 2.53.0 From nobody Sat Sep 26 21:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1787928450; cv=none; d=zohomail.com; s=zohoarc; b=kAQgXJ4hXKSAB0uEhl3kchanjo22TgWI/6/esMAwu5SCIg9fBqvKIsUl16zrcbPLZcoRKVymrYr1O2DpmMcvqbq07mu7Ytc/HM6gjtvPERDU31R/XaatCqmJt+BOImKdKx2v2CJ83h7YJcpjTt/AK6BmK2B1sV3yquGkFjGTC+I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1787928450; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=rJs5q0k6Ad65mdVAHbHvZTkV+6jv0q/NjjLKksIrv+E=; b=nDDt2Bp5hS8y6DoTLrlRj9r+jN/OY7vyo0O4AnChZgx8qrQsfnri02ifg+h6zNrWkkl7EmO3y+LzTM3XXZ9s/CpzoF0IOyHBylXsHhblMR3/nkeDUT9gAbSJZhA0sFQ/RuHt789Kb1OYqCWWCd1Qti100ZCXU1qZMgLbzqEq0Xw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1787928450252812.8024512396464; Fri, 28 Aug 2026 07:47:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wzxqQ-0002kz-0A; Fri, 28 Aug 2026 10:46:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wzxqO-0002kF-N9 for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:12 -0400 Received: from mail-wm1-x32d.google.com ([2a00:1450:4864:20::32d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wzxqM-0000i6-B5 for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:12 -0400 Received: by mail-wm1-x32d.google.com with SMTP id 5b1f17b1804b1-496bb7cdf51so12750775e9.2 for ; Fri, 28 Aug 2026 07:46:09 -0700 (PDT) Received: from athena.sw.ru ([2a06:5b06:b600:300:9458:da19:ab04:373a]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49b94dcaf61sm53730925e9.4.2026.08.28.07.46.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 07:46:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openvz.org; s=google; t=1787928369; x=1788533169; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=rJs5q0k6Ad65mdVAHbHvZTkV+6jv0q/NjjLKksIrv+E=; b=GemDAjK7mTUBdEjLsaxEdpXq0gC6xgSb65YzQeT4+4qhsvjBQ4ogptpbacH9ptjIxQ WbR+1ZGs5eGNowd2zF5tdyPaNrgQUIBxw6bV7a7E3UsWd9EIQQptVQmEsjEdLRbj/P6v srI+4MTekdHGYSd4HH/2uI8jUhc3RTQYr3+eoCoTLD6/8YOKjgMnbEgCOdFnocEMUB6E 8q7iwb8GfNHz07tJvzG1aYBzAevwhYaUPlIZFl6JROCIumCkZQ/52m2dSDliU3yI8er4 V0IYUbNdkREb9NAtSXv+EGTxZDpxOCCBVE5oTnXU1PiW4TwGCjsWsa/mVSnstTx1xYGD XXYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787928369; x=1788533169; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=rJs5q0k6Ad65mdVAHbHvZTkV+6jv0q/NjjLKksIrv+E=; b=Yj1/TQhFsy20kniEb4RoxYjObmWXOa/X/bZuy/6QnF4SGO82oCYUXFrZAO40woj8MM klcpc4G97pTumw497Qz+i3w6V8mwdRjljYj2MQzw414D8TTLIu0oLr81CEjUwqVz8Q+b GA3BJgB4U0AT3twmMktdlyrxuxSA+FqqH1hPSqi2zGCdF0UC3gYS4HRyfdSYp3K1wJRc +zSdu7YkeuyFZPEOUelVALgp5rKOB5v/ddq7R86MzWjcVGCPXZQ+3Go8Aa1cp0nv101T Y7gdXBC8ir3bASMm5H9lHW0Kt6OLLa9z4414gVemm8shqp8T0SS+7ZoMZu4YUe5jqyZq HHtA== X-Gm-Message-State: AFuF++kXi9g6r3BS2/58WSf7MIhAZhatm0DGr07DDAgQ9fV6lkZgY/pN xf7sAWfPa2jj3OYlsF7BHVYHS85rlYz5EC2k7+e0U9arXwQ1QrWEQ4vcxd/55/mHm9y2AwGwihH oO7Qw X-Gm-Gg: AR+sD10uQe+EVRMK6adgcwH+SVa0zShCbAh9iJKypcpJRfoJ3cZ8LLBMCDreLS1ckf4 UH6iGQUhTvn1MManKycOV0gdVX8uUg7/5EGWaDcPFBvGOBYCvYaGeGX1s1UQVQnssEMl7Yd7u4J tzY42PlicZJ0A4mirSWtGc2OpoxQ5udk/imlrkvsn2EeJiBLBOBT6CNETUjCq/gG8S9zHe0cSYK z2qCwMCLmDaxgnb2H935Cbn6HUxA2I8i9cvoPrnRsPPy/vXWKwta5l5n27QuDevP/dO2RhQeTip wlmYSCUMmqIuD4ziJ0Z3W7shmLYKoW0BO3wV+Q44P8DjMqgN8TPaItcrpd1s7+x+2uhNYBVP73n yR1ZDEb0dvVOHdZvC7RLT94sPY2J57Xi1lnDJT8RaC8zSvEtDMbEkPdZHk+b056MONwzVWLU4C/ ZzHxwb6fw6foeovZEwf4EHTbnaw+ms2cltB7beAhOR00jKrBdVzo4S9LkXzw== X-Received: by 2002:a05:600c:19ca:b0:495:52a5:8829 with SMTP id 5b1f17b1804b1-49b91c47b3dmr108463655e9.11.1787928368601; Fri, 28 Aug 2026 07:46:08 -0700 (PDT) From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Vladimir Sementsov-Ogievskiy , John Snow , Andrey Drobyshev Subject: [PATCH 8/9] block/backup: pre-fill zero_bitmap for full/bitmap Date: Fri, 28 Aug 2026 16:45:53 +0200 Message-ID: <20260828144554.1741910-9-den@openvz.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260828144554.1741910-1-den@openvz.org> References: <20260828144554.1741910-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32d; envelope-from=den@openvz.org; helo=mail-wm1-x32d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @openvz.org) X-ZM-MESSAGEID: 1787928452981158500 Content-Type: text/plain; charset="utf-8" From: Denis V. Lunev Only sync=3Dtop runs an up-front scan, so the previous patch did nothing for the other modes. FULL and BITMAP know their complete copy_bitmap before copying starts, and the same copy-before-write guarantee holds: a cluster still dirty when the copy loop reaches it has not been written since the scan. Add block_copy_calculate_zero_bitmap(), driven one query at a time from backup_scan_source() so the graph rdlock and the pause point are taken per query, and query each confirmed span once however many dirty areas it covers. The scan only fills zero_bitmap and never clears copy_bitmap, so sync=3Dfull still writes zeroes for clusters the source never allocated; it just stops re-querying to find that out. Unlike the per-task queries it replaces the scan is serial, so a sparsely dirtied job against a slow source may spend more time here than it saves. The sync=3Dtop loop had the same shape, so both fold into backup_scan_source(). sync=3Dnone keeps the per-task query. Its bitmap says "anything may be copied", not "this will be copied", and it only ever copies what the guest writes during the fleecing window, so scanning the whole image at attach is the wrong trade. Signed-off-by: Denis V. Lunev CC: Vladimir Sementsov-Ogievskiy CC: John Snow CC: Andrey Drobyshev Reviewed-by: Andrey Drobyshev --- block/backup.c | 94 ++++++++++++++++++++++++-------------- block/block-copy.c | 41 +++++++++++++++++ include/block/block-copy.h | 4 ++ 3 files changed, 105 insertions(+), 34 deletions(-) diff --git a/block/backup.c b/block/backup.c index 11d70243e2..b6e559a97e 100644 --- a/block/backup.c +++ b/block/backup.c @@ -247,59 +247,85 @@ static void backup_init_bcs_bitmap(BackupBlockJob *jo= b) job_progress_set_remaining(&job->common.job, estimate); } =20 -static int coroutine_fn backup_run(Job *job, Error **errp) +/* + * Walk the source before copying starts: fill zero_bitmap and, for sync= =3Dtop, + * drop the clusters that are not allocated above the backing. + */ +static int coroutine_fn backup_scan_source(BackupBlockJob *s) { - BackupBlockJob *s =3D container_of(job, BackupBlockJob, common.job); - int ret; - - backup_init_bcs_bitmap(s); - - if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_TOP) { - int64_t offset =3D 0; - int64_t count; + Job *job =3D &s->common.job; + bool top =3D s->sync_mode =3D=3D MIRROR_SYNC_MODE_TOP; + int64_t offset, cached_end =3D 0, count; + int ret =3D 0; =20 - for (offset =3D 0; offset < s->len; ) { - if (job_is_cancelled(job)) { - return -ECANCELED; - } + for (offset =3D 0; offset < s->len; offset +=3D count) { + if (job_is_cancelled(job)) { + return -ECANCELED; + } =20 - job_pause_point(job); + job_pause_point(job); =20 - if (job_is_cancelled(job)) { - return -ECANCELED; - } + if (job_is_cancelled(job)) { + return -ECANCELED; + } =20 - /* rdlock protects the subsequent call to bdrv_is_allocated() = */ - bdrv_graph_co_rdlock(); + /* rdlock protects the block-status queries below */ + bdrv_graph_co_rdlock(); + if (top) { ret =3D block_copy_reset_unallocated(s->bcs, offset, &count); - bdrv_graph_co_rdunlock(); - if (ret < 0) { - return ret; - } + } else { + block_copy_calculate_zero_bitmap(s->bcs, offset, &cached_end, + &count); + } + bdrv_graph_co_rdunlock(); =20 - offset +=3D count; + if (ret < 0) { + return ret; } + } + + if (top) { block_copy_set_skip_unallocated(s->bcs, false); - block_copy_set_zero_bitmap_valid(s->bcs); } + block_copy_set_zero_bitmap_valid(s->bcs); =20 - if (s->sync_mode =3D=3D MIRROR_SYNC_MODE_NONE) { + return 0; +} + +static int coroutine_fn backup_run(Job *job, Error **errp) +{ + BackupBlockJob *s =3D container_of(job, BackupBlockJob, common.job); + int ret; + + backup_init_bcs_bitmap(s); + + switch (s->sync_mode) { + case MIRROR_SYNC_MODE_TOP: + case MIRROR_SYNC_MODE_FULL: + case MIRROR_SYNC_MODE_BITMAP: + ret =3D backup_scan_source(s); + if (ret < 0) { + return ret; + } + break; + + case MIRROR_SYNC_MODE_NONE: /* * All bits are set in bcs bitmap to allow any cluster to be copie= d. - * This does not actually require them to be copied. + * This does not actually require them to be copied. Yield until t= he + * job is cancelled and let the before_write notify callback servi= ce + * CoW requests. */ while (!job_is_cancelled(job)) { - /* - * Yield until the job is cancelled. We just let our before_w= rite - * notify callback service CoW requests. - */ job_yield(job); } - } else { - return backup_loop(s); + return 0; + + default: + break; } =20 - return 0; + return backup_loop(s); } =20 static void coroutine_fn backup_pause(Job *job) diff --git a/block/block-copy.c b/block/block-copy.c index 94d4f3dd69..c0a3359938 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -813,6 +813,47 @@ int64_t coroutine_fn block_copy_reset_unallocated(Bloc= kCopyState *s, return ret; } =20 +/* + * One scan step, like block_copy_reset_unallocated(); @cached_end + * tracks progress across calls, @count how far @offset should move. + */ +void coroutine_fn GRAPH_RDLOCK +block_copy_calculate_zero_bitmap(BlockCopyState *s, int64_t offset, + int64_t *cached_end, int64_t *count) +{ + int64_t dirty_offset, dirty_bytes, dirty_end; + bool found; + + /* CBW mutates copy_bitmap from its own AioContext meanwhile. */ + WITH_QEMU_LOCK_GUARD(&s->lock) { + found =3D bdrv_dirty_bitmap_next_dirty_area(s->copy_bitmap, offset, + s->len, INT64_MAX, + &dirty_offset, &dirty_by= tes); + } + + if (!found) { + *count =3D s->len - offset; + return; + } + dirty_end =3D dirty_offset + dirty_bytes; + + if (*cached_end < dirty_end) { + int64_t clusters; + int64_t query_offset =3D MAX(dirty_offset, *cached_end); + int ret =3D block_copy_is_cluster_allocated(s, query_offset, &clus= ters); + + if (ret >=3D 0) { + *cached_end =3D query_offset + clusters * s->cluster_size; + } else { + /* Best-effort: leave this area unmarked; it copies as data. */ + *cached_end =3D dirty_end; + } + } + + /* 0 if the query above didn't yet reach dirty_end: try again next cal= l. */ + *count =3D *cached_end >=3D dirty_end ? dirty_end - offset : 0; +} + /* * Decide how @task is copied: COPY_WRITE_ZEROES if it reads as zero. May * shrink @task. Returns false if @task is to be skipped (already ended, diff --git a/include/block/block-copy.h b/include/block/block-copy.h index e4e5b56753..3a61066b55 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -44,6 +44,10 @@ void block_copy_reset(BlockCopyState *s, int64_t offset,= int64_t bytes); int64_t coroutine_fn GRAPH_RDLOCK block_copy_reset_unallocated(BlockCopyState *s, int64_t offset, int64_t *c= ount); =20 +void coroutine_fn GRAPH_RDLOCK +block_copy_calculate_zero_bitmap(BlockCopyState *s, int64_t offset, + int64_t *cached_end, int64_t *count); + int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t byt= es, bool ignore_ratelimit, uint64_t timeout_ns, BlockCopyAsyncCallbackFunc cb, --=20 2.53.0 From nobody Sat Sep 26 21:37:05 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=openvz.org ARC-Seal: i=1; a=rsa-sha256; t=1787928467; cv=none; d=zohomail.com; s=zohoarc; b=dye/x1dnzIfBYPMu+WJSJWeTrfjgJ4j9LG1TSUNv9NC8H8FlpInbtMx2bXMByIQpDw6QMtv/op1ZMPVslWcPdyp/bOAXtQMR4I4OKk1wHpUiNUnOP+xz9Jynt7KwSCJ9VhOkrH9jsYi+IhJ9ldxTMLoUffPocEBwqb7DfsWME9U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1787928467; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fkEN44L0ZdFVpa/GxkjZduejqvJWgpOrsGopU0sTJpY=; b=c3T6pLb/HRVdojqwSeI0CqX3tie1UPGxC1VJCFrT8htZU/jEIv46aVAjma7PKp32shxelVpvjGSROPAFFF3wW1VrMn0na48xR3ZoajM3xzzOp8skOcihsHbc74u/VGSncQf2P8ueqQbp6cSWbpsdH1h7mBrv7L8IKlIE6EPCgwA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists1p.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1787928466918998.8099904018687; Fri, 28 Aug 2026 07:47:46 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists1p.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wzxqQ-0002l5-I3; Fri, 28 Aug 2026 10:46:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wzxqP-0002kd-2E for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:13 -0400 Received: from mail-wm1-x330.google.com ([2a00:1450:4864:20::330]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1wzxqN-0000iQ-BP for qemu-devel@nongnu.org; Fri, 28 Aug 2026 10:46:12 -0400 Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-4953e04ef16so10155935e9.2 for ; Fri, 28 Aug 2026 07:46:10 -0700 (PDT) Received: from athena.sw.ru ([2a06:5b06:b600:300:9458:da19:ab04:373a]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49b94dcaf61sm53730925e9.4.2026.08.28.07.46.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Aug 2026 07:46:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openvz.org; s=google; t=1787928370; x=1788533170; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=fkEN44L0ZdFVpa/GxkjZduejqvJWgpOrsGopU0sTJpY=; b=n028BXODmqCWbmkkQ06TTfF0vzL1uYwcKCCKA8afQYOLZgNeTFt8XCYwtSj7zJiapm zfz8YnxKnIE+9ohgRjv3ZwJoGe1wILmFPxXR8MhmNW0xm7FQtpDlU3aTKskfxXha5ldT 1NJ9JUdyaOueVhiNQy7ESpl7xMs9Sff2ZPyA7TrztnxTHhxdQxPmJ3je96BgOj5lUz4j GQVvdvaj9szXFWpfsTQ0OMiBy5d3b0fbG8efhNwWbwgcf1CP8X9+DFGdeZ7PvMrtfbtk 5HuG4XwOt6kq0wWq212r3323Y8w1DltgDxZk2SoSxvkhMPazPWXte1c9vFnoN5uX63Tn MQdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787928370; x=1788533170; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=fkEN44L0ZdFVpa/GxkjZduejqvJWgpOrsGopU0sTJpY=; b=n6PzrcOh7VvpQEB6Kf8AMEXlTe0vYU2cx+wLy0jUFJFaHdkh0Syha5AQORzVfqvdI+ XRkukz7bYxDyhAScJxtxxmfi3OuZtmj/ccYV/Prcq7txiTpz2zgeRliYJ8xzYoVJj9cc Jba3yM91W5CrOVsk3QyYMzJeBJyCcIDimsGGwv1XgkCTwVQSZtn6T3JX7c0SX6P9fBP/ 9vctmlNUb3D8jg9xbkcp1o9Ek5dINjSIDoic2mJgYlEwvsf0oJ72pnOx2KA36QRng6Fm 0dDESyWpLrpSkIgM1s5UDkzX/vq7+OvnEXkEs+tGR3j5dn85Nm7k0zCOHizmmZQmtT1X tO+g== X-Gm-Message-State: AFuF++lOZiUWcMbOwqLdar26EHyLxJRCyt7FNn3wXa7j2nhUG5iC8Zq9 +3px1kCAtvjGhdMdPNBzY+BQ0qAgN3jQowjcoGyN0aku5EpRM3R0QWNFJBmWg2K8sHdvfGcA9G6 1tmCS X-Gm-Gg: AR+sD12T5QnmpWOXpdk9nlZdhB/h9c8i0sESJ2UpUkoDfbGSJn8s+aVCS5TmqTV2C2A rDexMjpIN3asylT0AT0vHQPAW4h29dZJhFKhf4orck0gsJwUUEq4Do63iwwmoctk7g3qWC0eejV IDqKvrhjKhso5SuX784K3MCEgsvZEsLrJz2lK2V6Lhqer3mjC+DYQ24TW3TmzUL1ORJ0Ss9sbH5 qE1Mj8X/1ah5/+1aZT55yScewhzkeYACB1iz6YSZtMTA9ZP1ScxSGa5FO5UN22nCeme3+DxzBTB +xFD1t5reMHRqKxTBF69hEklgr63FeKuxO8H138hcJplkSquNvFmGfbgqGbDikPOpBdNrhrYdIf xfNZFw61Wz1qHxWriPbOI2QKsgM44Pv4XhxtLZH5X/yJbcSXtUlFOj0dUnkKsERrh3hSgDEmHgr 8ab0VpJbXxtv3KFXgGvFqo51bALpjuDVltIMfiYiV6nzMiy3JAYAWyIngmCMHSjh40+vMi X-Received: by 2002:a05:600c:6388:b0:499:d95a:414 with SMTP id 5b1f17b1804b1-49b91c26a17mr128752355e9.6.1787928369725; Fri, 28 Aug 2026 07:46:09 -0700 (PDT) From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, "Denis V. Lunev" , Vladimir Sementsov-Ogievskiy , John Snow , Andrey Drobyshev Subject: [PATCH 9/9] block/block-copy: coalesce write-zeroes tasks Date: Fri, 28 Aug 2026 16:45:54 +0200 Message-ID: <20260828144554.1741910-10-den@openvz.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260828144554.1741910-1-den@openvz.org> References: <20260828144554.1741910-1-den@openvz.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists1p.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::330; envelope-from=den@openvz.org; helo=mail-wm1-x330.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @openvz.org) X-ZM-MESSAGEID: 1787928469007158500 Content-Type: text/plain; charset="utf-8" From: Denis V. Lunev Task size was capped to block_copy_chunk_size() regardless of method, which sizes a task by the copy buffer. A write-zeroes task carries no buffer, so that cap only splits one long known-zero run into a crowd of small COPY_WRITE_ZEROES tasks, each with its own request against the target. block_copy_task_create() already decides from zero_bitmap how far a task may run, so add block_copy_widen_zero_area() to that decision: for a write-zeroes task, re-search the dirty area under BLOCK_COPY_MAX_ZERO_CHUNK instead of the buffer chunk size, and let the existing clamp cut it back to where the zero run ends. Widening is safe: an overlapping caller waits on the task's BlockReq via reqlist_wait_one() rather than observing it mid-flight. The 256M cap stays well under BDRV_REQUEST_MAX_BYTES, where block_copy_do_copy() would trip assert(nbytes < INT_MAX), keeps the rate limiter usable, leaves enough tasks to fill BLOCK_COPY_MAX_WORKERS and bounds how long a cancel waits on one request. It is aligned at the use site rather than in the constant, since nothing bounds cluster_size. Signed-off-by: Denis V. Lunev CC: Vladimir Sementsov-Ogievskiy CC: John Snow CC: Andrey Drobyshev --- block/block-copy.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/block/block-copy.c b/block/block-copy.c index c0a3359938..16b76484cb 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -37,6 +37,8 @@ #define BLOCK_COPY_MAX_WORKERS 64 #define BLOCK_COPY_SLICE_TIME 100000000ULL /* ns */ #define BLOCK_COPY_CLUSTER_SIZE_DEFAULT (1 << 16) +/* Bounded by BDRV_REQUEST_MAX_BYTES, rate limiting and cancel latency. */ +#define BLOCK_COPY_MAX_ZERO_CHUNK (256 * MiB) =20 typedef enum { COPY_READ_WRITE_CLUSTER, @@ -187,6 +189,34 @@ static int64_t block_copy_chunk_size(BlockCopyState *s) } } =20 +/* + * A write-zeroes task carries no buffer, so it may cover far more than + * block_copy_chunk_size(). Return how far it may run; the caller clamps it + * to where the zero run ends. + */ +static int64_t block_copy_widen_zero_area(BlockCopyState *s, + BlockCopyCallState *call_state, + int64_t offset, int64_t search_e= nd, + int64_t bytes) +{ + int64_t aligned =3D QEMU_ALIGN_DOWN(BLOCK_COPY_MAX_ZERO_CHUNK, + s->cluster_size); + int64_t zero_chunk =3D MIN_NON_ZERO(MAX(aligned, s->cluster_size), + call_state->max_chunk); + int64_t wide_offset, wide_bytes; + + if (!bdrv_dirty_bitmap_next_dirty_area(s->copy_bitmap, offset, search_= end, + zero_chunk, &wide_offset, + &wide_bytes)) { + return bytes; + } + + /* @offset is dirty, so the search cannot have moved past it. */ + assert(wide_offset =3D=3D offset); + + return wide_bytes; +} + /* * Search for the first dirty area in offset/bytes range and create task at * the beginning of it. @@ -198,6 +228,7 @@ block_copy_task_create(BlockCopyState *s, BlockCopyCall= State *call_state, BlockCopyTask *task; BlockCopyMethod method; int64_t max_chunk; + int64_t search_end =3D offset + bytes; =20 QEMU_LOCK_GUARD(&s->lock); max_chunk =3D MIN_NON_ZERO(block_copy_chunk_size(s), call_state->max_c= hunk); @@ -219,6 +250,8 @@ block_copy_task_create(BlockCopyState *s, BlockCopyCall= State *call_state, =20 if (hbitmap_get(s->zero_bitmap, offset)) { method =3D COPY_WRITE_ZEROES; + bytes =3D block_copy_widen_zero_area(s, call_state, offset, + search_end, bytes); boundary =3D hbitmap_next_zero(s->zero_bitmap, offset, bytes); } else { boundary =3D hbitmap_next_dirty(s->zero_bitmap, offset, bytes); --=20 2.53.0