From nobody Fri May 3 14:44:13 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642526386810745.8732646122425; Tue, 18 Jan 2022 09:19:46 -0800 (PST) Received: from localhost ([::1]:43448 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n9s93-0004DM-JQ for importer@patchew.org; Tue, 18 Jan 2022 12:19:45 -0500 Received: from eggs.gnu.org ([209.51.188.92]:49972) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n9s3V-0007EA-9Y for qemu-devel@nongnu.org; Tue, 18 Jan 2022 12:14:01 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22680) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n9s3T-0003hg-Hs for qemu-devel@nongnu.org; Tue, 18 Jan 2022 12:14:00 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-154-Gy3j1E8JNo2AjLQMLPO-5g-1; Tue, 18 Jan 2022 12:13:56 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 957441091B11; Tue, 18 Jan 2022 17:00:11 +0000 (UTC) Received: from localhost (unknown [10.39.194.149]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2643384D03; Tue, 18 Jan 2022 17:00:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642526039; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=u41Kqjld1T5T5bJY/38Fai5bRQ2UK99Tzm6Danarohk=; b=KpV3bf5Qk5ae424TtTe77UKjdWW1fNiU14/Oei0NdA89eItOz+B4AU/YwMARdVyvbBw342 YRKNpcqHbF1yPkSN+UWsJDgtjztqLFCro39TxYNEuVyOhBxCjYOQhzKN65XBzGlzZsWAEm uqpEfnHRarhIigLODVHsdnxgkXsDyO8= X-MC-Unique: Gy3j1E8JNo2AjLQMLPO-5g-1 From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PATCH v2 1/2] block/io: Update BSC only if want_zero is true Date: Tue, 18 Jan 2022 17:59:59 +0100 Message-Id: <20220118170000.49423-2-hreitz@redhat.com> In-Reply-To: <20220118170000.49423-1-hreitz@redhat.com> References: <20220118170000.49423-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Nir Soffer , Hanna Reitz , qemu-devel@nongnu.org, qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642526387430100001 Content-Type: text/plain; charset="utf-8" We update the block-status cache whenever we get new information from a bdrv_co_block_status() call to the block driver. However, if we have passed want_zero=3Dfalse to that call, it may flag areas containing zeroes as data, and so we would update the block-status cache with wrong information. Therefore, we should not update the cache with want_zero=3Dfalse. Reported-by: Nir Soffer Fixes: 0bc329fbb00 ("block: block-status cache for data regions") Reviewed-by: Nir Soffer Cc: qemu-stable@nongnu.org Signed-off-by: Hanna Reitz Reviewed-by: Eric Blake --- block/io.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/block/io.c b/block/io.c index bb0a254def..4e4cb556c5 100644 --- a/block/io.c +++ b/block/io.c @@ -2497,8 +2497,12 @@ static int coroutine_fn bdrv_co_block_status(BlockDr= iverState *bs, * non-protocol nodes, and then it is never used. However, fi= lling * the cache requires an RCU update, so double check here to a= void * such an update if possible. + * + * Check want_zero, because we only want to update the cache w= hen we + * have accurate information about what is zero and what is da= ta. */ - if (ret =3D=3D (BDRV_BLOCK_DATA | BDRV_BLOCK_OFFSET_VALID) && + if (want_zero && + ret =3D=3D (BDRV_BLOCK_DATA | BDRV_BLOCK_OFFSET_VALID) && QLIST_EMPTY(&bs->children)) { /* --=20 2.33.1 From nobody Fri May 3 14:44:13 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642527482542547.0780706816894; Tue, 18 Jan 2022 09:38:02 -0800 (PST) Received: from localhost ([::1]:50792 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n9sQj-0004Sc-7j for importer@patchew.org; Tue, 18 Jan 2022 12:38:01 -0500 Received: from eggs.gnu.org ([209.51.188.92]:49764) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n9s2S-00062o-5w for qemu-devel@nongnu.org; Tue, 18 Jan 2022 12:13:03 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44071) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n9s2Q-0003cf-2B for qemu-devel@nongnu.org; Tue, 18 Jan 2022 12:12:55 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-558-XdrUFlaPOuS7xc6-NOGKPQ-1; Tue, 18 Jan 2022 12:12:35 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7124710A2609; Tue, 18 Jan 2022 17:00:14 +0000 (UTC) Received: from localhost (unknown [10.39.194.149]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B641184D07; Tue, 18 Jan 2022 17:00:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642525973; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yJiKI/EfP3Kb9cR4D+B6fiifYdDk2ui7y40xn8IgahE=; b=Cl8KJaGH8NVVKZsqkmzI6E15AhAeQ7TpAmZWDIY651yHHNcFQv9YEKB7S9TBb4dpVzG4ms Jht53nQJzTXESb18+0PYqKx4nzJo5YktfbUOyFMW7GnIsEjqAAXNNDuXF72S432smJ4yHm fF4l8hetYX53wNh48TEO1UcynvdnApU= X-MC-Unique: XdrUFlaPOuS7xc6-NOGKPQ-1 From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PATCH v2 2/2] iotests/block-status-cache: New test Date: Tue, 18 Jan 2022 18:00:00 +0100 Message-Id: <20220118170000.49423-3-hreitz@redhat.com> In-Reply-To: <20220118170000.49423-1-hreitz@redhat.com> References: <20220118170000.49423-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Nir Soffer , Hanna Reitz , qemu-devel@nongnu.org, qemu-stable@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642527484041100001 Content-Type: text/plain; charset="utf-8" Add a new test to verify that want_zero=3Dfalse block-status calls do not pollute the block-status cache for want_zero=3Dtrue calls. We check want_zero=3Dtrue calls and their results using `qemu-img map` (over NBD), and want_zero=3Dfalse calls also using `qemu-img map` over NBD, but using the qemu:allocation-depth context. (This test case cannot be integrated into nbd-qemu-allocation, because that is a qcow2 test, and this is a raw test.) Signed-off-by: Hanna Reitz Reviewed-by: Eric Blake Reviewed-by: Nir Soffer Tested-by: Eric Blake --- tests/qemu-iotests/tests/block-status-cache | 139 ++++++++++++++++++ .../qemu-iotests/tests/block-status-cache.out | 5 + 2 files changed, 144 insertions(+) create mode 100755 tests/qemu-iotests/tests/block-status-cache create mode 100644 tests/qemu-iotests/tests/block-status-cache.out diff --git a/tests/qemu-iotests/tests/block-status-cache b/tests/qemu-iotes= ts/tests/block-status-cache new file mode 100755 index 0000000000..6fa10bb8f8 --- /dev/null +++ b/tests/qemu-iotests/tests/block-status-cache @@ -0,0 +1,139 @@ +#!/usr/bin/env python3 +# group: rw quick +# +# Test cases for the block-status cache. +# +# Copyright (C) 2022 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import os +import signal +import iotests +from iotests import qemu_img_create, qemu_img_pipe, qemu_nbd + + +image_size =3D 1 * 1024 * 1024 +test_img =3D os.path.join(iotests.test_dir, 'test.img') + +nbd_pidfile =3D os.path.join(iotests.test_dir, 'nbd.pid') +nbd_sock =3D os.path.join(iotests.sock_dir, 'nbd.sock') + + +class TestBscWithNbd(iotests.QMPTestCase): + def setUp(self) -> None: + """Just create an empty image with a read-only NBD server on it""" + assert qemu_img_create('-f', iotests.imgfmt, test_img, + str(image_size)) =3D=3D 0 + + # Pass --allocation-depth to enable the qemu:allocation-depth cont= ext, + # which we are going to query to provoke a block-status inquiry wi= th + # want_zero=3Dfalse. + assert qemu_nbd(f'--socket=3D{nbd_sock}', + f'--format=3D{iotests.imgfmt}', + '--persistent', + '--allocation-depth', + '--read-only', + f'--pid-file=3D{nbd_pidfile}', + test_img) \ + =3D=3D 0 + + def tearDown(self) -> None: + with open(nbd_pidfile, encoding=3D'utf-8') as f: + pid =3D int(f.read()) + os.kill(pid, signal.SIGTERM) + os.remove(nbd_pidfile) + os.remove(test_img) + + def test_with_zero_bug(self) -> None: + """ + Verify that the block-status cache is not corrupted by a + want_zero=3Dfalse call. + We can provoke a want_zero=3Dfalse call with `qemu-img map` over N= BD with + x-dirty-bitmap=3Dqemu:allocation-depth, so we first run a normal `= map` + (which results in want_zero=3Dtrue), then using said + qemu:allocation-depth context, and finally another normal `map` to + verify that the cache has not been corrupted. + """ + + nbd_img_opts =3D f'driver=3Dnbd,server.type=3Dunix,server.path=3D{= nbd_sock}' + nbd_img_opts_alloc_depth =3D nbd_img_opts + \ + ',x-dirty-bitmap=3Dqemu:allocation-depth' + + # Normal map, results in want_zero=3Dtrue. + # This will probably detect an allocated data sector first (qemu l= ikes + # to allocate the first sector to facilitate alignment probing), a= nd + # then the rest to be zero. The BSC will thus contain (if anythin= g) + # one range covering the first sector. + map_pre =3D qemu_img_pipe('map', '--output=3Djson', '--image-opts', + nbd_img_opts) + + # qemu:allocation-depth maps for want_zero=3Dfalse. + # want_zero=3Dfalse should (with the file driver, which the server= is + # using) report everything as data. While this is sufficient for + # want_zero=3Dfalse, this is nothing that should end up in the + # block-status cache. + # Due to a bug, this information did end up in the cache, though, = and + # this would lead to wrong information being returned on subsequent + # want_zero=3Dtrue calls. + # + # We need to run this map twice: On the first call, we probably st= ill + # have the first sector in the cache, and so this will be served f= rom + # the cache; and only the subsequent range will be queried from the + # block driver. This subsequent range will then be entered into t= he + # cache. + # If we did a want_zero=3Dtrue call at this point, we would thus g= et + # correct information: The first sector is not covered by the cach= e, so + # we would get fresh block-status information from the driver, whi= ch + # would return a data range, and this would then go into the cache, + # evicting the wrong range from the want_zero=3Dfalse call before. + # + # Therefore, we need a second want_zero=3Dfalse map to reproduce: + # Since the first sector is not in the cache, the query for its st= atus + # will go to the driver, which will return a result that reports t= he + # whole image to be a single data area. This result will then go = into + # the cache, and so the cache will then report the whole image to + # contain data. + # + # Note that once the cache reports the whole image to contain data= , any + # subsequent map operation will be served from the cache, and so w= e can + # never loop too many times here. + for _ in range(2): + # (Ignore the result, this is just to contaminate the cache) + qemu_img_pipe('map', '--output=3Djson', '--image-opts', + nbd_img_opts_alloc_depth) + + # Now let's see whether the cache reports everything as data, or + # whether we get correct information (i.e. the same as we got on o= ur + # first attempt). + map_post =3D qemu_img_pipe('map', '--output=3Djson', '--image-opts= ', + nbd_img_opts) + + if map_pre !=3D map_post: + print('ERROR: Map information differs before and after queryin= g ' + + 'qemu:allocation-depth') + print('Before:') + print(map_pre) + print('After:') + print(map_post) + + self.fail("Map information differs") + + +if __name__ =3D=3D '__main__': + # The block-status cache only works on the protocol layer, so to test = it, + # we can only use the raw format + iotests.main(supported_fmts=3D['raw'], + supported_protocols=3D['file']) diff --git a/tests/qemu-iotests/tests/block-status-cache.out b/tests/qemu-i= otests/tests/block-status-cache.out new file mode 100644 index 0000000000..ae1213e6f8 --- /dev/null +++ b/tests/qemu-iotests/tests/block-status-cache.out @@ -0,0 +1,5 @@ +. +---------------------------------------------------------------------- +Ran 1 tests + +OK --=20 2.33.1