From nobody Tue Nov 4 21:22:00 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531413496441661.6969266244499; Thu, 12 Jul 2018 09:38:16 -0700 (PDT) Received: from localhost ([::1]:33012 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdebe-00008y-Ud for importer@patchew.org; Thu, 12 Jul 2018 12:38:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56788) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdeVn-0004hP-HR for qemu-devel@nongnu.org; Thu, 12 Jul 2018 12:32:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fdeVk-00067k-FQ for qemu-devel@nongnu.org; Thu, 12 Jul 2018 12:32:11 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:59382 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fdeVb-0005z7-Dp; Thu, 12 Jul 2018 12:31:59 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0E33A40201BF; Thu, 12 Jul 2018 16:31:59 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-16.ams2.redhat.com [10.36.117.16]) by smtp.corp.redhat.com (Postfix) with ESMTP id 372AD2022BCB; Thu, 12 Jul 2018 16:31:58 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 12 Jul 2018 18:31:46 +0200 Message-Id: <20180712163152.12521-2-kwolf@redhat.com> In-Reply-To: <20180712163152.12521-1-kwolf@redhat.com> References: <20180712163152.12521-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 12 Jul 2018 16:31:59 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 12 Jul 2018 16:31:59 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 1/7] iotests: 153: Fix dead code X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Fam Zheng This step was left behind my mistake. As suggested by the echoed text, the intention was to test two devices with the same image, with different options. The behavior should be the same as two QEMU processes. Complete it. Signed-off-by: Fam Zheng Signed-off-by: Kevin Wolf --- tests/qemu-iotests/153 | 2 ++ tests/qemu-iotests/153.out | 25 +++++++++++++++++++++++++ 2 files changed, 27 insertions(+) diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153 index 673813cd39..0daeb1b005 100755 --- a/tests/qemu-iotests/153 +++ b/tests/qemu-iotests/153 @@ -162,6 +162,7 @@ for opts1 in "" "read-only=3Don" "read-only=3Don,force-= share=3Don"; do _cleanup_qemu done =20 +test_opts=3D"read-only=3Doff read-only=3Don read-only=3Don,force-share=3Do= n" for opt1 in $test_opts; do for opt2 in $test_opts; do echo @@ -170,6 +171,7 @@ for opt1 in $test_opts; do done done =20 +echo echo "=3D=3D Creating ${TEST_IMG}.[abc] =3D=3D" | _filter_testdir ( $QEMU_IMG create -f qcow2 "${TEST_IMG}.a" -b "${TEST_IMG}" diff --git a/tests/qemu-iotests/153.out b/tests/qemu-iotests/153.out index 3492ba7a29..93eaf10486 100644 --- a/tests/qemu-iotests/153.out +++ b/tests/qemu-iotests/153.out @@ -369,6 +369,31 @@ _qemu_img_wrapper bench -U -w -c 1 TEST_DIR/t.qcow2 qemu-img: Could not open 'TEST_DIR/t.qcow2': force-share=3Don can only be = used with read-only images =20 Round done + +=3D=3D Two devices with the same image (read-only=3Doff - read-only=3Doff)= =3D=3D +QEMU_PROG: -drive if=3Dnone,file=3DTEST_DIR/t.qcow2,read-only=3Doff: Faile= d to get "write" lock +Is another process using the image? + +=3D=3D Two devices with the same image (read-only=3Doff - read-only=3Don) = =3D=3D +QEMU_PROG: -drive if=3Dnone,file=3DTEST_DIR/t.qcow2,read-only=3Don: Failed= to get shared "write" lock +Is another process using the image? + +=3D=3D Two devices with the same image (read-only=3Doff - read-only=3Don,f= orce-share=3Don) =3D=3D + +=3D=3D Two devices with the same image (read-only=3Don - read-only=3Doff) = =3D=3D +QEMU_PROG: -drive if=3Dnone,file=3DTEST_DIR/t.qcow2,read-only=3Doff: Faile= d to get "write" lock +Is another process using the image? + +=3D=3D Two devices with the same image (read-only=3Don - read-only=3Don) = =3D=3D + +=3D=3D Two devices with the same image (read-only=3Don - read-only=3Don,fo= rce-share=3Don) =3D=3D + +=3D=3D Two devices with the same image (read-only=3Don,force-share=3Don - = read-only=3Doff) =3D=3D + +=3D=3D Two devices with the same image (read-only=3Don,force-share=3Don - = read-only=3Don) =3D=3D + +=3D=3D Two devices with the same image (read-only=3Don,force-share=3Don - = read-only=3Don,force-share=3Don) =3D=3D + =3D=3D Creating TEST_DIR/t.qcow2.[abc] =3D=3D Formatting 'TEST_DIR/t.IMGFMT.a', fmt=3DIMGFMT size=3D33554432 backing_fil= e=3DTEST_DIR/t.IMGFMT Formatting 'TEST_DIR/t.IMGFMT.b', fmt=3DIMGFMT size=3D33554432 backing_fil= e=3DTEST_DIR/t.IMGFMT --=20 2.13.6 From nobody Tue Nov 4 21:22:00 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531413285325855.8722945764777; Thu, 12 Jul 2018 09:34:45 -0700 (PDT) Received: from localhost ([::1]:32987 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdeY7-0005sp-4L for importer@patchew.org; Thu, 12 Jul 2018 12:34:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56685) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdeVg-0004cQ-E3 for qemu-devel@nongnu.org; Thu, 12 Jul 2018 12:32:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fdeVf-00063I-F3 for qemu-devel@nongnu.org; Thu, 12 Jul 2018 12:32:04 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:54716 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fdeVc-00060S-HU; Thu, 12 Jul 2018 12:32:00 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2430E44D0E; Thu, 12 Jul 2018 16:32:00 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-16.ams2.redhat.com [10.36.117.16]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4CAEC2026D76; Thu, 12 Jul 2018 16:31:59 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 12 Jul 2018 18:31:47 +0200 Message-Id: <20180712163152.12521-3-kwolf@redhat.com> In-Reply-To: <20180712163152.12521-1-kwolf@redhat.com> References: <20180712163152.12521-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Thu, 12 Jul 2018 16:32:00 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Thu, 12 Jul 2018 16:32:00 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 2/7] iotests: nbd: Stop qemu-nbd before remaking image X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Fam Zheng 197 is one example where _make_test_img is used twice without stopping the NBD server in between. An error will occur like this: @@ -26,9 +26,13 @@ =3D=3D=3D Partial final cluster =3D=3D=3D +qemu-img: TEST_DIR/t.IMGFMT: Failed to get "resize" lock +Is another process using the image? Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1024 +Failed to find an available port: Address already in use read 1024/1024 bytes at offset 0 Patch _make_test_img to stop the old qemu-nbd before starting a new one, which fixes this problem, and similarly 215. Signed-off-by: Fam Zheng Signed-off-by: Kevin Wolf --- tests/qemu-iotests/common.rc | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index d054cb97fc..44bee16a5e 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -195,6 +195,16 @@ _use_sample_img() fi } =20 +_stop_nbd_server() +{ + if [ -f "${QEMU_TEST_DIR}/qemu-nbd.pid" ]; then + local QEMU_NBD_PID + read QEMU_NBD_PID < "${QEMU_TEST_DIR}/qemu-nbd.pid" + kill ${QEMU_NBD_PID} + rm -f "${QEMU_TEST_DIR}/qemu-nbd.pid" + fi +} + _make_test_img() { # extra qemu-img options can be added by tests @@ -234,6 +244,10 @@ _make_test_img() extra_img_options=3D"-o $optstr $extra_img_options" fi =20 + if [ $IMGPROTO =3D "nbd" ]; then + _stop_nbd_server + fi + # XXX(hch): have global image options? ( if [ $use_backing =3D 1 ]; then @@ -274,12 +288,7 @@ _cleanup_test_img() case "$IMGPROTO" in =20 nbd) - if [ -f "${QEMU_TEST_DIR}/qemu-nbd.pid" ]; then - local QEMU_NBD_PID - read QEMU_NBD_PID < "${QEMU_TEST_DIR}/qemu-nbd.pid" - kill ${QEMU_NBD_PID} - rm -f "${QEMU_TEST_DIR}/qemu-nbd.pid" - fi + _stop_nbd_server rm -f "$TEST_IMG_FILE" ;; vxhs) --=20 2.13.6 From nobody Tue Nov 4 21:22:00 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531413288000889.9681706373641; Thu, 12 Jul 2018 09:34:48 -0700 (PDT) Received: from localhost ([::1]:32988 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdeYF-0005zI-TQ for importer@patchew.org; Thu, 12 Jul 2018 12:34:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56690) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdeVg-0004cS-Jn for qemu-devel@nongnu.org; Thu, 12 Jul 2018 12:32:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fdeVf-00063b-O4 for qemu-devel@nongnu.org; Thu, 12 Jul 2018 12:32:04 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:40538 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fdeVd-00061X-Kg; Thu, 12 Jul 2018 12:32:01 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3A82D14B398; Thu, 12 Jul 2018 16:32:01 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-16.ams2.redhat.com [10.36.117.16]) by smtp.corp.redhat.com (Postfix) with ESMTP id 624092026D76; Thu, 12 Jul 2018 16:32:00 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 12 Jul 2018 18:31:48 +0200 Message-Id: <20180712163152.12521-4-kwolf@redhat.com> In-Reply-To: <20180712163152.12521-1-kwolf@redhat.com> References: <20180712163152.12521-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Thu, 12 Jul 2018 16:32:01 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Thu, 12 Jul 2018 16:32:01 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 3/7] qemu-img: Document copy offloading implications with -S and -c X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Fam Zheng Explicitly enabling zero detection or compression suppresses copy offloading during convert. Document it. Suggested-by: Kevin Wolf Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi Signed-off-by: Kevin Wolf --- qemu-img.texi | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/qemu-img.texi b/qemu-img.texi index aeb1b9e66c..5853cd18d1 100644 --- a/qemu-img.texi +++ b/qemu-img.texi @@ -96,7 +96,8 @@ will enumerate information about backing files in a disk = image chain. Refer below for further description. =20 @item -c -indicates that target image must be compressed (qcow format only) +indicates that target image must be compressed (qcow format only). If this +option is used, copy offloading will not be attempted. =20 @item -h with or without a command shows help and lists the supported formats @@ -115,7 +116,8 @@ in case both @var{-q} and @var{-p} options are used. indicates the consecutive number of bytes that must contain only zeros for qemu-img to create a sparse image during conversion. This value is rou= nded down to the nearest 512 bytes. You may use the common size suffixes like -@code{k} for kilobytes. +@code{k} for kilobytes. If this option is used, copy offloading will not be +attempted. =20 @item -t @var{cache} specifies the cache mode that should be used with the (destination) file. = See --=20 2.13.6 From nobody Tue Nov 4 21:22:00 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 15314134962971009.6078427204809; Thu, 12 Jul 2018 09:38:16 -0700 (PDT) Received: from localhost ([::1]:33011 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdebb-0008WU-QK for importer@patchew.org; Thu, 12 Jul 2018 12:38:11 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56755) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdeVj-0004fZ-OU for qemu-devel@nongnu.org; Thu, 12 Jul 2018 12:32:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fdeVi-00066Z-Ap for qemu-devel@nongnu.org; Thu, 12 Jul 2018 12:32:07 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:54724 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fdeVe-00062Y-Mg; Thu, 12 Jul 2018 12:32:02 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4ECBC7A7EB; Thu, 12 Jul 2018 16:32:02 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-16.ams2.redhat.com [10.36.117.16]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7761D2026D76; Thu, 12 Jul 2018 16:32:01 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 12 Jul 2018 18:31:49 +0200 Message-Id: <20180712163152.12521-5-kwolf@redhat.com> In-Reply-To: <20180712163152.12521-1-kwolf@redhat.com> References: <20180712163152.12521-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Thu, 12 Jul 2018 16:32:02 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Thu, 12 Jul 2018 16:32:02 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 4/7] file-posix: specify expected filetypes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: John Snow Adjust each caller of raw_open_common to specify if they are expecting host and character devices or not. Tighten expectations of file types upon open in the common code and refuse types that are not expected. This has two effects: (1) Character and block devices are now considered deprecated for the 'file' driver, which expects only S_IFREG, and (2) no file-posix driver (file, host_cdrom, or host_device) can open directories now. I don't think there's a legitimate reason to open directories as if they were files. This prevents QEMU from opening and attempting to probe a directory inode, which can break in exciting ways. One of those ways is lseek on ext4/xfs, which will return 0x7fffffffffffffff as the file size instead of EISDIR. This can coax QEMU into responding with a confusing "file too big" instead of "Hey, that's not a file". See: https://bugs.launchpad.net/qemu/+bug/1739304/ Signed-off-by: John Snow Signed-off-by: Kevin Wolf --- block/file-posix.c | 39 +++++++++++++++++++++++++++++++-------- qemu-doc.texi | 6 ++++++ 2 files changed, 37 insertions(+), 8 deletions(-) diff --git a/block/file-posix.c b/block/file-posix.c index 28824aae65..60af4b3d51 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -438,7 +438,8 @@ static QemuOptsList raw_runtime_opts =3D { }; =20 static int raw_open_common(BlockDriverState *bs, QDict *options, - int bdrv_flags, int open_flags, Error **errp) + int bdrv_flags, int open_flags, + bool device, Error **errp) { BDRVRawState *s =3D bs->opaque; QemuOpts *opts; @@ -585,10 +586,32 @@ static int raw_open_common(BlockDriverState *bs, QDic= t *options, error_setg_errno(errp, errno, "Could not stat file"); goto fail; } - if (S_ISREG(st.st_mode)) { - s->discard_zeroes =3D true; - s->has_fallocate =3D true; + + if (!device) { + if (S_ISBLK(st.st_mode)) { + warn_report("Opening a block device as a file using the '%s' " + "driver is deprecated", bs->drv->format_name); + } else if (S_ISCHR(st.st_mode)) { + warn_report("Opening a character device as a file using the '%= s' " + "driver is deprecated", bs->drv->format_name); + } else if (!S_ISREG(st.st_mode)) { + error_setg(errp, "A regular file was expected by the '%s' driv= er, " + "but something else was given", bs->drv->format_nam= e); + ret =3D -EINVAL; + goto fail; + } else { + s->discard_zeroes =3D true; + s->has_fallocate =3D true; + } + } else { + if (!(S_ISCHR(st.st_mode) || S_ISBLK(st.st_mode))) { + error_setg(errp, "'%s' driver expects either " + "a character or block device", bs->drv->format_name= ); + ret =3D -EINVAL; + goto fail; + } } + if (S_ISBLK(st.st_mode)) { #ifdef BLKDISCARDZEROES unsigned int arg; @@ -641,7 +664,7 @@ static int raw_open(BlockDriverState *bs, QDict *option= s, int flags, BDRVRawState *s =3D bs->opaque; =20 s->type =3D FTYPE_FILE; - return raw_open_common(bs, options, flags, 0, errp); + return raw_open_common(bs, options, flags, 0, false, errp); } =20 typedef enum { @@ -2939,7 +2962,7 @@ hdev_open_Mac_error: =20 s->type =3D FTYPE_FILE; =20 - ret =3D raw_open_common(bs, options, flags, 0, &local_err); + ret =3D raw_open_common(bs, options, flags, 0, true, &local_err); if (ret < 0) { error_propagate(errp, local_err); #if defined(__APPLE__) && defined(__MACH__) @@ -3170,7 +3193,7 @@ static int cdrom_open(BlockDriverState *bs, QDict *op= tions, int flags, s->type =3D FTYPE_CD; =20 /* open will not fail even if no CD is inserted, so add O_NONBLOCK */ - return raw_open_common(bs, options, flags, O_NONBLOCK, errp); + return raw_open_common(bs, options, flags, O_NONBLOCK, true, errp); } =20 static int cdrom_probe_device(const char *filename) @@ -3284,7 +3307,7 @@ static int cdrom_open(BlockDriverState *bs, QDict *op= tions, int flags, =20 s->type =3D FTYPE_CD; =20 - ret =3D raw_open_common(bs, options, flags, 0, &local_err); + ret =3D raw_open_common(bs, options, flags, 0, true, &local_err); if (ret) { error_propagate(errp, local_err); return ret; diff --git a/qemu-doc.texi b/qemu-doc.texi index 0fbf8b108b..1047c407e7 100644 --- a/qemu-doc.texi +++ b/qemu-doc.texi @@ -2969,6 +2969,12 @@ replacement since it is not needed anymore. The @option{-enable-hax} option has been replaced by @option{-accel hax}. Both options have been introduced in QEMU version 2.9.0. =20 +@subsection -drive file=3Djson:@{...@{'driver':'file'@}@} (since 3.0) + +The 'file' driver for drives is no longer appropriate for character or host +devices and will only accept regular files (S_IFREG). The correct driver +for these file types is 'host_cdrom' or 'host_device' as appropriate. + @section QEMU Machine Protocol (QMP) commands =20 @subsection block-dirty-bitmap-add "autoload" parameter (since 2.12.0) --=20 2.13.6 From nobody Tue Nov 4 21:22:00 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531413524405274.9079705280508; Thu, 12 Jul 2018 09:38:44 -0700 (PDT) Received: from localhost ([::1]:33013 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdec7-0000UQ-8B for importer@patchew.org; Thu, 12 Jul 2018 12:38:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56763) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdeVk-0004gY-8y for qemu-devel@nongnu.org; Thu, 12 Jul 2018 12:32:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fdeVj-00066w-06 for qemu-devel@nongnu.org; Thu, 12 Jul 2018 12:32:08 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:47842 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fdeVf-00063M-QN; Thu, 12 Jul 2018 12:32:03 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6585140122CD; Thu, 12 Jul 2018 16:32:03 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-16.ams2.redhat.com [10.36.117.16]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8D36D2026D76; Thu, 12 Jul 2018 16:32:02 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 12 Jul 2018 18:31:50 +0200 Message-Id: <20180712163152.12521-6-kwolf@redhat.com> In-Reply-To: <20180712163152.12521-1-kwolf@redhat.com> References: <20180712163152.12521-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 12 Jul 2018 16:32:03 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 12 Jul 2018 16:32:03 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 5/7] iotests: add test 226 for file driver types X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: John Snow Test that we're rejecting what we ought to for file, host_driver and host_cdrom drivers. Test that we're seeing the deprecated message for block and chardevs on the file driver. Signed-off-by: John Snow Signed-off-by: Kevin Wolf --- tests/qemu-iotests/226 | 66 ++++++++++++++++++++++++++++++++++++++++++= ++++ tests/qemu-iotests/226.out | 26 ++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 93 insertions(+) create mode 100755 tests/qemu-iotests/226 create mode 100644 tests/qemu-iotests/226.out diff --git a/tests/qemu-iotests/226 b/tests/qemu-iotests/226 new file mode 100755 index 0000000000..460aea2fc9 --- /dev/null +++ b/tests/qemu-iotests/226 @@ -0,0 +1,66 @@ +#!/bin/bash +# +# This test covers expected filetypes for the file, host_cdrom and +# host_device drivers. +# +# Copyright (C) 2018 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=3Djsnow@redhat.com + +seq=3D`basename $0` +echo "QA output created by $seq" + +here=3D`pwd` +status=3D1 # failure is the default! + +_cleanup() +{ + rmdir "$TEST_IMG" +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter +. ./common.pattern + +# Generic format, but tests file-protocol specific error handling +_supported_fmt generic +_supported_proto file +_supported_os Linux + +# Create something decidedly not a file, blockdev or chardev... +mkdir "$TEST_IMG" + +for PROTO in "file" "host_device" "host_cdrom"; do + echo + echo "=3D=3D=3D Testing with driver:$PROTO =3D=3D=3D" + echo + echo "=3D=3D Testing RO =3D=3D" + $QEMU_IO -c "open -r -o driver=3D$PROTO,filename=3D$TEST_IMG" 2>&1 | _= filter_imgfmt | _filter_testdir + $QEMU_IO -c "open -r -o driver=3D$PROTO,filename=3D/dev/null" 2>&1 | _= filter_imgfmt + echo "=3D=3D Testing RW =3D=3D" + $QEMU_IO -c "open -o driver=3D$PROTO,filename=3D$TEST_IMG" 2>&1 | _fil= ter_imgfmt | _filter_testdir + $QEMU_IO -c "open -o driver=3D$PROTO,filename=3D/dev/null" 2>&1 | _fil= ter_imgfmt +done + +# success, all done +echo +echo "*** done" +rm -f $seq.full +status=3D0 diff --git a/tests/qemu-iotests/226.out b/tests/qemu-iotests/226.out new file mode 100644 index 0000000000..8c0d060ffc --- /dev/null +++ b/tests/qemu-iotests/226.out @@ -0,0 +1,26 @@ +QA output created by 226 + +=3D=3D=3D Testing with driver:file =3D=3D=3D + +=3D=3D Testing RO =3D=3D +can't open: A regular file was expected by the 'file' driver, but somethin= g else was given +warning: Opening a character device as a file using the 'file' driver is d= eprecated +=3D=3D Testing RW =3D=3D +can't open: Could not open 'TEST_DIR/t.IMGFMT': Is a directory +warning: Opening a character device as a file using the 'file' driver is d= eprecated + +=3D=3D=3D Testing with driver:host_device =3D=3D=3D + +=3D=3D Testing RO =3D=3D +can't open: 'host_device' driver expects either a character or block device +=3D=3D Testing RW =3D=3D +can't open: Could not open 'TEST_DIR/t.IMGFMT': Is a directory + +=3D=3D=3D Testing with driver:host_cdrom =3D=3D=3D + +=3D=3D Testing RO =3D=3D +can't open: 'host_cdrom' driver expects either a character or block device +=3D=3D Testing RW =3D=3D +can't open: Could not open 'TEST_DIR/t.IMGFMT': Is a directory + +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 1c9f679821..68f66259b2 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -223,3 +223,4 @@ 222 rw auto quick 223 rw auto quick 225 rw auto quick +226 auto quick --=20 2.13.6 From nobody Tue Nov 4 21:22:00 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531413296229887.3986175852332; Thu, 12 Jul 2018 09:34:56 -0700 (PDT) Received: from localhost ([::1]:32989 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdeYR-00065w-3b for importer@patchew.org; Thu, 12 Jul 2018 12:34:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56762) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdeVk-0004gV-8b for qemu-devel@nongnu.org; Thu, 12 Jul 2018 12:32:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fdeVj-000678-97 for qemu-devel@nongnu.org; Thu, 12 Jul 2018 12:32:08 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:54728 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fdeVg-00064c-Rs; Thu, 12 Jul 2018 12:32:04 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7B2687A7EB; Thu, 12 Jul 2018 16:32:04 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-16.ams2.redhat.com [10.36.117.16]) by smtp.corp.redhat.com (Postfix) with ESMTP id A35522026D76; Thu, 12 Jul 2018 16:32:03 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 12 Jul 2018 18:31:51 +0200 Message-Id: <20180712163152.12521-7-kwolf@redhat.com> In-Reply-To: <20180712163152.12521-1-kwolf@redhat.com> References: <20180712163152.12521-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Thu, 12 Jul 2018 16:32:04 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Thu, 12 Jul 2018 16:32:04 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 6/7] scsi-disk: Block Device Characteristics emulation fix X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Daniel Henrique Barboza The current BDC VPD page (page 0xb1) is too short. This can be seen running sg_utils: $ sg_vpd --page=3Dbdc /dev/sda Block device characteristics VPD page (SBC): Block device characteristics VPD page length too short=3D8 By the SCSI spec, the expected size of the SBC page is 0x40. There is no telling how the guest will behave with a shorter message - it can ignore it, or worse, make (wrong) assumptions. This patch fixes the emulation by setting the size to 0x40. This is the output of the previous sg_vpd command after applying it: $ sg_vpd --page=3Dbdc /dev/sda -v inquiry cdb: 12 01 b1 00 fc 00 Block device characteristics VPD page (SBC): [PQual=3D0 Peripheral device type: disk] Medium rotation rate is not reported Product type: Not specified WABEREQ=3D0 WACEREQ=3D0 Nominal form factor not reported FUAB=3D0 VBULS=3D0 To improve readability, this patch also adds the VBULS value explictly and add comments on the existing fields we're setting. Signed-off-by: Daniel Henrique Barboza Acked-by: Paolo Bonzini Signed-off-by: Kevin Wolf --- hw/scsi/scsi-disk.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c index d7df357029..5bb390773b 100644 --- a/hw/scsi/scsi-disk.c +++ b/hw/scsi/scsi-disk.c @@ -775,11 +775,12 @@ int scsi_disk_emulate_vpd_page(SCSIRequest *req, uint= 8_t *outbuf) } case 0xb1: /* block device characteristics */ { - buflen =3D 8; + buflen =3D 0x40; outbuf[4] =3D (s->rotation_rate >> 8) & 0xff; outbuf[5] =3D s->rotation_rate & 0xff; - outbuf[6] =3D 0; - outbuf[7] =3D 0; + outbuf[6] =3D 0; /* PRODUCT TYPE */ + outbuf[7] =3D 0; /* WABEREQ | WACEREQ | NOMINAL FORM FACTOR */ + outbuf[8] =3D 0; /* VBULS */ break; } case 0xb2: /* thin provisioning */ --=20 2.13.6 From nobody Tue Nov 4 21:22:00 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1531413655115678.5769695206143; Thu, 12 Jul 2018 09:40:55 -0700 (PDT) Received: from localhost ([::1]:33036 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdee2-0002cZ-TH for importer@patchew.org; Thu, 12 Jul 2018 12:40:42 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56817) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdeVq-0004jw-1y for qemu-devel@nongnu.org; Thu, 12 Jul 2018 12:32:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fdeVo-00069Q-HO for qemu-devel@nongnu.org; Thu, 12 Jul 2018 12:32:14 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:47846 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fdeVh-00065z-W8; Thu, 12 Jul 2018 12:32:06 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9271C40122CD; Thu, 12 Jul 2018 16:32:05 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-16.ams2.redhat.com [10.36.117.16]) by smtp.corp.redhat.com (Postfix) with ESMTP id B9A472026D76; Thu, 12 Jul 2018 16:32:04 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Date: Thu, 12 Jul 2018 18:31:52 +0200 Message-Id: <20180712163152.12521-8-kwolf@redhat.com> In-Reply-To: <20180712163152.12521-1-kwolf@redhat.com> References: <20180712163152.12521-1-kwolf@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 12 Jul 2018 16:32:05 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 12 Jul 2018 16:32:05 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'kwolf@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 7/7] qemu-img: align result of is_allocated_sectors X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Peter Lieven We currently don't enforce that the sparse segments we detect during conver= t are aligned. This leads to unnecessary and costly read-modify-write cycles eith= er internally in Qemu or in the background on the storage device as nearly all modern filesystems or hardware have a 4k alignment internally. This patch modifies is_allocated_sectors so that its *pnum result will alwa= ys end at an alignment boundary. This way all requests will end at an alignment boundary. The start of all requests will also be aligned as long as the res= ults of get_block_status do not lead to an unaligned offset. The number of RMW cycles when converting an example image [1] to a raw devi= ce that has 4k sector size is about 4600 4k read requests to perform a total of abo= ut 15000 write requests. With this path the additional 4600 read requests are elimin= ated while the number of total write requests stays constant. [1] https://cloud-images.ubuntu.com/releases/16.04/release/ubuntu-16.04-ser= ver-cloudimg-amd64-disk1.vmdk Signed-off-by: Peter Lieven Signed-off-by: Kevin Wolf --- qemu-img.c | 44 ++++++++++++++++++++++++++++++++++++++----= -- tests/qemu-iotests/122.out | 18 ++++++++---------- 2 files changed, 46 insertions(+), 16 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index f4074ebf75..4a7ce43dc9 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -1105,11 +1105,15 @@ static int64_t find_nonzero(const uint8_t *buf, int= 64_t n) * * 'pnum' is set to the number of sectors (including and immediately follo= wing * the first one) that are known to be in the same allocated/unallocated s= tate. + * The function will try to align the end offset to alignment boundaries so + * that the request will at least end aligned and consequtive requests will + * also start at an aligned offset. */ -static int is_allocated_sectors(const uint8_t *buf, int n, int *pnum) +static int is_allocated_sectors(const uint8_t *buf, int n, int *pnum, + int64_t sector_num, int alignment) { bool is_zero; - int i; + int i, tail; =20 if (n <=3D 0) { *pnum =3D 0; @@ -1122,6 +1126,23 @@ static int is_allocated_sectors(const uint8_t *buf, = int n, int *pnum) break; } } + + tail =3D (sector_num + i) & (alignment - 1); + if (tail) { + if (is_zero && i <=3D tail) { + /* treat unallocated areas which only consist + * of a small tail as allocated. */ + is_zero =3D false; + } + if (!is_zero) { + /* align up end offset of allocated areas. */ + i +=3D alignment - tail; + i =3D MIN(i, n); + } else { + /* align down end offset of zero areas. */ + i -=3D tail; + } + } *pnum =3D i; return !is_zero; } @@ -1132,7 +1153,7 @@ static int is_allocated_sectors(const uint8_t *buf, i= nt n, int *pnum) * breaking up write requests for only small sparse areas. */ static int is_allocated_sectors_min(const uint8_t *buf, int n, int *pnum, - int min) + int min, int64_t sector_num, int alignment) { int ret; int num_checked, num_used; @@ -1141,7 +1162,7 @@ static int is_allocated_sectors_min(const uint8_t *bu= f, int n, int *pnum, min =3D n; } =20 - ret =3D is_allocated_sectors(buf, n, pnum); + ret =3D is_allocated_sectors(buf, n, pnum, sector_num, alignment); if (!ret) { return ret; } @@ -1149,13 +1170,15 @@ static int is_allocated_sectors_min(const uint8_t *= buf, int n, int *pnum, num_used =3D *pnum; buf +=3D BDRV_SECTOR_SIZE * *pnum; n -=3D *pnum; + sector_num +=3D *pnum; num_checked =3D num_used; =20 while (n > 0) { - ret =3D is_allocated_sectors(buf, n, pnum); + ret =3D is_allocated_sectors(buf, n, pnum, sector_num, alignment); =20 buf +=3D BDRV_SECTOR_SIZE * *pnum; n -=3D *pnum; + sector_num +=3D *pnum; num_checked +=3D *pnum; if (ret) { num_used =3D num_checked; @@ -1560,6 +1583,7 @@ typedef struct ImgConvertState { bool wr_in_order; bool copy_range; int min_sparse; + int alignment; size_t cluster_sectors; size_t buf_sectors; long num_coroutines; @@ -1724,7 +1748,8 @@ static int coroutine_fn convert_co_write(ImgConvertSt= ate *s, int64_t sector_num, * zeroed. */ if (!s->min_sparse || (!s->compressed && - is_allocated_sectors_min(buf, n, &n, s->min_sparse)) || + is_allocated_sectors_min(buf, n, &n, s->min_sparse, + sector_num, s->alignment)) || (s->compressed && !buffer_is_zero(buf, n * BDRV_SECTOR_SIZE))) { @@ -2368,6 +2393,13 @@ static int img_convert(int argc, char **argv) out_bs->bl.pdiscard_alignment >> BDRV_SECTOR_BITS))); =20 + /* try to align the write requests to the destination to avoid unneces= sary + * RMW cycles. */ + s.alignment =3D MAX(pow2floor(s.min_sparse), + DIV_ROUND_UP(out_bs->bl.request_alignment, + BDRV_SECTOR_SIZE)); + assert(is_power_of_2(s.alignment)); + if (skip_create) { int64_t output_sectors =3D blk_nb_sectors(s.target); if (output_sectors < 0) { diff --git a/tests/qemu-iotests/122.out b/tests/qemu-iotests/122.out index 6c7ee1da6c..c576705284 100644 --- a/tests/qemu-iotests/122.out +++ b/tests/qemu-iotests/122.out @@ -194,12 +194,12 @@ wrote 1024/1024 bytes at offset 17408 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) =20 convert -S 4k -[{ "start": 0, "length": 1024, "depth": 0, "zero": false, "data": true, "o= ffset": OFFSET}, -{ "start": 1024, "length": 7168, "depth": 0, "zero": true, "data": false}, -{ "start": 8192, "length": 1024, "depth": 0, "zero": false, "data": true, = "offset": OFFSET}, -{ "start": 9216, "length": 8192, "depth": 0, "zero": true, "data": false}, -{ "start": 17408, "length": 1024, "depth": 0, "zero": false, "data": true,= "offset": OFFSET}, -{ "start": 18432, "length": 67090432, "depth": 0, "zero": true, "data": fa= lse}] +[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "o= ffset": OFFSET}, +{ "start": 4096, "length": 4096, "depth": 0, "zero": true, "data": false}, +{ "start": 8192, "length": 4096, "depth": 0, "zero": false, "data": true, = "offset": OFFSET}, +{ "start": 12288, "length": 4096, "depth": 0, "zero": true, "data": false}, +{ "start": 16384, "length": 4096, "depth": 0, "zero": false, "data": true,= "offset": OFFSET}, +{ "start": 20480, "length": 67088384, "depth": 0, "zero": true, "data": fa= lse}] =20 convert -c -S 4k [{ "start": 0, "length": 1024, "depth": 0, "zero": false, "data": true}, @@ -210,10 +210,8 @@ convert -c -S 4k { "start": 18432, "length": 67090432, "depth": 0, "zero": true, "data": fa= lse}] =20 convert -S 8k -[{ "start": 0, "length": 9216, "depth": 0, "zero": false, "data": true, "o= ffset": OFFSET}, -{ "start": 9216, "length": 8192, "depth": 0, "zero": true, "data": false}, -{ "start": 17408, "length": 1024, "depth": 0, "zero": false, "data": true,= "offset": OFFSET}, -{ "start": 18432, "length": 67090432, "depth": 0, "zero": true, "data": fa= lse}] +[{ "start": 0, "length": 24576, "depth": 0, "zero": false, "data": true, "= offset": OFFSET}, +{ "start": 24576, "length": 67084288, "depth": 0, "zero": true, "data": fa= lse}] =20 convert -c -S 8k [{ "start": 0, "length": 1024, "depth": 0, "zero": false, "data": true}, --=20 2.13.6