From nobody Sat May 18 05:34:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1568656585; cv=none; d=zoho.com; s=zohoarc; b=W3QdYzpB+JnMzpBRamATQLqBSHYhTBEzEgDs/BDQ1nj8iSef3WIhr8E0LtvDybU8kFBVjofnhttqMLg0AXLBFrN49nLdTXp23sYy6NWNYGf+1ddPJutPRozm0ll29BY2qGNrJ2eTaRPmawrJ0kBphJr/eymnc8/IfZwhEihwVak= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1568656585; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=AbiNGdu+EqU58+G1DqMdKo9HY34DxTjeRUhK1OMcmxM=; b=O6xutRbbwumQ1NeIlStUa60qlbFPK7c2tZwMpXh/bNCdOHNlNJBY89/tKOuW90ogdynSW6q8oQy5qE6YINXq1O3naJIvZiknl3YNvsSLGaIhICpnmYmYTINpTsYJiEZa6A4ardyjASMG5CXrCpobKafwQLmwZLjtEoQ1gSjvDJw= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1568656585956975.1164386866342; Mon, 16 Sep 2019 10:56:25 -0700 (PDT) Received: from localhost ([::1]:38244 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i9vEY-0008LH-7w for importer@patchew.org; Mon, 16 Sep 2019 13:56:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33428) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i9vC9-0006Ys-Eb for qemu-devel@nongnu.org; Mon, 16 Sep 2019 13:53:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1i9vC3-0004ug-6N for qemu-devel@nongnu.org; Mon, 16 Sep 2019 13:53:49 -0400 Received: from relay.sw.ru ([185.231.240.75]:36810) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1i9vBp-0004nd-Sh; Mon, 16 Sep 2019 13:53:30 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1i9vBm-0004Nc-8v; Mon, 16 Sep 2019 20:53:26 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Mon, 16 Sep 2019 20:53:20 +0300 Message-Id: <20190916175324.18478-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190916175324.18478-1-vsementsov@virtuozzo.com> References: <20190916175324.18478-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v5 1/5] qemu-iotests: ignore leaks on failure paths in 026 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Upcoming asynchronous handling of sub-parts of qcow2 requests will change number of leaked clusters and even make it racy. As a preparation, ignore leaks on failure parts in 026. It's not trivial to just grep or substitute qemu-img output for such thing. Instead do better: 3 is a error code of qemu-img check, if only leaks are found. Catch this case and print success output. Suggested-by: Anton Nefedov Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/026 | 6 +-- tests/qemu-iotests/026.out | 80 ++++++++---------------------- tests/qemu-iotests/026.out.nocache | 80 ++++++++---------------------- tests/qemu-iotests/common.rc | 17 +++++++ 4 files changed, 60 insertions(+), 123 deletions(-) diff --git a/tests/qemu-iotests/026 b/tests/qemu-iotests/026 index ffb18ab6b5..3430029ed6 100755 --- a/tests/qemu-iotests/026 +++ b/tests/qemu-iotests/026 @@ -107,7 +107,7 @@ if [ "$event" =3D=3D "l2_load" ]; then $QEMU_IO -c "read $vmstate 0 128k " "$BLKDBG_TEST_IMG" | _filter_qemu_= io fi =20 -_check_test_img 2>&1 | grep -v "refcount=3D1 reference=3D0" +_check_test_img_ignore_leaks 2>&1 | grep -v "refcount=3D1 reference=3D0" =20 done done @@ -152,7 +152,7 @@ echo echo "Event: $event; errno: $errno; imm: $imm; once: $once; write $vmstate" $QEMU_IO -c "write $vmstate 0 64M" "$BLKDBG_TEST_IMG" | _filter_qemu_io =20 -_check_test_img 2>&1 | grep -v "refcount=3D1 reference=3D0" +_check_test_img_ignore_leaks 2>&1 | grep -v "refcount=3D1 reference=3D0" =20 done done @@ -191,7 +191,7 @@ echo echo "Event: $event; errno: $errno; imm: $imm; once: $once" $QEMU_IO -c "write -b 0 64k" "$BLKDBG_TEST_IMG" | _filter_qemu_io =20 -_check_test_img 2>&1 | grep -v "refcount=3D1 reference=3D0" +_check_test_img_ignore_leaks 2>&1 | grep -v "refcount=3D1 reference=3D0" =20 done done diff --git a/tests/qemu-iotests/026.out b/tests/qemu-iotests/026.out index fb89b8480c..ff0817b6f2 100644 --- a/tests/qemu-iotests/026.out +++ b/tests/qemu-iotests/026.out @@ -17,18 +17,14 @@ Event: l1_update; errno: 5; imm: off; once: off; write qemu-io: Failed to flush the L2 table cache: Input/output error qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error - -1 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: l1_update; errno: 5; imm: off; once: off; write -b qemu-io: Failed to flush the L2 table cache: Input/output error qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error - -1 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: l1_update; errno: 28; imm: off; once: on; write @@ -45,18 +41,14 @@ Event: l1_update; errno: 28; imm: off; once: off; write qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -1 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: l1_update; errno: 28; imm: off; once: off; write -b qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -1 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: l2_load; errno: 5; imm: off; once: on; write @@ -137,18 +129,14 @@ Event: l2_update; errno: 5; imm: off; once: off; write qemu-io: Failed to flush the L2 table cache: Input/output error qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error - -127 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: l2_update; errno: 5; imm: off; once: off; write -b qemu-io: Failed to flush the L2 table cache: Input/output error qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error - -127 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: l2_update; errno: 28; imm: off; once: on; write @@ -165,18 +153,14 @@ Event: l2_update; errno: 28; imm: off; once: off; wri= te qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -127 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: l2_update; errno: 28; imm: off; once: off; write -b qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -127 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: l2_alloc_write; errno: 5; imm: off; once: on; write @@ -200,9 +184,7 @@ Event: l2_alloc_write; errno: 5; imm: off; once: off; w= rite -b qemu-io: Failed to flush the L2 table cache: Input/output error qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error - -1 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: l2_alloc_write; errno: 28; imm: off; once: on; write @@ -226,9 +208,7 @@ Event: l2_alloc_write; errno: 28; imm: off; once: off; = write -b qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -1 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: write_aio; errno: 5; imm: off; once: on; write @@ -480,18 +460,14 @@ Event: refblock_alloc_hookup; errno: 28; imm: off; on= ce: off; write qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -55 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: refblock_alloc_hookup; errno: 28; imm: off; once: off; write -b qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -251 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: refblock_alloc_write; errno: 28; imm: off; once: on; write @@ -532,18 +508,14 @@ Event: refblock_alloc_write_blocks; errno: 28; imm: o= ff; once: off; write qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -10 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: refblock_alloc_write_blocks; errno: 28; imm: off; once: off; write = -b qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -23 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: refblock_alloc_write_table; errno: 28; imm: off; once: on; write @@ -560,18 +532,14 @@ Event: refblock_alloc_write_table; errno: 28; imm: of= f; once: off; write qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -10 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: refblock_alloc_write_table; errno: 28; imm: off; once: off; write -b qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -23 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: refblock_alloc_switch_table; errno: 28; imm: off; once: on; write @@ -588,18 +556,14 @@ Event: refblock_alloc_switch_table; errno: 28; imm: o= ff; once: off; write qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -10 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: refblock_alloc_switch_table; errno: 28; imm: off; once: off; write = -b qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -23 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. =20 =3D=3D=3D L1 growth tests =3D=3D=3D =20 @@ -658,9 +622,7 @@ Event: l1_grow_activate_table; errno: 5; imm: off; once= : off qemu-io: Failed to flush the L2 table cache: Input/output error qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error - -96 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 =20 Event: l1_grow_activate_table; errno: 28; imm: off; once: on @@ -672,9 +634,7 @@ Event: l1_grow_activate_table; errno: 28; imm: off; onc= e: off qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -96 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. =20 =3D=3D=3D Avoid cluster leaks after temporary failure =3D=3D=3D =20 diff --git a/tests/qemu-iotests/026.out.nocache b/tests/qemu-iotests/026.ou= t.nocache index 6dda95dfb4..495d013007 100644 --- a/tests/qemu-iotests/026.out.nocache +++ b/tests/qemu-iotests/026.out.nocache @@ -17,18 +17,14 @@ Event: l1_update; errno: 5; imm: off; once: off; write qemu-io: Failed to flush the L2 table cache: Input/output error qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error - -1 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: l1_update; errno: 5; imm: off; once: off; write -b qemu-io: Failed to flush the L2 table cache: Input/output error qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error - -1 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: l1_update; errno: 28; imm: off; once: on; write=20 @@ -45,18 +41,14 @@ Event: l1_update; errno: 28; imm: off; once: off; write qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -1 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: l1_update; errno: 28; imm: off; once: off; write -b qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -1 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: l2_load; errno: 5; imm: off; once: on; write=20 @@ -140,9 +132,7 @@ qemu-io: Failed to flush the L2 table cache: Input/outp= ut error qemu-io: Failed to flush the refcount block cache: Input/output error wrote 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) - -127 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: l2_update; errno: 5; imm: off; once: off; write -b @@ -150,9 +140,7 @@ qemu-io: Failed to flush the L2 table cache: Input/outp= ut error qemu-io: Failed to flush the refcount block cache: Input/output error wrote 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) - -127 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: l2_update; errno: 28; imm: off; once: on; write=20 @@ -172,9 +160,7 @@ qemu-io: Failed to flush the L2 table cache: No space l= eft on device qemu-io: Failed to flush the refcount block cache: No space left on device wrote 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) - -127 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: l2_update; errno: 28; imm: off; once: off; write -b @@ -182,9 +168,7 @@ qemu-io: Failed to flush the L2 table cache: No space l= eft on device qemu-io: Failed to flush the refcount block cache: No space left on device wrote 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) - -127 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: l2_alloc_write; errno: 5; imm: off; once: on; write=20 @@ -208,9 +192,7 @@ Event: l2_alloc_write; errno: 5; imm: off; once: off; w= rite -b qemu-io: Failed to flush the L2 table cache: Input/output error qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error - -1 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: l2_alloc_write; errno: 28; imm: off; once: on; write=20 @@ -234,9 +216,7 @@ Event: l2_alloc_write; errno: 28; imm: off; once: off; = write -b qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -1 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: write_aio; errno: 5; imm: off; once: on; write=20 @@ -488,18 +468,14 @@ Event: refblock_alloc_hookup; errno: 28; imm: off; on= ce: off; write qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -55 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: refblock_alloc_hookup; errno: 28; imm: off; once: off; write -b qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -251 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: refblock_alloc_write; errno: 28; imm: off; once: on; write=20 @@ -540,18 +516,14 @@ Event: refblock_alloc_write_blocks; errno: 28; imm: o= ff; once: off; write qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -10 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: refblock_alloc_write_blocks; errno: 28; imm: off; once: off; write = -b qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -23 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: refblock_alloc_write_table; errno: 28; imm: off; once: on; write=20 @@ -568,18 +540,14 @@ Event: refblock_alloc_write_table; errno: 28; imm: of= f; once: off; write qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -10 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: refblock_alloc_write_table; errno: 28; imm: off; once: off; write -b qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -23 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: refblock_alloc_switch_table; errno: 28; imm: off; once: on; write=20 @@ -596,18 +564,14 @@ Event: refblock_alloc_switch_table; errno: 28; imm: o= ff; once: off; write qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -10 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: refblock_alloc_switch_table; errno: 28; imm: off; once: off; write = -b qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -23 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. =20 =3D=3D=3D L1 growth tests =3D=3D=3D =20 @@ -666,9 +630,7 @@ Event: l1_grow_activate_table; errno: 5; imm: off; once= : off qemu-io: Failed to flush the L2 table cache: Input/output error qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error - -96 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824=20 =20 Event: l1_grow_activate_table; errno: 28; imm: off; once: on @@ -680,9 +642,7 @@ Event: l1_grow_activate_table; errno: 28; imm: off; onc= e: off qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device - -96 leaked clusters were found on the image. -This means waste of disk space, but no harm to data. +No errors were found on the image. =20 =3D=3D=3D Avoid cluster leaks after temporary failure =3D=3D=3D =20 diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index e45cdfa66b..12b4751848 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -405,6 +405,23 @@ _check_test_img() $QEMU_IMG check "$@" -f $IMGFMT "$TEST_IMG" 2>&1 fi ) | _filter_testdir | _filter_qemu_img_check + + # return real qemu_img check status, to analyze in + # _check_test_img_ignore_leaks + return ${PIPESTATUS[0]} +} + +_check_test_img_ignore_leaks() +{ + out=3D$(_check_test_img "$@") + status=3D$? + if [ $status =3D 3 ]; then + # This must correspond to success output in dump_human_image_check= () + echo "No errors were found on the image." + return 0 + fi + echo "$out" + return $status } =20 _img_info() --=20 2.21.0 From nobody Sat May 18 05:34:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1568656587; cv=none; d=zoho.com; s=zohoarc; b=cIULpcQUgtyXbXfrKW3wbgHZwx1tkBtWLnaT1ppW/aFmIO6JEp17q/y9WSZnHoTIGTbsKWoGcThK2DKvo5qYoY6igezKSudDwmSXbY4QZZD7CxinOs4/cEooSdPw5yU+PYELDGMmZeybvLx9DRmK+ol5SVe3/hgHWVG1sUbeLN4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1568656587; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=fB+82SJlp6NCXRlL0F01gDC+6XWyfafJXCXxQGRmzMs=; b=gyqTYXt7PmlDDeKfCPRa7K6MHlAvYMzXKl7i9sN1fzGwPGSuc6rJXBmtRxMXQEAMgl3a/LYCcIYK0cZGNv8BoPVxn9BlvqJutpwbTT8dBa6n4GyHjeZ6/tTCjro2kM0/xLfFkUqpnANZoVPJB5O7S7d6+BZvWnNO5nUVjZMpiag= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1568656587505375.3384555097076; Mon, 16 Sep 2019 10:56:27 -0700 (PDT) Received: from localhost ([::1]:38246 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i9vEd-0008TM-AT for importer@patchew.org; Mon, 16 Sep 2019 13:56:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33377) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i9vC4-0006SY-1q for qemu-devel@nongnu.org; Mon, 16 Sep 2019 13:53:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1i9vC0-0004tP-Hj for qemu-devel@nongnu.org; Mon, 16 Sep 2019 13:53:43 -0400 Received: from relay.sw.ru ([185.231.240.75]:36822) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1i9vBp-0004ng-RN; Mon, 16 Sep 2019 13:53:30 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1i9vBm-0004Nc-CN; Mon, 16 Sep 2019 20:53:26 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Mon, 16 Sep 2019 20:53:21 +0300 Message-Id: <20190916175324.18478-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190916175324.18478-1-vsementsov@virtuozzo.com> References: <20190916175324.18478-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v5 2/5] block: introduce aio task pool X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Common interface for aio task loops. To be used for improving performance of synchronous io loops in qcow2, block-stream, copy-on-read, and may be other places. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- include/block/aio_task.h | 54 +++++++++++++++++ block/aio_task.c | 124 +++++++++++++++++++++++++++++++++++++++ block/Makefile.objs | 2 + 3 files changed, 180 insertions(+) create mode 100644 include/block/aio_task.h create mode 100644 block/aio_task.c diff --git a/include/block/aio_task.h b/include/block/aio_task.h new file mode 100644 index 0000000000..50bc1e1817 --- /dev/null +++ b/include/block/aio_task.h @@ -0,0 +1,54 @@ +/* + * Aio tasks loops + * + * Copyright (c) 2019 Virtuozzo International GmbH. + * + * Permission is hereby granted, free of charge, to any person obtaining a= copy + * of this software and associated documentation files (the "Software"), t= o deal + * in the Software without restriction, including without limitation the r= ights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or se= ll + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included= in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS= OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OT= HER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING= FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS = IN + * THE SOFTWARE. + */ + +#ifndef BLOCK_AIO_TASK_H +#define BLOCK_AIO_TASK_H + +#include "qemu/coroutine.h" + +typedef struct AioTaskPool AioTaskPool; +typedef struct AioTask AioTask; +typedef int coroutine_fn (*AioTaskFunc)(AioTask *task); +struct AioTask { + AioTaskPool *pool; + AioTaskFunc func; + int ret; +}; + +AioTaskPool *coroutine_fn aio_task_pool_new(int max_busy_tasks); +void aio_task_pool_free(AioTaskPool *); + +/* error code of failed task or 0 if all is OK */ +int aio_task_pool_status(AioTaskPool *pool); + +bool aio_task_pool_empty(AioTaskPool *pool); + +/* User provides filled @task, however task->pool will be set automaticall= y */ +void coroutine_fn aio_task_pool_start_task(AioTaskPool *pool, AioTask *tas= k); + +void coroutine_fn aio_task_pool_wait_slot(AioTaskPool *pool); +void coroutine_fn aio_task_pool_wait_one(AioTaskPool *pool); +void coroutine_fn aio_task_pool_wait_all(AioTaskPool *pool); + +#endif /* BLOCK_AIO_TASK_H */ diff --git a/block/aio_task.c b/block/aio_task.c new file mode 100644 index 0000000000..88989fa248 --- /dev/null +++ b/block/aio_task.c @@ -0,0 +1,124 @@ +/* + * Aio tasks loops + * + * Copyright (c) 2019 Virtuozzo International GmbH. + * + * Permission is hereby granted, free of charge, to any person obtaining a= copy + * of this software and associated documentation files (the "Software"), t= o deal + * in the Software without restriction, including without limitation the r= ights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or se= ll + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included= in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS= OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OT= HER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING= FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS = IN + * THE SOFTWARE. + */ + +#include "qemu/osdep.h" +#include "block/aio.h" +#include "block/aio_task.h" + +struct AioTaskPool { + Coroutine *main_co; + int status; + int max_busy_tasks; + int busy_tasks; + bool waiting; +}; + +static void coroutine_fn aio_task_co(void *opaque) +{ + AioTask *task =3D opaque; + AioTaskPool *pool =3D task->pool; + + assert(pool->busy_tasks < pool->max_busy_tasks); + pool->busy_tasks++; + + task->ret =3D task->func(task); + + pool->busy_tasks--; + + if (task->ret < 0 && pool->status =3D=3D 0) { + pool->status =3D task->ret; + } + + g_free(task); + + if (pool->waiting) { + pool->waiting =3D false; + aio_co_wake(pool->main_co); + } +} + +void coroutine_fn aio_task_pool_wait_one(AioTaskPool *pool) +{ + assert(pool->busy_tasks > 0); + assert(qemu_coroutine_self() =3D=3D pool->main_co); + + pool->waiting =3D true; + qemu_coroutine_yield(); + + assert(!pool->waiting); + assert(pool->busy_tasks < pool->max_busy_tasks); +} + +void coroutine_fn aio_task_pool_wait_slot(AioTaskPool *pool) +{ + if (pool->busy_tasks < pool->max_busy_tasks) { + return; + } + + aio_task_pool_wait_one(pool); +} + +void coroutine_fn aio_task_pool_wait_all(AioTaskPool *pool) +{ + while (pool->busy_tasks > 0) { + aio_task_pool_wait_one(pool); + } +} + +void coroutine_fn aio_task_pool_start_task(AioTaskPool *pool, AioTask *tas= k) +{ + aio_task_pool_wait_slot(pool); + + task->pool =3D pool; + qemu_coroutine_enter(qemu_coroutine_create(aio_task_co, task)); +} + +AioTaskPool *coroutine_fn aio_task_pool_new(int max_busy_tasks) +{ + AioTaskPool *pool =3D g_new0(AioTaskPool, 1); + + pool->main_co =3D qemu_coroutine_self(); + pool->max_busy_tasks =3D max_busy_tasks; + + return pool; +} + +void aio_task_pool_free(AioTaskPool *pool) +{ + g_free(pool); +} + +int aio_task_pool_status(AioTaskPool *pool) +{ + if (!pool) { + return 0; /* Sugar for lazy allocation of aio pool */ + } + + return pool->status; +} + +bool aio_task_pool_empty(AioTaskPool *pool) +{ + return pool->busy_tasks =3D=3D 0; +} diff --git a/block/Makefile.objs b/block/Makefile.objs index 35f3bca4d9..c2eb8c8769 100644 --- a/block/Makefile.objs +++ b/block/Makefile.objs @@ -40,6 +40,8 @@ block-obj-y +=3D throttle.o copy-on-read.o =20 block-obj-y +=3D crypto.o =20 +block-obj-y +=3D aio_task.o + common-obj-y +=3D stream.o =20 nfs.o-libs :=3D $(LIBNFS_LIBS) --=20 2.21.0 From nobody Sat May 18 05:34:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1568656742; cv=none; d=zoho.com; s=zohoarc; b=T0RGU1HnEWKNzKUAn14N2g7c40DmPk49ETPHMXfxJwouvc1qxKN/Hw5oY370CP9E/YDC/FhfmbRoQxyn7zt148E875Yex6HRDQkNmq9LGJ7bu471UTBf0txxhqXwdrd7BFjkV36f7PAbyY56wn7FiKMNNoi7xNic1u0Zt80R7GI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1568656742; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=oqFvC5BpLadsBYKFIm5F487tiePLz37WXNYPQwgzLmg=; b=Ft19hTVkP+DrYpR/tdxYu3gANIpgml+JHEv45ybydfEKV5b5RfoY+B5O7IUbzzn+KhM+JMCnpiyfjOLC/mw+LSa4vi46Tui9t75FkngP6jT75mJcvit6TqBj0opKks7c4S34r8FmMCrBYi8khwVfXIJNAFEIc0TFmtATQI24YqE= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1568656742610655.1057193327273; Mon, 16 Sep 2019 10:59:02 -0700 (PDT) Received: from localhost ([::1]:38276 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i9vHB-0002Xv-Ep for importer@patchew.org; Mon, 16 Sep 2019 13:59:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33386) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i9vC5-0006UI-Ge for qemu-devel@nongnu.org; Mon, 16 Sep 2019 13:53:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1i9vC0-0004ta-Um for qemu-devel@nongnu.org; Mon, 16 Sep 2019 13:53:45 -0400 Received: from relay.sw.ru ([185.231.240.75]:36820) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1i9vBp-0004ni-Qs; Mon, 16 Sep 2019 13:53:30 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1i9vBm-0004Nc-HF; Mon, 16 Sep 2019 20:53:26 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Mon, 16 Sep 2019 20:53:22 +0300 Message-Id: <20190916175324.18478-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190916175324.18478-1-vsementsov@virtuozzo.com> References: <20190916175324.18478-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v5 3/5] block/qcow2: refactor qcow2_co_preadv_part X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Further patch will run partial requests of iterations of qcow2_co_preadv in parallel for performance reasons. To prepare for this, separate part which may be parallelized into separate function (qcow2_co_preadv_task). While being here, also separate encrypted clusters reading to own function, like it is done for compressed reading. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2.c | 209 +++++++++++++++++++++++++++----------------------- 1 file changed, 113 insertions(+), 96 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 4d16393e61..6feb169f7c 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1972,17 +1972,117 @@ out: return ret; } =20 +static coroutine_fn int +qcow2_co_preadv_encrypted(BlockDriverState *bs, + uint64_t file_cluster_offset, + uint64_t offset, + uint64_t bytes, + QEMUIOVector *qiov, + uint64_t qiov_offset) +{ + int ret; + BDRVQcow2State *s =3D bs->opaque; + uint8_t *buf; + + assert(bs->encrypted && s->crypto); + assert(bytes <=3D QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); + + /* + * For encrypted images, read everything into a temporary + * contiguous buffer on which the AES functions can work. + * Also, decryption in a separate buffer is better as it + * prevents the guest from learning information about the + * encrypted nature of the virtual disk. + */ + + buf =3D qemu_try_blockalign(s->data_file->bs, bytes); + if (buf =3D=3D NULL) { + return -ENOMEM; + } + + BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO); + ret =3D bdrv_co_pread(s->data_file, + file_cluster_offset + offset_into_cluster(s, offse= t), + bytes, buf, 0); + if (ret < 0) { + goto fail; + } + + assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE)); + assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE)); + if (qcow2_co_decrypt(bs, + file_cluster_offset + offset_into_cluster(s, offs= et), + offset, buf, bytes) < 0) + { + ret =3D -EIO; + goto fail; + } + qemu_iovec_from_buf(qiov, qiov_offset, buf, bytes); + +fail: + qemu_vfree(buf); + + return ret; +} + +static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs, + QCow2ClusterType cluster_type, + uint64_t file_cluster_offset, + uint64_t offset, uint64_t byt= es, + QEMUIOVector *qiov, + size_t qiov_offset) +{ + BDRVQcow2State *s =3D bs->opaque; + int offset_in_cluster =3D offset_into_cluster(s, offset); + + switch (cluster_type) { + case QCOW2_CLUSTER_ZERO_PLAIN: + case QCOW2_CLUSTER_ZERO_ALLOC: + /* Both zero types are handled in qcow2_co_preadv_part */ + g_assert_not_reached(); + + case QCOW2_CLUSTER_UNALLOCATED: + assert(bs->backing); /* otherwise handled in qcow2_co_preadv_part = */ + + BLKDBG_EVENT(bs->file, BLKDBG_READ_BACKING_AIO); + return bdrv_co_preadv_part(bs->backing, offset, bytes, + qiov, qiov_offset, 0); + + case QCOW2_CLUSTER_COMPRESSED: + return qcow2_co_preadv_compressed(bs, file_cluster_offset, + offset, bytes, qiov, qiov_offset= ); + + case QCOW2_CLUSTER_NORMAL: + if ((file_cluster_offset & 511) !=3D 0) { + return -EIO; + } + + if (bs->encrypted) { + return qcow2_co_preadv_encrypted(bs, file_cluster_offset, + offset, bytes, qiov, qiov_off= set); + } + + BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO); + return bdrv_co_preadv_part(s->data_file, + file_cluster_offset + offset_in_cluster, + bytes, qiov, qiov_offset, 0); + + default: + g_assert_not_reached(); + } + + g_assert_not_reached(); +} + static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, uint64_t offset, uint64_t byt= es, QEMUIOVector *qiov, size_t qiov_offset, int flags) { BDRVQcow2State *s =3D bs->opaque; - int offset_in_cluster; int ret; unsigned int cur_bytes; /* number of bytes in current iteration */ uint64_t cluster_offset =3D 0; - uint8_t *cluster_data =3D NULL; =20 while (bytes !=3D 0) { =20 @@ -1997,112 +2097,29 @@ static coroutine_fn int qcow2_co_preadv_part(Block= DriverState *bs, ret =3D qcow2_get_cluster_offset(bs, offset, &cur_bytes, &cluster_= offset); qemu_co_mutex_unlock(&s->lock); if (ret < 0) { - goto fail; + return ret; } =20 - offset_in_cluster =3D offset_into_cluster(s, offset); - - switch (ret) { - case QCOW2_CLUSTER_UNALLOCATED: - - if (bs->backing) { - BLKDBG_EVENT(bs->file, BLKDBG_READ_BACKING_AIO); - ret =3D bdrv_co_preadv_part(bs->backing, offset, cur_bytes, - qiov, qiov_offset, 0); - if (ret < 0) { - goto fail; - } - } else { - /* Note: in this case, no need to wait */ - qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes); - } - break; - - case QCOW2_CLUSTER_ZERO_PLAIN: - case QCOW2_CLUSTER_ZERO_ALLOC: + if (ret =3D=3D QCOW2_CLUSTER_ZERO_PLAIN || + ret =3D=3D QCOW2_CLUSTER_ZERO_ALLOC || + (ret =3D=3D QCOW2_CLUSTER_UNALLOCATED && !bs->backing)) + { qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes); - break; - - case QCOW2_CLUSTER_COMPRESSED: - ret =3D qcow2_co_preadv_compressed(bs, cluster_offset, - offset, cur_bytes, - qiov, qiov_offset); + } else { + ret =3D qcow2_co_preadv_task(bs, ret, + cluster_offset, offset, cur_bytes, + qiov, qiov_offset); if (ret < 0) { - goto fail; - } - - break; - - case QCOW2_CLUSTER_NORMAL: - if ((cluster_offset & 511) !=3D 0) { - ret =3D -EIO; - goto fail; - } - - if (bs->encrypted) { - assert(s->crypto); - - /* - * For encrypted images, read everything into a temporary - * contiguous buffer on which the AES functions can work. - */ - if (!cluster_data) { - cluster_data =3D - qemu_try_blockalign(s->data_file->bs, - QCOW_MAX_CRYPT_CLUSTERS - * s->cluster_size); - if (cluster_data =3D=3D NULL) { - ret =3D -ENOMEM; - goto fail; - } - } - - assert(cur_bytes <=3D QCOW_MAX_CRYPT_CLUSTERS * s->cluster= _size); - - BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO); - ret =3D bdrv_co_pread(s->data_file, - cluster_offset + offset_in_cluster, - cur_bytes, cluster_data, 0); - if (ret < 0) { - goto fail; - } - - assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE)); - assert(QEMU_IS_ALIGNED(cur_bytes, BDRV_SECTOR_SIZE)); - if (qcow2_co_decrypt(bs, cluster_offset + offset_in_cluste= r, - offset, - cluster_data, cur_bytes) < 0) { - ret =3D -EIO; - goto fail; - } - qemu_iovec_from_buf(qiov, qiov_offset, cluster_data, cur_b= ytes); - } else { - BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO); - ret =3D bdrv_co_preadv_part(s->data_file, - cluster_offset + offset_in_clust= er, - cur_bytes, qiov, qiov_offset, 0); - if (ret < 0) { - goto fail; - } + return ret; } - break; - - default: - g_assert_not_reached(); - ret =3D -EIO; - goto fail; } =20 bytes -=3D cur_bytes; offset +=3D cur_bytes; qiov_offset +=3D cur_bytes; } - ret =3D 0; - -fail: - qemu_vfree(cluster_data); =20 - return ret; + return 0; } =20 /* Check if it's possible to merge a write request with the writing of --=20 2.21.0 From nobody Sat May 18 05:34:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1568656743; cv=none; d=zoho.com; s=zohoarc; b=LawX6P2a4ZxSyKu6XXQWctSfmA0kQNQBlbyD55f0YPZuEmvgRMBUk8N9aTO8kd9d/ZdttrgEF+uoBKtaJn/C77QREXUBwSyJ7l5wtNO6fikBCMA27NM/MdA0Li65wT1C42RvT3o3/u9D0KZK02KvFsaEm6v1Z/oRXuwrp6zfYg0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1568656743; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=P9R93ro3IgPa5cOQ/6yu8M0zXbU+eMeq7AHxkaS1Z+s=; b=IJ5P8CBL4BjuEDzKt87XHguaQ+jApYEFQhAsAVV2AIkMha6baK7n1Et17aEbaf5OCC0xfoGZJa33QbxSuW93Lh6BieeVsEScglocBTlDmYTp/TScwCsA+hG5SSVcpE9ljxytRncmIQAGcu6E7HAJ9QF/1QsYm9jSUgh0BEIMOYk= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1568656743150629.9148030739469; Mon, 16 Sep 2019 10:59:03 -0700 (PDT) Received: from localhost ([::1]:38278 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i9vHC-0002YM-05 for importer@patchew.org; Mon, 16 Sep 2019 13:59:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33398) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i9vC6-0006Vw-PU for qemu-devel@nongnu.org; Mon, 16 Sep 2019 13:53:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1i9vC0-0004tK-HL for qemu-devel@nongnu.org; Mon, 16 Sep 2019 13:53:46 -0400 Received: from relay.sw.ru ([185.231.240.75]:36800) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1i9vBp-0004ne-RR; Mon, 16 Sep 2019 13:53:30 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1i9vBm-0004Nc-MQ; Mon, 16 Sep 2019 20:53:26 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Mon, 16 Sep 2019 20:53:23 +0300 Message-Id: <20190916175324.18478-5-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190916175324.18478-1-vsementsov@virtuozzo.com> References: <20190916175324.18478-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v5 4/5] block/qcow2: refactor qcow2_co_pwritev_part X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Similarly to previous commit, prepare for parallelizing write-loop iterations. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2.c | 154 +++++++++++++++++++++++++++++--------------------- 1 file changed, 90 insertions(+), 64 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 6feb169f7c..164b22e4a2 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2242,6 +2242,88 @@ static int handle_alloc_space(BlockDriverState *bs, = QCowL2Meta *l2meta) return 0; } =20 +/* + * qcow2_co_pwritev_task + * Called with s->lock unlocked + * l2meta - if not NULL, qcow2_co_pwritev_task() will consume it. Caller = must + * not use it somehow after qcow2_co_pwritev_task() call + */ +static coroutine_fn int qcow2_co_pwritev_task(BlockDriverState *bs, + uint64_t file_cluster_offset, + uint64_t offset, uint64_t by= tes, + QEMUIOVector *qiov, + uint64_t qiov_offset, + QCowL2Meta *l2meta) +{ + int ret; + BDRVQcow2State *s =3D bs->opaque; + void *crypt_buf =3D NULL; + int offset_in_cluster =3D offset_into_cluster(s, offset); + QEMUIOVector encrypted_qiov; + + if (bs->encrypted) { + assert(s->crypto); + assert(bytes <=3D QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); + crypt_buf =3D qemu_try_blockalign(bs->file->bs, bytes); + if (crypt_buf =3D=3D NULL) { + ret =3D -ENOMEM; + goto out_unlocked; + } + qemu_iovec_to_buf(qiov, qiov_offset, crypt_buf, bytes); + + if (qcow2_co_encrypt(bs, file_cluster_offset + offset_in_cluster, + offset, crypt_buf, bytes) < 0) + { + ret =3D -EIO; + goto out_unlocked; + } + + qemu_iovec_init_buf(&encrypted_qiov, crypt_buf, bytes); + qiov =3D &encrypted_qiov; + qiov_offset =3D 0; + } + + /* Try to efficiently initialize the physical space with zeroes */ + ret =3D handle_alloc_space(bs, l2meta); + if (ret < 0) { + goto out_unlocked; + } + + /* + * If we need to do COW, check if it's possible to merge the + * writing of the guest data together with that of the COW regions. + * If it's not possible (or not necessary) then write the + * guest data now. + */ + if (!merge_cow(offset, bytes, qiov, qiov_offset, l2meta)) { + BLKDBG_EVENT(bs->file, BLKDBG_WRITE_AIO); + trace_qcow2_writev_data(qemu_coroutine_self(), + file_cluster_offset + offset_in_cluster); + ret =3D bdrv_co_pwritev_part(s->data_file, + file_cluster_offset + offset_in_cluster, + bytes, qiov, qiov_offset, 0); + if (ret < 0) { + goto out_unlocked; + } + } + + qemu_co_mutex_lock(&s->lock); + + ret =3D qcow2_handle_l2meta(bs, &l2meta, true); + goto out_locked; + +out_unlocked: + qemu_co_mutex_lock(&s->lock); + +out_locked: + qcow2_handle_l2meta(bs, &l2meta, false); + qemu_co_mutex_unlock(&s->lock); + + qemu_vfree(crypt_buf); + + return ret; +} + static coroutine_fn int qcow2_co_pwritev_part( BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, size_t qiov_offset, int flags) @@ -2251,15 +2333,10 @@ static coroutine_fn int qcow2_co_pwritev_part( int ret; unsigned int cur_bytes; /* number of sectors in current iteration */ uint64_t cluster_offset; - QEMUIOVector encrypted_qiov; - uint64_t bytes_done =3D 0; - uint8_t *cluster_data =3D NULL; QCowL2Meta *l2meta =3D NULL; =20 trace_qcow2_writev_start_req(qemu_coroutine_self(), offset, bytes); =20 - qemu_co_mutex_lock(&s->lock); - while (bytes !=3D 0) { =20 l2meta =3D NULL; @@ -2273,6 +2350,8 @@ static coroutine_fn int qcow2_co_pwritev_part( - offset_in_cluster); } =20 + qemu_co_mutex_lock(&s->lock); + ret =3D qcow2_alloc_cluster_offset(bs, offset, &cur_bytes, &cluster_offset, &l2meta); if (ret < 0) { @@ -2290,73 +2369,20 @@ static coroutine_fn int qcow2_co_pwritev_part( =20 qemu_co_mutex_unlock(&s->lock); =20 - if (bs->encrypted) { - assert(s->crypto); - if (!cluster_data) { - cluster_data =3D qemu_try_blockalign(bs->file->bs, - QCOW_MAX_CRYPT_CLUSTERS - * s->cluster_size); - if (cluster_data =3D=3D NULL) { - ret =3D -ENOMEM; - goto out_unlocked; - } - } - - assert(cur_bytes <=3D QCOW_MAX_CRYPT_CLUSTERS * s->cluster_siz= e); - qemu_iovec_to_buf(qiov, qiov_offset + bytes_done, - cluster_data, cur_bytes); - - if (qcow2_co_encrypt(bs, cluster_offset + offset_in_cluster, o= ffset, - cluster_data, cur_bytes) < 0) { - ret =3D -EIO; - goto out_unlocked; - } - - qemu_iovec_init_buf(&encrypted_qiov, cluster_data, cur_bytes); - } - - /* Try to efficiently initialize the physical space with zeroes */ - ret =3D handle_alloc_space(bs, l2meta); + ret =3D qcow2_co_pwritev_task(bs, cluster_offset, offset, cur_byte= s, + qiov, qiov_offset, l2meta); + l2meta =3D NULL; /* l2meta is consumed by qcow2_co_pwritev_task() = */ if (ret < 0) { - goto out_unlocked; - } - - /* If we need to do COW, check if it's possible to merge the - * writing of the guest data together with that of the COW regions. - * If it's not possible (or not necessary) then write the - * guest data now. */ - if (!merge_cow(offset, cur_bytes, - bs->encrypted ? &encrypted_qiov : qiov, - bs->encrypted ? 0 : qiov_offset + bytes_done, l2met= a)) - { - BLKDBG_EVENT(bs->file, BLKDBG_WRITE_AIO); - trace_qcow2_writev_data(qemu_coroutine_self(), - cluster_offset + offset_in_cluster); - ret =3D bdrv_co_pwritev_part( - s->data_file, cluster_offset + offset_in_cluster, cur_= bytes, - bs->encrypted ? &encrypted_qiov : qiov, - bs->encrypted ? 0 : qiov_offset + bytes_done, 0); - if (ret < 0) { - goto out_unlocked; - } - } - - qemu_co_mutex_lock(&s->lock); - - ret =3D qcow2_handle_l2meta(bs, &l2meta, true); - if (ret) { - goto out_locked; + goto fail_nometa; } =20 bytes -=3D cur_bytes; offset +=3D cur_bytes; - bytes_done +=3D cur_bytes; + qiov_offset +=3D cur_bytes; trace_qcow2_writev_done_part(qemu_coroutine_self(), cur_bytes); } ret =3D 0; - goto out_locked; =20 -out_unlocked: qemu_co_mutex_lock(&s->lock); =20 out_locked: @@ -2364,7 +2390,7 @@ out_locked: =20 qemu_co_mutex_unlock(&s->lock); =20 - qemu_vfree(cluster_data); +fail_nometa: trace_qcow2_writev_done_req(qemu_coroutine_self(), ret); =20 return ret; --=20 2.21.0 From nobody Sat May 18 05:34:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=virtuozzo.com ARC-Seal: i=1; a=rsa-sha256; t=1568656950; cv=none; d=zoho.com; s=zohoarc; b=JBA+i/tlntddrmqgbEFhoGFsIDJJU3LBnM3gP9ARETUrdxDWa0BFKscx+tyHWXN/TODjiKjEIoWyf+Eav/LVOvpNiboXXK4lRPgYOHBwasa4jcqq7FgIHi9bEGKk+TH8O+m95nPUf5RVrZfZSxBSEnVfnNI1DOZMSyYvC7hYhcA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1568656950; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=xwEdEVp20vloxWAyD8ZqCzXp07mfOnX43hDbwacxLNI=; b=Ji/Cz0N1oeNTmxvyLg+JV0GSgFk+1NH3u5/DATQjBrqYahELsbXBBKxL4PoO2YH6rJ7+t8iTGxNwIF85r9QzGMJ8E6bkQJAIS6NMN1i+2Bmh8IqTFrBLXjGmnCpC33pObBs0oBf3lXLi1qrNWJPXxBiR+ccLX1nVpL3K0jfzl/Q= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1568656950671322.02855367232985; Mon, 16 Sep 2019 11:02:30 -0700 (PDT) Received: from localhost ([::1]:38318 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i9vKR-0005PT-SK for importer@patchew.org; Mon, 16 Sep 2019 14:02:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33417) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i9vC8-0006Xu-Ey for qemu-devel@nongnu.org; Mon, 16 Sep 2019 13:53:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1i9vC3-0004v3-Tr for qemu-devel@nongnu.org; Mon, 16 Sep 2019 13:53:48 -0400 Received: from relay.sw.ru ([185.231.240.75]:36812) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1i9vBp-0004nh-RQ; Mon, 16 Sep 2019 13:53:30 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1i9vBm-0004Nc-RW; Mon, 16 Sep 2019 20:53:26 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Mon, 16 Sep 2019 20:53:24 +0300 Message-Id: <20190916175324.18478-6-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190916175324.18478-1-vsementsov@virtuozzo.com> References: <20190916175324.18478-1-vsementsov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v5 5/5] block/qcow2: introduce parallel subrequest handling in read and write X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" It improves performance for fragmented qcow2 images. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 3 ++ block/qcow2.c | 125 ++++++++++++++++++++++++++++++++++++++++----- block/trace-events | 1 + 3 files changed, 117 insertions(+), 12 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index a488d761ff..f51f478e34 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -65,6 +65,9 @@ #define QCOW2_MAX_BITMAPS 65535 #define QCOW2_MAX_BITMAP_DIRECTORY_SIZE (1024 * QCOW2_MAX_BITMAPS) =20 +/* Maximum of parallel sub-request per guest request */ +#define QCOW2_MAX_WORKERS 8 + /* indicate that the refcount of the referenced cluster is exactly one. */ #define QCOW_OFLAG_COPIED (1ULL << 63) /* indicate that the cluster is compressed (they never have the copied fla= g) */ diff --git a/block/qcow2.c b/block/qcow2.c index 164b22e4a2..7961c05783 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -41,6 +41,7 @@ #include "qapi/qobject-input-visitor.h" #include "qapi/qapi-visit-block-core.h" #include "crypto.h" +#include "block/aio_task.h" =20 /* Differences with QCOW: @@ -2025,6 +2026,60 @@ fail: return ret; } =20 +typedef struct Qcow2AioTask { + AioTask task; + + BlockDriverState *bs; + QCow2ClusterType cluster_type; /* only for read */ + uint64_t file_cluster_offset; + uint64_t offset; + uint64_t bytes; + QEMUIOVector *qiov; + uint64_t qiov_offset; + QCowL2Meta *l2meta; /* only for write */ +} Qcow2AioTask; + +static coroutine_fn int qcow2_co_preadv_task_entry(AioTask *task); +static coroutine_fn int qcow2_add_task(BlockDriverState *bs, + AioTaskPool *pool, + AioTaskFunc func, + QCow2ClusterType cluster_type, + uint64_t file_cluster_offset, + uint64_t offset, + uint64_t bytes, + QEMUIOVector *qiov, + size_t qiov_offset, + QCowL2Meta *l2meta) +{ + Qcow2AioTask local_task; + Qcow2AioTask *task =3D pool ? g_new(Qcow2AioTask, 1) : &local_task; + + *task =3D (Qcow2AioTask) { + .task.func =3D func, + .bs =3D bs, + .cluster_type =3D cluster_type, + .qiov =3D qiov, + .file_cluster_offset =3D file_cluster_offset, + .offset =3D offset, + .bytes =3D bytes, + .qiov_offset =3D qiov_offset, + .l2meta =3D l2meta, + }; + + trace_qcow2_add_task(qemu_coroutine_self(), bs, pool, + func =3D=3D qcow2_co_preadv_task_entry ? "read" := "write", + cluster_type, file_cluster_offset, offset, bytes, + qiov, qiov_offset); + + if (!pool) { + return func(&task->task); + } + + aio_task_pool_start_task(pool, &task->task); + + return 0; +} + static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs, QCow2ClusterType cluster_type, uint64_t file_cluster_offset, @@ -2074,18 +2129,28 @@ static coroutine_fn int qcow2_co_preadv_task(BlockD= riverState *bs, g_assert_not_reached(); } =20 +static coroutine_fn int qcow2_co_preadv_task_entry(AioTask *task) +{ + Qcow2AioTask *t =3D container_of(task, Qcow2AioTask, task); + + assert(!t->l2meta); + + return qcow2_co_preadv_task(t->bs, t->cluster_type, t->file_cluster_of= fset, + t->offset, t->bytes, t->qiov, t->qiov_offs= et); +} + static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, uint64_t offset, uint64_t byt= es, QEMUIOVector *qiov, size_t qiov_offset, int flags) { BDRVQcow2State *s =3D bs->opaque; - int ret; + int ret =3D 0; unsigned int cur_bytes; /* number of bytes in current iteration */ uint64_t cluster_offset =3D 0; + AioTaskPool *aio =3D NULL; =20 - while (bytes !=3D 0) { - + while (bytes !=3D 0 && aio_task_pool_status(aio) =3D=3D 0) { /* prepare next request */ cur_bytes =3D MIN(bytes, INT_MAX); if (s->crypto) { @@ -2097,7 +2162,7 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDri= verState *bs, ret =3D qcow2_get_cluster_offset(bs, offset, &cur_bytes, &cluster_= offset); qemu_co_mutex_unlock(&s->lock); if (ret < 0) { - return ret; + goto out; } =20 if (ret =3D=3D QCOW2_CLUSTER_ZERO_PLAIN || @@ -2106,11 +2171,14 @@ static coroutine_fn int qcow2_co_preadv_part(BlockD= riverState *bs, { qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes); } else { - ret =3D qcow2_co_preadv_task(bs, ret, - cluster_offset, offset, cur_bytes, - qiov, qiov_offset); + if (!aio && cur_bytes !=3D bytes) { + aio =3D aio_task_pool_new(QCOW2_MAX_WORKERS); + } + ret =3D qcow2_add_task(bs, aio, qcow2_co_preadv_task_entry, re= t, + cluster_offset, offset, cur_bytes, + qiov, qiov_offset, NULL); if (ret < 0) { - return ret; + goto out; } } =20 @@ -2119,7 +2187,16 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDr= iverState *bs, qiov_offset +=3D cur_bytes; } =20 - return 0; +out: + if (aio) { + aio_task_pool_wait_all(aio); + if (ret =3D=3D 0) { + ret =3D aio_task_pool_status(aio); + } + g_free(aio); + } + + return ret; } =20 /* Check if it's possible to merge a write request with the writing of @@ -2324,6 +2401,17 @@ out_locked: return ret; } =20 +static coroutine_fn int qcow2_co_pwritev_task_entry(AioTask *task) +{ + Qcow2AioTask *t =3D container_of(task, Qcow2AioTask, task); + + assert(!t->cluster_type); + + return qcow2_co_pwritev_task(t->bs, t->file_cluster_offset, + t->offset, t->bytes, t->qiov, t->qiov_off= set, + t->l2meta); +} + static coroutine_fn int qcow2_co_pwritev_part( BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, size_t qiov_offset, int flags) @@ -2334,10 +2422,11 @@ static coroutine_fn int qcow2_co_pwritev_part( unsigned int cur_bytes; /* number of sectors in current iteration */ uint64_t cluster_offset; QCowL2Meta *l2meta =3D NULL; + AioTaskPool *aio =3D NULL; =20 trace_qcow2_writev_start_req(qemu_coroutine_self(), offset, bytes); =20 - while (bytes !=3D 0) { + while (bytes !=3D 0 && aio_task_pool_status(aio) =3D=3D 0) { =20 l2meta =3D NULL; =20 @@ -2369,8 +2458,12 @@ static coroutine_fn int qcow2_co_pwritev_part( =20 qemu_co_mutex_unlock(&s->lock); =20 - ret =3D qcow2_co_pwritev_task(bs, cluster_offset, offset, cur_byte= s, - qiov, qiov_offset, l2meta); + if (!aio && cur_bytes !=3D bytes) { + aio =3D aio_task_pool_new(QCOW2_MAX_WORKERS); + } + ret =3D qcow2_add_task(bs, aio, qcow2_co_pwritev_task_entry, 0, + cluster_offset, offset, cur_bytes, + qiov, qiov_offset, l2meta); l2meta =3D NULL; /* l2meta is consumed by qcow2_co_pwritev_task() = */ if (ret < 0) { goto fail_nometa; @@ -2391,6 +2484,14 @@ out_locked: qemu_co_mutex_unlock(&s->lock); =20 fail_nometa: + if (aio) { + aio_task_pool_wait_all(aio); + if (ret =3D=3D 0) { + ret =3D aio_task_pool_status(aio); + } + g_free(aio); + } + trace_qcow2_writev_done_req(qemu_coroutine_self(), ret); =20 return ret; diff --git a/block/trace-events b/block/trace-events index 04209f058d..3aa27e6b1e 100644 --- a/block/trace-events +++ b/block/trace-events @@ -62,6 +62,7 @@ file_paio_submit(void *acb, void *opaque, int64_t offset,= int count, int type) " file_copy_file_range(void *bs, int src, int64_t src_off, int dst, int64_t = dst_off, int64_t bytes, int flags, int64_t ret) "bs %p src_fd %d offset %"P= RIu64" dst_fd %d offset %"PRIu64" bytes %"PRIu64" flags %d ret %"PRId64 =20 # qcow2.c +qcow2_add_task(void *co, void *bs, void *pool, const char *action, int clu= ster_type, uint64_t file_cluster_offset, uint64_t offset, uint64_t bytes, v= oid *qiov, size_t qiov_offset) "co %p bs %p pool %p: %s: cluster_type %d fi= le_cluster_offset %" PRIu64 " offset %" PRIu64 " bytes %" PRIu64 " qiov %p = qiov_offset %zu" qcow2_writev_start_req(void *co, int64_t offset, int bytes) "co %p offset = 0x%" PRIx64 " bytes %d" qcow2_writev_done_req(void *co, int ret) "co %p ret %d" qcow2_writev_start_part(void *co) "co %p" --=20 2.21.0