From nobody Mon May 6 17:32:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1572602660; cv=none; d=zoho.com; s=zohoarc; b=Q+jjzO7ETo/c1qIzMT17seMdb1yZ4EaRagRjn0Nqmq1lNoQbITEPB+ArVQhL9fOb7ExzWIf3zt8T9hGkETu4m7aNV18hCrM93T6gBhKYneXIDLIYYO1WhDO379EainsP7TEjPJWD1HHIs0ko1Q+KRSGBMv3e3okuA4eI0rTqL8I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1572602660; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=mZbWS4z5zx9Uj24IYu9jd20MrPNOf8ssfnRGYm5Ywpc=; b=WwoObu5DdLdj6kaNidDzSc5hGTrjwYJYIs75zmdFxkBM3yc2LnwTT1R3ugU9M7HKysPjOyN2iOwhofEsHSDLQYYKQ4Pb8EcdnBy0AWd2g1f78AcI46sK8DxkaiotU0BRqqTfCFtJ9kcGM9PBKMcSph46MLJxcXcUqfwjKRrPqIQ= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1572602660148345.8103780944608; Fri, 1 Nov 2019 03:04:20 -0700 (PDT) Received: from localhost ([::1]:37190 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iQTn1-0000Ax-48 for importer@patchew.org; Fri, 01 Nov 2019 06:04:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40365) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iQTjW-00071n-RN for qemu-devel@nongnu.org; Fri, 01 Nov 2019 06:00:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iQTjQ-0004vS-LF for qemu-devel@nongnu.org; Fri, 01 Nov 2019 06:00:39 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:54179 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iQTjQ-0004j1-HD for qemu-devel@nongnu.org; Fri, 01 Nov 2019 06:00:36 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-409-lpwiKjK1OtGHt595ylDJEg-1; Fri, 01 Nov 2019 06:00:32 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 46F021005500; Fri, 1 Nov 2019 10:00:31 +0000 (UTC) Received: from localhost (ovpn-117-28.ams2.redhat.com [10.36.117.28]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BB45E5DA32; Fri, 1 Nov 2019 10:00:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572602433; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mZbWS4z5zx9Uj24IYu9jd20MrPNOf8ssfnRGYm5Ywpc=; b=fa0OeZKRe2Z+qdAE/8PSeLYZc92uvxeW7XCZuOR8VIRSI5PT3JqXpBJZ55jtmc2/h2QB1X usqAT1fof+IGeEHhbWerbTld0+TGwA8TS9M9QqpCUYiJjGSAUKDCdRIc0Gxz9H9H/bj2SI DZeZWvOcryW1CNSgBM1wAXN9pmxgdMk= From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH for-4.2 1/4] Revert "qcow2: skip writing zero buffers to empty COW areas" Date: Fri, 1 Nov 2019 11:00:16 +0100 Message-Id: <20191101100019.9512-2-mreitz@redhat.com> In-Reply-To: <20191101100019.9512-1-mreitz@redhat.com> References: <20191101100019.9512-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: lpwiKjK1OtGHt595ylDJEg-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Anton Nefedov , qemu-devel@nongnu.org, Max Reitz , Vladimir Sementsov-Ogievskiy , Stefan Hajnoczi Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" This reverts commit c8bb23cbdbe32f5c326365e0a82e1b0e68cdcd8a. This commit causes fundamental performance problems on XFS (because fallocate() stalls the AIO pipeline), and as such it is not clear that we should unconditionally enable this behavior. We expect subclusters to alleviate the performance penalty of small writes to newly allocated clusters, so when we get them, the originally intended performance gain may actually no longer be significant. If we want to reintroduce something similar to c8bb23cbdbe, it will require extensive benchmarking on various systems with subclusters enabled. Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 4 +- block/qcow2.h | 6 --- block/qcow2-cluster.c | 2 +- block/qcow2.c | 86 -------------------------------------- block/trace-events | 1 - tests/qemu-iotests/060 | 7 +--- tests/qemu-iotests/060.out | 5 +-- 7 files changed, 4 insertions(+), 107 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index aa97ee2641..f053f15431 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -3304,8 +3304,6 @@ # # @cor_write: a write due to copy-on-read (since 2.11) # -# @cluster_alloc_space: an allocation of file space for a cluster (since 4= .1) -# # @none: triggers once at creation of the blkdebug node (since 4.1) # # Since: 2.9 @@ -3326,7 +3324,7 @@ 'pwritev_rmw_tail', 'pwritev_rmw_after_tail', 'pwritev', 'pwritev_zero', 'pwritev_done', 'empty_image_prepare', 'l1_shrink_write_table', 'l1_shrink_free_l2_clusters', - 'cor_write', 'cluster_alloc_space', 'none'] } + 'cor_write', 'none'] } =20 ## # @BlkdebugIOType: diff --git a/block/qcow2.h b/block/qcow2.h index 601c2e4c82..8166f6e311 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -418,12 +418,6 @@ typedef struct QCowL2Meta */ Qcow2COWRegion cow_end; =20 - /* - * Indicates that COW regions are already handled and do not require - * any more processing. - */ - bool skip_cow; - /** * The I/O vector with the data from the actual guest write request. * If non-NULL, this is meant to be merged together with the data diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 8982b7b762..fbfea8c817 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -809,7 +809,7 @@ static int perform_cow(BlockDriverState *bs, QCowL2Meta= *m) assert(start->nb_bytes + end->nb_bytes <=3D UINT_MAX - data_bytes); assert(start->offset + start->nb_bytes <=3D end->offset); =20 - if ((start->nb_bytes =3D=3D 0 && end->nb_bytes =3D=3D 0) || m->skip_co= w) { + if (start->nb_bytes =3D=3D 0 && end->nb_bytes =3D=3D 0) { return 0; } =20 diff --git a/block/qcow2.c b/block/qcow2.c index 7c18721741..17555cb0a1 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2274,11 +2274,6 @@ static bool merge_cow(uint64_t offset, unsigned byte= s, continue; } =20 - /* If COW regions are handled already, skip this too */ - if (m->skip_cow) { - continue; - } - /* The data (middle) region must be immediately after the * start region */ if (l2meta_cow_start(m) + m->cow_start.nb_bytes !=3D offset) { @@ -2305,81 +2300,6 @@ static bool merge_cow(uint64_t offset, unsigned byte= s, return false; } =20 -static bool is_unallocated(BlockDriverState *bs, int64_t offset, int64_t b= ytes) -{ - int64_t nr; - return !bytes || - (!bdrv_is_allocated_above(bs, NULL, false, offset, bytes, &nr) && - nr =3D=3D bytes); -} - -static bool is_zero_cow(BlockDriverState *bs, QCowL2Meta *m) -{ - /* - * This check is designed for optimization shortcut so it must be - * efficient. - * Instead of is_zero(), use is_unallocated() as it is faster (but not - * as accurate and can result in false negatives). - */ - return is_unallocated(bs, m->offset + m->cow_start.offset, - m->cow_start.nb_bytes) && - is_unallocated(bs, m->offset + m->cow_end.offset, - m->cow_end.nb_bytes); -} - -static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta) -{ - BDRVQcow2State *s =3D bs->opaque; - QCowL2Meta *m; - - if (!(s->data_file->bs->supported_zero_flags & BDRV_REQ_NO_FALLBACK)) { - return 0; - } - - if (bs->encrypted) { - return 0; - } - - for (m =3D l2meta; m !=3D NULL; m =3D m->next) { - int ret; - - if (!m->cow_start.nb_bytes && !m->cow_end.nb_bytes) { - continue; - } - - if (!is_zero_cow(bs, m)) { - continue; - } - - /* - * instead of writing zero COW buffers, - * efficiently zero out the whole clusters - */ - - ret =3D qcow2_pre_write_overlap_check(bs, 0, m->alloc_offset, - m->nb_clusters * s->cluster_si= ze, - true); - if (ret < 0) { - return ret; - } - - BLKDBG_EVENT(bs->file, BLKDBG_CLUSTER_ALLOC_SPACE); - ret =3D bdrv_co_pwrite_zeroes(s->data_file, m->alloc_offset, - m->nb_clusters * s->cluster_size, - BDRV_REQ_NO_FALLBACK); - if (ret < 0) { - if (ret !=3D -ENOTSUP && ret !=3D -EAGAIN) { - return ret; - } - continue; - } - - trace_qcow2_skip_cow(qemu_coroutine_self(), m->offset, m->nb_clust= ers); - m->skip_cow =3D true; - } - return 0; -} - /* * qcow2_co_pwritev_task * Called with s->lock unlocked @@ -2421,12 +2341,6 @@ static coroutine_fn int qcow2_co_pwritev_task(BlockD= riverState *bs, qiov_offset =3D 0; } =20 - /* Try to efficiently initialize the physical space with zeroes */ - ret =3D handle_alloc_space(bs, l2meta); - if (ret < 0) { - goto out_unlocked; - } - /* * If we need to do COW, check if it's possible to merge the * writing of the guest data together with that of the COW regions. diff --git a/block/trace-events b/block/trace-events index 6ba86decca..c615b26d71 100644 --- a/block/trace-events +++ b/block/trace-events @@ -72,7 +72,6 @@ qcow2_writev_done_part(void *co, int cur_bytes) "co %p cu= r_bytes %d" qcow2_writev_data(void *co, uint64_t offset) "co %p offset 0x%" PRIx64 qcow2_pwrite_zeroes_start_req(void *co, int64_t offset, int count) "co %p = offset 0x%" PRIx64 " count %d" qcow2_pwrite_zeroes(void *co, int64_t offset, int count) "co %p offset 0x%= " PRIx64 " count %d" -qcow2_skip_cow(void *co, uint64_t offset, int nb_clusters) "co %p offset 0= x%" PRIx64 " nb_clusters %d" =20 # qcow2-cluster.c qcow2_alloc_clusters_offset(void *co, uint64_t offset, int bytes) "co %p o= ffset 0x%" PRIx64 " bytes %d" diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060 index b91d8321bb..89e911400c 100755 --- a/tests/qemu-iotests/060 +++ b/tests/qemu-iotests/060 @@ -150,15 +150,10 @@ $QEMU_IO -c "$OPEN_RO" -c "read -P 1 0 512" | _filter= _qemu_io echo echo "=3D=3D=3D Testing overlap while COW is in flight =3D=3D=3D" echo -BACKING_IMG=3D$TEST_IMG.base -TEST_IMG=3D$BACKING_IMG _make_test_img 1G - -$QEMU_IO -c 'write 0k 64k' "$BACKING_IMG" | _filter_qemu_io - # compat=3D0.10 is required in order to make the following discard actually # unallocate the sector rather than make it a zero sector - we want COW, a= fter # all. -IMGOPTS=3D'compat=3D0.10' _make_test_img -b "$BACKING_IMG" 1G +IMGOPTS=3D'compat=3D0.10' _make_test_img 1G # Write two clusters, the second one enforces creation of an L2 table after # the first data cluster. $QEMU_IO -c 'write 0k 64k' -c 'write 512M 64k' "$TEST_IMG" | _filter_qemu_= io diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out index 0f6b0658a1..e42bf8c5a9 100644 --- a/tests/qemu-iotests/060.out +++ b/tests/qemu-iotests/060.out @@ -97,10 +97,7 @@ read 512/512 bytes at offset 0 =20 =3D=3D=3D Testing overlap while COW is in flight =3D=3D=3D =20 -Formatting 'TEST_DIR/t.IMGFMT.base', fmt=3DIMGFMT size=3D1073741824 -wrote 65536/65536 bytes at offset 0 -64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 backing_fil= e=3DTEST_DIR/t.IMGFMT.base +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D1073741824 wrote 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 65536/65536 bytes at offset 536870912 --=20 2.21.0 From nobody Mon May 6 17:32:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1572602872; cv=none; d=zoho.com; s=zohoarc; b=HQ5jimVtO73N7RfvaNnSDGvyOLPwDbNuZX8ZQgjY28O8+ipM401K66QLNQtplwwZG7iujiKnI8ciNYzMKjY9MexoKiA1kjy61gLiSkNyo9kpuhC/O6ZK5yR6F7Cu4UF5XD0b+A99STO1g/352w1rdAwguH2T3JTiYewanRLssSU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1572602872; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=tK6nGHtnBLwNSehBsJRlYIHpvmXJp/Go8FcoP4eXX78=; b=op21vluvXIjTWExDEQgSWNgOmsBLmsqBc355xJn5SCanjElhp8l67o/1gns4CPx3iulloxemxMM6SL6KrlsapZmoIRRhyMiq4xCHwJ1TyTa+YGKLAIFHRN0V+5MpugOTKwYg3SuGFgIeTx8aC6TypfAshphYC5v24wEmQDFDoU4= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1572602872845636.3513743015168; Fri, 1 Nov 2019 03:07:52 -0700 (PDT) Received: from localhost ([::1]:37216 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iQTqR-0003nb-M0 for importer@patchew.org; Fri, 01 Nov 2019 06:07:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40377) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iQTjX-000743-2l for qemu-devel@nongnu.org; Fri, 01 Nov 2019 06:00:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iQTjV-0005FX-Gh for qemu-devel@nongnu.org; Fri, 01 Nov 2019 06:00:42 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:33183 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iQTjV-0005Ax-CQ for qemu-devel@nongnu.org; Fri, 01 Nov 2019 06:00:41 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-415-2yKPsUPFNHuyRrIFuM5DRg-1; Fri, 01 Nov 2019 06:00:38 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9BA1C800D49; Fri, 1 Nov 2019 10:00:37 +0000 (UTC) Received: from localhost (ovpn-117-28.ams2.redhat.com [10.36.117.28]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B0DC11001B07; Fri, 1 Nov 2019 10:00:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572602440; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tK6nGHtnBLwNSehBsJRlYIHpvmXJp/Go8FcoP4eXX78=; b=R1sIabNKXst6FGGuTfK8UnjtOlLOiiIYERCrCSLNBSsoYlwn0GYj1u8g/f6gime8Qr3A90 ZVD+OsE62kgacSl6T6x7769RUDRBK6xhKtxP2TyuYI+ppDMG7RmmoteKEmAxZq/vJ5FvAB x+TsfvSsR108ty3sRyuBgzYb99hgedg= From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH for-4.2 2/4] block: Make wait/mark serialising requests public Date: Fri, 1 Nov 2019 11:00:17 +0100 Message-Id: <20191101100019.9512-3-mreitz@redhat.com> In-Reply-To: <20191101100019.9512-1-mreitz@redhat.com> References: <20191101100019.9512-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: 2yKPsUPFNHuyRrIFuM5DRg-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Anton Nefedov , qemu-devel@nongnu.org, Max Reitz , Vladimir Sementsov-Ogievskiy , Stefan Hajnoczi Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Make both bdrv_mark_request_serialising() and bdrv_wait_serialising_requests() public so they can be used from block drivers. Signed-off-by: Max Reitz --- include/block/block_int.h | 3 +++ block/io.c | 24 ++++++++++++------------ 2 files changed, 15 insertions(+), 12 deletions(-) diff --git a/include/block/block_int.h b/include/block/block_int.h index 02dc0034a2..32fa323b63 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -999,6 +999,9 @@ extern unsigned int bdrv_drain_all_count; void bdrv_apply_subtree_drain(BdrvChild *child, BlockDriverState *new_pare= nt); void bdrv_unapply_subtree_drain(BdrvChild *child, BlockDriverState *old_pa= rent); =20 +bool coroutine_fn bdrv_wait_serialising_requests(BdrvTrackedRequest *self); +void bdrv_mark_request_serialising(BdrvTrackedRequest *req, uint64_t align= ); + int get_tmp_filename(char *filename, int size); BlockDriver *bdrv_probe_all(const uint8_t *buf, int buf_size, const char *filename); diff --git a/block/io.c b/block/io.c index 02659f994d..039c0d49c9 100644 --- a/block/io.c +++ b/block/io.c @@ -715,7 +715,7 @@ static void tracked_request_begin(BdrvTrackedRequest *r= eq, qemu_co_mutex_unlock(&bs->reqs_lock); } =20 -static void mark_request_serialising(BdrvTrackedRequest *req, uint64_t ali= gn) +void bdrv_mark_request_serialising(BdrvTrackedRequest *req, uint64_t align) { int64_t overlap_offset =3D req->offset & ~(align - 1); uint64_t overlap_bytes =3D ROUND_UP(req->offset + req->bytes, align) @@ -805,7 +805,7 @@ void bdrv_dec_in_flight(BlockDriverState *bs) bdrv_wakeup(bs); } =20 -static bool coroutine_fn wait_serialising_requests(BdrvTrackedRequest *sel= f) +bool coroutine_fn bdrv_wait_serialising_requests(BdrvTrackedRequest *self) { BlockDriverState *bs =3D self->bs; BdrvTrackedRequest *req; @@ -1437,14 +1437,14 @@ static int coroutine_fn bdrv_aligned_preadv(BdrvChi= ld *child, * with each other for the same cluster. For example, in copy-on-= read * it ensures that the CoR read and write operations are atomic and * guest writes cannot interleave between them. */ - mark_request_serialising(req, bdrv_get_cluster_size(bs)); + bdrv_mark_request_serialising(req, bdrv_get_cluster_size(bs)); } =20 /* BDRV_REQ_SERIALISING is only for write operation */ assert(!(flags & BDRV_REQ_SERIALISING)); =20 if (!(flags & BDRV_REQ_NO_SERIALISING)) { - wait_serialising_requests(req); + bdrv_wait_serialising_requests(req); } =20 if (flags & BDRV_REQ_COPY_ON_READ) { @@ -1841,10 +1841,10 @@ bdrv_co_write_req_prepare(BdrvChild *child, int64_t= offset, uint64_t bytes, assert(!(flags & ~BDRV_REQ_MASK)); =20 if (flags & BDRV_REQ_SERIALISING) { - mark_request_serialising(req, bdrv_get_cluster_size(bs)); + bdrv_mark_request_serialising(req, bdrv_get_cluster_size(bs)); } =20 - waited =3D wait_serialising_requests(req); + waited =3D bdrv_wait_serialising_requests(req); =20 assert(!waited || !req->serialising || is_request_serialising_and_aligned(req)); @@ -2008,8 +2008,8 @@ static int coroutine_fn bdrv_co_do_zero_pwritev(BdrvC= hild *child, =20 padding =3D bdrv_init_padding(bs, offset, bytes, &pad); if (padding) { - mark_request_serialising(req, align); - wait_serialising_requests(req); + bdrv_mark_request_serialising(req, align); + bdrv_wait_serialising_requests(req); =20 bdrv_padding_rmw_read(child, req, &pad, true); =20 @@ -2111,8 +2111,8 @@ int coroutine_fn bdrv_co_pwritev_part(BdrvChild *chil= d, } =20 if (bdrv_pad_request(bs, &qiov, &qiov_offset, &offset, &bytes, &pad)) { - mark_request_serialising(&req, align); - wait_serialising_requests(&req); + bdrv_mark_request_serialising(&req, align); + bdrv_wait_serialising_requests(&req); bdrv_padding_rmw_read(child, &req, &pad, false); } =20 @@ -3205,7 +3205,7 @@ static int coroutine_fn bdrv_co_copy_range_internal( /* BDRV_REQ_SERIALISING is only for write operation */ assert(!(read_flags & BDRV_REQ_SERIALISING)); if (!(read_flags & BDRV_REQ_NO_SERIALISING)) { - wait_serialising_requests(&req); + bdrv_wait_serialising_requests(&req); } =20 ret =3D src->bs->drv->bdrv_co_copy_range_from(src->bs, @@ -3336,7 +3336,7 @@ int coroutine_fn bdrv_co_truncate(BdrvChild *child, i= nt64_t offset, bool exact, * new area, we need to make sure that no write requests are made to it * concurrently or they might be overwritten by preallocation. */ if (new_bytes) { - mark_request_serialising(&req, 1); + bdrv_mark_request_serialising(&req, 1); } if (bs->read_only) { error_setg(errp, "Image is read-only"); --=20 2.21.0 From nobody Mon May 6 17:32:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1572602751; cv=none; d=zoho.com; s=zohoarc; b=BEOV0Y1IV1B5pmJxDI3+obVWeii0E6/ULA8NC7MMqSACKgfLGmKKxNVKF4+a07DBUqz4jaM0HOS5yRtRlF/zugC6Z37VWsr9sGbxtQQXeM2X7TDlLA9I9vq5RbPhoBU5W91xmeUAt8VU8duTJSDkoiZ7xmbP2Y3a2RMgiPczQ9c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1572602751; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=u4OEPj71kqlrt31oYuTE5It4Um95mf+PdKtBZRC/t5U=; b=kMrwQ84QaNHg6WAzrynjE2GBz4WFyMneDLZ024SFYuXYhPUNky6nawmw9PWs+unIXgDMthiMHPqwIbymUybZaw1jyb/AZmeIbWa5QpfkMmeXZu1WCSryx9BT9x1Rt+I85nT6YGr1pRdvOqruMS3Liu4cF9WqOc5oQwUDeSXdASg= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 157260275131776.12640920919489; Fri, 1 Nov 2019 03:05:51 -0700 (PDT) Received: from localhost ([::1]:37192 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iQToU-0001WZ-9v for importer@patchew.org; Fri, 01 Nov 2019 06:05:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40412) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iQTjb-000770-Ra for qemu-devel@nongnu.org; Fri, 01 Nov 2019 06:00:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iQTjZ-0005U0-KG for qemu-devel@nongnu.org; Fri, 01 Nov 2019 06:00:46 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:52744 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iQTjZ-0005NY-CR for qemu-devel@nongnu.org; Fri, 01 Nov 2019 06:00:45 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-345-Doq3Ui9hPLSFppQ5m8WmFA-1; Fri, 01 Nov 2019 06:00:41 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AAC482A3; Fri, 1 Nov 2019 10:00:40 +0000 (UTC) Received: from localhost (ovpn-117-28.ams2.redhat.com [10.36.117.28]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2095B101E812; Fri, 1 Nov 2019 10:00:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572602443; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=u4OEPj71kqlrt31oYuTE5It4Um95mf+PdKtBZRC/t5U=; b=Hq2ZnV6X13Y3LXI38DaRgT35IKIMFa5SnXdKOle4hXDZNWW+mX45i4Jl5r/ylcya5Usxpq tDIQWHHqaIGCvVXOHZq8aMfaIMTy1d4R4mbm0Hphj3wDHSNw7WoDBVFOpy9IxqqgByK7Mb /Ik4ao8Vo7+IolHGAaCtfS7w21GBeE8= From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH for-4.2 3/4] block: Add bdrv_co_get_self_request() Date: Fri, 1 Nov 2019 11:00:18 +0100 Message-Id: <20191101100019.9512-4-mreitz@redhat.com> In-Reply-To: <20191101100019.9512-1-mreitz@redhat.com> References: <20191101100019.9512-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: Doq3Ui9hPLSFppQ5m8WmFA-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Anton Nefedov , qemu-devel@nongnu.org, Max Reitz , Vladimir Sementsov-Ogievskiy , Stefan Hajnoczi Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Signed-off-by: Max Reitz --- include/block/block_int.h | 1 + block/io.c | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+) diff --git a/include/block/block_int.h b/include/block/block_int.h index 32fa323b63..4fc531f9b2 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -1001,6 +1001,7 @@ void bdrv_unapply_subtree_drain(BdrvChild *child, Blo= ckDriverState *old_parent); =20 bool coroutine_fn bdrv_wait_serialising_requests(BdrvTrackedRequest *self); void bdrv_mark_request_serialising(BdrvTrackedRequest *req, uint64_t align= ); +BdrvTrackedRequest *bdrv_co_get_self_request(BlockDriverState *bs); =20 int get_tmp_filename(char *filename, int size); BlockDriver *bdrv_probe_all(const uint8_t *buf, int buf_size, diff --git a/block/io.c b/block/io.c index 039c0d49c9..e9205994a8 100644 --- a/block/io.c +++ b/block/io.c @@ -742,6 +742,24 @@ static bool is_request_serialising_and_aligned(BdrvTra= ckedRequest *req) (req->bytes =3D=3D req->overlap_bytes); } =20 +/** + * Return the tracked request on @bs for the current coroutine, or + * NULL if there is none. + */ +BdrvTrackedRequest *bdrv_co_get_self_request(BlockDriverState *bs) +{ + BdrvTrackedRequest *req; + Coroutine *self =3D qemu_coroutine_self(); + + QLIST_FOREACH(req, &bs->tracked_requests, list) { + if (req->co =3D=3D self) { + return req; + } + } + + return NULL; +} + /** * Round a region to cluster boundaries */ --=20 2.21.0 From nobody Mon May 6 17:32:45 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1572602856; cv=none; d=zoho.com; s=zohoarc; b=KisBMVAGT86Iq1fNyX8/ElmXHT+CkCeaKIaweVB4WwwNXZjdnByD6xGVC3WTeI+ZAG2VKXs0UfeTzleJPSblSF+72igDAcC4eKptQrAwVvAzH9xyPDVDi9sjE3i4UUkwdVabFtoEC3aqhERCMjPjkqlLP+EeBieCUVF8cFlkBFE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1572602856; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CSfDgngGZU4Dl9sNhabdfyOvZuYyfclSjEtzmRhnADY=; b=VxTc2ljjjNXN3Zv+Zf7J1CBItayB9e+LjLMAhAlJCJ5IGN/npglGhFN0XITMQ6N3dKEqmNPQLBloGODf4xd//tEktgEI9t8AjSmNIxExxPcHcGNan8oMbSGSHM/mAVXTV2++9lefR0wsAT8QmPhhZZx8mO/Ee7Lm358slUfHR7M= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1572602856805879.3272520843188; Fri, 1 Nov 2019 03:07:36 -0700 (PDT) Received: from localhost ([::1]:37210 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iQTq8-0003OP-Gq for importer@patchew.org; Fri, 01 Nov 2019 06:07:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40483) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iQTk4-0007Li-GS for qemu-devel@nongnu.org; Fri, 01 Nov 2019 06:01:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iQTk1-0006oq-8v for qemu-devel@nongnu.org; Fri, 01 Nov 2019 06:01:15 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:37162 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iQTjw-00068C-Sm for qemu-devel@nongnu.org; Fri, 01 Nov 2019 06:01:09 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-228-2rhOLzh1P56fZs8QHJF0lw-1; Fri, 01 Nov 2019 06:00:52 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 54FFE2A3; Fri, 1 Nov 2019 10:00:51 +0000 (UTC) Received: from localhost (ovpn-117-28.ams2.redhat.com [10.36.117.28]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3332860C18; Fri, 1 Nov 2019 10:00:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572602456; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CSfDgngGZU4Dl9sNhabdfyOvZuYyfclSjEtzmRhnADY=; b=GttoklKSG2Fu5ECDKsIXM4CkpUN/NvCNS65BXuL4JhRu+3l6iXyTS5uZKxHs+765u4U0dA v4CPnRbSl9n4yfYYZRtfhgTUEvzPhhNODPLXP4DhzS5aJ43Sy8KpVMiqm4/oVpbpfhjAHH tS+a2OnkHSoK0GzhVKr70Rz3L/j/zWU= From: Max Reitz To: qemu-block@nongnu.org Subject: [PATCH for-4.2 4/4] block/file-posix: Let post-EOF fallocate serialize Date: Fri, 1 Nov 2019 11:00:19 +0100 Message-Id: <20191101100019.9512-5-mreitz@redhat.com> In-Reply-To: <20191101100019.9512-1-mreitz@redhat.com> References: <20191101100019.9512-1-mreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: 2rhOLzh1P56fZs8QHJF0lw-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Anton Nefedov , qemu-devel@nongnu.org, Max Reitz , Vladimir Sementsov-Ogievskiy , Stefan Hajnoczi Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" The XFS kernel driver has a bug that may cause data loss when using fallocate() in an I/O path, i.e. writing zeroes. We can work around it by treating post-EOF fallocates as serializing up until infinity (INT64_MAX in practice). Signed-off-by: Max Reitz --- block/file-posix.c | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/block/file-posix.c b/block/file-posix.c index 0b7e904d48..d5460f3e45 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -2721,6 +2721,44 @@ raw_do_pwrite_zeroes(BlockDriverState *bs, int64_t o= ffset, int bytes, RawPosixAIOData acb; ThreadPoolFunc *handler; =20 +#ifdef CONFIG_FALLOCATE + if (offset + bytes > bs->total_sectors * BDRV_SECTOR_SIZE) { + BdrvTrackedRequest *req; + uint64_t end; + + /* + * This is a workaround for a bug in the Linux XFS driver, + * where writes submitted through the AIO interface will be + * discarded if they happen beyond a concurrently running + * fallocate() that increases the file length (i.e., both the + * write and the fallocate() happen beyond the EOF). + * + * To work around it, we extend the tracked request for this + * zero write until INT64_MAX (effectively infinity), and mark + * it as serializing. + * + * We have to enable this workaround for all filesystems and + * AIO modes (not just XFS with aio=3Dnative), because for + * remote filesystems we do not know the host configuration. + * However, this should have no effect as long as no block + * driver submits zero writes beyond the EOF. + */ + + req =3D bdrv_co_get_self_request(bs); + assert(req); + assert(req->type =3D=3D BDRV_TRACKED_WRITE); + assert(req->offset <=3D offset); + assert(req->offset + req->bytes >=3D offset + bytes); + + end =3D INT64_MAX & -(uint64_t)bs->bl.request_alignment; + req->bytes =3D end - req->offset; + req->overlap_bytes =3D req->bytes; + + bdrv_mark_request_serialising(req, bs->bl.request_alignment); + bdrv_wait_serialising_requests(req); + } +#endif + acb =3D (RawPosixAIOData) { .bs =3D bs, .aio_fildes =3D s->fd, --=20 2.21.0