From nobody Wed May 15 07:07:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1693396211; cv=none; d=zohomail.com; s=zohoarc; b=dB17aEfwpJqwMu1IaPusc88Z0knuIApm7PKcVY9h/RMzAUf9tleLFu36DyzQ32oU0OJZJiIUs19xsJivIObmTtmGwELirAha12kcm+AD+ZtgHaXOYgqF1wtBCfrkIjT7Gk7jjP4H8HDVvuf7GCnOaUyI4RLDGaNPm6CCV49h2FE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1693396211; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=2mrbL2L5n4dlKXoxPhQ4bdIS4/YMrsyeedolVZYfUKA=; b=bhb2/+hZCnEarfwft8s5p8dLIbaBjn9UMDMaDFkgxMqMlOHtL2bVNjcP/u06v8uJ9eQqFBW0CyMnyhVJcNuqq59Z13YHPYuid3Vm7/oAFpNf1kx7B5SD/69arshrELriTMUTcoyhyJwn7QsLdHforbTiLJvFm9Gj9In3LnOAV3U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1693396211882650.8504202121957; Wed, 30 Aug 2023 04:50:11 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qbJhs-0003Pp-TA; Wed, 30 Aug 2023 07:49:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qbJhr-0003Ms-8O for qemu-devel@nongnu.org; Wed, 30 Aug 2023 07:49:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qbJho-0003ft-TZ for qemu-devel@nongnu.org; Wed, 30 Aug 2023 07:49:54 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-228-E8xyf0fpMY-PthX8a-0lKQ-1; Wed, 30 Aug 2023 07:49:47 -0400 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2068585CCE0; Wed, 30 Aug 2023 11:49:47 +0000 (UTC) Received: from localhost (unknown [10.39.193.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8AD88492C13; Wed, 30 Aug 2023 11:49:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1693396192; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2mrbL2L5n4dlKXoxPhQ4bdIS4/YMrsyeedolVZYfUKA=; b=izlz/w8zbNQ3rp1AJ9YWfHim/l9XlPIDMkysyEl0jbEB5VcJ9iWN+elSpjr7vlW6wtktGg RE+syFVq8dRL+MZ+m53CKVtN4NR4h6tL4MhHeogDoTr6IKSLAVfvHThP97Qt1aGRgfhtHM FnbqR/CCZfptaXgkS5tnk2eOCDRvz3Q= X-MC-Unique: E8xyf0fpMY-PthX8a-0lKQ-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , Stefan Hajnoczi , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Jeuk Kim , qemu-block@nongnu.org, Vladimir Sementsov-Ogievskiy , Leonardo Bras , Laurent Vivier , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Kevin Wolf , Peter Xu , Paolo Bonzini , John Snow , Fam Zheng , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Thomas Huth , Hanna Reitz , Fabiano Rosas Subject: [PULL v3 1/5] block-migration: Ensure we don't crash during migration cleanup Date: Wed, 30 Aug 2023 07:49:38 -0400 Message-ID: <20230830114942.449060-2-stefanha@redhat.com> In-Reply-To: <20230830114942.449060-1-stefanha@redhat.com> References: <20230830114942.449060-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1693396212906100001 Content-Type: text/plain; charset="utf-8" From: Fabiano Rosas We can fail the blk_insert_bs() at init_blk_migration(), leaving the BlkMigDevState without a dirty_bitmap and BlockDriverState. Account for the possibly missing elements when doing cleanup. Fix the following crashes: Thread 1 "qemu-system-x86" received signal SIGSEGV, Segmentation fault. 0x0000555555ec83ef in bdrv_release_dirty_bitmap (bitmap=3D0x0) at ../block/= dirty-bitmap.c:359 359 BlockDriverState *bs =3D bitmap->bs; #0 0x0000555555ec83ef in bdrv_release_dirty_bitmap (bitmap=3D0x0) at ../b= lock/dirty-bitmap.c:359 #1 0x0000555555bba331 in unset_dirty_tracking () at ../migration/block.c:= 371 #2 0x0000555555bbad98 in block_migration_cleanup_bmds () at ../migration/= block.c:681 Thread 1 "qemu-system-x86" received signal SIGSEGV, Segmentation fault. 0x0000555555e971ff in bdrv_op_unblock (bs=3D0x0, op=3DBLOCK_OP_TYPE_BACKUP_= SOURCE, reason=3D0x0) at ../block.c:7073 7073 QLIST_FOREACH_SAFE(blocker, &bs->op_blockers[op], list, next) { #0 0x0000555555e971ff in bdrv_op_unblock (bs=3D0x0, op=3DBLOCK_OP_TYPE_BA= CKUP_SOURCE, reason=3D0x0) at ../block.c:7073 #1 0x0000555555e9734a in bdrv_op_unblock_all (bs=3D0x0, reason=3D0x0) at = ../block.c:7095 #2 0x0000555555bbae13 in block_migration_cleanup_bmds () at ../migration/= block.c:690 Signed-off-by: Fabiano Rosas Message-id: 20230731203338.27581-1-farosas@suse.de Signed-off-by: Stefan Hajnoczi --- migration/block.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/migration/block.c b/migration/block.c index b9580a6c7e..86c2256a2b 100644 --- a/migration/block.c +++ b/migration/block.c @@ -368,7 +368,9 @@ static void unset_dirty_tracking(void) BlkMigDevState *bmds; =20 QSIMPLEQ_FOREACH(bmds, &block_mig_state.bmds_list, entry) { - bdrv_release_dirty_bitmap(bmds->dirty_bitmap); + if (bmds->dirty_bitmap) { + bdrv_release_dirty_bitmap(bmds->dirty_bitmap); + } } } =20 @@ -676,13 +678,18 @@ static int64_t get_remaining_dirty(void) static void block_migration_cleanup_bmds(void) { BlkMigDevState *bmds; + BlockDriverState *bs; AioContext *ctx; =20 unset_dirty_tracking(); =20 while ((bmds =3D QSIMPLEQ_FIRST(&block_mig_state.bmds_list)) !=3D NULL= ) { QSIMPLEQ_REMOVE_HEAD(&block_mig_state.bmds_list, entry); - bdrv_op_unblock_all(blk_bs(bmds->blk), bmds->blocker); + + bs =3D blk_bs(bmds->blk); + if (bs) { + bdrv_op_unblock_all(bs, bmds->blocker); + } error_free(bmds->blocker); =20 /* Save ctx, because bmds->blk can disappear during blk_unref. */ --=20 2.41.0 From nobody Wed May 15 07:07:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1693396250; cv=none; d=zohomail.com; s=zohoarc; b=b96LAqc4Mo/qf4EMng1c8y0QxBiOPvh2A+Pe1Z6rwO4j1lEC2jv6WwdaIPdxC4KbVB/QJJgN+sxIkWSpEiTwycUfAPD2MLpd2Shkno+lCeb7yCYYXtjpITK8fQoXAWDOFaQaxmE5ZYBV9c/0x1pMzXG2WsQrR/izP9WD1k7pVrg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1693396250; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=gKxF0jpSy2jpgUjoIwkX9hfmhlsS/bKIAWxJtzJQZrE=; b=nHiMptb1MncBL1P+/WVfNYKA/DruX2V5ClQ8GvaJ1Pj9/p9k2k/vDq3GUJZ5X3pPks+NjIiIiwX4TVG2SNkCjbdswy6x/6v3pXjw/TnJD2zgg+smDDIY3lJf96LapDbIOjoVZRP3f3gLx04LmeTihkUIRTltDrri6B+mCVdpTiI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 169339625070942.50711036772418; Wed, 30 Aug 2023 04:50:50 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qbJhu-0003Rb-LS; Wed, 30 Aug 2023 07:49:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qbJhs-0003PA-Dj for qemu-devel@nongnu.org; Wed, 30 Aug 2023 07:49:56 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qbJhp-0003gI-W0 for qemu-devel@nongnu.org; Wed, 30 Aug 2023 07:49:56 -0400 Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-367-E06kURzINPOnAd1oidh3hQ-1; Wed, 30 Aug 2023 07:49:50 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6F71A280D588; Wed, 30 Aug 2023 11:49:49 +0000 (UTC) Received: from localhost (unknown [10.39.193.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id 74A1540C6F4C; Wed, 30 Aug 2023 11:49:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1693396193; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gKxF0jpSy2jpgUjoIwkX9hfmhlsS/bKIAWxJtzJQZrE=; b=PdNrFSBrg7LahcneNdzj5Q/pj/5kWXQhVcbpv1vHW+HGwwH7vbjz9AtyyYlk4HszLdMtYw uV8TqZwoSmjbRX29Rx1wLrqZ+cR6FmLgsF5NSUedTnXrMoYVoV7G3SyqiIecAX2VcZXlsH /ilFH9YcY/pcDlaUHPW0rHBlcwdS1WU= X-MC-Unique: E06kURzINPOnAd1oidh3hQ-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , Stefan Hajnoczi , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Jeuk Kim , qemu-block@nongnu.org, Vladimir Sementsov-Ogievskiy , Leonardo Bras , Laurent Vivier , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Kevin Wolf , Peter Xu , Paolo Bonzini , John Snow , Fam Zheng , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Thomas Huth , Hanna Reitz , Andrey Drobyshev , Eric Blake , "Denis V . Lunev" Subject: [PULL v3 2/5] block: add subcluster_size field to BlockDriverInfo Date: Wed, 30 Aug 2023 07:49:39 -0400 Message-ID: <20230830114942.449060-3-stefanha@redhat.com> In-Reply-To: <20230830114942.449060-1-stefanha@redhat.com> References: <20230830114942.449060-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1693396251696100001 Content-Type: text/plain; charset="utf-8" From: Andrey Drobyshev This is going to be used in the subsequent commit as requests alignment (in particular, during copy-on-read). This value only makes sense for the formats which support subclusters (currently QCOW2 only). If this field isn't set by driver's own bdrv_get_info() implementation, we simply set it equal to the cluster size thus treating each cluster as having a single subcluster. Reviewed-by: Eric Blake Reviewed-by: Denis V. Lunev Signed-off-by: Andrey Drobyshev Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Stefan Hajnoczi Message-ID: <20230711172553.234055-2-andrey.drobyshev@virtuozzo.com> --- include/block/block-common.h | 5 +++++ block.c | 7 +++++++ block/qcow2.c | 1 + 3 files changed, 13 insertions(+) diff --git a/include/block/block-common.h b/include/block/block-common.h index e15395f2cb..df5ffc8d09 100644 --- a/include/block/block-common.h +++ b/include/block/block-common.h @@ -132,6 +132,11 @@ typedef struct BlockZoneWps { typedef struct BlockDriverInfo { /* in bytes, 0 if irrelevant */ int cluster_size; + /* + * A fraction of cluster_size, if supported (currently QCOW2 only); if + * disabled or unsupported, set equal to cluster_size. + */ + int subcluster_size; /* offset at which the VM state can be saved (0 if not possible) */ int64_t vm_state_offset; bool is_dirty; diff --git a/block.c b/block.c index a307c151a8..0af890f647 100644 --- a/block.c +++ b/block.c @@ -6480,6 +6480,13 @@ int coroutine_fn bdrv_co_get_info(BlockDriverState *= bs, BlockDriverInfo *bdi) } memset(bdi, 0, sizeof(*bdi)); ret =3D drv->bdrv_co_get_info(bs, bdi); + if (bdi->subcluster_size =3D=3D 0) { + /* + * If the driver left this unset, subclusters are not supported. + * Then it is safe to treat each cluster as having only one subclu= ster. + */ + bdi->subcluster_size =3D bdi->cluster_size; + } if (ret < 0) { return ret; } diff --git a/block/qcow2.c b/block/qcow2.c index c51388e99d..b48cd9ce63 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -5197,6 +5197,7 @@ qcow2_co_get_info(BlockDriverState *bs, BlockDriverIn= fo *bdi) { BDRVQcow2State *s =3D bs->opaque; bdi->cluster_size =3D s->cluster_size; + bdi->subcluster_size =3D s->subcluster_size; bdi->vm_state_offset =3D qcow2_vm_state_offset(s); bdi->is_dirty =3D s->incompatible_features & QCOW2_INCOMPAT_DIRTY; return 0; --=20 2.41.0 From nobody Wed May 15 07:07:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1693396230; cv=none; d=zohomail.com; s=zohoarc; b=glrI5dTEUw/z8t2Msrgf1lI43SKNYJJvr05gjzgEmvIv1Nsv1LVcpPE7rFCrCP+33DEwNw8+x6yeV64vcAS+Cu5VonC1oe0gdQ+7dh0LMnH2lBXvsxl4y7TTQMTGKOo2qdFra+Rd/AA0pAkcXjmlXeN8kjmRqLzxEyISozY2zIo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1693396230; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7BekLkQHdNl+nON36AsTjo2RLArJ5w0AroGSMvtVbtI=; b=hHLrFL/qSzLTCud06UBYNnMYORzvHMz60Z2dNxdRpy77mLxLVl1Ck5kMToOOS/VBFP7djG2koQUiBoESUTHWaWEXdJwWUBed4M0Y3qmccNTo0VI516rZyUkuh8IL9qHVgIVsLf0R7OwhUwgsrqGPyXtOFRoRqEex4RqdxgNir58= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1693396230324964.1028927406065; Wed, 30 Aug 2023 04:50:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qbJhy-0003VG-5E; Wed, 30 Aug 2023 07:50:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qbJhw-0003Sk-Ez for qemu-devel@nongnu.org; Wed, 30 Aug 2023 07:50:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qbJhu-0003ii-0r for qemu-devel@nongnu.org; Wed, 30 Aug 2023 07:50:00 -0400 Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-340-48Nm6KIxN32zGDVVvn0LWg-1; Wed, 30 Aug 2023 07:49:52 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A5FD61C25A6C; Wed, 30 Aug 2023 11:49:51 +0000 (UTC) Received: from localhost (unknown [10.39.193.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id 00D7F2026D35; Wed, 30 Aug 2023 11:49:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1693396197; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7BekLkQHdNl+nON36AsTjo2RLArJ5w0AroGSMvtVbtI=; b=deJtnQqdlMMfKLl3+dgRdGtDGd6A+AVkGP8shi4WWxhtIbdZZzmL9uidPMMXG4QMvotB1T Qgg500kmPPbBS839TaxGUDqBLydt1QOsegfXgff+UFTuWkOfUGzD0U9xMyDzmNvGpWukPG 72ZsatNIrCuiUhY4RDC0VjD+0g+lzIU= X-MC-Unique: 48Nm6KIxN32zGDVVvn0LWg-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , Stefan Hajnoczi , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Jeuk Kim , qemu-block@nongnu.org, Vladimir Sementsov-Ogievskiy , Leonardo Bras , Laurent Vivier , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Kevin Wolf , Peter Xu , Paolo Bonzini , John Snow , Fam Zheng , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Thomas Huth , Hanna Reitz , Andrey Drobyshev , Eric Blake , "Denis V . Lunev" Subject: [PULL v3 3/5] block/io: align requests to subcluster_size Date: Wed, 30 Aug 2023 07:49:40 -0400 Message-ID: <20230830114942.449060-4-stefanha@redhat.com> In-Reply-To: <20230830114942.449060-1-stefanha@redhat.com> References: <20230830114942.449060-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1693396231140100001 Content-Type: text/plain; charset="utf-8" From: Andrey Drobyshev When target image is using subclusters, and we align the request during copy-on-read, it makes sense to align to subcluster_size rather than cluster_size. Otherwise we end up with unnecessary allocations. This commit renames bdrv_round_to_clusters() to bdrv_round_to_subclusters() and utilizes subcluster_size field of BlockDriverInfo to make necessary alignments. It affects copy-on-read as well as mirror job (which is using bdrv_round_to_clusters()). This change also fixes the following bug with failing assert (covered by the test in the subsequent commit): qemu-img create -f qcow2 base.qcow2 64K qemu-img create -f qcow2 -o extended_l2=3Don,backing_file=3Dbase.qcow2,back= ing_fmt=3Dqcow2 img.qcow2 64K qemu-io -c "write -P 0xaa 0 2K" img.qcow2 qemu-io -C -c "read -P 0x00 2K 62K" img.qcow2 qemu-io: ../block/io.c:1236: bdrv_co_do_copy_on_readv: Assertion `skip_byte= s < pnum' failed. Reviewed-by: Eric Blake Reviewed-by: Denis V. Lunev Signed-off-by: Andrey Drobyshev Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Stefan Hajnoczi Message-ID: <20230711172553.234055-3-andrey.drobyshev@virtuozzo.com> --- include/block/block-io.h | 8 +++---- block/io.c | 50 ++++++++++++++++++++-------------------- block/mirror.c | 8 +++---- 3 files changed, 33 insertions(+), 33 deletions(-) diff --git a/include/block/block-io.h b/include/block/block-io.h index 4415506e40..6db48f2d35 100644 --- a/include/block/block-io.h +++ b/include/block/block-io.h @@ -189,10 +189,10 @@ bdrv_get_info(BlockDriverState *bs, BlockDriverInfo *= bdi); ImageInfoSpecific *bdrv_get_specific_info(BlockDriverState *bs, Error **errp); BlockStatsSpecific *bdrv_get_specific_stats(BlockDriverState *bs); -void bdrv_round_to_clusters(BlockDriverState *bs, - int64_t offset, int64_t bytes, - int64_t *cluster_offset, - int64_t *cluster_bytes); +void bdrv_round_to_subclusters(BlockDriverState *bs, + int64_t offset, int64_t bytes, + int64_t *cluster_offset, + int64_t *cluster_bytes); =20 void bdrv_get_backing_filename(BlockDriverState *bs, char *filename, int filename_size); diff --git a/block/io.c b/block/io.c index 055fcf7438..76e7df18d8 100644 --- a/block/io.c +++ b/block/io.c @@ -728,21 +728,21 @@ BdrvTrackedRequest *coroutine_fn bdrv_co_get_self_req= uest(BlockDriverState *bs) } =20 /** - * Round a region to cluster boundaries + * Round a region to subcluster (if supported) or cluster boundaries */ void coroutine_fn GRAPH_RDLOCK -bdrv_round_to_clusters(BlockDriverState *bs, int64_t offset, int64_t bytes, - int64_t *cluster_offset, int64_t *cluster_bytes) +bdrv_round_to_subclusters(BlockDriverState *bs, int64_t offset, int64_t by= tes, + int64_t *align_offset, int64_t *align_bytes) { BlockDriverInfo bdi; IO_CODE(); - if (bdrv_co_get_info(bs, &bdi) < 0 || bdi.cluster_size =3D=3D 0) { - *cluster_offset =3D offset; - *cluster_bytes =3D bytes; + if (bdrv_co_get_info(bs, &bdi) < 0 || bdi.subcluster_size =3D=3D 0) { + *align_offset =3D offset; + *align_bytes =3D bytes; } else { - int64_t c =3D bdi.cluster_size; - *cluster_offset =3D QEMU_ALIGN_DOWN(offset, c); - *cluster_bytes =3D QEMU_ALIGN_UP(offset - *cluster_offset + bytes,= c); + int64_t c =3D bdi.subcluster_size; + *align_offset =3D QEMU_ALIGN_DOWN(offset, c); + *align_bytes =3D QEMU_ALIGN_UP(offset - *align_offset + bytes, c); } } =20 @@ -1168,8 +1168,8 @@ bdrv_co_do_copy_on_readv(BdrvChild *child, int64_t of= fset, int64_t bytes, void *bounce_buffer =3D NULL; =20 BlockDriver *drv =3D bs->drv; - int64_t cluster_offset; - int64_t cluster_bytes; + int64_t align_offset; + int64_t align_bytes; int64_t skip_bytes; int ret; int max_transfer =3D MIN_NON_ZERO(bs->bl.max_transfer, @@ -1203,28 +1203,28 @@ bdrv_co_do_copy_on_readv(BdrvChild *child, int64_t = offset, int64_t bytes, * BDRV_REQUEST_MAX_BYTES (even when the original read did not), which * is one reason we loop rather than doing it all at once. */ - bdrv_round_to_clusters(bs, offset, bytes, &cluster_offset, &cluster_by= tes); - skip_bytes =3D offset - cluster_offset; + bdrv_round_to_subclusters(bs, offset, bytes, &align_offset, &align_byt= es); + skip_bytes =3D offset - align_offset; =20 trace_bdrv_co_do_copy_on_readv(bs, offset, bytes, - cluster_offset, cluster_bytes); + align_offset, align_bytes); =20 - while (cluster_bytes) { + while (align_bytes) { int64_t pnum; =20 if (skip_write) { ret =3D 1; /* "already allocated", so nothing will be copied */ - pnum =3D MIN(cluster_bytes, max_transfer); + pnum =3D MIN(align_bytes, max_transfer); } else { - ret =3D bdrv_is_allocated(bs, cluster_offset, - MIN(cluster_bytes, max_transfer), &pnu= m); + ret =3D bdrv_is_allocated(bs, align_offset, + MIN(align_bytes, max_transfer), &pnum); if (ret < 0) { /* * Safe to treat errors in querying allocation as if * unallocated; we'll probably fail again soon on the * read, but at least that will set a decent errno. */ - pnum =3D MIN(cluster_bytes, max_transfer); + pnum =3D MIN(align_bytes, max_transfer); } =20 /* Stop at EOF if the image ends in the middle of the cluster = */ @@ -1242,7 +1242,7 @@ bdrv_co_do_copy_on_readv(BdrvChild *child, int64_t of= fset, int64_t bytes, /* Must copy-on-read; use the bounce buffer */ pnum =3D MIN(pnum, MAX_BOUNCE_BUFFER); if (!bounce_buffer) { - int64_t max_we_need =3D MAX(pnum, cluster_bytes - pnum); + int64_t max_we_need =3D MAX(pnum, align_bytes - pnum); int64_t max_allowed =3D MIN(max_transfer, MAX_BOUNCE_BUFFE= R); int64_t bounce_buffer_len =3D MIN(max_we_need, max_allowed= ); =20 @@ -1254,7 +1254,7 @@ bdrv_co_do_copy_on_readv(BdrvChild *child, int64_t of= fset, int64_t bytes, } qemu_iovec_init_buf(&local_qiov, bounce_buffer, pnum); =20 - ret =3D bdrv_driver_preadv(bs, cluster_offset, pnum, + ret =3D bdrv_driver_preadv(bs, align_offset, pnum, &local_qiov, 0, 0); if (ret < 0) { goto err; @@ -1266,13 +1266,13 @@ bdrv_co_do_copy_on_readv(BdrvChild *child, int64_t = offset, int64_t bytes, /* FIXME: Should we (perhaps conditionally) be setting * BDRV_REQ_MAY_UNMAP, if it will allow for a sparser copy * that still correctly reads as zero? */ - ret =3D bdrv_co_do_pwrite_zeroes(bs, cluster_offset, pnum, + ret =3D bdrv_co_do_pwrite_zeroes(bs, align_offset, pnum, BDRV_REQ_WRITE_UNCHANGED); } else { /* This does not change the data on the disk, it is not * necessary to flush even in cache=3Dwritethrough mode. */ - ret =3D bdrv_driver_pwritev(bs, cluster_offset, pnum, + ret =3D bdrv_driver_pwritev(bs, align_offset, pnum, &local_qiov, 0, BDRV_REQ_WRITE_UNCHANGED); } @@ -1301,8 +1301,8 @@ bdrv_co_do_copy_on_readv(BdrvChild *child, int64_t of= fset, int64_t bytes, } } =20 - cluster_offset +=3D pnum; - cluster_bytes -=3D pnum; + align_offset +=3D pnum; + align_bytes -=3D pnum; progress +=3D pnum - skip_bytes; skip_bytes =3D 0; } diff --git a/block/mirror.c b/block/mirror.c index d3cacd1708..e213a892db 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -283,8 +283,8 @@ static int coroutine_fn mirror_cow_align(MirrorBlockJob= *s, int64_t *offset, need_cow |=3D !test_bit((*offset + *bytes - 1) / s->granularity, s->cow_bitmap); if (need_cow) { - bdrv_round_to_clusters(blk_bs(s->target), *offset, *bytes, - &align_offset, &align_bytes); + bdrv_round_to_subclusters(blk_bs(s->target), *offset, *bytes, + &align_offset, &align_bytes); } =20 if (align_bytes > max_bytes) { @@ -576,8 +576,8 @@ static void coroutine_fn mirror_iteration(MirrorBlockJo= b *s) int64_t target_offset; int64_t target_bytes; WITH_GRAPH_RDLOCK_GUARD() { - bdrv_round_to_clusters(blk_bs(s->target), offset, io_bytes, - &target_offset, &target_bytes); + bdrv_round_to_subclusters(blk_bs(s->target), offset, io_by= tes, + &target_offset, &target_bytes); } if (target_offset =3D=3D offset && target_bytes =3D=3D io_bytes) { --=20 2.41.0 From nobody Wed May 15 07:07:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1693396253; cv=none; d=zohomail.com; s=zohoarc; b=KMnqg7p4mf2IwQ7wmigF18ma4NrCeYlSsRMCOi+nMs8lc6jkROqgwGqAyQPZVXJpNeEN16MwJnY+bUPYef1NANuIBuyTh5a/szs3U82cGnZCrGN0oP8Yzt8W4XzwGhzjUBkbU1RxMzm30o7P+4FF3onNglku0kblLcjqzf8bZnM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1693396253; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Ga6qSUQdwzhmfn/lQ9lhh6t3Yiayp8nRSobjZ382ohU=; b=GNd9jg+yI6rmW0nt1w7sAHi39+T9OpqFaHnBMYMxvqjArhH8IgPBf8R8xzWGMA1p92hwHdxOWLLMrvT8t7VdpGNzEmtt8vzxBKO8BExaN6b2PhzR9NpaF3ZRnePE9jDgigGdluDX9Gbpu1Jvn6SQ+3YdjthHU9ig416CZ3OXEDg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1693396252989800.9793787187357; Wed, 30 Aug 2023 04:50:52 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qbJi0-0003Zt-FY; Wed, 30 Aug 2023 07:50:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qbJhx-0003T2-8H for qemu-devel@nongnu.org; Wed, 30 Aug 2023 07:50:01 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qbJhv-0003iz-0W for qemu-devel@nongnu.org; Wed, 30 Aug 2023 07:50:00 -0400 Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-613-s8R2zKAEPTC61o899lSztQ-1; Wed, 30 Aug 2023 07:49:54 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 87D1F381C166; Wed, 30 Aug 2023 11:49:53 +0000 (UTC) Received: from localhost (unknown [10.39.193.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0095E40C6F4C; Wed, 30 Aug 2023 11:49:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1693396198; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ga6qSUQdwzhmfn/lQ9lhh6t3Yiayp8nRSobjZ382ohU=; b=SsVF7tZAtHz99Lh+WLZoxtdMafuNvbcAkpQSGBgZCwUvkrOs36UdlDaZqnC5Ws3h/HVMmF j2aKdnBvLToUM5cA9+kM8HH7gKfs3Z4l01fCSgs7XSOXFDzlm/ns0Yh3g+gvWsCKBWaXaD u0TmQA7v4HBhy2Gg5aNdoJrg6I01fgU= X-MC-Unique: s8R2zKAEPTC61o899lSztQ-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , Stefan Hajnoczi , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Jeuk Kim , qemu-block@nongnu.org, Vladimir Sementsov-Ogievskiy , Leonardo Bras , Laurent Vivier , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Kevin Wolf , Peter Xu , Paolo Bonzini , John Snow , Fam Zheng , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Thomas Huth , Hanna Reitz , Andrey Drobyshev , Eric Blake , "Denis V . Lunev" Subject: [PULL v3 4/5] tests/qemu-iotests/197: add testcase for CoR with subclusters Date: Wed, 30 Aug 2023 07:49:41 -0400 Message-ID: <20230830114942.449060-5-stefanha@redhat.com> In-Reply-To: <20230830114942.449060-1-stefanha@redhat.com> References: <20230830114942.449060-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1693396254584100007 Content-Type: text/plain; charset="utf-8" From: Andrey Drobyshev Add testcase which checks that allocations during copy-on-read are performed on the subcluster basis when subclusters are enabled in target image. This testcase also triggers the following assert with previous commit not being applied, so we check that as well: qemu-io: ../block/io.c:1236: bdrv_co_do_copy_on_readv: Assertion `skip_byte= s < pnum' failed. Reviewed-by: Eric Blake Reviewed-by: Denis V. Lunev Signed-off-by: Andrey Drobyshev Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Stefan Hajnoczi Message-ID: <20230711172553.234055-4-andrey.drobyshev@virtuozzo.com> --- tests/qemu-iotests/197 | 29 +++++++++++++++++++++++++++++ tests/qemu-iotests/197.out | 24 ++++++++++++++++++++++++ 2 files changed, 53 insertions(+) diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197 index a2547bc280..f07a9da136 100755 --- a/tests/qemu-iotests/197 +++ b/tests/qemu-iotests/197 @@ -122,6 +122,35 @@ $QEMU_IO -f qcow2 -C -c 'read 0 1024' "$TEST_WRAP" | _= filter_qemu_io $QEMU_IO -f qcow2 -c map "$TEST_WRAP" _check_test_img =20 +echo +echo '=3D=3D=3D Copy-on-read with subclusters =3D=3D=3D' +echo + +# Create base and top images 64K (1 cluster) each. Make subclusters enabl= ed +# for the top image +_make_test_img 64K +IMGPROTO=3Dfile IMGFMT=3Dqcow2 TEST_IMG_FILE=3D"$TEST_WRAP" \ + _make_test_img --no-opts -o extended_l2=3Dtrue -F "$IMGFMT" -b "$TEST_= IMG" \ + 64K | _filter_img_create + +$QEMU_IO -c "write -P 0xaa 0 64k" "$TEST_IMG" | _filter_qemu_io + +# Allocate individual subclusters in the top image, and not the whole clus= ter +$QEMU_IO -c "write -P 0xbb 28K 2K" -c "write -P 0xcc 34K 2K" "$TEST_WRAP" \ + | _filter_qemu_io + +# Only 2 subclusters should be allocated in the top image at this point +$QEMU_IMG map "$TEST_WRAP" | _filter_qemu_img_map + +# Actual copy-on-read operation +$QEMU_IO -C -c "read -P 0xaa 30K 4K" "$TEST_WRAP" | _filter_qemu_io + +# And here we should have 4 subclusters allocated right in the middle of t= he +# top image. Make sure the whole cluster remains unallocated +$QEMU_IMG map "$TEST_WRAP" | _filter_qemu_img_map + +_check_test_img + # success, all done echo '*** done' status=3D0 diff --git a/tests/qemu-iotests/197.out b/tests/qemu-iotests/197.out index ad414c3b0e..8f34a30afe 100644 --- a/tests/qemu-iotests/197.out +++ b/tests/qemu-iotests/197.out @@ -31,4 +31,28 @@ read 1024/1024 bytes at offset 0 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 1 KiB (0x400) bytes allocated at offset 0 bytes (0x0) No errors were found on the image. + +=3D=3D=3D Copy-on-read with subclusters =3D=3D=3D + +Formatting 'TEST_DIR/t.IMGFMT', fmt=3DIMGFMT size=3D65536 +Formatting 'TEST_DIR/t.wrap.IMGFMT', fmt=3DIMGFMT size=3D65536 backing_fil= e=3DTEST_DIR/t.IMGFMT backing_fmt=3DIMGFMT +wrote 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 28672 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 34816 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Offset Length File +0 0x7000 TEST_DIR/t.IMGFMT +0x7000 0x800 TEST_DIR/t.wrap.IMGFMT +0x7800 0x1000 TEST_DIR/t.IMGFMT +0x8800 0x800 TEST_DIR/t.wrap.IMGFMT +0x9000 0x7000 TEST_DIR/t.IMGFMT +read 4096/4096 bytes at offset 30720 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Offset Length File +0 0x7000 TEST_DIR/t.IMGFMT +0x7000 0x2000 TEST_DIR/t.wrap.IMGFMT +0x9000 0x7000 TEST_DIR/t.IMGFMT +No errors were found on the image. *** done --=20 2.41.0 From nobody Wed May 15 07:07:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1693396301; cv=none; d=zohomail.com; s=zohoarc; b=GhcPyfIkGAVTKdoAtD/MvEA85JBgcpy7y4j1IHWydbGLi5Lcw2F7mJdDqeXOW7D0a+S5kO1n8ayO7akrVgMz7hZ90if7rOCnjbN/xVlAbQzf2cDbb72Y5uLLwD2/Q9VaYALbxqj5Vs7z6akgZpcPnZUNq2RKRg4jlU7nDFdlMx8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1693396301; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=TTOlhnU8FQgncj4r5I4EbHjsKY1j8lGUXnUus2JhPoI=; b=UiXKircRxJvxLnrl4pSNnyZmmJ8gLpQp5l6Iuq9Ez6naPPf0nEaaZYjzGNRCybezp3xUg6WgQ/5orUsqjYFP6J7KVhn25CwyVkez0gqvA6aM8QP5R7ML4fnHGZ/MdlzDbzKLQbJu+GTZRbXIT64oiaudAvIg98MIOgyrdna6N0E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1693396301735928.2153876339744; Wed, 30 Aug 2023 04:51:41 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qbJi1-0003fp-HO; Wed, 30 Aug 2023 07:50:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qbJhy-0003X2-Kx for qemu-devel@nongnu.org; Wed, 30 Aug 2023 07:50:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qbJhw-0003jM-Is for qemu-devel@nongnu.org; Wed, 30 Aug 2023 07:50:02 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-540-nAKD0yb5PEC7Sjq9YDoyyQ-1; Wed, 30 Aug 2023 07:49:56 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 64CF2858290; Wed, 30 Aug 2023 11:49:55 +0000 (UTC) Received: from localhost (unknown [10.39.193.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id D6F3EC15BB8; Wed, 30 Aug 2023 11:49:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1693396200; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TTOlhnU8FQgncj4r5I4EbHjsKY1j8lGUXnUus2JhPoI=; b=ZDjXYOBsUH175Obzzx8CmljKtMUM1M7bESxngyAID8D5AznYSgh2mveKoQQ9M/bBotFSIr Uns/M5mJ0JNT5t72W/Hkn/zTBJ/QuDgRwuZZwWWtR5Q2GDt0Luk2/tfZyR4Cq8rP0DOLaY c8h48hWc4MCttNmAqGsBYNr2KrvEE1s= X-MC-Unique: nAKD0yb5PEC7Sjq9YDoyyQ-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Cc: Marcel Apfelbaum , Stefan Hajnoczi , Juan Quintela , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Jeuk Kim , qemu-block@nongnu.org, Vladimir Sementsov-Ogievskiy , Leonardo Bras , Laurent Vivier , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Kevin Wolf , Peter Xu , Paolo Bonzini , John Snow , Fam Zheng , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , "Michael S. Tsirkin" , Thomas Huth , Hanna Reitz Subject: [PULL v3 5/5] aio-posix: zero out io_uring sqe user_data Date: Wed, 30 Aug 2023 07:49:42 -0400 Message-ID: <20230830114942.449060-6-stefanha@redhat.com> In-Reply-To: <20230830114942.449060-1-stefanha@redhat.com> References: <20230830114942.449060-1-stefanha@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1693396303724100003 Content-Type: text/plain; charset="utf-8" liburing does not clear sqe->user_data. We must do it ourselves to avoid undefined behavior in process_cqe() when user_data is used. Note that fdmon-io_uring is currently disabled, so this is a latent bug that does not affect users. Let's merge this fix now to make it easier to enable fdmon-io_uring in the future (and I'm working on that). Signed-off-by: Stefan Hajnoczi Message-ID: <20230426212639.82310-1-stefanha@redhat.com> --- util/fdmon-io_uring.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/util/fdmon-io_uring.c b/util/fdmon-io_uring.c index 17ec18b7bd..16054c5ede 100644 --- a/util/fdmon-io_uring.c +++ b/util/fdmon-io_uring.c @@ -184,6 +184,7 @@ static void add_poll_remove_sqe(AioContext *ctx, AioHan= dler *node) #else io_uring_prep_poll_remove(sqe, node); #endif + io_uring_sqe_set_data(sqe, NULL); } =20 /* Add a timeout that self-cancels when another cqe becomes ready */ @@ -197,6 +198,7 @@ static void add_timeout_sqe(AioContext *ctx, int64_t ns) =20 sqe =3D get_sqe(ctx); io_uring_prep_timeout(sqe, &ts, 1, 0); + io_uring_sqe_set_data(sqe, NULL); } =20 /* Add sqes from ctx->submit_list for submission */ --=20 2.41.0