From nobody Mon May 20 16:07:49 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=virtuozzo.com); dmarc=pass(p=quarantine dis=none) header.from=virtuozzo.com ARC-Seal: i=2; a=rsa-sha256; t=1616148645; cv=pass; d=zohomail.com; s=zohoarc; b=bO4Uvr81PVn7RLv8TOOgsp03g1xnLzVr4Tkqw1dV0yuIPEqQ+swNOy5RKOhNLbbuElB3Crk5Lx5otmENjISah0Qj4LXO5Ri2AISAzFZ2p5rzxAZMgUjW8ZkEJiBLccede5zvDbUdfFpSHNcT0Uji+NABFMizmj8PJ3mJctHz+/0= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616148645; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=tqyq/wzrWvPl3Y8oX+bTp4PhAiv+/b/Ay3++YaCl4A8=; b=BlEbfVs94G9SF734LhlWsoKwheLONzKK+Gfe8/Z+hNzOmxDNnJBHx9rY3L5+2jVIEHyX/JpiS1r88mYtpfdgn4s8njz3rrwfph6SVqmstZKegzU7VlciK0c3qXFIqbzNS34yXWlbQDhwCXXFNSiCJqcX9KeDt9JOXu+KSgjefgk= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=virtuozzo.com); dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1616148645701674.2709752548719; Fri, 19 Mar 2021 03:10:45 -0700 (PDT) Received: from localhost ([::1]:37536 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lNC5c-0006ZH-Fe for importer@patchew.org; Fri, 19 Mar 2021 06:10:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46578) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lNC3Y-00056r-Jh; Fri, 19 Mar 2021 06:08:36 -0400 Received: from mail-db8eur05on2125.outbound.protection.outlook.com ([40.107.20.125]:53216 helo=EUR05-DB8-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lNC3W-0005Ve-Nz; Fri, 19 Mar 2021 06:08:36 -0400 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM6PR08MB3720.eurprd08.prod.outlook.com (2603:10a6:20b:8f::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3955.18; Fri, 19 Mar 2021 10:08:29 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::f1f0:6610:11f5:5e4a]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::f1f0:6610:11f5:5e4a%7]) with mapi id 15.20.3955.018; Fri, 19 Mar 2021 10:08:28 +0000 Received: from localhost.localdomain (185.215.60.202) by AM0PR01CA0082.eurprd01.prod.exchangelabs.com (2603:10a6:208:10e::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3955.18 via Frontend Transport; Fri, 19 Mar 2021 10:08:26 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nU8/YHr/TXNAPuCoNz5EMW+8CV+aJVq2NubolC9z7vJzpreX7r15DIZHfBHdawSD5JXogV9U9U3jS+9cIk3FW61Kucwtgz+RYD+Q9j+mJWMvqCuN+lvHNLdqC1gob/zIxJhaSVjjkincOwkdePbN6IQr7yPZ04CAvPXInBqnGCxR0YPSxo4YsMwLM8JOmlIxNaWprHNI5Fo8iyXLulFJUEVYRJavIHTSVmghI8pEXRVeGst/hnfK9WLYJKsaIWOJI0grRbquQpesKGTxhQorCDDPgR+Z+N8FmFpVmGgdZSx7f0avpYQJyLJAAOLi7w52NdcuCU/8hlyfHjat0oEw7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tqyq/wzrWvPl3Y8oX+bTp4PhAiv+/b/Ay3++YaCl4A8=; b=JNQt/NjcPwZsr+9Q3j47TMEGQ5pXm1J0gifRtZdmRMOnkQhgNulshUmhlbNoVbr20GeapfLEabEPshJL8VsLLwWaEFM2WGXf+ZvxRaaOHv/Brb26ToB/hA3MzqL3l+n/hy9er9HnDziS4BOd8zQfOdbnxf2o/3wplN84xZdrkNUy42ewQVwNG/aXFsNjhuJFdme3PXlhcxTJSmG7WR0gEgY/zjEMzDxj/RI72nE/LeSoMJLExuxFsOvItv7As9E8XHK3sBIrCWnn3zpCUV7ufZjgICq/rDiiEHjrlatkSbWSB0ob+hr+wRnttrEbdADK3uaiNj1C7YunZLCxpdiNJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tqyq/wzrWvPl3Y8oX+bTp4PhAiv+/b/Ay3++YaCl4A8=; b=HkHPG6nV9SUllhavV/zzctcsOHoJkZBLZUenfNVhApUl4Wkj81V3MUEZB3c/qVInaXxAbzEBScKxNBLrIdYD/kKMrrkXJH0K70TqWWV0PzWUJLismoubb2TVcmPDYa87iSRa7JJR7TeDcSZxCDt1nnF9+lbB76/l6pR80NDaDFY= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=virtuozzo.com; From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, mreitz@redhat.com, kwolf@redhat.com, vsementsov@virtuozzo.com, den@openvz.org Subject: [PATCH v4 1/3] qemu-io: add aio_discard Date: Fri, 19 Mar 2021 13:08:09 +0300 Message-Id: <20210319100811.342464-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210319100811.342464-1-vsementsov@virtuozzo.com> References: <20210319100811.342464-1-vsementsov@virtuozzo.com> Content-Transfer-Encoding: quoted-printable X-Originating-IP: [185.215.60.202] X-ClientProxiedBy: AM0PR01CA0082.eurprd01.prod.exchangelabs.com (2603:10a6:208:10e::23) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f28bde9e-3ebc-475d-83ef-08d8eabef087 X-MS-TrafficTypeDiagnostic: AM6PR08MB3720: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oTJ/muLfNFiKyXKaNS4eJ03kmcJ/i0MPMMKgNpjoZWuxrvvwNKQuaDm8RvVqUkfWksa+/S6TzbsP7Pq/pZMgqfs+H+kyGsYnokklGJ6ldtB8L4Q/UEgwmGza8E4rpbPRiTIxrZa0U2EDNX1yB4Kd9RDIBzaga69t9M+FQgqKrtMx+/1U+SpkC6N0se94VAT0Qut/boYuZ4OxiezrlHIlueB4eGVNC4T+bIDQH6AAHWNsOTORmmlM4h4/XJHoS02PBdgAoXKUUCypCcYtZoiMq+tMDcuTxl1TcCIumKy52+1ixGPILBooAd3Mk4I0OwX71wIM2MuR2EPcjn1zfBMb3whS8bR53sI3peytLJIe3Aw4QDvK/iejW/DsBw8eaWey2jBE7PXdoeUGwYwM+rQTmBaLk52V/ZdB/y266PYH48beivAUKZujnYMeEMB4CqmxkRE3AqkdFCMOCPp5FlGwfrvrBkGQgbvZDyXzHDbEZ19denUbmvoFZ+LeclNaJbEe1lDvNN0WWGDKABj64nsZQ3Fs720N1HpOUyyqMTNt+Gk1dHyElWx2fudACL4l8ZHHYAyDa+GulEgZt0ktugdZ5932wJFseun2dWeOlz1KrCpPThrLPMHh4oqZBf6+9cuOJ+NlSmDihWO4AO80whjlEzyyCrodAq1Q6U6qR7b2YqYexm1GPRFdjRdtgz86Gtsg X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(376002)(396003)(39840400004)(346002)(136003)(186003)(956004)(69590400012)(2616005)(83380400001)(16526019)(107886003)(66556008)(6506007)(38100700001)(66946007)(316002)(2906002)(86362001)(6512007)(5660300002)(8936002)(1076003)(6916009)(8676002)(36756003)(6666004)(478600001)(26005)(4326008)(66476007)(6486002)(52116002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: =?us-ascii?Q?Ovp0NZJEpONnfIDjonp+Mcqu8i/lf2n3cCJoAEhuoeLB6fZrKkjTu3/bvuG6?= =?us-ascii?Q?QLhA0EYaX0XqbWwtU/blQy8xxPpdjovPX3ODHeYEo6DKrrYleH+0bQBuQ1ni?= =?us-ascii?Q?WTbo/xWskCAY7+CuqveBSZlObTG9xALrUQzA0W7Y1spgbChtiNBv/3siRHsJ?= =?us-ascii?Q?2MzDQECYPLluvfQ05cSsS+aKvavSmdBx6xn4WiA18JJG+GGSb8hFk8tYq0Fu?= =?us-ascii?Q?uFV6mry70mXl75e0F2w2U2nQxio3WKmQfM7aYynOOk8eN1oMXdQQHGu85ja8?= =?us-ascii?Q?J4aUmjMDnhjxYUfgpQ2XXF0J/VEjsttPbmnRZM0PrFrMuhmkHzEgJ5uAeSpk?= =?us-ascii?Q?eOM8CJHZPQkLGdjijUkSEUDQGPqBxMNmH6JzZ8jlPt2huAI8TxRNcHj1E8Wt?= =?us-ascii?Q?GZfR/AOFxWW02p2X1l7yvJFiMCq9esXferWoZgjVis0i4gnaZPercMiOIfYp?= =?us-ascii?Q?+WHUnU3V4Y6e85o9H2hgl0cQA1PCPnrsdEPX2RFu/Y8rDt1OI9BZBqZEp1r1?= =?us-ascii?Q?RYVEpENZ4c5MSAD0o4w5HxJGTqkvAXTguOLpp5t4WB5CDUTPdaL34WqxIoPz?= =?us-ascii?Q?e1KJNpoIFCpD7RsgPST+yK08+MNm99IKEP2Zx+ySCYx7EW1YU4r6BCIMrnm/?= =?us-ascii?Q?nwVVwVzOoUYFBMXxXthMzZcdxgwIbotEJXd3laeeHU0LLVK3uTXvfr8VP5ES?= =?us-ascii?Q?dLxPWaXbZDwkV6xfAfPc0S7jX3OdcBm31q8FCjpFu20bnWBqzelJmaxU3yXt?= =?us-ascii?Q?JTjD+v35U5QvSHXhE9yQr/OGPNE1nsg7UxADNqmmeKBHJWLaznYBJdqwPa/E?= =?us-ascii?Q?xtLrkeTZyHM+BMAhaYa+TDvPDpnS9i7kVVY0z/JJJmdrJcuoD7RAp9ybCXXx?= =?us-ascii?Q?F79qp+65D8fbXxRXsx85ifGyOlA8YI36yr2sLg8kJ3t3dY43SyDvytOpo3PR?= =?us-ascii?Q?QdjlbMHSmX1Hj6gdUUSxNWD9xx1u88buJwl82V7/OJCd2T73vNGzL5kuFLp1?= =?us-ascii?Q?X8x0Xp92OuQHyWiAoX+wEK52ENzYc8WPWm+uLgPKGJGPBj7AiulkWLLu1Pa2?= =?us-ascii?Q?grRDH/39dFbWSPVcVVrpjJDuCUcFExV8vinnlz2YRufYlU3+siPyTAM4xjsj?= =?us-ascii?Q?0P6C/H1+7SPKm6QUp6UKEthROaZN/RmNAYOYF2cEnpWWwwoed6K+fNhumkEq?= =?us-ascii?Q?DbAGbqKvo2Lf7ZUhA5wr5T+nCGfZ1FsOzQgCISpFgFgPP9d+8sEtSrfiyjGT?= =?us-ascii?Q?lSDND5ag8SDtGuLAGSJ+uuTqnM+7L5LHvcceP/GVnoH5T2TDemq4d0vox3Fa?= =?us-ascii?Q?vwP2T1Wgbzvoc3O5jbdb1QG7?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: f28bde9e-3ebc-475d-83ef-08d8eabef087 X-MS-Exchange-CrossTenant-AuthSource: AM7PR08MB5494.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2021 10:08:27.9216 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hmUxH31PENI1YBaYdS+YYiz9Gmvn56XI8cpPSx+DfOfy8u0xsq9A7gTrz+Kv8wrf6qXyVoeQdjciqzNslgBXOEMI34z1k0xHVlC0AxSmiUA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3720 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=40.107.20.125; envelope-from=vsementsov@virtuozzo.com; helo=EUR05-DB8-obe.outbound.protection.outlook.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @virtuozzo.com) Content-Type: text/plain; charset="utf-8" Add aio_discard command like existing aio_write. It will be used in further test. Signed-off-by: Vladimir Sementsov-Ogievskiy --- qemu-io-cmds.c | 117 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 117 insertions(+) diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c index 97611969cb..28b5c3c092 100644 --- a/qemu-io-cmds.c +++ b/qemu-io-cmds.c @@ -1332,6 +1332,7 @@ struct aio_ctx { BlockBackend *blk; QEMUIOVector qiov; int64_t offset; + int64_t discard_bytes; char *buf; bool qflag; bool vflag; @@ -1343,6 +1344,34 @@ struct aio_ctx { struct timespec t1; }; =20 +static void aio_discard_done(void *opaque, int ret) +{ + struct aio_ctx *ctx =3D opaque; + struct timespec t2; + + clock_gettime(CLOCK_MONOTONIC, &t2); + + + if (ret < 0) { + printf("aio_discard failed: %s\n", strerror(-ret)); + block_acct_failed(blk_get_stats(ctx->blk), &ctx->acct); + goto out; + } + + block_acct_done(blk_get_stats(ctx->blk), &ctx->acct); + + if (ctx->qflag) { + goto out; + } + + /* Finally, report back -- -C gives a parsable format */ + t2 =3D tsub(t2, ctx->t1); + print_report("discarded", &t2, ctx->offset, ctx->discard_bytes, + ctx->discard_bytes, 1, ctx->Cflag); +out: + g_free(ctx); +} + static void aio_write_done(void *opaque, int ret) { struct aio_ctx *ctx =3D opaque; @@ -1671,6 +1700,93 @@ static int aio_write_f(BlockBackend *blk, int argc, = char **argv) return 0; } =20 +static void aio_discard_help(void) +{ + printf( +"\n" +" asynchronously discards a range of bytes from the given offset\n" +"\n" +" Example:\n" +" 'aio_discard 0 64k' - discards 64K at start of a disk\n" +"\n" +" Note that due to its asynchronous nature, this command will be\n" +" considered successful once the request is submitted, independently\n" +" of potential I/O errors or pattern mismatches.\n" +" -C, -- report statistics in a machine parsable format\n" +" -i, -- treat request as invalid, for exercising stats\n" +" -q, -- quiet mode, do not show I/O statistics\n" +"\n"); +} + +static int aio_discard_f(BlockBackend *blk, int argc, char **argv); + +static const cmdinfo_t aio_discard_cmd =3D { + .name =3D "aio_discard", + .cfunc =3D aio_discard_f, + .perm =3D BLK_PERM_WRITE, + .argmin =3D 2, + .argmax =3D -1, + .args =3D "[-Ciq] off len", + .oneline =3D "asynchronously discards a number of bytes", + .help =3D aio_discard_help, +}; + +static int aio_discard_f(BlockBackend *blk, int argc, char **argv) +{ + int ret; + int c; + struct aio_ctx *ctx =3D g_new0(struct aio_ctx, 1); + + ctx->blk =3D blk; + while ((c =3D getopt(argc, argv, "Ciq")) !=3D -1) { + switch (c) { + case 'C': + ctx->Cflag =3D true; + break; + case 'q': + ctx->qflag =3D true; + break; + case 'i': + printf("injecting invalid discard request\n"); + block_acct_invalid(blk_get_stats(blk), BLOCK_ACCT_UNMAP); + g_free(ctx); + return 0; + default: + g_free(ctx); + qemuio_command_usage(&aio_write_cmd); + return -EINVAL; + } + } + + if (optind !=3D argc - 2) { + g_free(ctx); + qemuio_command_usage(&aio_write_cmd); + return -EINVAL; + } + + ctx->offset =3D cvtnum(argv[optind]); + if (ctx->offset < 0) { + ret =3D ctx->offset; + print_cvtnum_err(ret, argv[optind]); + g_free(ctx); + return ret; + } + optind++; + + ctx->discard_bytes =3D cvtnum(argv[optind]); + if (ctx->discard_bytes < 0) { + ret =3D ctx->discard_bytes; + print_cvtnum_err(ret, argv[optind]); + g_free(ctx); + return ret; + } + + blk_aio_pdiscard(blk, ctx->offset, ctx->discard_bytes, + aio_discard_done, ctx); + + return 0; +} + static int aio_flush_f(BlockBackend *blk, int argc, char **argv) { BlockAcctCookie cookie; @@ -2494,6 +2610,7 @@ static void __attribute((constructor)) init_qemuio_co= mmands(void) qemuio_add_command(&readv_cmd); qemuio_add_command(&write_cmd); qemuio_add_command(&writev_cmd); + qemuio_add_command(&aio_discard_cmd); qemuio_add_command(&aio_read_cmd); qemuio_add_command(&aio_write_cmd); qemuio_add_command(&aio_flush_cmd); --=20 2.29.2 From nobody Mon May 20 16:07:49 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=virtuozzo.com); dmarc=pass(p=quarantine dis=none) header.from=virtuozzo.com ARC-Seal: i=2; a=rsa-sha256; t=1616148741; cv=pass; d=zohomail.com; s=zohoarc; b=Lirx/dfllakDkFsEdR5C1FBQBnQr/y+NI6OEr7fBC3CmcTpJPG4L0uIELTGXDSMIJj2WcoFasOghyO1ki0a4y2KZ09ROSVyLBjFO+2IpWephpOUssKixR1b3Rd+DK4peMGXqJaT8F0umswzdYSKi63F9s0r5bTsI5sTnoz3wPq0= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1616148741; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=16r3DiadWOxkQ42LojADZo+zQt/YAQng9PMqIguR3yA=; b=k4M+MlqqBPGP567zgJ38CX+0BETrrh2Lo7+c5bF/SxLmhEbLgi9eXlGC4KDLoFeue7dMLC297kHxhM6krjWQJgKkt0LOqTK4CeU0OCdtVur6hLEkRDUc6T11a/UplibAxbnu33CLfybQMkhuP/o4Ztg/CNuPOsqFCvc0DCyEU+I= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1dmarc=pass fromdomain=virtuozzo.com); dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1616148741054396.2554632048905; Fri, 19 Mar 2021 03:12:21 -0700 (PDT) Received: from localhost ([::1]:40982 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lNC79-00084W-Tr for importer@patchew.org; Fri, 19 Mar 2021 06:12:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46600) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lNC3b-0005BR-EX; Fri, 19 Mar 2021 06:08:39 -0400 Received: from mail-db8eur05on2125.outbound.protection.outlook.com ([40.107.20.125]:53216 helo=EUR05-DB8-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lNC3Z-0005Ve-7d; Fri, 19 Mar 2021 06:08:39 -0400 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM6PR08MB3720.eurprd08.prod.outlook.com (2603:10a6:20b:8f::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3955.18; Fri, 19 Mar 2021 10:08:29 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::f1f0:6610:11f5:5e4a]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::f1f0:6610:11f5:5e4a%7]) with mapi id 15.20.3955.018; Fri, 19 Mar 2021 10:08:29 +0000 Received: from localhost.localdomain (185.215.60.202) by AM0PR01CA0082.eurprd01.prod.exchangelabs.com (2603:10a6:208:10e::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3955.18 via Frontend Transport; Fri, 19 Mar 2021 10:08:28 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=P9vik0a3RumVeeu1P3+QaolNSk3jwgf99mnqFobFdCBdtvaFZRij3MYecjAJPJaP80fOowYs98fbFDvJhMyE//fkDfDhGkPIb/mjMTlxhzY61NVCmOGJ5kwBjczwBYGHrNZRD+09botrscoH+cSrOLZkGAhkQ8B2C0j8pUpuhXbhJfmoaYGUTPK10dBHpSA5DJD7LcG51beaDXccrYOFObPb1t/WH+i5Yor1DmZ8uvGSLZqb1qskvAU+/3iT5XwszOPSYUSVWmD11AuvFFPQKI3o5qZ2Ce/9l1NqJH8piqt+UhddfGEPc9NZlDpXr9KKF4AX3I2Q8zlmto+WjtAMUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=16r3DiadWOxkQ42LojADZo+zQt/YAQng9PMqIguR3yA=; b=fzErwfjVvv0QQeNJNX+Jvx1D4quEKp06lfdUffQr3UerOGnIANNkLLpMU2ZDBnViS83Rm5Hi8J434R3KRry1Bf/Jns33nEJZUy2265ocs8uSW9LCcAYosRwd9Oam3oZ4xuj69GKtNp0r3HSUeB/yhsQLjf1JdKg1X4chFRE538JYoOfZ7xValB/dsrd9pKyVoN7JYl7Lf2JT5OqMRfMq/wdlpzaNPV2lDeIyBOfzEbjoVGPWgOf+AcRWACTDyyD5nmqeDWxvmmMfM8e6DHLMG922ciFjk0MCzSkRWZLzX+VgvZ7LbnTNombenxXT38AhQHjuFWOt3Ds71H3Le4Tt2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=16r3DiadWOxkQ42LojADZo+zQt/YAQng9PMqIguR3yA=; b=NiBjj8FCfh4YJ3E+fcirtD3CDRm+slacCkw4F+8oNaqnnfHGuI3EkjYe+ii5bDLM5lQh2F4i8Z6W29SzvPYyY6t7c6Md4wNTpPvzfczEMiDYPm3G9edjfI3XdRnefGumL07u9yEBdfQYQJ87CiIOK9B1fpEJmSyqnD6/CUgydsc= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=virtuozzo.com; From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, mreitz@redhat.com, kwolf@redhat.com, vsementsov@virtuozzo.com, den@openvz.org Subject: [PATCH v4 2/3] iotests: add qcow2-discard-during-rewrite Date: Fri, 19 Mar 2021 13:08:10 +0300 Message-Id: <20210319100811.342464-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210319100811.342464-1-vsementsov@virtuozzo.com> References: <20210319100811.342464-1-vsementsov@virtuozzo.com> Content-Transfer-Encoding: quoted-printable X-Originating-IP: [185.215.60.202] X-ClientProxiedBy: AM0PR01CA0082.eurprd01.prod.exchangelabs.com (2603:10a6:208:10e::23) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c5fb078a-da50-43f1-fbf4-08d8eabef136 X-MS-TrafficTypeDiagnostic: AM6PR08MB3720: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4502; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: RMsnL8pDuJjkObXe68ObKoDHeeVkucbhRNCnkGWzECjNH9MVJMEyON/Aa56Ow/7yjslCNsZDC7MtLzYPtzl0bAmmNVWfiA/CsoZplFgtHaPoXrTAzDoM8HEy0Li+77OF6ubyfNB106XB0rWWfclQQAmgZGCzulXp88EEcnc3wbufjf/TrXODpFXCfbGCWnynDbMlkm/aV+0gOwJvCfPMmdAySohEvQoSnxoflu3fD1vxwzsPMIqeu87xUHxq06jMZFicK8JeX5oc1/ZEmI2mBMzqFCMZr4nMTcSV1vd4qgCRsczKL/GSjl2/AYSW+L9ywvknCkE+xlu2YOTABbXOtAa17KU+EmFD4FiuGUNUK1Bro+v0UEfNJEIFDGRwydlhFanYph+ZPCXSjWqmnt8thXwbpseFMhjHKN8LKw4V3oLFKkxDMLZPKuagz9JlTfbYek3tAmuqXc+zelKg4/BoKqo/xoJ1bUVVVWpVjsLoOmJh6AyYuMAubcUT8kDlNSqNT0JQsWCDlPhgRA9U+x9jgpAMtKweE3C+H91SKVdmSUhLvLB0HCR5fSnIdIrGn32bDLpFFoC4m4zctXIn/hC/VFagfInfHlSlJmecRZCFPH1OqXCZkd0+g0kbPp3EIzkIzkcFRcdqpHOY7GaMcU3vUQV+ppilNNIp9EQWrieB+GfDn83A2dqgDb2o53MpE8rH5mKxfTZyxw0aAm5BxJnae6JvbfpAI3EH+QuO0bV6hKW4aoPmhvlwu+RaQZr5dyTOZhXzacptWus+98di52JWXhhrwAjYbOpy/m5/KaRJSN+bYgr0ak75aO0S/pFiddE8 X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(376002)(396003)(39840400004)(346002)(136003)(186003)(956004)(69590400012)(2616005)(83380400001)(16526019)(107886003)(66556008)(6506007)(38100700001)(66946007)(316002)(2906002)(86362001)(6512007)(5660300002)(8936002)(1076003)(6916009)(8676002)(36756003)(6666004)(478600001)(26005)(4326008)(66476007)(6486002)(52116002)(2004002)(21314003); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: =?us-ascii?Q?1mXbv1p5rHHtmFHcAtQ9SbgFg3QjkiwveAp9bikWkWLE4TId9MgB5dwfw5KY?= =?us-ascii?Q?PJOH7uM92PVC3HDwLunKQ/4sQZow3cpog6cmfCG0WfbrnV7NYtz/xr3TUt0I?= =?us-ascii?Q?J+BDxUTws0qji2pWtCFyUJ3WuPDtrlrpgAuFy7704Le85iXwi2Ehuo7cmP7W?= =?us-ascii?Q?SzY8Mv3p+DmlD+BZsV20JGchUyx4/e5qVMDfTxyOHBPIoOk2DMK86HFfHadD?= =?us-ascii?Q?p4/cqz3cKDexCetbPJzBqrrZglZ7hQESfXqKKBF+sGT3qqgLUKQcxOvZa/uy?= =?us-ascii?Q?7liXp0GjurZ+r08pYPENP5uTMwtKnNF4GOtTAin3/lXl7F9peSkV0J3607PB?= =?us-ascii?Q?RBGaO5JrCswPU9YVq+h5414Fn8Ox3B85S9FkF1YhM6sttsMmtlbahJklkGs7?= =?us-ascii?Q?+DTfzNtjTPFTCRHacPN5sna9+vz1+fgFd2VCEQZKKP8HYiFEMCytKIxQdoXg?= =?us-ascii?Q?DhYEaQHdQxOeSa4qpSHQAZAPnd3mRYfEyo4HTRyPWmOJaIqv7wnTUabl+O43?= =?us-ascii?Q?K/02qqCrV9VQ5FjDERjtJq+guckxQmV/06VQ0+euKIgl1FbxaDnky26s69f5?= =?us-ascii?Q?qVX7FvlVMImuki1TiujWi/5xILEvwHXUvztqL3V+uA6nTegshc1R4Qd9SSvA?= =?us-ascii?Q?QS9L3KCgciCOsFWFWEjZmzibS1WOHYLe5mAs5WJ03iVPPE7yIDDL5lIi0syl?= =?us-ascii?Q?Q9xbYtRXJITXmmSMvxzV3qg3qTeqPMaCQqd18mbNYpkalmdhUTIOrGPJ9jBR?= =?us-ascii?Q?Ml+5HMHvr17ZS/eLu/dKFuGpibHdXM3GXUOcZfpPGVFcq0EDJhI4+501mNVt?= =?us-ascii?Q?5oHsQyaZzhaYAIhCOmad5RMW3TprBUrCQlWclBVrUsf9884I1TDjy3u28EsN?= =?us-ascii?Q?UvhYkU/hyDnMll+7UV26WJTnkPSlXgLlYjUH0WoSwd4/ZT20b16LbPOp7jKR?= =?us-ascii?Q?rfZA5f2+6s0i/HZBNCzPuuj6AKT8Zzyi9gd7/FapEvpe+pjZt5mnePZwKOfD?= =?us-ascii?Q?cVCTDXxdOzXX/pTZkWr2Oq2lMA7KQAWQusq/ORoDIc4CvQ6wfR1C79LoyZ0n?= =?us-ascii?Q?lp5F4cz1kg7CmOlRpqtCWXh/jY+M2JQwgTS7jN5OLoTZ4KBTx0TloZWRjsP+?= =?us-ascii?Q?MAMl6c7Atuj1otgcU+DnRdRbYDTxhiSQ68HiAA3H9Nie5fgRjFgVrfbmk7dz?= =?us-ascii?Q?S6l2FAyN7pOomqZ1NhlMrLAPCHzZMI2SVRrsMbTB7q3oOHH9yGYEiV7LQ7nw?= =?us-ascii?Q?8j9DnpnQkdKC2/5U9vc+48Iqk/YL7l5jJqIHt90GiDpqbXQK7ujqolUXgx1B?= =?us-ascii?Q?Ye/68rcxwDDQZwW0IPeJ/Y1A?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: c5fb078a-da50-43f1-fbf4-08d8eabef136 X-MS-Exchange-CrossTenant-AuthSource: AM7PR08MB5494.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2021 10:08:28.6534 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Pv6i0iqZx6Xp1+m++vnb0HwXf+dQU+MnywvADQOckJLc3lDYQ/s6RAr0KX2k04p6ANHJEThDA6zZjxO7ZgDl99pYRdFGWZz4C63SmwYNq4g= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3720 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=40.107.20.125; envelope-from=vsementsov@virtuozzo.com; helo=EUR05-DB8-obe.outbound.protection.outlook.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @virtuozzo.com) Content-Type: text/plain; charset="utf-8" Simple test: - start writing to allocated cluster A - discard this cluster - write to another unallocated cluster B (it's allocated in same place where A was allocated) - continue writing to A For now last action pollutes cluster B which is a bug fixed by the following commit. For now, add test to "disabled" group, so that it doesn't run automatically. Signed-off-by: Vladimir Sementsov-Ogievskiy --- .../tests/qcow2-discard-during-rewrite | 99 +++++++++++++++++++ .../tests/qcow2-discard-during-rewrite.out | 17 ++++ 2 files changed, 116 insertions(+) create mode 100755 tests/qemu-iotests/tests/qcow2-discard-during-rewrite create mode 100644 tests/qemu-iotests/tests/qcow2-discard-during-rewrite.o= ut diff --git a/tests/qemu-iotests/tests/qcow2-discard-during-rewrite b/tests/= qemu-iotests/tests/qcow2-discard-during-rewrite new file mode 100755 index 0000000000..dd9964ef3f --- /dev/null +++ b/tests/qemu-iotests/tests/qcow2-discard-during-rewrite @@ -0,0 +1,99 @@ +#!/usr/bin/env bash +# group: quick disabled +# +# Test discarding (and reusing) host cluster during writing data to it. +# +# Copyright (c) 2021 Virtuozzo International GmbH. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=3Dvsementsov@virtuozzo.com + +seq=3D`basename $0` +echo "QA output created by $seq" + +status=3D1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./../common.rc +. ./../common.filter + +_supported_fmt qcow2 +_supported_proto file fuse +_supported_os Linux + +size=3D1M +_make_test_img $size + +( +cat < (p=quarantine dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1616148871886290.5328338239294; Fri, 19 Mar 2021 03:14:31 -0700 (PDT) Received: from localhost ([::1]:45118 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lNC9G-0001Px-Jr for importer@patchew.org; Fri, 19 Mar 2021 06:14:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46624) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lNC3e-0005Hm-IE; Fri, 19 Mar 2021 06:08:42 -0400 Received: from mail-db8eur05on2125.outbound.protection.outlook.com ([40.107.20.125]:53216 helo=EUR05-DB8-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lNC3c-0005Ve-5k; Fri, 19 Mar 2021 06:08:42 -0400 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM6PR08MB3720.eurprd08.prod.outlook.com (2603:10a6:20b:8f::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3955.18; Fri, 19 Mar 2021 10:08:30 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::f1f0:6610:11f5:5e4a]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::f1f0:6610:11f5:5e4a%7]) with mapi id 15.20.3955.018; Fri, 19 Mar 2021 10:08:29 +0000 Received: from localhost.localdomain (185.215.60.202) by AM0PR01CA0082.eurprd01.prod.exchangelabs.com (2603:10a6:208:10e::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3955.18 via Frontend Transport; Fri, 19 Mar 2021 10:08:28 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=k/1RgS60K8NZ3jlvqnIfdiTcrOH/EL+8mChSDb9SOWZLMKddDshrm9onhVmXmnvI+/e4P0N6gViCyiEqoJ8v2KUl30nB4iB4WG3/N8RAOhc5G56qdqQfkHOoVPzTd5jEwAtd90cq83mzEbgDY5BU3Lfs8CSC5qNPvcgmTY2C3MjLfiqxbr1AeOm3A0sYZMoEqW/2A2lLVeNikV5ewrY9wrsSny4+21+D77kHVqWz7Obt8WLaefD7F0FM7j5zUy6JOzaLL6TorWKHzhZGRveXpLrdtgcVN6EAz5bsdaw0765LK7Ck6FITKrQ50Kqb7Vqslq6n0tDeKoF4TCVobDokpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qirjD67N4m2AzG4FVamcGSn90wLaySM3buH1Sr/5gUo=; b=LGAO51SbbIeAlwa6uCDsLyeJ85WNb9SyyDwRwUGRaidakVZqbNZ9XKMZWdS7WP517KFbk+E1HtywTV6m1rnuJRZJmCwMqi6kEjR6tFqwtDAlEYnc+lw3sGUk/XvKOv5IaJd4+fRSiQjRPmog950NPyCU2fG1pc5Z3nJSnksRlRbbpFIR4T0gAyyEQy8IsgK2BqEVXYGzQFDSOLCz1Uz+tnEv0x6Y41DVe1wRBg5Id2saDyj8EuHmNxvDzMjf6xoKJ1PkR1bDco5hZz/c6ll1/2orChqdIOcz5pnYZqwA+98ge4T9y8Aw6ylbXdlQ2jpyh0qPgs2j6i/HFtRVO8qNWw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qirjD67N4m2AzG4FVamcGSn90wLaySM3buH1Sr/5gUo=; b=j5Wbm1E+jSvndsUgeW5n5PbTA6Gnbs6kABYcsjSEWJevzfXw5r2CfWgMMiH6wSbTQy7PgwXING+3DdzS8QcgVws7ok16x+kiFapi+gfPO+IMkd0Nv0B+V0QIg59nV2OeJO3lFSxxVWD0X7kKAY8Y+uoMmNSEluWEbDj/XSTx+DE= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=virtuozzo.com; From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, mreitz@redhat.com, kwolf@redhat.com, vsementsov@virtuozzo.com, den@openvz.org Subject: [PATCH v4 3/3] block/qcow2: introduce discard_rw_lock: fix discarding host clusters Date: Fri, 19 Mar 2021 13:08:11 +0300 Message-Id: <20210319100811.342464-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210319100811.342464-1-vsementsov@virtuozzo.com> References: <20210319100811.342464-1-vsementsov@virtuozzo.com> Content-Transfer-Encoding: quoted-printable X-Originating-IP: [185.215.60.202] X-ClientProxiedBy: AM0PR01CA0082.eurprd01.prod.exchangelabs.com (2603:10a6:208:10e::23) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8a7123dd-40f4-4b49-0a30-08d8eabef1b2 X-MS-TrafficTypeDiagnostic: AM6PR08MB3720: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6790; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YqEfO73rCi5KRQ8Z/MWehO0FSblltTZFmWpvrIhZ1GBosxuG4qcOYV78yT6DJrPJ7UD8ecsyJPUkwger+P5ptWqqzNxw9HVIqydgsOYWkLW4uaALsDGGJVCMohowqpcu2Ax7shRJSgrZb1L5LKToxIkRJbPL4KdXDO1/Xfq99geHnRAT4KZn6Bpq7YV4kOdLEbzJIykp4Bc4mW6xqunOCgieJfPwmyKx7CkWmscVEJWmTxiiJuuwX2be4NRHrl15TKmLlhLZ7CtoeX8lIeTnMeu9unHYj8yb1zWqI7BeArNhtLPtD5OaxRB0gJwfdQFZuIrFd2e5G2fWYN6VVir5g/ExMwwVEB9dqdTgFybpRUYPx5aTfTlbsoyTTQFUA0VfK9PsD53dW4AxrnoT+UOc6i/Dws9856eF1rjyk92qvWj5bUYh9xdkn94eE8G4Ne70N5bnGukqoaXmsB7NyuxG6aAmi0N8DbLcEJZRVgRRczHhSVfRsiF/V+3FERcLBZDj4YtstLzDPsw3dDssOsJUarEU4WH6dMGvdRyMhYo47dPOG7U5kMtTNPr5gDL4e+PHTFmIvWZDtgLZU8Os9K88OTVOSyc1ZKpy+VzEQJU2Teuu2Hrsuk5H4k08CpKBoGIaVpQis6WnHU4p/+P2lgl0pDKchcdm7MLer/Jp5ddzG7tz1sbQtZuKuM4pS0VlByys X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(376002)(396003)(39840400004)(346002)(136003)(186003)(956004)(69590400012)(2616005)(83380400001)(16526019)(107886003)(66556008)(6506007)(38100700001)(66946007)(316002)(2906002)(86362001)(6512007)(5660300002)(8936002)(1076003)(19627235002)(6916009)(8676002)(36756003)(6666004)(478600001)(26005)(4326008)(66476007)(6486002)(52116002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: =?us-ascii?Q?Ski3GUTbdYO69NB9Mng3i5AIowQrcu9w6+IFDw0emAwZ5z2LaBd2FTd5QXQO?= =?us-ascii?Q?07+XR32clQwxcbMo3UR4PcdTg1uhvHdvXHYh5qsheUBMJXeLDf5EP9qTAmqC?= =?us-ascii?Q?Ioo24WaSutECJLptgqO/hqGHo8D7JR+BBUhlpAXefwiZFbUTx3xieAVvJfi3?= =?us-ascii?Q?FHQgWX4gdBvcgO97SpJmbiDZWbu96nFen4nkGicpeHxE+Hzpw5NYZXbkP1Ia?= =?us-ascii?Q?xr6HbBNWy78bFdjGy9rxfOi5Pmo5/RYemevTi6pvuZQW8Y4xRYYrVnn8k0F2?= =?us-ascii?Q?mNjpS6Hr4+hpnYp6pvxfTBtqJAbO+W3sWTFTly1tm+8lm1Ctr+OmeWZhVcVr?= =?us-ascii?Q?bKYPymZm8nV+9GTw7GbcyHK8OhLTqpA0h/Dw6PU+Ne3/fYha7+ZdAkpAKriS?= =?us-ascii?Q?mVh5OooYk2zfpxo6pvAda6ypaQWGDK37cCgbwKTdEWGQ7ho/XT5wtmia4qN9?= =?us-ascii?Q?UfYdUw0CHVgQzEU6bGabTT0POSZOG7ggVNfKZ3eZfd4QbaT03zo2RfmHP2Kd?= =?us-ascii?Q?G3uvovqzDwsXvWcQbzxo0ASxRJLSPAczuWNCnmKpzaQQf0kVZRwYE+kulKuH?= =?us-ascii?Q?BfGHlljpReRHPYdo1jrsgK4pQ/HmidDDFyC/5tvDOcv8zsBL5Llng4iILUb+?= =?us-ascii?Q?UgCQMJmlQa3Ka6qv+M7TAypCyHJRpXE/3XmfdDKhhDevCQXjYz/MFFPAIcnY?= =?us-ascii?Q?1WWtqgta5ut+CdX4XtpxfP0r/qBCn5aKljj4RjuFGZmMto91d1f5l0iLacvj?= =?us-ascii?Q?s8AOWy9+/HZPjTUopfsXpmaFb0E/Nsgl/YHFkvPAlr9xX8rkSPST52fr99yO?= =?us-ascii?Q?yY6qnZ3S7Y31dOavDX7/Uaol8Fsx8XVLnerPty7ffH7CRn6GPWHwnRKAfQ7C?= =?us-ascii?Q?2ErCWnMbp+kojDpcoL3KYeTmNhuWZgpe8E9fVlGx00y9CfwJlZkan/KWJ2u2?= =?us-ascii?Q?PIP5oMCqDpQ7YAZbylfjNoRuqmN+g8DU4FKPYz16DrPcIEPbA7ONozKDm8fJ?= =?us-ascii?Q?KxXCvPj9A76esECtaEe974Q2zIMlbBrZ82PVYQr0w3u8q6jd9CkggJej6FtA?= =?us-ascii?Q?IS30+3FfF3EXdZE1PsGFtSCa1apxYisJ65r440uZGfN9QMC1/Lz4JDqzKZq6?= =?us-ascii?Q?nlBdFVUroqjebvf4OB0+yJ1/S0sbkrs2ijjCpuMFQnsHOevUzg3J6HSAKWGP?= =?us-ascii?Q?4OD+lfMAMgJynQE+z3C1h0QZhobUlanfvMWvuDtoSwkV5HZriIxKtQGXahSe?= =?us-ascii?Q?35jNKe5D6JYdeiAiYSTFecUPHY4BQHFSvPB1/eY+C9ecZE7T7fis+OwEkAh+?= =?us-ascii?Q?adUcRptYdI5CM5XL+YjSnDXV?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8a7123dd-40f4-4b49-0a30-08d8eabef1b2 X-MS-Exchange-CrossTenant-AuthSource: AM7PR08MB5494.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2021 10:08:29.4400 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: felUL3u/Dv032u9yi5lknJlVcwm+ezcRXgjkgBfDS5XalCaFRY+XpYIXMdvyl8Z53f2WAoRuJt3HvHG6afiTgpXuH6bA+LuBCwN+MhHvYfI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3720 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=40.107.20.125; envelope-from=vsementsov@virtuozzo.com; helo=EUR05-DB8-obe.outbound.protection.outlook.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @virtuozzo.com) Content-Type: text/plain; charset="utf-8" There is a bug in qcow2: host cluster can be discarded (refcount becomes 0) and reused during data write. In this case data write may pollute another data cluster (recently allocated) or even metadata. To fix the issue introduce rw-lock: hold read-lock during data writing take write-lock when refcount becomes 0. Enable qcow2-discard-during-rewrite as it is fixed. Suggested-by: Kevin Wolf Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 20 +++++ block/qcow2-refcount.c | 22 ++++++ block/qcow2.c | 73 +++++++++++++++++-- .../tests/qcow2-discard-during-rewrite | 2 +- 4 files changed, 109 insertions(+), 8 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 0fe5f74ed3..7d387fe39d 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -420,6 +420,26 @@ typedef struct BDRVQcow2State { * is to convert the image with the desired compression type set. */ Qcow2CompressionType compression_type; + + /* + * discard_rw_lock prevents discarding host cluster when there are in-= flight + * writes into it. So, in-flight writes are "readers" and the code red= ucing + * cluster refcount to zero in update_refcount() is "writer". + * + * Data-writes should works as follows: + * 1. rd-lock should be acquired in the same s->lock critical section = where + * corresponding host clusters were allocated or somehow got from t= he + * metadata. Otherwise we risk miss discarding the cluster on s->lo= ck + * unlocking (unlock should only schedule another coroutine, not en= ter + * it, but better be absolutely safe). + * + * 2. rd-lock should be released after data writing finish. + * + * For reducing refcount to zero (and therefore allowing reallocating = the + * host cluster for other needs) it's enough to take rw-lock (to wait = for + * all in-flight writes) and immediately release it (see update_refcou= nt()). + */ + CoRwlock discard_rw_lock; } BDRVQcow2State; =20 typedef struct Qcow2COWRegion { diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 8e649b008e..bb6842cd8f 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -878,6 +878,28 @@ static int QEMU_WARN_UNUSED_RESULT update_refcount(Blo= ckDriverState *bs, } else { refcount +=3D addend; } + + if (qemu_in_coroutine() && refcount =3D=3D 0) { + /* + * Refcount becomes zero, but there are still may be in-fligth + * writes, that writing data to the cluster (that's done witho= ut + * qcow2 mutext held). + * + * Data writing protected by rd-locked discard_rw_lock. So here + * it's enough to take and immediately release wr-lock on it. + * We can immediately release it, because new write requests c= an't + * came to cluster which refcount becomes 0 (There should not = be any + * links from L2 tables to it). + * + * We can't do it if we are not in coroutine. But if we are no= t in + * coroutine, that also means that we are modifying metadata n= ot + * taking qcow2 s->lock mutex, which is wrong as well.. So, le= t's + * hope for a bright future. + */ + qemu_co_rwlock_wrlock(&s->discard_rw_lock); + qemu_co_rwlock_unlock(&s->discard_rw_lock); + } + if (refcount =3D=3D 0 && cluster_index < s->free_cluster_index) { s->free_cluster_index =3D cluster_index; } diff --git a/block/qcow2.c b/block/qcow2.c index 0db1227ac9..aea7aea334 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1899,6 +1899,7 @@ static int qcow2_open(BlockDriverState *bs, QDict *op= tions, int flags, =20 /* Initialise locks */ qemu_co_mutex_init(&s->lock); + qemu_co_rwlock_init(&s->discard_rw_lock); =20 if (qemu_in_coroutine()) { /* From bdrv_co_create. */ @@ -2182,9 +2183,17 @@ typedef struct Qcow2AioTask { QEMUIOVector *qiov; uint64_t qiov_offset; QCowL2Meta *l2meta; /* only for write */ + bool rdlock; /* only for write */ } Qcow2AioTask; =20 static coroutine_fn int qcow2_co_preadv_task_entry(AioTask *task); + +/* + * @rdlock: If true, it means that qcow2_add_task is called with discard_r= w_lock + * rd-locked, and this rd-lock must be transaferred to the task. The task = itself + * will release the lock. The caller expects that after qcow2_add_task() c= all + * the lock is already released. + */ static coroutine_fn int qcow2_add_task(BlockDriverState *bs, AioTaskPool *pool, AioTaskFunc func, @@ -2194,8 +2203,10 @@ static coroutine_fn int qcow2_add_task(BlockDriverSt= ate *bs, uint64_t bytes, QEMUIOVector *qiov, size_t qiov_offset, - QCowL2Meta *l2meta) + QCowL2Meta *l2meta, + bool rdlock) { + BDRVQcow2State *s =3D bs->opaque; Qcow2AioTask local_task; Qcow2AioTask *task =3D pool ? g_new(Qcow2AioTask, 1) : &local_task; =20 @@ -2209,6 +2220,7 @@ static coroutine_fn int qcow2_add_task(BlockDriverSta= te *bs, .bytes =3D bytes, .qiov_offset =3D qiov_offset, .l2meta =3D l2meta, + .rdlock =3D rdlock, }; =20 trace_qcow2_add_task(qemu_coroutine_self(), bs, pool, @@ -2217,10 +2229,24 @@ static coroutine_fn int qcow2_add_task(BlockDriverS= tate *bs, qiov, qiov_offset); =20 if (!pool) { + /* + * func will release rd-lock if needed and caller's expectation wo= uld be + * satisfied, so we should not care. + */ return func(&task->task); } =20 + /* + * We are going to run task in a different coroutine. We can't acquire= lock + * in one coroutine and release in another. So, the new coroutine shou= ld + * take it's own rd-lock, and we should release ours one. + */ + task->rdlock =3D false; aio_task_pool_start_task(pool, &task->task); + if (rdlock) { + assert(task->rdlock); /* caller took ownership */ + qemu_co_rwlock_unlock(&s->discard_rw_lock); + } =20 return 0; } @@ -2274,6 +2300,7 @@ static coroutine_fn int qcow2_co_preadv_task_entry(Ai= oTask *task) Qcow2AioTask *t =3D container_of(task, Qcow2AioTask, task); =20 assert(!t->l2meta); + assert(!t->rdlock); =20 return qcow2_co_preadv_task(t->bs, t->subcluster_type, t->host_offset, t->offset, t->bytes, @@ -2320,7 +2347,7 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDri= verState *bs, } ret =3D qcow2_add_task(bs, aio, qcow2_co_preadv_task_entry, ty= pe, host_offset, offset, cur_bytes, - qiov, qiov_offset, NULL); + qiov, qiov_offset, NULL, false); if (ret < 0) { goto out; } @@ -2483,19 +2510,32 @@ static int handle_alloc_space(BlockDriverState *bs,= QCowL2Meta *l2meta) * Called with s->lock unlocked * l2meta - if not NULL, qcow2_co_pwritev_task() will consume it. Caller = must * not use it somehow after qcow2_co_pwritev_task() call + * + * @rdlock must be non-NULL. + * If *@rdlock is true it means that discard_rw_lock is already taken. We = should + * not reacquire it, but caller expects that we release it. + * If *@rdlock is false, we should take it ourselves (and still release in= the + * end). When rd-lock is taken, we should set *@rdlock to true, so that pa= rent + * coroutine can check it. */ static coroutine_fn int qcow2_co_pwritev_task(BlockDriverState *bs, uint64_t host_offset, uint64_t offset, uint64_t by= tes, QEMUIOVector *qiov, uint64_t qiov_offset, - QCowL2Meta *l2meta) + QCowL2Meta *l2meta, + bool *rdlock) { int ret; BDRVQcow2State *s =3D bs->opaque; void *crypt_buf =3D NULL; QEMUIOVector encrypted_qiov; =20 + if (!*rdlock) { + qemu_co_rwlock_rdlock(&s->discard_rw_lock); + *rdlock =3D true; + } + if (bs->encrypted) { assert(s->crypto); assert(bytes <=3D QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); @@ -2538,12 +2578,14 @@ static coroutine_fn int qcow2_co_pwritev_task(Block= DriverState *bs, } } =20 + qemu_co_rwlock_unlock(&s->discard_rw_lock); qemu_co_mutex_lock(&s->lock); =20 ret =3D qcow2_handle_l2meta(bs, &l2meta, true); goto out_locked; =20 out_unlocked: + qemu_co_rwlock_unlock(&s->discard_rw_lock); qemu_co_mutex_lock(&s->lock); =20 out_locked: @@ -2563,7 +2605,7 @@ static coroutine_fn int qcow2_co_pwritev_task_entry(A= ioTask *task) =20 return qcow2_co_pwritev_task(t->bs, t->host_offset, t->offset, t->bytes, t->qiov, t->qiov_off= set, - t->l2meta); + t->l2meta, &t->rdlock); } =20 static coroutine_fn int qcow2_co_pwritev_part( @@ -2607,6 +2649,8 @@ static coroutine_fn int qcow2_co_pwritev_part( goto out_locked; } =20 + qemu_co_rwlock_rdlock(&s->discard_rw_lock); + qemu_co_mutex_unlock(&s->lock); =20 if (!aio && cur_bytes !=3D bytes) { @@ -2614,7 +2658,10 @@ static coroutine_fn int qcow2_co_pwritev_part( } ret =3D qcow2_add_task(bs, aio, qcow2_co_pwritev_task_entry, 0, host_offset, offset, - cur_bytes, qiov, qiov_offset, l2meta); + cur_bytes, qiov, qiov_offset, l2meta, true); + /* + * now discard_rw_lock is released and we are safe to take s->lock= again + */ l2meta =3D NULL; /* l2meta is consumed by qcow2_co_pwritev_task() = */ if (ret < 0) { goto fail_nometa; @@ -4094,10 +4141,15 @@ qcow2_co_copy_range_to(BlockDriverState *bs, goto fail; } =20 + qemu_co_rwlock_rdlock(&s->discard_rw_lock); qemu_co_mutex_unlock(&s->lock); + ret =3D bdrv_co_copy_range_to(src, src_offset, s->data_file, host_= offset, cur_bytes, read_flags, write_flags); + + qemu_co_rwlock_unlock(&s->discard_rw_lock); qemu_co_mutex_lock(&s->lock); + if (ret < 0) { goto fail; } @@ -4533,13 +4585,19 @@ qcow2_co_pwritev_compressed_task(BlockDriverState *= bs, } =20 ret =3D qcow2_pre_write_overlap_check(bs, 0, cluster_offset, out_len, = true); - qemu_co_mutex_unlock(&s->lock); if (ret < 0) { + qemu_co_mutex_unlock(&s->lock); goto fail; } =20 + qemu_co_rwlock_rdlock(&s->discard_rw_lock); + qemu_co_mutex_unlock(&s->lock); + BLKDBG_EVENT(s->data_file, BLKDBG_WRITE_COMPRESSED); ret =3D bdrv_co_pwrite(s->data_file, cluster_offset, out_len, out_buf,= 0); + + qemu_co_rwlock_unlock(&s->discard_rw_lock); + if (ret < 0) { goto fail; } @@ -4608,7 +4666,8 @@ qcow2_co_pwritev_compressed_part(BlockDriverState *bs, } =20 ret =3D qcow2_add_task(bs, aio, qcow2_co_pwritev_compressed_task_e= ntry, - 0, 0, offset, chunk_size, qiov, qiov_offset, = NULL); + 0, 0, offset, chunk_size, qiov, qiov_offset, = NULL, + false); if (ret < 0) { break; } diff --git a/tests/qemu-iotests/tests/qcow2-discard-during-rewrite b/tests/= qemu-iotests/tests/qcow2-discard-during-rewrite index dd9964ef3f..5df0048757 100755 --- a/tests/qemu-iotests/tests/qcow2-discard-during-rewrite +++ b/tests/qemu-iotests/tests/qcow2-discard-during-rewrite @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# group: quick disabled +# group: quick # # Test discarding (and reusing) host cluster during writing data to it. # --=20 2.29.2