From nobody Tue Oct 7 06:50:02 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D776F27B4E0; Sat, 12 Jul 2025 14:13:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329599; cv=none; b=qOI9DaYQTSyCuTxwkU/amHaYGA+l/+8SV+faxD4N6GKOQVwjQd3Mig3mPsdeuHfSwr2gj6GUsAWE9LFtquWnvoypdquYJvbkotq1xEsfEtirhXYlfGGVY3LzESHAVYFEwWDEAh7aYq1NAajeQMYCep95irLbnrFMaADb1NN3a9A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329599; c=relaxed/simple; bh=KGnhA3OEPNi5mPEFOwuBdga0KZcoROXpHFOdoqRmo3c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cZOkP4AN1rBHfluyJfFv1Rm/k+MowSE/XoNOIVFO+aLvYIe5HI9Ajnw5c54lX/jBoL6BsqsWEwxotXNVotRD7FPsP6U3ek4Cr4/B6PYgV8XT20NIyAzwjhZicXEUT16zHzsT+CxeD8QaqxqUH/tlnh3ZhzGRzU8bgmbMsJtklDU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=JoNUnKp1; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="JoNUnKp1" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 56C1ufbo022923; Sat, 12 Jul 2025 14:13:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=GeM76khIzlsvA662v qoGYtRNLFzJUS8exvmZZPtUZg0=; b=JoNUnKp1SQu0r0WGgAF1G+O2QdPNomQ6c WD66QnALnSJYnM6hI56d/+vKfvaMJtD/RlGo5HUqO97JYSWkWHr1E84VK44Th4Kz Hvhtt6pOtSe7U9KIh1dRmlpWN8o3HO+LUsri2mrXr1aPhDKcuuFOwFiaIE3Fedv+ RAwCkgEV9yNUd63+5V7oiwwTLbPbrV76dqbRiCBO7Qecwoik2DKyn5NXCBrKpLPa nZLSzY81OC9rwa4DG9kkLbKL2nJUbuebtrJsjaNrKH6gUXzqirXzhO2TobJ3W5r5 5f0dz++3hBGcG2rNSREBwGRCLq60ESuuuY43acdqNLTjwfgQzGDjQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ud4st2yb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:04 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 56CED46X004280; Sat, 12 Jul 2025 14:13:04 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ud4st2y9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:04 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 56CA6C91025586; Sat, 12 Jul 2025 14:13:03 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 47qfcpqe4m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:03 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 56CED1GR51708224 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 12 Jul 2025 14:13:01 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 689842004E; Sat, 12 Jul 2025 14:13:01 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2A93220040; Sat, 12 Jul 2025 14:12:59 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.252]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 12 Jul 2025 14:12:58 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v3 01/13] common/rc: Add _min() and _max() helpers Date: Sat, 12 Jul 2025 19:42:43 +0530 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNzEyMDEwNCBTYWx0ZWRfX2L/Go5CZivrC 6xm/Qmv8NZaqLc0VhIzmm6ubODY0xUjKgjwqLXp2iIf7xqlDmhtXy6BdMEfkl8UhOBqt3e+3bXM ORRrB8rYbsRP/QoY2cyBIimKmIvMaVnMMyAo74HXYacRf8GFE9qPONGHtSwAotPmyoSFA9q4VKn +pMho93AtsXEfBLy4VQPXIzC8bYOdY1kVjfRCE6qZg5byYAk6I6LAoEamq4oA5VWB4eMEanIbJ9 HN8D0nFMw8VcRRv8M9k7GGltIwXY3tbNxLLcoTBfGeQKBu/b574rPZa6MrHxiDjWWSN3afmTjjv SsA5DiObHVe+x6N4X/L3vmsulHYdsQhR/TuKnTKYYB0sAuIzkpdfNF+kBPy9OHuJ5dt+haEFBJW vcD11BktbQun2bUZftMLWoRrR0BIrX1vnrVzXcR26tazaELjLVsUGpPJv1NC7gKxIB/a35UX X-Proofpoint-GUID: zwQC4WZBwAVM2snA3fKW-PRE5P4BrNfV X-Proofpoint-ORIG-GUID: MtMJvNoX0Jo_DgfCM5YqKVAMf0QdS2Ru X-Authority-Analysis: v=2.4 cv=KezSsRYD c=1 sm=1 tr=0 ts=68726d70 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=Wb1JkmetP80A:10 a=VnNF1IyMAAAA:8 a=wDx1mg2EYdB0JXDzLs4A:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-07-12_02,2025-07-09_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 adultscore=0 clxscore=1015 suspectscore=0 phishscore=0 mlxlogscore=912 mlxscore=0 bulkscore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 malwarescore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2505280000 definitions=main-2507120104 Content-Type: text/plain; charset="utf-8" Many programs open code these functionalities so add it as a generic helper in common/rc Signed-off-by: Ojaswin Mujoo Reviewed-by: "Darrick J. Wong" --- common/rc | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/common/rc b/common/rc index f71cc8f0..9a9d3cc8 100644 --- a/common/rc +++ b/common/rc @@ -5817,6 +5817,28 @@ _require_program() { _have_program "$1" || _notrun "$tag required" } =20 +_min() { + local ret + + for arg in "$@"; do + if [ -z "$ret" ] || (( $arg < $ret )); then + ret=3D"$arg" + fi + done + echo $ret +} + +_max() { + local ret + + for arg in "$@"; do + if [ -z "$ret" ] || (( $arg > $ret )); then + ret=3D"$arg" + fi + done + echo $ret +} + ##########################################################################= ###### # make sure this script returns success /bin/true --=20 2.49.0 From nobody Tue Oct 7 06:50:02 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F0A027E7DE; Sat, 12 Jul 2025 14:13:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329602; cv=none; b=ihLClI0l/mxoWEVsi0/KPza20JoYfVN/IfAyRcC+ay03LaFgIX9DooBPmpB7bcuU/+bCUiFDnXi2maYX4Fz7vuIi0qZ/bxSzLwgnZDjc0CAdcRF7gG7wjhgLs/FH64PzGsrNxvmGDLxC5ga+e6egbFEb3f0by/3GW+4G9ONOeqE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329602; c=relaxed/simple; bh=L65ns+3OHyXvWJnP/IHECk63yip6LPqKKecJne2Bbt4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WH7t6bCRh0YpGr8aivbiXDm/VuUBpWrDOTiDFpwOzlOsxfbmr56njfkYC9MvZptw5JpSdd+gAwIAUqedqVWhvJa5nSglP8D6OMdliSJF+9p//nfrvX+gsT/3m2xLIvR3MCN9cJ+dGCsHuqXQMObdk4iE2oOpimOhE2UVV9BD0/U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=nEcxT5n1; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="nEcxT5n1" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 56CDsafk011353; Sat, 12 Jul 2025 14:13:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=2r62yGIrPPaKZ7oFj 2c6pL63818SyYbPJ5gyNw4/7yQ=; b=nEcxT5n18CQAxIaByNvyi2m6AEtoRXpl6 6bEsphSzzopw/85iyksd20TZYTymTDRhTNi4+sKd/n8xNeK5ptOZlLJV8b+I+2wE zHc7NKMzhBKQqlkMICpYJuuNBEj9weirLKa+2Vr+wWYWyssTJyMvwcFo8l5x6/q5 l/NwmaNhpJvRAs8gritK1HASIIvB8vYQYvVcE0TLJiAUlgLvja3AasY9lKGRD3fh wICoB/6BudphC2G9YHCCIGqzjQ37MRB3KK/f5Qv2Xy9bKHLuxGG6clgJeFb8gIKt 8unh9U4gndI/QpiOtRm7De8rq4rmu9W3CXiUJP4OXwMnLduHn9aXQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ue4thv41-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:07 +0000 (GMT) Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 56CE9W5i003156; Sat, 12 Jul 2025 14:13:06 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ue4thv3y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:06 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 56C9u6AP021566; Sat, 12 Jul 2025 14:13:06 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 47qecu7kx9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:05 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 56CED4tx17170730 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 12 Jul 2025 14:13:04 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 48ED020043; Sat, 12 Jul 2025 14:13:04 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CC4D220040; Sat, 12 Jul 2025 14:13:01 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.252]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 12 Jul 2025 14:13:01 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v3 02/13] common/rc: Fix fsx for ext4 with bigalloc Date: Sat, 12 Jul 2025 19:42:44 +0530 Message-ID: <84a1820482419a1f1fb599bc35c2b7dcc1abbcb9.1752329098.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=baBrUPPB c=1 sm=1 tr=0 ts=68726d73 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=Wb1JkmetP80A:10 a=pGLkceISAAAA:8 a=VnNF1IyMAAAA:8 a=SMpRl3u3EVCsjR3gKCoA:9 X-Proofpoint-GUID: wPYYBxaqqJzXEf40DcdhiFLFC1O8moGX X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNzEyMDEwNyBTYWx0ZWRfX0FmcW4qSAKYz ZWj09QziqfQehNSM3gigI9mk0ik+4IzJZTX8CjvJesrFntJPQBVVTRyF7yU+QHKdOs0M0B2JIBl eFMEWXdYiU3QGuDRjPmQfp5NP9GADcL8vFgcKWUMmvk/yOArm6nEtpq62LHWuSnBIo/LPDDDugN NHSbhiBzcFdu3mgS8GUcjrtSI8NTus/E4Nsc/pHbdGRfbY/z2MTkcB/Y9tls2aOdWxgwhxybI6+ 69ePpA+PQEheDWLQiutS6FiMo2NgHa2BKF28n0CKgyFptdU6Z5u8xTQ7XQy6XUry58wKRFr8qbS WycDZ3dkw1faJN5nPPVgQFOJUFquvK+YzYAJ1cc0/RWLZT8OGD7G42tGGZotjrJ27f/jsXnhzXS o7UytiC5uTJl2rP7mZ1cU0hacEa6B9AxkXReWbKQlUwUHexMZxGKHJHvU3mjU+wt0PC37eve X-Proofpoint-ORIG-GUID: Etm8vHtN0NX0TiSLUp5xPbW51wo9OnR7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-07-12_02,2025-07-09_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 impostorscore=0 phishscore=0 lowpriorityscore=0 priorityscore=1501 clxscore=1015 mlxscore=0 malwarescore=0 spamscore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2505280000 definitions=main-2507120107 Content-Type: text/plain; charset="utf-8" Insert range and collapse range only works with bigalloc in case the range is cluster size aligned, which fsx doesnt take care. To work past this, disable insert range and collapse range on ext4, if bigalloc is enabled. This is achieved by defining a new function _set_default_fsx_avoid called via run_fsx helper. This can be used to selectively disable fsx options based on the configuration. Co-developed-by: Ritesh Harjani (IBM) Signed-off-by: Ritesh Harjani (IBM) Signed-off-by: Ojaswin Mujoo --- common/rc | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/common/rc b/common/rc index 9a9d3cc8..218cf253 100644 --- a/common/rc +++ b/common/rc @@ -5113,10 +5113,37 @@ _require_hugepage_fsx() _notrun "fsx binary does not support MADV_COLLAPSE" } =20 +_set_default_fsx_avoid() { + local file=3D$1 + + case "$FSTYP" in + "ext4") + local dev=3D$(findmnt -n -o SOURCE --target $file) + + # open code instead of _require_dumpe2fs cause we don't + # want to _notrun if dumpe2fs is not available + if [ -z "$DUMPE2FS_PROG" ]; then + echo "_set_default_fsx_avoid: dumpe2fs not found, skipping bigalloc che= ck." >> $seqres.full + return + fi + + $DUMPE2FS_PROG -h $dev 2>&1 | grep -q bigalloc && { + export FSX_AVOID+=3D" -I -C" + } + ;; + # Add other filesystem types here as needed + *) + ;; + esac +} + _run_fsx() { echo "fsx $*" local args=3D`echo $@ | sed -e "s/ BSIZE / $bsize /g" -e "s/ PSIZE / $psi= ze /g"` + + _set_default_fsx_avoid $testfile + set -- $FSX_PROG $args $FSX_AVOID $TEST_DIR/junk echo "$@" >>$seqres.full rm -f $TEST_DIR/junk --=20 2.49.0 From nobody Tue Oct 7 06:50:02 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B71F27F19B; Sat, 12 Jul 2025 14:13:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329604; cv=none; b=F6FvnkOA51qdTj8eNdkcSPDRa844YQvbJEMW3OKmFkOKqbHN6tVeLISkGWiwL6V7vPLaSeG8v6wDxL3g6c1rUvEghqJnu8yDHINjP3N1NAR7MKSm/nPHTJN6a9Y0/gYFP10bBsz7obFiY4A9FtQMlxWTZ3me8Ai7MMNkWwTiSG8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329604; c=relaxed/simple; bh=JhF6Y6fJZhdiYaNs6VV47lmoUmyn0ue7rO4KOjdZttI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tmLFU0X6NBxKVY0OiBxwZpjL02oXZ+EHWpLVh1B46O01fw9n1YJVT8uZgViipnIWK+ZLv+OFtnfcvq+vmUywfFM0E859YtNX8qQkL3On/AR6LYBgwrGakYmeUALIS+FTkGaYxx0SF1b0J4Xx+Sm8O3lwDZJQBvLuMnX3kNOp9mY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=qH0fs1RS; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="qH0fs1RS" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 56C5tlQc024712; Sat, 12 Jul 2025 14:13:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=rR9isvtN7ai/NIAtB zCznRhNhwv7hZuUJH4LTrJyRAI=; b=qH0fs1RS7ErerAVBPDED0AVW6MK9+PlQ4 hvef/ox+lSQQ5pX0oJNWChDqirpeMHB2xD8rgkXoYlNx6cTT1XWKGg1gtNJTPVHl FAOGqZL88oNtk8zyA9ZXTOMIbB01tOF4vtzd0MhETuBlrgz7a1HEQvV6GQ7kIW9S hIoAAjYDtrUZqPHCzoCmhyI0Sd4oi/nSIvmWKH6xANrYFY5iwF4bRDi4dXnv5koi cy+j+oY01b+k2+ywbZ29LiE2LcF+O1uMrz19pLYXoUyMViuTU1ilCXD95IWZNS8+ wf+bakG9tKAisiqjpt9oNJ40YeWEHXc3jVeq58U2acvJCDFtbu0fw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ud4st2yg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:10 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 56CED9mn004363; Sat, 12 Jul 2025 14:13:09 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ud4st2ye-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:09 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 56CAjAlZ025623; Sat, 12 Jul 2025 14:13:09 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 47qfcpqe4q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:09 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 56CED7Bs18481598 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 12 Jul 2025 14:13:07 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 12ABF20043; Sat, 12 Jul 2025 14:13:07 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C581B20040; Sat, 12 Jul 2025 14:13:04 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.252]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 12 Jul 2025 14:13:04 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v3 03/13] common/rc: Add a helper to run fsx on a given file Date: Sat, 12 Jul 2025 19:42:45 +0530 Message-ID: <5334e25feda819fdc91e533d2ceaba05d9f711ac.1752329098.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNzEyMDEwNCBTYWx0ZWRfX/B9QOG68Y1rP +RwGEzL/EGJkf940fCLkfie7sRc7G36AtgylnjQtEkcszCWeYOENgvpggSmw62QX1R/BWQyxUzR 3IzwbmeZ6itVjfL+9SRNAwZ1RajbbDgS1JI8jawsbCfsjdaOpEIYqVa2hppXiawo58o/gS88CkL M+zoEMcJHUy/0nD+nm6aI8LdPxiGdOobKPMU8sqvdziT4lrZeaQhW8mi5zGd+jGEQoVCuS3zg8G BFLTXFZ7u1fEpJ1pp71JqUBXGMEwuIGGvTG5dy1HTsUIL3V9G7z+UNWB1rRjhUxPUOtjNYRwB7d AWe66rZri2gDtuNJv2lxXbgcGtA6Suzndt/4DiQQbqe0g9tZpvpjke1mI3oqKnxUPEgabM+PLiR 0mzKKPyiFdgv1O0P75emBhKRnOJ9NsWlm2+lWJgZsKlTVWTMAtZuVrgcYQyhYsKXyX9/CvfH X-Proofpoint-GUID: wZbtfa2c4j2n5vgGTM6LuHWCaEHiFLZl X-Proofpoint-ORIG-GUID: I0VXMQmKLIE6CoYSOhFxATCge8ZshRsa X-Authority-Analysis: v=2.4 cv=KezSsRYD c=1 sm=1 tr=0 ts=68726d76 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=u_VYNAqGXo6OSSAA:21 a=Wb1JkmetP80A:10 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=EfcJlRrZmV05EoLhlkQA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-07-12_02,2025-07-09_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 adultscore=0 clxscore=1015 suspectscore=0 phishscore=0 mlxlogscore=920 mlxscore=0 bulkscore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 malwarescore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2505280000 definitions=main-2507120104 Content-Type: text/plain; charset="utf-8" Currently run_fsx is hardcoded to run on a file in $TEST_DIR. Add a helper _run_fsx_on_file so that we can run fsx on any given file including in $SCRATCH_MNT. Also, refactor _run_fsx to use this helper. No functional change is intended in this patch. Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo --- common/rc | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/common/rc b/common/rc index 218cf253..2d4592b6 100644 --- a/common/rc +++ b/common/rc @@ -5137,16 +5137,26 @@ _set_default_fsx_avoid() { esac } =20 -_run_fsx() +_run_fsx_on_file() { + local testfile=3D$1 + shift + + if ! [ -f $testfile ] + then + echo "_run_fsx_on_file: $testfile doesn't exist. Creating" >> $seqres.fu= ll + touch $testfile + fi + echo "fsx $*" local args=3D`echo $@ | sed -e "s/ BSIZE / $bsize /g" -e "s/ PSIZE / $psi= ze /g"` =20 _set_default_fsx_avoid $testfile =20 - set -- $FSX_PROG $args $FSX_AVOID $TEST_DIR/junk + set -- $FSX_PROG $args $FSX_AVOID $testfile + echo "$@" >>$seqres.full - rm -f $TEST_DIR/junk + rm -f $testfile "$@" 2>&1 | tee -a $seqres.full >$tmp.fsx local res=3D${PIPESTATUS[0]} if [ $res -ne 0 ]; then @@ -5158,6 +5168,12 @@ _run_fsx() return 0 } =20 +_run_fsx() +{ + _run_fsx_on_file $TEST_DIR/junk $@ + return $? +} + # Run fsx with -h(ugepage buffers). If we can't set up a hugepage then sk= ip # the test, but if any other error occurs then exit the test. _run_hugepage_fsx() { --=20 2.49.0 From nobody Tue Oct 7 06:50:02 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 21F3027E052; Sat, 12 Jul 2025 14:13:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329601; cv=none; b=RXJ17eTVcKIYc/9v7lpLG3w5muMOMYjCYtnf1h7SHmDuCHYzPdcqHdH87cnM4j44b5TFm67FEYhdvHkBY5vktGL5GGTjrbHa7Ipkmjt821IeIuDhTeVpDUi/bcnmlEw8ZefdnXxH3cKaHStMiDpyY0dg4XYktxsbOm6cyutLDDU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329601; c=relaxed/simple; bh=d7Zgw8QUj1NAkq/4ExPCxrIx++FdxvLpdzEAi4p1Hoc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=X7LeF7SIkEGF2SHATSvDsafzTAxPEz6/lWklB3J3bQs/XihWzH3L/bMBwhIoJ3z5BZD4IwsJZxtmTrz5oYQyLV4fljO9pUhltcFu3JRp8oBqcG/krA8NeiMiTy91EyoRyD+PFFCQYdQ+x9f4muWD0jUNOI2k0IUPgefIXmidfnM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=WVMPybpy; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="WVMPybpy" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 56C0PiMT024270; Sat, 12 Jul 2025 14:13:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=+KKpbQs87NanNdRjU BLA8pJrlXjlbz/3c7g89DS6dS4=; b=WVMPybpyjEPR6hndhuS/qpBJNtea6BaDq usVJ7YY/ZOSuEZMwsyJniBZ9yPSspqLQ/g9BTlsgKD555ukX5AX15TLwiMvJt+cd 9yBDeYQY17c6dZkXMz6Q6GAjkSyeeRu2puwLQ10WT2q5mWML4X2vfg4ttwXyPuRv E5SLbkWlKlm9GbH+YQLTB3NuRqDh/Fgmqub9ruKgs6S+m5v4avJfzI2Z7jDpRcUM etI2FxUY/TGjTPGo3toAQLMUGgau0TcFtY+yJm2C4ydi3+QIJgV5Q28GazNSQiRx Q0aoYt4YfmXM3m7xukwox3TPql5corFdCjGKr9PzNYo+hbXDzdraw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ud4st2ym-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:13 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 56CEDCCP004443; Sat, 12 Jul 2025 14:13:12 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ud4st2yj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:12 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 56CA1uSg025593; Sat, 12 Jul 2025 14:13:11 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 47qfcpqe4r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:11 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 56CED9cZ54198630 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 12 Jul 2025 14:13:10 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D09E920043; Sat, 12 Jul 2025 14:13:09 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 761AF20040; Sat, 12 Jul 2025 14:13:07 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.252]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 12 Jul 2025 14:13:07 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v3 04/13] ltp/fsx.c: Add atomic writes support to fsx Date: Sat, 12 Jul 2025 19:42:46 +0530 Message-ID: <5bbd19e1615ca2a485b3b430c92f0260ee576f5e.1752329098.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNzEyMDEwNCBTYWx0ZWRfXz6BBk2hcdQjm hXqEKZ+L+p9aoG4hJhg8P9VG6yqKj5SWzGBTZVAvruGMt+5vy3SIMLCJt31ps7AwQZv1pJzWUww PzCPn5bVmmUyK0dNN3fGgUD78+sHbR3LwqxfGxPP3diC5mOa6RVkriId1or67Jb3+EkT//ozOqG 5Q6HacI4fqB5AvoAbwYJa79629ycZ6pRGnqvEtXO6lC5NgTCgEZfFrjf67pbXipizeMWZYKjuh9 MbmDpeIE1LZ+n7j92tpe89IpbINUKV2Tj0sKeEYrijQBJf8/bssLa+y4VjhV/ELxqGbn4YF59hu W+wK4n951c3SgBPdoNYtncAN+9u5yXUrj9EtaylCz06YnuMVpCH9J/4Cjcz3z3Tk+RuMfuD7U+S tswfLNYhNVotsA5HliwKYS90Efe3VN7DoH3pEzJ1qhsQmmKIOxseDMe4XqUcX3YbK6tLcxhn X-Proofpoint-GUID: giGXp_R72a6TqscCI9uyzRTn-Hk13kcs X-Proofpoint-ORIG-GUID: PpW8O8N-NFvxeXerkaDbs9h7882uNFKe X-Authority-Analysis: v=2.4 cv=KezSsRYD c=1 sm=1 tr=0 ts=68726d79 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=Wb1JkmetP80A:10 a=pGLkceISAAAA:8 a=VnNF1IyMAAAA:8 a=T5_vRgAD47iBWrMonTEA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-07-12_02,2025-07-09_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 adultscore=0 clxscore=1015 suspectscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 bulkscore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 malwarescore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2505280000 definitions=main-2507120104 Content-Type: text/plain; charset="utf-8" Implement atomic write support to help fuzz atomic writes with fsx. Suggested-by: Ritesh Harjani (IBM) Signed-off-by: Ojaswin Mujoo Reviewed-by: "Darrick J. Wong" --- ltp/fsx.c | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 104 insertions(+), 5 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 163b9453..ea39ca29 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -40,6 +40,7 @@ #include #endif #include +#include "statx.h" =20 #ifndef MAP_FILE # define MAP_FILE 0 @@ -49,6 +50,10 @@ #define RWF_DONTCACHE 0x80 #endif =20 +#ifndef RWF_ATOMIC +#define RWF_ATOMIC 0x40 +#endif + #define NUMPRINTCOLUMNS 32 /* # columns of data to print on each line */ =20 /* Operation flags (bitmask) */ @@ -110,6 +115,7 @@ enum { OP_READ_DONTCACHE, OP_WRITE, OP_WRITE_DONTCACHE, + OP_WRITE_ATOMIC, OP_MAPREAD, OP_MAPWRITE, OP_MAX_LITE, @@ -200,6 +206,11 @@ int uring =3D 0; int mark_nr =3D 0; int dontcache_io =3D 1; int hugepages =3D 0; /* -h flag */ +int do_atomic_writes =3D 1; /* -a flag disables */ + +/* User for atomic writes */ +int awu_min =3D 0; +int awu_max =3D 0; =20 /* Stores info needed to periodically collapse hugepages */ struct hugepages_collapse_info { @@ -288,6 +299,7 @@ static const char *op_names[] =3D { [OP_READ_DONTCACHE] =3D "read_dontcache", [OP_WRITE] =3D "write", [OP_WRITE_DONTCACHE] =3D "write_dontcache", + [OP_WRITE_ATOMIC] =3D "write_atomic", [OP_MAPREAD] =3D "mapread", [OP_MAPWRITE] =3D "mapwrite", [OP_TRUNCATE] =3D "truncate", @@ -422,6 +434,7 @@ logdump(void) prt("\t***RRRR***"); break; case OP_WRITE_DONTCACHE: + case OP_WRITE_ATOMIC: case OP_WRITE: prt("WRITE 0x%x thru 0x%x\t(0x%x bytes)", lp->args[0], lp->args[0] + lp->args[1] - 1, @@ -1073,6 +1086,25 @@ update_file_size(unsigned offset, unsigned size) file_size =3D offset + size; } =20 +static int is_power_of_2(unsigned n) { + return ((n & (n - 1)) =3D=3D 0); +} + +/* + * Round down n to nearest power of 2. + * If n is already a power of 2, return n; + */ +static int rounddown_pow_of_2(int n) { + int i =3D 0; + + if (is_power_of_2(n)) + return n; + + for (; (1 << i) < n; i++); + + return 1 << (i - 1); +} + void dowrite(unsigned offset, unsigned size, int flags) { @@ -1081,6 +1113,27 @@ dowrite(unsigned offset, unsigned size, int flags) offset -=3D offset % writebdy; if (o_direct) size -=3D size % writebdy; + if (flags & RWF_ATOMIC) { + /* atomic write len must be inbetween awu_min and awu_max */ + if (size < awu_min) + size =3D awu_min; + if (size > awu_max) + size =3D awu_max; + + /* atomic writes need power-of-2 sizes */ + size =3D rounddown_pow_of_2(size); + + /* atomic writes need naturally aligned offsets */ + offset -=3D offset % size; + + /* Skip the write if we are crossing max filesize */ + if ((offset + size) > maxfilelen) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping atomic write past maxfilelen\n"); + log4(OP_WRITE_ATOMIC, offset, size, FL_SKIPPED); + return; + } + } if (size =3D=3D 0) { if (!quiet && testcalls > simulatedopcount && !o_direct) prt("skipping zero size write\n"); @@ -1088,7 +1141,10 @@ dowrite(unsigned offset, unsigned size, int flags) return; } =20 - log4(OP_WRITE, offset, size, FL_NONE); + if (flags & RWF_ATOMIC) + log4(OP_WRITE_ATOMIC, offset, size, FL_NONE); + else + log4(OP_WRITE, offset, size, FL_NONE); =20 gendata(original_buf, good_buf, offset, size); if (offset + size > file_size) { @@ -1108,8 +1164,9 @@ dowrite(unsigned offset, unsigned size, int flags) (monitorstart =3D=3D -1 || (offset + size > monitorstart && (monitorend =3D=3D -1 || offset <=3D monitorend)))))) - prt("%lld write\t0x%x thru\t0x%x\t(0x%x bytes)\tdontcache=3D%d\n", testc= alls, - offset, offset + size - 1, size, (flags & RWF_DONTCACHE) !=3D 0); + prt("%lld write\t0x%x thru\t0x%x\t(0x%x bytes)\tdontcache=3D%d atomic_wr= =3D%d\n", testcalls, + offset, offset + size - 1, size, (flags & RWF_DONTCACHE) !=3D 0, + (flags & RWF_ATOMIC) !=3D 0); iret =3D fsxwrite(fd, good_buf + offset, size, offset, flags); if (iret !=3D size) { if (iret =3D=3D -1) @@ -1785,6 +1842,30 @@ do_dedupe_range(unsigned offset, unsigned length, un= signed dest) } #endif =20 +int test_atomic_writes(void) { + int ret; + struct statx stx; + + ret =3D xfstests_statx(AT_FDCWD, fname, 0, STATX_WRITE_ATOMIC, &stx); + if (ret < 0) { + fprintf(stderr, "main: Statx failed with %d." + " Failed to determine atomic write limits, " + " disabling!\n", ret); + return 0; + } + + if (stx.stx_attributes & STATX_ATTR_WRITE_ATOMIC && + stx.stx_atomic_write_unit_min > 0) { + awu_min =3D stx.stx_atomic_write_unit_min; + awu_max =3D stx.stx_atomic_write_unit_max; + return 1; + } + + fprintf(stderr, "main: IO Stack does not support " + "atomic writes, disabling!\n"); + return 0; +} + #ifdef HAVE_COPY_FILE_RANGE int test_copy_range(void) @@ -2356,6 +2437,12 @@ have_op: goto out; } break; + case OP_WRITE_ATOMIC: + if (!do_atomic_writes) { + log4(OP_WRITE_ATOMIC, offset, size, FL_SKIPPED); + goto out; + } + break; } =20 switch (op) { @@ -2385,6 +2472,11 @@ have_op: dowrite(offset, size, 0); break; =20 + case OP_WRITE_ATOMIC: + TRIM_OFF_LEN(offset, size, maxfilelen); + dowrite(offset, size, RWF_ATOMIC); + break; + case OP_MAPREAD: TRIM_OFF_LEN(offset, size, file_size); domapread(offset, size); @@ -2511,13 +2603,14 @@ void usage(void) { fprintf(stdout, "usage: %s", - "fsx [-dfhknqxyzBEFHIJKLORWXZ0]\n\ + "fsx [-adfhknqxyzBEFHIJKLORWXZ0]\n\ [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid]\n\ [-l flen] [-m start:end] [-o oplen] [-p progressinterval]\n\ [-r readbdy] [-s style] [-t truncbdy] [-w writebdy]\n\ [-A|-U] [-D startingop] [-N numops] [-P dirpath] [-S seed]\n\ [--replay-ops=3Dopsfile] [--record-ops[=3Dopsfile]] [--duration=3Dseco= nds]\n\ ... fname\n\ + -a: disable atomic writes\n\ -b opnum: beginning operation number (default 1)\n\ -c P: 1 in P chance of file close+open at each op (default infinity)\n\ -d: debug output for all operations\n\ @@ -3059,9 +3152,13 @@ main(int argc, char **argv) setvbuf(stdout, (char *)0, _IOLBF, 0); /* line buffered stdout */ =20 while ((ch =3D getopt_long(argc, argv, - "0b:c:de:fg:hi:j:kl:m:no:p:qr:s:t:uw:xyABD:EFJKHzCILN:OP:RS:UWXZ", + "0ab:c:de:fg:hi:j:kl:m:no:p:qr:s:t:uw:xyABD:EFJKHzCILN:OP:RS:UWXZ", longopts, NULL)) !=3D EOF) switch (ch) { + case 'a': + prt("main(): Atomic writes disabled\n"); + do_atomic_writes =3D 0; + break; case 'b': simulatedopcount =3D getnum(optarg, &endp); if (!quiet) @@ -3475,6 +3572,8 @@ main(int argc, char **argv) exchange_range_calls =3D test_exchange_range(); if (dontcache_io) dontcache_io =3D test_dontcache_io(); + if (do_atomic_writes) + do_atomic_writes =3D test_atomic_writes(); =20 while (keep_running()) if (!test()) --=20 2.49.0 From nobody Tue Oct 7 06:50:02 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9870927FD5A; Sat, 12 Jul 2025 14:13:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329605; cv=none; b=K9jrxbkwf14asr6nQCv5QsbEXtgBTRYikrKLv6hLtcWNqCTu9NO7+ww63Bz4poYg0WRn7GGNQroI4G7A2HpObE8dfpZAfQVEgPBhH5IGMXMYx5RcAYx+IAXk5s3FYLOqpY0HxEW1X4uj/4Xl5EeuYB7XtZOQp1XmPp3DrSBs8pU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329605; c=relaxed/simple; bh=33sNqFw/dIEXLfBV+bssmhsXLFSDEC+TSaoWvP3jet0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HSc40q9MTozuR/qsbstl4Mm3YA1qXmIWips5C6CM05yieaX9fgIgnBG52YgODkjRt1H42m/QSYkrKzezDAAUUgO2Jb/iFbsqo43ujSeMOWbjBoS7l/Zrt8zJB9wK+hcrocvGe0l/TCaTHpHmpduKDJZ2hv5Hmn5G5FnhbbHmR+c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=MTz6as9M; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="MTz6as9M" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 56CBRevL015518; Sat, 12 Jul 2025 14:13:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=Dpo+GxVAnJKnVsHbr uMPnygjKAgCy7bk1yjwL0JberA=; b=MTz6as9MH0VerFuEq21S8hDC3kENw5fSO EuE8F02o9vFSDstWnPkdIMn9e1fAJo9bG9NiERp+qHCNdoqAHG4k8JxQ6KvJ09Ah S1YltDUAMq+85Rq9xZsofzPmCQwyPvUn9c6XK44Qt7TCVNZUMdNoR/c2qmLHjgcq TBv2gwhyhwKkgt/inhsDz++Wb0vMdk7JOExbahgbdFIsLcvAcAt/PIiww814PspO vgsnHdqz7M1kDma00V27WB/dGefORC2VDtorDA296Ogl9+t+vo0r5/hX2MAgy0/v Z3GcQAezw3K0VOLnEmwC8M60W5rIYd6NrJzMIdGIwwjCoH+8IrZlA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ue4thv47-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:15 +0000 (GMT) Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 56CE9mqS003407; Sat, 12 Jul 2025 14:13:15 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ue4thv45-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:15 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 56C8rYg9010832; Sat, 12 Jul 2025 14:13:14 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 47qes0qk6r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:14 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 56CEDCNr34406734 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 12 Jul 2025 14:13:12 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 748CE20043; Sat, 12 Jul 2025 14:13:12 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3961B20040; Sat, 12 Jul 2025 14:13:10 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.252]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 12 Jul 2025 14:13:09 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v3 05/13] generic/1226: Add atomic write test using fio crc check verifier Date: Sat, 12 Jul 2025 19:42:47 +0530 Message-ID: <1e6dad5f4bdc8107e670cc0bd3ce0fccd0c9037a.1752329098.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=baBrUPPB c=1 sm=1 tr=0 ts=68726d7b cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=Wb1JkmetP80A:10 a=pGLkceISAAAA:8 a=VnNF1IyMAAAA:8 a=TDUsxBfbQoAVH7yvDjsA:9 X-Proofpoint-GUID: RLkQSbKDyKq2eRQvOYpk5xaOf_f92E3n X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNzEyMDEwNyBTYWx0ZWRfX/FS3B8SfoU9B ddy/nYnVUzuKIhQpzswmzPsbRhXLzYafQWgoAxWPcmby1UQXxDJjjThH3hdwDlcPpj4Ld8Ma8w3 F+bUxUuvleGPiSbExESM9PPyIJECu9Xm7ys2ld8etRjuTxgbMD5dDcDVSGOH+C9uHbjb/ikuAuW 79FzAJyVHxzs38hycMgfADpxfrtJtjI2M5uhR6jayV3ou96JBLrsKZzE+2k8oD+7XCyuGGTUdjq uzOWmbU8Ut/mktvZmi6HoQFV1hCorD61N9jEuGkVpV6ZxBCgXLSSWoAT5Druy216qFB0ILypXqi hYpldXg1qR7DEiJcFdyxmg3CEe5i7J906dNet4ONvZVKSIznhRtwwBVf0kT2HLJbPFv1ZCZa+ni TzQZy7PHApEDG5GL0kRSHfrgrC4CtrBq2AkS+lTIQiGbARRnBQTq2o/Yi3Guzja/IHXLP28s X-Proofpoint-ORIG-GUID: MYTLZeoP1tUSq9jCDda48dRAEi0JEKiN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-07-12_02,2025-07-09_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 mlxlogscore=918 suspectscore=0 adultscore=0 impostorscore=0 phishscore=0 lowpriorityscore=0 priorityscore=1501 clxscore=1015 mlxscore=0 malwarescore=0 spamscore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2505280000 definitions=main-2507120107 Content-Type: text/plain; charset="utf-8" From: "Ritesh Harjani (IBM)" This adds atomic write test using fio based on it's crc check verifier. fio adds a crc for each data block. If the underlying device supports atomic write then it is guaranteed that we will never have a mix data from two threads writing on the same physical block. Co-developed-by: Ritesh Harjani (IBM) Signed-off-by: Ritesh Harjani (IBM) Signed-off-by: Ojaswin Mujoo --- tests/generic/1226 | 101 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1226.out | 2 + 2 files changed, 103 insertions(+) create mode 100755 tests/generic/1226 create mode 100644 tests/generic/1226.out diff --git a/tests/generic/1226 b/tests/generic/1226 new file mode 100755 index 00000000..455fc55f --- /dev/null +++ b/tests/generic/1226 @@ -0,0 +1,101 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1226 +# +# Validate FS atomic write using fio crc check verifier. +# +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto aio rw atomicwrites + +_require_scratch_write_atomic +_require_odirect +_require_aio + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +touch "$SCRATCH_MNT/f1" +awu_min_write=3D$(_get_atomic_write_unit_min "$SCRATCH_MNT/f1") +awu_max_write=3D$(_get_atomic_write_unit_max "$SCRATCH_MNT/f1") +blocksize=3D$(_max "$awu_min_write" "$((awu_max_write/2))") + +fio_config=3D$tmp.fio +fio_out=3D$tmp.fio.out + +FIO_LOAD=3D$(($(nproc) * 2 * LOAD_FACTOR)) +SIZE=3D$((100 * 1024 * 1024)) + +function create_fio_configs() +{ + create_fio_aw_config + create_fio_verify_config +} + +function create_fio_verify_config() +{ +cat >$fio_verify_config <$fio_aw_config <> $seqres.full +cat $fio_verify_config >> $seqres.full + +$FIO_PROG $fio_aw_config >> $seqres.full +ret1=3D$? +$FIO_PROG $fio_verify_config >> $seqres.full +ret2=3D$? + +[[ $ret1 -eq 0 && $ret2 -eq 0 ]] || _fail "fio with atomic write failed" + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/generic/1226.out b/tests/generic/1226.out new file mode 100644 index 00000000..6dce0ea5 --- /dev/null +++ b/tests/generic/1226.out @@ -0,0 +1,2 @@ +QA output created by 1226 +Silence is golden --=20 2.49.0 From nobody Tue Oct 7 06:50:02 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A1F2284B56; Sat, 12 Jul 2025 14:13:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329607; cv=none; b=nqhLJ6uEg+YofNGp0NuGuBcch7xQ/w4/ccOZ5ud6svuNHXBgT8Zh+9ypiolgTFgn5/GIQO9oPMFovYcBfmCO9qL/rvXOXWlBK5UmMUPdrY7NQtYLHxcx6wieXzOnl08NSVG5K3VpCNWOdD940o6q4aHXa/OArXcKbX8hqrhxiYM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329607; c=relaxed/simple; bh=NbKJl4MIkBjbKokRgks+75U1IqsVpspVnPb0d087uRU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IsG2WJhxc0nl6JX6Ko3cPOi9mdN4QKy6tT1a4NIFnUEmhh5hkRCc+wkqsawHlCJZC+lCCAijU56ViVW5pC6aINLxa9xkdEgdZ3qp/Wh1b0DCA8Em8C6gCSwbVCseu8TikCj5uQxZ97uQiczG9mIzs7DTVmn5aGM/McsMlNyaidY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=WLTSc3pa; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="WLTSc3pa" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 56C2wLYO005951; Sat, 12 Jul 2025 14:13:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=rT9EgbiIW9JY2UPbN E2nI/jlnTr+TgFGnm5C5P1lbkk=; b=WLTSc3papKwPjvsBOdYjGSNc7LJpOe4UR fBNl/9xNY5Glf1XsK7q1oO5cnBVKJeJlp1ocTx66C88Is3t3FDkGdiG4PzJHU6AO UvHa/ZK/TCr2ToNTeTQHFr4FWdNbmhpICXLwIjQhJBNtrkh4lGJT/3jeFV59O3/u aYaZGUBJbp3HexECWAHRBkpsgh0jVFSTC9ezrYe1pwEjfbVXNqsUiYEuh+mJwBl7 cfA7PeLkwLER/IH7tc223omSM11h3wirQ4k0tR8p6Ee9+bSjHwkeIuo8/E/T+1IC RSaOWEyHiYrEBe1l/qXEWUBuOfv8J5voyqDwC4pIMCxKkXJ3EE0Lg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ufcysr7d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:18 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 56CEDIb4008429; Sat, 12 Jul 2025 14:13:18 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ufcysr7b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:18 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 56C91GFZ010841; Sat, 12 Jul 2025 14:13:16 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 47qes0qk6u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:16 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 56CEDErO35521154 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 12 Jul 2025 14:13:15 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D289520043; Sat, 12 Jul 2025 14:13:14 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BEA9020040; Sat, 12 Jul 2025 14:13:12 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.252]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 12 Jul 2025 14:13:12 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v3 06/13] generic/1227: Add atomic write test using fio verify on file mixed mappings Date: Sat, 12 Jul 2025 19:42:48 +0530 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=J6iq7BnS c=1 sm=1 tr=0 ts=68726d7e cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=Wb1JkmetP80A:10 a=pGLkceISAAAA:8 a=VnNF1IyMAAAA:8 a=ZzjZKRc1KEgCYT2eLxwA:9 X-Proofpoint-GUID: vQXvLn4qNjY_a4grMjnswC7N7woNlctU X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNzEyMDEwNyBTYWx0ZWRfX5zNynLNRfb34 dnNuOLGOJmdvoEof9O0hyHsnDYOkQ6JE8SWLiAV0RpWG6OeQCzJURXqE/PF7O9/1aLJbXbqvERC ouKgbbL+/9kp/TORFdO11YNPKnHdwbhlKX/S/D/2KdJ0u81ov5ghHn4mCZkAmjzD+gTCYgvXdPI DlNsyxvTmP1O+/bP6RV19/XpYzgQcnpbWoTzeJG/coMSV3XR0pLBSKexP8SEq+5h16kTOOMsQwz Hkp02rxowNS9cyiJZ8NMzwxXTwWjeqdRspOwu4pseWOV0kDyeibqVLCTAulYMx4fusI351m0UmT aNK/fJHkirn9+kWZw7p2PYp6YE9Dn4eu0uP9VM0J0S5NKUq0PkP39/m0B7UFEVZYfQuskWNwgEv 7MkdzpIRLTiTJ6KTp+BgPk7jW0BskI00a4KWqQ2ffa/QK3gor5XCgNDWRGGqI94luLguhLmH X-Proofpoint-ORIG-GUID: amrTx6z_leAjrOAm5_agPxdX2t6vVbfu X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-07-12_02,2025-07-09_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 impostorscore=0 malwarescore=0 priorityscore=1501 adultscore=0 mlxscore=0 suspectscore=0 bulkscore=0 mlxlogscore=999 spamscore=0 lowpriorityscore=0 phishscore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2505280000 definitions=main-2507120107 Content-Type: text/plain; charset="utf-8" From: "Ritesh Harjani (IBM)" This tests uses fio to first create a file with mixed mappings. Then it does atomic writes using aio dio with parallel jobs to the same file with mixed mappings. This forces the filesystem allocator to allocate extents over mixed mapping regions to stress FS block allocators. Co-developed-by: Ritesh Harjani (IBM) Signed-off-by: Ritesh Harjani (IBM) Signed-off-by: Ojaswin Mujoo Reviewed-by: "Darrick J. Wong" Reviewed-by: Zorro Lang --- tests/generic/1227 | 123 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1227.out | 2 + 2 files changed, 125 insertions(+) create mode 100755 tests/generic/1227 create mode 100644 tests/generic/1227.out diff --git a/tests/generic/1227 b/tests/generic/1227 new file mode 100755 index 00000000..cfdc54ec --- /dev/null +++ b/tests/generic/1227 @@ -0,0 +1,123 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1227 +# +# Validate FS atomic write using fio crc check verifier on mixed mappings +# of a file. +# +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto aio rw atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_odirect +_require_aio + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +touch "$SCRATCH_MNT/f1" +awu_min_write=3D$(_get_atomic_write_unit_min "$SCRATCH_MNT/f1") +awu_max_write=3D$(_get_atomic_write_unit_max "$SCRATCH_MNT/f1") +aw_bsize=3D$(_max "$awu_min_write" "$((awu_max_write/4))") + +fsbsize=3D$(_get_block_size $SCRATCH_MNT) + +fio_prep_config=3D$tmp.prep.fio +fio_aw_config=3D$tmp.aw.fio +fio_verify_config=3D$tmp.verify.fio +fio_out=3D$tmp.fio.out + +FIO_LOAD=3D$(($(nproc) * 2 * LOAD_FACTOR)) +SIZE=3D$((128 * 1024 * 1024)) + +cat >$fio_prep_config <$fio_aw_config <$fio_verify_config <> $seqres.full +cat $fio_aw_config >> $seqres.full +cat $fio_verify_config >> $seqres.full + +#prepare file with mixed mappings +$FIO_PROG $fio_prep_config >> $seqres.full + +# do atomic writes without verifying +$FIO_PROG $fio_aw_config >> $seqres.full + +# verify data is not torn +$FIO_PROG $fio_verify_config >> $seqres.full + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/generic/1227.out b/tests/generic/1227.out new file mode 100644 index 00000000..2605d062 --- /dev/null +++ b/tests/generic/1227.out @@ -0,0 +1,2 @@ +QA output created by 1227 +Silence is golden --=20 2.49.0 From nobody Tue Oct 7 06:50:02 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2B1A27F19B; Sat, 12 Jul 2025 14:13:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329609; cv=none; b=Il5+LU1seoqX9GzvgK1l8suMPiqkZtxrHMufbC12jDrJsQSJLEqlP53wKLCW8N++hqJSVNy0GjKBYD5zmgbi+rrEElPeLRw5m2BXSFQaFVDqK/xFl+DZX7QcQhW0MJtWZLtWjGQVxVSG6eQEnWflNiBnFgBP2EqRCTC7B6pB+9g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329609; c=relaxed/simple; bh=qfTNrVszUJrJ9X5OWWdIiNeQVrPCrDeBfKMcJnCHPrI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FiBA8jl7JMFx8jTODohN8OrBZ7XUBvCKIONv4L8Kjhj2md/lZKWH/hUarPIaRu4BSrUkTy6rOLxY8d6NlROExgNWKePTn6K3ch0VdoGTTudXabihjRWTFgcxtG65gX2uf24csmMXFSRvlEgzoqsTiVajXermLPurlw7hPkzYbv4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=U5L4cUF2; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="U5L4cUF2" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 56C8i00h028787; Sat, 12 Jul 2025 14:13:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=jnPUZ5Gvv3upS+qTq KyPnVTG8IdhzTg0fV6sfLAbL/4=; b=U5L4cUF2H0ffhGibTyfbiluxMXmRBNKr2 cmUgqxZxyZQMbMLE0LBwVi6C7Xi4e9JtHnjADi2O2NSyJjeQs72AsRM1Y3vQprvV qlKeUVf/VHEuFaxHrWYZzZARh6xj6RvtvhW7Kip7mbDKI63uZuQHXQ3E5UWRVrGy GfTpiWNctBKgltlItiAQkfbnIDaHM1xkUCyRiJXWcz7vWewcnLqKbiQJwM7i5aqm 164Nzd2jyWAAmvOQbhBR0JaxBBim4/UrHpMUBRPmaajAKQS3AxqMj279VAOICbIT iWoCABjp4oFg/e7+kp/H1xvMeNFemuZhp28EZ4KNYhID+d75a1XjQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ue4thv4b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:20 +0000 (GMT) Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 56CEDKUS008828; Sat, 12 Jul 2025 14:13:20 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ue4thv49-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:20 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 56CB0DgD002888; Sat, 12 Jul 2025 14:13:19 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 47qfvmy96h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:19 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 56CEDHfP50594128 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 12 Jul 2025 14:13:17 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 953DA20043; Sat, 12 Jul 2025 14:13:17 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4685320040; Sat, 12 Jul 2025 14:13:15 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.252]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 12 Jul 2025 14:13:15 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v3 07/13] generic/1228: Add atomic write multi-fsblock O_[D]SYNC tests Date: Sat, 12 Jul 2025 19:42:49 +0530 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=baBrUPPB c=1 sm=1 tr=0 ts=68726d81 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=Wb1JkmetP80A:10 a=pGLkceISAAAA:8 a=VnNF1IyMAAAA:8 a=gLbHiHCgn5DP7liA1jIA:9 X-Proofpoint-GUID: F6qj_X1kYoVpbdHmmJO4E2QYVEfRETtR X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNzEyMDEwNyBTYWx0ZWRfX7zPPLAUMXZc6 oSHwu1lad0XqCRZQKY1xJRJTPFN+pkHjW5eCXDEJ9cgY4SrzSoxbld7hflVX28VdEQHyXjfmy5+ q9PFGIEvXwdyj589A7OyqYng8xbLL2vroBZTBolv0h+eZ9xNs2h50OiOxQRfHM2G7sXZtyyKZfx kZeGUF423+k2mzXLRydJp06rHWRko4ki7379NzF1YUATDooZ2ml0EeRLxt5y9RkBzUlJ/67adCX Fq7Ur4Baeb3bmc8fQC0sh3jb1tQA+t8+JlAeHndRewMmlsYodFXPD124qaglqmahAz2t1e13wfj h9IrW5v4qZauAYTdJdz9bN/5epbT4TWBwvMj21D8QlbHxvWXhmP0ut4A6AQ8Ofn+TIVJ8n4mwq7 p9RInQ/M2MVeOwuBme7afqBOGqe1GH5zF2LQfIy+FsgsPAIoRLTCi/Et10bcxeCNvIq2VwHT X-Proofpoint-ORIG-GUID: 9f9BEVOGHkrhG0L7XQpWn_B8L0EQ-6aL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-07-12_02,2025-07-09_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 impostorscore=0 phishscore=0 lowpriorityscore=0 priorityscore=1501 clxscore=1015 mlxscore=0 malwarescore=0 spamscore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2505280000 definitions=main-2507120107 Content-Type: text/plain; charset="utf-8" This adds various atomic write multi-fsblock stresst tests with mixed mappings and O_SYNC, to ensure the data and metadata is atomically persisted even if there is a shutdown. Suggested-by: Ritesh Harjani (IBM) Signed-off-by: Ojaswin Mujoo --- tests/generic/1228 | 139 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1228.out | 2 + 2 files changed, 141 insertions(+) create mode 100755 tests/generic/1228 create mode 100644 tests/generic/1228.out diff --git a/tests/generic/1228 b/tests/generic/1228 new file mode 100755 index 00000000..3f9a6af1 --- /dev/null +++ b/tests/generic/1228 @@ -0,0 +1,139 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1228 +# +# Atomic write multi-fsblock data integrity tests with mixed mappings +# and O_SYNC +# +. ./common/preamble +. ./common/atomicwrites +_begin_fstest auto quick rw atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_atomic_write_test_commands +_require_scratch_shutdown +_require_xfs_io_command "truncate" + +_scratch_mkfs >> $seqres.full +_scratch_mount >> $seqres.full + +check_data_integrity() { + actual=3D$(_hexdump $testfile) + if [[ "$expected" !=3D "$actual" ]] + then + echo "Integrity check failed" + echo "Integrity check failed" >> $seqres.full + echo "# Expected file contents:" >> $seqres.full + echo "$expected" >> $seqres.full + echo "# Actual file contents:" >> $seqres.full + echo "$actual" >> $seqres.full + + _fail "Data integrity check failed. The atomic write was torn." + fi +} + +prep_mixed_mapping() { + $XFS_IO_PROG -c "truncate 0" $testfile >> $seqres.full + local off=3D0 + local mapping=3D"" + + local operations=3D("W" "H" "U") + local num_blocks=3D$((awu_max / blksz)) + for ((i=3D0; i /d= ev/null + ;; + "H") + # No operation needed for hole + ;; + "U") + $XFS_IO_PROG -c "falloc $off $blksz" $testfile >> /dev/null + ;; + esac + off=3D$((off + blksz)) + done + + echo "+ + Mixed mapping prep done. Full mapping pattern: $mapping" >> $se= qres.full + + sync $testfile +} + +verify_atomic_write() { + if [[ "$1" =3D=3D "shutdown" ]] + then + local do_shutdown=3D1 + fi + + test $bytes_written -eq $awu_max || _fail "atomic write len=3D$awu_max as= sertion failed" + + if [[ $do_shutdown -eq "1" ]] + then + echo "Shutting down filesystem" >> $seqres.full + _scratch_shutdown >> $seqres.full + _scratch_cycle_mount >>$seqres.full 2>&1 || _fail "remount failed for Te= st-3" + fi + + check_data_integrity +} + +mixed_mapping_test() { + prep_mixed_mapping + + echo "+ + Performing O_DSYNC atomic write from 0 to $awu_max" >> $seqres.= full + bytes_written=3D$($XFS_IO_PROG -dc "pwrite -DA -V1 -b $awu_max 0 $awu_max= " $testfile | \ + grep wrote | awk -F'[/ ]' '{print $2}') + + verify_atomic_write $1 +} + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile + +awu_max=3D$(_get_atomic_write_unit_max $testfile) +blksz=3D$(_get_block_size $SCRATCH_MNT) + +# Create an expected pattern to compare with +$XFS_IO_PROG -tc "pwrite -b $awu_max 0 $awu_max" $testfile >> $seqres.full +expected=3D$(_hexdump $testfile) +echo "# Expected file contents:" >> $seqres.full +echo "$expected" >> $seqres.full +echo >> $seqres.full + +echo "# Test 1: Do O_DSYNC atomic write on random mixed mapping:" >> $seqr= es.full +echo >> $seqres.full +for ((iteration=3D1; iteration<=3D10; iteration++)); do + echo "=3D=3D=3D Mixed Mapping Test Iteration $iteration =3D=3D=3D" >> $se= qres.full + + echo "+ Testing without shutdown..." >> $seqres.full + mixed_mapping_test + echo "Passed!" >> $seqres.full + + echo "+ Testing with sudden shutdown..." >> $seqres.full + mixed_mapping_test "shutdown" + echo "Passed!" >> $seqres.full + + echo "Iteration $iteration completed: OK" >> $seqres.full + echo >> $seqres.full +done +echo "# Test 1: Do O_SYNC atomic write on random mixed mapping (10 iterati= ons): OK" >> $seqres.full + + +echo >> $seqres.full +echo "# Test 2: Do extending O_SYNC atomic writes: " >> $seqres.full +bytes_written=3D$($XFS_IO_PROG -dstc "pwrite -A -V1 -b $awu_max 0 $awu_max= " $testfile | \ + grep wrote | awk -F'[/ ]' '{print $2}') +verify_atomic_write "shutdown" +echo "# Test 2: Do extending O_SYNC atomic writes: OK" >> $seqres.full + +# success, all done +echo "Silence is golden" +status=3D0 +exit + diff --git a/tests/generic/1228.out b/tests/generic/1228.out new file mode 100644 index 00000000..1baffa91 --- /dev/null +++ b/tests/generic/1228.out @@ -0,0 +1,2 @@ +QA output created by 1228 +Silence is golden --=20 2.49.0 From nobody Tue Oct 7 06:50:02 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC881289814; Sat, 12 Jul 2025 14:13:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329613; cv=none; b=uHePtPVkIEBzQnHFZW/jUgUqsvGu6NXUieDNS5ec/FTgA/qnIgCRQf6RsK0QXnBO5s/kYHMYsH51XeWxI57nuZekTXW03pBSIZBNG3vwAzaTquVjT/Y267YTtlc7pAkqsoFyrCzwfiOeM4ipCExdpEI5xcRFTgk680RB11E9mUM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329613; c=relaxed/simple; bh=WanGwxHlBHd/+1ooGk/OJ6/nLWjp8/ahCzh2gOmWpkQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QRRte+2QECoo5361g8R/gwENyBdRmjmQ84MMUWzkz2vFxbOayIwM172lRUcWXolvjxYpnv3aqFk5ti8SIn9wsE04Ze1PuqevFU9K0pSaLTov3SQgUyv5b6qhxf+2TyvGy7DVHCbknThZIpv14qPi9SE3JL3g0c/qbLvW1SlHl6M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=c42IurbO; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="c42IurbO" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 56CBwYlL024719; Sat, 12 Jul 2025 14:13:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=QfO5LbuVEdFD6MYBG +KY8MBTI7LF3bPGUyNsn/KAUzs=; b=c42IurbOOoLYl4C8qiZA1+PN7eGLhdJi6 mgyISr0X+FW648wWmgQlggXVdv7SG5aB0THersOoDAOEvLuIMV/VRQUdt+6R/OpN swRjA6o6SM9T2dfMvtKFk4ii7BWgfQY48Y9bolDRtqiW54PzOmOCNuOOA861JgL5 hq3+Xe9uFb8MHxmp/aMvA2SINpxAxbu4xf8ExDFP6CDtm/iVHCbZdQiHpbuibjgB hNQniJVoUvkpMD3IXn3bxn66cmqXkg41PFM0OA1HtJlPtiZutFEi1o3Nyw/K7LwL TsJi/bUxecrDS1l4q32nR2zQJe16ug7f0vmlkXwqRe6PqVYvkP6SQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ufeehqtj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:24 +0000 (GMT) Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 56CEDNKJ031205; Sat, 12 Jul 2025 14:13:23 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ufeehqtf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:23 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 56C9obF7025642; Sat, 12 Jul 2025 14:13:22 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 47qfcpqe4u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:22 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 56CEDKg835848628 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 12 Jul 2025 14:13:20 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 42F8320043; Sat, 12 Jul 2025 14:13:20 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 03DDF20040; Sat, 12 Jul 2025 14:13:18 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.252]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 12 Jul 2025 14:13:17 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v3 08/13] generic/1229: Stress fsx with atomic writes enabled Date: Sat, 12 Jul 2025 19:42:50 +0530 Message-ID: <1e1e7d552e91fab58037b7b35ffbf8b2e7070be5.1752329098.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNzEyMDEwNyBTYWx0ZWRfX8MhNBY7hH4Dd ghO7a6Kx/Wu1PomAqgGgcFCFCyiW5Y524JsRxtJQ1bTCet/2UzErl2zuBkZFeWDkTuq/vNWCBEp gRd6r/VvgU63FZ+LfT48RhWpGUFsbhECQIDMro1YUPlnphegsrU5djjZkycGHiZDgalWHeSkGTM KVNVQ1i4SH5g5Lb7pAj1ja0aNZLYT6URXLv1ZGZj4cLecmVYW2kRPe3ngYtWdW72c3bZrK7slRP nekGDcSGyfD5JCKHt7te7kcYwDdGHDUDXWz4B4pynWH/a/HMLgFHiuECMOnLJaWopP690/6DQSa bIoveumr3+kpqpAuxHaqTCIFwJunC38oXMkX4lIS6dZBvG1wNaB5qwG5F25TWKjHES7HEkqs/oB pwGXUELwlzpMcrXdJrlBWjA2rJbOBttphJPWXBpKCSvi8rSKBcrjNMl33OpESuhAXiCozcNW X-Proofpoint-ORIG-GUID: q2YYk-6xTqq9a0JkHl6WMN6Sm1MayKAv X-Authority-Analysis: v=2.4 cv=C9/pyRP+ c=1 sm=1 tr=0 ts=68726d84 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=Wb1JkmetP80A:10 a=pGLkceISAAAA:8 a=VnNF1IyMAAAA:8 a=JyoSrJM7ovCTkwbihYMA:9 a=U1FKsahkfWQA:10 X-Proofpoint-GUID: 68j3FQTHx7-vvIQ4Dc-nFslgOgBc8Pbg X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-07-12_02,2025-07-09_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 impostorscore=0 spamscore=0 suspectscore=0 priorityscore=1501 lowpriorityscore=0 mlxlogscore=796 bulkscore=0 adultscore=0 clxscore=1015 mlxscore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2505280000 definitions=main-2507120107 Content-Type: text/plain; charset="utf-8" Stress file with atomic writes to ensure we excercise codepaths where we are mixing different FS operations with atomic writes Suggested-by: Ritesh Harjani (IBM) Signed-off-by: Ojaswin Mujoo Reviewed-by: "Darrick J. Wong" --- tests/generic/1229 | 41 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1229.out | 2 ++ 2 files changed, 43 insertions(+) create mode 100755 tests/generic/1229 create mode 100644 tests/generic/1229.out diff --git a/tests/generic/1229 b/tests/generic/1229 new file mode 100755 index 00000000..98e9b50c --- /dev/null +++ b/tests/generic/1229 @@ -0,0 +1,41 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1229 +# +# fuzz fsx with atomic writes +# +. ./common/preamble +. ./common/atomicwrites +_begin_fstest rw auto quick atomicwrites + +_require_odirect +_require_scratch_write_atomic + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount >> $seqres.full 2>&1 + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile + +awu_max=3D$(_get_atomic_write_unit_max $testfile) +blksz=3D$(_get_block_size $SCRATCH_MNT) +bsize=3D`$here/src/min_dio_alignment $SCRATCH_MNT $SCRATCH_DEV` + +# fsx usage: +# +# -N numops: total # operations to do +# -l flen: the upper bound on file size +# -o oplen: the upper bound on operation size (64k default) +# -Z: O_DIRECT () + +_run_fsx_on_file $testfile -N 10000 -o $awu_max -A -l 500000 -r $bsize -w = $bsize -Z $FSX_AVOID >> $seqres.full +if [[ "$?" !=3D "0" ]] +then + _fail "fsx returned error: $?" +fi + +echo "Silence is golden" +status=3D0 +exit diff --git a/tests/generic/1229.out b/tests/generic/1229.out new file mode 100644 index 00000000..737d61c6 --- /dev/null +++ b/tests/generic/1229.out @@ -0,0 +1,2 @@ +QA output created by 1229 +Silence is golden --=20 2.49.0 From nobody Tue Oct 7 06:50:02 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23C1B289E04; Sat, 12 Jul 2025 14:13:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329615; cv=none; b=qEnNpAA60wQTQzZSwRSL+B9bLCSnA59Xkxh3MffcNdJv/KvxpflGMLh+AcFwc3Xf415kGMIM0XJoAghVdPJJ61fYZIkz7TZLjZTdsZpshYwAsj3wnGNFcFSFefE3M7ZqOlAZXGQXMyVD0RK6sTjGKvBHwCZu95CeTNPXGhNDVkA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329615; c=relaxed/simple; bh=oG66v5+saskaYQ2no9zivWGFaMVszm5sN7jm7hp2IRk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kki/v+vyuXiiDUIMFlTNRQbY1qIfr+UIB6dh6JLPYjHJ5jfJg2pbl+x9/kHzkqh0cn6tI7sNpFdSe8JIPPNGa05tl0UxIy1cXp5Yuy4waxASEvbDjz53lErjIwapsqVl+00vlMk4ZW9WtTnObG0IL3kCuX3i66HBkCrNi1Y9QM0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=X5y0GgtO; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="X5y0GgtO" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 56CDmMhh028355; Sat, 12 Jul 2025 14:13:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=cGleFsAB3xzsmGy49 OyV6tiTUbRlnuyCR1fzCNLJUws=; b=X5y0GgtOZwZhyrWnph1B3h3Y7lUdR8lXF 8L/HhJd5/vL8sybrEY1cJO+wycHzsfOGc7f9LLjSqvN6xXTIixBzFgIiLeyqVaSq 5+3XVdDrOb629hIkPcGzD5w8DBV7VkTGkpp/DEUVEH3umJKqVv7O7Wecc+6vWUMs Yc/XinaNURx8WRA2FU2HPMU+FtmAZogRt0Bq1yL/sm6wbpmxl4GqufJevfFktTAY +oBcZQp2xdHEil9RN1Igcbs4UsbgI+Fotdaesap/whzfHhEG1VegpJq46jC9M1cW /c1ttk3maQE6jRCVZQCud2lChMSPQEOPKHoG0ayb/1oiGHW65SF8A== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ufc6hpfs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:26 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 56CEDP0c003211; Sat, 12 Jul 2025 14:13:25 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ufc6hpfj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:25 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 56C93TIe010807; Sat, 12 Jul 2025 14:13:25 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 47qes0qk71-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:24 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 56CEDNH852101450 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 12 Jul 2025 14:13:23 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E5D8F2004B; Sat, 12 Jul 2025 14:13:22 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9983520040; Sat, 12 Jul 2025 14:13:20 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.252]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 12 Jul 2025 14:13:20 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v3 09/13] generic/1230: Add sudden shutdown tests for multi block atomic writes Date: Sat, 12 Jul 2025 19:42:51 +0530 Message-ID: <4965ba3e281a81558f1fe06fe1c478d29adca1e5.1752329098.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=Je68rVKV c=1 sm=1 tr=0 ts=68726d86 cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=Wb1JkmetP80A:10 a=pGLkceISAAAA:8 a=VnNF1IyMAAAA:8 a=jzLDx6uKbS-OZFe_CzQA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNzEyMDEwNyBTYWx0ZWRfX5nktr7wHyXo4 BHajc5waTsszHa2QeA5uH/cF0BEEYPBk8rD8lf+m1zVeSQ9+PSHbfM09qD13gtQG8iYzLKgQfMO pJjMMkQUQx0Em4DlsnjV8/VZXQzgWF7hoKJrXQc7IK/GveGWVRB4TuRorCtRTTwHIYtbrz99PDq xo+EVF0YfWVGe2Fdtz4JPYuvv2OzCEy1sYUAJ9Xt4KPSlMNzQexfwoNf/ABvjTJrAnZZ401d2i5 rFqCZqupg4b9y3d+yWmMXTB3kdi1eDxw4p+CB8I+XzIGyoRJpEc4yfIQUnBDpenvr9Cx5JtpWvo tkbhWB4bhZI2DaKaD8Y8JFQ/1yNIb6nF4JVY7n44R/g/baMSX3wgA+n6SmWlusZFz59UaWDClwW 0jWwtXdJFWAS1vJGChUc31TxuZR9OoQINi24WlbRLY6DYlnFYeLMdHnel1ZzSh/snI4rbcLd X-Proofpoint-GUID: Lrj3qEx_mM_gwb7EF1Jx4DcEo_thGqka X-Proofpoint-ORIG-GUID: Kh5BdLU6o5UIQjrD-g3HcHFkXDlCyYCU X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-07-12_02,2025-07-09_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 malwarescore=0 lowpriorityscore=0 suspectscore=0 adultscore=0 priorityscore=1501 clxscore=1015 bulkscore=0 phishscore=0 spamscore=0 mlxlogscore=999 mlxscore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2505280000 definitions=main-2507120107 Content-Type: text/plain; charset="utf-8" This test is intended to ensure that multi blocks atomic writes maintain atomic guarantees across sudden FS shutdowns. The way we work is that we lay out a file with random mix of written, unwritten and hole extents. Then we start performing atomic writes sequentially on the file while we parallely shutdown the FS. Then we note the last offset where the atomic write happened just before shut down and then make sure blocks around it either have completely old data or completely new data, ie the write was not torn during shutdown. We repeat the same with completely written, completely unwritten and comple= tely empty file to ensure these cases are not torn either. Finally, we have a similar test for append atomic writes Suggested-by: Ritesh Harjani (IBM) Signed-off-by: Ojaswin Mujoo Reviewed-by: "Darrick J. Wong" --- tests/generic/1230 | 397 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1230.out | 2 + 2 files changed, 399 insertions(+) create mode 100755 tests/generic/1230 create mode 100644 tests/generic/1230.out diff --git a/tests/generic/1230 b/tests/generic/1230 new file mode 100755 index 00000000..cff5adc0 --- /dev/null +++ b/tests/generic/1230 @@ -0,0 +1,397 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test No. 1230 +# +# Test multi block atomic writes with sudden FS shutdowns to ensure +# the FS is not tearing the write operation +. ./common/preamble +. ./common/atomicwrites +_begin_fstest auto atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_atomic_write_test_commands +_require_scratch_shutdown +_require_xfs_io_command "truncate" + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount >> $seqres.full + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile + +awu_max=3D$(_get_atomic_write_unit_max $testfile) +blksz=3D$(_get_block_size $SCRATCH_MNT) +echo "Awu max: $awu_max" >> $seqres.full + +num_blocks=3D$((awu_max / blksz)) +# keep initial value high for dry run. This will be +# tweaked in dry_run() based on device write speed. +filesize=3D$(( 10 * 1024 * 1024 * 1024 )) + +_cleanup() { + [ -n "$awloop_pid" ] && kill $awloop_pid &> /dev/null + wait +} + +atomic_write_loop() { + local off=3D0 + local size=3D$awu_max + for ((i=3D0; i<$((filesize / $size )); i++)); do + # Due to sudden shutdown this can produce errors so just + # redirect them to seqres.full + $XFS_IO_PROG -c "open -fsd $testfile" -c "pwrite -S 0x61 -DA -V1 -b $siz= e $off $size" >> /dev/null 2>>$seqres.full + echo "Written to offset: $off" >> $tmp.aw + off=3D$((off + $size)) + done +} + +# This test has the following flow: +# 1. Start doing sequential atomic writes in bg, upto $filesize +# 2. Sleep for 0.2s and shutdown the FS +# 3. kill the atomic write process +# 4. verify the writes were not torn +# +# We ideally want the shutdown to happen while an atomic write is ongoing +# but this gets tricky since faster devices can actually finish the whole +# atomic write loop before sleep 0.2s completes, resulting in the shutdown +# happening after the write loop which is not what we want. A simple solut= ion +# to this is to increase $filesize so step 1 takes long enough but a big +# $filesize leads to create_mixed_mappings() taking very long, which is not +# ideal. +# +# Hence, use the dry_run function to figure out the rough device speed and= set +# $filesize accordingly. +dry_run() { + echo >> $seqres.full + echo "# Estimating ideal filesize..." >> $seqres.full + atomic_write_loop & + awloop_pid=3D$! + + local i=3D0 + # Wait for atleast first write to be recorded or 10s + while [ ! -f "$tmp.aw" -a $i -le 50 ]; do i=3D$((i + 1)); sleep 0.2; done + + if [[ $i -gt 50 ]] + then + _fail "atomic write process took too long to start" + fi + + echo >> $seqres.full + echo "# Shutting down filesystem while write is running" >> $seqres.full + _scratch_shutdown + + kill $awloop_pid 2>/dev/null # the process might have finished already + wait $awloop_pid + unset $awloop_pid + + bytes_written=3D$(tail -n 1 $tmp.aw | cut -d" " -f4) + echo "# Bytes written in 0.2s: $bytes_written" >> $seqres.full + + filesize=3D$((bytes_written * 3)) + echo "# Setting \$filesize=3D$filesize" >> $seqres.full + + rm $tmp.aw + sleep 0.5 + + _scratch_cycle_mount + +} + +create_mixed_mappings() { + local file=3D$1 + local size_bytes=3D$2 + + echo "# Filling file $file with alternate mappings till size $size_bytes"= >> $seqres.full + #Fill the file with alternate written and unwritten blocks + local off=3D0 + local operations=3D("W" "U") + + for ((i=3D0; i<$((size_bytes / blksz )); i++)); do + index=3D$(($i % ${#operations[@]})) + map=3D"${operations[$index]}" + + case "$map" in + "W") + $XFS_IO_PROG -fc "pwrite -b $blksz $off $blksz" $file >> /dev/null + ;; + "U") + $XFS_IO_PROG -fc "falloc $off $blksz" $file >> /dev/null + ;; + esac + off=3D$((off + blksz)) + done + + sync $file +} + +populate_expected_data() { + # create a dummy file with expected old data for different cases + create_mixed_mappings $testfile.exp_old_mixed $awu_max + expected_data_old_mixed=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_o= ld_mixed) + + $XFS_IO_PROG -fc "falloc 0 $awu_max" $testfile.exp_old_zeroes >> $seqres.= full + expected_data_old_zeroes=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_= old_zeroes) + + $XFS_IO_PROG -fc "pwrite -b $awu_max 0 $awu_max" $testfile.exp_old_mapped= >> $seqres.full + expected_data_old_mapped=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_= old_mapped) + + # create a dummy file with expected new data + $XFS_IO_PROG -fc "pwrite -S 0x61 -b $awu_max 0 $awu_max" $testfile.exp_ne= w >> $seqres.full + expected_data_new=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_new) +} + +verify_data_blocks() { + local verify_start=3D$1 + local verify_end=3D$2 + local expected_data_old=3D"$3" + local expected_data_new=3D"$4" + + echo >> $seqres.full + echo "# Checking data integrity from $verify_start to $verify_end" >> $se= qres.full + + # After an atomic write, for every chunk we ensure that the underlying + # data is either the old data or new data as writes shouldn't get torn. + local off=3D$verify_start + while [[ "$off" -lt "$verify_end" ]] + do + #actual_data=3D$(xxd -s $off -l $awu_max -p $testfile) + actual_data=3D$(od -An -t x1 -j $off -N $awu_max $testfile) + if [[ "$actual_data" !=3D "$expected_data_new" ]] && [[ "$actual_data" != =3D "$expected_data_old" ]] + then + echo "Checksum match failed at off: $off size: $awu_max" + echo "Expected contents: (Either of the 2 below):" + echo + echo "Expected old: " + echo "$expected_data_old" + echo + echo "Expected new: " + echo "$expected_data_new" + echo + echo "Actual contents: " + echo "$actual_data" + + _fail + fi + echo -n "Check at offset $off suceeded! " >> $seqres.full + if [[ "$actual_data" =3D=3D "$expected_data_new" ]] + then + echo "matched new" >> $seqres.full + elif [[ "$actual_data" =3D=3D "$expected_data_old" ]] + then + echo "matched old" >> $seqres.full + fi + off=3D$(( off + awu_max )) + done +} + +# test data integrity for file by shutting down in between atomic writes +test_data_integrity() { + echo >> $seqres.full + echo "# Writing atomically to file in background" >> $seqres.full + atomic_write_loop & + awloop_pid=3D$! + + local i=3D0 + # Wait for atleast first write to be recorded or 10s + while [ ! -f "$tmp.aw" -a $i -le 50 ]; do i=3D$((i + 1)); sleep 0.2; done + + if [[ $i -gt 50 ]] + then + _fail "atomic write process took too long to start" + fi + + echo >> $seqres.full + echo "# Shutting down filesystem while write is running" >> $seqres.full + _scratch_shutdown + + kill $awloop_pid 2>/dev/null # the process might have finished already + wait $awloop_pid + unset $awloop_pid + + last_offset=3D$(tail -n 1 $tmp.aw | cut -d" " -f4) + if [[ -z $last_offset ]] + then + last_offset=3D0 + fi + + echo >> $seqres.full + echo "# Last offset of atomic write: $last_offset" >> $seqres.full + + rm $tmp.aw + sleep 0.5 + + _scratch_cycle_mount + + # we want to verify all blocks around which the shutdown happended + verify_start=3D$(( last_offset - (awu_max * 5))) + if [[ $verify_start < 0 ]] + then + verify_start=3D0 + fi + + verify_end=3D$(( last_offset + (awu_max * 5))) + if [[ "$verify_end" -gt "$filesize" ]] + then + verify_end=3D$filesize + fi +} + +# test data integrity for file wiht written and unwritten mappings +test_data_integrity_mixed() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with mixed mappings" >> $seqres.full + create_mixed_mappings $testfile $filesize + + test_data_integrity + + verify_data_blocks $verify_start $verify_end "$expected_data_old_mixed" "= $expected_data_new" +} + +# test data integrity for file with completely written mappings +test_data_integrity_writ() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with fully written mapping" >> $seqres.full + $XFS_IO_PROG -c "pwrite -b $filesize 0 $filesize" $testfile >> $seqres.fu= ll + sync $testfile + + test_data_integrity + + verify_data_blocks $verify_start $verify_end "$expected_data_old_mapped" = "$expected_data_new" +} + +# test data integrity for file with completely unwritten mappings +test_data_integrity_unwrit() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with fully unwritten mappings" >> $seqres.full + $XFS_IO_PROG -c "falloc 0 $filesize" $testfile >> $seqres.full + sync $testfile + + test_data_integrity + + verify_data_blocks $verify_start $verify_end "$expected_data_old_zeroes" = "$expected_data_new" +} + +# test data integrity for file with no mappings +test_data_integrity_hole() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with no mappings" >> $seqres.full + $XFS_IO_PROG -c "truncate $filesize" $testfile >> $seqres.full + sync $testfile + + test_data_integrity + + verify_data_blocks $verify_start $verify_end "$expected_data_old_zeroes" = "$expected_data_new" +} + +test_filesize_integrity() { + $XFS_IO_PROG -c "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Performing extending atomic writes over file in background" >> $s= eqres.full + atomic_write_loop & + awloop_pid=3D$! + + local i=3D0 + # Wait for atleast first write to be recorded or 10s + while [ ! -f "$tmp.aw" -a $i -le 50 ]; do i=3D$((i + 1)); sleep 0.2; done + + if [[ $i -gt 50 ]] + then + _fail "atomic write process took too long to start" + fi + + echo >> $seqres.full + echo "# Shutting down filesystem while write is running" >> $seqres.full + _scratch_shutdown + + kill $awloop_pid 2>/dev/null # the process might have finished already + wait $awloop_pid + unset $awloop_pid + + local last_offset=3D$(tail -n 1 $tmp.aw | cut -d" " -f4) + if [[ -z $last_offset ]] + then + last_offset=3D0 + fi + + echo >> $seqres.full + echo "# Last offset of atomic write: $last_offset" >> $seqres.full + rm $tmp.aw + sleep 0.5 + + _scratch_cycle_mount + local filesize=3D$(_get_filesize $testfile) + echo >> $seqres.full + echo "# Filesize after shutdown: $filesize" >> $seqres.full + + # To confirm that the write went atomically, we check: + # 1. The last block should be a multiple of awu_max + # 2. The last block should be the completely new data + + if (( $filesize % $awu_max )) + then + echo "Filesize after shutdown ($filesize) not a multiple of atomic write= unit ($awu_max)" + fi + + verify_start=3D$(( filesize - (awu_max * 5))) + if [[ $verify_start < 0 ]] + then + verify_start=3D0 + fi + + local verify_end=3D$filesize + + # Here the blocks should always match new data hence, for simplicity of + # code, just corrupt the $expected_data_old buffer so it never matches + local expected_data_old=3D"POISON" + verify_data_blocks $verify_start $verify_end "$expected_data_old" "$expec= ted_data_new" +} + +$XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + +dry_run + +echo >> $seqres.full +echo "# Populating expected data buffers" >> $seqres.full +populate_expected_data + +# Loop 20 times to shake out any races due to shutdown +for ((iter=3D0; iter<20; iter++)) +do + echo >> $seqres.full + echo "------ Iteration $iter ------" >> $seqres.full + + echo >> $seqres.full + echo "# Starting data integrity test for atomic writes over mixed mapping= " >> $seqres.full + test_data_integrity_mixed + + echo >> $seqres.full + echo "# Starting data integrity test for atomic writes over fully written= mapping" >> $seqres.full + test_data_integrity_writ + + echo >> $seqres.full + echo "# Starting data integrity test for atomic writes over fully unwritt= en mapping" >> $seqres.full + test_data_integrity_unwrit + + echo >> $seqres.full + echo "# Starting data integrity test for atomic writes over holes" >> $se= qres.full + test_data_integrity_hole + + echo >> $seqres.full + echo "# Starting filesize integrity test for atomic writes" >> $seqres.fu= ll + test_filesize_integrity +done + +echo "Silence is golden" +status=3D0 +exit diff --git a/tests/generic/1230.out b/tests/generic/1230.out new file mode 100644 index 00000000..d01f54ea --- /dev/null +++ b/tests/generic/1230.out @@ -0,0 +1,2 @@ +QA output created by 1230 +Silence is golden --=20 2.49.0 From nobody Tue Oct 7 06:50:02 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 80D3827EFE4; Sat, 12 Jul 2025 14:14:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329646; cv=none; b=A9mCKbQC9Y/dSazxwaAxG7R8M7hMy2ii+fU2L+913c3h3/kt39IHjKoj1/0b9PdBQl59GWoaWWMMg5KqBL1MSaLHsMYHup0rZH2ylinVwzXWnWrYZ5GhfiXWdksQ4IoV1v4Bvb1XSls18NFmAgJuOVBOLNUfFbGgF5Jt0CrFpRY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329646; c=relaxed/simple; bh=I3iYrD5WwI1ocmafgqtxBAhBJc+MnL6IfmTgqCCGZnE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AxvD+EXiGrI/j/lOIJfz7Wxu3xad0YcSD9i069bhMV1krUYkezdKnJ0TFoz/ki1fDNQizq0gazdct8KYPLJ3t+1t+D7SKiAlIOkspEHKtQZg3mhvPivlIbudBdFjDFK5YB5LkVFn+TR4Q5qs0k+xyB5/F15OWcvvjPg/RX1QTBc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=fjKPOK+T; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="fjKPOK+T" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 56C8kboV026979; Sat, 12 Jul 2025 14:13:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=Sb47yk01e7J3qOOmE /cujCmZTIsqm4s0zL7KCUqoS5w=; b=fjKPOK+T62PHfP50K1kzQo7GNnqoJEslF Z7Lc5o0AMNxy/1F2+vLRBcJS+xwz2xkY6Ug7A+YZoKXTFWFzZ16PMl9Fj84vRU2k 0CuGuzyERxEZnh1/L0gnwkOm4eLnQOP3aAmkLAN+cEWdmpRRmLAFTdQJkyAP97oO q+hJ4rriWmPMGHklWYHdGV1yYdNH5j4Un6cjS2zuVbWZ22OHSoURZ8dZudrxO9uM voThorOzsr4wLlSaYVo5zg/YmOUE4Rb7o8sHD95rOaItNPEIOFUGHV/4bqAxj7RD 3HYtcLPJ/hG9Yhz8YM8Sl6Q74gCiP7Az5CyXOJ5OUxfCP8/NXGZZA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ud4st30t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:29 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 56CEDSYZ004827; Sat, 12 Jul 2025 14:13:28 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ud4st30j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:28 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 56C9tMR3010790; Sat, 12 Jul 2025 14:13:27 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 47qes0qk7c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:27 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 56CEDPPx30343694 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 12 Jul 2025 14:13:25 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AAF9B20043; Sat, 12 Jul 2025 14:13:25 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 55A3620040; Sat, 12 Jul 2025 14:13:23 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.252]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 12 Jul 2025 14:13:23 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v3 10/13] ext4/061: Atomic writes stress test for bigalloc using fio crc verifier Date: Sat, 12 Jul 2025 19:42:52 +0530 Message-ID: <9b59eb50b171dece1a15bc7c1b6cadff438586d6.1752329098.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNzEyMDEwNCBTYWx0ZWRfX5KGzAWTU3qBu N/JlvOAAzw1myelIEJyeLgEOJMjCCIkJgO73d93zY9MldqWJj9RT3l65VoSn9NW6wK3AEUhMeOu nrnK14Ip+EbdAS+OYqVKUOLjb80mloZHvqjuTd/v7jPS8zUNDPEkKHb4t2vRIN7RosfFIL+j/WT nPLGYKyagwPhZYAHkYdY5ECxkbVZADZxk9smyWKNRODzFbFNFGesF7KwkFhYl5dMYKIyklVGEAR S3Vn3dgx0nNmruBwn/Dm87QkwVaZijDzor3HHILpetsjPa2jTiONmnrVffehnzcKrWLa08ma6wQ rASiQnpBFJCS7BD5C4cDv6w3XGT1klLe09GgprgLbITNGewV9WmUVLfMfnSAAGfTyUTTjfG4ohk 8rBoGaxObkvSAGpoYMlgakmCG4m03Va0HJz8gQDtn4ja6ueabAbg84BXN8oi8U3LrVXUXZYz X-Proofpoint-GUID: bgzu00rkDSaEKEAAK2_bFebARTH_GFyb X-Proofpoint-ORIG-GUID: Eq5_KCTouqDEGWF_fTlL_SK0fRYtfMj1 X-Authority-Analysis: v=2.4 cv=KezSsRYD c=1 sm=1 tr=0 ts=68726d89 cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=Wb1JkmetP80A:10 a=pGLkceISAAAA:8 a=VnNF1IyMAAAA:8 a=jHICElQ_nnKT0EPUS5YA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-07-12_02,2025-07-09_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 adultscore=0 clxscore=1015 suspectscore=0 phishscore=0 mlxlogscore=822 mlxscore=0 bulkscore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 malwarescore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2505280000 definitions=main-2507120104 Content-Type: text/plain; charset="utf-8" From: "Ritesh Harjani (IBM)" We brute force all possible blocksize & clustersize combinations on a bigalloc filesystem for stressing atomic write using fio data crc verifier. We run nproc * $LOAD_FACTOR threads in parallel writing to a single $SCRATCH_MNT/test-file. With atomic writes this test ensures that we never see the mix of data contents from different threads on a given bsrange. Signed-off-by: Ritesh Harjani (IBM) Signed-off-by: Ojaswin Mujoo --- tests/ext4/061 | 130 +++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/061.out | 2 + 2 files changed, 132 insertions(+) create mode 100755 tests/ext4/061 create mode 100644 tests/ext4/061.out diff --git a/tests/ext4/061 b/tests/ext4/061 new file mode 100755 index 00000000..a0e49249 --- /dev/null +++ b/tests/ext4/061 @@ -0,0 +1,130 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 061 +# +# Brute force all possible blocksize clustersize combination on a bigalloc +# filesystem for stressing atomic write using fio data crc verifier. We run +# nproc * 2 * $LOAD_FACTOR threads in parallel writing to a single +# $SCRATCH_MNT/test-file. With fio aio-dio atomic write this test ensures = that +# we should never see the mix of data contents from different threads for = any +# given fio blocksize. +# + +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto rw stress atomicwrites + +_require_scratch_write_atomic +_require_aiodio + +FIO_LOAD=3D$(($(nproc) * 2 * LOAD_FACTOR)) +SIZE=3D$((100*1024*1024)) +fiobsize=3D4096 + +# Calculate fsblocksize as per bdev atomic write units. +bdev_awu_min=3D$(_get_atomic_write_unit_min $SCRATCH_DEV) +bdev_awu_max=3D$(_get_atomic_write_unit_max $SCRATCH_DEV) +fsblocksize=3D$(_max 4096 "$bdev_awu_min") + +function create_fio_configs() +{ + create_fio_aw_config + create_fio_verify_config +} + +function create_fio_verify_config() +{ +cat >$fio_verify_config <$fio_aw_config <> $seqres.full 2>&1 || continue + if _try_scratch_mount >> $seqres.full 2>&1; then + echo "=3D=3D FIO test for fsblocksize=3D$fsblocksize fsclustersize=3D$= fsclustersize fiobsize=3D$fiobsize =3D=3D" >> $seqres.full + + touch $SCRATCH_MNT/f1 + create_fio_configs + + cat $fio_aw_config >> $seqres.full + echo >> $seqres.full + cat $fio_verify_config >> $seqres.full + + $FIO_PROG $fio_aw_config >> $seqres.full + ret1=3D$? + + $FIO_PROG $fio_verify_config >> $seqres.full + ret2=3D$? + + _scratch_unmount + + [[ $ret1 -eq 0 && $ret2 -eq 0 ]] || _fail "fio with atomic write faile= d" + fi + done + done +done + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/ext4/061.out b/tests/ext4/061.out new file mode 100644 index 00000000..273be9e0 --- /dev/null +++ b/tests/ext4/061.out @@ -0,0 +1,2 @@ +QA output created by 061 +Silence is golden --=20 2.49.0 From nobody Tue Oct 7 06:50:02 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F81827E047; Sat, 12 Jul 2025 14:14:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329647; cv=none; b=sL0okDnQpb4K/cIa2ku0nbE5+jfO/KNBPFGBUSspwQcuKXw3CnKzjy/1aZzbq45J51cNndt3Y4fmMJmSKBoQlPbGDMbkL+Al1MKvWP8hYUAtVuxubq34Zo1gP8+v5LsnUZQ+BnNTyaPm8YOMrFYAwLaNG3qRpkpUHcrmZedCmg8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329647; c=relaxed/simple; bh=KNrAQHgMsXhyOwPfV9u/T0Jg4bGopTaJ35cfz1MUYuc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SYd/SdhIMWSG8jE5iOyshW1crI/4xH5cDtnE4+ajRFF/I3hi2UxNL0K8E/zbgvnDCfViGu7NJG5XA7juoB/QPx6X6lVTL0Ar1GMK7xwYwrgj8Z/nlN86IOMOP7EXWbt14BdJ/HzE8xhgz/n2RImmfnzhDXhFnHLnXMSzTTRVJuQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=BfjF9u7W; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="BfjF9u7W" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 56CCwcta031931; Sat, 12 Jul 2025 14:13:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=/HrkyBHl/xr+a469M 1SoZyj2BKanRPoZvRyiqaCwunU=; b=BfjF9u7WsH8OV+iqk8CBJmh/qICPDneKF HQMk9Vm1LGvYCrc3gfBM9XP1sFdEqe6IS/Z8N3zgzYC5j8QQYE5KIPuorQE0w9lo JU5drRolIs0fTldDDWnczweuR1ut+JopmfxUjo2N6HQ5/j13sOqWt5q2/k2EG9bt 4pqBHOpx88qWM19o2UdZsNiRWUVwT+cYkMb2fGtcqmwMSQtNmM746xo54LR/tX6p EUKmv4XrEeoTSgjPxJjoemBTagrBniLZ0DpBiqQrBtK/kyyYJCf1EfhLj8NKLHeV d6t+C+mrg8QChHF1QL75/1j6p2ReXiTDj0JYxbYyL9Uq6wtM76xiA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ufeehqvb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:31 +0000 (GMT) Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 56CEDUuM031405; Sat, 12 Jul 2025 14:13:31 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ufeehqv2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:30 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 56CBdNON013616; Sat, 12 Jul 2025 14:13:30 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 47qgkmf65d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:29 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 56CEDS2n18481612 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 12 Jul 2025 14:13:28 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5838020043; Sat, 12 Jul 2025 14:13:28 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 19CEA20040; Sat, 12 Jul 2025 14:13:26 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.252]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 12 Jul 2025 14:13:25 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v3 11/13] ext4/062: Atomic writes test for bigalloc using fio crc verifier on multiple files Date: Sat, 12 Jul 2025 19:42:53 +0530 Message-ID: <0116498a542ba9ea9026b2b61ad747eca31f931d.1752329098.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNzEyMDEwNyBTYWx0ZWRfX3VBEYmIfO40y 4J8D/pu03m/sfmH209oeigTuMnbNP3TnXAGND+LG9lXxC1i6nuf3gyLWRFVJ7TyZds1Xlcb0rwP LmKgx/JXJHkKblGMPquNOJqDLVimlBKkSL/QUadmtSz/NXQ+2cs0ld7Bhp82yis2Tm/RIsDJJLA DZ+SDtEGTnFAVlHB0EJe/iZ2pwnmHQulPzbbdDLgzYUqA1kN0aa1nMVyHK/mdLrq3anFwc3c4Zp fbXt7oTyiTj5BmK2NKJgHKO61NYGmt6xv9pgMWvxe1l3lEydzp7YvlYoyT6GlEcn9hEP27iq1MH qDD4CybmVxy5mHGG8FhjT1JnPNzjpQr01fbW+6SUL4RCMop3B6oQhxj9mwVqL++nacgCOkBsOBa doqhoug5KrssOuLol5NJ4phoQWUMmOHM9+VW9OSTuixzCsg92N87MljQAriqsmvi3jc5FIw0 X-Proofpoint-ORIG-GUID: Bn_q3_yU56Y9fED776peuyFyy11fk7qw X-Authority-Analysis: v=2.4 cv=C9/pyRP+ c=1 sm=1 tr=0 ts=68726d8b cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=Wb1JkmetP80A:10 a=pGLkceISAAAA:8 a=VnNF1IyMAAAA:8 a=yVCtIWGlblad6CDWSz4A:9 X-Proofpoint-GUID: qKwxC4nF61vVidSZYV87Wfu-Xi13qJl2 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-07-12_02,2025-07-09_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 impostorscore=0 spamscore=0 suspectscore=0 priorityscore=1501 lowpriorityscore=0 mlxlogscore=991 bulkscore=0 adultscore=0 clxscore=1015 mlxscore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2505280000 definitions=main-2507120107 Content-Type: text/plain; charset="utf-8" From: "Ritesh Harjani (IBM)" Brute force all possible blocksize clustersize combination on a bigalloc filesystem for stressing atomic write using fio data crc verifier. We run multiple threads in parallel with each job writing to its own file. The parallel jobs running on a constrained filesystem size ensure that we stress the ext4 allocator to allocate contiguous extents. Signed-off-by: Ritesh Harjani (IBM) Signed-off-by: Ojaswin Mujoo Reviewed-by: "Darrick J. Wong" --- tests/ext4/062 | 176 +++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/062.out | 2 + 2 files changed, 178 insertions(+) create mode 100755 tests/ext4/062 create mode 100644 tests/ext4/062.out diff --git a/tests/ext4/062 b/tests/ext4/062 new file mode 100755 index 00000000..85b82f97 --- /dev/null +++ b/tests/ext4/062 @@ -0,0 +1,176 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 061 +# +# Brute force all possible blocksize clustersize combination on a bigalloc +# filesystem for stressing atomic write using fio data crc verifier. We run +# nproc * $LOAD_FACTOR threads in parallel writing to a single +# $SCRATCH_MNT/test-file. We also create 8 such parallel jobs to run on +# a constrained filesystem size to stress the ext4 allocator to allocate +# contiguous extents. +# + +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto rw stress atomicwrites + +_require_scratch_write_atomic +_require_aiodio + +FSSIZE=3D$((360*1024*1024)) +FIO_LOAD=3D$(($(nproc) * LOAD_FACTOR)) +fiobsize=3D4096 + +# Calculate fsblocksize as per bdev atomic write units. +bdev_awu_min=3D$(_get_atomic_write_unit_min $SCRATCH_DEV) +bdev_awu_max=3D$(_get_atomic_write_unit_max $SCRATCH_DEV) +fsblocksize=3D$(_max 4096 "$bdev_awu_min") + +function create_fio_configs() +{ + create_fio_aw_config + create_fio_verify_config +} + +function create_fio_verify_config() +{ +cat >$fio_verify_config <$fio_aw_config <> $seqres.full 2>&1 || continue + if _try_scratch_mount >> $seqres.full 2>&1; then + echo "=3D=3D FIO test for fsblocksize=3D$fsblocksize fsclustersize=3D$= fsclustersize fiobsize=3D$fiobsize =3D=3D" >> $seqres.full + + touch $SCRATCH_MNT/f1 + create_fio_configs + + cat $fio_aw_config >> $seqres.full + cat $fio_verify_config >> $seqres.full + + $FIO_PROG $fio_aw_config >> $seqres.full + ret1=3D$? + + $FIO_PROG $fio_verify_config >> $seqres.full + ret2=3D$? + + _scratch_unmount + + [[ $ret1 -eq 0 && $ret2 -eq 0 ]] || _fail "fio with atomic write faile= d" + fi + done + done +done + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/ext4/062.out b/tests/ext4/062.out new file mode 100644 index 00000000..a1578f48 --- /dev/null +++ b/tests/ext4/062.out @@ -0,0 +1,2 @@ +QA output created by 062 +Silence is golden --=20 2.49.0 From nobody Tue Oct 7 06:50:02 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA1D317A2EC; Sat, 12 Jul 2025 14:14:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329661; cv=none; b=SssqUTmGkYAN3MrkYeYQF4PaAwE67dYqV5l19CVaMUetIn1xInu02kp/3Yur+HqQHd1uHEVoT7JQCAkYSeL9efsxeNKjSXh2AOFTjZZ6lgwJwBbr6Zsi/8gk2rAjWLRA3ZV4ZjcVr0W2oHdAWTihgWqmxcBclWuoLz8ZbKQKxQs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329661; c=relaxed/simple; bh=WqXXipFPQ1tVUxnA3odb4wEvTzAaPB/Apy2RQvWokIY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=c9DUZ3q6v4D+49tfVVH335LmBCydmL34MXtQpHD9dpaDYaSv3/B1S8SCcJ4Ky54fOsztINBHsQqeL3+B6M34U1RrGTIUwTl4WcCJxwYCzR3BcLiSsT9COGD5Jbv95iZV4tsisrmfkewObrqljRkEbqcYn+bo2XkJG5ON0pEOEaw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Xk+aqn6E; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Xk+aqn6E" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 56CBkc90018194; Sat, 12 Jul 2025 14:13:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=ZRdoJ9TTuDQV9ijiB aHQiZbirmNkt1nR1KInfMDU9Ag=; b=Xk+aqn6El/7nP14bdhWjx0SFONupIwsTH kFZLfIs7ruzL3GWn6SEN6Cnrb1esduHvQGrYyyz4Mce/Ci1JWHxZktc7B9XSEoK4 x/sgvjhfrc20/t7/EQe65ALZz/tDtcEohpAruFA1SmXipDjOamynYbXivgdKGBgj hR2/p/pmvYx32G+J9/P/MOdra8HvCmkvO5k8ly3bezNWld3K8omz3YbodfSbLBrP kx2UA+GwM6TAN1ir++7EvayR+/Lilr9tyH+n+EJUsjEqdFbHC/pmhT6drlJ1etlE XXpi1JqTTy80s+ylw5rLvDs84NOgX1R4LS3FRRlvZcMFpJjYFOvxg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ue4thv6s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:34 +0000 (GMT) Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 56CEDKUU008828; Sat, 12 Jul 2025 14:13:33 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ue4thv6m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:33 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 56CB3wU9013577; Sat, 12 Jul 2025 14:13:32 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 47qgkmf667-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:32 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 56CEDVDK30409160 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 12 Jul 2025 14:13:31 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 06C4920043; Sat, 12 Jul 2025 14:13:31 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BB2AB20040; Sat, 12 Jul 2025 14:13:28 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.252]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 12 Jul 2025 14:13:28 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v3 12/13] ext4/063: Atomic write test for extent split across leaf nodes Date: Sat, 12 Jul 2025 19:42:54 +0530 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=baBrUPPB c=1 sm=1 tr=0 ts=68726d8e cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=Wb1JkmetP80A:10 a=VnNF1IyMAAAA:8 a=BtNrGSYHu5IGmLxdGZgA:9 X-Proofpoint-GUID: x5iyryN_uXFvgrbwGwYH5FGrdNTMxDoZ X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNzEyMDEwNyBTYWx0ZWRfX4jcauGXp+Juh dxhMZER8JAm8uzzPyO5QBtJb9bYsSXrrQLfV192tvvwz05Re5A9LtUAC55FSsUyv8aFcXSfNHB5 pb1uNHP3qD60BU/sMkoyNX+wiO6VO7jhSVVilcoU3STVhCxDL9vGCnvSfGmi5S8wmSRrqkgcNq+ kom7SM+/0DIEAfJK4grJNu3hEVHLJH8m1ywReseNGX92NCwtKhMhUSzIDwyz1XOE+vV6nGQmeSk lt4FsovYNzTI7g4k6RW01ujDHzJjS/ALsIduHKg3Cg4LCLBI5wVO7C/5eOWeq6pKw+c/5g6vNOp Hr0vkh3q3ggYBJWTQaXLNLnCvDr+JtV5TcPX4eEgq+QpUixYrbdC5ziEqqlWmQYj9hbRoh1zwFe J8iNBqKAg/qjfPeDkfJIzCBL8eya61zH7DTic2vo9o765HPftVvWPJXDWLUL8kZ3RlKGzIon X-Proofpoint-ORIG-GUID: q3o_HD03o1lzVRCUAtH9nIuWh66upTFR X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-07-12_02,2025-07-09_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 impostorscore=0 phishscore=0 lowpriorityscore=0 priorityscore=1501 clxscore=1015 mlxscore=0 malwarescore=0 spamscore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2505280000 definitions=main-2507120107 Content-Type: text/plain; charset="utf-8" In ext4, even if an allocated range is physically and logically contiguous, it can still be split into 2 extents. This is because ext4 does not merge extents across leaf nodes. This is an issue for atomic writes since even for a continuous extent the map block could (in rare cases) return a shorter map, hence tearning the write. This test creates such a file and ensures that the atomic write handles this case correctly Signed-off-by: Ojaswin Mujoo --- tests/ext4/063 | 125 +++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/063.out | 2 + 2 files changed, 127 insertions(+) create mode 100755 tests/ext4/063 create mode 100644 tests/ext4/063.out diff --git a/tests/ext4/063 b/tests/ext4/063 new file mode 100755 index 00000000..25b5693d --- /dev/null +++ b/tests/ext4/063 @@ -0,0 +1,125 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# In ext4, even if an allocated range is physically and logically contiguo= us, +# it can still be split into 2 extents. This is because ext4 does not merge +# extents across leaf nodes. This is an issue for atomic writes since even= for +# a continuous extent the map block could (in rare cases) return a shorter= map, +# hence tearning the write. This test creates such a file and ensures that= the +# atomic write handles this case correctly +# +. ./common/preamble +. ./common/atomicwrites +_begin_fstest auto atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_atomic_write_test_commands +_require_command "$DEBUGFS_PROG" debugfs + +prep() { + local bs=3D`_get_block_size $SCRATCH_MNT` + local ex_hdr_bytes=3D12 + local ex_entry_bytes=3D12 + local entries_per_blk=3D$(( (bs - ex_hdr_bytes) / ex_entry_bytes )) + + # fill the extent tree leaf which bs len extents at alternate offsets. Fo= r example, + # for 4k bs the tree should look as follows + # + # +---------+---------+ + # | index 1 | index 2 | + # +-----+---+-----+---+ + # +--------+ +-------+ + # | | + # +----------+--------------+ +-----+-----+ + # | ex 1 | ex 2 |... | ex n | | ex n + 1 | + # +-------------------------+ +-----------+ + # 0 2 680 682 + for i in $(seq 0 $entries_per_blk) + do + $XFS_IO_PROG -fc "pwrite -b $bs $((i * 2 * bs)) $bs" $testfile > /dev/nu= ll + done + sync $testfile + + echo >> $seqres.full + echo "Create file with extents spanning 2 leaves. Extents:">> $seqres.full + echo "...">> $seqres.full + $DEBUGFS_PROG -R "ex `basename $testfile`" $SCRATCH_DEV |& tail >> $seqre= s.full + + # Now try to insert a new extent ex(new) between ex(n) and ex(n+1). Since + # this is a new FS the allocator would find continuous blocks such that + # ex(n) ex(new) ex(n+1) are physically(and logically) contiguous. However, + # since we dont merge extents across leaf we will end up with a tree as: + # + # +---------+---------+ + # | index 1 | index 2 | + # +-----+---+-----+---+ + # +--------+ +-------+ + # | | + # +----------+--------------+ +-----+-----+ + # | ex 1 | ex 2 |... | ex n | | ex merged | + # +-------------------------+ +-----------+ + # 0 2 680 681 682 684 + # + echo >> $seqres.full + torn_ex_offset=3D$((((entries_per_blk * 2) - 1) * bs)) + $XFS_IO_PROG -c "pwrite $torn_ex_offset $bs" $testfile >> /dev/null + sync $testfile + + echo >> $seqres.full + echo "Perform 1 block write at $torn_ex_offset to create torn extent. Ext= ents:">> $seqres.full + echo "...">> $seqres.full + $DEBUGFS_PROG -R "ex `basename $testfile`" $SCRATCH_DEV |& tail >> $seqre= s.full + + _scratch_cycle_mount +} + +_scratch_mkfs >> $seqres.full +_scratch_mount >> $seqres.full + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile +awu_max=3D$(_get_atomic_write_unit_max $testfile) + +echo >> $seqres.full +echo "# Prepping the file" >> $seqres.full +prep + +torn_aw_offset=3D$((torn_ex_offset - (torn_ex_offset % awu_max))) + +echo >> $seqres.full +echo "# Performing atomic IO on the torn extent range. Command: " >> $seqr= es.full +echo $XFS_IO_PROG -c "open -fsd $testfile" -c "pwrite -S 0x61 -DA -V1 -b $= awu_max $torn_aw_offset $awu_max" >> $seqres.full +$XFS_IO_PROG -c "open -fsd $testfile" -c "pwrite -S 0x61 -DA -V1 -b $awu_m= ax $torn_aw_offset $awu_max" >> $seqres.full + +echo >> $seqres.full +echo "Extent state after atomic write:">> $seqres.full +echo "...">> $seqres.full +$DEBUGFS_PROG -R "ex `basename $testfile`" $SCRATCH_DEV |& tail >> $seqres= .full + +echo >> $seqres.full +echo "# Checking data integrity" >> $seqres.full + +# create a dummy file with expected data +$XFS_IO_PROG -fc "pwrite -S 0x61 -b $awu_max 0 $awu_max" $testfile.exp >> = /dev/null +expected_data=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp) + +# We ensure that the data after atomic writes should match the expected da= ta +actual_data=3D$(od -An -t x1 -j $torn_aw_offset -N $awu_max $testfile) +if [[ "$actual_data" !=3D "$expected_data" ]] +then + echo "Checksum match failed at off: $torn_aw_offset size: $awu_max" + echo + echo "Expected: " + echo "$expected_data" + echo + echo "Actual contents: " + echo "$actual_data" + + _fail +fi + +echo -n "Data verification at offset $torn_aw_offset suceeded!" >> $seqres= .full +echo "Silence is golden" +status=3D0 +exit diff --git a/tests/ext4/063.out b/tests/ext4/063.out new file mode 100644 index 00000000..de35fc52 --- /dev/null +++ b/tests/ext4/063.out @@ -0,0 +1,2 @@ +QA output created by 063 +Silence is golden --=20 2.49.0 From nobody Tue Oct 7 06:50:02 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 401B427E1C0; Sat, 12 Jul 2025 14:13:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329626; cv=none; b=ADg0j/huNvh1/OoaSbvNpgyYM5dtPinEL601clkVq0kYWOrRKVnUUYWeUXW0FiLNKeYcoEB3WCjH6qlpkTmXQ+WekLrEPq/aWaYcsuRVHs4dWOFTzg+We6Q2mKaEaIqRUsCZwfkbQcoR4OVik6TUVWMCe+BS08Ji5KbTjxBbzWg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752329626; c=relaxed/simple; bh=EWZ5P3hK76Z6OvtYAHftlDXYHxIiGpzvT+Filq2kvaI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YPiqwyijXX4kbCiXzLBfrp3YEUkaNiA9L5tgOpqpNkDHt1squZkdAhuv96LGMEvkOV+jy4yfmEh3ArUQXAl6v0KFqO5LiboWcZPGR2/ymESboS9qW0NSV8AOJ4DVtzvdNQXQtEb6L1w4YzhWv9LfOo2B7RpXQ7MD2b/pGc+drb4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=JYBcmFye; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="JYBcmFye" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 56C9wska026642; Sat, 12 Jul 2025 14:13:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=0Qp+LtOx7GmPwaZCz tYC+3HTAkC8Ob0pTvDIkziBgsk=; b=JYBcmFyeey5C5PVtBOB5r2LuuaXbkiYLT cELvrXh86tTU1VSltORKs09K8H6zxQOKfHS/cLG+4R+QZ+DNzzMM2gsrsoOma0E1 KzI7DTtdnWuNez4aggiYNJ2I45DKSLdEc+CF/jdavU1FjW17eZJsFoKR+o3zF7k0 4cVd12psBQDgyVcNN0mFBhdzPjMAP/GiW4Hl/xNfhhcEW0FZIgQwtRqV74NAV4+P tldp5ADgfxF7AZmBLKtB4zmIVT3Ax2aPjgjZEBM+f+w4+OMGU5M0leg/S7Cm/Fzq QLfgSi/xuPC+5VESPJiLoeVYmDWn0/Ghl6HzaM+l8zl88kp7J8dBg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ue4thv7q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:37 +0000 (GMT) Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 56CEDaGs009854; Sat, 12 Jul 2025 14:13:36 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 47ue4thv7b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:36 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 56CCZ2xm013555; Sat, 12 Jul 2025 14:13:35 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 47qgkmf66u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jul 2025 14:13:35 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 56CEDXoE34406746 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 12 Jul 2025 14:13:34 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D966E20043; Sat, 12 Jul 2025 14:13:33 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 70FEB20040; Sat, 12 Jul 2025 14:13:31 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.252]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 12 Jul 2025 14:13:31 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v3 13/13] ext4/064: Add atomic write tests for journal credit calculation Date: Sat, 12 Jul 2025 19:42:55 +0530 Message-ID: <77fb2f74dfce591aed65364984803904da9c1408.1752329098.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=baBrUPPB c=1 sm=1 tr=0 ts=68726d91 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=Wb1JkmetP80A:10 a=pGLkceISAAAA:8 a=VnNF1IyMAAAA:8 a=s1tx9Ns7CSuJm8OCEucA:9 X-Proofpoint-GUID: pgtEJvlABbJuumr1DfJgI2bo07qs3x1n X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNzEyMDEwNyBTYWx0ZWRfXxgm8ylP+urXk qAwz5ANPLBgnCJSpfvxccV+1H64nnIpck/5EzmhHqiQTDlxxq7qyllGj4x4XTlBIVYrWJTuus/i 52ZTDpza3m65DuB+egO2aFgXmaP8zdmTURCNlrKm45qClt1Y1cJXRns5INis+/7LAWY4SHQl+e4 8/p1MzBIvIXHjDy122aBSpRMvrMGBIbYVvNW+T/f6m1HkF6F2/ShCJa/5HQTE3Gg4ObuGTyyyer j+YU6gJw7pgdoajEGkNKJ3UOpIYb8IEOmKuVA/2GCn+y0y3kNQdXMG9NKqE9HfSgrJiU2pGFWPB +xC7Ax8LQEqtClIhkQwRQwcXWxyQCHI5L9fnRwy4bL9x2lL9mqGxp1VBg6gSxnOI8YyInHiLVEd +AloaB92263am6Ru4y8mvsv7DX1u9O2tvADNn6r7bfhL2tD9PHGG6gB/ndLvmtDrMtGLcguM X-Proofpoint-ORIG-GUID: 1XC8Tw3vG4vd7dXU-Fghyt3WBvMrZ3-P X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.7,FMLib:17.12.80.40 definitions=2025-07-12_02,2025-07-09_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 impostorscore=0 phishscore=0 lowpriorityscore=0 priorityscore=1501 clxscore=1015 mlxscore=0 malwarescore=0 spamscore=0 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2505280000 definitions=main-2507120107 Content-Type: text/plain; charset="utf-8" Test atomic writes with journal credit calculation. We take 2 cases here: 1. Atomic writes on single mapping causing tree to collapse into the inode 2. Atomic writes on mixed mapping causing tree to collapse into the inode This test is inspired by ext4/034. Suggested-by: Ritesh Harjani (IBM) Signed-off-by: Ojaswin Mujoo --- tests/ext4/064 | 75 ++++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/064.out | 2 ++ 2 files changed, 77 insertions(+) create mode 100755 tests/ext4/064 create mode 100644 tests/ext4/064.out diff --git a/tests/ext4/064 b/tests/ext4/064 new file mode 100755 index 00000000..ec31f983 --- /dev/null +++ b/tests/ext4/064 @@ -0,0 +1,75 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 034 +# +# Test proper credit reservation is done when performing +# tree collapse during an aotmic write based allocation +# +. ./common/preamble +. ./common/atomicwrites +_begin_fstest auto quick quota fiemap prealloc atomicwrites + +# Import common functions. + + +# Modify as appropriate. +_exclude_fs ext2 +_exclude_fs ext3 +_require_xfs_io_command "falloc" +_require_xfs_io_command "fiemap" +_require_xfs_io_command "syncfs" +_require_scratch_write_atomic_multi_fsblock +_require_atomic_write_test_commands + +echo "----- Testing with atomic write on non-mixed mapping -----" >> $seqr= es.full + +echo "Format and mount" >> $seqres.full +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount > $seqres.full 2>&1 + +echo "Create the original file" >> $seqres.full +touch $SCRATCH_MNT/foobar >> $seqres.full + +echo "Create 2 level extent tree (btree) for foobar with a unwritten exten= t" >> $seqres.full +$XFS_IO_PROG -f -c "pwrite 0 4k" -c "falloc 4k 4k" -c "pwrite 8k 4k" \ + -c "pwrite 20k 4k" -c "pwrite 28k 4k" -c "pwrite 36k 4k" \ + -c "fsync" $SCRATCH_MNT/foobar >> $seqres.full + +$XFS_IO_PROG -c "fiemap -v" $SCRATCH_MNT/foobar >> $seqres.full + +echo "Convert unwritten extent to written and collapse extent tree to inod= e" >> $seqres.full +$XFS_IO_PROG -dc "pwrite -A -V1 4k 4k" $SCRATCH_MNT/foobar >> $seqres.full + +echo "Create a new file and do fsync to force a jbd2 commit" >> $seqres.fu= ll +$XFS_IO_PROG -f -c "pwrite 0 4k" -c "fsync" $SCRATCH_MNT/dummy >> $seqres.= full + +echo "sync $SCRATCH_MNT to writeback" >> $seqres.full +$XFS_IO_PROG -c "syncfs" $SCRATCH_MNT >> $seqres.full + +echo "----- Testing with atomi write on mixed mapping -----" >> $seqres.fu= ll + +echo "Create the original file" >> $seqres.full +touch $SCRATCH_MNT/foobar2 >> $seqres.full + +echo "Create 2 level extent tree (btree) for foobar2 with a unwritten exte= nt" >> $seqres.full +$XFS_IO_PROG -f -c "pwrite 0 4k" -c "falloc 4k 4k" -c "pwrite 8k 4k" \ + -c "pwrite 20k 4k" -c "pwrite 28k 4k" -c "pwrite 36k 4k" \ + -c "fsync" $SCRATCH_MNT/foobar2 >> $seqres.full + +$XFS_IO_PROG -c "fiemap -v" $SCRATCH_MNT/foobar2 >> $seqres.full + +echo "Convert unwritten extent to written and collapse extent tree to inod= e" >> $seqres.full +$XFS_IO_PROG -dc "pwrite -A -V1 0k 12k" $SCRATCH_MNT/foobar2 >> $seqres.fu= ll + +echo "Create a new file and do fsync to force a jbd2 commit" >> $seqres.fu= ll +$XFS_IO_PROG -f -c "pwrite 0 4k" -c "fsync" $SCRATCH_MNT/dummy2 >> $seqres= .full + +echo "sync $SCRATCH_MNT to writeback" >> $seqres.full +$XFS_IO_PROG -c "syncfs" $SCRATCH_MNT >> $seqres.full + +# success, all done +echo "Silence is golden" +status=3D0 +exit diff --git a/tests/ext4/064.out b/tests/ext4/064.out new file mode 100644 index 00000000..d9076546 --- /dev/null +++ b/tests/ext4/064.out @@ -0,0 +1,2 @@ +QA output created by 064 +Silence is golden --=20 2.49.0