From nobody Sun Sep 7 23:48:49 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 94FB72E7166; Fri, 22 Aug 2025 08:02:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849751; cv=none; b=oiMgBi410u8CBSBBouO9m48saGkQzww0s8nFdiIW6eSXFtuh+pfNOXpmzqxkH7vpMFHc0dtm2ndJFUEAHhuLNYUpIM7DZU1CnJiznRPBBi0pSxNzokvW+BmQjkxLGe+AGnqKn0HibTXyFzUAgJh5Jyuda4NoNiifsZvPxsFMsK4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849751; c=relaxed/simple; bh=rwemJAsET1QAd67z1zthbFOTnKYTlW3d35XQz3vetPs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Qlf1eolW5aTz7l3I3vNJpGXzszn31zTPZvPzoCiwbxXdS/dH/dg4nsyZsftxglQH9lqFGMoB/ibyhZZrSkh+Ga/vBAWsq72wd+Y8qP8dpySSTMGCIeXKR9q2SL1i7noK69He7huwZ4pv8FjEQFP9Rx4YehOJucfFK/1Oepl8HpI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=bExb2aTR; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="bExb2aTR" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57M5onvN011916; Fri, 22 Aug 2025 08:02:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=MfELG6FC8BYGZ51xy ZXxPCf5stkdh9Ke37+72AOOym4=; b=bExb2aTRXf1+niWIIWVBJF0cTg9TTBRnf o6IFFFkbxkfktidr36IjN5kSV92raZiieoXG9rDarY29ERjeSdCDr2CeuAMFAnJo m8f8r5Ykt6F9aMwJSJH7DHHb3BfIOzRz4VqOvG0LZGgiX+VMxOuhMa7l9gEO5jvL wEM3ZpJl1BoFAVFkqqLi4Q/56QHSyUbll0YXZXUe+MatME7BfN1M024+dRIWveHN ihm2LftHd+9MacDJymxVqpdte1IqW7L7r8CwEi49FoFcWYQpFK9OFBd+/ndJWMeb EBCMUxJZhWztyIUVi93q1zOgz1TEJ9N5aKV+qaS9oFbsatGRkyfYg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vw4cp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:20 +0000 (GMT) Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 57M82K19024216; Fri, 22 Aug 2025 08:02:20 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vw4cj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:20 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57M3tj9C031881; Fri, 22 Aug 2025 08:02:19 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 48my5ycafy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:19 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57M82HjS51708290 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Aug 2025 08:02:17 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7457620043; Fri, 22 Aug 2025 08:02:17 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3573C20040; Fri, 22 Aug 2025 08:02:15 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.210.10]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 Aug 2025 08:02:14 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v5 01/12] common/rc: Add _min() and _max() helpers Date: Fri, 22 Aug 2025 13:32:00 +0530 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODE5MDIyMiBTYWx0ZWRfXzzObLmZRTntO /ieerJqJZ1VLoXwBinbPqRuW9vfC4qzkfFaUJPEx+ZWB6nupwvCIyDpoBKfYBGh+5Wx9VJ08h83 UZ9ytc416XG1uG2JVBoVnHFOvOlxEXt+ax8PBfu1ExTQ8zF/A++zjD4heSsGtJkW0o450tLNn2Y xNRyOi95kKUFNbhR+8W4pAnRJkuBDnA/BIjq932cNcZQ/iZikXSPswBZJuHg0Fe7s2jepJut1RH eKk52bfjAVgaOOrmaKTA5fh3w950czJypgKkKNiNkvwAQFre5Agk49JLp0DyuCyac/21B0Z4pt9 4dHks6lD5jO7tFRmWG7CMKryjfieTqFIuxVa77Y3w2GD7d2vCFL6itCFEmw484TlXy5iE+TsXG+ Uq6OnMaKCrGRgNlwtN9zDSCMxBkvZw== X-Proofpoint-ORIG-GUID: 4tTTkPKftnEx5t4ixF_8MEZQ-95yuz0q X-Authority-Analysis: v=2.4 cv=T9nVj/KQ c=1 sm=1 tr=0 ts=68a8240c cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=2OwXVqhp2XgA:10 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=wDx1mg2EYdB0JXDzLs4A:9 a=cPQSjfK2_nFv0Q5t_7PE:22 X-Proofpoint-GUID: yBheFqQRLRTkq6ReGE_gElyfI4wBjMBn X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-22_02,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 malwarescore=0 phishscore=0 spamscore=0 clxscore=1015 bulkscore=0 suspectscore=0 lowpriorityscore=0 priorityscore=1501 adultscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2508110000 definitions=main-2508190222 Content-Type: text/plain; charset="utf-8" Many programs open code these functionalities so add it as a generic helper in common/rc Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo --- common/rc | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/common/rc b/common/rc index ff5df203..35a1c835 100644 --- a/common/rc +++ b/common/rc @@ -5975,6 +5975,28 @@ _require_inplace_writes() fi } =20 +_min() { + local ret + + for arg in "$@"; do + if [ -z "$ret" ] || (( $arg < $ret )); then + ret=3D"$arg" + fi + done + echo $ret +} + +_max() { + local ret + + for arg in "$@"; do + if [ -z "$ret" ] || (( $arg > $ret )); then + ret=3D"$arg" + fi + done + echo $ret +} + ##########################################################################= ###### # make sure this script returns success /bin/true --=20 2.49.0 From nobody Sun Sep 7 23:48:49 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7454C2E54C7; Fri, 22 Aug 2025 08:02:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849760; cv=none; b=MXpnLpwwyE2qIHUIX8/kDgeiLsWik5idj3sQpkkD7MFuUPZ0asu1Y1LS9Dpc6EoTTInvSZkICjeYI/zKyQRLRysSYS3KKXStuRPnALYy5MLD+pg5x6zCp9We34Ah2W7zC87tt01xiTiNNnM4m63LXHrp1cjLFeeEZp1g5+gGY1g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849760; c=relaxed/simple; bh=h7wbbamg4JolKV3mimLB8bIlNNmks/rM2pudmV6msgw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fveoaE3VOGTvLbMpWILDIEvm5bZeiMo+o+rhK17PYSd6OoNtF4Kz/shdjjbPqCrkjWjRcLgEmk1AN78sx/GTUfesmR0NugPB/PM2rQKoPbEkjLOU+B86m5NH9sgeOKSHW6BZXLz8vBXYCjHjmUOsJ5b/WbbW3g7xgWMq8usgAGo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=FZ8afuQx; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="FZ8afuQx" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57LIXov7012217; Fri, 22 Aug 2025 08:02:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=zpb085wPGmEPbAst3 E2KRxlV+xk0WkLGfPuijn8WhgI=; b=FZ8afuQxmlJ6/eBt68nDdc7Yr1/Oju5Xq 2/0WSaN120Tl8VSBGOa4vQ0mDUvd1S+u+LT+auebq+5KUqVP9RngZPVZcIiS87/i OF7BBVjd1ceUTC7Vsbtta3dniD7O5dgjKHmS4EAy2hDAAPDEUV4OfWgWNFv9vO4R UlFB/Q4/WDuuZTu0EpUXbeAwttoPpvG40lg4UDlpL1s9k1rAwB6KuLHhMEPFdxYY iUyDzm/w8fjwt26SpsQiXvPXe0sWRgHNAfSWfNUxTHJ8qOor68bhx9NSOsD5RQa0 DPIqB/XClOpwz/wMCi/GBlNA0y2sfa3YW9D9tbJ9P2Dpaw5ow1A/g== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vnawp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:23 +0000 (GMT) Received: from m0360083.ppops.net (m0360083.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 57M82NWT010015; Fri, 22 Aug 2025 08:02:23 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vnawg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:23 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57M4KUCm008726; Fri, 22 Aug 2025 08:02:22 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 48my42cadf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:22 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57M82KnQ58917324 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Aug 2025 08:02:20 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 219C420040; Fri, 22 Aug 2025 08:02:20 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C964D20043; Fri, 22 Aug 2025 08:02:17 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.210.10]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 Aug 2025 08:02:17 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v5 02/12] common/rc: Add _require_fio_version helper Date: Fri, 22 Aug 2025 13:32:01 +0530 Message-ID: <955d47b2534d9236adbd2bbd13598bbd1da8fc04.1755849134.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: FQ5llxCM6DH61Rgdbx68JJqk60laNYSh X-Authority-Analysis: v=2.4 cv=IrhHsL/g c=1 sm=1 tr=0 ts=68a82410 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=0B_TZz9YXGle3_EXArAA:9 a=cPQSjfK2_nFv0Q5t_7PE:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODE5MDIyMiBTYWx0ZWRfXxrD64K5fCyeB O4tZskCw6In2YscmVoKx7c1Blj1vvn2bymwK+Jfa5hSNrt+tN8XXgYR8OdIEmcgX+RaXUgu56NV YaZqDe0v/thsY+VhM8DxtuAjXPD711TYRCjJLv6+YR9nbxoKN1OCBBiYqyIx9f5BABM5/byApzx dXxVrdNYC1HuT1p3la0Y/mMWkTNsQsJLtli5VqHTPaVeuJcJdaFTsW7oUiksF3iUJEcI/Zcr34o dm+3gdBkXrMfOmkbLvtHbuADHJICEgdliTyU5U0D9FKzMGZjIZNtzlQtRtpk2iont8o+QSE8MJ/ RPZzA2wrmMKL5OcmGt5ocaofa4Sf2lHZw4QgNqPgbGBqkIQ5vXoUdpwYRsE/7LA4POsTGEQ1iPA 6rsTsjEfsTZAUrFs39rSvxwKXtAKgw== X-Proofpoint-ORIG-GUID: bB2KwJJOyyLW5hz9oicfrl2rFA6IoRer X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-22_02,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 bulkscore=0 suspectscore=0 spamscore=0 lowpriorityscore=0 impostorscore=0 adultscore=0 priorityscore=1501 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2508110000 definitions=main-2508190222 Content-Type: text/plain; charset="utf-8" The main motivation of adding this function on top of _require_fio is that there has been a case in fio where atomic=3D option was added but later it was changed to noop since kernel didn't yet have support for atomic writes. It was then again utilized to do atomic writes in a later version, once kernel got the support. Due to this there is a point in fio where _require_fio w/ atomic=3D1 will succeed even though it would not be doing atomic writes. Hence, add an explicit helper to ensure tests to require specific versions of fio to work past such issues. Signed-off-by: Ojaswin Mujoo Reviewed-by: Zorro Lang --- common/rc | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/common/rc b/common/rc index 35a1c835..f45b9a38 100644 --- a/common/rc +++ b/common/rc @@ -5997,6 +5997,38 @@ _max() { echo $ret } =20 +# Check the required fio version. Examples: +# _require_fio_version 3.38 (matches 3.38 only) +# _require_fio_version 3.38+ (matches 3.38 and above) +# _require_fio_version 3.38- (matches 3.38 and below) +_require_fio_version() { + local req_ver=3D"$1" + local fio_ver + + _require_fio + _require_math + + fio_ver=3D$(fio -v | cut -d"-" -f2) + + case "$req_ver" in + *+) + req_ver=3D${req_ver%+} + test $(_math "$fio_ver >=3D $req_ver") -eq 1 || \ + _notrun "need fio >=3D $req_ver (found $fio_ver)" + ;; + *-) + req_ver=3D${req_ver%-} + test $(_math "$fio_ver <=3D $req_ver") -eq 1 || \ + _notrun "need fio <=3D $req_ver (found $fio_ver)" + ;; + *) + req_ver=3D${req_ver%-} + test $(_math "$fio_ver =3D=3D $req_ver") -eq 1 || \ + _notrun "need fio =3D $req_ver (found $fio_ver)" + ;; + esac +} + ##########################################################################= ###### # make sure this script returns success /bin/true --=20 2.49.0 From nobody Sun Sep 7 23:48:49 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 745DA2EAB8E; Fri, 22 Aug 2025 08:02:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849761; cv=none; b=dbpEKkFDYdXph9UuuYROczaOMqBlyLSJQyyxTH1oqkYpTho71FI6c34rpINFDU8fivm1LNh2zK0jmhaAdxme3LfAEvK+63sZZ1XjkVcr/26Rcr6x40VXG2lCCo6GxG9DPS/7/41zfTxRqDQwjrb9OPaulu/akVPvwpc26UhS7s0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849761; c=relaxed/simple; bh=PGymx0PrCWx0rSx5jirmfScMWFXcrc17cbYCRh65R8c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TIAdNnJnFjTkWK6nqHjVhIL256RZZq3da9mL+rsIMV+soeMM3jrM5+oz4uMr4XipjZu/NAuQV/We33RgtFkhnknettq7t+ZCZf8a57cSQXKUoexcDVNNVm5j1iqj4mtvlMhmZ6vF79Vm0deuhwIIVm74eT6fUN6hUKIhu83//FY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=GH64L5LO; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="GH64L5LO" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57M7cxEO026013; Fri, 22 Aug 2025 08:02:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=Ydj4+7gOr5w73AX63 sXYfyK96HWJzRLoYcAq068iRWI=; b=GH64L5LOaDHopeW7OJMhxUt6uSNEm6NTL Bu0bbc71/kdYd1NcEigLfH0PvFqb2tBEa8PB74KAFyzUZW/ahUloqml79dpKQyrn LONxuF57oxvQPBho5x5I8LM2oL1/NXn2qy4HlNeHVNbHOfAse8VevPJVX4SfCA76 BJkOoCQ53mAZNo58HXuD6pS4lv2wv307nFmt7OkKRZycTSU4M4iRa8nXfXvpN/VZ GrIR+LnatbEQWkJ8Po3yNkNfMSMRTEke4jV2vS/hosJHfIbOpMx47th/9S934UVS 84+4MfCA/cJmHxSbrz1QHYpJtQ1uHIKaqxCdNHaSdO+0hsphUiMlA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vnbpd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:25 +0000 (GMT) Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 57M82Pqw031133; Fri, 22 Aug 2025 08:02:25 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vnbp8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:25 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57M7dJR1016047; Fri, 22 Aug 2025 08:02:24 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 48my42cace-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:24 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57M82MAl53412202 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Aug 2025 08:02:22 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C336420040; Fri, 22 Aug 2025 08:02:22 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 848AA2004B; Fri, 22 Aug 2025 08:02:20 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.210.10]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 Aug 2025 08:02:20 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v5 03/12] common/rc: Add a helper to run fsx on a given file Date: Fri, 22 Aug 2025 13:32:02 +0530 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODE5MDIyMiBTYWx0ZWRfXwsQrW6ALsu+k dJ2LpZieupJ0leTUo+Z2EzRq8Q30oppXvBlf/+aBBuvRVReENKeHKhPp99CsO5LKZ5oJRAjvcVA KQYg4agw2MOR856oZrfJcrTVsFFHunKrRAO8IRvvfa+yu5/wrEDL4s4bgznbDYEU54LcUDupl4r CdQjaALYMKwvdAn0Qc615j33hnc2RAA0oODlpSMPxHk/AXlfgqQAihxMU399jf1DgAyjvmWqk7B CdVyAsV3+OeKQpI5zd/BbCmnWvcNBhg1xPfI8cwcwJNtXRPQnOptK1Ql+mlEt2qMGI0ff5exL+P W9iamVb0aqC0zaidLHaaPCh+FfYxLfCvICGXPbeOpuZHxgs6mCia5EYNn7XLCpck2zclNTpbYue 2dAP9UdTWZQMAoB1RXBBp9A62iasaA== X-Authority-Analysis: v=2.4 cv=PMlWOfqC c=1 sm=1 tr=0 ts=68a82412 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=u_VYNAqGXo6OSSAA:21 a=2OwXVqhp2XgA:10 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=EfcJlRrZmV05EoLhlkQA:9 a=cPQSjfK2_nFv0Q5t_7PE:22 X-Proofpoint-GUID: cbwpB5PbxV34VfShdcjJEKjsx1MpRGmQ X-Proofpoint-ORIG-GUID: BIW_vhtrVPiD9Uz1f_C9WJxVB0rwskoS X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-22_02,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 bulkscore=0 adultscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 spamscore=0 phishscore=0 suspectscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2508110000 definitions=main-2508190222 Content-Type: text/plain; charset="utf-8" Currently run_fsx is hardcoded to run on a file in $TEST_DIR. Add a helper _run_fsx_on_file so that we can run fsx on any given file including in $SCRATCH_MNT. Also, refactor _run_fsx to use this helper. No functional change is intended in this patch. Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo --- common/rc | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/common/rc b/common/rc index f45b9a38..85baa677 100644 --- a/common/rc +++ b/common/rc @@ -5200,13 +5200,24 @@ _require_hugepage_fsx() _notrun "fsx binary does not support MADV_COLLAPSE" } =20 -_run_fsx() +_run_fsx_on_file() { + local testfile=3D$1 + shift + + if ! [ -f $testfile ] + then + echo "_run_fsx_on_file: $testfile doesn't exist. Creating" >> $seqres.fu= ll + touch $testfile + fi + echo "fsx $*" local args=3D`echo $@ | sed -e "s/ BSIZE / $bsize /g" -e "s/ PSIZE / $psi= ze /g"` - set -- $FSX_PROG $args $FSX_AVOID $TEST_DIR/junk + + set -- $FSX_PROG $args $FSX_AVOID $testfile + echo "$@" >>$seqres.full - rm -f $TEST_DIR/junk + rm -f $testfile "$@" 2>&1 | tee -a $seqres.full >$tmp.fsx local res=3D${PIPESTATUS[0]} if [ $res -ne 0 ]; then @@ -5218,6 +5229,12 @@ _run_fsx() return 0 } =20 +_run_fsx() +{ + _run_fsx_on_file $TEST_DIR/junk $@ + return $? +} + # Run fsx with -h(ugepage buffers). If we can't set up a hugepage then sk= ip # the test, but if any other error occurs then exit the test. _run_hugepage_fsx() { --=20 2.49.0 From nobody Sun Sep 7 23:48:49 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E1062EAB89; Fri, 22 Aug 2025 08:02:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849758; cv=none; b=Uhcw6Pju2W9RHDcq1eENf79FmqXHQQxnIkJOXyv/9CDhfWAo3cg/w2ODuRKr+MWBLmQEcbHD3+3tQvQtBy+fW5zn7f7AXjWEYdoHqphiRdP+p++bztJ76X4oa/eWvlN946cA2F6gj0J5h4I4j02jaMDTK5hGS2+DzWaMYriPCMU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849758; c=relaxed/simple; bh=dhBJ3VIPWMssZ/rPWvWflKK34SHaDD06SBMA0iSNSKs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WxCuwRXe3VwNFm7Ya436RMk23hDtgQe9yTy6lQHgSUwgWjjyKPaf5ORRZiMTnLA1pZxq4NrVmZEq/fnHir2gPPm3ggJABBEthCUyaEhcyG2/VsNSS3Fy+GPhT6NyNjPoNxC7zJsOpMwAn+JvOf9pnVB0lyjna62MSudcz3K4ttg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Kk38Fk3p; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Kk38Fk3p" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57M5UW1s012155; Fri, 22 Aug 2025 08:02:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=dtrzSBLIxKzNH/3Bh JjPBCqET9CFnPbTO0ltX/5I0eM=; b=Kk38Fk3pLNwPODt+ntVgE4GX/I/J75Vmh 5LiB9gvQ/2B5eWf02ObQni1p2BLdVeiu9ZJX+dSCNWC1EN4wJ4U4GJPL9KfCKGhn EdbXo+pfATynCB4OxTb+tItDvwGXV/pfXYLy/P2DsCCuyvkGITnH46lW94yaeEBN vlpR9NJx5m/s24tTUr6mmUCKSvYVbcHdvz3vdhviWnkrPN1NUFlrTW5W/D/QTR6X IUvZRFkyWxJxaG3/vAuhOIUy8sT2OCVmeYrZYvvrlKMfPnivhOP8R1WzDoGEpEp/ EPmp+8HAS+/M2FZbW6r90XzV/khpIcOFO5wme6r3zIKDVUZIDHD4A== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vw4d6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:28 +0000 (GMT) Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 57M82Sr2024566; Fri, 22 Aug 2025 08:02:28 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vw4d3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:28 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57M3o9KG027145; Fri, 22 Aug 2025 08:02:27 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 48my4wcamf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:27 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57M82PPD47251808 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Aug 2025 08:02:25 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9610A20043; Fri, 22 Aug 2025 08:02:25 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 26C8A20040; Fri, 22 Aug 2025 08:02:23 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.210.10]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 Aug 2025 08:02:22 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v5 04/12] ltp/fsx.c: Add atomic writes support to fsx Date: Fri, 22 Aug 2025 13:32:03 +0530 Message-ID: <8b7e007fd87918a0c3976ca7d06c089ed9b0070c.1755849134.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODE5MDIyMiBTYWx0ZWRfX8D9yZlJVY0Ih INf1RPkzha5UODJNfclq/0vu66ZfLjOqcWB4HpflZqNxcYhCv5EZR0AKLbRVrd2luqJMqrrn/9a quSxCJXfWesiiUZFa9DTH+u2IFOXS0MYTu9k/SaJHzc5L1Aj8GTjF6Xv98xMmxSukcNaOoFMc9W mcliqu0inj26thDlu0p9pj8+4vnQsW/1jl3wKJBALdFrxtdGbf2hI+YrumnJ+er33YhlCP8xo1T dCqXtIFMoekCskLbue8RcJPkw3BREko/U6dp4zDWlrkP+18fCxzrSzXGwxOdsywKDFBhe3K9ctM mZjVT5ZerAJIdYy28/qSlxAte1ozco/E0IRMFr6QmcomupEUhNfWyYg/ins6Dbo54jQYg2rywRG aXRMVgyb3PlEZTrMyQ31TAyd9mPWdw== X-Proofpoint-ORIG-GUID: pPdboEDvMNyinyHAFXUwwKcnjyOAQrH- X-Authority-Analysis: v=2.4 cv=T9nVj/KQ c=1 sm=1 tr=0 ts=68a82415 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=2OwXVqhp2XgA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=oUtnbjD3gOVQiAq08lgA:9 a=cPQSjfK2_nFv0Q5t_7PE:22 X-Proofpoint-GUID: qir_D8ueDVu_TrYAlXPX5ChbEyk1lVhN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-22_02,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 malwarescore=0 phishscore=0 spamscore=0 clxscore=1015 bulkscore=0 suspectscore=0 lowpriorityscore=0 priorityscore=1501 adultscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2508110000 definitions=main-2508190222 Content-Type: text/plain; charset="utf-8" Implement atomic write support to help fuzz atomic writes with fsx. Suggested-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo Reviewed-by: John Garry --- ltp/fsx.c | 115 +++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 110 insertions(+), 5 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 163b9453..1582f6d1 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -40,6 +40,7 @@ #include #endif #include +#include "statx.h" =20 #ifndef MAP_FILE # define MAP_FILE 0 @@ -49,6 +50,10 @@ #define RWF_DONTCACHE 0x80 #endif =20 +#ifndef RWF_ATOMIC +#define RWF_ATOMIC 0x40 +#endif + #define NUMPRINTCOLUMNS 32 /* # columns of data to print on each line */ =20 /* Operation flags (bitmask) */ @@ -110,6 +115,7 @@ enum { OP_READ_DONTCACHE, OP_WRITE, OP_WRITE_DONTCACHE, + OP_WRITE_ATOMIC, OP_MAPREAD, OP_MAPWRITE, OP_MAX_LITE, @@ -200,6 +206,11 @@ int uring =3D 0; int mark_nr =3D 0; int dontcache_io =3D 1; int hugepages =3D 0; /* -h flag */ +int do_atomic_writes =3D 1; /* -a flag disables */ + +/* User for atomic writes */ +int awu_min =3D 0; +int awu_max =3D 0; =20 /* Stores info needed to periodically collapse hugepages */ struct hugepages_collapse_info { @@ -288,6 +299,7 @@ static const char *op_names[] =3D { [OP_READ_DONTCACHE] =3D "read_dontcache", [OP_WRITE] =3D "write", [OP_WRITE_DONTCACHE] =3D "write_dontcache", + [OP_WRITE_ATOMIC] =3D "write_atomic", [OP_MAPREAD] =3D "mapread", [OP_MAPWRITE] =3D "mapwrite", [OP_TRUNCATE] =3D "truncate", @@ -422,6 +434,7 @@ logdump(void) prt("\t***RRRR***"); break; case OP_WRITE_DONTCACHE: + case OP_WRITE_ATOMIC: case OP_WRITE: prt("WRITE 0x%x thru 0x%x\t(0x%x bytes)", lp->args[0], lp->args[0] + lp->args[1] - 1, @@ -1073,6 +1086,25 @@ update_file_size(unsigned offset, unsigned size) file_size =3D offset + size; } =20 +static int is_power_of_2(unsigned n) { + return ((n & (n - 1)) =3D=3D 0); +} + +/* + * Round down n to nearest power of 2. + * If n is already a power of 2, return n; + */ +static int rounddown_pow_of_2(int n) { + int i =3D 0; + + if (is_power_of_2(n)) + return n; + + for (; (1 << i) < n; i++); + + return 1 << (i - 1); +} + void dowrite(unsigned offset, unsigned size, int flags) { @@ -1081,6 +1113,27 @@ dowrite(unsigned offset, unsigned size, int flags) offset -=3D offset % writebdy; if (o_direct) size -=3D size % writebdy; + if (flags & RWF_ATOMIC) { + /* atomic write len must be inbetween awu_min and awu_max */ + if (size < awu_min) + size =3D awu_min; + if (size > awu_max) + size =3D awu_max; + + /* atomic writes need power-of-2 sizes */ + size =3D rounddown_pow_of_2(size); + + /* atomic writes need naturally aligned offsets */ + offset -=3D offset % size; + + /* Skip the write if we are crossing max filesize */ + if ((offset + size) > maxfilelen) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping atomic write past maxfilelen\n"); + log4(OP_WRITE_ATOMIC, offset, size, FL_SKIPPED); + return; + } + } if (size =3D=3D 0) { if (!quiet && testcalls > simulatedopcount && !o_direct) prt("skipping zero size write\n"); @@ -1088,7 +1141,10 @@ dowrite(unsigned offset, unsigned size, int flags) return; } =20 - log4(OP_WRITE, offset, size, FL_NONE); + if (flags & RWF_ATOMIC) + log4(OP_WRITE_ATOMIC, offset, size, FL_NONE); + else + log4(OP_WRITE, offset, size, FL_NONE); =20 gendata(original_buf, good_buf, offset, size); if (offset + size > file_size) { @@ -1108,8 +1164,9 @@ dowrite(unsigned offset, unsigned size, int flags) (monitorstart =3D=3D -1 || (offset + size > monitorstart && (monitorend =3D=3D -1 || offset <=3D monitorend)))))) - prt("%lld write\t0x%x thru\t0x%x\t(0x%x bytes)\tdontcache=3D%d\n", testc= alls, - offset, offset + size - 1, size, (flags & RWF_DONTCACHE) !=3D 0); + prt("%lld write\t0x%x thru\t0x%x\t(0x%x bytes)\tdontcache=3D%d atomic_wr= =3D%d\n", testcalls, + offset, offset + size - 1, size, (flags & RWF_DONTCACHE) !=3D 0, + (flags & RWF_ATOMIC) !=3D 0); iret =3D fsxwrite(fd, good_buf + offset, size, offset, flags); if (iret !=3D size) { if (iret =3D=3D -1) @@ -1785,6 +1842,36 @@ do_dedupe_range(unsigned offset, unsigned length, un= signed dest) } #endif =20 +int test_atomic_writes(void) { + int ret; + struct statx stx; + + if (o_direct !=3D O_DIRECT) { + fprintf(stderr, "main: atomic writes need O_DIRECT (-Z), " + "disabling!\n"); + return 0; + } + + ret =3D xfstests_statx(AT_FDCWD, fname, 0, STATX_WRITE_ATOMIC, &stx); + if (ret < 0) { + fprintf(stderr, "main: Statx failed with %d." + " Failed to determine atomic write limits, " + " disabling!\n", ret); + return 0; + } + + if (stx.stx_attributes & STATX_ATTR_WRITE_ATOMIC && + stx.stx_atomic_write_unit_min > 0) { + awu_min =3D stx.stx_atomic_write_unit_min; + awu_max =3D stx.stx_atomic_write_unit_max; + return 1; + } + + fprintf(stderr, "main: IO Stack does not support " + "atomic writes, disabling!\n"); + return 0; +} + #ifdef HAVE_COPY_FILE_RANGE int test_copy_range(void) @@ -2356,6 +2443,12 @@ have_op: goto out; } break; + case OP_WRITE_ATOMIC: + if (!do_atomic_writes) { + log4(OP_WRITE_ATOMIC, offset, size, FL_SKIPPED); + goto out; + } + break; } =20 switch (op) { @@ -2385,6 +2478,11 @@ have_op: dowrite(offset, size, 0); break; =20 + case OP_WRITE_ATOMIC: + TRIM_OFF_LEN(offset, size, maxfilelen); + dowrite(offset, size, RWF_ATOMIC); + break; + case OP_MAPREAD: TRIM_OFF_LEN(offset, size, file_size); domapread(offset, size); @@ -2511,13 +2609,14 @@ void usage(void) { fprintf(stdout, "usage: %s", - "fsx [-dfhknqxyzBEFHIJKLORWXZ0]\n\ + "fsx [-adfhknqxyzBEFHIJKLORWXZ0]\n\ [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid]\n\ [-l flen] [-m start:end] [-o oplen] [-p progressinterval]\n\ [-r readbdy] [-s style] [-t truncbdy] [-w writebdy]\n\ [-A|-U] [-D startingop] [-N numops] [-P dirpath] [-S seed]\n\ [--replay-ops=3Dopsfile] [--record-ops[=3Dopsfile]] [--duration=3Dseco= nds]\n\ ... fname\n\ + -a: disable atomic writes\n\ -b opnum: beginning operation number (default 1)\n\ -c P: 1 in P chance of file close+open at each op (default infinity)\n\ -d: debug output for all operations\n\ @@ -3059,9 +3158,13 @@ main(int argc, char **argv) setvbuf(stdout, (char *)0, _IOLBF, 0); /* line buffered stdout */ =20 while ((ch =3D getopt_long(argc, argv, - "0b:c:de:fg:hi:j:kl:m:no:p:qr:s:t:uw:xyABD:EFJKHzCILN:OP:RS:UWXZ", + "0ab:c:de:fg:hi:j:kl:m:no:p:qr:s:t:uw:xyABD:EFJKHzCILN:OP:RS:UWXZ", longopts, NULL)) !=3D EOF) switch (ch) { + case 'a': + prt("main(): Atomic writes disabled\n"); + do_atomic_writes =3D 0; + break; case 'b': simulatedopcount =3D getnum(optarg, &endp); if (!quiet) @@ -3475,6 +3578,8 @@ main(int argc, char **argv) exchange_range_calls =3D test_exchange_range(); if (dontcache_io) dontcache_io =3D test_dontcache_io(); + if (do_atomic_writes) + do_atomic_writes =3D test_atomic_writes(); =20 while (keep_running()) if (!test()) --=20 2.49.0 From nobody Sun Sep 7 23:48:49 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7B2B17BA3; Fri, 22 Aug 2025 08:02:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849760; cv=none; b=FiGEJXKVPgXpu8qoyH9T6SgzMWU4Lcov2zsniXgm0L769gTh2EuRw9ASYggc0qrjSrYDHBXPjhn/l0uIaOU1zyywJX9PeeBq4xONg0I6f8kn7P67YITb/6fIKe0/Mo7b33oYKsLKLVHMVep2KDr0Y0MHTCV3SDMigK0ydZSdemw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849760; c=relaxed/simple; bh=1Zv5IS2OEvsc1CKEUq7+MH1dZdSqezFI+uP8i9Vi4oQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mlIXjcBcsH1USHDyn2oRutRAoI26GjpKe8nwzpGtIWE151T/20ga80f6JklYsxb/qmqVUvTJQNYa/J6WhYdxQGUsFZWvQoBeWjQybKoTVy76TT/3FDg6a92beoE+lxU8IAR/jfWCIe6uSCxlAej+v5NZUjchc9GWm0BbxRDLPb8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=cnWmEWuI; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="cnWmEWuI" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57LKhJRe012171; Fri, 22 Aug 2025 08:02:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=5CB5aNgIhq1p3l6D2 7WH6MDjxvFiRVYR09zhXrF2E5E=; b=cnWmEWuISpHcTUNBNdilLfq0zLjwL5q39 6ImS3qkHmHP6cA7HoPAtPibEuMxwfxoaEvyeQIZlVOQFowvGGqMI402juoOnsaJp hh2YTtdYFyigYVTxL8HZSdDDyUzz70gxT5kFJ+phIyGLGl9ZiwBFbh7/Fm6UZ0p6 4JEDFfjf9He7C79b6Vo4sgEf+1BwIF5KhS9/W6dF1wxikIH68jpSC88HbfNlCIVi Ngeah7zzEWp0OX719lumXz6/zWu7pGsy+f6ZxLkQcDHuUP1QR5TmhluT5liM1/Gn dEiFnh/tJEYb9IhIeraRLerMjeFpAKVqvFK4nyc/8E6Bxd80kcqAg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vnaxk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:31 +0000 (GMT) Received: from m0360083.ppops.net (m0360083.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 57M82Vj3010348; Fri, 22 Aug 2025 08:02:31 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vnaxg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:31 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57M7dJR2016047; Fri, 22 Aug 2025 08:02:30 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 48my42cacu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:30 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57M82SLh20709776 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Aug 2025 08:02:28 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8241F2004B; Fri, 22 Aug 2025 08:02:28 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 13A3920040; Fri, 22 Aug 2025 08:02:26 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.210.10]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 Aug 2025 08:02:25 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v5 05/12] generic: Add atomic write test using fio crc check verifier Date: Fri, 22 Aug 2025 13:32:04 +0530 Message-ID: <90241ec96d84e6e87d8cf8bd0d0d75fcc296757c.1755849134.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: U5r_lrmHFMLHvoCrNeKlwncZMwkiABg8 X-Authority-Analysis: v=2.4 cv=IrhHsL/g c=1 sm=1 tr=0 ts=68a82417 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=2OwXVqhp2XgA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=TDUsxBfbQoAVH7yvDjsA:9 a=cPQSjfK2_nFv0Q5t_7PE:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODE5MDIyMiBTYWx0ZWRfX0/LaShGk/rgI zdx2Q5dqP8In2Qaccy0XTrxFNqYPUVZDNrTziKuxW1vqzbIYk2Qx10VQkeX4yUjrqEc3R6eEhIx ou+YgOVHBWsWRxh/i8LItvl4fdQDIOi5bTDpXRv0E5HiUlOobFoUiRhQ7gwmUclZ3uaaEPiLaR0 PO3pmENPTxBCA34NL9fMo0FdPY9sr2kfwTYoObUZUkGwBsrXEjc8l10xJ0f/seiyzsqzlDADI2S GKEbA/2Omrk80cqCVPLMBfz3wBols4zKTJzmrg13Kgg/WsN7YKULR8EEBm9eR+JYb4045KT49IJ ZTfllDi+DILWcaPtxaDGHeTRpBlzWOMzJzXGbdXVy+j6LRTpUim92wQU1gushQxyScS+sO17L3p elgXZl8JDqaR3Vs4NBwejvDEsQhTRg== X-Proofpoint-ORIG-GUID: 0FV5Qu1ubGrCdpfBVXmPrtCOfFuk0lJ9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-22_02,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 bulkscore=0 suspectscore=0 spamscore=0 lowpriorityscore=0 impostorscore=0 adultscore=0 priorityscore=1501 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2508110000 definitions=main-2508190222 Content-Type: text/plain; charset="utf-8" This adds atomic write test using fio based on it's crc check verifier. fio adds a crc header for each data block, which is verified later to ensure there is no data corruption or torn write. This test essentially does a lot of parallel RWF_ATOMIC IO on a preallocated file to stress the write and end-io unwritten conversion code paths. The idea is to increase code coverage to ensure RWF_ATOMIC hasn't introduced any issues. Avoid doing overlapping parallel atomic writes because it might give unexpected results. Use offset_increment=3D, size=3D fio options to achieve this behavior. Co-developed-by: Ritesh Harjani (IBM) Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo Reviewed-by: John Garry --- tests/generic/1226 | 108 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1226.out | 2 + 2 files changed, 110 insertions(+) create mode 100755 tests/generic/1226 create mode 100644 tests/generic/1226.out diff --git a/tests/generic/1226 b/tests/generic/1226 new file mode 100755 index 00000000..4584f062 --- /dev/null +++ b/tests/generic/1226 @@ -0,0 +1,108 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1226 +# +# Validate FS atomic write using fio crc check verifier. +# +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto aio rw atomicwrites + +_require_scratch_write_atomic +_require_odirect +_require_aio +_require_fio_version "3.38+" + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount +_require_xfs_io_command "falloc" + +touch "$SCRATCH_MNT/f1" +awu_min_write=3D$(_get_atomic_write_unit_min "$SCRATCH_MNT/f1") +awu_max_write=3D$(_get_atomic_write_unit_max "$SCRATCH_MNT/f1") + +blocksize=3D$(_max "$awu_min_write" "$((awu_max_write/2))") +threads=3D$(_min "$(($(nproc) * 2 * LOAD_FACTOR))" "100") +filesize=3D$((blocksize * threads * 100)) +depth=3D$threads +io_size=3D$((filesize / threads)) +io_inc=3D$io_size +testfile=3D$SCRATCH_MNT/test-file + +fio_config=3D$tmp.fio +fio_out=3D$tmp.fio.out + +fio_aw_config=3D$tmp.aw.fio +fio_verify_config=3D$tmp.verify.fio + +function create_fio_configs() +{ + create_fio_aw_config + create_fio_verify_config +} + +function create_fio_verify_config() +{ +cat >$fio_verify_config <$fio_aw_config <> $seqres.full +cat $fio_verify_config >> $seqres.full + +$XFS_IO_PROG -fc "falloc 0 $filesize" $testfile >> $seqres.full + +$FIO_PROG $fio_aw_config >> $seqres.full +ret1=3D$? +$FIO_PROG $fio_verify_config >> $seqres.full +ret2=3D$? + +[[ $ret1 -eq 0 && $ret2 -eq 0 ]] || _fail "fio with atomic write failed" + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/generic/1226.out b/tests/generic/1226.out new file mode 100644 index 00000000..6dce0ea5 --- /dev/null +++ b/tests/generic/1226.out @@ -0,0 +1,2 @@ +QA output created by 1226 +Silence is golden --=20 2.49.0 From nobody Sun Sep 7 23:48:49 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8FD62C327C; Fri, 22 Aug 2025 08:02:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849763; cv=none; b=tK2w+n9yxhQyw7bOKJXe7GyRuwfG1YpcPkuvzpk+GGBmULwRoEuorqPgOR5JW5lDBG6aGuEKcmEC0w12Yam8ZTZNfzPoP/bART7MBaxVSZve99Gr/w/W5JFcF9TvL5VHX2T8FPHxw41htWA2ohVSBkIo0SmHP5XX0nTM/Bm5pso= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849763; c=relaxed/simple; bh=8CMIXGQU36RgTOG94vHlD8QeoCxA0yHAB4Z5AiKWrAY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ipC7FPmhe6rRThs0tCxvKXe7p7BnvmYD09EPXL4fO/LWMMDnSwiqoUIQrSmPn84uoDAmldNLkpcSZFZTaEVXDqWtxMz2trkLaOaRIiTp50TGbtPEsMvYW6OeCprGR/YVOx1Sc8Ml9aOPeVlKed9KB1h4CIFnIu2A4LzyVd6oxGs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=GyLEeS1c; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="GyLEeS1c" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57M7sieW012030; Fri, 22 Aug 2025 08:02:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=cQvp/JuP9aF8QNkil Z43A0YrULZxNlobH9EraofDX70=; b=GyLEeS1cV7wZLXaa/dSHNPeoTrUksrnTs +nrwEMxLyct5ILwocCQ9bofGldVJ5jV97rUw7TGZaEt4Aw3TgYK9nOX9y9W5NgKC kBR3sum9JnFe5vkdplW/NEgSyNvgjyiBL5jnb8lMx0vEMiP/aM/WzPIVxrwHpWfi nN/6zzil4TJTgY16+/tC3pPDbAmseU8kEZtFRshlQGOp4NpnsbizUanGcw8f7XgG amXKmYialDnaGQBLa/IqMnZbfy0Bhm0XjCaA0newQjBBQX4j71R5gKJ4Eu8u0nQQ IB3kRrSu4S210J7dLiVZsPR8xskcB/Eb9XxH0aXK35g0ziGCG5eJg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vw4dk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:34 +0000 (GMT) Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 57M7rf4Z005362; Fri, 22 Aug 2025 08:02:34 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vw4df-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:34 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57M4KUCp008726; Fri, 22 Aug 2025 08:02:33 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 48my42caej-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:33 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57M82VvY46334360 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Aug 2025 08:02:31 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6243220040; Fri, 22 Aug 2025 08:02:31 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E58CC2004B; Fri, 22 Aug 2025 08:02:28 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.210.10]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 Aug 2025 08:02:28 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v5 06/12] generic: Add atomic write test using fio verify on file mixed mappings Date: Fri, 22 Aug 2025 13:32:05 +0530 Message-ID: <7c508d4ecfa8ff1be5c031d1675e0b378c581028.1755849134.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODE5MDIyMiBTYWx0ZWRfX1qRvB1ZeUMhk cfXYovvDCMrPg1/fvLoNXUr9mfHjWOYqIfyISnYxrzbluHHPbuC52xa+4ne7YBiiwkUI3p7xsZK SK8G9NHP6nwt/09CqgePD3ChfON44xv3n28rgR/zbQ1ecMe722b2o28wHnJMAhB7HRhbigYr8PU krDiv4LH00Ymx/N+UthvtcUtGhKgXHypsoY1kt3b9VodG3HYBU+VCTktt6QbFUs0Ly6AfkoVRPu yoQUQe0VzBkBc0U9oyx6WrpxQT5aOSknWr1hG0WQmvv9m6ZnMdqoZV8hYUTfRVFmrnU+aMvKodB SLLEt4Qxs4Gbk91BkHUZyXVXzNoebk777UpiwTgZ2wW9M3aJTwhIUPlMrorE5hG6iFfLpAOqz+a KcRFUOiXXvFNgqXPpNcDqo5+PLOlWw== X-Proofpoint-ORIG-GUID: UVXTeVZnlpsZD66Iv7JYZTXG1qhGTbWD X-Authority-Analysis: v=2.4 cv=T9nVj/KQ c=1 sm=1 tr=0 ts=68a8241a cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=2OwXVqhp2XgA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=qMsZUPF6o8un45dIlBUA:9 a=cPQSjfK2_nFv0Q5t_7PE:22 X-Proofpoint-GUID: 0kFVMbE8BDgVBdpmILCNjPixcoR2XesK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-22_02,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 malwarescore=0 phishscore=0 spamscore=0 clxscore=1015 bulkscore=0 suspectscore=0 lowpriorityscore=0 priorityscore=1501 adultscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2508110000 definitions=main-2508190222 Content-Type: text/plain; charset="utf-8" This test uses fio to first create a file with mixed mappings. Then it does atomic writes using aio dio with parallel jobs to the same file with mixed mappings. Finally, we perform a fio verify step to ensure there is no data corruption or torn write. The aim is to stress the FS block allocation and extent handling logic to ensure it handles mixed mappings with RWF_ATOMIC correctly without tearing or losing data. Avoid doing overlapping parallel atomic writes because it might give unexpected results. Use offset_increment=3D, size=3D fio options to achieve this behavior. Co-developed-by: Ritesh Harjani (IBM) Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo Reviewed-by: John Garry --- tests/generic/1227 | 132 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1227.out | 2 + 2 files changed, 134 insertions(+) create mode 100755 tests/generic/1227 create mode 100644 tests/generic/1227.out diff --git a/tests/generic/1227 b/tests/generic/1227 new file mode 100755 index 00000000..a5ff9fca --- /dev/null +++ b/tests/generic/1227 @@ -0,0 +1,132 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1227 +# +# Validate FS atomic write using fio crc check verifier on mixed mappings +# of a file. +# +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto aio rw atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_odirect +_require_aio +_require_fio_version "3.38+" +_require_xfs_io_command "truncate" + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +touch "$SCRATCH_MNT/f1" +awu_min_write=3D$(_get_atomic_write_unit_min "$SCRATCH_MNT/f1") +awu_max_write=3D$(_get_atomic_write_unit_max "$SCRATCH_MNT/f1") + +aw_bsize=3D$(_max "$awu_min_write" "$((awu_max_write/4))") +fsbsize=3D$(_get_block_size $SCRATCH_MNT) + +threads=3D$(_min "$(($(nproc) * 2 * LOAD_FACTOR))" "100") +filesize=3D$((aw_bsize * threads * 100)) +depth=3D$threads +aw_io_size=3D$((filesize / threads)) +aw_io_inc=3D$aw_io_size +testfile=3D$SCRATCH_MNT/test-file + +fio_prep_config=3D$tmp.prep.fio +fio_aw_config=3D$tmp.aw.fio +fio_verify_config=3D$tmp.verify.fio +fio_out=3D$tmp.fio.out + +cat >$fio_prep_config <$fio_aw_config <$fio_verify_config <> $seqres.full +cat $fio_aw_config >> $seqres.full +cat $fio_verify_config >> $seqres.full + +$XFS_IO_PROG -fc "truncate $filesize" $testfile >> $seqres.full + +#prepare file with mixed mappings +$FIO_PROG $fio_prep_config >> $seqres.full + +# do atomic writes without verifying +$FIO_PROG $fio_aw_config >> $seqres.full + +# verify data is not torn +$FIO_PROG $fio_verify_config >> $seqres.full + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/generic/1227.out b/tests/generic/1227.out new file mode 100644 index 00000000..2605d062 --- /dev/null +++ b/tests/generic/1227.out @@ -0,0 +1,2 @@ +QA output created by 1227 +Silence is golden --=20 2.49.0 From nobody Sun Sep 7 23:48:49 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F64D2ECD35; Fri, 22 Aug 2025 08:02:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849767; cv=none; b=IwZGrVQUtb5kXlMLjLbY6ynl8aUsCYqxfEGjvY2YKgpgRcTJs10A1JJXHoWoOzzyN26s0OMxO5QIdcLOuHppKkiVTaSFOrsqxa8kPaI6jBhVrT1Q9rZPMR7GGgyFzNBTXIgyXQZCLnDPuOc2Nb2RQAf6foKxsDhu28qzcRfH7Z0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849767; c=relaxed/simple; bh=duzUlj0n0K27jnqJHj3/3MK0/1qL/ma7YHwUL9HIQgg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YbKLZHH+J4NNlLMTCQNuQ596TrZ0qO3eNF8tpezSasSO44+omDGJSks+NVuLMKvSpXOMp3+mWlh1VYfeJKDs37rSqSZUlfpA32kUbMot3wJp7ORHbRhlWULg49sljWxWBt+fr5p/bzBWQ/RfugCb3nmQu9MlSLW41XQ76XW87/8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=YjfIHH1s; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="YjfIHH1s" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57LIuaRb007983; Fri, 22 Aug 2025 08:02:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=sMpg0a048nM/v8MIy ZOuHjJkc8vOd5G56C4gl3yrYJQ=; b=YjfIHH1shKzMfDuI6ueL2hibOuPpxlX9m IkH+WvDFZVl1wR/yQ8RzjlauDtRpngbxWeby2sCOJiNsRj1pHvafpU1VXRFbQj87 bleWXVWXIb1219A6oMkwameZg65+svBf1Gzn2ECbLoC3Yjm9e3ATzwjKDe8Sux00 JBIiRh3c/Hgwi8qJO5iBe6Drr15lxSMz8Xo5W15Tb3iexSiUXdd3DXZO8K/iJ+YH MAE4oCIRAqmQL8o2bKuLwuNl6bXkROmMPnvrcyuXxOhypNh0UD7L97wsyVFxWgOW L/yRyCK8gAMDfph15uZFaGDY0Ke12CmCzvrC/f1FYC3GVdJYv47+w== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38w5419-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:37 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 57M81Bff014547; Fri, 22 Aug 2025 08:02:36 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38w540y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:36 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57M473Lw024299; Fri, 22 Aug 2025 08:02:35 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 48my43vahs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:35 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57M82Yik47579618 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Aug 2025 08:02:34 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 27ECF2004B; Fri, 22 Aug 2025 08:02:34 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BDB2F20040; Fri, 22 Aug 2025 08:02:31 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.210.10]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 Aug 2025 08:02:31 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v5 07/12] generic: Add atomic write multi-fsblock O_[D]SYNC tests Date: Fri, 22 Aug 2025 13:32:06 +0530 Message-ID: <55583acdfb6cb69bcea84a56cbc20754e1d2f4f4.1755849134.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: TcmE-7V-XyhBiC_3y35MDlU7mWm-deRg X-Proofpoint-GUID: ZAGid3oNachqK8YjOyOr06knXh3X5rk7 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODE5MDIyMiBTYWx0ZWRfX4R/hnF7kk/Gb mLlhBeDxiw+P7USqZx+Ivtk1OTjEb6n5WqtJmXVVRiTYxrMNU+3S/HsT3WrudYFW0esahdURKd8 2VuRf4Q50rxxPXobutN9csR86Qx7Cx1RCHDNR7Rc2TGIhiB9MWlN/KhUMBYjPhid22wbxM1KmjW TsS0mrF8B3olAxF9dKUSABo1AMW6YYKvuDhKvNLOMa01d7KRJeJvgN1i47vW2DA04X1ganY1dZt pqhl7+RHaSPgu8rlvjWJ7h41Y58/ABN4jyoOgrsi/69MnRYPXbS9NMK73hdjIdlA1z0bMJN30s7 FBR4eq0v1xMe/ZfbTVCy6HRn/BAjWqsAlZFq3AH8Gn/WdnRcZJeOa7Hwzw/46A2SOFhXw58NYfA h0DCirbcmzQ/W8RWEpMFrop5R8b10Q== X-Authority-Analysis: v=2.4 cv=a9dpNUSF c=1 sm=1 tr=0 ts=68a8241d cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=2OwXVqhp2XgA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=udGWdZ-Mnbz566dSa5EA:9 a=cPQSjfK2_nFv0Q5t_7PE:22 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-22_02,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 spamscore=0 suspectscore=0 adultscore=0 impostorscore=0 malwarescore=0 bulkscore=0 priorityscore=1501 clxscore=1015 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2508110000 definitions=main-2508190222 Content-Type: text/plain; charset="utf-8" This adds various atomic write multi-fsblock stresst tests with mixed mappings and O_SYNC, to ensure the data and metadata is atomically persisted even if there is a shutdown. Suggested-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo Reviewed-by: John Garry --- tests/generic/1228 | 137 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1228.out | 2 + 2 files changed, 139 insertions(+) create mode 100755 tests/generic/1228 create mode 100644 tests/generic/1228.out diff --git a/tests/generic/1228 b/tests/generic/1228 new file mode 100755 index 00000000..888599ce --- /dev/null +++ b/tests/generic/1228 @@ -0,0 +1,137 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1228 +# +# Atomic write multi-fsblock data integrity tests with mixed mappings +# and O_SYNC +# +. ./common/preamble +. ./common/atomicwrites +_begin_fstest auto quick rw atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_atomic_write_test_commands +_require_scratch_shutdown +_require_xfs_io_command "truncate" + +_scratch_mkfs >> $seqres.full +_scratch_mount >> $seqres.full + +check_data_integrity() { + actual=3D$(_hexdump $testfile) + if [[ "$expected" !=3D "$actual" ]] + then + echo "Integrity check failed" + echo "Integrity check failed" >> $seqres.full + echo "# Expected file contents:" >> $seqres.full + echo "$expected" >> $seqres.full + echo "# Actual file contents:" >> $seqres.full + echo "$actual" >> $seqres.full + + _fail "Data integrity check failed. The atomic write was torn." + fi +} + +prep_mixed_mapping() { + $XFS_IO_PROG -c "truncate 0" $testfile >> $seqres.full + local off=3D0 + local mapping=3D"" + + local operations=3D("W" "H" "U") + local num_blocks=3D$((awu_max / blksz)) + for ((i=3D0; i /d= ev/null + ;; + "H") + # No operation needed for hole + ;; + "U") + $XFS_IO_PROG -c "falloc $off $blksz" $testfile >> /dev/null + ;; + esac + off=3D$((off + blksz)) + done + + echo "+ + Mixed mapping prep done. Full mapping pattern: $mapping" >> $se= qres.full + + sync $testfile +} + +verify_atomic_write() { + test $bytes_written -eq $awu_max || _fail "atomic write len=3D$awu_max as= sertion failed" + check_data_integrity +} + +mixed_mapping_test() { + prep_mixed_mapping + + echo -"+ + Performing O_DSYNC atomic write from 0 to $awu_max" >> $seqres= .full + if [[ "$1" =3D=3D "shutdown" ]] + then + bytes_written=3D$($XFS_IO_PROG -x -dc \ + "pwrite -DA -V1 -b $awu_max 0 $awu_max" \ + -c "shutdown" $testfile | grep wrote | \ + awk -F'[/ ]' '{print $2}') + _scratch_cycle_mount >>$seqres.full 2>&1 || _fail "remount failed" + else + bytes_written=3D$($XFS_IO_PROG -dc \ + "pwrite -DA -V1 -b $awu_max 0 $awu_max" $testfile | \ + grep wrote | awk -F'[/ ]' '{print $2}') + fi + + verify_atomic_write +} + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile + +awu_max=3D$(_get_atomic_write_unit_max $testfile) +blksz=3D$(_get_block_size $SCRATCH_MNT) + +# Create an expected pattern to compare with +$XFS_IO_PROG -tc "pwrite -b $awu_max 0 $awu_max" $testfile >> $seqres.full +expected=3D$(_hexdump $testfile) +echo "# Expected file contents:" >> $seqres.full +echo "$expected" >> $seqres.full +echo >> $seqres.full + +echo "# Test 1: Do O_DSYNC atomic write on random mixed mapping:" >> $seqr= es.full +echo >> $seqres.full +for ((iteration=3D1; iteration<=3D10; iteration++)); do + echo "=3D=3D=3D Mixed Mapping Test Iteration $iteration =3D=3D=3D" >> $se= qres.full + + echo "+ Testing without shutdown..." >> $seqres.full + mixed_mapping_test + echo "Passed!" >> $seqres.full + + echo "+ Testing with sudden shutdown..." >> $seqres.full + mixed_mapping_test "shutdown" + echo "Passed!" >> $seqres.full + + echo "Iteration $iteration completed: OK" >> $seqres.full + echo >> $seqres.full +done +echo "# Test 1: Do O_SYNC atomic write on random mixed mapping (10 iterati= ons): OK" >> $seqres.full + + +echo >> $seqres.full +echo "# Test 2: Do extending O_SYNC atomic writes: " >> $seqres.full +bytes_written=3D$($XFS_IO_PROG -x -dstc "pwrite -A -V1 -b $awu_max 0 $awu_= max" \ + -c "shutdown" $testfile | grep wrote | awk -F'[/ ]' '{print $2}') +_scratch_cycle_mount >>$seqres.full 2>&1 || _fail "remount failed" +verify_atomic_write +echo "# Test 2: Do extending O_SYNC atomic writes: OK" >> $seqres.full + +# success, all done +echo "Silence is golden" +status=3D0 +exit + diff --git a/tests/generic/1228.out b/tests/generic/1228.out new file mode 100644 index 00000000..1baffa91 --- /dev/null +++ b/tests/generic/1228.out @@ -0,0 +1,2 @@ +QA output created by 1228 +Silence is golden --=20 2.49.0 From nobody Sun Sep 7 23:48:49 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 741572EACE1; Fri, 22 Aug 2025 08:02:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849769; cv=none; b=pto1RKDhBe2e2UOg80sh7S8dNphTpiLswEyg9ld7M8tuw6qc0sHOYMyYoRsXYkwd4IZQPdapnesmIorRVqD+vB8TeHhS6SVkl6ImVEa/OdMDtkJP1oFE+NTNpUgqTbbet1a7DVhaHsl+c7BsqPGlfrrChDqmn8oKYiUs9p9Ap/0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849769; c=relaxed/simple; bh=RxPrTaQAs9qgU+jIKC0eIPg2SC3HuaGHuhdMAdcX158=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=f4jr4+oC0y5hxfQwQ6tXkScMBXaLvm11baZFwX2XvEz1IpUb5pYRRYuzzB+HfkvgguhMrwv1Sq39wwp3/I15rNnWVP1s8GQux8zkm2Q1D4G2DpLwQ9jD8n1rlNAETTVurtmTOPctIVQtq9BYgfqHvzn0glHHFZDC/dWoAgh87j8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=pRQYB1Jq; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="pRQYB1Jq" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57M7cDCg003075; Fri, 22 Aug 2025 08:02:40 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=U0xLYgn5+zfKzcdPT vows24RLUS6/k6+Ygwt8UBcx3A=; b=pRQYB1Jqh5BCs27irvY51EWIZ72Ly7pf8 STTI33xDqx+5f/roDeSQK+/voLIhrpDsj+HKhCoCXCFM7jj8shb83p7vmE1kaNk2 zN6/+2qhqeASpKcsLA8d6XGCbqjMn7jYA/qXb/YgQocjczsaseoZwO3pPrwi/6qA AvyJHr3ebnzRv+MG2Kko76fNN+kmaO4xymsX/X71+U8q6Iaj7Fm4qc867nj14j8u O4SfpOMVNX4GJY4QBx9O0drZzDlAaz7PEADEUUSMXzLXnWvvGVJ94jjCVAkoDFcR PK6q7TBKExrH8TRdvrdW30ZcJEzHMeWdgX/efhwmy2JwqZoDP/WWQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38wd32r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:39 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 57M7v4Af030742; Fri, 22 Aug 2025 08:02:39 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38wd32p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:39 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57M7hfte024279; Fri, 22 Aug 2025 08:02:38 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 48my43vaj0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:38 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57M82axU55443768 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Aug 2025 08:02:36 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id ADF532004E; Fri, 22 Aug 2025 08:02:36 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7FE9120040; Fri, 22 Aug 2025 08:02:34 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.210.10]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 Aug 2025 08:02:34 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v5 08/12] generic: Stress fsx with atomic writes enabled Date: Fri, 22 Aug 2025 13:32:07 +0530 Message-ID: <6d0de75776499a6751ccf12d2c3a1f059396b631.1755849134.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODE5MDIyMiBTYWx0ZWRfX/orhR4pHhBNC Q0F2PZDpOcxvqeGxyOkpvxgw4fdXATasAnNbnL1Ie9IWBVehyE48hM4tCvWmHbpXhOz4J/dDXS+ sxGvA+FdUEbUI7BP8W9ZUhWfdacXopu2udT3haNWXCX7LlF7rMUtfwc2EWp6nbkhamlNV89cF6Q lA4RfXgNXTu4twwX/OdAcflRaAe89Lw03dCpvzimZHQHQgudkStFVSwq3vRFIrplq06SgLb4AZk SapuTZWz1LPmW4ABkpv8I+YADgQhRoFYm3FBFRTJ2YVrdM7SuR0Rfz5hStllUd9wdIbuWFQDjh8 2lMNplSLKAF1ira9oGzz2RfVaj+xSTEmIY4LD0aYHXK9CR9S4vDUIehwQa6NkBK+TJUWQhpfeJl AKUI4vthl6/bKmk3mm9QpBtbqY4BKw== X-Authority-Analysis: v=2.4 cv=H62CA+Yi c=1 sm=1 tr=0 ts=68a8241f cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=2OwXVqhp2XgA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=zPW1cLLGUcrnTIp6xCMA:9 a=cPQSjfK2_nFv0Q5t_7PE:22 X-Proofpoint-ORIG-GUID: _bE4VOMz1WfjTP9FBajJj9dl5XLTStgq X-Proofpoint-GUID: Ox_AOXjZOX-9tYmeV3ZeUBQW7oV0bQqv X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-22_02,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 suspectscore=0 bulkscore=0 adultscore=0 phishscore=0 priorityscore=1501 malwarescore=0 spamscore=0 impostorscore=0 lowpriorityscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2508110000 definitions=main-2508190222 Content-Type: text/plain; charset="utf-8" Stress file with atomic writes to ensure we excercise codepaths where we are mixing different FS operations with atomic writes Suggested-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo Reviewed-by: John Garry --- tests/generic/1229 | 68 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/1229.out | 2 ++ 2 files changed, 70 insertions(+) create mode 100755 tests/generic/1229 create mode 100644 tests/generic/1229.out diff --git a/tests/generic/1229 b/tests/generic/1229 new file mode 100755 index 00000000..7fa57105 --- /dev/null +++ b/tests/generic/1229 @@ -0,0 +1,68 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1229 +# +# fuzz fsx with atomic writes +# +. ./common/preamble +. ./common/atomicwrites +_begin_fstest rw auto quick atomicwrites + +_require_odirect +_require_scratch_write_atomic + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount >> $seqres.full 2>&1 + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile + +awu_max=3D$(_get_atomic_write_unit_max $testfile) +blksz=3D$(_get_block_size $SCRATCH_MNT) +bsize=3D`$here/src/min_dio_alignment $SCRATCH_MNT $SCRATCH_DEV` + +set_fsx_avoid() { + local file=3D$1 + + case "$FSTYP" in + "ext4") + local dev=3D$(findmnt -n -o SOURCE --target $testfile) + + # fsx insert/collpase range support for ext4+bigalloc is + # currently broken, so disable it. Also disable incase we can't + # detect bigalloc to be on safer side. + if [ -z "$DUMPE2FS_PROG" ]; then + echo "dumpe2fs not found, disabling insert/collapse range" >> $seqres.f= ull + FSX_AVOID+=3D" -I -C" + return + fi + + $DUMPE2FS_PROG -h $dev 2>&1 | grep -q bigalloc && { + echo "fsx insert/collapse range not supported with bigalloc. Disabling.= ." >> $seqres.full + FSX_AVOID+=3D" -I -C" + } + ;; + *) + ;; + esac +} + +# fsx usage: +# +# -N numops: total # operations to do +# -l flen: the upper bound on file size +# -o oplen: the upper bound on operation size (64k default) +# -Z: O_DIRECT () + +set_fsx_avoid +_run_fsx_on_file $testfile -N 10000 -o $awu_max -A -l 500000 -r $bsize -w = $bsize -Z $FSX_AVOID >> $seqres.full +if [[ "$?" !=3D "0" ]] +then + _fail "fsx returned error: $?" +fi + +echo "Silence is golden" +status=3D0 +exit diff --git a/tests/generic/1229.out b/tests/generic/1229.out new file mode 100644 index 00000000..737d61c6 --- /dev/null +++ b/tests/generic/1229.out @@ -0,0 +1,2 @@ +QA output created by 1229 +Silence is golden --=20 2.49.0 From nobody Sun Sep 7 23:48:49 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B30EC2EDD78; Fri, 22 Aug 2025 08:02:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849773; cv=none; b=bMkwTRcUwtd9qRr+h/2aDs3XLmhQiZHSvn3Qns5650dkkzo4T82vt092C9s3BjLsrOn6AG059oAPWm4+zBzyogU46jKsDJby2/GWN7cr44Ldw5AYeuKhXjrQc/uwDTq84uBUNc5ZCV8e0UQiHHiE0jyei8JJ9+6VAFBod8FQMag= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849773; c=relaxed/simple; bh=5Whlb3M258/hYH1QZfI08fWCJYDAzM4q+pN3fIkE6zg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JDRb5y0G8J3QUUmHP1ipSJIKmDHOuuJ9+xyLKVqwNmJXOUgxLbnnDxcNf96Nb1PxIlSpK4bT+jfT98wtVrWT60OFxNcoVau0ozeb4aEpOmidvm40fTFk1IW+ITbA3c6NRwnWYcu90a6Km1eqoc4UBTX+Rm6jICWCvnGnHp5VQbY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=EZtN/3SD; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="EZtN/3SD" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57LIsLYl017631; Fri, 22 Aug 2025 08:02:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=K/X0pllH+N+VUDutZ XHOgTw9gsmcrn73ZXWsZBojlR0=; b=EZtN/3SDICFeMwlRUWXWJbBeDpW1ryAOW yidPKDcfZ4jcB5C0E9Zr09H65bRIRl8sbk/OMH8PTk6CTx6TPAnokiBYW4O4yrKb Fo8PYe2ggSmlhOJMMgn4aAY7ShCw3gzivAmBoUjYEhCeTYWe2InlEprIKaYnaCp8 Na9x1mbqFzqmRvuqGkvh+fjw4Z7vTdvs45kdXRNQFAvNhtSMaXnVJ3CIqVNqWJZm fejKh4CO+v0MT9JJtryaP8P66UnCd91yjfahxN7STX9om3Q58fJt1+chmbQ1QYlY 82HZXK3OInhnGEqXpvWFmYWt7epSStpbLekqY60KoA1uhzHHHMNZQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vw9eq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:42 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 57M81iUh004049; Fri, 22 Aug 2025 08:02:42 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vw9em-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:42 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57M3hKNC031902; Fri, 22 Aug 2025 08:02:41 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 48my5ycahv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:41 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57M82dHc31851004 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Aug 2025 08:02:39 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A9C0C2004B; Fri, 22 Aug 2025 08:02:39 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3A1BE20040; Fri, 22 Aug 2025 08:02:37 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.210.10]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 Aug 2025 08:02:36 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v5 09/12] generic: Add sudden shutdown tests for multi block atomic writes Date: Fri, 22 Aug 2025 13:32:08 +0530 Message-ID: <2b523de842ada3ac7759cedae80485ae201d7e5d.1755849134.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: g4cgeJoYBLSLclWQ3D0FY5BvqEmIueOQ X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODE5MDIyMiBTYWx0ZWRfX7UhP28ehQHmd a9tQuNv78hDx1HX2DZXHCxxZqGeN38HZFsRk87oFo7qGjjmsPhdcG/sY9utl8fLsDAtHsxNiG7o nr97lIRtgNgO9jPSYZ4A3hgOMLdNiUOUtRp8BHv3H+lqLMyaSRMew1ID44QLD6K4W7w3+pQlYmj 2VVAqbpBPYLG68pfwBXA+XfqfZcwfSAJhTadKXBC6VAdcXu0dDvEU3D9A3uhQlKHRwACL6IPT0l fgT6P026DBqMZ0FwPe+2RnqWPDrfTA8GaanRpvED9XaQW7BA39PI6Ar2W7e8liPHs60yAid+Z8Y 7vztWoqpqPyIo93uVOPSgKAzQERQjeUSH5U0Aqb2DhZwbAki6NQZLUt3mIXWf45a/tv+qWBsdo/ sQP+Qn8E9QIVeM0Wojg2M08whXvUyw== X-Proofpoint-GUID: PyMOLHLDRTMwOG2ksl_s5jIRw9q6LeSn X-Authority-Analysis: v=2.4 cv=KPwDzFFo c=1 sm=1 tr=0 ts=68a82423 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=2OwXVqhp2XgA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=jzLDx6uKbS-OZFe_CzQA:9 a=cPQSjfK2_nFv0Q5t_7PE:22 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-22_02,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 spamscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 bulkscore=0 priorityscore=1501 suspectscore=0 phishscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2508110000 definitions=main-2508190222 Content-Type: text/plain; charset="utf-8" This test is intended to ensure that multi blocks atomic writes maintain atomic guarantees across sudden FS shutdowns. The way we work is that we lay out a file with random mix of written, unwritten and hole extents. Then we start performing atomic writes sequentially on the file while we parallely shutdown the FS. Then we note the last offset where the atomic write happened just before shut down and then make sure blocks around it either have completely old data or completely new data, ie the write was not torn during shutdown. We repeat the same with completely written, completely unwritten and comple= tely empty file to ensure these cases are not torn either. Finally, we have a similar test for append atomic writes Suggested-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo --- tests/generic/1230 | 397 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1230.out | 2 + 2 files changed, 399 insertions(+) create mode 100755 tests/generic/1230 create mode 100644 tests/generic/1230.out diff --git a/tests/generic/1230 b/tests/generic/1230 new file mode 100755 index 00000000..cff5adc0 --- /dev/null +++ b/tests/generic/1230 @@ -0,0 +1,397 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test No. 1230 +# +# Test multi block atomic writes with sudden FS shutdowns to ensure +# the FS is not tearing the write operation +. ./common/preamble +. ./common/atomicwrites +_begin_fstest auto atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_atomic_write_test_commands +_require_scratch_shutdown +_require_xfs_io_command "truncate" + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount >> $seqres.full + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile + +awu_max=3D$(_get_atomic_write_unit_max $testfile) +blksz=3D$(_get_block_size $SCRATCH_MNT) +echo "Awu max: $awu_max" >> $seqres.full + +num_blocks=3D$((awu_max / blksz)) +# keep initial value high for dry run. This will be +# tweaked in dry_run() based on device write speed. +filesize=3D$(( 10 * 1024 * 1024 * 1024 )) + +_cleanup() { + [ -n "$awloop_pid" ] && kill $awloop_pid &> /dev/null + wait +} + +atomic_write_loop() { + local off=3D0 + local size=3D$awu_max + for ((i=3D0; i<$((filesize / $size )); i++)); do + # Due to sudden shutdown this can produce errors so just + # redirect them to seqres.full + $XFS_IO_PROG -c "open -fsd $testfile" -c "pwrite -S 0x61 -DA -V1 -b $siz= e $off $size" >> /dev/null 2>>$seqres.full + echo "Written to offset: $off" >> $tmp.aw + off=3D$((off + $size)) + done +} + +# This test has the following flow: +# 1. Start doing sequential atomic writes in bg, upto $filesize +# 2. Sleep for 0.2s and shutdown the FS +# 3. kill the atomic write process +# 4. verify the writes were not torn +# +# We ideally want the shutdown to happen while an atomic write is ongoing +# but this gets tricky since faster devices can actually finish the whole +# atomic write loop before sleep 0.2s completes, resulting in the shutdown +# happening after the write loop which is not what we want. A simple solut= ion +# to this is to increase $filesize so step 1 takes long enough but a big +# $filesize leads to create_mixed_mappings() taking very long, which is not +# ideal. +# +# Hence, use the dry_run function to figure out the rough device speed and= set +# $filesize accordingly. +dry_run() { + echo >> $seqres.full + echo "# Estimating ideal filesize..." >> $seqres.full + atomic_write_loop & + awloop_pid=3D$! + + local i=3D0 + # Wait for atleast first write to be recorded or 10s + while [ ! -f "$tmp.aw" -a $i -le 50 ]; do i=3D$((i + 1)); sleep 0.2; done + + if [[ $i -gt 50 ]] + then + _fail "atomic write process took too long to start" + fi + + echo >> $seqres.full + echo "# Shutting down filesystem while write is running" >> $seqres.full + _scratch_shutdown + + kill $awloop_pid 2>/dev/null # the process might have finished already + wait $awloop_pid + unset $awloop_pid + + bytes_written=3D$(tail -n 1 $tmp.aw | cut -d" " -f4) + echo "# Bytes written in 0.2s: $bytes_written" >> $seqres.full + + filesize=3D$((bytes_written * 3)) + echo "# Setting \$filesize=3D$filesize" >> $seqres.full + + rm $tmp.aw + sleep 0.5 + + _scratch_cycle_mount + +} + +create_mixed_mappings() { + local file=3D$1 + local size_bytes=3D$2 + + echo "# Filling file $file with alternate mappings till size $size_bytes"= >> $seqres.full + #Fill the file with alternate written and unwritten blocks + local off=3D0 + local operations=3D("W" "U") + + for ((i=3D0; i<$((size_bytes / blksz )); i++)); do + index=3D$(($i % ${#operations[@]})) + map=3D"${operations[$index]}" + + case "$map" in + "W") + $XFS_IO_PROG -fc "pwrite -b $blksz $off $blksz" $file >> /dev/null + ;; + "U") + $XFS_IO_PROG -fc "falloc $off $blksz" $file >> /dev/null + ;; + esac + off=3D$((off + blksz)) + done + + sync $file +} + +populate_expected_data() { + # create a dummy file with expected old data for different cases + create_mixed_mappings $testfile.exp_old_mixed $awu_max + expected_data_old_mixed=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_o= ld_mixed) + + $XFS_IO_PROG -fc "falloc 0 $awu_max" $testfile.exp_old_zeroes >> $seqres.= full + expected_data_old_zeroes=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_= old_zeroes) + + $XFS_IO_PROG -fc "pwrite -b $awu_max 0 $awu_max" $testfile.exp_old_mapped= >> $seqres.full + expected_data_old_mapped=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_= old_mapped) + + # create a dummy file with expected new data + $XFS_IO_PROG -fc "pwrite -S 0x61 -b $awu_max 0 $awu_max" $testfile.exp_ne= w >> $seqres.full + expected_data_new=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_new) +} + +verify_data_blocks() { + local verify_start=3D$1 + local verify_end=3D$2 + local expected_data_old=3D"$3" + local expected_data_new=3D"$4" + + echo >> $seqres.full + echo "# Checking data integrity from $verify_start to $verify_end" >> $se= qres.full + + # After an atomic write, for every chunk we ensure that the underlying + # data is either the old data or new data as writes shouldn't get torn. + local off=3D$verify_start + while [[ "$off" -lt "$verify_end" ]] + do + #actual_data=3D$(xxd -s $off -l $awu_max -p $testfile) + actual_data=3D$(od -An -t x1 -j $off -N $awu_max $testfile) + if [[ "$actual_data" !=3D "$expected_data_new" ]] && [[ "$actual_data" != =3D "$expected_data_old" ]] + then + echo "Checksum match failed at off: $off size: $awu_max" + echo "Expected contents: (Either of the 2 below):" + echo + echo "Expected old: " + echo "$expected_data_old" + echo + echo "Expected new: " + echo "$expected_data_new" + echo + echo "Actual contents: " + echo "$actual_data" + + _fail + fi + echo -n "Check at offset $off suceeded! " >> $seqres.full + if [[ "$actual_data" =3D=3D "$expected_data_new" ]] + then + echo "matched new" >> $seqres.full + elif [[ "$actual_data" =3D=3D "$expected_data_old" ]] + then + echo "matched old" >> $seqres.full + fi + off=3D$(( off + awu_max )) + done +} + +# test data integrity for file by shutting down in between atomic writes +test_data_integrity() { + echo >> $seqres.full + echo "# Writing atomically to file in background" >> $seqres.full + atomic_write_loop & + awloop_pid=3D$! + + local i=3D0 + # Wait for atleast first write to be recorded or 10s + while [ ! -f "$tmp.aw" -a $i -le 50 ]; do i=3D$((i + 1)); sleep 0.2; done + + if [[ $i -gt 50 ]] + then + _fail "atomic write process took too long to start" + fi + + echo >> $seqres.full + echo "# Shutting down filesystem while write is running" >> $seqres.full + _scratch_shutdown + + kill $awloop_pid 2>/dev/null # the process might have finished already + wait $awloop_pid + unset $awloop_pid + + last_offset=3D$(tail -n 1 $tmp.aw | cut -d" " -f4) + if [[ -z $last_offset ]] + then + last_offset=3D0 + fi + + echo >> $seqres.full + echo "# Last offset of atomic write: $last_offset" >> $seqres.full + + rm $tmp.aw + sleep 0.5 + + _scratch_cycle_mount + + # we want to verify all blocks around which the shutdown happended + verify_start=3D$(( last_offset - (awu_max * 5))) + if [[ $verify_start < 0 ]] + then + verify_start=3D0 + fi + + verify_end=3D$(( last_offset + (awu_max * 5))) + if [[ "$verify_end" -gt "$filesize" ]] + then + verify_end=3D$filesize + fi +} + +# test data integrity for file wiht written and unwritten mappings +test_data_integrity_mixed() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with mixed mappings" >> $seqres.full + create_mixed_mappings $testfile $filesize + + test_data_integrity + + verify_data_blocks $verify_start $verify_end "$expected_data_old_mixed" "= $expected_data_new" +} + +# test data integrity for file with completely written mappings +test_data_integrity_writ() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with fully written mapping" >> $seqres.full + $XFS_IO_PROG -c "pwrite -b $filesize 0 $filesize" $testfile >> $seqres.fu= ll + sync $testfile + + test_data_integrity + + verify_data_blocks $verify_start $verify_end "$expected_data_old_mapped" = "$expected_data_new" +} + +# test data integrity for file with completely unwritten mappings +test_data_integrity_unwrit() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with fully unwritten mappings" >> $seqres.full + $XFS_IO_PROG -c "falloc 0 $filesize" $testfile >> $seqres.full + sync $testfile + + test_data_integrity + + verify_data_blocks $verify_start $verify_end "$expected_data_old_zeroes" = "$expected_data_new" +} + +# test data integrity for file with no mappings +test_data_integrity_hole() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with no mappings" >> $seqres.full + $XFS_IO_PROG -c "truncate $filesize" $testfile >> $seqres.full + sync $testfile + + test_data_integrity + + verify_data_blocks $verify_start $verify_end "$expected_data_old_zeroes" = "$expected_data_new" +} + +test_filesize_integrity() { + $XFS_IO_PROG -c "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Performing extending atomic writes over file in background" >> $s= eqres.full + atomic_write_loop & + awloop_pid=3D$! + + local i=3D0 + # Wait for atleast first write to be recorded or 10s + while [ ! -f "$tmp.aw" -a $i -le 50 ]; do i=3D$((i + 1)); sleep 0.2; done + + if [[ $i -gt 50 ]] + then + _fail "atomic write process took too long to start" + fi + + echo >> $seqres.full + echo "# Shutting down filesystem while write is running" >> $seqres.full + _scratch_shutdown + + kill $awloop_pid 2>/dev/null # the process might have finished already + wait $awloop_pid + unset $awloop_pid + + local last_offset=3D$(tail -n 1 $tmp.aw | cut -d" " -f4) + if [[ -z $last_offset ]] + then + last_offset=3D0 + fi + + echo >> $seqres.full + echo "# Last offset of atomic write: $last_offset" >> $seqres.full + rm $tmp.aw + sleep 0.5 + + _scratch_cycle_mount + local filesize=3D$(_get_filesize $testfile) + echo >> $seqres.full + echo "# Filesize after shutdown: $filesize" >> $seqres.full + + # To confirm that the write went atomically, we check: + # 1. The last block should be a multiple of awu_max + # 2. The last block should be the completely new data + + if (( $filesize % $awu_max )) + then + echo "Filesize after shutdown ($filesize) not a multiple of atomic write= unit ($awu_max)" + fi + + verify_start=3D$(( filesize - (awu_max * 5))) + if [[ $verify_start < 0 ]] + then + verify_start=3D0 + fi + + local verify_end=3D$filesize + + # Here the blocks should always match new data hence, for simplicity of + # code, just corrupt the $expected_data_old buffer so it never matches + local expected_data_old=3D"POISON" + verify_data_blocks $verify_start $verify_end "$expected_data_old" "$expec= ted_data_new" +} + +$XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + +dry_run + +echo >> $seqres.full +echo "# Populating expected data buffers" >> $seqres.full +populate_expected_data + +# Loop 20 times to shake out any races due to shutdown +for ((iter=3D0; iter<20; iter++)) +do + echo >> $seqres.full + echo "------ Iteration $iter ------" >> $seqres.full + + echo >> $seqres.full + echo "# Starting data integrity test for atomic writes over mixed mapping= " >> $seqres.full + test_data_integrity_mixed + + echo >> $seqres.full + echo "# Starting data integrity test for atomic writes over fully written= mapping" >> $seqres.full + test_data_integrity_writ + + echo >> $seqres.full + echo "# Starting data integrity test for atomic writes over fully unwritt= en mapping" >> $seqres.full + test_data_integrity_unwrit + + echo >> $seqres.full + echo "# Starting data integrity test for atomic writes over holes" >> $se= qres.full + test_data_integrity_hole + + echo >> $seqres.full + echo "# Starting filesize integrity test for atomic writes" >> $seqres.fu= ll + test_filesize_integrity +done + +echo "Silence is golden" +status=3D0 +exit diff --git a/tests/generic/1230.out b/tests/generic/1230.out new file mode 100644 index 00000000..d01f54ea --- /dev/null +++ b/tests/generic/1230.out @@ -0,0 +1,2 @@ +QA output created by 1230 +Silence is golden --=20 2.49.0 From nobody Sun Sep 7 23:48:49 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9CD2D2EACE1; Fri, 22 Aug 2025 08:02:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849775; cv=none; b=AtHuTCuEB1Vaw7ID9+drya1WuannTJsD2Xv5pg7ZajahnZR3Kr8zKWcrg1uRqCsNIJQR/A7YaCOPTgffFB87M4RUrnnFu5V3VTWqUAcMqr59uRUdfgsXUAQ5//6T07eQ/t1Td6gUCgvrN5pMX0eWIifs/HXvTBY+0xo7NZvYkbI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849775; c=relaxed/simple; bh=g3w21KjqeY+mdjFwZ9JmSJ54JRxkKppYSjcHP7SC0QM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SOaU8xYasYZP6AWGP3znHmZJMcPGxR2u/V2f8EIMyQb5eXFT62eoCabCl0RiZbZHhzw7lipUkDF9LLb72rtCOR/5AfSXqf6cj/CX+IeCKyH1ALEplD81Wsw2whBWvqgc1MFJcLqrPVnRpOXqz4GVFW3NwRlZuLpwppN6wYz1e44= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Q3smdvy4; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Q3smdvy4" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57M7UCLG017413; Fri, 22 Aug 2025 08:02:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=rKMKSgqvLMQ8HeyFG FDgfgVrcaGypQx2FC//nyy1y68=; b=Q3smdvy40lL++Zpvc4SHOBK0zDEwZGnMI UpTq/xSrOpznrPPTMpXm5Yvaekcaa53ODOy+SUKV3g2Tf+JR08CYFaZZBd8qpdR5 i5qNeQysEeH1gcyE53H9KR28U3ejAX9Ny+zciwWDLJMS/qG8kgRs1MmQW7RMW7zN nwX5Zs8LLK0d1b0rv/iMKWEXNfxpr04zmXXtlx+tJDj6yABbUrri91TbNdMSZtjQ rs4YWIxP3ftHaWYjdQU9XJjIFLWisbIdvri+tdC7PNmWKe6l0jSbGmsj6bDO23lw WK37vsJLwQMeSFW6va1W2Qvllw2V2/5C+wakK8XOVbP4ZvVXfZ+Yw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vw9ex-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:46 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 57M82jGC006388; Fri, 22 Aug 2025 08:02:45 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vw9et-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:45 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57M7hftg024279; Fri, 22 Aug 2025 08:02:44 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 48my43vajc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:44 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57M82g1G20709790 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Aug 2025 08:02:42 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BB9F820040; Fri, 22 Aug 2025 08:02:42 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4AA9920043; Fri, 22 Aug 2025 08:02:40 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.210.10]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 Aug 2025 08:02:39 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v5 10/12] ext4: test atomic write and ioend codepaths with bigalloc Date: Fri, 22 Aug 2025 13:32:09 +0530 Message-ID: <5a39bfbbd73f8598e9f85fb4420955c8a95c78a2.1755849134.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: c7Eefrw8S_dwePmhjRpFHSCWG3n-NJz- X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODE5MDIyMiBTYWx0ZWRfX+a3cDKvr6wLs ey7/YSEANsAhRDBprsluqxbWhkwM/CHsUfPiYYoAapTp2QbFRXyJuDZ8BHmP4zVsK6LO2H82EcK KDrJeogC+LilbP5u3+kvQronSxMYCnNbYucEM8F/stca2oiaAKzRSGCq6+s6LI/3dMbE/aiLzzH fMmJaVWpXJEFHBSY9nFRDCQS+wptLdlMUpXPkaeMD0CMOBKaxIggEIS3vqvCjigKu4HI8pUeia1 swb1gecESMydfPospncp1b5FfFgC4k3PXMvKqfXUiVAM7HWw6Xc4juelwmfJEzPr04cne3ib9SW TnJpJQmPrxIdlTORMilYGqn+jgyMBrtzFAeVoDmy32hBEIdrue6AKfm34HMKJ5/XUvx/42zZepQ HuJytDPXUA8Flvn/OGm+8vFzszrQWA== X-Proofpoint-GUID: ydLOqbuDgKa7FgZxA-RBrwHg8R3WzikW X-Authority-Analysis: v=2.4 cv=KPwDzFFo c=1 sm=1 tr=0 ts=68a82426 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=2OwXVqhp2XgA:10 a=pGLkceISAAAA:8 a=VnNF1IyMAAAA:8 a=KekteOT-tD_VQUBAREkA:9 a=cPQSjfK2_nFv0Q5t_7PE:22 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-22_02,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 spamscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 bulkscore=0 priorityscore=1501 suspectscore=0 phishscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2508110000 definitions=main-2508190222 Content-Type: text/plain; charset="utf-8" From: "Ritesh Harjani (IBM)" This test does a lot of parallel RWF_ATOMIC IO on a preallocated file to stress the write and end-io unwritten conversion code paths. We brute force this for different blocksize and clustersizes and after each iteration we ensure the data was not torn or corrupted using fio crc verification. Note that in this test we use overlapping atomic writes of same io size. Although serializing racing writes is not guaranteed for RWF_ATOMIC, NVMe and SCSI provide this guarantee as an inseparable feature to power-fail atomicity. Keeping the iosize as same also ensures that ext4 doesn't tear the write due to racing ioend unwritten conversion. The value of this test is that we make sure the RWF_ATOMIC is handled correctly by ext4 as well as test that the block layer doesn't split or only generate multiple bios for an atomic write. Signed-off-by: Ritesh Harjani (IBM) Signed-off-by: Ojaswin Mujoo Reviewed-by: "Darrick J. Wong" Reviewed-by: John Garry --- tests/ext4/061 | 155 +++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/061.out | 2 + 2 files changed, 157 insertions(+) create mode 100755 tests/ext4/061 create mode 100644 tests/ext4/061.out diff --git a/tests/ext4/061 b/tests/ext4/061 new file mode 100755 index 00000000..0ccf9f69 --- /dev/null +++ b/tests/ext4/061 @@ -0,0 +1,155 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 061 +# +# This test does a lot of parallel RWF_ATOMIC IO on a preallocated file to +# stress the write and end-io unwritten conversion code paths. We brute fo= rce +# this for all possible blocksize and clustersizes and after each iteratio= n we +# ensure the data was not torn or corrupted using fio crc verification. +# +# Note that in this test we use overlapping atomic writes of same io size. +# Although serializing racing writes is not guaranteed for RWF_ATOMIC, NVM= e and +# SCSI provide this guarantee as an inseparable feature to power-fail +# atomicity. Keeping the iosize as same also ensures that ext4 doesn't tea= r the +# write due to racing ioend unwritten conversion. +# +# The value of this test is that we make sure the RWF_ATOMIC is handled +# correctly by ext4 as well as test that the block layer doesn't split or = only +# generate multiple bios for an atomic write. + +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto rw stress atomicwrites + +_require_scratch_write_atomic +_require_aiodio +_require_fio_version "3.38+" + +FIO_LOAD=3D$(($(nproc) * 2 * LOAD_FACTOR)) +SIZE=3D$((100*1024*1024)) + +# Calculate fsblocksize as per bdev atomic write units. +bdev_awu_min=3D$(_get_atomic_write_unit_min $SCRATCH_DEV) +bdev_awu_max=3D$(_get_atomic_write_unit_max $SCRATCH_DEV) +bs=3D$(_max 4096 "$bdev_awu_min") + +function create_fio_configs() +{ + local bsize=3D$1 + create_fio_aw_config $bsize + create_fio_verify_config $bsize +} + +function create_fio_verify_config() +{ + local bsize=3D$1 +cat >$fio_verify_config <$fio_aw_config <> $seqres.full 2>&1 || continue + if _try_scratch_mount >> $seqres.full 2>&1; then + echo "=3D=3D Testing: bs=3D$bs cs=3D$cs iosize=3D$iosize =3D=3D" >> $seq= res.full + + touch $SCRATCH_MNT/f1 + create_fio_configs $iosize + + cat $fio_aw_config >> $seqres.full + echo >> $seqres.full + cat $fio_verify_config >> $seqres.full + + $FIO_PROG $fio_aw_config >> $seqres.full + ret1=3D$? + + $FIO_PROG $fio_verify_config >> $seqres.full + ret2=3D$? + + _scratch_unmount + + [[ $ret1 -eq 0 && $ret2 -eq 0 ]] || _fail "fio with atomic write failed" + fi +} + +run_test() { + local bs=3D$1 + + # cluster sizes above 16 x blocksize are experimental so avoid them + # Also, cap cluster size at 128kb to keep it reasonable for large + # blocks size + max_cs=3D$(_min $((16 * bs)) "$bdev_awu_max" $((128 * 1024))) + + # Fuzz for combinations of blocksize, clustersize and + # iosize that cover most of the cases + run_test_one $bs $bs $bs + run_test_one $bs $max_cs $bs + run_test_one $bs $max_cs $max_cs + run_test_one $bs $max_cs $(_max "$((max_cs/2))" $bs) +} + +# Let's create a sample fio config to check whether fio supports all optio= ns. +fio_aw_config=3D$tmp.aw.fio +fio_verify_config=3D$tmp.verify.fio +fio_out=3D$tmp.fio.out + +create_fio_configs $bs +_require_fio $fio_aw_config + +for ((bs=3D$bs; bs <=3D $(_get_page_size); bs =3D $bs << 1)); do + run_test $bs +done + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/ext4/061.out b/tests/ext4/061.out new file mode 100644 index 00000000..273be9e0 --- /dev/null +++ b/tests/ext4/061.out @@ -0,0 +1,2 @@ +QA output created by 061 +Silence is golden --=20 2.49.0 From nobody Sun Sep 7 23:48:49 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4D952EE615; Fri, 22 Aug 2025 08:02:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849778; cv=none; b=H3w7VLeZKahWl+NeQHNYGUJFrXyJqkGTLsxpasguxc2oKmk33FtoIUeZOOC/9oLvJCPzmaU1iF36pkal1BSXsR4vL3kL3Wa3HeZJXB+IhPXOHcHIMWbs/u1Px6eWmCW4zX8QHDdePyY7zE+q2i8F7HqufcBCa6nLAzno0PLHPHU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849778; c=relaxed/simple; bh=2wdVnJkwYeXW7x3GQfVg5pByW22kDKehXKlG4uwt6M0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pjORSV7Xm/HSoACPT8kLKAi9b4Kby3dGF8douoKBxLM8K8OndzeEjheblkx3XQq7vTBg4zVURqYFD7Kg83XbOw5rQjqXzrvpYT49oFcIsPWL+PTh2tp3PhFbAK+LcQHfyZ+BqDhyvejkPeLjLKNNgbUHTzxc8ZvVDKUL8eeAhQM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=IMZCTaXr; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="IMZCTaXr" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57M7SdRe016626; Fri, 22 Aug 2025 08:02:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=yNbSQIv91voq6nVLy j5r+K1DTIwAWE6KyJ4qtnZes4g=; b=IMZCTaXr8tlzfqgaAyqIk8a7LQ11qzQFe F2shbRA+CFrBTwlJxmgDlMU4g7uPiiUZP5bCKb5JEHN8GBKGyUQOq0GvLZ983fAt y7E5HweiFQ2BveG9OysrJy0nGhhKltJWCdcF4xDf0aOsui4zyGzsvWphHZRNuYgn 7YsEBUfFOjT1n4sldVb6SB70s+R4GIVcOeRCiThZ5Ytl5n3v3qffZemn/4/25R9r wkaa2k5quW17X+Wd/MeTPfCpkclCtEUggWtyR3ccl3KaRHsuee/6K/C/vjjjhJG6 1Jrrds3uIknA+JPENKkmjFK0EDNH993PmTXN86LdqYkhhqHpelCJQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vw9f9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:48 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 57M82maC006425; Fri, 22 Aug 2025 08:02:48 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vw9f3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:48 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57M82c3V031905; Fri, 22 Aug 2025 08:02:47 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 48my5ycajb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:47 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57M82jlm41681250 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Aug 2025 08:02:45 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 52A7420043; Fri, 22 Aug 2025 08:02:45 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1F78B20040; Fri, 22 Aug 2025 08:02:43 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.210.10]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 Aug 2025 08:02:42 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v5 11/12] ext4: Test atomic writes allocation and write codepaths with bigalloc Date: Fri, 22 Aug 2025 13:32:10 +0530 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: YuBsMEhNRXzZxdML_JkVPj5SR83duF2C X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODE5MDIyMiBTYWx0ZWRfXw1yTeOAwBnj6 rWSbvtLB1hz7wDCM7tCX6euWis+rwHeU8O4ZrJ5Or5AR5uIdYpnHHr63sBoiK62zxuURMI7HZaE K58PF2CjIR2+DmtwNp9asuz34r8ZnKyfOS2BmT4kyukY8eDmD+8aUrUz8LY6qcACdPnTPNrGK7G SmMUiYXln8TMNkueVZBKsVbzpSsSJRbvpoeeI3fDhPantHsup3BvgYm46eI7OD9HmeXCygpykfV a1QwA6JjNO/0kE+JAFHwuDDVIBrhr5B6GfKEmgpCQa9uDIFj1al8cg9oAAlW3BI+rouTsbTEX2x 9doHAMswaANtGrMO+I8RT3RKD8BJ79N61dYFUAzFmeb7yZLEyzVn0y6U4P5lY1dTeAO8Sf5p0xH qn060AgIFynF8XNmTFetwueZywaE3A== X-Proofpoint-GUID: CJnYrdIPhsgyGEpVeLwu9H36y2Qrlyns X-Authority-Analysis: v=2.4 cv=KPwDzFFo c=1 sm=1 tr=0 ts=68a82428 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=2OwXVqhp2XgA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=o-vKpyFH2n2aPF30nXYA:9 a=cPQSjfK2_nFv0Q5t_7PE:22 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-22_02,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 spamscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 bulkscore=0 priorityscore=1501 suspectscore=0 phishscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2508110000 definitions=main-2508190222 Content-Type: text/plain; charset="utf-8" From: "Ritesh Harjani (IBM)" This test does a parallel RWF_ATOMIC IO on a multiple truncated files in a small FS. The idea is to stress ext4 allocator to ensure we are able to handle low space scenarios correctly with atomic writes. We brute force this for different blocksize and clustersizes and after each iteration we ensure the data was not torn or corrupted using fio crc verification. Note that in this test we use overlapping atomic writes of same io size. Although serializing racing writes is not guaranteed for RWF_ATOMIC, NVMe and SCSI provide this guarantee as an inseparable feature to power-fail atomicity. Keeping the iosize as same also ensures that ext4 doesn't tear the write due to racing ioend unwritten conversion. The value of this test is that we make sure the RWF_ATOMIC is handled correctly by ext4 as well as test that the block layer doesn't split or only generate multiple bios for an atomic write. Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo --- tests/ext4/062 | 203 +++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/062.out | 2 + 2 files changed, 205 insertions(+) create mode 100755 tests/ext4/062 create mode 100644 tests/ext4/062.out diff --git a/tests/ext4/062 b/tests/ext4/062 new file mode 100755 index 00000000..d48f69d3 --- /dev/null +++ b/tests/ext4/062 @@ -0,0 +1,203 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 062 +# +# This test does a parallel RWF_ATOMIC IO on a multiple truncated files in= a +# small FS. The idea is to stress ext4 allocator to ensure we are able to +# handle low space scenarios correctly with atomic writes.. We brute force= this +# for all possible blocksize and clustersizes and after each iteration we +# ensure the data was not torn or corrupted using fio crc verification. +# +# Note that in this test we use overlapping atomic writes of same io size. +# Although serializing racing writes is not guaranteed for RWF_ATOMIC, NVM= e and +# SCSI provide this guarantee as an inseparable feature to power-fail +# atomicity. Keeping the iosize as same also ensures that ext4 doesn't tea= r the +# write due to racing ioend unwritten conversion. +# +# The value of this test is that we make sure the RWF_ATOMIC is handled +# correctly by ext4 as well as test that the block layer doesn't split or = only +# generate multiple bios for an atomic write. +# + +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto rw stress atomicwrites + +_require_scratch_write_atomic +_require_aiodio +_require_fio_version "3.38+" + +FSSIZE=3D$((360*1024*1024)) +FIO_LOAD=3D$(($(nproc) * LOAD_FACTOR)) + +# Calculate bs as per bdev atomic write units. +bdev_awu_min=3D$(_get_atomic_write_unit_min $SCRATCH_DEV) +bdev_awu_max=3D$(_get_atomic_write_unit_max $SCRATCH_DEV) +bs=3D$(_max 4096 "$bdev_awu_min") + +function create_fio_configs() +{ + local bsize=3D$1 + create_fio_aw_config $bsize + create_fio_verify_config $bsize +} + +function create_fio_verify_config() +{ + local bsize=3D$1 +cat >$fio_verify_config <$fio_aw_config <> $seqres.full 2>&1 || continue + if _try_scratch_mount >> $seqres.full 2>&1; then + echo "Testing: bs=3D$bs cs=3D$cs iosize=3D$iosize" >> $seqres.full + + touch $SCRATCH_MNT/f1 + create_fio_configs $iosize + + cat $fio_aw_config >> $seqres.full + cat $fio_verify_config >> $seqres.full + + $FIO_PROG $fio_aw_config >> $seqres.full + ret1=3D$? + + $FIO_PROG $fio_verify_config >> $seqres.full + ret2=3D$? + + _scratch_unmount + + [[ $ret1 -eq 0 && $ret2 -eq 0 ]] || _fail "fio with atomic write failed" + fi +} + +run_test() { + local bs=3D$1 + + # cluster sizes above 16 x blocksize are experimental so avoid them + # Also, cap cluster size at 128kb to keep it reasonable for large + # blocks size + max_cs=3D$(_min $((16 * bs)) "$bdev_awu_max" $((128 * 1024))) + + # Fuzz for combinations of blocksize, clustersize and + # iosize that cover most of the cases + run_test_one $bs $bs $bs + run_test_one $bs $max_cs $bs + run_test_one $bs $max_cs $max_cs + run_test_one $bs $max_cs $(_max "$((max_cs/2))" $bs) +} + +# Let's create a sample fio config to check whether fio supports all optio= ns. +fio_aw_config=3D$tmp.aw.fio +fio_verify_config=3D$tmp.verify.fio +fio_out=3D$tmp.fio.out + +create_fio_configs $bs +_require_fio $fio_aw_config + +for ((bs=3D$bs; bs <=3D $(_get_page_size); bs =3D $bs << 1)); do + run_test $bs $cs $iosize +done + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/ext4/062.out b/tests/ext4/062.out new file mode 100644 index 00000000..a1578f48 --- /dev/null +++ b/tests/ext4/062.out @@ -0,0 +1,2 @@ +QA output created by 062 +Silence is golden --=20 2.49.0 From nobody Sun Sep 7 23:48:49 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A4C702EFDB2; Fri, 22 Aug 2025 08:03:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849782; cv=none; b=G+xECOMBTtP8CL42KbhdLkqKzhIFPfD9DV3GeQKCYqyTQAZ30SKLKfJhSsExzVPRtP70uD66PBzqY9muVtjknYPUua7BNkCjKK7z6P34/6qBRfNWO+4TW0SRaFneQXabFT2WNx8bdTiWhbSFd3XMVb7qtI3I4W7M0UuyepiHzE8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755849782; c=relaxed/simple; bh=oaKqYHlsVtfa1zZFOpdl1nzBlYmVBXPTbCPK/8lQQq8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LsclmI4zExvVa+iWhZ39wlkB8P5xj9lDmXpC8tGzcixoKT9+KaT7XqF4Hg/rHrKtQorB/FoDgQzHbT+GupDXuoh6AMfna5/FiHeDgA1cHMerphM0mKDf4y1yp7pVx25cFOqQyixoyEForoCCHzMG452g5nR+tYqoozHqxSLNiBs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=aD3BK0lM; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="aD3BK0lM" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57M7cxER026013; Fri, 22 Aug 2025 08:02:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=c/tYPpUPqg61RgADq IE5QLIuPAEA+t+uU9KouQvjvHE=; b=aD3BK0lMGr+NYB/tnTq96O8rkl6Tb8FkD GUQH8ZZLbvmrQX3RRMM7wzVmm5EPHW+XNuTUBIDDyCmKESo0J2a4s3B90D0CLxOg vtwQ1okLkz+lMDkc3ESRFrUISOFqQWMPvplwVmjl5YBprDN+JUnGC+Vj/I/5npm0 Mzo2iUe43Zb46G8SdB/GiRB+Ab8rRs4vYGfbbcU6E0vwTAWNse9yuB1wfa9KEJ1i 6BnT1lVCffCePRH2rXbhUncUEg4Pmc9NCozcNMfV9ToDZJezQ58OEu3z3JriThSI FYKn2e7JUquksCo3fLoRCsIH9xack0IKzV9eRi2lCRBIoURA/qPEA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vnbs0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:52 +0000 (GMT) Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 57M7qQL7008128; Fri, 22 Aug 2025 08:02:51 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48n38vnbrv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:51 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57M7woeu008761; Fri, 22 Aug 2025 08:02:50 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 48my42cafs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 Aug 2025 08:02:50 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57M82muM51970336 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 Aug 2025 08:02:48 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 161A32005A; Fri, 22 Aug 2025 08:02:48 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B374420043; Fri, 22 Aug 2025 08:02:45 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.210.10]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 Aug 2025 08:02:45 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v5 12/12] ext4: Atomic write test for extent split across leaf nodes Date: Fri, 22 Aug 2025 13:32:11 +0530 Message-ID: <370b882f57455e2ca5d7568a26aa5d6ea068ffd4.1755849134.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODE5MDIyMiBTYWx0ZWRfX7NstifbbIyyc FPwQeIMeNM5ybkydVcgNMAJYf6iIDftL5ydF41gTCS+k9oYh6AwJlZR1crg2O3g92PS6IYYK57d XzXnCY8uP8DCWjXO2agCNtq1WsuHv03CgJ4kkjAWjPkGmG8Iv1U7qh1gPoaIGt8Cit4Q2xh3Zp7 sYo5rINC6KvvzZGOlHZt78wEExcyb5GUcLw602CAWZVCpRJW/9jg6oeEL+iiD5UqlhdtQaf/YNU 293ZfwyeWwbYGn+qK9r7dHWupHs8ZDU4Tn/A89cmEKBzqFptzljJg0U0VcI+kkXwqLlImjgeE5R DS924O7Hc2Ffy7Ky+6eXJrbvfBdTUSBFGArK19btvW9X6vUGdKJM8VOnjRmi2tGM+9jmGwwYlMC Epi/jza71exiPszA9HjkpMiosOHJUg== X-Authority-Analysis: v=2.4 cv=PMlWOfqC c=1 sm=1 tr=0 ts=68a8242c cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=2OwXVqhp2XgA:10 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=BtNrGSYHu5IGmLxdGZgA:9 a=cPQSjfK2_nFv0Q5t_7PE:22 X-Proofpoint-GUID: 0nSw54AFiHO5mtnIvUgqRaWhPlTloyFL X-Proofpoint-ORIG-GUID: 1CKuFpzvUubDDZHmqxvfygvChOBhLR3Q X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-22_02,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 bulkscore=0 adultscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 spamscore=0 phishscore=0 suspectscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2508110000 definitions=main-2508190222 Content-Type: text/plain; charset="utf-8" In ext4, even if an allocated range is physically and logically contiguous, it can still be split into 2 extents. This is because ext4 does not merge extents across leaf nodes. This is an issue for atomic writes since even for a continuous extent the map block could (in rare cases) return a shorter map, hence tearning the write. This test creates such a file and ensures that the atomic write handles this case correctly Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo --- tests/ext4/063 | 129 +++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/063.out | 2 + 2 files changed, 131 insertions(+) create mode 100755 tests/ext4/063 create mode 100644 tests/ext4/063.out diff --git a/tests/ext4/063 b/tests/ext4/063 new file mode 100755 index 00000000..19fb611b --- /dev/null +++ b/tests/ext4/063 @@ -0,0 +1,129 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# In ext4, even if an allocated range is physically and logically contiguo= us, +# it can still be split into 2 or more extents. This is because ext4 does = not +# merge extents across leaf nodes. This is an issue for atomic writes since +# even for a continuous extent the map block could (in rare cases) return a +# shorter map, hence tearing the write. This test creates such a file and +# ensures that the atomic write handles this case correctly +# +. ./common/preamble +. ./common/atomicwrites +_begin_fstest auto atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_atomic_write_test_commands +_require_command "$DEBUGFS_PROG" debugfs + +prep() { + local bs=3D`_get_block_size $SCRATCH_MNT` + local ex_hdr_bytes=3D12 + local ex_entry_bytes=3D12 + local entries_per_blk=3D$(( (bs - ex_hdr_bytes) / ex_entry_bytes )) + + # fill the extent tree leaf with bs len extents at alternate offsets. + # The tree should look as follows + # + # +---------+---------+ + # | index 1 | index 2 | + # +-----+---+-----+---+ + # +------+ +-----------+ + # | | + # +-------+-------+---+---------+ +-----+----+ + # | ex 1 | ex 2 | | ex n | | ex n+1 | + # | off:0 | off:2 |...| off:678 | | off:680 | + # | len:1 | len:1 | | len:1 | | len:1 | + # +-------+-------+---+---------+ +----------+ + # + for i in $(seq 0 $entries_per_blk) + do + $XFS_IO_PROG -fc "pwrite -b $bs $((i * 2 * bs)) $bs" $testfile > /dev/nu= ll + done + sync $testfile + + echo >> $seqres.full + echo "Create file with extents spanning 2 leaves. Extents:">> $seqres.full + echo "...">> $seqres.full + $DEBUGFS_PROG -R "ex `basename $testfile`" $SCRATCH_DEV |& tail >> $seqre= s.full + + # Now try to insert a new extent ex(new) between ex(n) and ex(n+1). + # Since this is a new FS the allocator would find continuous blocks + # such that ex(n) ex(new) ex(n+1) are physically(and logically) + # contiguous. However, since we don't merge extents across leaf we will + # end up with a tree as: + # + # +---------+---------+ + # | index 1 | index 2 | + # +-----+---+-----+---+ + # +------+ +------------+ + # | | + # +-------+-------+---+---------+ +------+-----------+ + # | ex 1 | ex 2 | | ex n | | ex n+1 (merged) | + # | off:0 | off:2 |...| off:678 | | off:679 | + # | len:1 | len:1 | | len:1 | | len:2 | + # +-------+-------+---+---------+ +------------------+ + # + echo >> $seqres.full + torn_ex_offset=3D$((((entries_per_blk * 2) - 1) * bs)) + $XFS_IO_PROG -c "pwrite $torn_ex_offset $bs" $testfile >> /dev/null + sync $testfile + + echo >> $seqres.full + echo "Perform 1 block write at $torn_ex_offset to create torn extent. Ext= ents:">> $seqres.full + echo "...">> $seqres.full + $DEBUGFS_PROG -R "ex `basename $testfile`" $SCRATCH_DEV |& tail >> $seqre= s.full + + _scratch_cycle_mount +} + +_scratch_mkfs >> $seqres.full +_scratch_mount >> $seqres.full + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile +awu_max=3D$(_get_atomic_write_unit_max $testfile) + +echo >> $seqres.full +echo "# Prepping the file" >> $seqres.full +prep + +torn_aw_offset=3D$((torn_ex_offset - (torn_ex_offset % awu_max))) + +echo >> $seqres.full +echo "# Performing atomic IO on the torn extent range. Command: " >> $seqr= es.full +echo $XFS_IO_PROG -c "open -fsd $testfile" -c "pwrite -S 0x61 -DA -V1 -b $= awu_max $torn_aw_offset $awu_max" >> $seqres.full +$XFS_IO_PROG -c "open -fsd $testfile" -c "pwrite -S 0x61 -DA -V1 -b $awu_m= ax $torn_aw_offset $awu_max" >> $seqres.full + +echo >> $seqres.full +echo "Extent state after atomic write:">> $seqres.full +echo "...">> $seqres.full +$DEBUGFS_PROG -R "ex `basename $testfile`" $SCRATCH_DEV |& tail >> $seqres= .full + +echo >> $seqres.full +echo "# Checking data integrity" >> $seqres.full + +# create a dummy file with expected data +$XFS_IO_PROG -fc "pwrite -S 0x61 -b $awu_max 0 $awu_max" $testfile.exp >> = /dev/null +expected_data=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp) + +# We ensure that the data after atomic writes should match the expected da= ta +actual_data=3D$(od -An -t x1 -j $torn_aw_offset -N $awu_max $testfile) +if [[ "$actual_data" !=3D "$expected_data" ]] +then + echo "Checksum match failed at off: $torn_aw_offset size: $awu_max" + echo + echo "Expected: " + echo "$expected_data" + echo + echo "Actual contents: " + echo "$actual_data" + + _fail +fi + +echo -n "Data verification at offset $torn_aw_offset suceeded!" >> $seqres= .full +echo "Silence is golden" +status=3D0 +exit diff --git a/tests/ext4/063.out b/tests/ext4/063.out new file mode 100644 index 00000000..de35fc52 --- /dev/null +++ b/tests/ext4/063.out @@ -0,0 +1,2 @@ +QA output created by 063 +Silence is golden --=20 2.49.0