From nobody Thu Oct 2 07:43:36 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1FFA2BF000; Fri, 19 Sep 2025 06:48:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264509; cv=none; b=G0taVOJva//iZjZKq4yjooUjklOlYLquPRCr0HQB7AEzS1BXh6jjLT7Mbg7Enq9Bs7fcDQmRegdjjFFknP6Otank0oPEtY7q1O043zWWUretlx6tePx8sY1R/aMHlaKoxjquvPPXVzGa2JTbyD8TYRsAkmlcxCFAY9sWcb+yXJk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264509; c=relaxed/simple; bh=IdUEutyZ93amHarGXgGsMBB4Qfcd7ks1oarQIAtu0Pc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Auek4dldL+5ZWyOXQPqxfFf71T1xP6uV1n/HshLIq+jpPB/4ba2ef/Q0zR4py1Fso2FYgObMQdH+nFSf3LhXoY03Yoft6+8rkpiNFhqYGeVKgWV3UE3Bo5xpF9X3anT9EqdyvdEtm6mWqTM43jCtJYcBPPWeIzEEyuaw1baOhdI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=IAEDYpRk; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="IAEDYpRk" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58INsFh1023812; Fri, 19 Sep 2025 06:48:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=q9h9zW9aKtQcC8/jU 8xJ+MdlBeFU8vQJmkg5Cw9+FY0=; b=IAEDYpRkeL0U3+/VBZB5oZ9H16H9vkV14 dbSb5czyLfijIeuwRIPCuEGeQTXiK9xNS3A/cfF9kkakbmDLR0I+cxtdTILogZXz X9XGHj50IdTpZFPijVsWm8AiBDGuWOWcMjbXN2XYkoScGVEWKWWIkIXoBQRbTTHZ lwiQ4FgeKnYmiCKr32PYdKnCeSJ2qC25Dlqb7zHIu/JZHJLH03vmzr4rQmlptYNx TDkoFeUB/YkjpkE1B6Z8NOC2078EIl1H52pvcwSWNtUeRDclGN4Akrf6roos0bgM h2cSfWxhO/XR8dNTxMz2WQ+LFbYQ/KbAfPMULFCer3IprXm2vclkA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4qxar1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:18 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58J6UDv5022899; Fri, 19 Sep 2025 06:48:18 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4qxaqy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:17 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58J4lk1h029498; Fri, 19 Sep 2025 06:48:17 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 495kb1atnw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:17 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58J6mF6X28705182 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 19 Sep 2025 06:48:15 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3F2A82004E; Fri, 19 Sep 2025 06:48:15 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A2A1A20040; Fri, 19 Sep 2025 06:48:11 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.51]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 19 Sep 2025 06:48:11 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org, John Garry Subject: [PATCH v7 01/12] common/rc: Add _min() and _max() helpers Date: Fri, 19 Sep 2025 12:17:54 +0530 Message-ID: <97ed192e3cfe41810aa8c599b32f5ba439402bcf.1758264169.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: -LeqR-nHFhZ0fCOmw3qvHkf79uF2n_ww X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTE2MDIwNCBTYWx0ZWRfX/CFrXAzbok/m ZELppsHV0pX/CuQwkeig64h292voZ1+xvRfx3BMCDn5+cA2p0PP2YHMGBF5SpOqray2OwoWST+b Oe+MeGNwdr5uah096SwjFGs/s6Jeh47OIbA1tbMAIJtU5iqv9EdvzfEGuNz5luL+cXPfIr0Jno2 z25oE1nVSOPLq79GIwxKEr/W6f1qVkG1diebe1JNVctAvD6vyDBHYP0aWFvC4MDyRh47IR9I4Tm qYf7d6WfxzxlDmreJ5pVx32ahJpDrlqJl5QanrZFs9YWoIr10x0jndHtF1CpclvJak9S6yM8xea jwqnuEGOsRwlucgDOKcPBp0Tclsnay7eWV8+9XF9NqyvCSO++eCcvpzWD1Hu4Pz1LFlHNs+MJzv sFpSOjh5 X-Authority-Analysis: v=2.4 cv=R8oDGcRX c=1 sm=1 tr=0 ts=68ccfcb2 cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=yJojWOMRYYMA:10 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=wDx1mg2EYdB0JXDzLs4A:9 X-Proofpoint-GUID: 8DcyIrDMwvGvu5JOIJ9K12NvO6K96mwD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-18_03,2025-09-19_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1011 suspectscore=0 malwarescore=0 bulkscore=0 spamscore=0 adultscore=0 impostorscore=0 priorityscore=1501 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509160204 Content-Type: text/plain; charset="utf-8" Many programs open code these functionalities so add it as a generic helper in common/rc Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- common/rc | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/common/rc b/common/rc index 81587dad..28fbbcbb 100644 --- a/common/rc +++ b/common/rc @@ -5978,6 +5978,28 @@ _require_inplace_writes() fi } =20 +_min() { + local ret + + for arg in "$@"; do + if [ -z "$ret" ] || (( $arg < $ret )); then + ret=3D"$arg" + fi + done + echo $ret +} + +_max() { + local ret + + for arg in "$@"; do + if [ -z "$ret" ] || (( $arg > $ret )); then + ret=3D"$arg" + fi + done + echo $ret +} + ##########################################################################= ###### # make sure this script returns success /bin/true --=20 2.49.0 From nobody Thu Oct 2 07:43:36 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7027D2C21F2; Fri, 19 Sep 2025 06:48:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264512; cv=none; b=Mg1fKGmUpIMG3ZOtrj3TyIyPhB8HVnbdCLp5ExqETkA6a+W+j8p4AteW09U8al78PUyK3JCQTweukkuyVx7pu1vGbEmFL3LXxbqleKZD9WhiTo7yq3icF3GHf1t5AwPWvzLfgngg8W+PkBhUkg9201BdUPieF4W9QaXn5OrKCbQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264512; c=relaxed/simple; bh=2JqvJ/NmK7Vl8fIu8O+3jE4mt2NGdF8erJY3mQhxTPM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GoAqee9OffZkz7P/QRayIOYVZuuU2s+eTHbh9VBLX9gMh+qa9sjW38kBUM3/j24elEPxtc3fQ5dtSzFxwM5rncqWryH9HwsSLZHUlhlxBbMrpMlq8ofXA2qRze2k7h6EOXJrpUEka7/E0qJGWzcKgdY0YbrtCDT/j4h58oqVRtM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=DTMyF5nn; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="DTMyF5nn" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58ILtVgX011259; Fri, 19 Sep 2025 06:48:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=DuIzAiNFOh2JfC6oT 8Q9P797UvbN9UNxjQc+oYLCF3Q=; b=DTMyF5nnGsGMzt3833oTdaWtO+doEaE5b XepCA2wVMYpMEw1NFBG7gu8AwWGyWCVkFxI4+p0jNJwBYoE0UGxyoGdNEopvazqz t8BrcldgqowLHdNJIAZ+5fRPcYscnPfUP4YEs41lvwqmZ7JUOpcEYrOFmOzm3Lg9 LwStFgUsOWmrX6dwX89TAVTSWetkQoOyY4ZisK7pF1PzD2If0yxDClTLvhE40V59 LAbgH8zWn1n7PCQZFUvh0xT7zBt5UhU8whKPbflVQGclSIEseNVi02w2rO527/k+ Fi1Xod4znGb7FgEXaUIfT7WUCwlPTybCMtfSdZbohqspQLp/X+gpw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4npbsr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:22 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58J6gH9W015223; Fri, 19 Sep 2025 06:48:22 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4npbsn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:22 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58J5YZEQ027308; Fri, 19 Sep 2025 06:48:21 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 495menjhcc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:21 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58J6mJUP57016726 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 19 Sep 2025 06:48:19 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6A9242004B; Fri, 19 Sep 2025 06:48:19 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C231D20040; Fri, 19 Sep 2025 06:48:15 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.51]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 19 Sep 2025 06:48:15 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v7 02/12] common/rc: Add fio atomic write helpers Date: Fri, 19 Sep 2025 12:17:55 +0530 Message-ID: <8e87c6c800f6ca53f0c89af554b85197c7e397f1.1758264169.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=MN5gmNZl c=1 sm=1 tr=0 ts=68ccfcb6 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=yJojWOMRYYMA:10 a=20KFwNOVAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=nFHpeJSrNhMDmVjzN9sA:9 X-Proofpoint-GUID: WO0zhzJyxN9gPLEM8QrMEPOlC41qcHCQ X-Proofpoint-ORIG-GUID: w3W7rvgRaH8eT618dXO1MWaAxNKzBZsW X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTE2MDIwNCBTYWx0ZWRfX9chtEZFALBsE q4+0ccx3a7XLzdVxejNUo6/1jEoAMYiFIqe/TRSfaVtAYzPi5r3jbbvX2/s8IDu1f/8CuSjXR1r hxfCqSRTSR39Gq8N02Yv9jw+MwhfuSFWVa5Pz14VdP5k6gchrdrX2CkDSQAlCqsZxHXqSfkSEvh 69C/atzlxcXQptsS9iVB8/J/6DfFwiSKsWw6dxOKRr55to4jCHQLLV4SWBDJlreYpgoFxk+/Czm e806pwjVZdOKBBKc9//3gWsaiV2aTIl+tfZaoZGIr0TFnFW0La8yJWiN0SJoIW9xvHI8JgfByPM 3vWkP8zhyM6boLXkR8Eq+wDaZVCapIHcn0Dl0yaVdy9SsSmemtzIzQ9fxybysysTJ0Jx7os9eaI R8/UIzam X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-18_03,2025-09-19_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 spamscore=0 priorityscore=1501 bulkscore=0 impostorscore=0 malwarescore=0 adultscore=0 phishscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509160204 Content-Type: text/plain; charset="utf-8" The main motivation of adding this function on top of _require_fio is that there has been a case in fio where atomic=3D option was added but later it was changed to noop since kernel didn't yet have support for atomic writes. It was then again utilized to do atomic writes in a later version, once kernel got the support. Due to this there is a point in fio where _require_fio w/ atomic=3D1 will succeed even though it would not be doing atomic writes. Hence, add an internal helper __require_fio_version to require specific versions of fio to work past such issues. Further, add the high level _require_fio_atomic_writes helper which tests can use to ensure fio has the right version for atomic writes. Reviewed-by: Zorro Lang Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo Reviewed-by: "Darrick J. Wong" --- common/rc | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/common/rc b/common/rc index 28fbbcbb..8a023b9d 100644 --- a/common/rc +++ b/common/rc @@ -6000,6 +6000,49 @@ _max() { echo $ret } =20 +# Due to reasons explained in fio commit 40f1fc11d, fio version between +# v3.33 and v3.38 have atomic=3D feature but it is a no-op and doesn't do +# RWF_ATOMIC write. Hence, use this helper to ensure fio has the +# required support. Currently, the simplest way we have is to ensure +# the version. +_require_fio_atomic_writes() { + __require_fio_version "3.38+" +} + +# Check the required fio version. Examples: +# __require_fio_version 3.38 (matches 3.38 only) +# __require_fio_version 3.38+ (matches 3.38 and above) +# __require_fio_version 3.38- (matches 3.38 and below) +# +# Internal helper, avoid using directly in tests. +__require_fio_version() { + local req_ver=3D"$1" + local fio_ver + + _require_fio + _require_math + + fio_ver=3D$(fio -v | cut -d"-" -f2) + + case "$req_ver" in + *+) + req_ver=3D${req_ver%+} + test $(_math "$fio_ver >=3D $req_ver") -eq 1 || \ + _notrun "need fio >=3D $req_ver (found $fio_ver)" + ;; + *-) + req_ver=3D${req_ver%-} + test $(_math "$fio_ver <=3D $req_ver") -eq 1 || \ + _notrun "need fio <=3D $req_ver (found $fio_ver)" + ;; + *) + req_ver=3D${req_ver%-} + test $(_math "$fio_ver =3D=3D $req_ver") -eq 1 || \ + _notrun "need fio =3D $req_ver (found $fio_ver)" + ;; + esac +} + ##########################################################################= ###### # make sure this script returns success /bin/true --=20 2.49.0 From nobody Thu Oct 2 07:43:36 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 359AC2BEC26; Fri, 19 Sep 2025 06:48:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264517; cv=none; b=cOgojQ4va2ryyvafJIYhj7a8HL/FgoWmrClYVCMuPxPsAzXww3qLGIBIP3/8O+abVkSaNOXbqJNGYK2H2pPcMQq12NUU9ZKENGT1s8QMBdXVjWzcqPBLUfIqlt7GadXEXlPU6ssjoCancyCCJ4Q757wwsbxYmPVQXqirSnQ03yo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264517; c=relaxed/simple; bh=eLun/YRHHykazaVnn6i45Z+pzdc3kn72i/dVNUgTmwA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XjIwOTuKNlOZgIKmRWwOVdCxGKHSp6EvHapkC8MyC8dXZ00cCq4YHcHjlGQftBFNgRrZlufja+jJjNHeE5a9+1HwB5WAuNyhMI5MjdmzeuXxODTPhvpz4nV8nZ54CigaErjjYPFw6R5I/MAqlEVl/sUlN4TPbQ/kT8vEWFUH7QQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=CTZrBG3k; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="CTZrBG3k" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58J4tQs9011358; Fri, 19 Sep 2025 06:48:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=se562Xeeq4/F+TqKO yQTZeEEX6qaNTC32TSkbPlkRY0=; b=CTZrBG3kGXeToviSzNujvI6Az8gFTxaYL KefFludfpTi+F3SXxOd2Zy8rLDj69PnP1tNgzz0NZH8ibYkxvuWG5ZKoexAeKN5m HbHmZa3lX2NP5vx4dxNoaQxq8X50bNMv8gaBbcl4xxhubnfA7l+gR77vVKi00N06 vU78wGYhVPiDg68GOEIch8aaMgm71bsgGIVrC8TOUiHB3edNGjKNLAw7YZch5rGI 5YUoosSplQoloNw/K+/hVq7s6NYWiXB9hsFtGxgdOC2/RBbTW/Eh90twOOFjHuoB oX0j5QMDTbJN2cE5Nwmh7wsDqOA3UTGu76b3U5pu9fvgQVJf2HE5w== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4npbt7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:26 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58J6khrv024091; Fri, 19 Sep 2025 06:48:26 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4npbt4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:25 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58J6QrCJ027393; Fri, 19 Sep 2025 06:48:25 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 495menjhch-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:25 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58J6mNew52429298 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 19 Sep 2025 06:48:23 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5323520043; Fri, 19 Sep 2025 06:48:23 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0DA8C20040; Fri, 19 Sep 2025 06:48:20 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.51]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 19 Sep 2025 06:48:19 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v7 03/12] common/rc: Add a helper to run fsx on a given file Date: Fri, 19 Sep 2025 12:17:56 +0530 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=MN5gmNZl c=1 sm=1 tr=0 ts=68ccfcba cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=u_VYNAqGXo6OSSAA:21 a=yJojWOMRYYMA:10 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=EfcJlRrZmV05EoLhlkQA:9 X-Proofpoint-GUID: E_nYEFYF7MOoCdpq2kOjIs9U6dyr0psM X-Proofpoint-ORIG-GUID: cZ-G_5miBbkywdq8494acneFJAUwTcUL X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTE2MDIwNCBTYWx0ZWRfX/twCDT4VFxaW WAlG8lXJUMKfaF2Zrhx8vI0gGidpfarc/ccJL+ttZABAD1dVsChu/6M8QJnsTODZBtj9D1yk2Js LWjIkERTlm/Gs5h3wZ1Y7wqvxCWC6wUMBiDY0kSIAB4qqO3KFFW3CTmjUxwsnlSjMb5aULWBH7z CmuWOTrY+A5/Bz2N7pR3ceg4/sHZyhnbn7gyiahZ6/51YZ49jak0ijtcVAUONW76yMJYqCrRnXu mHC6j2h4BqqbgEr+nb7XffBmkiwfIi1AJdrQ1E1wKaivCTm9tBHCT/VKuAAI/L9enGG1zZC8DjC mPBMFX101beqlK9qgylBLXQv1rwqLb+sNOLoIwaB98BMo8UETsKxkMHbqDQ4n504isc6aRk3DTD MK6wpVlm X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-18_03,2025-09-19_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 spamscore=0 priorityscore=1501 bulkscore=0 impostorscore=0 malwarescore=0 adultscore=0 phishscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509160204 Content-Type: text/plain; charset="utf-8" Currently run_fsx is hardcoded to run on a file in $TEST_DIR. Add a helper _run_fsx_on_file so that we can run fsx on any given file including in $SCRATCH_MNT. Also, refactor _run_fsx to use this helper. No functional change is intended in this patch. Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- common/rc | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/common/rc b/common/rc index 8a023b9d..ac77a650 100644 --- a/common/rc +++ b/common/rc @@ -5203,13 +5203,24 @@ _require_hugepage_fsx() _notrun "fsx binary does not support MADV_COLLAPSE" } =20 -_run_fsx() +_run_fsx_on_file() { + local testfile=3D$1 + shift + + if ! [ -f $testfile ] + then + echo "_run_fsx_on_file: $testfile doesn't exist. Creating" >> $seqres.fu= ll + touch $testfile + fi + echo "fsx $*" local args=3D`echo $@ | sed -e "s/ BSIZE / $bsize /g" -e "s/ PSIZE / $psi= ze /g"` - set -- $FSX_PROG $args $FSX_AVOID $TEST_DIR/junk + + set -- $FSX_PROG $args $FSX_AVOID $testfile + echo "$@" >>$seqres.full - rm -f $TEST_DIR/junk + rm -f $testfile "$@" 2>&1 | tee -a $seqres.full >$tmp.fsx local res=3D${PIPESTATUS[0]} if [ $res -ne 0 ]; then @@ -5221,6 +5232,12 @@ _run_fsx() return 0 } =20 +_run_fsx() +{ + _run_fsx_on_file $TEST_DIR/junk $@ + return $? +} + # Run fsx with -h(ugepage buffers). If we can't set up a hugepage then sk= ip # the test, but if any other error occurs then exit the test. _run_hugepage_fsx() { --=20 2.49.0 From nobody Thu Oct 2 07:43:36 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 392D72C11DB; Fri, 19 Sep 2025 06:48:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264527; cv=none; b=NIseWR+e5xEkgNnKcpN6bULwdSoIMk4B6gz+CuyUiRftsSdiaLLT9VA/tvfbg0FjMlfNmDEn0MEcVjENWzM77W5zMS46bclIiSZo2bKEy9FL3k1RNACzGTJTXqaqhT3gXpMu52A63BP2rcBlIra2iUY9WVVpqsjpKiR0H55ubsA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264527; c=relaxed/simple; bh=LioGuSv8gh+FACLKetBabaA1JxM4E2bYxoWLiAjfvzw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sIY16Cfaa+ykxss64tJTX1xDodQTYupP+Ohtaqh66tM+rAt5G1RXQO8bX4nF1BWm3gyNF0C5y4+g46Dmq+MQiDKsp+wr/f6qwMMjPwIVYCSULQlkyeWYW+dXBkrnhOsmT3+7G/VAHLwnfx0vXYcdARtJeRzccfcz2l3VBrzHuE8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=dzRe8Sjt; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="dzRe8Sjt" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58IKYRiQ012254; Fri, 19 Sep 2025 06:48:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=x4NGr1m/Xveh2pPYm VjLzXxjLOVjo2c7XJc+kcjJ/70=; b=dzRe8Sjt/8IAFSNjE8CuNj4EpvPSpPZ86 QRo5Y6WQmiOtlD3RH6cGGjZFVzQfZG/A1ZiFJUQ7DQFIg0SRkN0/rDqWTiA0YeQ4 pOnrozojl7wEif2XDBqzHRw3duLCJWemLKdp/rP0YqL+VUkyKguzF6Z0tisswvog p2Lz3b4WauctaIILb1VqMyNFkyPpHoHVJ88+yHh/2mzlFkCx90SuWBcKjUdGb8sD fa8jKs48BIjnzCOEu7gcBreObKzfPWVva93p5HJn1Lvo69PR+vD2ySV9uDTlq/5b 3lKxm7P1dtugQeacPoHILKIiIbPn2k5oAeO9WAanDRi1qg8pGefJg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4jeya0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:30 +0000 (GMT) Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58J6mUOi019914; Fri, 19 Sep 2025 06:48:30 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4jey9v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:30 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58J4mvsk029536; Fri, 19 Sep 2025 06:48:29 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 495kb1atpj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:28 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58J6mRTv51380576 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 19 Sep 2025 06:48:27 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0408D20043; Fri, 19 Sep 2025 06:48:27 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E86CB20040; Fri, 19 Sep 2025 06:48:23 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.51]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 19 Sep 2025 06:48:23 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v7 04/12] ltp/fsx.c: Add atomic writes support to fsx Date: Fri, 19 Sep 2025 12:17:57 +0530 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=Qf5mvtbv c=1 sm=1 tr=0 ts=68ccfcbe cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=oUtnbjD3gOVQiAq08lgA:9 X-Proofpoint-ORIG-GUID: lQfq2T531J8HqO0bs_vjtDuPYDZQz_ha X-Proofpoint-GUID: FE9MHaDSlG2E1bQT-Z1BwvEg56If9nQ8 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTE2MDIwNCBTYWx0ZWRfX72WePjYQfV1T zfG3PubLCjKuXILzQgZ/pvKLAn/YUsZPzHZU0wu8yb4zkD8GORLmveY6jfd+eXD4mLgy63D6j2n GGDEjvY75j9/h3d0dFoQVuiPP5CQXf1ejpkkfv24e4XQRG1UeItEcF68+0ux8lpLTaTaVBpbouO 5/G5FU+i90KA3w+hB+XfR7uoO6pCCg64Wc9vFYwL8DZE+JHA3Vw4kweNjT4o8bujTD8L59zgJSe 2R3w3WPyaRV/OQksB46aH6R8Jy0SNsGpNrhvhjtrzjUgTNSBEIAILxsBitE0rbQwP+0gYSy5eXZ QdC20z6borqb4eRZonsQNW7YIkFWHaCm0oDiMgYooq02ig5m4vv5L6n53LSudIkRJXdClWylHmG wuRejA/3 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-18_03,2025-09-19_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 clxscore=1015 phishscore=0 suspectscore=0 adultscore=0 priorityscore=1501 malwarescore=0 bulkscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509160204 Content-Type: text/plain; charset="utf-8" Implement atomic write support to help fuzz atomic writes with fsx. Suggested-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- ltp/fsx.c | 115 +++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 110 insertions(+), 5 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 163b9453..bdb87ca9 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -40,6 +40,7 @@ #include #endif #include +#include "statx.h" =20 #ifndef MAP_FILE # define MAP_FILE 0 @@ -49,6 +50,10 @@ #define RWF_DONTCACHE 0x80 #endif =20 +#ifndef RWF_ATOMIC +#define RWF_ATOMIC 0x40 +#endif + #define NUMPRINTCOLUMNS 32 /* # columns of data to print on each line */ =20 /* Operation flags (bitmask) */ @@ -110,6 +115,7 @@ enum { OP_READ_DONTCACHE, OP_WRITE, OP_WRITE_DONTCACHE, + OP_WRITE_ATOMIC, OP_MAPREAD, OP_MAPWRITE, OP_MAX_LITE, @@ -200,6 +206,11 @@ int uring =3D 0; int mark_nr =3D 0; int dontcache_io =3D 1; int hugepages =3D 0; /* -h flag */ +int do_atomic_writes =3D 1; /* -a flag disables */ + +/* User for atomic writes */ +int awu_min =3D 0; +int awu_max =3D 0; =20 /* Stores info needed to periodically collapse hugepages */ struct hugepages_collapse_info { @@ -288,6 +299,7 @@ static const char *op_names[] =3D { [OP_READ_DONTCACHE] =3D "read_dontcache", [OP_WRITE] =3D "write", [OP_WRITE_DONTCACHE] =3D "write_dontcache", + [OP_WRITE_ATOMIC] =3D "write_atomic", [OP_MAPREAD] =3D "mapread", [OP_MAPWRITE] =3D "mapwrite", [OP_TRUNCATE] =3D "truncate", @@ -422,6 +434,7 @@ logdump(void) prt("\t***RRRR***"); break; case OP_WRITE_DONTCACHE: + case OP_WRITE_ATOMIC: case OP_WRITE: prt("WRITE 0x%x thru 0x%x\t(0x%x bytes)", lp->args[0], lp->args[0] + lp->args[1] - 1, @@ -1073,6 +1086,25 @@ update_file_size(unsigned offset, unsigned size) file_size =3D offset + size; } =20 +static int is_power_of_2(unsigned n) { + return ((n & (n - 1)) =3D=3D 0); +} + +/* + * Round down n to nearest power of 2. + * If n is already a power of 2, return n; + */ +static int rounddown_pow_of_2(int n) { + int i =3D 0; + + if (is_power_of_2(n)) + return n; + + for (; (1 << i) < n; i++); + + return 1 << (i - 1); +} + void dowrite(unsigned offset, unsigned size, int flags) { @@ -1081,6 +1113,27 @@ dowrite(unsigned offset, unsigned size, int flags) offset -=3D offset % writebdy; if (o_direct) size -=3D size % writebdy; + if (flags & RWF_ATOMIC) { + /* atomic write len must be between awu_min and awu_max */ + if (size < awu_min) + size =3D awu_min; + if (size > awu_max) + size =3D awu_max; + + /* atomic writes need power-of-2 sizes */ + size =3D rounddown_pow_of_2(size); + + /* atomic writes need naturally aligned offsets */ + offset -=3D offset % size; + + /* Skip the write if we are crossing max filesize */ + if ((offset + size) > maxfilelen) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping atomic write past maxfilelen\n"); + log4(OP_WRITE_ATOMIC, offset, size, FL_SKIPPED); + return; + } + } if (size =3D=3D 0) { if (!quiet && testcalls > simulatedopcount && !o_direct) prt("skipping zero size write\n"); @@ -1088,7 +1141,10 @@ dowrite(unsigned offset, unsigned size, int flags) return; } =20 - log4(OP_WRITE, offset, size, FL_NONE); + if (flags & RWF_ATOMIC) + log4(OP_WRITE_ATOMIC, offset, size, FL_NONE); + else + log4(OP_WRITE, offset, size, FL_NONE); =20 gendata(original_buf, good_buf, offset, size); if (offset + size > file_size) { @@ -1108,8 +1164,9 @@ dowrite(unsigned offset, unsigned size, int flags) (monitorstart =3D=3D -1 || (offset + size > monitorstart && (monitorend =3D=3D -1 || offset <=3D monitorend)))))) - prt("%lld write\t0x%x thru\t0x%x\t(0x%x bytes)\tdontcache=3D%d\n", testc= alls, - offset, offset + size - 1, size, (flags & RWF_DONTCACHE) !=3D 0); + prt("%lld write\t0x%x thru\t0x%x\t(0x%x bytes)\tdontcache=3D%d atomic_wr= =3D%d\n", testcalls, + offset, offset + size - 1, size, (flags & RWF_DONTCACHE) !=3D 0, + (flags & RWF_ATOMIC) !=3D 0); iret =3D fsxwrite(fd, good_buf + offset, size, offset, flags); if (iret !=3D size) { if (iret =3D=3D -1) @@ -1785,6 +1842,36 @@ do_dedupe_range(unsigned offset, unsigned length, un= signed dest) } #endif =20 +int test_atomic_writes(void) { + int ret; + struct statx stx; + + if (o_direct !=3D O_DIRECT) { + fprintf(stderr, "main: atomic writes need O_DIRECT (-Z), " + "disabling!\n"); + return 0; + } + + ret =3D xfstests_statx(AT_FDCWD, fname, 0, STATX_WRITE_ATOMIC, &stx); + if (ret < 0) { + fprintf(stderr, "main: Statx failed with %d." + " Failed to determine atomic write limits, " + " disabling!\n", ret); + return 0; + } + + if (stx.stx_attributes & STATX_ATTR_WRITE_ATOMIC && + stx.stx_atomic_write_unit_min > 0) { + awu_min =3D stx.stx_atomic_write_unit_min; + awu_max =3D stx.stx_atomic_write_unit_max; + return 1; + } + + fprintf(stderr, "main: IO Stack does not support " + "atomic writes, disabling!\n"); + return 0; +} + #ifdef HAVE_COPY_FILE_RANGE int test_copy_range(void) @@ -2356,6 +2443,12 @@ have_op: goto out; } break; + case OP_WRITE_ATOMIC: + if (!do_atomic_writes) { + log4(OP_WRITE_ATOMIC, offset, size, FL_SKIPPED); + goto out; + } + break; } =20 switch (op) { @@ -2385,6 +2478,11 @@ have_op: dowrite(offset, size, 0); break; =20 + case OP_WRITE_ATOMIC: + TRIM_OFF_LEN(offset, size, maxfilelen); + dowrite(offset, size, RWF_ATOMIC); + break; + case OP_MAPREAD: TRIM_OFF_LEN(offset, size, file_size); domapread(offset, size); @@ -2511,13 +2609,14 @@ void usage(void) { fprintf(stdout, "usage: %s", - "fsx [-dfhknqxyzBEFHIJKLORWXZ0]\n\ + "fsx [-adfhknqxyzBEFHIJKLORWXZ0]\n\ [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid]\n\ [-l flen] [-m start:end] [-o oplen] [-p progressinterval]\n\ [-r readbdy] [-s style] [-t truncbdy] [-w writebdy]\n\ [-A|-U] [-D startingop] [-N numops] [-P dirpath] [-S seed]\n\ [--replay-ops=3Dopsfile] [--record-ops[=3Dopsfile]] [--duration=3Dseco= nds]\n\ ... fname\n\ + -a: disable atomic writes\n\ -b opnum: beginning operation number (default 1)\n\ -c P: 1 in P chance of file close+open at each op (default infinity)\n\ -d: debug output for all operations\n\ @@ -3059,9 +3158,13 @@ main(int argc, char **argv) setvbuf(stdout, (char *)0, _IOLBF, 0); /* line buffered stdout */ =20 while ((ch =3D getopt_long(argc, argv, - "0b:c:de:fg:hi:j:kl:m:no:p:qr:s:t:uw:xyABD:EFJKHzCILN:OP:RS:UWXZ", + "0ab:c:de:fg:hi:j:kl:m:no:p:qr:s:t:uw:xyABD:EFJKHzCILN:OP:RS:UWXZ", longopts, NULL)) !=3D EOF) switch (ch) { + case 'a': + prt("main(): Atomic writes disabled\n"); + do_atomic_writes =3D 0; + break; case 'b': simulatedopcount =3D getnum(optarg, &endp); if (!quiet) @@ -3475,6 +3578,8 @@ main(int argc, char **argv) exchange_range_calls =3D test_exchange_range(); if (dontcache_io) dontcache_io =3D test_dontcache_io(); + if (do_atomic_writes) + do_atomic_writes =3D test_atomic_writes(); =20 while (keep_running()) if (!test()) --=20 2.49.0 From nobody Thu Oct 2 07:43:36 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 92AF92C0F66; Fri, 19 Sep 2025 06:48:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264523; cv=none; b=CvKFTW+xxKuLMNH6avKWQfsroQJjwxIUwPhDlDapOOLzVe8+qWeMbsqePFoNdLEAMrIycZ9bskGyPUg/fKMAKFkcX0PR5IZTREz6M6XQ9QzxP2aBbpWWRA2Pcxd8cbzhtNA8Lm3G7LeasQ9dQYdvjh5n0RLoUyIHfHl5u5iXHvo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264523; c=relaxed/simple; bh=v26UflPQnu2h2Q5+BBo6b3eR9y7md2rK29p0D+NRWAc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uxU1NK3C8rgwOe1ZhxRYmeFnlno+PZ6y4QQ1/2rdIxO0z+MAmJJg3SyMfeZSq/nL7UcqvGqADLfATdRXVY/lKHhMwhzIKlnhOqSRNLgdCRG6Q7lIRHVQyqN47n5vBt/E/cDoOfQP6Lk+hgSFqMnTLIed0+bJdVOpFWTRP92WU/I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=QcAiWp3x; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="QcAiWp3x" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58J5Ibxf000333; Fri, 19 Sep 2025 06:48:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=rZYDWIxNta4tNx7sM Hsk5+3wUSG6W0FYsTLTVuBCzOs=; b=QcAiWp3x9cnGyZZBqS3/U66sCEayE/aBy kKkc+l8M9Rzaftpsco+2nMDNw8cTJ0qE+dEpIDY9UVAYzzmz72Bmp56Ft26uWiln vKvGDr2uxSqmSFQWurVs3OTaIm4M2D6agnIPMc7kWzuaWYg2pT6taJZPf4p22qxW UreJMT3vzjX4YDoB6zZ8gxRjDkhYmgfd6LEGLZcoXCCki3hP12Cupyx0Fhk6PaYF b+FHGqwoXBC3AtHn2OZGmRRgu1rS6lHhR2Y2bafRa0fs0FP18cUz5yxT7/P80w9p TQoWxDd5macDv0eNdRYrgLsiABlFOW1Tr0LsFj7p/NO956XuHvTWQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4hxwe0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:34 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58J6imam031168; Fri, 19 Sep 2025 06:48:34 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4hxwdw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:34 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58J5PEuo022316; Fri, 19 Sep 2025 06:48:32 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 495kxq2nvx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:32 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58J6mUHs52429300 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 19 Sep 2025 06:48:30 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CC38A20043; Fri, 19 Sep 2025 06:48:30 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8807120040; Fri, 19 Sep 2025 06:48:27 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.51]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 19 Sep 2025 06:48:27 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v7 05/12] generic: Add atomic write test using fio crc check verifier Date: Fri, 19 Sep 2025 12:17:58 +0530 Message-ID: <3c034b2fb5b81b3a043f1851f3905ce6a41c827a.1758264169.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: bKpOq1dU1XUEzF1yZT7hmEKFpYVnJRE1 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTE2MDIwNCBTYWx0ZWRfX5DbidT1VgjBu phwGKMAYMM4IZpfJrBGl6ias5CUeMhF9cV/og9udoSNhXXhHNy9Is4TSF3ea7uWX+0WLIX+1Nuy m0wLGbwY6JrO55TP2r88zNA110GSf7pHEzhHgyv9po+h3J9Gd9lfG8O0VzuLMRkuz2Jrpxq0s2q +AU4h+wu7PDIiUhpeWfIxrTl7wEaJ1rr1a+Kh78Hx4qXc1GqmQ+hqYNBnqKfnh0cPcfg62LDURN oEaaFF5MLbl/daK1I4zSDS78V8pmAhJoevQ5lTI/y/wbhsHZ6DgdaSR0+4Cx5+KcG0+TzTKWHI6 KNY8kTuaVdI8tg7h9dmZX08duhgwl5cjxHoNHRVpjN5ABPRtrzdK08tDSBUsmwXbqSa+5bzRfd6 wX2IpQwX X-Proofpoint-GUID: WE7Y3O_XPlLqIfSLwAWYff4znxNnoFda X-Authority-Analysis: v=2.4 cv=co2bk04i c=1 sm=1 tr=0 ts=68ccfcc2 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=TDUsxBfbQoAVH7yvDjsA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-18_03,2025-09-19_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 impostorscore=0 priorityscore=1501 suspectscore=0 adultscore=0 phishscore=0 malwarescore=0 spamscore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509160204 Content-Type: text/plain; charset="utf-8" This adds atomic write test using fio based on it's crc check verifier. fio adds a crc header for each data block, which is verified later to ensure there is no data corruption or torn write. This test essentially does a lot of parallel RWF_ATOMIC IO on a preallocated file to stress the write and end-io unwritten conversion code paths. The idea is to increase code coverage to ensure RWF_ATOMIC hasn't introduced any issues. Avoid doing overlapping parallel atomic writes because it might give unexpected results. Use offset_increment=3D, size=3D fio options to achieve this behavior. Co-developed-by: Ritesh Harjani (IBM) Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- tests/generic/1226 | 108 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1226.out | 2 + 2 files changed, 110 insertions(+) create mode 100755 tests/generic/1226 create mode 100644 tests/generic/1226.out diff --git a/tests/generic/1226 b/tests/generic/1226 new file mode 100755 index 00000000..7ad74554 --- /dev/null +++ b/tests/generic/1226 @@ -0,0 +1,108 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1226 +# +# Validate FS atomic write using fio crc check verifier. +# +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto aio rw atomicwrites + +_require_scratch_write_atomic +_require_odirect +_require_aio +_require_fio_atomic_writes + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount +_require_xfs_io_command "falloc" + +touch "$SCRATCH_MNT/f1" +awu_min_write=3D$(_get_atomic_write_unit_min "$SCRATCH_MNT/f1") +awu_max_write=3D$(_get_atomic_write_unit_max "$SCRATCH_MNT/f1") + +blocksize=3D$(_max "$awu_min_write" "$((awu_max_write/2))") +threads=3D$(_min "$(($(nproc) * 2 * LOAD_FACTOR))" "100") +filesize=3D$((blocksize * threads * 100)) +depth=3D$threads +io_size=3D$((filesize / threads)) +io_inc=3D$io_size +testfile=3D$SCRATCH_MNT/test-file + +fio_config=3D$tmp.fio +fio_out=3D$tmp.fio.out + +fio_aw_config=3D$tmp.aw.fio +fio_verify_config=3D$tmp.verify.fio + +function create_fio_configs() +{ + create_fio_aw_config + create_fio_verify_config +} + +function create_fio_verify_config() +{ +cat >$fio_verify_config <$fio_aw_config <> $seqres.full +cat $fio_verify_config >> $seqres.full + +$XFS_IO_PROG -fc "falloc 0 $filesize" $testfile >> $seqres.full + +$FIO_PROG $fio_aw_config >> $seqres.full +ret1=3D$? +$FIO_PROG $fio_verify_config >> $seqres.full +ret2=3D$? + +[[ $ret1 -eq 0 && $ret2 -eq 0 ]] || _fail "fio with atomic write failed" + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/generic/1226.out b/tests/generic/1226.out new file mode 100644 index 00000000..6dce0ea5 --- /dev/null +++ b/tests/generic/1226.out @@ -0,0 +1,2 @@ +QA output created by 1226 +Silence is golden --=20 2.49.0 From nobody Thu Oct 2 07:43:36 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A91AB2D23BF; Fri, 19 Sep 2025 06:48:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264526; cv=none; b=mWV4lDHR70yPf2UtLGa8cfZAP3EYMup3jMyiUv+5z94eFzBBoKqaOlRfv2d/l34Ha4dmvO+pxYnjIORcx3bsjU+m9ljJrj8jbLa0RDMdhJn0IGyN8DyclLAkqalJHh8qnrqOBojJgRxB6SyfJcDSF5V5ixLeON21EaXNnjKzgM8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264526; c=relaxed/simple; bh=U2+DFjhyIqPMB1ayIZuWmHVDK4O9xd4a2D0R+RU4N6Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Gn0JArZu0OuyG9f/3ZJWpPIX24PReBKGqux+UbENgm5LGR/hAWtOhvr5s6ntVXDL9115d7DyFXcDbDRiAAyM9iYuOK2oINIkXUOSblTnz1sj8HUTB/EGm17nED/UZDiarkd3LYqVCXLv2PlWYR/3hpGqpKORKnsDaueb41LsmjI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=f8hS+mQQ; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="f8hS+mQQ" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58IM6GCE011736; Fri, 19 Sep 2025 06:48:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=5rUBmHsTxULczha7j OsuDxqz5QJHjQC/dMqlzcYVYyA=; b=f8hS+mQQACVwlaG7bzeTn8NHr7SUaM9aY LNU/udBSirS6m1I7KysLX+08oW7/O65OgPH+k++3c7AY4HrzihZDl6CO+ysEGMQC BlI8goJnZR86G3Q0p1mHGr/dUMTiFhl5yOTHhIy1dr37wkf9HZHVD/VU9nD7ZXWX qa0MYGMkuy3oBKzE1eaNDK/FmYecBUXh0927cCAyyoSUUVpOPHItdTeLPRSWz4Ya 3JDuKlsRGwtMr1cYjd8UO9S6yPBZ+Et1hXV9bsLyf09gjmuJ0GiSOtjS54pTEBdJ xhjDNqwJfVtfZ26nao4apE/zCoZhEBjmcxt6BSqOWjEly2mQoI0rg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4npbtw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:37 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58J6khrx024091; Fri, 19 Sep 2025 06:48:37 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4npbtr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:37 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58J30khO008987; Fri, 19 Sep 2025 06:48:36 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 495nn3t9w7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:36 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58J6mYvs51380586 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 19 Sep 2025 06:48:34 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8993B20043; Fri, 19 Sep 2025 06:48:34 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 56E4020040; Fri, 19 Sep 2025 06:48:31 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.51]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 19 Sep 2025 06:48:31 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v7 06/12] generic: Add atomic write test using fio verify on file mixed mappings Date: Fri, 19 Sep 2025 12:17:59 +0530 Message-ID: <410f90263c448ae3115bdc5523dd73e3d9885e04.1758264169.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=MN5gmNZl c=1 sm=1 tr=0 ts=68ccfcc5 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=qMsZUPF6o8un45dIlBUA:9 X-Proofpoint-GUID: i9apJd-HgoWVKP464UzzqDwD980mxmQX X-Proofpoint-ORIG-GUID: nfECkdjKbAMmqwxsfNDKLua5s-QwMDen X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTE2MDIwNCBTYWx0ZWRfX+S+84iga9pkp gi/jwqgXMkf8j5maXtFGZH8Q0VBkvJEuWOPXQICOzUo0G1zpZdejxDBYCV7HCYwHk2hGST7gKfi 48ndGK0KdnfIPd624xKpVBTKoBpEY9cg070d8W0z9gSKDRseBbleFrof51DzuZGGlGfW+gDSm9/ mdSBl7WfzJRU8RWsc+uqFTFxkjv5/2CgmvA2+fycy973UMwO40WXbuU+JskmSAYhUkY9H1n0JP6 oERzmvw9l7cohS0E0RSqpXv7r1PWO/wHp2s5dGdIycuHAPdlW7UEGD2SN5jrxwPoIz7ls36oVOF iWicN3FxOb8Rc2rritg2r1rXQ1PsSooQqD6+I6wdw9T4hzkCeRolE0O97Ag4KW0XQNojGvs1fYp SuFeAnIh X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-18_03,2025-09-19_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 spamscore=0 priorityscore=1501 bulkscore=0 impostorscore=0 malwarescore=0 adultscore=0 phishscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509160204 Content-Type: text/plain; charset="utf-8" This test uses fio to first create a file with mixed mappings. Then it does atomic writes using aio dio with parallel jobs to the same file with mixed mappings. Finally, we perform a fio verify step to ensure there is no data corruption or torn write. The aim is to stress the FS block allocation and extent handling logic to ensure it handles mixed mappings with RWF_ATOMIC correctly without tearing or losing data. Avoid doing overlapping parallel atomic writes because it might give unexpected results. Use offset_increment=3D, size=3D fio options to achieve this behavior. Co-developed-by: Ritesh Harjani (IBM) Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- tests/generic/1227 | 132 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1227.out | 2 + 2 files changed, 134 insertions(+) create mode 100755 tests/generic/1227 create mode 100644 tests/generic/1227.out diff --git a/tests/generic/1227 b/tests/generic/1227 new file mode 100755 index 00000000..26177508 --- /dev/null +++ b/tests/generic/1227 @@ -0,0 +1,132 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1227 +# +# Validate FS atomic write using fio crc check verifier on mixed mappings +# of a file. +# +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto aio rw atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_odirect +_require_aio +_require_fio_atomic_writes +_require_xfs_io_command "truncate" + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +touch "$SCRATCH_MNT/f1" +awu_min_write=3D$(_get_atomic_write_unit_min "$SCRATCH_MNT/f1") +awu_max_write=3D$(_get_atomic_write_unit_max "$SCRATCH_MNT/f1") + +aw_bsize=3D$(_max "$awu_min_write" "$((awu_max_write/4))") +fsbsize=3D$(_get_block_size $SCRATCH_MNT) + +threads=3D$(_min "$(($(nproc) * 2 * LOAD_FACTOR))" "100") +filesize=3D$((aw_bsize * threads * 100)) +depth=3D$threads +aw_io_size=3D$((filesize / threads)) +aw_io_inc=3D$aw_io_size +testfile=3D$SCRATCH_MNT/test-file + +fio_prep_config=3D$tmp.prep.fio +fio_aw_config=3D$tmp.aw.fio +fio_verify_config=3D$tmp.verify.fio +fio_out=3D$tmp.fio.out + +cat >$fio_prep_config <$fio_aw_config <$fio_verify_config <> $seqres.full +cat $fio_aw_config >> $seqres.full +cat $fio_verify_config >> $seqres.full + +$XFS_IO_PROG -fc "truncate $filesize" $testfile >> $seqres.full + +#prepare file with mixed mappings +$FIO_PROG $fio_prep_config >> $seqres.full + +# do atomic writes without verifying +$FIO_PROG $fio_aw_config >> $seqres.full + +# verify data is not torn +$FIO_PROG $fio_verify_config >> $seqres.full + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/generic/1227.out b/tests/generic/1227.out new file mode 100644 index 00000000..2605d062 --- /dev/null +++ b/tests/generic/1227.out @@ -0,0 +1,2 @@ +QA output created by 1227 +Silence is golden --=20 2.49.0 From nobody Thu Oct 2 07:43:36 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B4A32D2499; Fri, 19 Sep 2025 06:48:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264530; cv=none; b=UW/JJE6GHW6CxeSsy/h978zRYmHrw/5HOiQ6JN3itzXH/CKtGM/WuVdRzUi3o/95fhYCWJPLw8lQDChS8zlG2J6SUHQN8EkkvwhTZuzkCDGW6j8sIOFnHb38VsOWIsPWxsRkvZe3l3LKqzozHTtEKto9OdQ3XL3VqghH9dHmEFA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264530; c=relaxed/simple; bh=+Q/vBM7008NAemI0PvMW6eBWXjGySS3eWBUZ6ZGCzi8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PbYz1qwI5PoI/TV4ihT6UmbXQfP2XnlZtTv+x8ApzQfe2l/04q1MSxJO3FN9AldHqAUWO390KnmHZp2NxW/XAabxg2i/u6g9UXNol6zp4iolFnVspQeFGlIWAUCks9iEKJguyWfZkK+q8svpJdaNULX+RZBgrE/fJXJPWVaaLt0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=mgUCr252; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="mgUCr252" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58J2WLt8000352; Fri, 19 Sep 2025 06:48:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=Qi50oiRdPA4zdEsWw o82Eki8KnW/N3w2AfOe2CPDP0U=; b=mgUCr2521B7r3sBU1CC0sGijpR6T2ndpO kIaoKqXri7wOWDljgmFoqQApY8+D2NcafZ3+42RfVRx6r27m5fagqMixNApkA+A9 A/mxGjFSdhG6xKWaDju1SoIXMoinp0sqWSE6loPOxOA11kpyqry8YuQPrMGDyiB1 8VrFq8tckknjdL/l/KKbjYgFyjsXzJYEsJ+jf/hqKXtSkDV+0v1xx5NXGGV8JEgo +/oPB4DMPmrvSDR8g6/dWLLWIZrwG9LjszubWUaBA+1Xx1zfiHC5a/VMSl/IQVN2 0/i9J5NWJIXfIY9j/Ngzwnx+IWFOu0cw9JLTjFhVBf59TOQxinV7w== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4hxwem-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:41 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58J6XAcG008034; Fri, 19 Sep 2025 06:48:41 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4hxwed-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:41 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58J5PEuq022316; Fri, 19 Sep 2025 06:48:40 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 495kxq2nwa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:40 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58J6mcol50463100 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 19 Sep 2025 06:48:38 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1232F20043; Fri, 19 Sep 2025 06:48:38 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0AB7820040; Fri, 19 Sep 2025 06:48:35 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.51]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 19 Sep 2025 06:48:34 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v7 07/12] generic: Add atomic write multi-fsblock O_[D]SYNC tests Date: Fri, 19 Sep 2025 12:18:00 +0530 Message-ID: <3e255a7b60838df5442a6beb971827853eb9f999.1758264169.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 1fRzuiRpwZl3ZUlcREDrCffCsTnqR8b2 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTE2MDIwNCBTYWx0ZWRfXytga5XznK4tT MqWskT6rxy1/p0jtPLP3bFTdgA/w+yXYQ/TicTox83kf6fJ8L5+BduiG9YvnfKRxc/Cjbogjoec yoSQOLXwe9Bj+YkIFH1PXf1kwxrMBn+rp9UIOli+c2ncmhNpCKp99JiLFvUQEBPKqDw/UMZ32DK DSafbJ1Zf5Ey10vxaIMpUaCzKCDXnGQfPYHakFKHNFRgyqVq5WD0uKq/UxG9HVGitA4h/giVsDJ Sbr1nAf6M9q4Tx5ddCFNzO9XlEDZl1qs+t631U1YzYvZDtrTXtizkfjpBM/q+oKAMTY36EjZXDq 7nYwSV7KbkkB7HVQG/XsiDUicznhYPETox7BjINNyADy9WpYhxsB53F6BjJ6HnVzyRARczWOuol qP3f+Glc X-Proofpoint-GUID: psEjK8g7Cgm96cQ_3TXdx-ZlE9BlAud4 X-Authority-Analysis: v=2.4 cv=co2bk04i c=1 sm=1 tr=0 ts=68ccfcc9 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=udGWdZ-Mnbz566dSa5EA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-18_03,2025-09-19_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 impostorscore=0 priorityscore=1501 suspectscore=0 adultscore=0 phishscore=0 malwarescore=0 spamscore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509160204 Content-Type: text/plain; charset="utf-8" This adds various atomic write multi-fsblock stress tests with mixed mappings and O_SYNC, to ensure the data and metadata is atomically persisted even if there is a shutdown. Suggested-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- tests/generic/1228 | 138 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1228.out | 2 + 2 files changed, 140 insertions(+) create mode 100755 tests/generic/1228 create mode 100644 tests/generic/1228.out diff --git a/tests/generic/1228 b/tests/generic/1228 new file mode 100755 index 00000000..730bf91e --- /dev/null +++ b/tests/generic/1228 @@ -0,0 +1,138 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1228 +# +# Atomic write multi-fsblock data integrity tests with mixed mappings +# and O_SYNC +# +. ./common/preamble +. ./common/atomicwrites +_begin_fstest auto quick rw atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_atomic_write_test_commands +_require_scratch_shutdown +_require_xfs_io_command "truncate" + +_scratch_mkfs >> $seqres.full +_scratch_mount >> $seqres.full + +check_data_integrity() { + actual=3D$(_hexdump $testfile) + if [[ "$expected" !=3D "$actual" ]] + then + echo "Integrity check failed" + echo "Integrity check failed" >> $seqres.full + echo "# Expected file contents:" >> $seqres.full + echo "$expected" >> $seqres.full + echo "# Actual file contents:" >> $seqres.full + echo "$actual" >> $seqres.full + + _fail "Data integrity check failed. The atomic write was torn." + fi +} + +prep_mixed_mapping() { + $XFS_IO_PROG -c "truncate 0" $testfile >> $seqres.full + local off=3D0 + local mapping=3D"" + + local operations=3D("W" "H" "U") + local num_blocks=3D$((awu_max / blksz)) + for ((i=3D0; i /d= ev/null + ;; + "H") + # No operation needed for hole + ;; + "U") + $XFS_IO_PROG -c "falloc $off $blksz" $testfile >> /dev/null + ;; + esac + off=3D$((off + blksz)) + done + + echo "+ + Mixed mapping prep done. Full mapping pattern: $mapping" >> $se= qres.full + + sync $testfile +} + +verify_atomic_write() { + test $bytes_written -eq $awu_max || _fail "atomic write len=3D$awu_max as= sertion failed" + check_data_integrity +} + +mixed_mapping_test() { + prep_mixed_mapping + + echo -"+ + Performing O_DSYNC atomic write from 0 to $awu_max" >> $seqres= .full + if [[ "$1" =3D=3D "shutdown" ]] + then + bytes_written=3D$($XFS_IO_PROG -x -dc \ + "pwrite -DA -V1 -b $awu_max 0 $awu_max" \ + -c "shutdown" $testfile | grep wrote | \ + awk -F'[/ ]' '{print $2}') + _scratch_cycle_mount >>$seqres.full 2>&1 || _fail "remount failed" + else + bytes_written=3D$($XFS_IO_PROG -dc \ + "pwrite -DA -V1 -b $awu_max 0 $awu_max" $testfile | \ + grep wrote | awk -F'[/ ]' '{print $2}') + fi + + verify_atomic_write +} + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile + +awu_max=3D$(_get_atomic_write_unit_max $testfile) +blksz=3D$(_get_block_size $SCRATCH_MNT) + +# Create an expected pattern to compare with +$XFS_IO_PROG -tc "pwrite -b $awu_max 0 $awu_max" $testfile >> $seqres.full +expected=3D$(_hexdump $testfile) +echo "# Expected file contents:" >> $seqres.full +echo "$expected" >> $seqres.full +echo >> $seqres.full + +echo "# Test 1: Do O_DSYNC atomic write on random mixed mapping:" >> $seqr= es.full +echo >> $seqres.full + +iterations=3D10 +for ((i=3D1; i<=3D$iterations; i++)); do + echo "=3D=3D=3D Mixed Mapping Test Iteration $i =3D=3D=3D" >> $seqres.full + + echo "+ Testing without shutdown..." >> $seqres.full + mixed_mapping_test + echo "Passed!" >> $seqres.full + + echo "+ Testing with sudden shutdown..." >> $seqres.full + mixed_mapping_test "shutdown" + echo "Passed!" >> $seqres.full + + echo "Iteration $i completed: OK" >> $seqres.full + echo >> $seqres.full +done +echo "# Test 1: Do O_SYNC atomic write on random mixed mapping ($iteration= s iterations): OK" >> $seqres.full + + +echo >> $seqres.full +echo "# Test 2: Do extending O_SYNC atomic writes: " >> $seqres.full +bytes_written=3D$($XFS_IO_PROG -x -dstc "pwrite -A -V1 -b $awu_max 0 $awu_= max" \ + -c "shutdown" $testfile | grep wrote | awk -F'[/ ]' '{print $2}') +_scratch_cycle_mount >>$seqres.full 2>&1 || _fail "remount failed" +verify_atomic_write +echo "# Test 2: Do extending O_SYNC atomic writes: OK" >> $seqres.full + +# success, all done +echo "Silence is golden" +status=3D0 +exit diff --git a/tests/generic/1228.out b/tests/generic/1228.out new file mode 100644 index 00000000..1baffa91 --- /dev/null +++ b/tests/generic/1228.out @@ -0,0 +1,2 @@ +QA output created by 1228 +Silence is golden --=20 2.49.0 From nobody Thu Oct 2 07:43:36 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F2C82D4B68; Fri, 19 Sep 2025 06:48:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264534; cv=none; b=iFEgwL9VXHt4aAfmSDoJ/WUPu9S263F4wuEpRp3OzhiC8rTQc15Ks8OK1KgbN0dL6ikmove7bXvqkJOBgOUb++QWik7K7BwhH1BkMaaE1RVAwwlYZP2G/flcnKxLXejH2S4/ximZJru6R354LL4zFR2qAiECHN1QwG2eMxKg75k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264534; c=relaxed/simple; bh=DuKA96AdpyKLqZYBHb58RoVvGwseEPk0CTPSH9y8nJQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PD/gUeEfDINghe+Bn4Q+wa5Q+qjdGDHJjtmDBPaYjI89p1yHioHnmE0bGf1n+HsmbFplz9zIdQq+WHQR2i3HLBUFGykvJSPKzbnKSP+yE0LinEXts6jIovTDVdQ1KlG+ZjG2PPhc58XrnVvtAJBaN5lcfkMFvvuoiGVDrsLLGPg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=iCCUACK0; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="iCCUACK0" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58J2WLt9000352; Fri, 19 Sep 2025 06:48:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=ot+islW9QqRcQihl1 cGnoj87N1ScOyDjNrHRSvRNX44=; b=iCCUACK0WXWC9tmkx8tOfZyXItW+qmUtW o77vs2b7QQXURdyA8ATPedve83OAeCWh10RJxFhlqnlYIZMXKv3Lh4cTHygiqrPM PttksOZ+3Ms8WXGZ43o1fdjjwxmWTq6iSAQsbzJ3r4s5owMWRbjsVI2zAmfMBTKV izsQIHgd1S/762j3L0+AoJU+QgoAIuvZfn3tNPkE+z4daMCIgkrm48YGVEQ70Qoy HFg1nPd9YwWik5biqP44PM9kT4Q3Nrkvbt4VUTtvggwjHJolhIWjZE3qDV1L7CxH bTd7a7eG29WpZAUXbn+I+TBdjuLlTIv9km7jrzQi9Wq5N4d5wFsYA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4hxwex-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:45 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58J6k897000735; Fri, 19 Sep 2025 06:48:44 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4hxweu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:44 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58J4KaYn005935; Fri, 19 Sep 2025 06:48:43 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 495jxujufn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:43 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58J6mfv251314994 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 19 Sep 2025 06:48:42 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D40B620043; Fri, 19 Sep 2025 06:48:41 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9859920040; Fri, 19 Sep 2025 06:48:38 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.51]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 19 Sep 2025 06:48:38 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v7 08/12] generic: Stress fsx with atomic writes enabled Date: Fri, 19 Sep 2025 12:18:01 +0530 Message-ID: <710bc35098ed16ceeeaa1214153cfb5e31130da3.1758264169.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: -bf3ix6oHCRkhNAX1VByXKZMqo2rZscI X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTE2MDIwNCBTYWx0ZWRfX1XY+3tEnsdKB T3z40UY7OaJLCwJ38WB0P5ua4lWl7t7mXZMk5FjSQdX+UfH/dFSiXvZ7fwqYkXyw/+teJEpVXF3 g49F9X1So0voMNiT40eVSuYz017hW6bx/CjyQCctiW9ZXBoAIiVDuUXUjCvRoY0Iiqs3GgYtOiN xead47vRUv3hbjHFe1VlXLj5HtnJCMl73um5tGNdh9wBmHT61Pw9AMOOrJolAHjOdBdZ8Na11ev nIlTUzMYsD87/hrPJ3P1juk3Kdc9RtV5ZgpVwQLxp2AQ9oe/zRl/gGvrDgIPXcTcv50SoKwh7fK +GhqZnzUqOQAh5GJ+hK2cTPO/Pemu6lX0UNT+sfKuZxstkk8Br7rrsuvj2bVBgGa112rveHfp+k D5a+ai2H X-Proofpoint-GUID: kdDeZCPsoaCyKMOckpdkWSgOiXvTfUh8 X-Authority-Analysis: v=2.4 cv=co2bk04i c=1 sm=1 tr=0 ts=68ccfccd cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=zPW1cLLGUcrnTIp6xCMA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-18_03,2025-09-19_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 impostorscore=0 priorityscore=1501 suspectscore=0 adultscore=0 phishscore=0 malwarescore=0 spamscore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509160204 Content-Type: text/plain; charset="utf-8" Stress file with atomic writes to ensure we exercise codepaths where we are mixing different FS operations with atomic writes Suggested-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- tests/generic/1229 | 68 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/1229.out | 2 ++ 2 files changed, 70 insertions(+) create mode 100755 tests/generic/1229 create mode 100644 tests/generic/1229.out diff --git a/tests/generic/1229 b/tests/generic/1229 new file mode 100755 index 00000000..6d4dcfed --- /dev/null +++ b/tests/generic/1229 @@ -0,0 +1,68 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1229 +# +# fuzz fsx with atomic writes +# +. ./common/preamble +. ./common/atomicwrites +_begin_fstest rw auto quick atomicwrites + +_require_odirect +_require_scratch_write_atomic + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount >> $seqres.full 2>&1 + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile + +awu_max=3D$(_get_atomic_write_unit_max $testfile) +blksz=3D$(_get_block_size $SCRATCH_MNT) +bsize=3D`$here/src/min_dio_alignment $SCRATCH_MNT $SCRATCH_DEV` + +set_fsx_avoid() { + local file=3D$1 + + case "$FSTYP" in + "ext4") + local dev=3D$(findmnt -n -o SOURCE --target $testfile) + + # fsx insert/collapse range support for ext4+bigalloc is + # currently broken, so disable it. Also disable in case we + # can't detect bigalloc to be on safer side. + if [ -z "$DUMPE2FS_PROG" ]; then + echo "dumpe2fs not found, disabling insert/collapse range" >> $seqres.f= ull + FSX_AVOID+=3D" -I -C" + return + fi + + $DUMPE2FS_PROG -h $dev 2>&1 | grep -q bigalloc && { + echo "fsx insert/collapse range not supported with bigalloc. Disabling.= ." >> $seqres.full + FSX_AVOID+=3D" -I -C" + } + ;; + *) + ;; + esac +} + +# fsx usage: +# +# -N numops: total # operations to do +# -l flen: the upper bound on file size +# -o oplen: the upper bound on operation size (64k default) +# -Z: O_DIRECT () + +set_fsx_avoid +_run_fsx_on_file $testfile -N 10000 -o $awu_max -A -l 500000 -r $bsize -w = $bsize -Z $FSX_AVOID >> $seqres.full +if [[ "$?" !=3D "0" ]] +then + _fail "fsx returned error: $?" +fi + +echo "Silence is golden" +status=3D0 +exit diff --git a/tests/generic/1229.out b/tests/generic/1229.out new file mode 100644 index 00000000..737d61c6 --- /dev/null +++ b/tests/generic/1229.out @@ -0,0 +1,2 @@ +QA output created by 1229 +Silence is golden --=20 2.49.0 From nobody Thu Oct 2 07:43:36 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60CBE265CBB; Fri, 19 Sep 2025 06:48:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264540; cv=none; b=LoJOWIGMz7bdhwUY8yPv63upsn03i+CvH5iv6jus2JTxBQEM2eIRz1/SN7yDhzSroyWMCieT7bHja9H6uRnfDWGZ+CKV4EzB67e2AYWCcckfifndtez1P3W367lSJUR1GDF2OtMW6xJ+9pVFiY5VG3wI4YGE+x9FJCJeoYeCk+Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264540; c=relaxed/simple; bh=7zFcMttLKMqJVZ4fcVUY/R4zC9cUxqDiDZoxmG9SqmY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=diUkTmtoOieG6Pa4sxM1K8pVneUyqSMdGxF1leAip2eQ+z3pKKrB2P877E7Vm7jll3tCewkOk7ogwVzhIiOFCXZ2iu0joU2O2YnD0UWj8MnQNZOIDALhp1yfvhU3UaxNgm1OooPoElBkFOGOb6PQbxYL6oYnfH+NbfoeMzOj5Ig= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=ONyo7m9r; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="ONyo7m9r" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58INDaaJ012207; Fri, 19 Sep 2025 06:48:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=UrioZmgTfKAivy8rh voElgdTxnsSA0lgnyocQ6OyEnM=; b=ONyo7m9rglO4DkdS0mYvvAK/GTaOsccYk oaMd9A7xGn91c1iH6HfbtUQ5Sl3ZxkYrOiORG0LkfSJRteouqlMhHOUEudqX3Dcl wKB7KOLkxX1LDUc6lqjdax16QuE7gYWILBNrJfkropUAzvwCE6gqJaFI+4KTzWVZ 1tZ4F5voZcojGi+H1gmH6LZW0rhQWxVRXtF1hGhmdJ3TcyqEzY4MFoNO9sWTaE+K 7x2pU0nyQ3sjbQaRHJ9d2fWatNejt6aYzYi4An27F19ALJI4oARgk6ndi0BPPxg1 iGHMn09uiNEgBwByLk0X0DWPJt8WktX2mP87JSom1tepyIqFgQJsA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4jeybe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:48 +0000 (GMT) Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58J6gAff008680; Fri, 19 Sep 2025 06:48:48 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4jeybb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:48 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58J30khU008987; Fri, 19 Sep 2025 06:48:47 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 495nn3t9wt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:47 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58J6mjXg29884726 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 19 Sep 2025 06:48:45 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A10AF2004B; Fri, 19 Sep 2025 06:48:45 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6556720040; Fri, 19 Sep 2025 06:48:42 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.51]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 19 Sep 2025 06:48:42 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v7 09/12] generic: Add sudden shutdown tests for multi block atomic writes Date: Fri, 19 Sep 2025 12:18:02 +0530 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=Qf5mvtbv c=1 sm=1 tr=0 ts=68ccfcd1 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=dyK_zsDHFtz5CmQ9vqoA:9 X-Proofpoint-ORIG-GUID: imFECujH9nHfNE438EfAJNB86qoQptSu X-Proofpoint-GUID: dmANn37mm5xYZPmZpYdJyRxI4MJKuCPW X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTE2MDIwNCBTYWx0ZWRfX3SpZt6mZXOK8 gAZPxy7ETqQRqjBLi268V3Tlu0jx9Wm7EobMyYoRAtkRlkt10aF0tPJSzqujZyEO7OdMacoM4j4 wt0/GpTljUEZtJI7SZlnaXXVpYHsa42Tg9cSlhmeZV+M1lP8uECCGY3LcJT1ZPDnpqpwY2QnOXa EGvAJtOiZrrYCznGV4rB4tZCOvpUnNp9CoyQxjn2Gdod2wkO09Uaq9+ZqDHvo/UwnhDV4AuiV2x mkwnyTfE/fP/nmm7aPmZY1jzbq/r9b0MzBivhTC7BMLpGRWBAjJcgBV4s6U513WNtWVPAvGz88f 6ndO8Qljk/SGV9uBurXu+adOXjK59uMaFQ8hKLR2/lAM/8wB07yb+aTyyBbAC2oi5AqN35DBzgc 7AXWtkvd X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-18_03,2025-09-19_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 clxscore=1015 phishscore=0 suspectscore=0 adultscore=0 priorityscore=1501 malwarescore=0 bulkscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509160204 Content-Type: text/plain; charset="utf-8" This test is intended to ensure that multi blocks atomic writes maintain atomic guarantees across sudden FS shutdowns. The way we work is that we lay out a file with random mix of written, unwritten and hole extents. Then we start performing atomic writes sequentially on the file while we parallelly shutdown the FS. Then we note the last offset where the atomic write happened just before shut down and then make sure blocks around it either have completely old data or completely new data, ie the write was not torn during shutdown. We repeat the same with completely written, completely unwritten and comple= tely empty file to ensure these cases are not torn either. Finally, we have a similar test for append atomic writes Suggested-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- tests/generic/1230 | 368 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1230.out | 2 + 2 files changed, 370 insertions(+) create mode 100755 tests/generic/1230 create mode 100644 tests/generic/1230.out diff --git a/tests/generic/1230 b/tests/generic/1230 new file mode 100755 index 00000000..cba75441 --- /dev/null +++ b/tests/generic/1230 @@ -0,0 +1,368 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test No. 1230 +# +# Test multi block atomic writes with sudden FS shutdowns to ensure +# the FS is not tearing the write operation +. ./common/preamble +. ./common/atomicwrites +_begin_fstest auto atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_atomic_write_test_commands +_require_scratch_shutdown +_require_xfs_io_command "truncate" + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount >> $seqres.full + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile + +awu_max=3D$(_get_atomic_write_unit_max $testfile) +blksz=3D$(_get_block_size $SCRATCH_MNT) +echo "Awu max: $awu_max" >> $seqres.full + +num_blocks=3D$((awu_max / blksz)) +# keep initial value high for dry run. This will be +# tweaked in dry_run() based on device write speed. +filesize=3D$(( 10 * 1024 * 1024 * 1024 )) + +_cleanup() { + [ -n "$awloop_pid" ] && kill $awloop_pid &> /dev/null + wait +} + +atomic_write_loop() { + local off=3D0 + local size=3D$awu_max + for ((i=3D0; i<$((filesize / $size )); i++)); do + # Due to sudden shutdown this can produce errors so just + # redirect them to seqres.full + $XFS_IO_PROG -c "open -fsd $testfile" -c "pwrite -S 0x61 -DA -V1 -b $siz= e $off $size" >> /dev/null 2>>$seqres.full + echo "Written to offset: $off" >> $tmp.aw + off=3D$((off + $size)) + done +} + +start_atomic_write_and_shutdown() { + atomic_write_loop & + awloop_pid=3D$! + + local i=3D0 + # Wait for at least first write to be recorded or 10s + while [ ! -f "$tmp.aw" -a $i -le 50 ]; do i=3D$((i + 1)); sleep 0.2; done + + if [[ $i -gt 50 ]] + then + _fail "atomic write process took too long to start" + fi + + echo >> $seqres.full + echo "# Shutting down filesystem while write is running" >> $seqres.full + _scratch_shutdown + + kill $awloop_pid 2>/dev/null # the process might have finished already + wait $awloop_pid + unset $awloop_pid +} + +# This test has the following flow: +# 1. Start doing sequential atomic writes in background, upto $filesize +# 2. Sleep for 0.2s and shutdown the FS +# 3. kill the atomic write process +# 4. verify the writes were not torn +# +# We ideally want the shutdown to happen while an atomic write is ongoing +# but this gets tricky since faster devices can actually finish the whole +# atomic write loop before sleep 0.2s completes, resulting in the shutdown +# happening after the write loop which is not what we want. A simple solut= ion +# to this is to increase $filesize so step 1 takes long enough but a big +# $filesize leads to create_mixed_mappings() taking very long, which is not +# ideal. +# +# Hence, use the dry_run function to figure out the rough device speed and= set +# $filesize accordingly. +dry_run() { + echo >> $seqres.full + echo "# Estimating ideal filesize..." >> $seqres.full + + start_atomic_write_and_shutdown + + bytes_written=3D$(tail -n 1 $tmp.aw | cut -d" " -f4) + echo "# Bytes written in 0.2s: $bytes_written" >> $seqres.full + + filesize=3D$((bytes_written * 3)) + echo "# Setting \$filesize=3D$filesize" >> $seqres.full + + rm $tmp.aw + sleep 0.5 + + _scratch_cycle_mount + +} + +create_mixed_mappings() { + local file=3D$1 + local size_bytes=3D$2 + + echo "# Filling file $file with alternate mappings till size $size_bytes"= >> $seqres.full + #Fill the file with alternate written and unwritten blocks + local off=3D0 + local operations=3D("W" "U") + + for ((i=3D0; i<$((size_bytes / blksz )); i++)); do + index=3D$(($i % ${#operations[@]})) + map=3D"${operations[$index]}" + + case "$map" in + "W") + $XFS_IO_PROG -fc "pwrite -b $blksz $off $blksz" $file >> /dev/null + ;; + "U") + $XFS_IO_PROG -fc "falloc $off $blksz" $file >> /dev/null + ;; + esac + off=3D$((off + blksz)) + done + + sync $file +} + +populate_expected_data() { + # create a dummy file with expected old data for different cases + create_mixed_mappings $testfile.exp_old_mixed $awu_max + expected_data_old_mixed=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_o= ld_mixed) + + $XFS_IO_PROG -fc "falloc 0 $awu_max" $testfile.exp_old_zeroes >> $seqres.= full + expected_data_old_zeroes=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_= old_zeroes) + + $XFS_IO_PROG -fc "pwrite -b $awu_max 0 $awu_max" $testfile.exp_old_mapped= >> $seqres.full + expected_data_old_mapped=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_= old_mapped) + + # create a dummy file with expected new data + $XFS_IO_PROG -fc "pwrite -S 0x61 -b $awu_max 0 $awu_max" $testfile.exp_ne= w >> $seqres.full + expected_data_new=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_new) +} + +verify_data_blocks() { + local verify_start=3D$1 + local verify_end=3D$2 + local expected_data_old=3D"$3" + local expected_data_new=3D"$4" + + echo >> $seqres.full + echo "# Checking for torn write from $verify_start to $verify_end" >> $se= qres.full + + # After an atomic write, for every chunk we ensure that the underlying + # data is either the old data or new data as writes shouldn't get torn. + local off=3D$verify_start + while [[ "$off" -lt "$verify_end" ]] + do + #actual_data=3D$(xxd -s $off -l $awu_max -p $testfile) + actual_data=3D$(od -An -t x1 -j $off -N $awu_max $testfile) + if [[ "$actual_data" !=3D "$expected_data_new" ]] && [[ "$actual_data" != =3D "$expected_data_old" ]] + then + echo "Checksum match failed at off: $off size: $awu_max" + echo "Expected contents: (Either of the 2 below):" + echo + echo "Old: " + echo "$expected_data_old" + echo + echo "New: " + echo "$expected_data_new" + echo + echo "Actual contents: " + echo "$actual_data" + + _fail + fi + echo -n "Check at offset $off succeeded! " >> $seqres.full + if [[ "$actual_data" =3D=3D "$expected_data_new" ]] + then + echo "matched new" >> $seqres.full + elif [[ "$actual_data" =3D=3D "$expected_data_old" ]] + then + echo "matched old" >> $seqres.full + fi + off=3D$(( off + awu_max )) + done +} + +# test torn write for file by shutting down in between atomic writes +test_torn_write() { + echo >> $seqres.full + echo "# Writing atomically to file in background" >> $seqres.full + + start_atomic_write_and_shutdown + + last_offset=3D$(tail -n 1 $tmp.aw | cut -d" " -f4) + if [[ -z $last_offset ]] + then + last_offset=3D0 + fi + + echo >> $seqres.full + echo "# Last offset of atomic write: $last_offset" >> $seqres.full + + rm $tmp.aw + sleep 0.5 + + _scratch_cycle_mount + + # we want to verify all blocks around which the shutdown happened + verify_start=3D$(( last_offset - (awu_max * 5))) + if [[ $verify_start < 0 ]] + then + verify_start=3D0 + fi + + verify_end=3D$(( last_offset + (awu_max * 5))) + if [[ "$verify_end" -gt "$filesize" ]] + then + verify_end=3D$filesize + fi +} + +# test torn write for file with written and unwritten mappings +test_torn_write_mixed() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with mixed mappings" >> $seqres.full + create_mixed_mappings $testfile $filesize + + test_torn_write + + verify_data_blocks $verify_start $verify_end "$expected_data_old_mixed" "= $expected_data_new" +} + +# test torn write for file with completely written mappings +test_torn_write_written() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with fully written mapping" >> $seqres.full + $XFS_IO_PROG -c "pwrite -b $filesize 0 $filesize" $testfile >> $seqres.fu= ll + sync $testfile + + test_torn_write + + verify_data_blocks $verify_start $verify_end "$expected_data_old_mapped" = "$expected_data_new" +} + +# test torn write for file with completely unwritten mappings +test_torn_write_unwritten() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with fully unwritten mappings" >> $seqres.full + $XFS_IO_PROG -c "falloc 0 $filesize" $testfile >> $seqres.full + sync $testfile + + test_torn_write + + verify_data_blocks $verify_start $verify_end "$expected_data_old_zeroes" = "$expected_data_new" +} + +# test torn write for file with no mappings +test_torn_write_hole() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with no mappings" >> $seqres.full + $XFS_IO_PROG -c "truncate $filesize" $testfile >> $seqres.full + sync $testfile + + test_torn_write + + verify_data_blocks $verify_start $verify_end "$expected_data_old_zeroes" = "$expected_data_new" +} + +test_append_torn_write() { + $XFS_IO_PROG -c "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Performing append atomic writes over file in background" >> $seqr= es.full + + start_atomic_write_and_shutdown + + local last_offset=3D$(tail -n 1 $tmp.aw | cut -d" " -f4) + if [[ -z $last_offset ]] + then + last_offset=3D0 + fi + + echo >> $seqres.full + echo "# Last offset of atomic write: $last_offset" >> $seqres.full + rm $tmp.aw + sleep 0.5 + + _scratch_cycle_mount + local filesize=3D$(_get_filesize $testfile) + echo >> $seqres.full + echo "# Filesize after shutdown: $filesize" >> $seqres.full + + # To confirm that the write went atomically, we check: + # 1. The last block should be a multiple of awu_max + # 2. The last block should be the completely new data + + if (( $filesize % $awu_max )) + then + echo "Filesize after shutdown ($filesize) not a multiple of atomic write= unit ($awu_max)" + fi + + verify_start=3D$(( filesize - (awu_max * 5))) + if [[ $verify_start < 0 ]] + then + verify_start=3D0 + fi + + local verify_end=3D$filesize + + # Here the blocks should always match new data hence, for simplicity of + # code, just corrupt the $expected_data_old buffer so it never matches + local expected_data_old=3D"POISON" + verify_data_blocks $verify_start $verify_end "$expected_data_old" "$expec= ted_data_new" +} + +$XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + +dry_run + +echo >> $seqres.full +echo "# Populating expected data buffers" >> $seqres.full +populate_expected_data + +# Loop 20 times to shake out any races due to shutdown +for ((iter=3D0; iter<20; iter++)) +do + echo >> $seqres.full + echo "------ Iteration $iter ------" >> $seqres.full + + echo >> $seqres.full + echo "# Starting torn write test for atomic writes over mixed mapping" >>= $seqres.full + test_torn_write_mixed + + echo >> $seqres.full + echo "# Starting torn write test for atomic writes over fully written map= ping" >> $seqres.full + test_torn_write_written + + echo >> $seqres.full + echo "# Starting torn write test for atomic writes over fully unwritten m= apping" >> $seqres.full + test_torn_write_unwritten + + echo >> $seqres.full + echo "# Starting torn write test for atomic writes over holes" >> $seqres= .full + test_torn_write_hole + + echo >> $seqres.full + echo "# Starting shutdown torn write test for append atomic writes" >> $s= eqres.full + test_append_torn_write +done + +echo "Silence is golden" +status=3D0 +exit diff --git a/tests/generic/1230.out b/tests/generic/1230.out new file mode 100644 index 00000000..d01f54ea --- /dev/null +++ b/tests/generic/1230.out @@ -0,0 +1,2 @@ +QA output created by 1230 +Silence is golden --=20 2.49.0 From nobody Thu Oct 2 07:43:36 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BDA272C2356; Fri, 19 Sep 2025 06:49:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264542; cv=none; b=rK3KnOXlrrD80YKgf8JWoxnsdGuHzctsmuJKwF7lrvTO+cE9sK/lX9qkBFe07oUkMIh1XmFBc0ihG34ljFQvOEjSfzzdIZ9FuOd2W2kroEArX5dAj2hb4lEjLaTcjEvW6gozk1i/PLGa3uvDGvbY0tYtFlEjxgqFC7eWZBrCGjs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264542; c=relaxed/simple; bh=/dAIy0nEihrCd/H2CLq+qnSEnaJT7cWIBOh5/Pwnnmw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jeKMris6wdQrqufpq1CdojgzDKTgFP2t428iYFUK7hzA50nVTZUHriTh20t6MvGp/1iR9HU4RwBHw49Wok7oaTVH32csom4mBonXlBC+uBKDa7eo0j4LsGIhGzDH1ULUCLLB/hSjs/0FY/9eWrOY4eUvGktW9DUNmDSffot5qjM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=LZaQA80U; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="LZaQA80U" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58J2U8X6011266; Fri, 19 Sep 2025 06:48:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=9DamNFw+yvGG3nqW3 SMA+ylj0WVex2u22K5E7mhw8/o=; b=LZaQA80UsR90BC2VFJiWN3mV9giiIthh1 QvVQNixUqaeeqp9TBUNGvY1dWxvqumKTM0Up4SKmXy4f7my02oRR9Asvmpp6OQRm kU8iLuta2qAz3ZyONRpCdB+HKLt8e5ZCQgqk06VAvr5an5QCaNRDrlHqC3yYUGTg ywRDEVCJwyRiSLt7kl9Ir4T977uT8e76bWNkF1bDcrHGt/r+QCd5412icsh+IghM lyY9FXHFCl+8k+5S/bTBNzd9wBqWNvAn0xV4TVdVVC7zAILx5cscbaiJBdEV1DXS usQstxNJSfiEmRActihFyCTGb0/wXelVz4HL2/5iL7wLYoAmf4wPw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4npbvx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:52 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58J6jvJt023032; Fri, 19 Sep 2025 06:48:52 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4npbvu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:51 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58J30khW008987; Fri, 19 Sep 2025 06:48:51 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 495nn3t9x9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:51 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58J6mn6Z56558058 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 19 Sep 2025 06:48:49 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 536E120040; Fri, 19 Sep 2025 06:48:49 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 300E82004B; Fri, 19 Sep 2025 06:48:46 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.51]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 19 Sep 2025 06:48:45 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v7 10/12] ext4: Test atomic write and ioend codepaths with bigalloc Date: Fri, 19 Sep 2025 12:18:03 +0530 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=MN5gmNZl c=1 sm=1 tr=0 ts=68ccfcd4 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=KekteOT-tD_VQUBAREkA:9 X-Proofpoint-GUID: uLWUbRL3MUXTw19gY0UjD8hcJtOhgd1K X-Proofpoint-ORIG-GUID: UIctzot_ScCafPkOo147-Q95M9C7fBk_ X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTE2MDIwNCBTYWx0ZWRfX6aioTMEJaAAG 3AdSrauEne0QrC5dX6DpmFFn/Dc5vFYC5V8haH0DeYtXFC5KEVfB1aIZUZt6bTKHu0EWJbl395Q iE413jfree2qsGiRPoZJ32sIYROaIhxxIWpzuI9lBSe2weZ1zQq1Q5PE6m7GQG/R1x9KY+eQrhA 5OdPZ+z628zmAKByF4YpNrTzwwmwSuGqmOTXwKNR9LXqCMMfLfrZmtJAaf3cL4+EsWCkhATjmEo NREmTRJ62tJLun+lURb+fcgHNKdLw3bdL63cd5oRmioYoImBWSsWkrYgzylcA23JOICQL52M1+N 5an/lk8bGoD6mxxhScxBXZYWzllgv2xz7/bCNWr52nAFyWI4R53S9l64isGpZW3ganY5MNHIz53 1VbkBAL6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-18_03,2025-09-19_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 spamscore=0 priorityscore=1501 bulkscore=0 impostorscore=0 malwarescore=0 adultscore=0 phishscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509160204 Content-Type: text/plain; charset="utf-8" From: "Ritesh Harjani (IBM)" This test does a lot of parallel RWF_ATOMIC IO on a preallocated file to stress the write and end-io unwritten conversion code paths. We brute force this for different blocksize and clustersizes and after each iteration we ensure the data was not torn or corrupted using fio crc verification. Note that in this test we use overlapping atomic writes of same io size. Although serializing racing writes is not guaranteed for RWF_ATOMIC, NVMe and SCSI provide this guarantee as an inseparable feature to power-fail atomicity. Keeping the iosize as same also ensures that ext4 doesn't tear the write due to racing ioend unwritten conversion. The value of this test is that we make sure the RWF_ATOMIC is handled correctly by ext4 as well as test that the block layer doesn't split or only generate multiple bios for an atomic write. Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- tests/ext4/061 | 155 +++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/061.out | 2 + 2 files changed, 157 insertions(+) create mode 100755 tests/ext4/061 create mode 100644 tests/ext4/061.out diff --git a/tests/ext4/061 b/tests/ext4/061 new file mode 100755 index 00000000..1d61c8b0 --- /dev/null +++ b/tests/ext4/061 @@ -0,0 +1,155 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 061 +# +# This test does a lot of parallel RWF_ATOMIC IO on a preallocated file to +# stress the write and end-io unwritten conversion code paths. We brute fo= rce +# this for all possible blocksize and clustersizes and after each iteratio= n we +# ensure the data was not torn or corrupted using fio crc verification. +# +# Note that in this test we use overlapping atomic writes of same io size. +# Although serializing racing writes is not guaranteed for RWF_ATOMIC, NVM= e and +# SCSI provide this guarantee as an inseparable feature to power-fail +# atomicity. Keeping the iosize as same also ensures that ext4 doesn't tea= r the +# write due to racing ioend unwritten conversion. +# +# The value of this test is that we make sure the RWF_ATOMIC is handled +# correctly by ext4 as well as test that the block layer doesn't split or = only +# generate multiple bios for an atomic write. + +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto rw stress atomicwrites + +_require_scratch_write_atomic +_require_fio_atomic_writes +_require_aiodio + +FIO_LOAD=3D$(($(nproc) * 2 * LOAD_FACTOR)) +SIZE=3D$((100*1024*1024)) + +# Calculate fsblocksize as per bdev atomic write units. +bdev_awu_min=3D$(_get_atomic_write_unit_min $SCRATCH_DEV) +bdev_awu_max=3D$(_get_atomic_write_unit_max $SCRATCH_DEV) +bs=3D$(_max 4096 "$bdev_awu_min") + +function create_fio_configs() +{ + local bsize=3D$1 + create_fio_aw_config $bsize + create_fio_verify_config $bsize +} + +function create_fio_verify_config() +{ + local bsize=3D$1 +cat >$fio_verify_config <$fio_aw_config <> $seqres.full 2>&1 || return + if _try_scratch_mount >> $seqres.full 2>&1; then + echo "=3D=3D Testing: bs=3D$bs cs=3D$cs iosize=3D$iosize =3D=3D" >> $seq= res.full + + touch $SCRATCH_MNT/f1 + create_fio_configs $iosize + + cat $fio_aw_config >> $seqres.full + echo >> $seqres.full + cat $fio_verify_config >> $seqres.full + + $FIO_PROG $fio_aw_config >> $seqres.full + ret1=3D$? + + $FIO_PROG $fio_verify_config >> $seqres.full + ret2=3D$? + + _scratch_unmount + + [[ $ret1 -eq 0 && $ret2 -eq 0 ]] || _fail "fio with atomic write failed" + fi +} + +run_test() { + local bs=3D$1 + + # cluster sizes above 16 x blocksize are experimental so avoid them + # Also, cap cluster size at 128kb to keep it reasonable for large + # blocks size + max_cs=3D$(_min $((16 * bs)) "$bdev_awu_max" $((128 * 1024))) + + # Fuzz for combinations of blocksize, clustersize and + # iosize that cover most of the cases + run_test_one $bs $bs $bs + run_test_one $bs $max_cs $bs + run_test_one $bs $max_cs $max_cs + run_test_one $bs $max_cs $(_max "$((max_cs/2))" $bs) +} + +# Let's create a sample fio config to check whether fio supports all optio= ns. +fio_aw_config=3D$tmp.aw.fio +fio_verify_config=3D$tmp.verify.fio +fio_out=3D$tmp.fio.out + +create_fio_configs $bs +_require_fio $fio_aw_config + +for ((bs=3D$bs; bs <=3D $(_get_page_size); bs =3D $bs << 1)); do + run_test $bs +done + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/ext4/061.out b/tests/ext4/061.out new file mode 100644 index 00000000..273be9e0 --- /dev/null +++ b/tests/ext4/061.out @@ -0,0 +1,2 @@ +QA output created by 061 +Silence is golden --=20 2.49.0 From nobody Thu Oct 2 07:43:36 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BA04729C33D; Fri, 19 Sep 2025 06:49:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264552; cv=none; b=CmC/jClW2hhTR0nPN3lpuTFnR2BbRof6wVspqkCOvq/tsozUdUtsySGpX0EMG75xH0x8maAEeMJfhGYwy+RG0QuWY4quiss1AH89NOGUvBNyzzXN8RaaOox1Z/eYviNNaTSx7+6da8ijDg7MwpJEE10BiV9plWZBJP2JgLxQo2M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264552; c=relaxed/simple; bh=AbuoGFdbt5Nlsepwq1pcM/3iTAcG8d6i8S+4mWjlAYE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TXwfLxyQ4Y9tLhKtgLPZr3em62QKEXI1wnHocHQK90wBQRR0n0HM5zIjt8ZW4d+abw/POm8l8x+GeCTmKSQkRAdX958Y3SR/2M5CmzMzhOv8f/x29wFH6spmjSok1KDeUUjuktYP9hDsrf4eTc3kLDhUKE5W+QkyajQAh524Gqc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Ac9TCyC0; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Ac9TCyC0" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58J0Q89j027677; Fri, 19 Sep 2025 06:48:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=DTRxxUzb0Dsn0/GBD P5OgVL94Y67Y6ZSK7VjxTYr85w=; b=Ac9TCyC0G0vyJslvG200Ekf99+najjGH+ +Fuyuew/vtnjzh5TENYxYL4BtuX987XL0YsOiElENY+19PHZhLteojR9P6vQnY3w IGaIz0yecE5mkK4WopXoBCmoK/gXO2sLCKrcoD6GO1yO+0I92LJg/5DIVUQmBoz3 m/0V6ONBSrXCkj4fdSJI7oOC2pWmThXByQKEDWKRnJevT+HFCW1S/h9UjJj6/MNw tRqHlPHDrqeQVYEXfn7xQC+b2kcidSPFEQkAP11TV/s9RJwSMOY84tKgVUbFIcyM 0LWNrKmOXmeAtAe20+gtmqTZTM1PJ3E0LE8eyFie/i1ZgFwCpCf+Q== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4pf155-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:57 +0000 (GMT) Received: from m0360083.ppops.net (m0360083.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58J6mnBE016892; Fri, 19 Sep 2025 06:48:56 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4pf153-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:56 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58J4lk1o029498; Fri, 19 Sep 2025 06:48:55 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 495kb1atr4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:55 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58J6mruQ34276046 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 19 Sep 2025 06:48:53 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 51AC120043; Fri, 19 Sep 2025 06:48:53 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D851820040; Fri, 19 Sep 2025 06:48:49 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.51]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 19 Sep 2025 06:48:49 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v7 11/12] ext4: Test atomic writes allocation and write codepaths with bigalloc Date: Fri, 19 Sep 2025 12:18:04 +0530 Message-ID: <98a68b6e35ef0b810ea46c007fbece75d41b911b.1758264169.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTE2MDIwNCBTYWx0ZWRfX4ZjqxM5x2xZv 3MNNbIK3hT7DRcJU6Y4RindtEHSgCyIEhejI++8Be2clDwwNSGjAMlORFPqazGa78MmQKQl1z+7 ygkvDrjy9pGY7CNVwwgy+WtuZ5FSzKLHcwytmZ6VbVhTZ6mRVTiiuVHt2iA8itjr0s/TbtKK1dQ GQPylO/b5qRV9qgzzflCEexjocwums6qFqkNy5OFuQ4o+5aqe0YJ5KZxriW5YD5RCnFm8v9UBXG p3mivpVW6xJHgmx2Zp8syCKPP5RZNktqam3w08j/c5dAvDm9tk5e3PoKoyHAQ7V9YQDxtE5IGoW 1NnzWDyqVxMIXlubXBf+K7Ynds/xVRrt51gQPVrVdkDuZGO0l4dLGosJ62sWMnDHlj1/RVLOPIc WYhzVT+P X-Proofpoint-ORIG-GUID: tZlZ9dwt5-8YiUkgycuNsO5dbUNbp_NH X-Proofpoint-GUID: 2ZZhgRa_VA7Tp4JGVyU8kexT_2uZ6cko X-Authority-Analysis: v=2.4 cv=cNzgskeN c=1 sm=1 tr=0 ts=68ccfcd9 cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=o-vKpyFH2n2aPF30nXYA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-18_03,2025-09-19_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 clxscore=1015 spamscore=0 bulkscore=0 malwarescore=0 adultscore=0 priorityscore=1501 impostorscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509160204 Content-Type: text/plain; charset="utf-8" From: "Ritesh Harjani (IBM)" This test does a parallel RWF_ATOMIC IO on a multiple truncated files in a small FS. The idea is to stress ext4 allocator to ensure we are able to handle low space scenarios correctly with atomic writes. We brute force this for different blocksize and clustersizes and after each iteration we ensure the data was not torn or corrupted using fio crc verification. Note that in this test we use overlapping atomic writes of same io size. Although serializing racing writes is not guaranteed for RWF_ATOMIC, NVMe and SCSI provide this guarantee as an inseparable feature to power-fail atomicity. Keeping the iosize as same also ensures that ext4 doesn't tear the write due to racing ioend unwritten conversion. The value of this test is that we make sure the RWF_ATOMIC is handled correctly by ext4 as well as test that the block layer doesn't split or only generate multiple bios for an atomic write. Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- tests/ext4/062 | 203 +++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/062.out | 2 + 2 files changed, 205 insertions(+) create mode 100755 tests/ext4/062 create mode 100644 tests/ext4/062.out diff --git a/tests/ext4/062 b/tests/ext4/062 new file mode 100755 index 00000000..05cce696 --- /dev/null +++ b/tests/ext4/062 @@ -0,0 +1,203 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 062 +# +# This test does a parallel RWF_ATOMIC IO on a multiple truncated files in= a +# small FS. The idea is to stress ext4 allocator to ensure we are able to +# handle low space scenarios correctly with atomic writes.. We brute force= this +# for all possible blocksize and clustersizes and after each iteration we +# ensure the data was not torn or corrupted using fio crc verification. +# +# Note that in this test we use overlapping atomic writes of same io size. +# Although serializing racing writes is not guaranteed for RWF_ATOMIC, NVM= e and +# SCSI provide this guarantee as an inseparable feature to power-fail +# atomicity. Keeping the iosize as same also ensures that ext4 doesn't tea= r the +# write due to racing ioend unwritten conversion. +# +# The value of this test is that we make sure the RWF_ATOMIC is handled +# correctly by ext4 as well as test that the block layer doesn't split or = only +# generate multiple bios for an atomic write. +# + +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto rw stress atomicwrites + +_require_scratch_write_atomic +_require_fio_atomic_writes +_require_aiodio + +FSSIZE=3D$((360*1024*1024)) +FIO_LOAD=3D$(($(nproc) * LOAD_FACTOR)) + +# Calculate bs as per bdev atomic write units. +bdev_awu_min=3D$(_get_atomic_write_unit_min $SCRATCH_DEV) +bdev_awu_max=3D$(_get_atomic_write_unit_max $SCRATCH_DEV) +bs=3D$(_max 4096 "$bdev_awu_min") + +function create_fio_configs() +{ + local bsize=3D$1 + create_fio_aw_config $bsize + create_fio_verify_config $bsize +} + +function create_fio_verify_config() +{ + local bsize=3D$1 +cat >$fio_verify_config <$fio_aw_config <> $seqres.full 2>&1 || return + if _try_scratch_mount >> $seqres.full 2>&1; then + echo "Testing: bs=3D$bs cs=3D$cs iosize=3D$iosize" >> $seqres.full + + touch $SCRATCH_MNT/f1 + create_fio_configs $iosize + + cat $fio_aw_config >> $seqres.full + cat $fio_verify_config >> $seqres.full + + $FIO_PROG $fio_aw_config >> $seqres.full + ret1=3D$? + + $FIO_PROG $fio_verify_config >> $seqres.full + ret2=3D$? + + _scratch_unmount + + [[ $ret1 -eq 0 && $ret2 -eq 0 ]] || _fail "fio with atomic write failed" + fi +} + +run_test() { + local bs=3D$1 + + # cluster sizes above 16 x blocksize are experimental so avoid them + # Also, cap cluster size at 128kb to keep it reasonable for large + # blocks size + max_cs=3D$(_min $((16 * bs)) "$bdev_awu_max" $((128 * 1024))) + + # Fuzz for combinations of blocksize, clustersize and + # iosize that cover most of the cases + run_test_one $bs $bs $bs + run_test_one $bs $max_cs $bs + run_test_one $bs $max_cs $max_cs + run_test_one $bs $max_cs $(_max "$((max_cs/2))" $bs) +} + +# Let's create a sample fio config to check whether fio supports all optio= ns. +fio_aw_config=3D$tmp.aw.fio +fio_verify_config=3D$tmp.verify.fio +fio_out=3D$tmp.fio.out + +create_fio_configs $bs +_require_fio $fio_aw_config + +for ((bs=3D$bs; bs <=3D $(_get_page_size); bs =3D $bs << 1)); do + run_test $bs $cs $iosize +done + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/ext4/062.out b/tests/ext4/062.out new file mode 100644 index 00000000..a1578f48 --- /dev/null +++ b/tests/ext4/062.out @@ -0,0 +1,2 @@ +QA output created by 062 +Silence is golden --=20 2.49.0 From nobody Thu Oct 2 07:43:36 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A16DC277C95; Fri, 19 Sep 2025 06:49:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264548; cv=none; b=VjKQUe+57da4rtVexuKK2lkhAj0n+Lyy9TD3JNlDwvu5ObDdpDJFBKalJuv2xsr/cgD/Z7qHJLwW792hxEARr0fVUmWieZkxOFB/8yDhkuc/D72hwqX+7f5aBFjX0q/e7X/aawYv89Z51F7u71nT/SBsYjv6kFtQ1ZDrvThhfsU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758264548; c=relaxed/simple; bh=/23TnMcmUpdflLUBR/TKi4uqya8y8NfSo8bYiPAZYPk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fdjLPEmJRUXuk6HcTD9qkxVHqnKBACpBeeNQwKkhY7vzh1yExbpZluLxVKTUXC6XbxHzIlgyN33HrtRdrmu3DdScgoE1OBEvYfUwVXyxMPl1/hp1tz8pAvj/ranY1yKihinGKYHTbDzR4klIhP/yMrag38cgV4kVJ2uvkQuby2I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=dixeyOKm; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="dixeyOKm" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58INLuSD023783; Fri, 19 Sep 2025 06:49:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=vRTNUGKdoq8A+YlHn I5gzyY7Rjkkw4gAzlowW/XKnQg=; b=dixeyOKmFhOlH03bWANFowZ6qWNru7M0B 4l4Ep41RmoJS9wYH9jhbBbtZrKnCpRXIuCPqqUP91ZcAhy/yl1v5tepV5cmt+dyS MH5+G50FdpXyqeg9ER0xETAaWMd5vgd+mjge2ngyip4qtNu+jYYPvdoZIMWuYLJs pF6LVVRXAtIrRA5stwxunm3o4qM5HWIHPsEBUy/3nviYxTLHciU6Tdf8s12bl8ls sfjNO7ERV6HmW2IK3D2nvfE3SYKxU3i5zxJjEAkPziEfjl3ggxBCg35HSoVL89OM fCIK6POyOiGCipA4qRBRLK8lLP6gBwtmQlivfYuCBdx2zeCQxxO5A== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4qxatp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:49:00 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58J6kx81022278; Fri, 19 Sep 2025 06:48:59 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 497g4qxatk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:59 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58J51vwm029468; Fri, 19 Sep 2025 06:48:59 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 495kb1atr9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Sep 2025 06:48:58 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58J6muNb56558066 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 19 Sep 2025 06:48:57 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C71ED20040; Fri, 19 Sep 2025 06:48:56 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EEBF220043; Fri, 19 Sep 2025 06:48:53 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.124.215.51]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 19 Sep 2025 06:48:53 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v7 12/12] ext4: Atomic write test for extent split across leaf nodes Date: Fri, 19 Sep 2025 12:18:05 +0530 Message-ID: <721505bb10ec191d93f0612f5ad98b864ba5c980.1758264169.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: wm3CUYiP2jcwdxdBnktZHmOUjCLCpJE9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTE2MDIwNCBTYWx0ZWRfX+dEkyR9DnSsM ivR3pqJnB5O5F3egjsOokko1KlRm/StTwbbiZjFEqKAIBt7t/M+KBNm6CeC/iSFtlnFpweUSRKe CNIzmt2QzAQuyHGUNFfD3Ad0bfnsQn6FXlkojp9l0+RhYcxIuCqmAYuBsUwI6Hf2B7g0oKYWSLn h8BddfsZOSKUzFXSAcjBFVsX5/lAjR7PLz8BTK7qykJJgJXlnj726WDie4RkgHQo+Zp9UBpP4l8 eKzhq0GaYAKklM8mwrBHXC/TcvOhm3zggtgXvlJuGAaREHJLsvCovN+/PqrLo2XphdQm9hOpH6W aw0nPD06sdjVYyJU45rrtmr6ccOIiHaBYbj6DvfHppn/4ujEwP2fAdc1lLJn4LYSoKBTUDbxepy jow2iNFL X-Authority-Analysis: v=2.4 cv=R8oDGcRX c=1 sm=1 tr=0 ts=68ccfcdc cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=yJojWOMRYYMA:10 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=BtNrGSYHu5IGmLxdGZgA:9 X-Proofpoint-GUID: L13-LolBoW-NPJ0NajznfdqwYExF1W6G X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-18_03,2025-09-19_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 suspectscore=0 malwarescore=0 bulkscore=0 spamscore=0 adultscore=0 impostorscore=0 priorityscore=1501 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509160204 Content-Type: text/plain; charset="utf-8" In ext4, even if an allocated range is physically and logically contiguous, it can still be split into 2 extents. This is because ext4 does not merge extents across leaf nodes. This is an issue for atomic writes since even for a continuous extent the map block could (in rare cases) return a shorter map, hence tearning the write. This test creates such a file and ensures that the atomic write handles this case correctly Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo --- tests/ext4/063 | 129 +++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/063.out | 2 + 2 files changed, 131 insertions(+) create mode 100755 tests/ext4/063 create mode 100644 tests/ext4/063.out diff --git a/tests/ext4/063 b/tests/ext4/063 new file mode 100755 index 00000000..9d6265a8 --- /dev/null +++ b/tests/ext4/063 @@ -0,0 +1,129 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# In ext4, even if an allocated range is physically and logically contiguo= us, +# it can still be split into 2 or more extents. This is because ext4 does = not +# merge extents across leaf nodes. This is an issue for atomic writes since +# even for a continuous extent the map block could (in rare cases) return a +# shorter map, hence tearing the write. This test creates such a file and +# ensures that the atomic write handles this case correctly +# +. ./common/preamble +. ./common/atomicwrites +_begin_fstest auto atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_atomic_write_test_commands +_require_command "$DEBUGFS_PROG" debugfs + +prep() { + local bs=3D`_get_block_size $SCRATCH_MNT` + local ex_hdr_bytes=3D12 + local ex_entry_bytes=3D12 + local entries_per_blk=3D$(( (bs - ex_hdr_bytes) / ex_entry_bytes )) + + # fill the extent tree leaf with bs len extents at alternate offsets. + # The tree should look as follows + # + # +---------+---------+ + # | index 1 | index 2 | + # +-----+---+-----+---+ + # +------+ +-----------+ + # | | + # +-------+-------+---+---------+ +-----+----+ + # | ex 1 | ex 2 | | ex n | | ex n+1 | + # | off:0 | off:2 |...| off:678 | | off:680 | + # | len:1 | len:1 | | len:1 | | len:1 | + # +-------+-------+---+---------+ +----------+ + # + for i in $(seq 0 $entries_per_blk) + do + $XFS_IO_PROG -fc "pwrite -b $bs $((i * 2 * bs)) $bs" $testfile > /dev/nu= ll + done + sync $testfile + + echo >> $seqres.full + echo "Create file with extents spanning 2 leaves. Extents:">> $seqres.full + echo "...">> $seqres.full + $DEBUGFS_PROG -R "ex `basename $testfile`" $SCRATCH_DEV |& tail >> $seqre= s.full + + # Now try to insert a new extent ex(new) between ex(n) and ex(n+1). + # Since this is a new FS the allocator would find continuous blocks + # such that ex(n) ex(new) ex(n+1) are physically(and logically) + # contiguous. However, since we don't merge extents across leaf we will + # end up with a tree as: + # + # +---------+---------+ + # | index 1 | index 2 | + # +-----+---+-----+---+ + # +------+ +------------+ + # | | + # +-------+-------+---+---------+ +------+-----------+ + # | ex 1 | ex 2 | | ex n | | ex n+1 (merged) | + # | off:0 | off:2 |...| off:678 | | off:679 | + # | len:1 | len:1 | | len:1 | | len:2 | + # +-------+-------+---+---------+ +------------------+ + # + echo >> $seqres.full + torn_ex_offset=3D$((((entries_per_blk * 2) - 1) * bs)) + $XFS_IO_PROG -c "pwrite $torn_ex_offset $bs" $testfile >> /dev/null + sync $testfile + + echo >> $seqres.full + echo "Perform 1 block write at $torn_ex_offset to create torn extent. Ext= ents:">> $seqres.full + echo "...">> $seqres.full + $DEBUGFS_PROG -R "ex `basename $testfile`" $SCRATCH_DEV |& tail >> $seqre= s.full + + _scratch_cycle_mount +} + +_scratch_mkfs >> $seqres.full +_scratch_mount >> $seqres.full + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile +awu_max=3D$(_get_atomic_write_unit_max $testfile) + +echo >> $seqres.full +echo "# Prepping the file" >> $seqres.full +prep + +torn_aw_offset=3D$((torn_ex_offset - (torn_ex_offset % awu_max))) + +echo >> $seqres.full +echo "# Performing atomic IO on the torn extent range. Command: " >> $seqr= es.full +echo $XFS_IO_PROG -c "open -fsd $testfile" -c "pwrite -S 0x61 -DA -V1 -b $= awu_max $torn_aw_offset $awu_max" >> $seqres.full +$XFS_IO_PROG -c "open -fsd $testfile" -c "pwrite -S 0x61 -DA -V1 -b $awu_m= ax $torn_aw_offset $awu_max" >> $seqres.full + +echo >> $seqres.full +echo "Extent state after atomic write:">> $seqres.full +echo "...">> $seqres.full +$DEBUGFS_PROG -R "ex `basename $testfile`" $SCRATCH_DEV |& tail >> $seqres= .full + +echo >> $seqres.full +echo "# Checking data integrity" >> $seqres.full + +# create a dummy file with expected data +$XFS_IO_PROG -fc "pwrite -S 0x61 -b $awu_max 0 $awu_max" $testfile.exp >> = /dev/null +expected_data=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp) + +# We ensure that the data after atomic writes should match the expected da= ta +actual_data=3D$(od -An -t x1 -j $torn_aw_offset -N $awu_max $testfile) +if [[ "$actual_data" !=3D "$expected_data" ]] +then + echo "Checksum match failed at off: $torn_aw_offset size: $awu_max" + echo + echo "Expected: " + echo "$expected_data" + echo + echo "Actual contents: " + echo "$actual_data" + + _fail +fi + +echo -n "Data verification at offset $torn_aw_offset succeeded!" >> $seqre= s.full +echo "Silence is golden" +status=3D0 +exit diff --git a/tests/ext4/063.out b/tests/ext4/063.out new file mode 100644 index 00000000..de35fc52 --- /dev/null +++ b/tests/ext4/063.out @@ -0,0 +1,2 @@ +QA output created by 063 +Silence is golden --=20 2.49.0