From nobody Thu Oct 2 19:24:13 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8AF3636207B; Thu, 11 Sep 2025 17:14:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610844; cv=none; b=CpkWuoODA/UD2G3+aIaUpGiRDES10/YTaEi2m7QmANFce2qZzuYNrEqDdtSiXLr40dtTZYWE6Z0YW3kk/erwP4+AKnjfYfrOo4TCnDiovQR17ff/+4S/QnkSsGH1a3+aRKAvjwBo7tl1VfO7GmITsn7723VY2A/B2KrmaxvUDaw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610844; c=relaxed/simple; bh=w+HAnnjKd3GJVEihEy0k9c5/uycjCPovrRjLi95YrnE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=S/LUvCjfkjm5VJQ05r3OD4+RxwopENVL9kVbb9VU13nJv6rIKT8OQ6LEuk1/P9uVE0ZXXLfz7SiIXmRxgBNo+hx8wvZNbapz69XngQV6YfbWJOqql9xUlwZR7xk+KU9l2xvv7DxNMzS24SiCT7Zb3bp4cvr6p/r1+N3wdpa70BI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=i/sI+mtq; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="i/sI+mtq" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BB05Qj009190; Thu, 11 Sep 2025 17:13:55 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=4w8BDwO1ZUf0Evazr ueiJY6KTF9inKx36aKsZrlnj8c=; b=i/sI+mtq9aLN6XcA5Pe9u69DZccSTGKcs d5g+cLjl1xEuRhBy3Ft26oTxeKpHaZdRlUTFsVAit5Q8jEL70SmTK/a6hOjGR1cH UdJznYw1RJKyMzjNKMU97ggNRpnKD0ALOZ+LJ9s05vo3dlSzS08J8HfyprTP5kxV Uy25Al636aY65fI6wyE207vdAlh85kFfAE4ccZ9zISyn7niLq+10zB9mkLI9aaPT Zoi8+d2FE7VKAzBDydf5PCZZks/FyNAcohNhVDQB2e8X4JWImwkI48/oMmDNrHVY qFc13CQjYAOonvtyK1/JyQyDpnGXUjblFRk0/tjU51fBxuwOiBTNg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490acrdgfc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:13:55 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58BH8sXN024933; Thu, 11 Sep 2025 17:13:54 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490acrdgf8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:13:54 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BGgkkO011435; Thu, 11 Sep 2025 17:13:53 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 490y9uq52m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:13:53 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BHDpVu53149964 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 17:13:52 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D7AC820063; Thu, 11 Sep 2025 17:13:51 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 727CC2004F; Thu, 11 Sep 2025 17:13:48 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.39.17.37]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 17:13:48 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v6 01/12] common/rc: Add _min() and _max() helpers Date: Thu, 11 Sep 2025 22:43:32 +0530 Message-ID: <9475f8da726b894dd152b54b1416823181052c2a.1757610403.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: XeNOgZgpvvw0Ue7ocIDcr884w84A20b3 X-Authority-Analysis: v=2.4 cv=Mp1S63ae c=1 sm=1 tr=0 ts=68c30353 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=yJojWOMRYYMA:10 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=wDx1mg2EYdB0JXDzLs4A:9 X-Proofpoint-ORIG-GUID: DkDTP1kNrScJXzhjjB6Pi-JU_pVfvOeN X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDAwMCBTYWx0ZWRfX43miWC2CRvZ5 ezGke3OYi6qAgjk1cpwebThQsAPGkRAoLe00VK8649ehFHby9qft+Epd0glqQQdN5a8OP3BiQDA OiGdyhErfJFFLWhvZ39ipzMo8okwLGg0Z5x1M+6TGf0WVpfpzkSW6Wt/VCCFBIUKvsufUPPV8jD gF/L8YLytxMoSyvfZjCVYt+fHopCFDTjoLHpX8ojqkBK1RAMbkrdilWcSoFBKHZtxhYpZoX1Y6O 9HjTvKwA0K2JkTxzaDyKnCEjygdwwVKpuv9K4WXtWyHvceRjEE5qAMc4Z9vfqoSrcbFN88jWYrx tuJKD1FgBwSqdK0o9CbbZVMC7FXKGKuNT+irP2zztlBx5jxg/gBgczXchcY3EnbHAVpEQNqZoRr bMc5Cofr X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_02,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 malwarescore=0 clxscore=1015 phishscore=0 spamscore=0 adultscore=0 priorityscore=1501 bulkscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060000 Content-Type: text/plain; charset="utf-8" Many programs open code these functionalities so add it as a generic helper in common/rc Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo Reviewed-by: John Garry --- common/rc | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/common/rc b/common/rc index 81587dad..28fbbcbb 100644 --- a/common/rc +++ b/common/rc @@ -5978,6 +5978,28 @@ _require_inplace_writes() fi } =20 +_min() { + local ret + + for arg in "$@"; do + if [ -z "$ret" ] || (( $arg < $ret )); then + ret=3D"$arg" + fi + done + echo $ret +} + +_max() { + local ret + + for arg in "$@"; do + if [ -z "$ret" ] || (( $arg > $ret )); then + ret=3D"$arg" + fi + done + echo $ret +} + ##########################################################################= ###### # make sure this script returns success /bin/true --=20 2.49.0 From nobody Thu Oct 2 19:24:13 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C4CE36207B; Thu, 11 Sep 2025 17:14:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610849; cv=none; b=TrpD+X9YuOEIXSEx7eRDBSRyLyzyLXcC78tg88jYF7/kbifZ2rWSn3k/IiFIfwIqYTdtJwdP4FG2MCqgBGpqZ7TdFQZtwLYe7Q0EnM63cId43sjUduc8Bmo27L9Gk2A+fjdQJ3E35llDID6N6lndDH2X1wJSgSFYcg6jh60AZ5g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610849; c=relaxed/simple; bh=W5XD++tnY8d0E4SXC6W9CchT17TpiD9tJGLiVrNOR6c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pk1KpPAlpgYEYr1JLK4Pl2vKPvtUXrsylpt1IhqH1QQATdkJC4OdFDz2c4nBn6U7fDiegBxl3dGpaDgkZGYpoZZ28UAE6dzaMeQmcu+EgSRUq4GchaXKtOf5tvHWY1oU3JWL7NBDrUjmv6Ewkg2xyVh9BKLnJJkCQhJCuqiigeM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=TtNh+pnX; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="TtNh+pnX" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BG6U9q008043; Thu, 11 Sep 2025 17:13:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=rNdvYmkqmpP8HC0yG I10lE8tHceC+TYn9c1ZHNzN2dY=; b=TtNh+pnXuiiVzAMgK2xusLcp93Ud2GwC3 fqzabSOQ317y4ppWvlr4Tgycrp1jauEWd5rpV6IYHEPHDDAvdgNs1KDzj7UohfbU tw4dqTJ8z8Rak70NBqwW3Qo2b1yiVLCcUj1xO+4MQmdlphGnZTtqzsC86JVecFt0 4MWDR6q43rTX5LhiYv81yrE7knb9VPkrwiz6tknq49+oVPODjbvcPecI+PGRx8G4 wwoMpNA5WvRpjzpLhcBsKK+65hPDXpMGXiHD3dcwH72OqnfuI0W2TZ+htLOLfqT/ Kz2WY+d5yINlVmgLfaLF7FrT/5yLR3utlLr0Hgp1vHbh1MOOuWMdA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490uketu17-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:13:59 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58BH6fsS029908; Thu, 11 Sep 2025 17:13:58 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490uketu14-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:13:58 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BH3c1p007929; Thu, 11 Sep 2025 17:13:57 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 49109pxvw0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:13:57 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BHDtR250397690 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 17:13:56 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D1DCF2004E; Thu, 11 Sep 2025 17:13:55 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8684420043; Thu, 11 Sep 2025 17:13:52 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.39.17.37]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 17:13:52 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v6 02/12] common/rc: Add fio atomic write helpers Date: Thu, 11 Sep 2025 22:43:33 +0530 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDE5NSBTYWx0ZWRfX3LlXug2v7ngr h0J1i8hMku8O1H6kBy/dY6BppfQLr+yuum0eG8V2RZIbfoAxJCsMP0Ror3oqhG3QQofQPrKxcOD 112GTDVyZ0in36EMrU/qXKb8BGlaQlVQrQ11CVeFbNnCukYt42k1ijvYKcGpeibmu0PgzNKzaXw 7BBtrOwERUs7jFXKwNXXjW/1uC86ieOQjp3gx78zI7wyFmh75Ww958qSlZJ2uUm04KMGlrdn45A Ej9iaf5jwLXFimbSKe/lh/LuqsLQbepmB3Won4Zp1hYdYpv4RMFzvAxMfczs3juZYXoY5woAfPj zX3os7vCq+7zJ3PbEJyKxlxdOEjgI2xNbiqEy/XMCZLw/E1X3WOeUGllkHWBUSkp5453ZlOlt/q 4rbhxb2C X-Proofpoint-ORIG-GUID: DnVKLSROxWaAh090llgYuxDbJ5pgLaRs X-Proofpoint-GUID: bMW8BhXSre6hCQzO1jBsUNMDaRyDSAQt X-Authority-Analysis: v=2.4 cv=StCQ6OO0 c=1 sm=1 tr=0 ts=68c30357 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=yJojWOMRYYMA:10 a=20KFwNOVAAAA:8 a=VnNF1IyMAAAA:8 a=nFHpeJSrNhMDmVjzN9sA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_02,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 malwarescore=0 bulkscore=0 clxscore=1015 adultscore=0 suspectscore=0 priorityscore=1501 impostorscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060195 Content-Type: text/plain; charset="utf-8" The main motivation of adding this function on top of _require_fio is that there has been a case in fio where atomic=3D option was added but later it was changed to noop since kernel didn't yet have support for atomic writes. It was then again utilized to do atomic writes in a later version, once kernel got the support. Due to this there is a point in fio where _require_fio w/ atomic=3D1 will succeed even though it would not be doing atomic writes. Hence, add an internal helper __require_fio_version to require specific versions of fio to work past such issues. Further, add the high level _require_fio_atomic_writes helper which tests can use to ensure fio has the right version for atomic writes. Reviewed-by: Zorro Lang Signed-off-by: Ojaswin Mujoo Reviewed-by: John Garry --- common/rc | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/common/rc b/common/rc index 28fbbcbb..8a023b9d 100644 --- a/common/rc +++ b/common/rc @@ -6000,6 +6000,49 @@ _max() { echo $ret } =20 +# Due to reasons explained in fio commit 40f1fc11d, fio version between +# v3.33 and v3.38 have atomic=3D feature but it is a no-op and doesn't do +# RWF_ATOMIC write. Hence, use this helper to ensure fio has the +# required support. Currently, the simplest way we have is to ensure +# the version. +_require_fio_atomic_writes() { + __require_fio_version "3.38+" +} + +# Check the required fio version. Examples: +# __require_fio_version 3.38 (matches 3.38 only) +# __require_fio_version 3.38+ (matches 3.38 and above) +# __require_fio_version 3.38- (matches 3.38 and below) +# +# Internal helper, avoid using directly in tests. +__require_fio_version() { + local req_ver=3D"$1" + local fio_ver + + _require_fio + _require_math + + fio_ver=3D$(fio -v | cut -d"-" -f2) + + case "$req_ver" in + *+) + req_ver=3D${req_ver%+} + test $(_math "$fio_ver >=3D $req_ver") -eq 1 || \ + _notrun "need fio >=3D $req_ver (found $fio_ver)" + ;; + *-) + req_ver=3D${req_ver%-} + test $(_math "$fio_ver <=3D $req_ver") -eq 1 || \ + _notrun "need fio <=3D $req_ver (found $fio_ver)" + ;; + *) + req_ver=3D${req_ver%-} + test $(_math "$fio_ver =3D=3D $req_ver") -eq 1 || \ + _notrun "need fio =3D $req_ver (found $fio_ver)" + ;; + esac +} + ##########################################################################= ###### # make sure this script returns success /bin/true --=20 2.49.0 From nobody Thu Oct 2 19:24:13 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B13D6362060; Thu, 11 Sep 2025 17:14:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610852; cv=none; b=VDA7VVNoHTiptq1kx9a6y0GvpDbkNqEOYbdYkKo4TPMiI9N93z7bdsLs4e6OLNJrl2a6Lv9MeiqxKH/tsyoKQSvR3vPKFn/blqqRsGBjW9ajdLFyoRpWm5H+YJ/gJYlOpS/+AUpiNeAr0au3KY4kEffL7ubc+ZLESLGEz9GTJ4o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610852; c=relaxed/simple; bh=6IuWXHYorbKtU5KjphGwTcVneKOi+9yl5hrmwhjpl5I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mjGv6r5gV3AJVKonmCmBFUgGXh4HmiJxmA9pVzEydZ5fdh00nfVlsvQ7RdUsQ3Ou87eawnjkNemPQOlZfn6iLKrjAd5IabjDJxUp0gmahrKs++LgXQUW7Jf1LQzADNtQgMlonLEZwqeUsQtpelEqPEhELnpZHVrJsW8CLtMvh3I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=JlHmvMoG; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="JlHmvMoG" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BBnATP021232; Thu, 11 Sep 2025 17:14:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=DN0Ra6Sl46iaxQ+Ne 5dURRCrbnWTuvAamB6MEUibkRg=; b=JlHmvMoGBfVG7yLqbYiKLxuM1ty7UBHRa NVn9LY155KG1Ydiya+eSTaySIR/bgRY/bIuNt3FVw0B9jpp3RnmgpePoQ+0eqeE/ mTiyxAU4jt5ZyZPnBXgqJqI4uor4QaabgXiymDi5HlJVnDeILIs31gd0NqPxgFNc Z8xCXx7UAuIcRMhkdKBOlMV8SJvpmOQco3DLMsAqTwgHeAqDQ2A/Bv7TmZ+/Ra+p /FAki/27xJnRBdlm18kfzh4FmEHCB/SUmh1az/2Y2bbUANz3+WtYNz1uTItTKvJr AT2au4K1fxnPn4sICYpjciomYggzd4rB3+QD10dx3cwubZpspd1Ow== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490uketu1f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:03 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58BHCdPF010852; Thu, 11 Sep 2025 17:14:02 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490uketu1c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:02 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BGNBOF020499; Thu, 11 Sep 2025 17:14:01 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 490yp171tq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:01 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BHDxUx33423658 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 17:13:59 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C8D4F2004E; Thu, 11 Sep 2025 17:13:59 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6B2F22005A; Thu, 11 Sep 2025 17:13:56 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.39.17.37]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 17:13:56 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v6 03/12] common/rc: Add a helper to run fsx on a given file Date: Thu, 11 Sep 2025 22:43:34 +0530 Message-ID: <18c7546ebcb6dbf72d591f3c6f6b29101aa48b95.1757610403.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDE5NSBTYWx0ZWRfX4K9UXJeFaGOQ GgnucfMrpjoW7rqAFQ6IdINU7RA6+PZp92f2Q4Rl8oYwqspBOdVhm7KaK9abZVCWLgMVtRT/+Hx A12LTyXAFyOqTO97V2KwjB6kN7D5n5nJxH05+jGKcqbBIhvWT+7zrg4ZOVdBhnOU63L5RUXtI6M 5euP46jMwfNyPknL7bYrMmvgSFpPSVPLf5w5qt2JPXCBEcbIUHlvBBMsOxEfXQBmYgtKL+0wueS a2YLJhD38O/vECqXofrdLKdpNLKvymYByzRaFxHebfuhLojN3UYc6tXPQVP2Zy2ejOs/Y7G8q3I +JcQVFNAfZ/NoDXduTJX2pcGzYs2+7mte8FnAJtPynkMaIK0MJQ56odm3zhtzaGPdxOZeF7+bZE f4TEcRAh X-Proofpoint-ORIG-GUID: aUsRPh8vz-wrOmch53l4wHefYSOFMP_t X-Proofpoint-GUID: EeKEIUW_o4SiP1PyBIK6o7oWO9bFRYjp X-Authority-Analysis: v=2.4 cv=StCQ6OO0 c=1 sm=1 tr=0 ts=68c3035b cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=u_VYNAqGXo6OSSAA:21 a=yJojWOMRYYMA:10 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=EfcJlRrZmV05EoLhlkQA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_02,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 malwarescore=0 bulkscore=0 clxscore=1015 adultscore=0 suspectscore=0 priorityscore=1501 impostorscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060195 Content-Type: text/plain; charset="utf-8" Currently run_fsx is hardcoded to run on a file in $TEST_DIR. Add a helper _run_fsx_on_file so that we can run fsx on any given file including in $SCRATCH_MNT. Also, refactor _run_fsx to use this helper. No functional change is intended in this patch. Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo Reviewed-by: John Garry --- common/rc | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/common/rc b/common/rc index 8a023b9d..ac77a650 100644 --- a/common/rc +++ b/common/rc @@ -5203,13 +5203,24 @@ _require_hugepage_fsx() _notrun "fsx binary does not support MADV_COLLAPSE" } =20 -_run_fsx() +_run_fsx_on_file() { + local testfile=3D$1 + shift + + if ! [ -f $testfile ] + then + echo "_run_fsx_on_file: $testfile doesn't exist. Creating" >> $seqres.fu= ll + touch $testfile + fi + echo "fsx $*" local args=3D`echo $@ | sed -e "s/ BSIZE / $bsize /g" -e "s/ PSIZE / $psi= ze /g"` - set -- $FSX_PROG $args $FSX_AVOID $TEST_DIR/junk + + set -- $FSX_PROG $args $FSX_AVOID $testfile + echo "$@" >>$seqres.full - rm -f $TEST_DIR/junk + rm -f $testfile "$@" 2>&1 | tee -a $seqres.full >$tmp.fsx local res=3D${PIPESTATUS[0]} if [ $res -ne 0 ]; then @@ -5221,6 +5232,12 @@ _run_fsx() return 0 } =20 +_run_fsx() +{ + _run_fsx_on_file $TEST_DIR/junk $@ + return $? +} + # Run fsx with -h(ugepage buffers). If we can't set up a hugepage then sk= ip # the test, but if any other error occurs then exit the test. _run_hugepage_fsx() { --=20 2.49.0 From nobody Thu Oct 2 19:24:13 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EDEBA35E4FF; Thu, 11 Sep 2025 17:14:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610859; cv=none; b=WD0NT+2SJwwgn88P4AnqupRh8sogTj99MBlmGBmDXUwMHS8ft3T23L8DloWYCdgnSr/l6GjwFZ030Q/o4EhVvk9/VnCEKTji3vUvIjRx7Apkk+HFEaVQYzYdHtwc3OZNcV63eY1s+jCcD1hc2NZ2vBR0f4ePjgbZaf+21/tX7jc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610859; c=relaxed/simple; bh=LioGuSv8gh+FACLKetBabaA1JxM4E2bYxoWLiAjfvzw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sXObBaE164DMOuaaPiAdmNcRQ7F2AidXfMf+PPXF9ltX6rE+rfYbEKDo8dRNqp+hiR9DBsVvBfEBzTS2O2GD/U990lrdoWvxfEwEhxIj45/fB9UO2Q577L3mnKPZHvkZW3t7g0FxBo1CoU6V9ensjvpIiI2nJrut48OvOBsw6hk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=oXwe4vcq; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="oXwe4vcq" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BBq34r015555; Thu, 11 Sep 2025 17:14:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=x4NGr1m/Xveh2pPYm VjLzXxjLOVjo2c7XJc+kcjJ/70=; b=oXwe4vcqFwH4Dayncj/+jYJwz+6zG4b9M ZgMXtmFiG+DfI8JfgVOWjTrz09IzvklXA9tqiIaVF9m5ED8/p1VU+ma97RCza6iP e1lG7zXK/0ZbXcvLqDS6xT5nm/BEnZYbV6RR8IxuIZizofVCI2loQPsv3jVfFUMn vjmrUWs14BmWRqKNEbSYJ1lM2dhVMNYeqIOPkzyli+od4kXeZYLygNLlZFMv81Ft zw9cEMVlyHA1nA/G2JSGafuW5gFcuvQJAticbWqeOLumP40bnvSSaURTuhUFI9fd K07lLcHjxu/Tcd2PlUo1kXcTtV1eS0F40lyShU5NmjcEQ13VWge8w== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490cffp78v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:07 +0000 (GMT) Received: from m0360083.ppops.net (m0360083.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58BH6PUf013270; Thu, 11 Sep 2025 17:14:06 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490cffp78q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:06 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BGGKBK011463; Thu, 11 Sep 2025 17:14:05 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 490y9uq539-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:05 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BHE44S21627332 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 17:14:04 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DE95520040; Thu, 11 Sep 2025 17:14:03 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 69F4E2004B; Thu, 11 Sep 2025 17:14:00 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.39.17.37]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 17:14:00 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v6 04/12] ltp/fsx.c: Add atomic writes support to fsx Date: Thu, 11 Sep 2025 22:43:35 +0530 Message-ID: <73ab3d957290bfc0c3d35e4cacbfe249f5b23f4c.1757610403.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: e-QSf8YWoLiIGeAMGSi8C82ZRjfkFE8_ X-Proofpoint-GUID: ER_6p5cDafKtJC20GTUlqKG4dmpWnAaL X-Authority-Analysis: v=2.4 cv=EYDIQOmC c=1 sm=1 tr=0 ts=68c3035f cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=oUtnbjD3gOVQiAq08lgA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDAyMCBTYWx0ZWRfX4BfdfpcIewKP KHE1BAa9uStLlwoRFxS+RL+BTh4/T9oClFwpEAVfBtJ8SY5es6WmMR8cHe2TwoltNTIgzLba0iz V+15udmQbMlUoyNjUVi/hiEODU7XxJoFoHL/CxBoJoVUYFGUskzM4hcOloaurDbFhcsXLinl/Hh uGkC8p3uCfaj2P6YFZmqQ7ffW13UfB1Mj3gFaNOeBRpO2x/ptl8sCB42ACskYxXDR0ZQqeUmvM6 zgPjzEFed/RUymM0Q/gukh0dmOi+1OmGA1nbXwWPdYIa1os5PJWAd9oy6M1BrakHVzmVvswVRtv CdtEf1x38MyNB7Cpj2WF9MSjcxVYQ2gFV2xvzvs3I1Asg3HxE05is+stGRB2EcUS0PBFg/X//si fw7OTxBn X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_02,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 adultscore=0 suspectscore=0 spamscore=0 impostorscore=0 priorityscore=1501 phishscore=0 clxscore=1015 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060020 Content-Type: text/plain; charset="utf-8" Implement atomic write support to help fuzz atomic writes with fsx. Suggested-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- ltp/fsx.c | 115 +++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 110 insertions(+), 5 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 163b9453..bdb87ca9 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -40,6 +40,7 @@ #include #endif #include +#include "statx.h" =20 #ifndef MAP_FILE # define MAP_FILE 0 @@ -49,6 +50,10 @@ #define RWF_DONTCACHE 0x80 #endif =20 +#ifndef RWF_ATOMIC +#define RWF_ATOMIC 0x40 +#endif + #define NUMPRINTCOLUMNS 32 /* # columns of data to print on each line */ =20 /* Operation flags (bitmask) */ @@ -110,6 +115,7 @@ enum { OP_READ_DONTCACHE, OP_WRITE, OP_WRITE_DONTCACHE, + OP_WRITE_ATOMIC, OP_MAPREAD, OP_MAPWRITE, OP_MAX_LITE, @@ -200,6 +206,11 @@ int uring =3D 0; int mark_nr =3D 0; int dontcache_io =3D 1; int hugepages =3D 0; /* -h flag */ +int do_atomic_writes =3D 1; /* -a flag disables */ + +/* User for atomic writes */ +int awu_min =3D 0; +int awu_max =3D 0; =20 /* Stores info needed to periodically collapse hugepages */ struct hugepages_collapse_info { @@ -288,6 +299,7 @@ static const char *op_names[] =3D { [OP_READ_DONTCACHE] =3D "read_dontcache", [OP_WRITE] =3D "write", [OP_WRITE_DONTCACHE] =3D "write_dontcache", + [OP_WRITE_ATOMIC] =3D "write_atomic", [OP_MAPREAD] =3D "mapread", [OP_MAPWRITE] =3D "mapwrite", [OP_TRUNCATE] =3D "truncate", @@ -422,6 +434,7 @@ logdump(void) prt("\t***RRRR***"); break; case OP_WRITE_DONTCACHE: + case OP_WRITE_ATOMIC: case OP_WRITE: prt("WRITE 0x%x thru 0x%x\t(0x%x bytes)", lp->args[0], lp->args[0] + lp->args[1] - 1, @@ -1073,6 +1086,25 @@ update_file_size(unsigned offset, unsigned size) file_size =3D offset + size; } =20 +static int is_power_of_2(unsigned n) { + return ((n & (n - 1)) =3D=3D 0); +} + +/* + * Round down n to nearest power of 2. + * If n is already a power of 2, return n; + */ +static int rounddown_pow_of_2(int n) { + int i =3D 0; + + if (is_power_of_2(n)) + return n; + + for (; (1 << i) < n; i++); + + return 1 << (i - 1); +} + void dowrite(unsigned offset, unsigned size, int flags) { @@ -1081,6 +1113,27 @@ dowrite(unsigned offset, unsigned size, int flags) offset -=3D offset % writebdy; if (o_direct) size -=3D size % writebdy; + if (flags & RWF_ATOMIC) { + /* atomic write len must be between awu_min and awu_max */ + if (size < awu_min) + size =3D awu_min; + if (size > awu_max) + size =3D awu_max; + + /* atomic writes need power-of-2 sizes */ + size =3D rounddown_pow_of_2(size); + + /* atomic writes need naturally aligned offsets */ + offset -=3D offset % size; + + /* Skip the write if we are crossing max filesize */ + if ((offset + size) > maxfilelen) { + if (!quiet && testcalls > simulatedopcount) + prt("skipping atomic write past maxfilelen\n"); + log4(OP_WRITE_ATOMIC, offset, size, FL_SKIPPED); + return; + } + } if (size =3D=3D 0) { if (!quiet && testcalls > simulatedopcount && !o_direct) prt("skipping zero size write\n"); @@ -1088,7 +1141,10 @@ dowrite(unsigned offset, unsigned size, int flags) return; } =20 - log4(OP_WRITE, offset, size, FL_NONE); + if (flags & RWF_ATOMIC) + log4(OP_WRITE_ATOMIC, offset, size, FL_NONE); + else + log4(OP_WRITE, offset, size, FL_NONE); =20 gendata(original_buf, good_buf, offset, size); if (offset + size > file_size) { @@ -1108,8 +1164,9 @@ dowrite(unsigned offset, unsigned size, int flags) (monitorstart =3D=3D -1 || (offset + size > monitorstart && (monitorend =3D=3D -1 || offset <=3D monitorend)))))) - prt("%lld write\t0x%x thru\t0x%x\t(0x%x bytes)\tdontcache=3D%d\n", testc= alls, - offset, offset + size - 1, size, (flags & RWF_DONTCACHE) !=3D 0); + prt("%lld write\t0x%x thru\t0x%x\t(0x%x bytes)\tdontcache=3D%d atomic_wr= =3D%d\n", testcalls, + offset, offset + size - 1, size, (flags & RWF_DONTCACHE) !=3D 0, + (flags & RWF_ATOMIC) !=3D 0); iret =3D fsxwrite(fd, good_buf + offset, size, offset, flags); if (iret !=3D size) { if (iret =3D=3D -1) @@ -1785,6 +1842,36 @@ do_dedupe_range(unsigned offset, unsigned length, un= signed dest) } #endif =20 +int test_atomic_writes(void) { + int ret; + struct statx stx; + + if (o_direct !=3D O_DIRECT) { + fprintf(stderr, "main: atomic writes need O_DIRECT (-Z), " + "disabling!\n"); + return 0; + } + + ret =3D xfstests_statx(AT_FDCWD, fname, 0, STATX_WRITE_ATOMIC, &stx); + if (ret < 0) { + fprintf(stderr, "main: Statx failed with %d." + " Failed to determine atomic write limits, " + " disabling!\n", ret); + return 0; + } + + if (stx.stx_attributes & STATX_ATTR_WRITE_ATOMIC && + stx.stx_atomic_write_unit_min > 0) { + awu_min =3D stx.stx_atomic_write_unit_min; + awu_max =3D stx.stx_atomic_write_unit_max; + return 1; + } + + fprintf(stderr, "main: IO Stack does not support " + "atomic writes, disabling!\n"); + return 0; +} + #ifdef HAVE_COPY_FILE_RANGE int test_copy_range(void) @@ -2356,6 +2443,12 @@ have_op: goto out; } break; + case OP_WRITE_ATOMIC: + if (!do_atomic_writes) { + log4(OP_WRITE_ATOMIC, offset, size, FL_SKIPPED); + goto out; + } + break; } =20 switch (op) { @@ -2385,6 +2478,11 @@ have_op: dowrite(offset, size, 0); break; =20 + case OP_WRITE_ATOMIC: + TRIM_OFF_LEN(offset, size, maxfilelen); + dowrite(offset, size, RWF_ATOMIC); + break; + case OP_MAPREAD: TRIM_OFF_LEN(offset, size, file_size); domapread(offset, size); @@ -2511,13 +2609,14 @@ void usage(void) { fprintf(stdout, "usage: %s", - "fsx [-dfhknqxyzBEFHIJKLORWXZ0]\n\ + "fsx [-adfhknqxyzBEFHIJKLORWXZ0]\n\ [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid]\n\ [-l flen] [-m start:end] [-o oplen] [-p progressinterval]\n\ [-r readbdy] [-s style] [-t truncbdy] [-w writebdy]\n\ [-A|-U] [-D startingop] [-N numops] [-P dirpath] [-S seed]\n\ [--replay-ops=3Dopsfile] [--record-ops[=3Dopsfile]] [--duration=3Dseco= nds]\n\ ... fname\n\ + -a: disable atomic writes\n\ -b opnum: beginning operation number (default 1)\n\ -c P: 1 in P chance of file close+open at each op (default infinity)\n\ -d: debug output for all operations\n\ @@ -3059,9 +3158,13 @@ main(int argc, char **argv) setvbuf(stdout, (char *)0, _IOLBF, 0); /* line buffered stdout */ =20 while ((ch =3D getopt_long(argc, argv, - "0b:c:de:fg:hi:j:kl:m:no:p:qr:s:t:uw:xyABD:EFJKHzCILN:OP:RS:UWXZ", + "0ab:c:de:fg:hi:j:kl:m:no:p:qr:s:t:uw:xyABD:EFJKHzCILN:OP:RS:UWXZ", longopts, NULL)) !=3D EOF) switch (ch) { + case 'a': + prt("main(): Atomic writes disabled\n"); + do_atomic_writes =3D 0; + break; case 'b': simulatedopcount =3D getnum(optarg, &endp); if (!quiet) @@ -3475,6 +3578,8 @@ main(int argc, char **argv) exchange_range_calls =3D test_exchange_range(); if (dontcache_io) dontcache_io =3D test_dontcache_io(); + if (do_atomic_writes) + do_atomic_writes =3D test_atomic_writes(); =20 while (keep_running()) if (!test()) --=20 2.49.0 From nobody Thu Oct 2 19:24:13 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 10D2F369341; Thu, 11 Sep 2025 17:14:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610861; cv=none; b=L42xeB+97tG8g3bTViR2AACyDmenZdAbLGGkOdONee10Yj3Hc1oLO8m2AYSvxpEzCHQ4/6oiPUY3Rv9/9of5RJu8gaAOI1LXyArbCUXM2/fi83wgXNtetFxsPVQgGkm3+tMzjBH/vHxDC04YMvNjcnsCpcwmtIPjz4hfbZf553E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610861; c=relaxed/simple; bh=v26UflPQnu2h2Q5+BBo6b3eR9y7md2rK29p0D+NRWAc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BTn7x/dl2REXUNczRPKQID1TDRAK1Zyho/uJWNrp0tGrECW4RbmYPDfRbrEgPRi1ClE1v6fIiN8k0+Mhg0uXYVLj8n8Jo75CkwGPI1xLbYw358PrQM/vh8/RY1JtKPoVjCHnaQdIyMdN973268G8GlNFU4+vMOCsg4tifhyVbKA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Fx3esfge; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Fx3esfge" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BDhJKp009660; Thu, 11 Sep 2025 17:14:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=rZYDWIxNta4tNx7sM Hsk5+3wUSG6W0FYsTLTVuBCzOs=; b=Fx3esfgea2YMexvvS+Msd1bFFU9eho9KQ x65HZZ8ruz/qZLyyIjwjc+xE+L0hFGm1l6+liqLdFd3eIggSP5jfxk5i2mwwYdcM mBHtfEjrZppjsRtP/eGFtBLNUEmwN0a+ZB01HHtrt+TUxqO7VR6hrXXToBl8NPHZ XHHuHR2lEcliGQjpZKWmq9b+IXlSUn9unZx9zBWdjVQJUrmYKq//TdLNIQN/bRSf e/QL3khtptbxC6HKSRBV5eRUzjWD2EKTDtQ3/LbWLq4MAxtiUh1uWb0t9cH5XCIQ /aKJ63yGK7bUyn3G5AUHdSJaMUwP9E8ti0NAsmO80p37br/4WBL2w== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490acrdgg9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:10 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58BH9HCY026022; Thu, 11 Sep 2025 17:14:10 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490acrdgg4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:10 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BEF4iG017172; Thu, 11 Sep 2025 17:14:09 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4911gmpppt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:09 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BHE7Tt61669810 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 17:14:07 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C672A20043; Thu, 11 Sep 2025 17:14:07 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 810A72004B; Thu, 11 Sep 2025 17:14:04 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.39.17.37]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 17:14:04 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v6 05/12] generic: Add atomic write test using fio crc check verifier Date: Thu, 11 Sep 2025 22:43:36 +0530 Message-ID: <38047ca73eb768e5fc71bc640185b1c0a032e40a.1757610403.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: j-ZVafVNBa7doXfPW8QzxacdSuTN8vwQ X-Authority-Analysis: v=2.4 cv=Mp1S63ae c=1 sm=1 tr=0 ts=68c30363 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=TDUsxBfbQoAVH7yvDjsA:9 X-Proofpoint-ORIG-GUID: RAmvoPdIzeRoY8bl0d-PKUcV8OXCjIME X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDAwMCBTYWx0ZWRfX6LHyvnOR4CwM s35ryb9EntFIfGf4DXkwmv0CrPLPIIIB3tzcPaJM/MeOLJxWZX6VIW29h84ZPmGCkrPF0JdA/aH Sd0YhwR2gzVfTx0rWLOqw6BHEvEtM1+Dh281OJOat+Yr9KElXzVl9L3l1V/VhCe+rEBAesNGG1M mj4l365DAKa0W9rHEXGMeuOqb+3K7a72jCbIzarrT9LE4606aaaTPwkKdnwKckJtyxaV9ZxXP0I 9AiR/XeRL47EV6sQkIUkoBtXAh04uFGA587rzsBnIW6ULDoJ9h8/MHr6Qz/xifu5+tMH8JrjC6l TWD8+6XRcdr6lutGiaN4PCuAWwNK2Nn/kwbUmfd+g/HrUYHVEazPAUiZuzlJryi16qAXV7WenSB zsT6/7Fh X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_02,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 malwarescore=0 clxscore=1015 phishscore=0 spamscore=0 adultscore=0 priorityscore=1501 bulkscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060000 Content-Type: text/plain; charset="utf-8" This adds atomic write test using fio based on it's crc check verifier. fio adds a crc header for each data block, which is verified later to ensure there is no data corruption or torn write. This test essentially does a lot of parallel RWF_ATOMIC IO on a preallocated file to stress the write and end-io unwritten conversion code paths. The idea is to increase code coverage to ensure RWF_ATOMIC hasn't introduced any issues. Avoid doing overlapping parallel atomic writes because it might give unexpected results. Use offset_increment=3D, size=3D fio options to achieve this behavior. Co-developed-by: Ritesh Harjani (IBM) Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- tests/generic/1226 | 108 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1226.out | 2 + 2 files changed, 110 insertions(+) create mode 100755 tests/generic/1226 create mode 100644 tests/generic/1226.out diff --git a/tests/generic/1226 b/tests/generic/1226 new file mode 100755 index 00000000..7ad74554 --- /dev/null +++ b/tests/generic/1226 @@ -0,0 +1,108 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1226 +# +# Validate FS atomic write using fio crc check verifier. +# +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto aio rw atomicwrites + +_require_scratch_write_atomic +_require_odirect +_require_aio +_require_fio_atomic_writes + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount +_require_xfs_io_command "falloc" + +touch "$SCRATCH_MNT/f1" +awu_min_write=3D$(_get_atomic_write_unit_min "$SCRATCH_MNT/f1") +awu_max_write=3D$(_get_atomic_write_unit_max "$SCRATCH_MNT/f1") + +blocksize=3D$(_max "$awu_min_write" "$((awu_max_write/2))") +threads=3D$(_min "$(($(nproc) * 2 * LOAD_FACTOR))" "100") +filesize=3D$((blocksize * threads * 100)) +depth=3D$threads +io_size=3D$((filesize / threads)) +io_inc=3D$io_size +testfile=3D$SCRATCH_MNT/test-file + +fio_config=3D$tmp.fio +fio_out=3D$tmp.fio.out + +fio_aw_config=3D$tmp.aw.fio +fio_verify_config=3D$tmp.verify.fio + +function create_fio_configs() +{ + create_fio_aw_config + create_fio_verify_config +} + +function create_fio_verify_config() +{ +cat >$fio_verify_config <$fio_aw_config <> $seqres.full +cat $fio_verify_config >> $seqres.full + +$XFS_IO_PROG -fc "falloc 0 $filesize" $testfile >> $seqres.full + +$FIO_PROG $fio_aw_config >> $seqres.full +ret1=3D$? +$FIO_PROG $fio_verify_config >> $seqres.full +ret2=3D$? + +[[ $ret1 -eq 0 && $ret2 -eq 0 ]] || _fail "fio with atomic write failed" + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/generic/1226.out b/tests/generic/1226.out new file mode 100644 index 00000000..6dce0ea5 --- /dev/null +++ b/tests/generic/1226.out @@ -0,0 +1,2 @@ +QA output created by 1226 +Silence is golden --=20 2.49.0 From nobody Thu Oct 2 19:24:13 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD4B4362092; Thu, 11 Sep 2025 17:14:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610862; cv=none; b=lRoKyLvxLGuyJB2AqCYF1GyuknsGAvSpcUuorvg2P/taZ42mqbZfN0SwSnOcacv5kGV7fMJVn/SeLhtu4eKtiBDAJKQmQZNXZHrejUkbk/+oBMoDriAe29KWVZTWfpIx1piRYyI8+L5hjQdBBBIPca8ldr8VzF20WIBp10A27ew= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610862; c=relaxed/simple; bh=U2+DFjhyIqPMB1ayIZuWmHVDK4O9xd4a2D0R+RU4N6Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nF6cqJX1Dkex1DuKzrA5xN8DcGicgcK/qrXTdo0zeLWPZrxNFuMppF3cKV3v9uIUYGNoFamfuCvz0PJQF6A6agm6TzeBW9eulHBBsRBYBJEUQyDOs1D+gkFFke5c4PZZ8aAZa5KFhoczh+ctRmPdRiddtNfHbESR3BsqBsBghy4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=UANVipZF; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="UANVipZF" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BG15T2011307; Thu, 11 Sep 2025 17:14:15 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=5rUBmHsTxULczha7j OsuDxqz5QJHjQC/dMqlzcYVYyA=; b=UANVipZFu3sv3eSNo1RjGHFb4lzPW/3WM TlQZRByIFDVIwtt7/FyzJ9wwPso7gMxmiN8X5ZJViwOWj1Mli9/07y8sI8mJ68ds QXMu6ZJT53/mtPPPKbPR3DkJbAU/OPpJXzxntrScyj9Y7raZ4Y3+AUFmrlwLURZH YHtUO8tWNW5RR14EBDKtvJQ4TR2sxcRoEy5H/f0uD4i3yzLNy0zsCCatpie7iBpH 8eVqK6uJqXceh3IM4mGMMvmeTiQg4wc/bED1RjJdbYvnVJXo1hMhiioFCIW8DOlU Dd1fBHSOu3goZyztymWWLgn7KPYGRlKWjZ2rupZp4VV5rt26xovng== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490bct5e5a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:14 +0000 (GMT) Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58BHEE81032340; Thu, 11 Sep 2025 17:14:14 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490bct5e56-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:14 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BH3c1w007929; Thu, 11 Sep 2025 17:14:13 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 49109pxvx2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:13 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BHEBrj52494704 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 17:14:11 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 957FA2004F; Thu, 11 Sep 2025 17:14:11 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 56E4120040; Thu, 11 Sep 2025 17:14:08 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.39.17.37]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 17:14:08 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v6 06/12] generic: Add atomic write test using fio verify on file mixed mappings Date: Thu, 11 Sep 2025 22:43:37 +0530 Message-ID: <1fba284833eb435bdc8b20910fd49f3bf1e1b504.1757610403.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDAxMCBTYWx0ZWRfX8+QTGC1SsPNV RSvLnNPw0y9VMeW4xFkwsOGwx106tKcdS0Vvu4of7Dz8WrjGH8yf6OdxF8upXlMMh2uFSaEf/Nm IF71wq8bccBKRBey3eXFg8esXWkNTwItFqrmHL8ZodmWIwW+I27UZpNFMoFa576aeVWUT9SO3CE 7ozo8RyHy9Q76oKb17DD9QBLX0NoZsTfrfROjo3BDcEcSBTnl6fKYiXn5xsLyq7wQIQhhwccjeb A5jTjMXaZyMO9PBPx1ghPZoP5SW9cWNzbnRM+cuhmbTg/v/OxArWfFpp2okKSMwXuAWs/Izezhe UOmSWKXBlrC/HPXUIqKq3/parMSMXMfdmNad7rgYQi5v0vQKzwqmc+6FaumFMzeuy5C2zNcegsO 4a/R2kJh X-Authority-Analysis: v=2.4 cv=SKNCVPvH c=1 sm=1 tr=0 ts=68c30366 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=qMsZUPF6o8un45dIlBUA:9 X-Proofpoint-GUID: fNwlUYfHe66fUQP6vZWUHGnccZ-LdSuJ X-Proofpoint-ORIG-GUID: mPMr_YVTD9BrGoSCqUggHp_vQYZCl6S6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_02,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 spamscore=0 priorityscore=1501 bulkscore=0 malwarescore=0 adultscore=0 suspectscore=0 impostorscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060010 Content-Type: text/plain; charset="utf-8" This test uses fio to first create a file with mixed mappings. Then it does atomic writes using aio dio with parallel jobs to the same file with mixed mappings. Finally, we perform a fio verify step to ensure there is no data corruption or torn write. The aim is to stress the FS block allocation and extent handling logic to ensure it handles mixed mappings with RWF_ATOMIC correctly without tearing or losing data. Avoid doing overlapping parallel atomic writes because it might give unexpected results. Use offset_increment=3D, size=3D fio options to achieve this behavior. Co-developed-by: Ritesh Harjani (IBM) Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- tests/generic/1227 | 132 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1227.out | 2 + 2 files changed, 134 insertions(+) create mode 100755 tests/generic/1227 create mode 100644 tests/generic/1227.out diff --git a/tests/generic/1227 b/tests/generic/1227 new file mode 100755 index 00000000..26177508 --- /dev/null +++ b/tests/generic/1227 @@ -0,0 +1,132 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1227 +# +# Validate FS atomic write using fio crc check verifier on mixed mappings +# of a file. +# +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto aio rw atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_odirect +_require_aio +_require_fio_atomic_writes +_require_xfs_io_command "truncate" + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +touch "$SCRATCH_MNT/f1" +awu_min_write=3D$(_get_atomic_write_unit_min "$SCRATCH_MNT/f1") +awu_max_write=3D$(_get_atomic_write_unit_max "$SCRATCH_MNT/f1") + +aw_bsize=3D$(_max "$awu_min_write" "$((awu_max_write/4))") +fsbsize=3D$(_get_block_size $SCRATCH_MNT) + +threads=3D$(_min "$(($(nproc) * 2 * LOAD_FACTOR))" "100") +filesize=3D$((aw_bsize * threads * 100)) +depth=3D$threads +aw_io_size=3D$((filesize / threads)) +aw_io_inc=3D$aw_io_size +testfile=3D$SCRATCH_MNT/test-file + +fio_prep_config=3D$tmp.prep.fio +fio_aw_config=3D$tmp.aw.fio +fio_verify_config=3D$tmp.verify.fio +fio_out=3D$tmp.fio.out + +cat >$fio_prep_config <$fio_aw_config <$fio_verify_config <> $seqres.full +cat $fio_aw_config >> $seqres.full +cat $fio_verify_config >> $seqres.full + +$XFS_IO_PROG -fc "truncate $filesize" $testfile >> $seqres.full + +#prepare file with mixed mappings +$FIO_PROG $fio_prep_config >> $seqres.full + +# do atomic writes without verifying +$FIO_PROG $fio_aw_config >> $seqres.full + +# verify data is not torn +$FIO_PROG $fio_verify_config >> $seqres.full + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/generic/1227.out b/tests/generic/1227.out new file mode 100644 index 00000000..2605d062 --- /dev/null +++ b/tests/generic/1227.out @@ -0,0 +1,2 @@ +QA output created by 1227 +Silence is golden --=20 2.49.0 From nobody Thu Oct 2 19:24:13 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CA011362999; Thu, 11 Sep 2025 17:14:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610869; cv=none; b=mc0QwJvdu4G8NJhDK6ahxYd2C+lUDFrUcJNKiW4/aD5zBhvvigPrgRG+2JY3agdLY7OxgsfPWRZRjYf/xdtIO7eXNL5jzqmpflBUvRCYKdhkPtq76zjT8C+a8KvqSj8GUWPR1XaCTO4fVR7Hj6Eouxp4UJj9EaOOvTJvgssP5ik= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610869; c=relaxed/simple; bh=P5cIYMqpFJmwCgxQgjMz1GxesNclep3gTW7Q3P4dypg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Xl/1AoVuN2yMC/UhpSub0IricSEs/OZZkLKD9b2uVvrGsvGTt/WhjlvdRyjxb572BBJ42GV50bZJg+2oWuF1XOEZLhyksrz+fjo7OtjjrkggNQA34Z3b71sATIicDOphzdk6+FtVvh7CCylToGuqJtxHRRRuNnMmcr2Ach+kTvg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=BCdLfIb7; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="BCdLfIb7" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58B8U4om021177; Thu, 11 Sep 2025 17:14:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=6YeF4FniBf8RotV2z uLYyAfpLhae4VcbwG1rB05hAa0=; b=BCdLfIb72Y7vPEst9q20X67ERzc3F4K0J R8Wf6zsUqGLG1pQb0tUc6CR+7HG/GVHK0nbfvSYP/BGhrh/BEtNpdqJ2sqUtRHMc Ruoo/xDfhFLe5X24lqMsU16q8f0V6bmbIshLx4WwZkp5L4TBrGavpEX5JRUWg9JH 7SHKiOJekfCom4sKwHXSrM+gXSw1S2TP6leXfW12tNl1omyKhEbHg0BHStpqVg84 LuSSU9sCneTu05Q7QvcmRi+Qut68C0/mCxt3g2/xp1HWZKs5nXuXHPAoBoOeNX9R pv8NO+SFAwBTrV5uD+3ixJqXBNnU3tb3Lrkq9M58fATTUyRMBs9dg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490bct5e5h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:18 +0000 (GMT) Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58BH4HCM010870; Thu, 11 Sep 2025 17:14:18 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490bct5e5e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:18 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BEsFBJ001163; Thu, 11 Sep 2025 17:14:17 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 491203pmn9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:17 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BHEFWm32113278 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 17:14:15 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 91E5520043; Thu, 11 Sep 2025 17:14:15 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4A7DA20040; Thu, 11 Sep 2025 17:14:12 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.39.17.37]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 17:14:11 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v6 07/12] generic: Add atomic write multi-fsblock O_[D]SYNC tests Date: Thu, 11 Sep 2025 22:43:38 +0530 Message-ID: <50f203dd770b326a867d11bfcb925d672dc083b4.1757610403.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDAxMCBTYWx0ZWRfX9Ne5/UKbOc7U lygOxcChyDHxZKPSNdU32yir7Oz563VGi0Ux4WuIauxgBAy5LfpocfY5eUOrGq5OSCIBAvlC8q+ bZeA1kOzKoBiFvUW5gsWhzSpg7alM4SKO0isGwJLP4Do9/8TUEdJ4f14n7lKzHIUZNme2NgEeAI pyWjUNo9WR3LR6qTfp5o+B5RGX+3T/q/6pYwDG9ZvEg0oKd/MJfrpYc6xTCEYUB24s9jH9QdcHG x5TzL50z76SJtphXyiodg+sX8RpoN+/VPAUmQO/PTBvd/CN6LsO3vXH1MZbU70pFA9IuTgqNNik 9Cj52RYb6x4DFo4ds7jaqmW1jtDS/1E4lo9GewCCpoyxZqCr4/si0A9Dyn019cokZAdhBtxtHTs /arIgAkM X-Authority-Analysis: v=2.4 cv=SKNCVPvH c=1 sm=1 tr=0 ts=68c3036a cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=udGWdZ-Mnbz566dSa5EA:9 X-Proofpoint-GUID: WJLI_QFuhWD8ZHESSN024Y0F8OjQY9c2 X-Proofpoint-ORIG-GUID: Cxg1RDEqDxKdalW_R8tT6zCER9rykIAc X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_02,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 spamscore=0 priorityscore=1501 bulkscore=0 malwarescore=0 adultscore=0 suspectscore=0 impostorscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060010 Content-Type: text/plain; charset="utf-8" This adds various atomic write multi-fsblock stress tests with mixed mappings and O_SYNC, to ensure the data and metadata is atomically persisted even if there is a shutdown. Suggested-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- tests/generic/1228 | 139 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1228.out | 2 + 2 files changed, 141 insertions(+) create mode 100755 tests/generic/1228 create mode 100644 tests/generic/1228.out diff --git a/tests/generic/1228 b/tests/generic/1228 new file mode 100755 index 00000000..20fbeb49 --- /dev/null +++ b/tests/generic/1228 @@ -0,0 +1,139 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1228 +# +# Atomic write multi-fsblock data integrity tests with mixed mappings +# and O_SYNC +# +. ./common/preamble +. ./common/atomicwrites +_begin_fstest auto quick rw atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_atomic_write_test_commands +_require_scratch_shutdown +_require_xfs_io_command "truncate" + +_scratch_mkfs >> $seqres.full +_scratch_mount >> $seqres.full + +check_data_integrity() { + actual=3D$(_hexdump $testfile) + if [[ "$expected" !=3D "$actual" ]] + then + echo "Integrity check failed" + echo "Integrity check failed" >> $seqres.full + echo "# Expected file contents:" >> $seqres.full + echo "$expected" >> $seqres.full + echo "# Actual file contents:" >> $seqres.full + echo "$actual" >> $seqres.full + + _fail "Data integrity check failed. The atomic write was torn." + fi +} + +prep_mixed_mapping() { + $XFS_IO_PROG -c "truncate 0" $testfile >> $seqres.full + local off=3D0 + local mapping=3D"" + + local operations=3D("W" "H" "U") + local num_blocks=3D$((awu_max / blksz)) + for ((i=3D0; i /d= ev/null + ;; + "H") + # No operation needed for hole + ;; + "U") + $XFS_IO_PROG -c "falloc $off $blksz" $testfile >> /dev/null + ;; + esac + off=3D$((off + blksz)) + done + + echo "+ + Mixed mapping prep done. Full mapping pattern: $mapping" >> $se= qres.full + + sync $testfile +} + +verify_atomic_write() { + test $bytes_written -eq $awu_max || _fail "atomic write len=3D$awu_max as= sertion failed" + check_data_integrity +} + +mixed_mapping_test() { + prep_mixed_mapping + + echo -"+ + Performing O_DSYNC atomic write from 0 to $awu_max" >> $seqres= .full + if [[ "$1" =3D=3D "shutdown" ]] + then + bytes_written=3D$($XFS_IO_PROG -x -dc \ + "pwrite -DA -V1 -b $awu_max 0 $awu_max" \ + -c "shutdown" $testfile | grep wrote | \ + awk -F'[/ ]' '{print $2}') + _scratch_cycle_mount >>$seqres.full 2>&1 || _fail "remount failed" + else + bytes_written=3D$($XFS_IO_PROG -dc \ + "pwrite -DA -V1 -b $awu_max 0 $awu_max" $testfile | \ + grep wrote | awk -F'[/ ]' '{print $2}') + fi + + verify_atomic_write +} + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile + +awu_max=3D$(_get_atomic_write_unit_max $testfile) +blksz=3D$(_get_block_size $SCRATCH_MNT) + +# Create an expected pattern to compare with +$XFS_IO_PROG -tc "pwrite -b $awu_max 0 $awu_max" $testfile >> $seqres.full +expected=3D$(_hexdump $testfile) +echo "# Expected file contents:" >> $seqres.full +echo "$expected" >> $seqres.full +echo >> $seqres.full + +echo "# Test 1: Do O_DSYNC atomic write on random mixed mapping:" >> $seqr= es.full +echo >> $seqres.full + +iterations=3D10 +for ((i=3D1; i<=3D$iterations; i++)); do + echo "=3D=3D=3D Mixed Mapping Test Iteration $i =3D=3D=3D" >> $seqres.full + + echo "+ Testing without shutdown..." >> $seqres.full + mixed_mapping_test + echo "Passed!" >> $seqres.full + + echo "+ Testing with sudden shutdown..." >> $seqres.full + mixed_mapping_test "shutdown" + echo "Passed!" >> $seqres.full + + echo "Iteration $i completed: OK" >> $seqres.full + echo >> $seqres.full +done +echo "# Test 1: Do O_SYNC atomic write on random mixed mapping ($iteration= s iterations): OK" >> $seqres.full + + +echo >> $seqres.full +echo "# Test 2: Do extending O_SYNC atomic writes: " >> $seqres.full +bytes_written=3D$($XFS_IO_PROG -x -dstc "pwrite -A -V1 -b $awu_max 0 $awu_= max" \ + -c "shutdown" $testfile | grep wrote | awk -F'[/ ]' '{print $2}') +_scratch_cycle_mount >>$seqres.full 2>&1 || _fail "remount failed" +verify_atomic_write +echo "# Test 2: Do extending O_SYNC atomic writes: OK" >> $seqres.full + +# success, all done +echo "Silence is golden" +status=3D0 +exit + diff --git a/tests/generic/1228.out b/tests/generic/1228.out new file mode 100644 index 00000000..1baffa91 --- /dev/null +++ b/tests/generic/1228.out @@ -0,0 +1,2 @@ +QA output created by 1228 +Silence is golden --=20 2.49.0 From nobody Thu Oct 2 19:24:13 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 56FD8362999; Thu, 11 Sep 2025 17:14:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610876; cv=none; b=Pb1qCFZpmJ8X3mjBR2MDXq3NAv50n4zJXT9OF3W56CS+yAyXj5/6N+aUOzCM/5pyBSeCzmRMgJDjjWcBqvf2hNosuxNQ0ew+FW2Bo+zmY5N9fRvtSiQIm3Gsw7xQ+slPy+rVlCa0FYhJv5HMCLeER90DC0/LITiJKJ1WuEpnFVM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610876; c=relaxed/simple; bh=DuKA96AdpyKLqZYBHb58RoVvGwseEPk0CTPSH9y8nJQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cZlfCM3UhmapJoQjWVowsVio62tSxhbPsFfDGy+0qm/l1K2wriSkGQybz1U7dg9cXoKnMqlihoxP7xtmwJcDhN7zmRVLY2gFLvm3Aue5nR5LGMwVpGrbU+G1wq1eKv0dkfZGYJ60uTxqg4eojXDL/u+ChDAmka5DNFLT8eZn66s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=nvwtnl7X; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="nvwtnl7X" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BDFnSx026794; Thu, 11 Sep 2025 17:14:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=ot+islW9QqRcQihl1 cGnoj87N1ScOyDjNrHRSvRNX44=; b=nvwtnl7XbdlQ2ZK3JwqbrCe0yU1jL/Fcv muTEkjfZTUEpwndM9C7enMffPURPcDoNC4ZHJM6jPXzdN8/SPQR9wTFj19kCMPS3 +qmwrTn8KpPwJVpSYZ6KFL7geRxzUSXqZPMWUygfwY6TBUtFdOh1PEcy9OfNFNAt DOM2TkrUvUX6dE55798ZOGW0R5KwDCNmN5pA8fcVgrbsiNhhYUCL6ob6mGsbGP3p 13kw9uTf0mTi0/TTIW53RqTtgG0gnlJeYrx2Zhdksa/VCVKnLiF+k3wkuHbTcIYL sltUeVNsUwh0V0QhrBsYopOMabg4gMHZYZ8KxeAZdskQFoBCFVLfQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490bct5e61-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:25 +0000 (GMT) Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58BHEPlA032470; Thu, 11 Sep 2025 17:14:25 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490bct5e5w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:25 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BH6Dnm020469; Thu, 11 Sep 2025 17:14:24 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 490yp171um-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:24 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BHEJek35455396 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 17:14:19 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C71E320040; Thu, 11 Sep 2025 17:14:19 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 324EA2004B; Thu, 11 Sep 2025 17:14:16 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.39.17.37]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 17:14:15 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v6 08/12] generic: Stress fsx with atomic writes enabled Date: Thu, 11 Sep 2025 22:43:39 +0530 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDAxMCBTYWx0ZWRfX1gb9549iKlLn xnTfhzufPZmMLoBEAdhxXKcaJIs/6Ni03bp0zhiBVTH0OxQX8ljaPWfRxdCkS0MeIcIS4pHLZ51 +JKkfDzSctpmKio9ZeCLnCECorb7oT5FhacCsp+iF0Olii+eoisz2xcIufJYsnIa3AcX9l/xfbm dwRv/dIuwDqBDbUpvAzFKwlqibSkzHrauEPx7XhyfKqQ6Rn4lMN7HICUffcZkDR2e5+FYgyHYce mpx3v+y6iTP1B/z68oIocwuDeYSKLRJTzd/ZQr9QKcMgyUV/Y1tyfgBEdCs7LbcpikA8l6jHsT0 COxZDiK6LPPLb35O1tfEcKnkeKQ4Bmu5FnTXP5OWFGWk1exLFk4hHDqO4w9kEPxeQRwBE0e3JBB MfW1RJ05 X-Authority-Analysis: v=2.4 cv=SKNCVPvH c=1 sm=1 tr=0 ts=68c30371 cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=zPW1cLLGUcrnTIp6xCMA:9 X-Proofpoint-GUID: aP5vXkxGVyMwLPUhnwVbqVX945YZ5cLv X-Proofpoint-ORIG-GUID: Cjsh6SxfKTPDKLtcR_ThEy2ISGYn5mCF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_02,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 spamscore=0 priorityscore=1501 bulkscore=0 malwarescore=0 adultscore=0 suspectscore=0 impostorscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060010 Content-Type: text/plain; charset="utf-8" Stress file with atomic writes to ensure we exercise codepaths where we are mixing different FS operations with atomic writes Suggested-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- tests/generic/1229 | 68 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/1229.out | 2 ++ 2 files changed, 70 insertions(+) create mode 100755 tests/generic/1229 create mode 100644 tests/generic/1229.out diff --git a/tests/generic/1229 b/tests/generic/1229 new file mode 100755 index 00000000..6d4dcfed --- /dev/null +++ b/tests/generic/1229 @@ -0,0 +1,68 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 1229 +# +# fuzz fsx with atomic writes +# +. ./common/preamble +. ./common/atomicwrites +_begin_fstest rw auto quick atomicwrites + +_require_odirect +_require_scratch_write_atomic + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount >> $seqres.full 2>&1 + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile + +awu_max=3D$(_get_atomic_write_unit_max $testfile) +blksz=3D$(_get_block_size $SCRATCH_MNT) +bsize=3D`$here/src/min_dio_alignment $SCRATCH_MNT $SCRATCH_DEV` + +set_fsx_avoid() { + local file=3D$1 + + case "$FSTYP" in + "ext4") + local dev=3D$(findmnt -n -o SOURCE --target $testfile) + + # fsx insert/collapse range support for ext4+bigalloc is + # currently broken, so disable it. Also disable in case we + # can't detect bigalloc to be on safer side. + if [ -z "$DUMPE2FS_PROG" ]; then + echo "dumpe2fs not found, disabling insert/collapse range" >> $seqres.f= ull + FSX_AVOID+=3D" -I -C" + return + fi + + $DUMPE2FS_PROG -h $dev 2>&1 | grep -q bigalloc && { + echo "fsx insert/collapse range not supported with bigalloc. Disabling.= ." >> $seqres.full + FSX_AVOID+=3D" -I -C" + } + ;; + *) + ;; + esac +} + +# fsx usage: +# +# -N numops: total # operations to do +# -l flen: the upper bound on file size +# -o oplen: the upper bound on operation size (64k default) +# -Z: O_DIRECT () + +set_fsx_avoid +_run_fsx_on_file $testfile -N 10000 -o $awu_max -A -l 500000 -r $bsize -w = $bsize -Z $FSX_AVOID >> $seqres.full +if [[ "$?" !=3D "0" ]] +then + _fail "fsx returned error: $?" +fi + +echo "Silence is golden" +status=3D0 +exit diff --git a/tests/generic/1229.out b/tests/generic/1229.out new file mode 100644 index 00000000..737d61c6 --- /dev/null +++ b/tests/generic/1229.out @@ -0,0 +1,2 @@ +QA output created by 1229 +Silence is golden --=20 2.49.0 From nobody Thu Oct 2 19:24:13 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 850BF35FC20; Thu, 11 Sep 2025 17:14:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610878; cv=none; b=DiIPlcq4SuguhhbHAAZJGlHPnMEK0hL8MRCic5GNmEGIsSY26bsRsEZc5k1V+FzFUC0vsWoWFCNK1itEswCClrfziGGhf9vnG2F5BMiuxG/vfPdCHGY6f5Kec/W5uIdE5XKBox168+Bd2tr9HN7ssB9rOb+nY9/8JsxIaT9l3Zw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610878; c=relaxed/simple; bh=zLdJPAcjf1P40hvRHx/GqzNa3/vxt4V+dwxgtrNL7vU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VL/PjXVijThiwOe2D6x/w9p8ZofeXo3PQS8GrrptDGK36dcjbfNumX+dXVeUB8N+bA0P/rQpDDF98igtD66ZAaFevjWj4bcpNvCEPj6fEQFdyyXU3XeRrvK2P2xJ+0hoHmD5zMmwLPDoUsLljCc0T97wN+4Tr+L2n0FtE4n/7FE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=ZUR9dw3g; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="ZUR9dw3g" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BBMeFu019651; Thu, 11 Sep 2025 17:14:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=tqT+fe3yhCbHbk8h8 +1A38CaOrbwYz4dPkmfjTQobTs=; b=ZUR9dw3g4+NNiKGudvbDXtV+XZcFvg2Hy GgZz4XTTe8a2YFiJnvqjCvZotOGSC+O/5It5RdI5CgYwThOC0WejNzZc3cS473Dv LJuke11E8lHLx+aCkzou4U0gVAmw36dLwk8jTHCvCIESKYL/tmVKqU0omXWr+0iu xYx/zR1Q8Clyjg72r3R06aw3f+Apb0B8y3tana6ZY1VPwFwvU7XB3gwfCqaF3bal el9BD5YtIsPUznCb0lpRN4FOffhF1b0L6qmujdYuVyELsHLbZtvLL4Wx258nItKv L+9Jx+X13FajikAJj2OL6awvvSkY+2RjuM3Bn6HY9DOLqBNaqHaGw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490xydb1aa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:27 +0000 (GMT) Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58BHD8xF001690; Thu, 11 Sep 2025 17:14:26 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490xydb1a7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:26 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BEq59m001177; Thu, 11 Sep 2025 17:14:25 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 491203pmnp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:25 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BHENT856558060 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 17:14:23 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BF8792004F; Thu, 11 Sep 2025 17:14:23 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 64D0720043; Thu, 11 Sep 2025 17:14:20 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.39.17.37]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 17:14:20 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v6 09/12] generic: Add sudden shutdown tests for multi block atomic writes Date: Thu, 11 Sep 2025 22:43:40 +0530 Message-ID: <25f77aa7ac816e48b5921601e3cf10445db1f36b.1757610403.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: jRgwHxbjUIr4EiL1nLkaoT-FQUA1TYEM X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDIzNSBTYWx0ZWRfX6bb6u+FRsOoV 2Ws9Hq9/LeRd5u/YzWrWIaTLL4PTMmW7ksbkCc3IKbWQmYMqRn1JxkgnNgURZgeuYW4iMqR7Ezu 1QymbhndOJIRZAFqoBfq+XittI8kSD7Q6djAxRKG7nwB5lC9EI16QI7ETMQ/wF0pReqpci7tFUQ bPmNoo/vbAB3zz86b114HDg/EIN9yP0UgQtvJOHnZs29p64QuVcjNM94iD3XBgYGcDD0ar/FOHh KfH0vvB1IJ24VF2KdlzATvJB4oBsv7/vXau5oA0swFfXPCxsujHeLTmJCmM4f5FT9rS1FkKH3Rp e6dc1A7MIVXORcQYZB7uW3aXmY73rszjNoEjU+Da4UVtRR9MOqSyWvBWSbRzxLO4SUpdQPhKTvw Yk97tz4M X-Proofpoint-GUID: MGI7NjpGifX1e1VZ5s2Oa3ZlvJUQw7ym X-Authority-Analysis: v=2.4 cv=F59XdrhN c=1 sm=1 tr=0 ts=68c30373 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=dyK_zsDHFtz5CmQ9vqoA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_02,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 clxscore=1015 impostorscore=0 bulkscore=0 adultscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060235 Content-Type: text/plain; charset="utf-8" This test is intended to ensure that multi blocks atomic writes maintain atomic guarantees across sudden FS shutdowns. The way we work is that we lay out a file with random mix of written, unwritten and hole extents. Then we start performing atomic writes sequentially on the file while we parallelly shutdown the FS. Then we note the last offset where the atomic write happened just before shut down and then make sure blocks around it either have completely old data or completely new data, ie the write was not torn during shutdown. We repeat the same with completely written, completely unwritten and comple= tely empty file to ensure these cases are not torn either. Finally, we have a similar test for append atomic writes Suggested-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo --- tests/generic/1230 | 368 +++++++++++++++++++++++++++++++++++++++++ tests/generic/1230.out | 2 + 2 files changed, 370 insertions(+) create mode 100755 tests/generic/1230 create mode 100644 tests/generic/1230.out diff --git a/tests/generic/1230 b/tests/generic/1230 new file mode 100755 index 00000000..28c2c4f5 --- /dev/null +++ b/tests/generic/1230 @@ -0,0 +1,368 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test No. 1230 +# +# Test multi block atomic writes with sudden FS shutdowns to ensure +# the FS is not tearing the write operation +. ./common/preamble +. ./common/atomicwrites +_begin_fstest auto atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_atomic_write_test_commands +_require_scratch_shutdown +_require_xfs_io_command "truncate" + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount >> $seqres.full + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile + +awu_max=3D$(_get_atomic_write_unit_max $testfile) +blksz=3D$(_get_block_size $SCRATCH_MNT) +echo "Awu max: $awu_max" >> $seqres.full + +num_blocks=3D$((awu_max / blksz)) +# keep initial value high for dry run. This will be +# tweaked in dry_run() based on device write speed. +filesize=3D$(( 10 * 1024 * 1024 * 1024 )) + +_cleanup() { + [ -n "$awloop_pid" ] && kill $awloop_pid &> /dev/null + wait +} + +atomic_write_loop() { + local off=3D0 + local size=3D$awu_max + for ((i=3D0; i<$((filesize / $size )); i++)); do + # Due to sudden shutdown this can produce errors so just + # redirect them to seqres.full + $XFS_IO_PROG -c "open -fsd $testfile" -c "pwrite -S 0x61 -DA -V1 -b $siz= e $off $size" >> /dev/null 2>>$seqres.full + echo "Written to offset: $off" >> $tmp.aw + off=3D$((off + $size)) + done +} + +start_atomic_write_and_shutdown() { + atomic_write_loop & + awloop_pid=3D$! + + local i=3D0 + # Wait for atleast first write to be recorded or 10s + while [ ! -f "$tmp.aw" -a $i -le 50 ]; do i=3D$((i + 1)); sleep 0.2; done + + if [[ $i -gt 50 ]] + then + _fail "atomic write process took too long to start" + fi + + echo >> $seqres.full + echo "# Shutting down filesystem while write is running" >> $seqres.full + _scratch_shutdown + + kill $awloop_pid 2>/dev/null # the process might have finished already + wait $awloop_pid + unset $awloop_pid +} + +# This test has the following flow: +# 1. Start doing sequential atomic writes in background, upto $filesize +# 2. Sleep for 0.2s and shutdown the FS +# 3. kill the atomic write process +# 4. verify the writes were not torn +# +# We ideally want the shutdown to happen while an atomic write is ongoing +# but this gets tricky since faster devices can actually finish the whole +# atomic write loop before sleep 0.2s completes, resulting in the shutdown +# happening after the write loop which is not what we want. A simple solut= ion +# to this is to increase $filesize so step 1 takes long enough but a big +# $filesize leads to create_mixed_mappings() taking very long, which is not +# ideal. +# +# Hence, use the dry_run function to figure out the rough device speed and= set +# $filesize accordingly. +dry_run() { + echo >> $seqres.full + echo "# Estimating ideal filesize..." >> $seqres.full + + start_atomic_write_and_shutdown + + bytes_written=3D$(tail -n 1 $tmp.aw | cut -d" " -f4) + echo "# Bytes written in 0.2s: $bytes_written" >> $seqres.full + + filesize=3D$((bytes_written * 3)) + echo "# Setting \$filesize=3D$filesize" >> $seqres.full + + rm $tmp.aw + sleep 0.5 + + _scratch_cycle_mount + +} + +create_mixed_mappings() { + local file=3D$1 + local size_bytes=3D$2 + + echo "# Filling file $file with alternate mappings till size $size_bytes"= >> $seqres.full + #Fill the file with alternate written and unwritten blocks + local off=3D0 + local operations=3D("W" "U") + + for ((i=3D0; i<$((size_bytes / blksz )); i++)); do + index=3D$(($i % ${#operations[@]})) + map=3D"${operations[$index]}" + + case "$map" in + "W") + $XFS_IO_PROG -fc "pwrite -b $blksz $off $blksz" $file >> /dev/null + ;; + "U") + $XFS_IO_PROG -fc "falloc $off $blksz" $file >> /dev/null + ;; + esac + off=3D$((off + blksz)) + done + + sync $file +} + +populate_expected_data() { + # create a dummy file with expected old data for different cases + create_mixed_mappings $testfile.exp_old_mixed $awu_max + expected_data_old_mixed=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_o= ld_mixed) + + $XFS_IO_PROG -fc "falloc 0 $awu_max" $testfile.exp_old_zeroes >> $seqres.= full + expected_data_old_zeroes=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_= old_zeroes) + + $XFS_IO_PROG -fc "pwrite -b $awu_max 0 $awu_max" $testfile.exp_old_mapped= >> $seqres.full + expected_data_old_mapped=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_= old_mapped) + + # create a dummy file with expected new data + $XFS_IO_PROG -fc "pwrite -S 0x61 -b $awu_max 0 $awu_max" $testfile.exp_ne= w >> $seqres.full + expected_data_new=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp_new) +} + +verify_data_blocks() { + local verify_start=3D$1 + local verify_end=3D$2 + local expected_data_old=3D"$3" + local expected_data_new=3D"$4" + + echo >> $seqres.full + echo "# Checking data integrity from $verify_start to $verify_end" >> $se= qres.full + + # After an atomic write, for every chunk we ensure that the underlying + # data is either the old data or new data as writes shouldn't get torn. + local off=3D$verify_start + while [[ "$off" -lt "$verify_end" ]] + do + #actual_data=3D$(xxd -s $off -l $awu_max -p $testfile) + actual_data=3D$(od -An -t x1 -j $off -N $awu_max $testfile) + if [[ "$actual_data" !=3D "$expected_data_new" ]] && [[ "$actual_data" != =3D "$expected_data_old" ]] + then + echo "Checksum match failed at off: $off size: $awu_max" + echo "Expected contents: (Either of the 2 below):" + echo + echo "Expected old: " + echo "$expected_data_old" + echo + echo "Expected new: " + echo "$expected_data_new" + echo + echo "Actual contents: " + echo "$actual_data" + + _fail + fi + echo -n "Check at offset $off succeeded! " >> $seqres.full + if [[ "$actual_data" =3D=3D "$expected_data_new" ]] + then + echo "matched new" >> $seqres.full + elif [[ "$actual_data" =3D=3D "$expected_data_old" ]] + then + echo "matched old" >> $seqres.full + fi + off=3D$(( off + awu_max )) + done +} + +# test data integrity for file by shutting down in between atomic writes +test_data_integrity() { + echo >> $seqres.full + echo "# Writing atomically to file in background" >> $seqres.full + + start_atomic_write_and_shutdown + + last_offset=3D$(tail -n 1 $tmp.aw | cut -d" " -f4) + if [[ -z $last_offset ]] + then + last_offset=3D0 + fi + + echo >> $seqres.full + echo "# Last offset of atomic write: $last_offset" >> $seqres.full + + rm $tmp.aw + sleep 0.5 + + _scratch_cycle_mount + + # we want to verify all blocks around which the shutdown happened + verify_start=3D$(( last_offset - (awu_max * 5))) + if [[ $verify_start < 0 ]] + then + verify_start=3D0 + fi + + verify_end=3D$(( last_offset + (awu_max * 5))) + if [[ "$verify_end" -gt "$filesize" ]] + then + verify_end=3D$filesize + fi +} + +# test data integrity for file with written and unwritten mappings +test_data_integrity_mixed() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with mixed mappings" >> $seqres.full + create_mixed_mappings $testfile $filesize + + test_data_integrity + + verify_data_blocks $verify_start $verify_end "$expected_data_old_mixed" "= $expected_data_new" +} + +# test data integrity for file with completely written mappings +test_data_integrity_written() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with fully written mapping" >> $seqres.full + $XFS_IO_PROG -c "pwrite -b $filesize 0 $filesize" $testfile >> $seqres.fu= ll + sync $testfile + + test_data_integrity + + verify_data_blocks $verify_start $verify_end "$expected_data_old_mapped" = "$expected_data_new" +} + +# test data integrity for file with completely unwritten mappings +test_data_integrity_unwritten() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with fully unwritten mappings" >> $seqres.full + $XFS_IO_PROG -c "falloc 0 $filesize" $testfile >> $seqres.full + sync $testfile + + test_data_integrity + + verify_data_blocks $verify_start $verify_end "$expected_data_old_zeroes" = "$expected_data_new" +} + +# test data integrity for file with no mappings +test_data_integrity_hole() { + $XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Creating testfile with no mappings" >> $seqres.full + $XFS_IO_PROG -c "truncate $filesize" $testfile >> $seqres.full + sync $testfile + + test_data_integrity + + verify_data_blocks $verify_start $verify_end "$expected_data_old_zeroes" = "$expected_data_new" +} + +test_append_data_integrity() { + $XFS_IO_PROG -c "truncate 0" $testfile >> $seqres.full + + echo >> $seqres.full + echo "# Performing append atomic writes over file in background" >> $seqr= es.full + + start_atomic_write_and_shutdown + + local last_offset=3D$(tail -n 1 $tmp.aw | cut -d" " -f4) + if [[ -z $last_offset ]] + then + last_offset=3D0 + fi + + echo >> $seqres.full + echo "# Last offset of atomic write: $last_offset" >> $seqres.full + rm $tmp.aw + sleep 0.5 + + _scratch_cycle_mount + local filesize=3D$(_get_filesize $testfile) + echo >> $seqres.full + echo "# Filesize after shutdown: $filesize" >> $seqres.full + + # To confirm that the write went atomically, we check: + # 1. The last block should be a multiple of awu_max + # 2. The last block should be the completely new data + + if (( $filesize % $awu_max )) + then + echo "Filesize after shutdown ($filesize) not a multiple of atomic write= unit ($awu_max)" + fi + + verify_start=3D$(( filesize - (awu_max * 5))) + if [[ $verify_start < 0 ]] + then + verify_start=3D0 + fi + + local verify_end=3D$filesize + + # Here the blocks should always match new data hence, for simplicity of + # code, just corrupt the $expected_data_old buffer so it never matches + local expected_data_old=3D"POISON" + verify_data_blocks $verify_start $verify_end "$expected_data_old" "$expec= ted_data_new" +} + +$XFS_IO_PROG -fc "truncate 0" $testfile >> $seqres.full + +dry_run + +echo >> $seqres.full +echo "# Populating expected data buffers" >> $seqres.full +populate_expected_data + +# Loop 20 times to shake out any races due to shutdown +for ((iter=3D0; iter<20; iter++)) +do + echo >> $seqres.full + echo "------ Iteration $iter ------" >> $seqres.full + + echo >> $seqres.full + echo "# Starting data integrity test for atomic writes over mixed mapping= " >> $seqres.full + test_data_integrity_mixed + + echo >> $seqres.full + echo "# Starting data integrity test for atomic writes over fully written= mapping" >> $seqres.full + test_data_integrity_written + + echo >> $seqres.full + echo "# Starting data integrity test for atomic writes over fully unwritt= en mapping" >> $seqres.full + test_data_integrity_unwritten + + echo >> $seqres.full + echo "# Starting data integrity test for atomic writes over holes" >> $se= qres.full + test_data_integrity_hole + + echo >> $seqres.full + echo "# Starting shutdown data integrity test for append atomic writes" >= > $seqres.full + test_append_data_integrity +done + +echo "Silence is golden" +status=3D0 +exit diff --git a/tests/generic/1230.out b/tests/generic/1230.out new file mode 100644 index 00000000..d01f54ea --- /dev/null +++ b/tests/generic/1230.out @@ -0,0 +1,2 @@ +QA output created by 1230 +Silence is golden --=20 2.49.0 From nobody Thu Oct 2 19:24:13 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D8F3A36CC84; Thu, 11 Sep 2025 17:14:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610880; cv=none; b=FKsDm1L+k0Wrm/vTZ+ghIz0X/50ndi+dhC4Sl00Uwybokx6pnxj8nhA2ImYvg4b+JHlMTmgW8YavVauz6XnYirjIl4Xq3bh5arU2WWtgnhh1KPFdmEclfzUhcXsMLoLUIika+CFbMz+CqtqwLvovsRoVu+/NcX4vKh1RBXaIq3I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610880; c=relaxed/simple; bh=/dAIy0nEihrCd/H2CLq+qnSEnaJT7cWIBOh5/Pwnnmw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TEotfrmOkmmWgnLlq/jH6mWtL/ze7CM6/lGRVKnJL0pAKajE9Lay9mCbxLyRjJ1B/3BtwD/Tae8u6grVE2KrANci4myjXR8jfPZ2epSNRh0twXSB0S9vcWEtwn1uudQsYc+NviRZzm7LvX1IIPIj7/nt2jRJQANwrEjKuo2sHLw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=GcDuIt6f; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="GcDuIt6f" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58B7qqbx011724; Thu, 11 Sep 2025 17:14:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=9DamNFw+yvGG3nqW3 SMA+ylj0WVex2u22K5E7mhw8/o=; b=GcDuIt6ft9Vwt11UZv/izCYB9VeOge2nY 5tpSlipR9uyBPGOo+AN8oyiMPDTAj9W8MsXGGJPYeSkO1q2rBZgvNCFJVE4wOTG0 Xm4JuNK51JQmOV3JbYeW5XxwtggxtX/h8dgFD7W3bRdplScePMkZ9ssPodwm9cHp KUoT2mfoqjk9diX2so9muIKibbOUOhTYb6wNU4I42eRQTv+0q60flwVxpBS08jv5 wCmEBcvpwKjaT3WBgr7QJXbkR+5GRUqmswVxz2iV82uHRA69NOsEEIvgHkwHqZI/ RufUm0iPRHMiZk/JVwDsn6MLCb0BmnjL7isF1KMXphRfn52lC0CWw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490xydb1aq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:31 +0000 (GMT) Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58BHCk5n000615; Thu, 11 Sep 2025 17:14:30 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490xydb1aj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:30 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BEOVXs017181; Thu, 11 Sep 2025 17:14:29 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4911gmppqw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:29 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BHER5u58392860 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 17:14:28 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D77E120040; Thu, 11 Sep 2025 17:14:27 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6129320043; Thu, 11 Sep 2025 17:14:24 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.39.17.37]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 17:14:24 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v6 10/12] ext4: Test atomic write and ioend codepaths with bigalloc Date: Thu, 11 Sep 2025 22:43:41 +0530 Message-ID: <79f496bc075a9fe5e2778e1433e4ffd1c8ab678e.1757610403.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: -7k2JEyt3o0ubTlbzHPx8Eu_LztEPdHs X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDIzNSBTYWx0ZWRfX3n284ruYq2Qz 2tvs8JOQakCLVM9DHmBnSZXJvoM0IRqRpofOtJX0+T09co22SN765V1euJgTj/gd+PEjbhhQ+di KaXNWnCbOgv6yRSfD7nSN6M6AR7KHM92y/eNsPlOsQfo6SHtp5ETeo8qrYrFwUfUWEYbbOjtYJj qD4PmZOdyA+WGseJotZgmJfJWibX3TfuSzYCEgO4uwpqLcXkSKrRgkRwYEs5ORvoMQH1mH3rm9a nVMEPdv543EFA5VhRwgzU0GIzNJFRKbFgmZFnZ6B4nDBGjYoLZEVrGOEACafwpC9xi4lErvmEc2 9gugDMIMDJbbvgILJUIQbzUS/uDoG2Wf1kw6JfQLH0C0ErJVt0PXaZfz+Zy4SXNYV1zNENUDOpF d2NgvOyP X-Proofpoint-GUID: 1O0yTUJJx2VnzXnSJ6maKuNHMObj6vRA X-Authority-Analysis: v=2.4 cv=F59XdrhN c=1 sm=1 tr=0 ts=68c30377 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=KekteOT-tD_VQUBAREkA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_02,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 clxscore=1015 impostorscore=0 bulkscore=0 adultscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060235 Content-Type: text/plain; charset="utf-8" From: "Ritesh Harjani (IBM)" This test does a lot of parallel RWF_ATOMIC IO on a preallocated file to stress the write and end-io unwritten conversion code paths. We brute force this for different blocksize and clustersizes and after each iteration we ensure the data was not torn or corrupted using fio crc verification. Note that in this test we use overlapping atomic writes of same io size. Although serializing racing writes is not guaranteed for RWF_ATOMIC, NVMe and SCSI provide this guarantee as an inseparable feature to power-fail atomicity. Keeping the iosize as same also ensures that ext4 doesn't tear the write due to racing ioend unwritten conversion. The value of this test is that we make sure the RWF_ATOMIC is handled correctly by ext4 as well as test that the block layer doesn't split or only generate multiple bios for an atomic write. Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- tests/ext4/061 | 155 +++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/061.out | 2 + 2 files changed, 157 insertions(+) create mode 100755 tests/ext4/061 create mode 100644 tests/ext4/061.out diff --git a/tests/ext4/061 b/tests/ext4/061 new file mode 100755 index 00000000..1d61c8b0 --- /dev/null +++ b/tests/ext4/061 @@ -0,0 +1,155 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 061 +# +# This test does a lot of parallel RWF_ATOMIC IO on a preallocated file to +# stress the write and end-io unwritten conversion code paths. We brute fo= rce +# this for all possible blocksize and clustersizes and after each iteratio= n we +# ensure the data was not torn or corrupted using fio crc verification. +# +# Note that in this test we use overlapping atomic writes of same io size. +# Although serializing racing writes is not guaranteed for RWF_ATOMIC, NVM= e and +# SCSI provide this guarantee as an inseparable feature to power-fail +# atomicity. Keeping the iosize as same also ensures that ext4 doesn't tea= r the +# write due to racing ioend unwritten conversion. +# +# The value of this test is that we make sure the RWF_ATOMIC is handled +# correctly by ext4 as well as test that the block layer doesn't split or = only +# generate multiple bios for an atomic write. + +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto rw stress atomicwrites + +_require_scratch_write_atomic +_require_fio_atomic_writes +_require_aiodio + +FIO_LOAD=3D$(($(nproc) * 2 * LOAD_FACTOR)) +SIZE=3D$((100*1024*1024)) + +# Calculate fsblocksize as per bdev atomic write units. +bdev_awu_min=3D$(_get_atomic_write_unit_min $SCRATCH_DEV) +bdev_awu_max=3D$(_get_atomic_write_unit_max $SCRATCH_DEV) +bs=3D$(_max 4096 "$bdev_awu_min") + +function create_fio_configs() +{ + local bsize=3D$1 + create_fio_aw_config $bsize + create_fio_verify_config $bsize +} + +function create_fio_verify_config() +{ + local bsize=3D$1 +cat >$fio_verify_config <$fio_aw_config <> $seqres.full 2>&1 || return + if _try_scratch_mount >> $seqres.full 2>&1; then + echo "=3D=3D Testing: bs=3D$bs cs=3D$cs iosize=3D$iosize =3D=3D" >> $seq= res.full + + touch $SCRATCH_MNT/f1 + create_fio_configs $iosize + + cat $fio_aw_config >> $seqres.full + echo >> $seqres.full + cat $fio_verify_config >> $seqres.full + + $FIO_PROG $fio_aw_config >> $seqres.full + ret1=3D$? + + $FIO_PROG $fio_verify_config >> $seqres.full + ret2=3D$? + + _scratch_unmount + + [[ $ret1 -eq 0 && $ret2 -eq 0 ]] || _fail "fio with atomic write failed" + fi +} + +run_test() { + local bs=3D$1 + + # cluster sizes above 16 x blocksize are experimental so avoid them + # Also, cap cluster size at 128kb to keep it reasonable for large + # blocks size + max_cs=3D$(_min $((16 * bs)) "$bdev_awu_max" $((128 * 1024))) + + # Fuzz for combinations of blocksize, clustersize and + # iosize that cover most of the cases + run_test_one $bs $bs $bs + run_test_one $bs $max_cs $bs + run_test_one $bs $max_cs $max_cs + run_test_one $bs $max_cs $(_max "$((max_cs/2))" $bs) +} + +# Let's create a sample fio config to check whether fio supports all optio= ns. +fio_aw_config=3D$tmp.aw.fio +fio_verify_config=3D$tmp.verify.fio +fio_out=3D$tmp.fio.out + +create_fio_configs $bs +_require_fio $fio_aw_config + +for ((bs=3D$bs; bs <=3D $(_get_page_size); bs =3D $bs << 1)); do + run_test $bs +done + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/ext4/061.out b/tests/ext4/061.out new file mode 100644 index 00000000..273be9e0 --- /dev/null +++ b/tests/ext4/061.out @@ -0,0 +1,2 @@ +QA output created by 061 +Silence is golden --=20 2.49.0 From nobody Thu Oct 2 19:24:13 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5678936CDFC; Thu, 11 Sep 2025 17:14:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610885; cv=none; b=DBkNfSgc2ldhBjRdYOLDpBd1D1W1CgROP4GpiCMiDr2q6Xgum6RvXfzeAllTGc2Q1MZaosNUxLFskFgBAaX9+JAgfKgeAf5vf5ENOuAbSckoWWLyPmnoP2vuP0tgd7RL7nuMsp7Eg87v5hnq+IBWhgXCST/c16VYVnbhk3ADZnM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610885; c=relaxed/simple; bh=AbuoGFdbt5Nlsepwq1pcM/3iTAcG8d6i8S+4mWjlAYE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Vl+9a1XwCH5lWBiSq+bwlbG/UcXS3ZuWKkytG/IxNi5ED+XjjzmOsPTtWSfibZ0wII5VslImnv+SMvxzPI/VXxmW4R6zJDJ00rgwSAA25cOjZszAHw4pwsuiXyz+Qau/jB0WhyuKcUK9l/BUj2h5E4yi5AcNij+k2QAFbTVsnaA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=iZqR/7iD; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="iZqR/7iD" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BEforI003430; Thu, 11 Sep 2025 17:14:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=DTRxxUzb0Dsn0/GBD P5OgVL94Y67Y6ZSK7VjxTYr85w=; b=iZqR/7iDKPkJEJ7Web7O3q6vLniaEQbtm qdtX/r7+8K+4LA+RBQtLudi4KAggoiag5Lt42sZqMFLl9jiRtj6M2vTvieuY1v+f 6X9GBRszNQOCkGiRM2v2FJpBHPEqPn8Q6UHzSo6RA3NZbjCTdsv6VqrP5PDAo4va Ob1k7SKg+/f1a8dceAiRhy6fshBFLJXFpEUVlXL07QeJCUYRKC4A9gtEflAOS1Dr zAaMvTWfzY/m5HDQ9zfB/5CWw/A24hCVLaEk1Zgvri/AHkPB3Y4iWMXNg1IGCE9K 3dpFmIUvlAkrHtzeCt9U3CL3tl64/6fsgutPjdVhvMxojTtONYQtQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490cffp7b1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:35 +0000 (GMT) Received: from m0360083.ppops.net (m0360083.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58BHEZZI031056; Thu, 11 Sep 2025 17:14:35 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490cffp7ax-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:35 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BEhF9B001156; Thu, 11 Sep 2025 17:14:34 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 491203pmp6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:34 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BHEW4l44761526 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 17:14:32 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5ED5A2004B; Thu, 11 Sep 2025 17:14:32 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 789B920043; Thu, 11 Sep 2025 17:14:28 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.39.17.37]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 17:14:28 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v6 11/12] ext4: Test atomic writes allocation and write codepaths with bigalloc Date: Thu, 11 Sep 2025 22:43:42 +0530 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: lUrgXj86GvGn1UaRciA1pKtiElJu-Ta2 X-Proofpoint-GUID: ReZgfDIdaIhecAkleHkjx9gqQIW1E2Gv X-Authority-Analysis: v=2.4 cv=EYDIQOmC c=1 sm=1 tr=0 ts=68c3037b cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=yJojWOMRYYMA:10 a=pGLkceISAAAA:8 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=VnNF1IyMAAAA:8 a=o-vKpyFH2n2aPF30nXYA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDAyMCBTYWx0ZWRfX3PybK4tbqx44 vblzkjFqHSndGt9ZPfao9Qie8cvXnjITt6SVxvq/gC3daywTvZ4Gde6zrVfW3g90KHVaWNyf0bS XOeY4i1NJMwmlnykjezuJbdUdjGM1hlAVWzoEA0idp7A7Z0m3xpwsQsc0hcuprCh50qB8dVpjiC qo44nRnwdHvoVUmSC8a56DHwd7LIV2xAvovtlQRKfo7oL1y2UVSWPUXrMG1GznF9ozOnUVIeRLm 7llCyNdczqSWm0N3ItUJwA2xdghYepmRpRDA89e1g19nIH+OEm1AXQe0aTNCiWSRzmitjh2y2dD VkIqG+yp5Ma13rBs3BrL4jsglC0JgUv8YWk6Mdc2gL/sBW/bp6ogJlSvSW5O9g+DXa1PqizkAxQ Zs7Kp3QK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_02,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 adultscore=0 suspectscore=0 spamscore=0 impostorscore=0 priorityscore=1501 phishscore=0 clxscore=1015 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060020 Content-Type: text/plain; charset="utf-8" From: "Ritesh Harjani (IBM)" This test does a parallel RWF_ATOMIC IO on a multiple truncated files in a small FS. The idea is to stress ext4 allocator to ensure we are able to handle low space scenarios correctly with atomic writes. We brute force this for different blocksize and clustersizes and after each iteration we ensure the data was not torn or corrupted using fio crc verification. Note that in this test we use overlapping atomic writes of same io size. Although serializing racing writes is not guaranteed for RWF_ATOMIC, NVMe and SCSI provide this guarantee as an inseparable feature to power-fail atomicity. Keeping the iosize as same also ensures that ext4 doesn't tear the write due to racing ioend unwritten conversion. The value of this test is that we make sure the RWF_ATOMIC is handled correctly by ext4 as well as test that the block layer doesn't split or only generate multiple bios for an atomic write. Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong Reviewed-by: John Garry Signed-off-by: Ojaswin Mujoo --- tests/ext4/062 | 203 +++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/062.out | 2 + 2 files changed, 205 insertions(+) create mode 100755 tests/ext4/062 create mode 100644 tests/ext4/062.out diff --git a/tests/ext4/062 b/tests/ext4/062 new file mode 100755 index 00000000..05cce696 --- /dev/null +++ b/tests/ext4/062 @@ -0,0 +1,203 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# FS QA Test 062 +# +# This test does a parallel RWF_ATOMIC IO on a multiple truncated files in= a +# small FS. The idea is to stress ext4 allocator to ensure we are able to +# handle low space scenarios correctly with atomic writes.. We brute force= this +# for all possible blocksize and clustersizes and after each iteration we +# ensure the data was not torn or corrupted using fio crc verification. +# +# Note that in this test we use overlapping atomic writes of same io size. +# Although serializing racing writes is not guaranteed for RWF_ATOMIC, NVM= e and +# SCSI provide this guarantee as an inseparable feature to power-fail +# atomicity. Keeping the iosize as same also ensures that ext4 doesn't tea= r the +# write due to racing ioend unwritten conversion. +# +# The value of this test is that we make sure the RWF_ATOMIC is handled +# correctly by ext4 as well as test that the block layer doesn't split or = only +# generate multiple bios for an atomic write. +# + +. ./common/preamble +. ./common/atomicwrites + +_begin_fstest auto rw stress atomicwrites + +_require_scratch_write_atomic +_require_fio_atomic_writes +_require_aiodio + +FSSIZE=3D$((360*1024*1024)) +FIO_LOAD=3D$(($(nproc) * LOAD_FACTOR)) + +# Calculate bs as per bdev atomic write units. +bdev_awu_min=3D$(_get_atomic_write_unit_min $SCRATCH_DEV) +bdev_awu_max=3D$(_get_atomic_write_unit_max $SCRATCH_DEV) +bs=3D$(_max 4096 "$bdev_awu_min") + +function create_fio_configs() +{ + local bsize=3D$1 + create_fio_aw_config $bsize + create_fio_verify_config $bsize +} + +function create_fio_verify_config() +{ + local bsize=3D$1 +cat >$fio_verify_config <$fio_aw_config <> $seqres.full 2>&1 || return + if _try_scratch_mount >> $seqres.full 2>&1; then + echo "Testing: bs=3D$bs cs=3D$cs iosize=3D$iosize" >> $seqres.full + + touch $SCRATCH_MNT/f1 + create_fio_configs $iosize + + cat $fio_aw_config >> $seqres.full + cat $fio_verify_config >> $seqres.full + + $FIO_PROG $fio_aw_config >> $seqres.full + ret1=3D$? + + $FIO_PROG $fio_verify_config >> $seqres.full + ret2=3D$? + + _scratch_unmount + + [[ $ret1 -eq 0 && $ret2 -eq 0 ]] || _fail "fio with atomic write failed" + fi +} + +run_test() { + local bs=3D$1 + + # cluster sizes above 16 x blocksize are experimental so avoid them + # Also, cap cluster size at 128kb to keep it reasonable for large + # blocks size + max_cs=3D$(_min $((16 * bs)) "$bdev_awu_max" $((128 * 1024))) + + # Fuzz for combinations of blocksize, clustersize and + # iosize that cover most of the cases + run_test_one $bs $bs $bs + run_test_one $bs $max_cs $bs + run_test_one $bs $max_cs $max_cs + run_test_one $bs $max_cs $(_max "$((max_cs/2))" $bs) +} + +# Let's create a sample fio config to check whether fio supports all optio= ns. +fio_aw_config=3D$tmp.aw.fio +fio_verify_config=3D$tmp.verify.fio +fio_out=3D$tmp.fio.out + +create_fio_configs $bs +_require_fio $fio_aw_config + +for ((bs=3D$bs; bs <=3D $(_get_page_size); bs =3D $bs << 1)); do + run_test $bs $cs $iosize +done + +# success, all done +echo Silence is golden +status=3D0 +exit diff --git a/tests/ext4/062.out b/tests/ext4/062.out new file mode 100644 index 00000000..a1578f48 --- /dev/null +++ b/tests/ext4/062.out @@ -0,0 +1,2 @@ +QA output created by 062 +Silence is golden --=20 2.49.0 From nobody Thu Oct 2 19:24:13 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96F4A362069; Thu, 11 Sep 2025 17:14:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610892; cv=none; b=h66XKA9AK5SnuN6fzrv1w6nos+09y/+ZLYAJUdMxs6boFUYjhNSiOLO7gjWtsNjyDH45+RqGr1RP+VEbaTFZferzfn3jDib/YzYT/zMABgDbTTDcrGNt8B40nzl04qEejZ3t/j1jIhX5nbibU2SiVvPn+56mVA6KNNaenaqWqJE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757610892; c=relaxed/simple; bh=/23TnMcmUpdflLUBR/TKi4uqya8y8NfSo8bYiPAZYPk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kZyRMZ7+Ug7tSqLClvwxtPnKgQ5oaeq9+ebEyKtVNE7W0QpAfqpMliMIYR1jW76cRRWAMeWzO95pC2xKkhaj5PIMJup7+PFtBksgLVkFPmqbmXYj4zGXyiM2wXkJD+xXxr4D+t9N0dzwMkXmuWtCHx7WEQgrLwP5v2Ut+Uo0E9M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=g7oFKh65; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="g7oFKh65" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58B9GCIN001309; Thu, 11 Sep 2025 17:14:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=vRTNUGKdoq8A+YlHn I5gzyY7Rjkkw4gAzlowW/XKnQg=; b=g7oFKh65f706tQETRS7T4gleRsUAX2pNz 3OFuS66GSxg0hXXYohBQE57Zemucju8UwN7S35d+1NYkVEyc0Mkyv80PCy5yoVQa U/eUBbjJS3b3NMV51C72uD4VRZZ2pieiq8rofJNquE/t3gbA5vCifPryZLR/xjuR LQzChNtnR+xrrV5GoL6OyElEsCNBKVVUwwuDZJxqrCH6FUe0ziBU9L2lfYRMKllD Y7PA1TkG3s7RL/RmrdMOfsFnRQbz30CI47wUYPeiLI4cHD6hnSph+mFhareNdoeX //TQG7OariD2tih8rdiL/xupvZqPgbo/hjnxUYxuDmDqcgfBp8o1g== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490acrdgj6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:39 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 58BH5MaS018264; Thu, 11 Sep 2025 17:14:38 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490acrdgj4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:38 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BFsZWT011447; Thu, 11 Sep 2025 17:14:37 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 490y9uq553-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 17:14:37 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BHEajE32113302 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 17:14:36 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3706C2004B; Thu, 11 Sep 2025 17:14:36 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D329520043; Thu, 11 Sep 2025 17:14:32 +0000 (GMT) Received: from li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com (unknown [9.39.17.37]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 17:14:32 +0000 (GMT) From: Ojaswin Mujoo To: Zorro Lang , fstests@vger.kernel.org Cc: Ritesh Harjani , djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH v6 12/12] ext4: Atomic write test for extent split across leaf nodes Date: Thu, 11 Sep 2025 22:43:43 +0530 Message-ID: <4ac353bfdf94c896345fdd308467a9666cb3d6f3.1757610403.git.ojaswin@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: A2SUQ2zU8gdMPIWg3z9EEzhwjt9p9N6w X-Authority-Analysis: v=2.4 cv=Mp1S63ae c=1 sm=1 tr=0 ts=68c3037f cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=yJojWOMRYYMA:10 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=BtNrGSYHu5IGmLxdGZgA:9 X-Proofpoint-ORIG-GUID: B9xwJd3nktRtiesckJVwiPCfb9yu33wk X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDAwMCBTYWx0ZWRfXxz0Gk4ElXfeB WNK84P68+k0tITlGE2QA+SSEvLGCnIhjFyrBVU/+GhkH214lPjhxZUCLrd2EDs0HxEMRTdm7Xx4 sAbFl5JyWshz1Kf1ivPuRYjrg7VVaniHS9041ciTJsjzAZdU8frFKFUFMmfpdlo44DvnO6hRFyR 47mxaoapJEYO1goT6ksIrSboNJVIF0ttNh0OiuLG7+JAFAusDLxwd+5n84WBbcb6iesGiPdu4hb NoerEo15LcWXvBD1hJTI2wE2JPgnJjNYYA59yIb4YenSrlXCzAw7kKhLXaI0j19mIEXFsU0q5ZF 6zbGtI4GeI0eP5fJQ3SBhwHhkb1hKZaTg9N9GbdXrd/F8ivFsV4GmeRWzWppNoJUZg/loE0YGdA 7pLENdrE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_02,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 malwarescore=0 clxscore=1015 phishscore=0 spamscore=0 adultscore=0 priorityscore=1501 bulkscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060000 Content-Type: text/plain; charset="utf-8" In ext4, even if an allocated range is physically and logically contiguous, it can still be split into 2 extents. This is because ext4 does not merge extents across leaf nodes. This is an issue for atomic writes since even for a continuous extent the map block could (in rare cases) return a shorter map, hence tearning the write. This test creates such a file and ensures that the atomic write handles this case correctly Reviewed-by: Darrick J. Wong Signed-off-by: Ojaswin Mujoo --- tests/ext4/063 | 129 +++++++++++++++++++++++++++++++++++++++++++++ tests/ext4/063.out | 2 + 2 files changed, 131 insertions(+) create mode 100755 tests/ext4/063 create mode 100644 tests/ext4/063.out diff --git a/tests/ext4/063 b/tests/ext4/063 new file mode 100755 index 00000000..9d6265a8 --- /dev/null +++ b/tests/ext4/063 @@ -0,0 +1,129 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 IBM Corporation. All Rights Reserved. +# +# In ext4, even if an allocated range is physically and logically contiguo= us, +# it can still be split into 2 or more extents. This is because ext4 does = not +# merge extents across leaf nodes. This is an issue for atomic writes since +# even for a continuous extent the map block could (in rare cases) return a +# shorter map, hence tearing the write. This test creates such a file and +# ensures that the atomic write handles this case correctly +# +. ./common/preamble +. ./common/atomicwrites +_begin_fstest auto atomicwrites + +_require_scratch_write_atomic_multi_fsblock +_require_atomic_write_test_commands +_require_command "$DEBUGFS_PROG" debugfs + +prep() { + local bs=3D`_get_block_size $SCRATCH_MNT` + local ex_hdr_bytes=3D12 + local ex_entry_bytes=3D12 + local entries_per_blk=3D$(( (bs - ex_hdr_bytes) / ex_entry_bytes )) + + # fill the extent tree leaf with bs len extents at alternate offsets. + # The tree should look as follows + # + # +---------+---------+ + # | index 1 | index 2 | + # +-----+---+-----+---+ + # +------+ +-----------+ + # | | + # +-------+-------+---+---------+ +-----+----+ + # | ex 1 | ex 2 | | ex n | | ex n+1 | + # | off:0 | off:2 |...| off:678 | | off:680 | + # | len:1 | len:1 | | len:1 | | len:1 | + # +-------+-------+---+---------+ +----------+ + # + for i in $(seq 0 $entries_per_blk) + do + $XFS_IO_PROG -fc "pwrite -b $bs $((i * 2 * bs)) $bs" $testfile > /dev/nu= ll + done + sync $testfile + + echo >> $seqres.full + echo "Create file with extents spanning 2 leaves. Extents:">> $seqres.full + echo "...">> $seqres.full + $DEBUGFS_PROG -R "ex `basename $testfile`" $SCRATCH_DEV |& tail >> $seqre= s.full + + # Now try to insert a new extent ex(new) between ex(n) and ex(n+1). + # Since this is a new FS the allocator would find continuous blocks + # such that ex(n) ex(new) ex(n+1) are physically(and logically) + # contiguous. However, since we don't merge extents across leaf we will + # end up with a tree as: + # + # +---------+---------+ + # | index 1 | index 2 | + # +-----+---+-----+---+ + # +------+ +------------+ + # | | + # +-------+-------+---+---------+ +------+-----------+ + # | ex 1 | ex 2 | | ex n | | ex n+1 (merged) | + # | off:0 | off:2 |...| off:678 | | off:679 | + # | len:1 | len:1 | | len:1 | | len:2 | + # +-------+-------+---+---------+ +------------------+ + # + echo >> $seqres.full + torn_ex_offset=3D$((((entries_per_blk * 2) - 1) * bs)) + $XFS_IO_PROG -c "pwrite $torn_ex_offset $bs" $testfile >> /dev/null + sync $testfile + + echo >> $seqres.full + echo "Perform 1 block write at $torn_ex_offset to create torn extent. Ext= ents:">> $seqres.full + echo "...">> $seqres.full + $DEBUGFS_PROG -R "ex `basename $testfile`" $SCRATCH_DEV |& tail >> $seqre= s.full + + _scratch_cycle_mount +} + +_scratch_mkfs >> $seqres.full +_scratch_mount >> $seqres.full + +testfile=3D$SCRATCH_MNT/testfile +touch $testfile +awu_max=3D$(_get_atomic_write_unit_max $testfile) + +echo >> $seqres.full +echo "# Prepping the file" >> $seqres.full +prep + +torn_aw_offset=3D$((torn_ex_offset - (torn_ex_offset % awu_max))) + +echo >> $seqres.full +echo "# Performing atomic IO on the torn extent range. Command: " >> $seqr= es.full +echo $XFS_IO_PROG -c "open -fsd $testfile" -c "pwrite -S 0x61 -DA -V1 -b $= awu_max $torn_aw_offset $awu_max" >> $seqres.full +$XFS_IO_PROG -c "open -fsd $testfile" -c "pwrite -S 0x61 -DA -V1 -b $awu_m= ax $torn_aw_offset $awu_max" >> $seqres.full + +echo >> $seqres.full +echo "Extent state after atomic write:">> $seqres.full +echo "...">> $seqres.full +$DEBUGFS_PROG -R "ex `basename $testfile`" $SCRATCH_DEV |& tail >> $seqres= .full + +echo >> $seqres.full +echo "# Checking data integrity" >> $seqres.full + +# create a dummy file with expected data +$XFS_IO_PROG -fc "pwrite -S 0x61 -b $awu_max 0 $awu_max" $testfile.exp >> = /dev/null +expected_data=3D$(od -An -t x1 -j 0 -N $awu_max $testfile.exp) + +# We ensure that the data after atomic writes should match the expected da= ta +actual_data=3D$(od -An -t x1 -j $torn_aw_offset -N $awu_max $testfile) +if [[ "$actual_data" !=3D "$expected_data" ]] +then + echo "Checksum match failed at off: $torn_aw_offset size: $awu_max" + echo + echo "Expected: " + echo "$expected_data" + echo + echo "Actual contents: " + echo "$actual_data" + + _fail +fi + +echo -n "Data verification at offset $torn_aw_offset succeeded!" >> $seqre= s.full +echo "Silence is golden" +status=3D0 +exit diff --git a/tests/ext4/063.out b/tests/ext4/063.out new file mode 100644 index 00000000..de35fc52 --- /dev/null +++ b/tests/ext4/063.out @@ -0,0 +1,2 @@ +QA output created by 063 +Silence is golden --=20 2.49.0