From nobody Sun Jun 14 17:34:54 2026 Received: from mail-dy1-f226.google.com (mail-dy1-f226.google.com [74.125.82.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E14723909C for ; Sat, 4 Apr 2026 03:23:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.226 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775273039; cv=none; b=O+Vd6xNL3LfLepqRbbOIv7pRJjEPWZ3X+HTQy2KHI35QDMv6UFgKQTdKmt3PRSXRcm1mywkAfmKKyerWTlFZ5QoGBmWkVB2iNIy5wpL1mKIJNHb1iIgyJHeMI8EeOK9Nfh5XcvCLfZBTnwa5KHVRzZ2CjTddTxpuAWxnhxsclIY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775273039; c=relaxed/simple; bh=GxzP0crHuoLNsQ25wbUfZ0tt2BhG4K8QmO1b7YEBuI8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dQGkrx2z/CF8HqEodLW/RZ7gK1lmMGStiEN3HOBVoCx9GczLQvb5238Q8RduIe3sLAyR4ElcivLPNWkRD3XYSJK1xVSDR19RVaKTgmGpTyN8Ay97g5t6Qjgc+TwQOS9zzuGpLdI2GzG2QrYVR6x0dmMSoZym31OikdN5TQl1DGI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=go8Xio4g; arc=none smtp.client-ip=74.125.82.226 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="go8Xio4g" Received: by mail-dy1-f226.google.com with SMTP id 5a478bee46e88-2ba895adfeaso2713963eec.0 for ; Fri, 03 Apr 2026 20:23:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1775273037; x=1775877837; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=EDuvDbXHwkEle99hrrUH04L5WbIEpbGMjepkPBSVZYU=; b=go8Xio4ghx3eymfjvTYS/YN/FGss6bGia32mXJuZrDwwHpirJj4oz2ObVWfAqfgTzd 1dL5+pq4N0Z2o2dVufKBCVqOZTVyfwnCkjIvlgYFGQhD9CRWwECToQVPySRwBZWkcVfd xs8ivljxd3vTpXa+x+iwtsKZiUxeEG2QjuSs5TOc8nNLrLg36R4JBus62UY+FvVTXs64 5L1uwb4dK31WNcjwkDx0M+SIgdoSOUP1OAnU3SLOyXeUyjifc5Y3SyB3s+Dn6Ote3zWL WpB12OnuT/nuH7x84kW4ll1wwW0izSDIMlrvHlYCiiOSXqSjSdoBO7zL8HGIQkUeZITJ qKBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775273037; x=1775877837; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=EDuvDbXHwkEle99hrrUH04L5WbIEpbGMjepkPBSVZYU=; b=pJtyxVt6iv/Uquqz283lDZAbMvLlP5Kwh0sK8UREiYQyLTP0ogcUu/CsQe3aH7SfHK Wl9bYiQt5I6jLS8XGlZzKNI1UYgMJu0Yi4XBaU/ostSqGToyeVAdDh2+ANDAHZP54XYL 15hKm8RSmaeiNTywIYA/MbUZOukDMMme8ky87iFIbP1kMVr/69Od2AmsVTz+B2WEChHN 33Hzm1GAR/Nib0mV3R09odmw4D3SBQ5Sc6IjEd9YSfOkEu8MlxZGjkX7Vj5uhCo7TaLC KPIAZHrBKKVjFwMHkegdktBgHWcv8XNg7yhVX6fHPGRGRfvdcdTTPgoAtfNx36ZMQBF+ bOqw== X-Forwarded-Encrypted: i=1; AJvYcCXwN6WctChWiYJpT9jYzr9eGTQ2lIuhCE9Jrs7PFp+/EeC3rViHgQLNX0ETRQXEEcRbeNwcQhKwOtMXRx8=@vger.kernel.org X-Gm-Message-State: AOJu0YyCGXy/keM9S8Fe3QNRvaXoNOcD4G6FK2ct7K3gWIe8pFT5clfY ZqgwTtgI0yF5e2EzwjPTsuweNmvWIpaIuZU74TOG7mD2Uv205p6V3CP+S0m1pNB2/GKaJJOru/K gHi6fJybDWKoXK2RBNBv3qrbmA0eYoRDCFiSs X-Gm-Gg: AeBDiesfB2WsfV3BOCziOVfkruKFAj/ChPFXf/aPMQZ5WmQcEHmE+sIU1wmlZ9OeSbw x2SVuKWp77shq1CfqvQu/q5H5i1vkVEvqDJ4aJcIdaQd3TsRVUuL1SgDUu3fYgmdSlA2Fn8coTw 48oX5z41XSlCGUdnYU5B6JHXXHhoDuQAjkMlm8d9yXjYpEGt2YiJ8tDoJOWKZ9a1Y9+hXyzW+oG kEw4hdKFhXBAPV/Yq/iWuG4D79ADpK+G7nnPpW42g4jpvSYuR8nFLYCREZkxImDrJqA3ConZtss 10hI3BIKLO7YIAuGFuUZMnFy/JDoLjgqA7BZUSZqNqkycNgPljl5K8onb+ajpq7O6CjDnotSYvJ PvZgB6YtGVPw5/MOH+uvj+43ydeP1FWZx9tXyHBHo1Yb2BAGhvwgoSdc= X-Received: by 2002:a05:7300:bc86:b0:2cb:53f5:553e with SMTP id 5a478bee46e88-2cbf64ab2c5mr2500520eec.0.1775273037061; Fri, 03 Apr 2026 20:23:57 -0700 (PDT) Received: from c7-smtp-2023.dev.purestorage.com ([208.88.159.128]) by smtp-relay.gmail.com with ESMTPS id 5a478bee46e88-2ca793f55d3sm518145eec.11.2026.04.03.20.23.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Apr 2026 20:23:57 -0700 (PDT) X-Relaying-Domain: purestorage.com Received: from dev-ushankar.dev.purestorage.com (unknown [IPv6:2620:125:9007:640:ffff::2c94]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id 9F4AA3405EC; Fri, 3 Apr 2026 21:23:55 -0600 (MDT) Received: by dev-ushankar.dev.purestorage.com (Postfix, from userid 1557716368) id BC324E54EC9; Fri, 3 Apr 2026 21:23:55 -0600 (MDT) From: Uday Shankar Date: Fri, 03 Apr 2026 21:23:55 -0600 Subject: [PATCH 1/2] ublk: reset per-IO canceled flag on each fetch Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260403-cancel-v1-1-86e5a6b3d3af@purestorage.com> References: <20260403-cancel-v1-0-86e5a6b3d3af@purestorage.com> In-Reply-To: <20260403-cancel-v1-0-86e5a6b3d3af@purestorage.com> To: Ming Lei , Caleb Sander Mateos , Jens Axboe , Shuah Khan Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Uday Shankar X-Mailer: b4 0.14.2 If a ublk server starts recovering devices but dies before issuing fetch commands for all IOs, cancellation of the fetch commands that were successfully issued may never complete. This is because the per-IO canceled flag can remain set even after the fetch for that IO has been submitted - the per-IO canceled flags for all IOs in a queue are reset together only once all IOs for that queue have been fetched. So if a nonempty proper subset of the IOs for a queue are fetched when the ublk server dies, the IOs in that subset will never successfully be canceled, as their canceled flags remain set, and this prevents ublk_cancel_cmd from actually calling io_uring_cmd_done on the commands, despite the fact that they are outstanding. Fix this by resetting the per-IO cancel flags immediately when each IO is fetched instead of waiting for all IOs for the queue (which may never happen). Signed-off-by: Uday Shankar Reviewed-by: Ming Lei Reviewed-by: zhang, the-essence-of-life. --- drivers/block/ublk_drv.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c index 3ba7da94d31499590a06a8b307ed151919a027cb..92dabeb820344107c9fadfae943= 96082b933d84e 100644 --- a/drivers/block/ublk_drv.c +++ b/drivers/block/ublk_drv.c @@ -2916,22 +2916,26 @@ static void ublk_stop_dev(struct ublk_device *ub) ublk_cancel_dev(ub); } =20 +static void ublk_reset_io_flags(struct ublk_queue *ubq, struct ublk_io *io) +{ + /* UBLK_IO_FLAG_CANCELED can be cleared now */ + spin_lock(&ubq->cancel_lock); + io->flags &=3D ~UBLK_IO_FLAG_CANCELED; + spin_unlock(&ubq->cancel_lock); +} + /* reset per-queue io flags */ static void ublk_queue_reset_io_flags(struct ublk_queue *ubq) { - int j; - - /* UBLK_IO_FLAG_CANCELED can be cleared now */ spin_lock(&ubq->cancel_lock); - for (j =3D 0; j < ubq->q_depth; j++) - ubq->ios[j].flags &=3D ~UBLK_IO_FLAG_CANCELED; ubq->canceling =3D false; spin_unlock(&ubq->cancel_lock); ubq->fail_io =3D false; } =20 /* device can only be started after all IOs are ready */ -static void ublk_mark_io_ready(struct ublk_device *ub, u16 q_id) +static void ublk_mark_io_ready(struct ublk_device *ub, u16 q_id, + struct ublk_io *io) __must_hold(&ub->mutex) { struct ublk_queue *ubq =3D ublk_get_queue(ub, q_id); @@ -2940,6 +2944,7 @@ static void ublk_mark_io_ready(struct ublk_device *ub= , u16 q_id) ub->unprivileged_daemons =3D true; =20 ubq->nr_io_ready++; + ublk_reset_io_flags(ubq, io); =20 /* Check if this specific queue is now fully ready */ if (ublk_queue_ready(ubq)) { @@ -3202,7 +3207,7 @@ static int ublk_fetch(struct io_uring_cmd *cmd, struc= t ublk_device *ub, if (!ret) ret =3D ublk_config_io_buf(ub, io, cmd, buf_addr, NULL); if (!ret) - ublk_mark_io_ready(ub, q_id); + ublk_mark_io_ready(ub, q_id, io); mutex_unlock(&ub->mutex); return ret; } @@ -3610,7 +3615,7 @@ static int ublk_batch_prep_io(struct ublk_queue *ubq, ublk_io_unlock(io); =20 if (!ret) - ublk_mark_io_ready(data->ub, ubq->q_id); + ublk_mark_io_ready(data->ub, ubq->q_id, io); =20 return ret; } --=20 2.34.1 From nobody Sun Jun 14 17:34:54 2026 Received: from mail-qk1-f226.google.com (mail-qk1-f226.google.com [209.85.222.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB9882EC086 for ; Sat, 4 Apr 2026 03:23:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.226 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775273041; cv=none; b=TirxENhoDO/OQJ2A23kBl7k+OA3HWJv3aqXuXHVuj8eY7MIONtcbHsGaPAbt7Jc9IMjxWDDnDzZLLYIzR/sQZFsHWY27rPxzwxdSC064h3ZeV9pDYznGZMq+3zv8SApk50i/1OhFpD7Nwb1v38xnegik3AtPy6IxyS+17kbqPGM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775273041; c=relaxed/simple; bh=EtIon0eFvVS3O6Xdg0WbBq1qhCViHvnzFkQSYVO/8H8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=a9yHaZ0FsBsQKUbA50xOcVQTsaY2JL1zyj9AVZQsTsq4hWAPlc7YWZLkZXv22J6lDkvr9WCuMUSt9GTlpuLy0j5KtEKT6ZZVIxRnc0kEVzyYwVzcevw0nVT5GMPHRAgLIp/nizg8vQMltDcejx5IPZi8eV1AaxwW4bHDkKNrLTE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=IKxg+0yK; arc=none smtp.client-ip=209.85.222.226 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="IKxg+0yK" Received: by mail-qk1-f226.google.com with SMTP id af79cd13be357-8d00cf835b7so313207385a.1 for ; Fri, 03 Apr 2026 20:23:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1775273037; x=1775877837; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=zX/5TzotSq2dC2MEEK2IilrTISaWwcCASj/MEFDm/KQ=; b=IKxg+0yKYnRx0xrQxTq2BUklL4qslZDAxzyBoX2Te+WSySbT7pTy+txia0A6iw/iPn c1OkBFmjyrR610TIPp+x5GSyoD5VgVbgCLOqLunLumW3FSQeHeANHMLEJVwK/JyfEwyP SpcW/yfTRwq72ozOt0FQ2wIyvbA0jBEVzxmP1o57nAKM0fvhMfGlWyqgEHp4wLN3exMB p20SLbxMmYAEQAnJHSGUYP8etn4qV66Tb8qyCV4cf2sZu1Z33lM2YtxWL2E/7NooCU7M iPvbfN9tYzIsc4JTs6cQ49qjONijWXZEa7bR9BOtWlJvT7Ow2WKwCjhYLDbOE4xn26Zu IAXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775273037; x=1775877837; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=zX/5TzotSq2dC2MEEK2IilrTISaWwcCASj/MEFDm/KQ=; b=Xiiv4lRRlCDLIAOc7QWB+c6Dqtmlyt1rMRSspxO0OL2Uf4GtINWpEBN4MzzXtsQ5Xa Lnqv9xwStXBSyF5rA19zLIFSbilLgcFWnXcXS3K3zw2loPJKlvL+E6wha5Gi1ZJvcLM7 M+q046t0cH9lO+Ux60xyKqoEtYiWsDUbP02cWaYE8a0FGvn9s15gDVE+YOBjOOxbE1Gg i7bMiFy/e6XpabFGpwm/zpDEHWbF9mTmytroEQYs/60+1+sl4ootHbzU1QMDYm+pCYb1 BrOo/u2o/O8N8NV1KiHL/OBSzCoUp9K2h/06iRx2RgE/h+L6X4uUbymNlAQ+F3JqZY6e YDjQ== X-Forwarded-Encrypted: i=1; AJvYcCVeUgR2WH+j9EuvN1u1ZHXH3QvWXkYqfQmIyomg/5Nu1ByZHg3ESTj0fmAmnPej/rke0y/wVeD53xPWdn4=@vger.kernel.org X-Gm-Message-State: AOJu0YziF/ufLd/fR7s6BCHvIyxbOS3WF9xUZhxvxCC7CTeDYa11COlE dStJXL27FKLJnhfIiGA+1WfC1QEYy5UBD+s/AiXFZOwZP6drEyw8Nphh++gN0zRtL4KO/c5t4co 6CN6ytkhGCH828z9wvBMGLBbokEI3i/ylzD9K X-Gm-Gg: AeBDiesi2WkShAOo4/b9QF/gkakegUlcTgVzlsq0BG7W+I9Wdl7yISqF8fyCsr4HitN rkc5svAGWgRqKKTqP2WxCKf0F3cLAuFHRmOk1kFoJCQu/aUCQVanTGIJtED+KQH5NMrufT6YVJh IRxt2SkZwa5+5NWKKLiC0JzZUQ/KXN8Lg/FGpEI/3P588U+xQI2DswqJW2dfp9TnXuVOD4NxuyA akWD2IRDMTe0iNNztH61YnLpYRktIWHjE7PBx5H+lPS8iqEz8NN7AF1x9kKUeAlTrCf9D/AvRFs 3dqzorrdlA6NvonU5Rpy2UmK7l/j39VOwy4t26r4L/IM259BW8KInOUvtlTVu/WTMjPo7kVW1Jz yMd/FCzkkQvOE6OCngpS/uYGMlICrQD4P+L61XjrVeCle5ymzUMz6p/A= X-Received: by 2002:a05:620a:171f:b0:8c6:ff8f:58af with SMTP id af79cd13be357-8d41e244447mr749101685a.51.1775273037502; Fri, 03 Apr 2026 20:23:57 -0700 (PDT) Received: from c7-smtp-2023.dev.purestorage.com ([208.88.159.129]) by smtp-relay.gmail.com with ESMTPS id af79cd13be357-8d2a6de9b33sm28188485a.5.2026.04.03.20.23.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Apr 2026 20:23:57 -0700 (PDT) X-Relaying-Domain: purestorage.com Received: from dev-ushankar.dev.purestorage.com (dev-ushankar.dev.purestorage.com [10.7.70.36]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id A2F783405FB; Fri, 3 Apr 2026 21:23:55 -0600 (MDT) Received: by dev-ushankar.dev.purestorage.com (Postfix, from userid 1557716368) id BFA81E54F5B; Fri, 3 Apr 2026 21:23:55 -0600 (MDT) From: Uday Shankar Date: Fri, 03 Apr 2026 21:23:56 -0600 Subject: [PATCH 2/2] selftests: ublk: test that teardown after incomplete recovery completes Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260403-cancel-v1-2-86e5a6b3d3af@purestorage.com> References: <20260403-cancel-v1-0-86e5a6b3d3af@purestorage.com> In-Reply-To: <20260403-cancel-v1-0-86e5a6b3d3af@purestorage.com> To: Ming Lei , Caleb Sander Mateos , Jens Axboe , Shuah Khan Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Uday Shankar X-Mailer: b4 0.14.2 Before the fix, teardown of a ublk server that was attempting to recover a device, but died when it had submitted a nonempty proper subset of the fetch commands to any queue would loop forever. Add a test to verify that, after the fix, teardown completes. This is done by: - Adding a new argument to the fault_inject target that causes it die after fetching a nonempty proper subset of the IOs to a queue - Using that argument in a new test while trying to recover an already-created device - Attempting to delete the ublk device at the end of the test; this hangs forever if teardown from the fault-injected ublk server never completed. It was manually verified that the test passes with the fix and hangs without it. Signed-off-by: Uday Shankar --- tools/testing/selftests/ublk/Makefile | 1 + tools/testing/selftests/ublk/fault_inject.c | 51 +++++++++++++++++++++= ++-- tools/testing/selftests/ublk/kublk.c | 4 ++ tools/testing/selftests/ublk/kublk.h | 3 ++ tools/testing/selftests/ublk/test_generic_17.sh | 35 +++++++++++++++++ 5 files changed, 91 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/ublk/Makefile b/tools/testing/selftest= s/ublk/Makefile index 8ac2d4a682a1768fb1eb9d2dd2a5d01294a67a03..d338668c5a5fbd73f6d70165455= a3551ab13e894 100644 --- a/tools/testing/selftests/ublk/Makefile +++ b/tools/testing/selftests/ublk/Makefile @@ -18,6 +18,7 @@ TEST_PROGS +=3D test_generic_10.sh TEST_PROGS +=3D test_generic_12.sh TEST_PROGS +=3D test_generic_13.sh TEST_PROGS +=3D test_generic_16.sh +TEST_PROGS +=3D test_generic_17.sh =20 TEST_PROGS +=3D test_batch_01.sh TEST_PROGS +=3D test_batch_02.sh diff --git a/tools/testing/selftests/ublk/fault_inject.c b/tools/testing/se= lftests/ublk/fault_inject.c index 3b897f69c014cc73b4b469d816e80284dd21b577..228a9605053409c84baaf255f97= c4abc271a8bfd 100644 --- a/tools/testing/selftests/ublk/fault_inject.c +++ b/tools/testing/selftests/ublk/fault_inject.c @@ -10,11 +10,17 @@ =20 #include "kublk.h" =20 +struct fi_opts { + long long delay_ns; + bool die_during_fetch; +}; + static int ublk_fault_inject_tgt_init(const struct dev_ctx *ctx, struct ublk_dev *dev) { const struct ublksrv_ctrl_dev_info *info =3D &dev->dev_info; unsigned long dev_size =3D 250UL << 30; + struct fi_opts *opts =3D NULL; =20 if (ctx->auto_zc_fallback) { ublk_err("%s: not support auto_zc_fallback\n", __func__); @@ -35,17 +41,51 @@ static int ublk_fault_inject_tgt_init(const struct dev_= ctx *ctx, }; ublk_set_integrity_params(ctx, &dev->tgt.params); =20 - dev->private_data =3D (void *)(unsigned long)(ctx->fault_inject.delay_us = * 1000); + opts =3D calloc(1, sizeof(*opts)); + if (!opts) { + ublk_err("%s: couldn't allocate memory for opts\n", __func__); + return -ENOMEM; + } + + opts->delay_ns =3D ctx->fault_inject.delay_us * 1000; + opts->die_during_fetch =3D ctx->fault_inject.die_during_fetch; + dev->private_data =3D opts; + return 0; } =20 +static void ublk_fault_inject_pre_fetch_io(struct ublk_thread *t, + struct ublk_queue *q, int tag) +{ + struct fi_opts *opts =3D q->dev->private_data; + + if (!opts->die_during_fetch) + return; + + /* + * Each queue fetches its IOs in increasing order of tags, so + * dying just before we're about to fetch tag 1 (regardless of + * what queue we're on) guarantees that we've fetched a nonempty + * proper subset of the tags on that queue. + */ + if (tag =3D=3D 1) { + /* + * Ensure our commands are actually live in the kernel + * before we die. + */ + io_uring_submit(&t->ring); + raise(SIGKILL); + } +} + static int ublk_fault_inject_queue_io(struct ublk_thread *t, struct ublk_queue *q, int tag) { const struct ublksrv_io_desc *iod =3D ublk_get_iod(q, tag); struct io_uring_sqe *sqe; + struct fi_opts *opts =3D q->dev->private_data; struct __kernel_timespec ts =3D { - .tv_nsec =3D (long long)q->dev->private_data, + .tv_nsec =3D opts->delay_ns, }; =20 ublk_io_alloc_sqes(t, &sqe, 1); @@ -77,29 +117,34 @@ static void ublk_fault_inject_cmd_line(struct dev_ctx = *ctx, int argc, char *argv { static const struct option longopts[] =3D { { "delay_us", 1, NULL, 0 }, + { "die_during_fetch", 1, NULL, 0 }, { 0, 0, 0, 0 } }; int option_idx, opt; =20 ctx->fault_inject.delay_us =3D 0; + ctx->fault_inject.die_during_fetch =3D false; while ((opt =3D getopt_long(argc, argv, "", longopts, &option_idx)) !=3D -1) { switch (opt) { case 0: if (!strcmp(longopts[option_idx].name, "delay_us")) ctx->fault_inject.delay_us =3D strtoll(optarg, NULL, 10); + if (!strcmp(longopts[option_idx].name, "die_during_fetch")) + ctx->fault_inject.die_during_fetch =3D strtoll(optarg, NULL, 10); } } } =20 static void ublk_fault_inject_usage(const struct ublk_tgt_ops *ops) { - printf("\tfault_inject: [--delay_us us (default 0)]\n"); + printf("\tfault_inject: [--delay_us us (default 0)] [--die_during_fetch 1= ]\n"); } =20 const struct ublk_tgt_ops fault_inject_tgt_ops =3D { .name =3D "fault_inject", .init_tgt =3D ublk_fault_inject_tgt_init, + .pre_fetch_io =3D ublk_fault_inject_pre_fetch_io, .queue_io =3D ublk_fault_inject_queue_io, .tgt_io_done =3D ublk_fault_inject_tgt_io_done, .parse_cmd_line =3D ublk_fault_inject_cmd_line, diff --git a/tools/testing/selftests/ublk/kublk.c b/tools/testing/selftests= /ublk/kublk.c index e1c3b3c55e565c8cad6b6fe9b9b764cd244818c0..8260c96a39c05584065f41a52f3= d9050614454c6 100644 --- a/tools/testing/selftests/ublk/kublk.c +++ b/tools/testing/selftests/ublk/kublk.c @@ -796,6 +796,8 @@ static void ublk_submit_fetch_commands(struct ublk_thre= ad *t) q =3D &t->dev->q[q_id]; io =3D &q->ios[tag]; io->buf_index =3D j++; + if (q->tgt_ops->pre_fetch_io) + q->tgt_ops->pre_fetch_io(t, q, tag); ublk_queue_io_cmd(t, io); } } else { @@ -807,6 +809,8 @@ static void ublk_submit_fetch_commands(struct ublk_thre= ad *t) for (i =3D 0; i < q->q_depth; i++) { io =3D &q->ios[i]; io->buf_index =3D i; + if (q->tgt_ops->pre_fetch_io) + q->tgt_ops->pre_fetch_io(t, q, i); ublk_queue_io_cmd(t, io); } } diff --git a/tools/testing/selftests/ublk/kublk.h b/tools/testing/selftests= /ublk/kublk.h index 02f0c55d006b4c791fea4456687d0d8757be7be2..f784946144ad3d6e347a867fb13= 89a8e057766a0 100644 --- a/tools/testing/selftests/ublk/kublk.h +++ b/tools/testing/selftests/ublk/kublk.h @@ -60,6 +60,7 @@ struct stripe_ctx { struct fault_inject_ctx { /* fault_inject */ unsigned long delay_us; + bool die_during_fetch; }; =20 struct dev_ctx { @@ -138,6 +139,8 @@ struct ublk_tgt_ops { int (*init_tgt)(const struct dev_ctx *ctx, struct ublk_dev *); void (*deinit_tgt)(struct ublk_dev *); =20 + void (*pre_fetch_io)(struct ublk_thread *t, struct ublk_queue *q, + int tag); int (*queue_io)(struct ublk_thread *, struct ublk_queue *, int tag); void (*tgt_io_done)(struct ublk_thread *, struct ublk_queue *, const struct io_uring_cqe *); diff --git a/tools/testing/selftests/ublk/test_generic_17.sh b/tools/testin= g/selftests/ublk/test_generic_17.sh new file mode 100755 index 0000000000000000000000000000000000000000..2278b5fc9dba523bb1bf8411d22= f4f95a28da905 --- /dev/null +++ b/tools/testing/selftests/ublk/test_generic_17.sh @@ -0,0 +1,35 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +. "$(cd "$(dirname "$0")" && pwd)"/test_common.sh + +ERR_CODE=3D0 + +_prep_test "fault_inject" "teardown after incomplete recovery" + +# First start and stop a ublk server with device configured for recovery +dev_id=3D$(_add_ublk_dev -t fault_inject -r 1) +_check_add_dev $TID $? +state=3D$(__ublk_kill_daemon "${dev_id}" "QUIESCED") +if [ "$state" !=3D "QUIESCED" ]; then + echo "device isn't quiesced($state) after $action" + ERR_CODE=3D255 +fi + +# Then recover the device, but use --die_during_fetch to have the ublk +# server die while a queue has some (but not all) I/Os fetched +${UBLK_PROG} recover -n "${dev_id}" --foreground -t fault_inject --die_dur= ing_fetch 1 +RECOVER_RES=3D$? +# 137 is the result when dying of SIGKILL +if (( RECOVER_RES !=3D 137 )); then + echo "recover command exited with unexpected code ${RECOVER_RES}!" + ERR_CODE=3D255 +fi + +# Clean up the device. This can only succeed once teardown of the above +# exited ublk server completes. So if teardown never completes, we will +# time out here +_ublk_del_dev "${dev_id}" + +_cleanup_test "fault_inject" +_show_result $TID $ERR_CODE --=20 2.34.1