From nobody Sat Sep 26 09:19:23 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 879DE3D1CAF for ; Wed, 2 Sep 2026 20:06:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788379567; cv=none; b=GYbcw80oNULBmE/dmo+iEb16gCozKDZFLMRCYMVVBhI59bmXyYGpkzfW1Co5kphFyiaqya5Mu2ryp4SWRp6HvtsGlpDFhAffs3CMj6bc+8AHL3vIAm2a6Bg/Mtx1LGPKDE6bhLwsILYvWBxDFDxcXtmjZpN4CoiYIfD5QzR0buc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788379567; c=relaxed/simple; bh=EG7jMXrQq32CibfG9Ke7DQ8xfXT/tW3mpnjzild10qU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=A69QwBx3aBwciC7NK0+5kGWMMmjU8pxwb9HhC59czwkd/A2RQm4ttzXwjVjrxuROZqRIwLZQxKmncVPgkfcaCzaid7PPffkkk3wRGxwuiOR+1uBM0JBTuAb8WhqI8BIcb24KrlaqnFNFDUp+XDzfynq6g9lPZkuG4VVvEHN//1M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=FYjDbGOD; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="FYjDbGOD" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1788379564; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fS8rV8Ai6nPQGvPzbdvPVCf76DBf0vaQn7m2V2kfIXg=; b=FYjDbGODX10Lxm0ph/uYTHcny4J1fdQiqURHZbU/TWHj44bqLik3OlwMI/Jd2aJkR6ro1p fQeWOnBy0klrcM3bV/oUhyrOE8aGCQzBNfC/xL/0XS9EdZUXLBa/Tm89j6ObDAMfXXunKj p06IVG5pxZvyxljWnVk6DzXvkZ4iLdg= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-397-2M5-PBdvMNqYV8QDxEtbfQ-1; Wed, 02 Sep 2026 16:05:59 -0400 X-MC-Unique: 2M5-PBdvMNqYV8QDxEtbfQ-1 X-Mimecast-MFC-AGG-ID: 2M5-PBdvMNqYV8QDxEtbfQ_1788379557 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E77F81801BF4; Wed, 2 Sep 2026 20:05:56 +0000 (UTC) Received: from jtaubepe-thinkpadx1carbongen12.ibmlowe.csb (unknown [10.17.16.153]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2A3F21955F03; Wed, 2 Sep 2026 20:05:52 +0000 (UTC) From: Jesse Taube To: linux-nvme@lists.infradead.org Cc: linux-scsi@vger.kernel.org, Jonathan Corbet , Shuah Khan , Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Justin Tee , Naresh Gottumukkala , Paul Ely , Chaitanya Kulkarni , "James E.J. Bottomley" , "Martin K. Petersen" , Nilesh Javali , GR-QLogic-Storage-Upstream@marvell.com, Hannes Reinecke , Jesse Taube , "Gustavo A. R. Silva" , John Meneghini , Bryan Gurney , Chris Leech , "Ewan D . Milne" , shinichiro.kawasaki@wdc.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org Subject: [PATCH v2 1/7] nvme: add NVME_CTRL_MARGINAL flag Date: Wed, 2 Sep 2026 16:05:41 -0400 Message-ID: <20260902200547.184734-2-jtaubepe@redhat.com> In-Reply-To: <20260902200547.184734-1-jtaubepe@redhat.com> References: <20260902200547.184734-1-jtaubepe@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Type: text/plain; charset="utf-8" From: Bryan Gurney Add a new controller flag, NVME_CTRL_MARGINAL, to help multipath I/O policies to react to a path that is set to a "marginal" state. The "marginal" flag is initialized to false in `nvme_init_ctrl` and cleared in `nvme_fc_ctrl_connectivity_loss` before re-association. It is only cleared there, not in the generic `nvme_reset_ctrl` path, because the marginal condition reflects a physical link problem. An arbitrary controller reset does not signal that a hardware fault has been resolved. Calling `nvme_fc_ctrl_connectivity_loss` means the FC association has failed and will be re-established, so the prior link quality state is no longer meaningful. Signed-off-by: Bryan Gurney Reviewed-by: Hannes Reinecke --- V10 -> V1: - No change V1 -> V2: - Add nvme_ctrl_assign_marginal - Fix nvme_fc_ctrl_connectivity_loss clearing wrong bit. - Update commit message --- drivers/nvme/host/core.c | 1 + drivers/nvme/host/fc.c | 4 ++++ drivers/nvme/host/nvme.h | 11 +++++++++++ 3 files changed, 16 insertions(+) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 453c1f0b2dd0..4ce3c0573707 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -5165,6 +5165,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct dev= ice *dev, WRITE_ONCE(ctrl->state, NVME_CTRL_NEW); ctrl->passthru_err_log_enabled =3D false; clear_bit(NVME_CTRL_FAILFAST_EXPIRED, &ctrl->flags); + nvme_ctrl_assign_marginal(ctrl, false); spin_lock_init(&ctrl->lock); mutex_init(&ctrl->namespaces_lock); =20 diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c index 04363b9c4489..7886c0dcc626 100644 --- a/drivers/nvme/host/fc.c +++ b/drivers/nvme/host/fc.c @@ -788,6 +788,10 @@ nvme_fc_ctrl_connectivity_loss(struct nvme_fc_ctrl *ct= rl) "Reconnect", ctrl->cnum); =20 set_bit(ASSOC_FAILED, &ctrl->flags); + + /* clear 'marginal' flag as controller will be reset */ + nvme_ctrl_assign_marginal(&ctrl->ctrl, false); + nvme_reset_ctrl(&ctrl->ctrl); } =20 diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 824651cc898d..640cb4747ba2 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -329,6 +329,7 @@ enum nvme_ctrl_flags { NVME_CTRL_SKIP_ID_CNS_CS =3D 4, NVME_CTRL_DIRTY_CAPABILITY =3D 5, NVME_CTRL_FROZEN =3D 6, + NVME_CTRL_MARGINAL =3D 7, }; =20 struct nvme_ctrl { @@ -479,6 +480,16 @@ static inline enum nvme_ctrl_state nvme_ctrl_state(str= uct nvme_ctrl *ctrl) return READ_ONCE(ctrl->state); } =20 +static inline bool nvme_ctrl_is_marginal(struct nvme_ctrl *ctrl) +{ + return test_bit(NVME_CTRL_MARGINAL, &ctrl->flags); +} + +static inline void nvme_ctrl_assign_marginal(struct nvme_ctrl *ctrl, bool = marginal) +{ + assign_bit(NVME_CTRL_MARGINAL, &ctrl->flags, marginal); +} + enum nvme_iopolicy { NVME_IOPOLICY_NUMA, NVME_IOPOLICY_RR, --=20 2.55.0 From nobody Sat Sep 26 09:19:23 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 87CF4330D4C for ; Wed, 2 Sep 2026 20:06:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788379573; cv=none; b=HjzR0gPR84spZoLoizY+jf2TgGOK7pGayyJy6kk15FbqrbexI+YKDH9li/zJ/sMcY/RD7037vFT47Qo8sc/SJTo6njJDeAWrTrm8sJr74lyUOR1NiTXUOoPZ4OCyn6Q1Iu1S96nMeJc6XmnadcwWgjbbIczF7TvR5deX5g0y8EA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788379573; c=relaxed/simple; bh=00jeLAyGx8pUqOu+C12Ta11isenUJ5gOIoXRiGQ/vOg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ShuYQhF7PpSfF43P9WtLulxI7+NHnipZgAn+Jq1Oep/7mjGdyLvK1V2lYcAeqHUqFSdq+HUY1nckr4iFArn+Qlk8/Awa7DfZV/XSH8EwLP8KVUTyYZU8PmV7I84HJO0MT2IumL214fC+SkyZn2P5h/0NyGf8jxx4HncPX0mduKE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=WqjrbuPN; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="WqjrbuPN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1788379571; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7S+46ev46/4yG1E8XUsVniDJoHCi+hDFBp54jOlRy1g=; b=WqjrbuPNcdA8Jas54eszpQcEmRftxOSIPJk6sKcCXbxQzAzTIXYrHtGZIp/ZHnayqOOcy4 ngkixpxniNEjJvtnWs2U2857oSS1GvHQyAsAiaJddGBhcNEtlHRouRii+ajZnFR75lscw/ Vx2cC+atThjHYFC+DvTMGHpiKts8dF0= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-687-206uQ71GPvKtm7JkxKrxQQ-1; Wed, 02 Sep 2026 16:06:05 -0400 X-MC-Unique: 206uQ71GPvKtm7JkxKrxQQ-1 X-Mimecast-MFC-AGG-ID: 206uQ71GPvKtm7JkxKrxQQ_1788379561 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id ECF571955D99; Wed, 2 Sep 2026 20:06:00 +0000 (UTC) Received: from jtaubepe-thinkpadx1carbongen12.ibmlowe.csb (unknown [10.17.16.153]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2590A1955F02; Wed, 2 Sep 2026 20:05:57 +0000 (UTC) From: Jesse Taube To: linux-nvme@lists.infradead.org Cc: linux-scsi@vger.kernel.org, Jonathan Corbet , Shuah Khan , Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Justin Tee , Naresh Gottumukkala , Paul Ely , Chaitanya Kulkarni , "James E.J. Bottomley" , "Martin K. Petersen" , Nilesh Javali , GR-QLogic-Storage-Upstream@marvell.com, Hannes Reinecke , Jesse Taube , "Gustavo A. R. Silva" , John Meneghini , Bryan Gurney , Chris Leech , "Ewan D . Milne" , shinichiro.kawasaki@wdc.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org Subject: [PATCH v2 2/7] nvme-multipath: numa support for marginal paths Date: Wed, 2 Sep 2026 16:05:42 -0400 Message-ID: <20260902200547.184734-3-jtaubepe@redhat.com> In-Reply-To: <20260902200547.184734-1-jtaubepe@redhat.com> References: <20260902200547.184734-1-jtaubepe@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Type: text/plain; charset="utf-8" FPIN LI (link integrity) messages are received when the attached fabric detects hardware errors. In response to these messages I/O should be directed away from the affected ports, and only used if no other non-marginal paths are available. To handle this a new controller flag 'NVME_CTRL_MARGINAL' is added which will cause the multipath scheduler to skip these paths when checking for 'optimized' paths. Signed-off-by: Jesse Taube --- This is a distinct change from the previous commit which treated marginal paths as non-optimized but still usable. This changes the priority of marginal paths to be lower than non-optimized paths. V10 -> V1: - New commit V1 -> V2: - Rewrite --- drivers/nvme/host/multipath.c | 33 ++++++++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-) diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index 9b9a657fa330..dbf09cdda815 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -305,16 +305,45 @@ static bool nvme_path_is_disabled(struct nvme_ns *ns) return false; } =20 +static bool nvme_path_is_usable(struct nvme_ns *ns) +{ + /* Only NVME_ANA_OPTIMIZED and NVME_ANA_NONOPTIMIZED are usable */ + return !nvme_path_is_disabled(ns) && + (ns->ana_state =3D=3D NVME_ANA_OPTIMIZED || + ns->ana_state =3D=3D NVME_ANA_NONOPTIMIZED); +} + +static bool nvme_all_paths_marginal(struct nvme_ns_head *head) +{ + struct nvme_ns *ns; + + list_for_each_entry_srcu(ns, &head->list, siblings, + srcu_read_lock_held(&head->srcu)) { + /* skip paths which can not be used */ + if (!nvme_path_is_usable(ns)) + continue; + if (!nvme_ctrl_is_marginal(ns->ctrl)) + return false; + } + + return true; +} + static struct nvme_ns *__nvme_find_path(struct nvme_ns_head *head, int nod= e) { int found_distance =3D INT_MAX, fallback_distance =3D INT_MAX, distance; struct nvme_ns *found =3D NULL, *fallback =3D NULL, *ns; + bool need_marginal =3D nvme_all_paths_marginal(head); =20 list_for_each_entry_srcu(ns, &head->list, siblings, srcu_read_lock_held(&head->srcu)) { if (nvme_path_is_disabled(ns)) continue; =20 + /* Skip marginal paths unless we need to use them */ + if (!need_marginal && nvme_ctrl_is_marginal(ns->ctrl)) + continue; + if (ns->ctrl->numa_node !=3D NUMA_NO_NODE && READ_ONCE(head->subsys->iopolicy) =3D=3D NVME_IOPOLICY_NUMA) distance =3D node_distance(node, ns->ctrl->numa_node); @@ -339,6 +368,7 @@ static struct nvme_ns *__nvme_find_path(struct nvme_ns_= head *head, int node) } } =20 + /* No optimized path found, use the fallback */ if (!found) found =3D fallback; if (found) @@ -444,7 +474,8 @@ static struct nvme_ns *nvme_queue_depth_path(struct nvm= e_ns_head *head) static inline bool nvme_path_is_optimized(struct nvme_ns *ns) { return nvme_ctrl_state(ns->ctrl) =3D=3D NVME_CTRL_LIVE && - ns->ana_state =3D=3D NVME_ANA_OPTIMIZED; + ns->ana_state =3D=3D NVME_ANA_OPTIMIZED && + !nvme_ctrl_is_marginal(ns->ctrl); } =20 static struct nvme_ns *nvme_numa_path(struct nvme_ns_head *head) --=20 2.55.0 From nobody Sat Sep 26 09:19:23 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 578EB42123D for ; Wed, 2 Sep 2026 20:06:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788379579; cv=none; b=rXFDowDKIOrSocgrzBZc+IljgdQ4EFhKwl5lXo6e/pv2RPeCdE/lTgERDFoWoem1yEmU6hk+BxWN3MXMUdr6S4L86RDyyCs7R9i2KCwCICuqMaQJbFktB+6SLwPy/wGRvIWk2Z7S8UfZC2s1Grz8iFS7HVy/7rvU14P4RWXye6o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788379579; c=relaxed/simple; bh=8surKw7udmsQwFPiPd6xTmHp71njgqhrlZSvjoy14eQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=F1P0nSFVBW80L7qIxJT0Bjl3yNpIkaIpIcBmTVWEfrgG9gA+zIh1NFGEbp2HUay9K+SjeNKpyNJjZlE3Cnv4dXJWDbj+YLBBnfM31fXBdLbwPjhjDh65vns2MXmsUlIWpl7co0wZvjLRHHhDlq2AdPrW1RvSLt3PWjFek127WV8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=bbDRczcl; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bbDRczcl" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1788379577; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B4SOQVx6UDmhZ26lSu1cCcYqEQXYt2vou3jNd344Y+M=; b=bbDRczclhhrDZPu89iXliJd7QFMc1XkR6Y6PahlUgfACt5AN4nUfLrxOtAhd0rQFJoppCL v082ICeWPQOAas9HVq2G85K3rQQpx2hOqU5ORk5uqdt4ybtOrJFWBomp2WVGgdKuCapcNF IRIKU3hpI1jbKUIQdL9egW9xKQJZOg8= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-296-wFa_C8tYM3y7FXo3Jv1X2Q-1; Wed, 02 Sep 2026 16:06:11 -0400 X-MC-Unique: wFa_C8tYM3y7FXo3Jv1X2Q-1 X-Mimecast-MFC-AGG-ID: wFa_C8tYM3y7FXo3Jv1X2Q_1788379565 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 74E081808898; Wed, 2 Sep 2026 20:06:04 +0000 (UTC) Received: from jtaubepe-thinkpadx1carbongen12.ibmlowe.csb (unknown [10.17.16.153]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2AF5D1955F03; Wed, 2 Sep 2026 20:06:01 +0000 (UTC) From: Jesse Taube To: linux-nvme@lists.infradead.org Cc: linux-scsi@vger.kernel.org, Jonathan Corbet , Shuah Khan , Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Justin Tee , Naresh Gottumukkala , Paul Ely , Chaitanya Kulkarni , "James E.J. Bottomley" , "Martin K. Petersen" , Nilesh Javali , GR-QLogic-Storage-Upstream@marvell.com, Hannes Reinecke , Jesse Taube , "Gustavo A. R. Silva" , John Meneghini , Bryan Gurney , Chris Leech , "Ewan D . Milne" , shinichiro.kawasaki@wdc.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org Subject: [PATCH v2 3/7] nvme-multipath: queue-depth support for marginal paths Date: Wed, 2 Sep 2026 16:05:43 -0400 Message-ID: <20260902200547.184734-4-jtaubepe@redhat.com> In-Reply-To: <20260902200547.184734-1-jtaubepe@redhat.com> References: <20260902200547.184734-1-jtaubepe@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Type: text/plain; charset="utf-8" From: John Meneghini Exclude marginal paths from queue-depth io policy. In the case where all paths are marginal and no optimized or non-optimized path is found, we fall back and select the best marginal path. Signed-off-by: Jesse Taube Signed-off-by: John Meneghini --- V10 -> V1: - New commit V1 -> V2: - Rewrite --- drivers/nvme/host/multipath.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index dbf09cdda815..f28797deab68 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -439,12 +439,17 @@ static struct nvme_ns *nvme_queue_depth_path(struct n= vme_ns_head *head) struct nvme_ns *best_opt =3D NULL, *best_nonopt =3D NULL, *ns; unsigned int min_depth_opt =3D UINT_MAX, min_depth_nonopt =3D UINT_MAX; unsigned int depth; + bool need_marginal =3D nvme_all_paths_marginal(head); =20 list_for_each_entry_srcu(ns, &head->list, siblings, srcu_read_lock_held(&head->srcu)) { if (nvme_path_is_disabled(ns)) continue; =20 + /* Skip marginal paths unless we need to use them */ + if (!need_marginal && nvme_ctrl_is_marginal(ns->ctrl)) + continue; + depth =3D atomic_read(&ns->ctrl->nr_active); =20 switch (ns->ana_state) { --=20 2.55.0 From nobody Sat Sep 26 09:19:23 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3B8E4A64D1 for ; Wed, 2 Sep 2026 20:06:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788379584; cv=none; b=GiNlX8Doaz8W1LsdwlBCAbSuwQOHvZhtbVhQZdt7Dmy1BP9fs7avCf5pieMXPDy7RH+9gdHN7pHvP2nuxyhmInqDcjzJWLdTzwAYB4Lpwk6gng/Ay8Z+q1nHQ5J9rYIGasM4Qq3aZRlLhX+sx4QBHHSd4i91b6Pm9kaDrV7//Z8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788379584; c=relaxed/simple; bh=KyigujruG9PSQ4X/h7TyBYjnfhYpTml3X/Nidvemv3c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rf0lZ3gIbU9f/XZGl1Fq8o/FiftpvsH1kfD3OP8PCb7x/Dz6H/HwZu2A1Dofl/UDiYhDGLvWpw+cHA5HCythWgYNR/jzjWT/MV0EcPVOFU7h3drxNUxrRzpqV5h8hfSG7+cyrujPdWeEXXnl6/bNEXGgEcUSmIaaJ2TxkxcHjnA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=iAdk/zPm; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="iAdk/zPm" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1788379581; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aOzuicW09ymDk9cexEa+3/Sews5a9NJ263znpzR+848=; b=iAdk/zPmrbBeH4oqp1GvfYp3OK2tO9SV+tWBNsQ4ozSVqvtOxp3Z8P9c3NXyZew7zFN3HY Py1znvpGI1AHKg0CmTz3Pp+SHgmoZYmU0IwzInD6PKwVTYeeNW04wKqVVx/PQH5J79Dl0G g5n6+wgEeN4Cap2mN4NmmTQbkWpnP6M= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-28-72-yttTKMveSGAZMlApg2A-1; Wed, 02 Sep 2026 16:06:17 -0400 X-MC-Unique: 72-yttTKMveSGAZMlApg2A-1 X-Mimecast-MFC-AGG-ID: 72-yttTKMveSGAZMlApg2A_1788379568 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E64BF1955D99; Wed, 2 Sep 2026 20:06:07 +0000 (UTC) Received: from jtaubepe-thinkpadx1carbongen12.ibmlowe.csb (unknown [10.17.16.153]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A8FE51955F02; Wed, 2 Sep 2026 20:06:04 +0000 (UTC) From: Jesse Taube To: linux-nvme@lists.infradead.org Cc: linux-scsi@vger.kernel.org, Jonathan Corbet , Shuah Khan , Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Justin Tee , Naresh Gottumukkala , Paul Ely , Chaitanya Kulkarni , "James E.J. Bottomley" , "Martin K. Petersen" , Nilesh Javali , GR-QLogic-Storage-Upstream@marvell.com, Hannes Reinecke , Jesse Taube , "Gustavo A. R. Silva" , John Meneghini , Bryan Gurney , Chris Leech , "Ewan D . Milne" , shinichiro.kawasaki@wdc.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org Subject: [PATCH v2 4/7] nvme-multipath: round-robin support for marginal paths Date: Wed, 2 Sep 2026 16:05:44 -0400 Message-ID: <20260902200547.184734-5-jtaubepe@redhat.com> In-Reply-To: <20260902200547.184734-1-jtaubepe@redhat.com> References: <20260902200547.184734-1-jtaubepe@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Type: text/plain; charset="utf-8" Exclude marginal paths from round-robin io policy. In the case where all paths are marginal and no optimized or non-optimized path is found, we fall back and perform round-robin on the marginal paths. Signed-off-by: Jesse Taube --- V10 -> V1: - New commit V1 -> V2: - Rewrite --- drivers/nvme/host/multipath.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-) diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index f28797deab68..738e3c742b74 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -392,22 +392,29 @@ static struct nvme_ns *nvme_round_robin_path(struct n= vme_ns_head *head) int node =3D numa_node_id(); struct nvme_ns *old =3D srcu_dereference(head->current_path[node], &head->srcu); + bool need_marginal; =20 if (unlikely(!old)) return __nvme_find_path(head, node); =20 if (list_is_singular(&head->list)) { - if (nvme_path_is_disabled(old)) - return NULL; - return old; + if (nvme_path_is_usable(old)) + return old; + return NULL; } =20 + need_marginal =3D nvme_all_paths_marginal(head); + for (ns =3D nvme_next_ns(head, old); ns && ns !=3D old; ns =3D nvme_next_ns(head, ns)) { if (nvme_path_is_disabled(ns)) continue; =20 + /* Skip marginal paths unless we need to use them */ + if (!need_marginal && nvme_ctrl_is_marginal(ns->ctrl)) + continue; + if (ns->ana_state =3D=3D NVME_ANA_OPTIMIZED) { found =3D ns; goto out; @@ -419,12 +426,23 @@ static struct nvme_ns *nvme_round_robin_path(struct n= vme_ns_head *head) /* * The loop above skips the current path for round-robin semantics. * Fall back to the current path if either: - * - no other optimized path found and current is optimized, + * - no other optimized path found and current is, + * optimized and not marginal. + * - no other non-marginal path found and current is, + * optimized and marginal. * - no other usable path found and current is usable. */ - if (!nvme_path_is_disabled(old) && - (old->ana_state =3D=3D NVME_ANA_OPTIMIZED || - (!found && old->ana_state =3D=3D NVME_ANA_NONOPTIMIZED))) + /* no other usable path found and current is usable. */ + if (nvme_path_is_usable(old) && !found) + return old; + /* + * - no other optimized path found and current is, + * optimized and not marginal. + * - no other non-marginal path found and current is, + * optimized and marginal. + */ + if (nvme_path_is_usable(old) && old->ana_state =3D=3D NVME_ANA_OPTIMIZED = && + (!nvme_ctrl_is_marginal(old->ctrl) || need_marginal)) return old; =20 if (!found) --=20 2.55.0 From nobody Sat Sep 26 09:19:23 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C73CB4A5EBA for ; Wed, 2 Sep 2026 20:06:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788379583; cv=none; b=doj35/PNX1nQh/bz3LftBshJw2Qdm0rRhvGNNs/ZynXwoH+TEbu+/YGnMPzulx5GVb0ha//BT2I48NZtNFr03UKnknzqIGauOsfo214FHbyfV+l1WIBXdhjjoI2TpZbQbAXZ7mXbv4Vu+SKba9H9gtJI2g1VX5hcYP4D4onbCZ4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788379583; c=relaxed/simple; bh=Eech0gy65GAyXynFCo9H4txIPW1uvlMnLzdl3n7agz8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Deidpl/tqcCwtzgAzeo5ZBNtWSBIK947pqmEZHwJBf9T1Z7bCMV/tOvpKZbuycJ0B5w8xOTo4zsHp9lkjXnhens1Gnrj/jcI6dTZYoIJVXW1pzZmTj6iwzLRA+dx4Bi9DRD7S/Qkmw2Y0HBnjozQOhx4e7eUJ9HLl4TYu/V/NqE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=N6Cs2fV1; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="N6Cs2fV1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1788379580; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UMvbLiMb+sF2mtOC2uLwr9SRzXheSA2PXAvP/FINdPY=; b=N6Cs2fV1fm8MwRqUkXPsx1vC9EvZkUA8pTnsCeYPYP80TaNBknMGml+uVr1vN7LmSbW90i VZiN/nPIb+nV7xgz9McBYjltoTK2ef75etkD8nPRiZo4jwsNOqpFI8VfuHjSnNjgnHAkrf /csKut2g6TilKq+qpQWXovvnQarWeoI= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-317-l6utWQtSM8iZ4kPy_-lR3g-1; Wed, 02 Sep 2026 16:06:15 -0400 X-MC-Unique: l6utWQtSM8iZ4kPy_-lR3g-1 X-Mimecast-MFC-AGG-ID: l6utWQtSM8iZ4kPy_-lR3g_1788379571 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7B45F195DE47; Wed, 2 Sep 2026 20:06:11 +0000 (UTC) Received: from jtaubepe-thinkpadx1carbongen12.ibmlowe.csb (unknown [10.17.16.153]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 247C61955F04; Wed, 2 Sep 2026 20:06:08 +0000 (UTC) From: Jesse Taube To: linux-nvme@lists.infradead.org Cc: linux-scsi@vger.kernel.org, Jonathan Corbet , Shuah Khan , Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Justin Tee , Naresh Gottumukkala , Paul Ely , Chaitanya Kulkarni , "James E.J. Bottomley" , "Martin K. Petersen" , Nilesh Javali , GR-QLogic-Storage-Upstream@marvell.com, Hannes Reinecke , Jesse Taube , "Gustavo A. R. Silva" , John Meneghini , Bryan Gurney , Chris Leech , "Ewan D . Milne" , shinichiro.kawasaki@wdc.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, Muneendra Kumar Subject: [PATCH v2 5/7] nvme: sysfs: emit the marginal path state in show_state() Date: Wed, 2 Sep 2026 16:05:45 -0400 Message-ID: <20260902200547.184734-6-jtaubepe@redhat.com> In-Reply-To: <20260902200547.184734-1-jtaubepe@redhat.com> References: <20260902200547.184734-1-jtaubepe@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Type: text/plain; charset="utf-8" From: Bryan Gurney If a controller has received a link integrity or congestion event, and has the NVME_CTRL_MARGINAL flag set, emit "marginal" in the state instead of "live", to identify the marginal paths. Co-developed-by: John Meneghini Signed-off-by: John Meneghini Reviewed-by: Hannes Reinecke Reviewed-by: Chaitanya Kulkarni Tested-by: Muneendra Kumar Signed-off-by: Bryan Gurney --- V10 -> V1: - No change V1 -> V2: - Only emit "marginal" if port is "live" --- drivers/nvme/host/sysfs.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/nvme/host/sysfs.c b/drivers/nvme/host/sysfs.c index 75b2d69b5957..9ede6b37f27a 100644 --- a/drivers/nvme/host/sysfs.c +++ b/drivers/nvme/host/sysfs.c @@ -526,6 +526,9 @@ static ssize_t nvme_sysfs_show_state(struct device *dev, [NVME_CTRL_DEAD] =3D "dead", }; =20 + if (state =3D=3D NVME_CTRL_LIVE && nvme_ctrl_is_marginal(ctrl)) + return sysfs_emit(buf, "%s\n", "marginal"); + if (state < ARRAY_SIZE(state_name) && state_name[state]) return sysfs_emit(buf, "%s\n", state_name[state]); =20 --=20 2.55.0 From nobody Sat Sep 26 09:19:23 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E89A9420469 for ; Wed, 2 Sep 2026 20:06:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788379586; cv=none; b=PvNLRZ+QPM1RyVa8xvGyjejT9G6pHNXsp7r2q08UrfQLU+Z6Ruq5HniD4YbhUmHVG7EoZaT5nBZwtxafnXlqxzJQIcfnAgCnoUzTLinx/I94/c8HHf8SqI6UVKLQrXKQke1E6gkkK4+l1CxekaBPfJh7tcyjrfwWk6feNFCDEp8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788379586; c=relaxed/simple; bh=TX/A9T7k2l7O+z/2UKblsu8QuGJl1aMR7tTvqKEq/WE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=izai+PP0Y7gH3heCp7ahhimZCmy+11PHCr4rpgOc77Ulqm7wQNfqYNoFKzDgRTB0oERjgN+gTRseit5xdl/TOAo5VMGyIfo0eCRstODnN99kQkMfUWM4P0y7+DZSiYhg75D3rheH1JkD+NTKXKKmFeAGz56EUjO1wViTJsEQWi8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Ldn/m6bx; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Ldn/m6bx" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1788379583; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XMYZkkfvLk4oMTmC0tcijBigxpVsXAU2YnaVB3q+VDM=; b=Ldn/m6bxo1R/5lep3Y/uTh+V6UZ5XqLq5Q/WVMNyR5rbvqZcWHtzPD1XzSUvSSlNNCPepc kogYM9tRRnIQlM/aMTgsqVYEqQNXdWEWOO1ciUupPuTZch+oeoMdM7yxD/h1djh47+KhJ6 dvWr1NfBUag+Q3sI7chpTb5T10C2K3U= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-119-Oa_6jVWVNQewd7r_--03mw-1; Wed, 02 Sep 2026 16:06:19 -0400 X-MC-Unique: Oa_6jVWVNQewd7r_--03mw-1 X-Mimecast-MFC-AGG-ID: Oa_6jVWVNQewd7r_--03mw_1788379575 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 691F9192DD55; Wed, 2 Sep 2026 20:06:15 +0000 (UTC) Received: from jtaubepe-thinkpadx1carbongen12.ibmlowe.csb (unknown [10.17.16.153]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B12581955F02; Wed, 2 Sep 2026 20:06:11 +0000 (UTC) From: Jesse Taube To: linux-nvme@lists.infradead.org Cc: linux-scsi@vger.kernel.org, Jonathan Corbet , Shuah Khan , Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Justin Tee , Naresh Gottumukkala , Paul Ely , Chaitanya Kulkarni , "James E.J. Bottomley" , "Martin K. Petersen" , Nilesh Javali , GR-QLogic-Storage-Upstream@marvell.com, Hannes Reinecke , Jesse Taube , "Gustavo A. R. Silva" , John Meneghini , Bryan Gurney , Chris Leech , "Ewan D . Milne" , shinichiro.kawasaki@wdc.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, Hannes Reinecke Subject: [PATCH v2 6/7] nvme-fc: add nvme_fc_set_remoteport_fpin() Date: Wed, 2 Sep 2026 16:05:46 -0400 Message-ID: <20260902200547.184734-7-jtaubepe@redhat.com> In-Reply-To: <20260902200547.184734-1-jtaubepe@redhat.com> References: <20260902200547.184734-1-jtaubepe@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Type: text/plain; charset="utf-8" Add nvme_fc_set_remoteport_fpin() and supporting functions. This function is called by the SCSI FC transport and driver layer to set or clear the 'marginal' path status for a specific rport. Co-developed-by: Hannes Reinecke Signed-off-by: Hannes Reinecke Signed-off-by: John Meneghini Signed-off-by: Jesse Taube --- V10 -> V1: - Remove nvme_fc_modify_rport_fpin_state - Use struct nvme_fc_remote_port instead of wwpn and wwnn V1 -> V2: - Use nvme_ctrl_assign_marginal over set/clear_bit --- drivers/nvme/host/fc.c | 13 +++++++++++++ include/linux/nvme-fc-driver.h | 2 ++ 2 files changed, 15 insertions(+) diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c index 7886c0dcc626..c4810d048b87 100644 --- a/drivers/nvme/host/fc.c +++ b/drivers/nvme/host/fc.c @@ -895,6 +895,19 @@ nvme_fc_set_remoteport_devloss(struct nvme_fc_remote_p= ort *portptr, } EXPORT_SYMBOL_GPL(nvme_fc_set_remoteport_devloss); =20 +void +nvme_fc_set_remoteport_fpin(struct nvme_fc_remote_port *portptr, bool marg= inal) +{ + struct nvme_fc_rport *rport =3D remoteport_to_rport(portptr); + struct nvme_fc_ctrl *ctrl; + unsigned long flags; + + spin_lock_irqsave(&rport->lock, flags); + list_for_each_entry(ctrl, &rport->ctrl_list, ctrl_list) + nvme_ctrl_assign_marginal(&ctrl->ctrl, marginal); + spin_unlock_irqrestore(&rport->lock, flags); +} +EXPORT_SYMBOL_GPL(nvme_fc_set_remoteport_fpin); =20 /* *********************** FC-NVME DMA Handling **************************= ** */ =20 diff --git a/include/linux/nvme-fc-driver.h b/include/linux/nvme-fc-driver.h index 9f6acadfe0c8..95d79386d126 100644 --- a/include/linux/nvme-fc-driver.h +++ b/include/linux/nvme-fc-driver.h @@ -536,6 +536,8 @@ void nvme_fc_rescan_remoteport(struct nvme_fc_remote_po= rt *remoteport); int nvme_fc_set_remoteport_devloss(struct nvme_fc_remote_port *remoteport, u32 dev_loss_tmo); =20 +void nvme_fc_set_remoteport_fpin(struct nvme_fc_remote_port *portptr, bool= marginal); + /* * Routine called to pass a NVME-FC LS request, received by the lldd, * to the nvme-fc transport. --=20 2.55.0 From nobody Sat Sep 26 09:19:23 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 739214AA039 for ; Wed, 2 Sep 2026 20:06:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788379588; cv=none; b=P5ZosB6gEJrYQM0Y7+sdL/bvQFmfGvjT0wRVaLi9TVXhETjVCol+ubnJiLXLnvn0t68lZHOPVkGs27NqM0Wv2sMV22xSpmT+BBa/9Xon6kSoEIsptkzrUb3wtrC0tCZ33TOtuhnetjSk/yF0AnnRk42ckSG+ar2A++tR/t+0T5E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788379588; c=relaxed/simple; bh=PUdwSTyoKchnGNvcKZneP8Bj182M6z3uu7Ne1W7EWu4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=P2AD9AP4iQeFyF9Obt3VsWvB7zuUs+JS/+3ue3U384XZHE5sEr+IpXh8ETjta8bAQQW/4Bb0v1lfOLMtqzNu7flOBqbaf/zhTYql8qY6QTkfD+zQAmjitiflTjSsy89tWZxEPeeGBXXeSPaeTVEE5La/aWF0O9B6/SSvAKPtXhk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=hz9G7uru; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hz9G7uru" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1788379586; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qj6Ej6aw6ESKMozrrmkT6gNikVjoJG57NJAgL2FNY6M=; b=hz9G7uru+XGP6j3G0nzv0M1m734X7td2QJc/TLj4ZTtdpz+3aWpEXJPX3vgAsrOUAEIiHb fe0nJZW/LTRSe6Sv/9o02fWn+/160Ec1bwZJ0MeUUK1q9zBZGZInzVij119DAKo0QpG1z3 4NRUAxBA6s7+Qc7oyoEPrph6V0MUO88= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-66-lfSnGyAPO7iHpTz7KpplyA-1; Wed, 02 Sep 2026 16:06:22 -0400 X-MC-Unique: lfSnGyAPO7iHpTz7KpplyA-1 X-Mimecast-MFC-AGG-ID: lfSnGyAPO7iHpTz7KpplyA_1788379579 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 69AD1195421A; Wed, 2 Sep 2026 20:06:19 +0000 (UTC) Received: from jtaubepe-thinkpadx1carbongen12.ibmlowe.csb (unknown [10.17.16.153]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 08C161955F02; Wed, 2 Sep 2026 20:06:15 +0000 (UTC) From: Jesse Taube To: linux-nvme@lists.infradead.org Cc: linux-scsi@vger.kernel.org, Jonathan Corbet , Shuah Khan , Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Justin Tee , Naresh Gottumukkala , Paul Ely , Chaitanya Kulkarni , "James E.J. Bottomley" , "Martin K. Petersen" , Nilesh Javali , GR-QLogic-Storage-Upstream@marvell.com, Hannes Reinecke , Jesse Taube , "Gustavo A. R. Silva" , John Meneghini , Bryan Gurney , Chris Leech , "Ewan D . Milne" , shinichiro.kawasaki@wdc.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org Subject: [PATCH v2 7/7] nvme: fcloop: Add set_marginal_rport to sysfs Date: Wed, 2 Sep 2026 16:05:47 -0400 Message-ID: <20260902200547.184734-8-jtaubepe@redhat.com> In-Reply-To: <20260902200547.184734-1-jtaubepe@redhat.com> References: <20260902200547.184734-1-jtaubepe@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Type: text/plain; charset="utf-8" To allow testing of multipath failover, add a sysfs attribute to set a remote port as marginal. This will allow the fcloop LLDD to set the marginal flag on a remote port, simulating a marginal link. Example: Turn on marginal for a remote port matching wwnn and wwpn: `echo 'wwnn=3D0x200000109b5f2956,wwpn=3D0x100000109b5f2956,marginal=3D1' > /sys/class/fcloop/ctl/set_marginal_rport` Turn off marginal for a remote port matching wwnn and wwpn: `echo 'wwnn=3D0x200000109b5f2956,wwpn=3D0x100000109b5f2956,marginal=3D0' > /sys/class/fcloop/ctl/set_marginal_rport` Suggested-by: John Meneghini Signed-off-by: Jesse Taube V10 -> V1: - New patch V1 -> V2: - Fix reference count leak in error path --- drivers/nvme/target/fcloop.c | 50 ++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/drivers/nvme/target/fcloop.c b/drivers/nvme/target/fcloop.c index b63af3b643a6..9977c2a71bd5 100644 --- a/drivers/nvme/target/fcloop.c +++ b/drivers/nvme/target/fcloop.c @@ -6,6 +6,7 @@ #include #include #include +#include =20 #include "../host/nvme.h" #include "../target/nvmet.h" @@ -21,6 +22,7 @@ enum { NVMF_OPT_FCADDR =3D 1 << 3, NVMF_OPT_LPWWNN =3D 1 << 4, NVMF_OPT_LPWWPN =3D 1 << 5, + NVMF_OPT_MARGINAL =3D 1 << 6, }; =20 struct fcloop_ctrl_options { @@ -31,6 +33,7 @@ struct fcloop_ctrl_options { u32 fcaddr; u64 lpwwnn; u64 lpwwpn; + u32 marginal; }; =20 static const match_table_t opt_tokens =3D { @@ -40,6 +43,7 @@ static const match_table_t opt_tokens =3D { { NVMF_OPT_FCADDR, "fcaddr=3D%x" }, { NVMF_OPT_LPWWNN, "lpwwnn=3D%s" }, { NVMF_OPT_LPWWPN, "lpwwpn=3D%s" }, + { NVMF_OPT_MARGINAL, "marginal=3D%d" }, { NVMF_OPT_ERR, NULL } }; =20 @@ -120,6 +124,13 @@ fcloop_parse_options(struct fcloop_ctrl_options *opts, } opts->lpwwpn =3D token64; break; + case NVMF_OPT_MARGINAL: + if (match_int(args, &token)) { + ret =3D -EINVAL; + goto out_free_options; + } + opts->marginal =3D token; + break; default: pr_warn("unknown parameter or missing value '%s'\n", p); ret =3D -EINVAL; @@ -199,6 +210,9 @@ fcloop_parse_nm_options(struct device *dev, u64 *nname,= u64 *pname, =20 #define TGTPORT_OPTS (NVMF_OPT_WWNN | NVMF_OPT_WWPN) =20 +#define MARGINAL_OPTS (NVMF_OPT_WWNN | NVMF_OPT_WWPN | \ + NVMF_OPT_MARGINAL) + =20 static DEFINE_SPINLOCK(fcloop_lock); static LIST_HEAD(fcloop_lports); @@ -1663,6 +1677,40 @@ fcloop_set_cmd_drop(struct device *dev, struct devic= e_attribute *attr, return count; } =20 +static ssize_t +fcloop_set_marginal_rport(struct device *dev, struct device_attribute *att= r, + const char *buf, size_t count) +{ + struct fcloop_nport *nport; + struct fcloop_ctrl_options opts =3D {}; + unsigned long flags; + int ret; + + ret =3D fcloop_parse_options(&opts, buf); + if (ret) + return ret; + + /* everything there ? */ + if ((opts.mask & MARGINAL_OPTS) !=3D MARGINAL_OPTS) + return -EINVAL; + + nport =3D fcloop_nport_lookup(opts.wwnn, opts.wwpn); + if (!nport) + return -ENOENT; + + spin_lock_irqsave(&fcloop_lock, flags); + if (!nport->rport || !nport->rport->remoteport) { + spin_unlock_irqrestore(&fcloop_lock, flags); + fcloop_nport_put(nport); + return -ENOENT; + } + + nvme_fc_set_remoteport_fpin(nport->rport->remoteport, opts.marginal); + spin_unlock_irqrestore(&fcloop_lock, flags); + fcloop_nport_put(nport); + + return count; +} =20 static DEVICE_ATTR(add_local_port, 0200, NULL, fcloop_create_local_port); static DEVICE_ATTR(del_local_port, 0200, NULL, fcloop_delete_local_port); @@ -1671,6 +1719,7 @@ static DEVICE_ATTR(del_remote_port, 0200, NULL, fcloo= p_delete_remote_port); static DEVICE_ATTR(add_target_port, 0200, NULL, fcloop_create_target_port); static DEVICE_ATTR(del_target_port, 0200, NULL, fcloop_delete_target_port); static DEVICE_ATTR(set_cmd_drop, 0200, NULL, fcloop_set_cmd_drop); +static DEVICE_ATTR(set_marginal_rport, 0200, NULL, fcloop_set_marginal_rpo= rt); =20 static struct attribute *fcloop_dev_attrs[] =3D { &dev_attr_add_local_port.attr, @@ -1680,6 +1729,7 @@ static struct attribute *fcloop_dev_attrs[] =3D { &dev_attr_add_target_port.attr, &dev_attr_del_target_port.attr, &dev_attr_set_cmd_drop.attr, + &dev_attr_set_marginal_rport.attr, NULL }; =20 --=20 2.55.0