From nobody Mon Feb 9 05:58:58 2026 Received: from mail-dl1-f48.google.com (mail-dl1-f48.google.com [74.125.82.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE08538A70F for ; Fri, 30 Jan 2026 22:36:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769812587; cv=none; b=hSAda6CL/9MD9+THaUwVsbd6yrO6bwVkTxZkyWyWadNsxNPnLbIJb8mAjxAWqb6zP2bU9kiRy/NNwl56AWx36WK3daBysm/RmtgwxM3f9xhz8GI6/p6IaFWuReiNDPdKYhn/zkBcauuuEcmEm9+XWFLPw5bqS0D64qJu2BrknYI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769812587; c=relaxed/simple; bh=N5sLm5Nh8pdXJyKaO0nmafIvgpZjyP2ePrjW4o1mWe8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EnCOGo2/VvhDSjc3JUg60kOfGktdjQ4O6RtvaT03HgCvFU/8BGrE3pj81+/YZKUSJTQKFLC95imcm37ngyDvfLXBg8WmNeN4oDlQubsLxUrtFAPjjcFWsSNqWqHA1TWsFOglBOyEgrwumeK11tbZxruk191LYNxmiELzpG4P5ck= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=OILvaSdj; arc=none smtp.client-ip=74.125.82.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="OILvaSdj" Received: by mail-dl1-f48.google.com with SMTP id a92af1059eb24-1233c155a42so4518746c88.1 for ; Fri, 30 Jan 2026 14:36:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1769812585; x=1770417385; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+6koEg3ORYOvIi3e26txsgR/nhwrNI8Ae21Mw02k0to=; b=OILvaSdj3kzI9qXPEH/R7ElmTjMQpOgxtD5zIHbD3jRPvAv4qfzsxz8uON2r9jrTq5 mOT4m7HlhX7l6h4/oKWQobGIXlsKa0UBJooi30sMuS3KbqrbuHSK+Fc4JtyVpFMgF6L4 4FO2+a2H8J0AAsPY6q8ELWuZsJpdGyrE469mL9EGMVZRdyjtJjLbTVI5AOcMJBHMXYA8 F1IcsNPTDZXYRzJvy9uSDekfxJfaYHgxW/Mo8egHsJjg7uOIiSI+5BK64IqivvB6AENJ 8UdJ0FyA7dRDWxRxN0kO3HXSJZ2ue9afwbGpzfnf6yO34wQuYKzBQBN/e2/JKPDK+3YN oobQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769812585; x=1770417385; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=+6koEg3ORYOvIi3e26txsgR/nhwrNI8Ae21Mw02k0to=; b=hCMOwhdE93sWSnUka6j4Q2eSFrIkCbOWrN1vAvcFqG5X+H0//UvAkUR4uq7WMchCp4 s8A9GYuIzM9QzJdPwkAYRqQrBJDMNWwLUc1j536bQMaY4bNh5uHcNMzsAMwyNheUegVR R3GDPVlvbViYQNYxit5GRm5W5prIf1UYzRtgoLBvo7Rf6jSz0Q4pO8wgLw2fbMji0UBy cyHGgdAgmG7sYgJz3xtB6zw77HBWEHAh5efQ9D/yQI9ctdiCfzrW/sqivO7UvdbAh0RG gNU+d7LVmNb8l0h9GAMFgcF3HXlt/I80GAdHW/vSlYETDvRr8joxh1kDVAG7SeviS/Ua RVKQ== X-Forwarded-Encrypted: i=1; AJvYcCUYvOl7L2KLJbIyc5Wo+DekKWl3CKL2vfSwP9L/OVDr9sTAmfTxKuXg5Ks4DgAJItKEha7M6gm7GA4d7YY=@vger.kernel.org X-Gm-Message-State: AOJu0YzCytpbVd/U8AMrMv37sqUdfwyK2yjMttLKT6aLulsVREQer3CK s2p724qR7vuhadBcaTLBEbUawMV1FLrlJQ5C92iZ3pks7e/Af4T18KnS1MQaaIQmLB8= X-Gm-Gg: AZuq6aLAR7CsfVcVNX7dUM8Jlz7mqQFmsKpRFg3dxE5zOUpuTwIwrrI+uLpy2qr6dkG RANmY+RUoI45GNwrjFPKFykTD66qJdyAms/E2a589Gw18HZF4qlJLjk94xCO7rG5iRWKnWUIpWI eEBuZP0EVMF6hIEsHzzFAxhEyJZVuYnxjKLA+wVckWZIGGt1WDAyMQLUv208jOnEybnyDwVWcLd RlOojtzoFHufk1v3449IM8Icnmh1P2uzuNgYPPGpBXfNcEELgWVC1euUtT+eGGWCwtZAzwAlMzQ j6+UmdfiOM/bcaA6RHYXmkxJ1Teb59Ut18tK8Tpmuw8gcgli92fFR8g7ZTNCrALsRfN76cqDnwG 2yzWwKwtWBuvrYbbHqRgQnmwPlWYhQGkU1VM5BXTcqeNYnR62O9lxGYbjd53ChFz4R07vMMtTWE kjuIrccbwXLtYOWSoT1hayzw2kJYBicv+Pnw== X-Received: by 2002:a05:7022:f103:b0:11d:f44d:401d with SMTP id a92af1059eb24-125c0fab3b7mr2177552c88.22.1769812584699; Fri, 30 Jan 2026 14:36:24 -0800 (PST) Received: from apollo.purestorage.com ([208.88.152.253]) by smtp.googlemail.com with ESMTPSA id a92af1059eb24-124a9d6b906sm13161717c88.4.2026.01.30.14.36.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Jan 2026 14:36:24 -0800 (PST) From: Mohamed Khalfella To: Justin Tee , Naresh Gottumukkala , Paul Ely , Chaitanya Kulkarni , Christoph Hellwig , Jens Axboe , Keith Busch , Sagi Grimberg Cc: Aaron Dailey , Randy Jennings , Dhaval Giani , Hannes Reinecke , linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Mohamed Khalfella Subject: [PATCH v2 01/14] nvmet: Rapid Path Failure Recovery set controller identify fields Date: Fri, 30 Jan 2026 14:34:05 -0800 Message-ID: <20260130223531.2478849-2-mkhalfella@purestorage.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260130223531.2478849-1-mkhalfella@purestorage.com> References: <20260130223531.2478849-1-mkhalfella@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" TP8028 Rapid Path Failure Recovery defined new fields in controller identify response. The newly defined fields are: - CIU (Controller Instance UNIQUIFIER): is an 8bit non-zero value that is assigned a random value when controller first created. The value is expected to be incremented when RDY bit in CSTS register is asserted - CIRN (Controller Instance Random Number): is 64bit random value that gets generated when controller is crated. CIRN is regenerated everytime RDY bit is CSTS register is asserted. - CCRL (Cross-Controller Reset Limit) is an 8bit value that defines the maximum number of in-progress controller reset operations. CCRL is hardcoded to 4 as recommended by TP8028. TP4129 KATO Corrections and Clarifications defined CQT (Command Quiesce Time) which is used along with KATO (Keep Alive Timeout) to set an upper time limit for attempting Cross-Controller Recovery. For NVME subsystem CQT is set to 0 by default to keep the current behavior. The value can be set from configfs if needed. Make the new fields available for IO controllers only since TP8028 is not very useful for discovery controllers. Signed-off-by: Mohamed Khalfella --- drivers/nvme/target/admin-cmd.c | 6 ++++++ drivers/nvme/target/configfs.c | 31 +++++++++++++++++++++++++++++++ drivers/nvme/target/core.c | 12 ++++++++++++ drivers/nvme/target/nvmet.h | 4 ++++ include/linux/nvme.h | 15 ++++++++++++--- 5 files changed, 65 insertions(+), 3 deletions(-) diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cm= d.c index 3da31bb1183e..ade1145df72d 100644 --- a/drivers/nvme/target/admin-cmd.c +++ b/drivers/nvme/target/admin-cmd.c @@ -696,6 +696,12 @@ static void nvmet_execute_identify_ctrl(struct nvmet_r= eq *req) =20 id->cntlid =3D cpu_to_le16(ctrl->cntlid); id->ver =3D cpu_to_le32(ctrl->subsys->ver); + if (!nvmet_is_disc_subsys(ctrl->subsys)) { + id->cqt =3D cpu_to_le16(ctrl->cqt); + id->ciu =3D ctrl->ciu; + id->cirn =3D cpu_to_le64(ctrl->cirn); + id->ccrl =3D NVMF_CCR_LIMIT; + } =20 /* XXX: figure out what to do about RTD3R/RTD3 */ id->oaes =3D cpu_to_le32(NVMET_AEN_CFG_OPTIONAL); diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c index e44ef69dffc2..035f6e75a818 100644 --- a/drivers/nvme/target/configfs.c +++ b/drivers/nvme/target/configfs.c @@ -1636,6 +1636,36 @@ static ssize_t nvmet_subsys_attr_pi_enable_store(str= uct config_item *item, CONFIGFS_ATTR(nvmet_subsys_, attr_pi_enable); #endif =20 +static ssize_t nvmet_subsys_attr_cqt_show(struct config_item *item, + char *page) +{ + return snprintf(page, PAGE_SIZE, "%u\n", to_subsys(item)->cqt); +} + +static ssize_t nvmet_subsys_attr_cqt_store(struct config_item *item, + const char *page, size_t cnt) +{ + struct nvmet_subsys *subsys =3D to_subsys(item); + struct nvmet_ctrl *ctrl; + u16 cqt; + + if (sscanf(page, "%hu\n", &cqt) !=3D 1) + return -EINVAL; + + down_write(&nvmet_config_sem); + if (subsys->cqt =3D=3D cqt) + goto out; + + subsys->cqt =3D cqt; + /* Force reconnect */ + list_for_each_entry(ctrl, &subsys->ctrls, subsys_entry) + ctrl->ops->delete_ctrl(ctrl); +out: + up_write(&nvmet_config_sem); + return cnt; +} +CONFIGFS_ATTR(nvmet_subsys_, attr_cqt); + static ssize_t nvmet_subsys_attr_qid_max_show(struct config_item *item, char *page) { @@ -1676,6 +1706,7 @@ static struct configfs_attribute *nvmet_subsys_attrs[= ] =3D { &nvmet_subsys_attr_attr_vendor_id, &nvmet_subsys_attr_attr_subsys_vendor_id, &nvmet_subsys_attr_attr_model, + &nvmet_subsys_attr_attr_cqt, &nvmet_subsys_attr_attr_qid_max, &nvmet_subsys_attr_attr_ieee_oui, &nvmet_subsys_attr_attr_firmware, diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c index cc88e5a28c8a..0d2a1206e08f 100644 --- a/drivers/nvme/target/core.c +++ b/drivers/nvme/target/core.c @@ -1393,6 +1393,10 @@ static void nvmet_start_ctrl(struct nvmet_ctrl *ctrl) return; } =20 + if (!nvmet_is_disc_subsys(ctrl->subsys)) { + ctrl->ciu =3D ((u8)(ctrl->ciu + 1)) ? : 1; + ctrl->cirn =3D get_random_u64(); + } ctrl->csts =3D NVME_CSTS_RDY; =20 /* @@ -1661,6 +1665,12 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_all= oc_ctrl_args *args) } ctrl->cntlid =3D ret; =20 + if (!nvmet_is_disc_subsys(ctrl->subsys)) { + ctrl->cqt =3D subsys->cqt; + ctrl->ciu =3D get_random_u8() ? : 1; + ctrl->cirn =3D get_random_u64(); + } + /* * Discovery controllers may use some arbitrary high value * in order to cleanup stale discovery sessions @@ -1853,10 +1863,12 @@ struct nvmet_subsys *nvmet_subsys_alloc(const char = *subsysnqn, =20 switch (type) { case NVME_NQN_NVME: + subsys->cqt =3D NVMF_CQT_MS; subsys->max_qid =3D NVMET_NR_QUEUES; break; case NVME_NQN_DISC: case NVME_NQN_CURR: + subsys->cqt =3D 0; subsys->max_qid =3D 0; break; default: diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h index b664b584fdc8..f5d9a01ec60c 100644 --- a/drivers/nvme/target/nvmet.h +++ b/drivers/nvme/target/nvmet.h @@ -264,7 +264,10 @@ struct nvmet_ctrl { =20 uuid_t hostid; u16 cntlid; + u16 cqt; + u8 ciu; u32 kato; + u64 cirn; =20 struct nvmet_port *port; =20 @@ -331,6 +334,7 @@ struct nvmet_subsys { #ifdef CONFIG_NVME_TARGET_DEBUGFS struct dentry *debugfs_dir; #endif + u16 cqt; u16 max_qid; =20 u64 ver; diff --git a/include/linux/nvme.h b/include/linux/nvme.h index 655d194f8e72..5135cdc3c120 100644 --- a/include/linux/nvme.h +++ b/include/linux/nvme.h @@ -21,6 +21,9 @@ #define NVMF_TRADDR_SIZE 256 #define NVMF_TSAS_SIZE 256 =20 +#define NVMF_CQT_MS 0 +#define NVMF_CCR_LIMIT 4 + #define NVME_DISC_SUBSYS_NAME "nqn.2014-08.org.nvmexpress.discovery" =20 #define NVME_NSID_ALL 0xffffffff @@ -328,7 +331,10 @@ struct nvme_id_ctrl { __le16 crdt1; __le16 crdt2; __le16 crdt3; - __u8 rsvd134[122]; + __u8 rsvd134[1]; + __u8 ciu; + __le64 cirn; + __u8 rsvd144[112]; __le16 oacs; __u8 acl; __u8 aerl; @@ -362,7 +368,9 @@ struct nvme_id_ctrl { __u8 anacap; __le32 anagrpmax; __le32 nanagrpid; - __u8 rsvd352[160]; + __u8 rsvd352[34]; + __le16 cqt; + __u8 rsvd388[124]; __u8 sqes; __u8 cqes; __le16 maxcmd; @@ -389,7 +397,8 @@ struct nvme_id_ctrl { __u8 msdbd; __u8 rsvd1804[2]; __u8 dctype; - __u8 rsvd1807[241]; + __u8 ccrl; + __u8 rsvd1808[240]; struct nvme_id_power_state psd[32]; __u8 vs[1024]; }; --=20 2.52.0