From nobody Fri Jul 24 21:29:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E00037F325 for ; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891492; cv=none; b=TTQgwJHWL3wArmed+fi7ga9SRzUPTZg5NEb5VaRnICl6mKttujzqJZ4gGNH7eTaRwkVQoIbI13AzYxlnrmhdyaZInFHszj2M7WWh08gpUasfcPIvkhWffDf9egci96JEUN3O1nsLgR9/fqWDfULUfw4d04rP2Cp7Ro41WJv9Ly8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891492; c=relaxed/simple; bh=N993Tbovv/OJQ5ehKfsCLbLkOsyM+Z9sDhkUcBQFuG8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dT0O3p9bb/nFH2OKKiWpd3ZJgOyAiHPZ/sGyfB/TZYrcsS+84TN20Tw6i6Ckdt/jD3WV/w34ypAsZoHbH2eKE6KTbquT++d83MLvBBZm5WoaxnbfTnnCVYOW1XwEUA3PndgIoa7l2NQoBVc848TacuAMycUGA0wtk2sBUkm808U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hG6veWoJ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hG6veWoJ" Received: by smtp.kernel.org (Postfix) with ESMTPS id 6A5A4C4FF61; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1784891492; bh=N993Tbovv/OJQ5ehKfsCLbLkOsyM+Z9sDhkUcBQFuG8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=hG6veWoJn+mVs92hN+5uRUO3eDs5wzOhIBNAvEYaaEeOtb64wBgkDAglvGOYZbTrR qpi+pk37uHfm7eLZHiu88eZW5oJZyOFhIGCZbcMRHWRwBHZPICXDw+YmX29e7mlkJg h+81vQf8w4+hf5LAVCmtf98SXJ0ayQuBT9g2gCRtap8+EAkSKBLy4opLvO4ZacDzDL 6Xm90neu8FKD182y01aA3erEEr2ANw03xoE/qgsq8R9zIwVzfoMWKXuaCEF6Q2+Cm8 Wxx6oFmSXuycI9LxSKdkTckJs5lcATLthfZuUjAdYahPBa8+sonb95TQ+VcG67Az3t brxRgo1OJvOag== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44376C531FA; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) From: Joel Granados Date: Fri, 24 Jul 2026 13:10:55 +0200 Subject: [PATCH RFC v2 01/14] nvme: Add NVME_AER_ONE_SHOT callback handler Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-jag-cdq-lkml-v2-1-659a7449e849@kernel.org> References: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> In-Reply-To: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jason Gunthorpe Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Joel Granados X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3112; i=joel.granados@kernel.org; h=from:subject:message-id; bh=N993Tbovv/OJQ5ehKfsCLbLkOsyM+Z9sDhkUcBQFuG8=; b=owJ4nAHtARL+kA0DAAoBupfNUreWQU8ByyZiAGpjSFlHhZ043yqa49QINd5arkgepGt3mncd+ qhtRiKsRCU2ZIkBswQAAQoAHRYhBK5HCVcl5jElzssnkLqXzVK3lkFPBQJqY0hZAAoJELqXzVK3 lkFPicML/iLuIyhE/0iBKriMsaxVZwdhPTF/Z4e1nEVOMBpWzRHnCu81fF8lUnEe3WwK1Xoaikh f5QSSghOdiijY8KpkkGHUFOt9svSWOXAA20TQGcaatvhoBZSlbnjzzktKoyMbLFZ3pW/doLiec6 k+zm5eHolzH9XBEC1rYszYV9f3A+1MsX350Gk1leOqrUH6yOCoRnf2tCG7a98H7fgArpox79j3Y VBJ5tw/eIgWma73KvzQVHxBHLG0cWCOlH+svBiYM2SHCCCexNikfSxx9AK9S32m7dTHBw3m4VzW G0M4iNR9GuSKdYNguZga3QFqIMy3ABKqnY41OwYqYs/IuuKVHXpsEsIgGugnpKoYPuA5GmQ9mmq xn2NC4gb3hqRQgMsPSFOrhY5wU6AQ3hG4YiGELPkZW1cJaY3YsOG30ADj63VOVRHAxPeC3HsCf9 lGvLzUx4Xnx+Ou+roLWg0uTlCKJQcssgT3uRsWtOk0MUTld4nsCq7G4MF/MbwKZWkhs6aAOyWR5 mk= X-Developer-Key: i=joel.granados@kernel.org; a=openpgp; fpr=F1F8E46D30F0F6C4A45FF4465895FAAC338C6E77 X-Endpoint-Received: by B4 Relay for joel.granados@kernel.org/default with auth_id=239 Handle NVME_AER_ONE_SHOT asynchronous event notifications by extracting the event parameter from the 64-bit result and passing it to the handler. Stub nvme_handle_aen_oneshot() with warnings for now; it will be extended by subsequent patches to handle specific one-shot event types like CDQ tail pointer events. Signed-off-by: Joel Granados --- drivers/nvme/host/core.c | 29 +++++++++++++++++++++++++++++ include/linux/nvme.h | 7 +++++++ 2 files changed, 36 insertions(+) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index fb13f85dd7844331566f05915dda21d91abfea30..6e7533ccf24bfe3df7188f44298= 6ec18a3200660 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -4748,6 +4748,30 @@ static u32 nvme_aer_subtype(u32 result) return (result & 0xff00) >> 8; } =20 +static bool nvme_handle_aen_oneshot(struct nvme_ctrl *ctrl, u32 result, u3= 2 event_param) +{ + u32 aer_subtype =3D nvme_aer_subtype(result); + + switch (aer_subtype) { + case NVME_AER_ONE_SHOT_CDQ_TAIL_PTR: + WARN_ONCE(1, "CDQ Tail Pointer one shot event ignored"); + break; + case NVME_AER_ONE_SHOT_CDQ_FULL: + WARN_ONCE(1, "CDQ Full Error one shot event ignored"); + break; + case NVME_AER_ONE_SHOT_PWR_TH: + WARN_ONCE(1, "Power Threshold Exceeded one shot event ignored"); + break; + default: + WARN_ONCE(1, "Unrecognized One Shot Async Event %d (ignored)", + aer_subtype); + break; + } + + /* Return true in all cases to reque after handling */ + return true; +} + static bool nvme_handle_aen_notice(struct nvme_ctrl *ctrl, u32 result) { u32 aer_notice_type =3D nvme_aer_subtype(result); @@ -4796,6 +4820,7 @@ void nvme_complete_async_event(struct nvme_ctrl *ctrl= , __le16 status, volatile union nvme_result *res) { u32 result =3D le32_to_cpu(res->u32); + u32 event_param =3D 0; u32 aer_type =3D nvme_aer_type(result); u32 aer_subtype =3D nvme_aer_subtype(result); bool requeue =3D true; @@ -4808,6 +4833,10 @@ void nvme_complete_async_event(struct nvme_ctrl *ctr= l, __le16 status, case NVME_AER_NOTICE: requeue =3D nvme_handle_aen_notice(ctrl, result); break; + case NVME_AER_ONE_SHOT: + event_param =3D le64_to_cpu(res->u64) >> 32; + requeue =3D nvme_handle_aen_oneshot(ctrl, result, event_param); + break; case NVME_AER_ERROR: /* * For a persistent internal error, don't run async_event_work diff --git a/include/linux/nvme.h b/include/linux/nvme.h index 69a8c48faa6ce19e695286d15be2585a4d4401d4..60fb5b6808615ff7388f30e12ff= e6ed5da2ef4f6 100644 --- a/include/linux/nvme.h +++ b/include/linux/nvme.h @@ -862,6 +862,7 @@ enum { NVME_AER_ERROR =3D 0, NVME_AER_SMART =3D 1, NVME_AER_NOTICE =3D 2, + NVME_AER_ONE_SHOT =3D 4, NVME_AER_CSS =3D 6, NVME_AER_VS =3D 7, }; @@ -877,6 +878,12 @@ enum { NVME_AER_NOTICE_DISC_CHANGED =3D 0xf0, }; =20 +enum { + NVME_AER_ONE_SHOT_CDQ_TAIL_PTR =3D 0x00, + NVME_AER_ONE_SHOT_CDQ_FULL =3D 0x01, + NVME_AER_ONE_SHOT_PWR_TH =3D 0x02, +}; + enum { NVME_AEN_BIT_NS_ATTR =3D 8, NVME_AEN_BIT_FW_ACT =3D 9, --=20 2.50.1 From nobody Fri Jul 24 21:29:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B5476431E74 for ; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891492; cv=none; b=JRMgAWjJavG3aQAPhuxE3LtNgkVnudqTAA02F8huJ5eS4wkzldNbL7HkG1YpcaeKRNFnN5JIHD3XlfOMkViu04C/JFDPohUtHreht2Ioje5di+Ei4fDGc20MrDKGoe2HmpwDiWlhOTySR02+c59njibneZWPiL0TV5033Ec2JjU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891492; c=relaxed/simple; bh=AZy2WTqOKzVRegCf28nvQJjMyVdHQz6BP3DV6TkjK9g=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=pfcNUz6OGsNMEY3LuPyZ/2JOIXx7Mh46SPSFvHwKO9uDE+l+RDdjiBJvfVzi3dGnUIS9vAQfYu6obA2bfcue7WLTFpz3UF4BVfelmLufXczV0icdKKZouPlrrOAKB5TEYeM5pEUEhREQyaAslSaHOzY+LeSDUjttJx5fBHobmbM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QkFWKwNF; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QkFWKwNF" Received: by smtp.kernel.org (Postfix) with ESMTPS id 7000AC4FF63; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1784891492; bh=AZy2WTqOKzVRegCf28nvQJjMyVdHQz6BP3DV6TkjK9g=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=QkFWKwNFesuxZhZZup0rZ4vakdAAyckDsSP9K7RxHvGwL/rYDcJJTsc+eiZA2UZCB oGz4FC1OfEe607Zn3+Q5BPiCu+gY+wdoqOnhK/xddnbL1wMIeBiaEGDcf2gfLvttvG QAV/mVMZI0zFTKcwCQShbEwF3+zCfSaO71e2V92qF/U1aOFV2Bq8uhZ1AcHArwxWH5 HL7IOPmy/N9koctD1v5PYGzi3B5hL83U4o36FigL9XCNY3W3C0OhqSPcBGfUFOELsy p+Pwh2+d1NK3BERhuj3Vz/z3MM/C3G5Ir8v+kSdAfN0aTBt0UMfD78DcAh8RgWSFHS wiS0cqyfE2b6g== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50BC3C531C9; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) From: Joel Granados Date: Fri, 24 Jul 2026 13:10:56 +0200 Subject: [PATCH RFC v2 02/14] nvme: Add NVMe Controller Data Queue command Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-jag-cdq-lkml-v2-2-659a7449e849@kernel.org> References: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> In-Reply-To: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jason Gunthorpe Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Joel Granados X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3542; i=joel.granados@kernel.org; h=from:subject:message-id; bh=AZy2WTqOKzVRegCf28nvQJjMyVdHQz6BP3DV6TkjK9g=; b=owJ4nAHtARL+kA0DAAoBupfNUreWQU8ByyZiAGpjSFrWCyLD22UG7qKq/OMjAgVA1sOtD01sl bmdPnT8hw5KD4kBswQAAQoAHRYhBK5HCVcl5jElzssnkLqXzVK3lkFPBQJqY0haAAoJELqXzVK3 lkFP4pkL/3PeJxuMTzMA9qAWTcWk0ycTHCE6SA2NFl5AYprn9XXu7TyOMAGzTsk6rEm3KWbj9L5 i3ert5Jd+DMdp5DQPKZKTblhyUa7QbYh8RM+Qnej04y7+Ysd+QdVNdV+NXNSV7jNIpd+n+m4iMF 7P62wmJP3qJi7f5+xNG3SS3n+FHiZUXW/jNa8ezZCKd3KW4/OMGxdz1pKlmsLOBoNKbsQU0v2gn H0AEDsUrInE1CNGrEp/LOVg3veYyzp/FhMiVdwR8tF5nVMXSFCEH23IKbJ+NJF8Bza7dzVbQRfs LWpmfNzsDvPTA0kMWVxVgmB9+iCS18c4hrvdR/2DO060kfLY3pDpCelylPlLUa2/GF+SIZmmirJ t2Ol2foyS65NYG/rQ187sm2ZwOBYIiqqkOK024VTgkNmoH2GOHDQqs8MhAZ8OBmn+KAiEo/s5zi 8kfTniQoGcnBeYZygauOoTGZJXP8+AoMParKfKkyZXVD/33lU2trn17wnT2sDsKebGTS91vyX56 zw= X-Developer-Key: i=joel.granados@kernel.org; a=openpgp; fpr=F1F8E46D30F0F6C4A45FF4465895FAAC338C6E77 X-Endpoint-Received: by B4 Relay for joel.granados@kernel.org/default with auth_id=239 Add the struct (nvme_cdq_cmd) and define the relative values to support NVMe Controller Data Queue management. * struct nvme_cdq_cmd: Is the CDQ mgmt command * nvme_cdq_cmd_mgmt_op: Specifies create or delete * NVME_CDQ_CMD_MGMT_CREATE_*: Specifies the create operation. No functional changes are included as this is a preparation commit. Signed-off-by: Joel Granados --- drivers/nvme/host/core.c | 1 + include/linux/nvme.h | 45 ++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 45 insertions(+), 1 deletion(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 6e7533ccf24bfe3df7188f442986ec18a3200660..d11858300ad43f3e229eab7f2dd= 77563075b801e 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -5387,6 +5387,7 @@ static inline void _nvme_check_size(void) BUILD_BUG_ON(sizeof(struct nvme_directive_cmd) !=3D 64); BUILD_BUG_ON(sizeof(struct nvme_lm_send_cmd) !=3D 64); BUILD_BUG_ON(sizeof(struct nvme_lm_recv_cmd) !=3D 64); + BUILD_BUG_ON(sizeof(struct nvme_cdq_command) !=3D 64); BUILD_BUG_ON(sizeof(struct nvme_feat_host_behavior) !=3D 512); } =20 diff --git a/include/linux/nvme.h b/include/linux/nvme.h index 60fb5b6808615ff7388f30e12ffe6ed5da2ef4f6..1c20ddc06a46066b5ab9acf1dc4= 4b03d941592a9 100644 --- a/include/linux/nvme.h +++ b/include/linux/nvme.h @@ -1639,6 +1639,7 @@ enum nvme_admin_opcode { nvme_admin_nvme_mi_recv =3D 0x1e, nvme_admin_lm_send =3D 0x41, nvme_admin_lm_recv =3D 0x42, + nvme_admin_cdq =3D 0x45, nvme_admin_dbbuf =3D 0x7C, nvme_admin_format_nvm =3D 0x80, nvme_admin_security_send =3D 0x81, @@ -1679,7 +1680,8 @@ enum nvme_admin_opcode { nvme_admin_opcode_name(nvme_admin_security_send), \ nvme_admin_opcode_name(nvme_admin_security_recv), \ nvme_admin_opcode_name(nvme_admin_sanitize_nvm), \ - nvme_admin_opcode_name(nvme_admin_get_lba_status)) + nvme_admin_opcode_name(nvme_admin_get_lba_status), \ + nvme_admin_opcode_name(nvme_admin_cdq)) =20 enum { NVME_QUEUE_PHYS_CONTIG =3D (1 << 0), @@ -1710,6 +1712,7 @@ enum { NVME_FEAT_PLM_WINDOW =3D 0x14, NVME_FEAT_HOST_BEHAVIOR =3D 0x16, NVME_FEAT_SANITIZE =3D 0x17, + NVME_FEAT_CDQ =3D 0x21, NVME_FEAT_FDP =3D 0x1d, NVME_FEAT_SW_PROGRESS =3D 0x80, NVME_FEAT_HOST_ID =3D 0x81, @@ -1739,6 +1742,10 @@ enum { NVME_FWACT_ACTV =3D (2 << 3), }; =20 +enum { + NVME_FEAT_CDQ_ID_MASK =3D GENMASK(15, 0), +}; + struct nvme_supported_log { __le32 lids[256]; }; @@ -1917,6 +1924,41 @@ struct nvme_directive_cmd { __u32 rsvd16[3]; }; =20 +//Select values for dword 10 +enum nvme_cdq_cmd_mgmt_op { + NVME_CDQ_CMD_MGMT_CREATE =3D 0x0, + NVME_CDQ_CMD_MGMT_DELETE =3D 0x1 +}; +#define NVME_CDQ_CMD_MGMT_CREATE_MOS_QT_UDMQ 0x0 +#define NVME_CDQ_CMD_MGMT_CREATE_PC_CONT 0x1 +#define NVME_CDQ_CMD_MGMT_CREATE_PC_DISCONT 0x0 + +union nvme_cdq_cmd_dw11 { + struct { + __le16 flags; + __le16 cqs; + }; + struct { + __le16 cdqid; + __le16 rsvd; + }; +}; + +struct nvme_cdq_command { + __u8 opcode; + __u8 flags; + __u16 command_id; + __u32 rsvd1[5]; + __le64 prp1; + __le32 rsvd8[2]; + __u8 sel; + __u8 rsvd10; + __le16 mos; + union nvme_cdq_cmd_dw11 dw11; + __le32 cdqsize; + __u32 rsvd13[3]; +}; + /* * Fabrics subcommands. */ @@ -2335,6 +2377,7 @@ struct nvme_command { struct nvme_directive_cmd directive; struct nvme_lm_command lm; struct nvme_io_mgmt_recv_cmd imr; + struct nvme_cdq_command cdq; }; }; =20 --=20 2.50.1 From nobody Fri Jul 24 21:29:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B557E4322E5 for ; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891492; cv=none; b=u4iw5OxH1zXsBVJBPebpqsL9DgoBW4oYmUXCI1jcvzWQsrLWrchBvF7Jizu0E6/R3IjmlgA3ept7+G8tzp0mzIF1p96yV4brixsC+Kd6SaTEi/zYeqAW1q4YlSRBhV4fSlXgumyiRI/w7OVkqo0QFj0t6FVtW8TU2Uznyz2cbJI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891492; c=relaxed/simple; bh=ymNcVYBztZcQ5ofo+4XnaaZ/MHgRlLgANYtiOPR0lYs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=jZ2PHAbT11cFq7t/accRp9dSoZyF7nEJ5CR1lyaAzmcOjONZrzWIgaKcAf3+0bJwyl2dyoLboJUGtl9u1sinfvPj8TYXVI1+csiopYDYK3GIIAlbiER6yWMxGffND6cmLnmbKuHY7EvFBZCNR7h9zbfC+8+gzuE3Tzf6pOYHHYo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Q6IG2Nxv; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Q6IG2Nxv" Received: by smtp.kernel.org (Postfix) with ESMTPS id 7CEC2C4FF6B; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1784891492; bh=ymNcVYBztZcQ5ofo+4XnaaZ/MHgRlLgANYtiOPR0lYs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Q6IG2NxvmBZnMi221AB3qgpzV4tRh6ec8IR07cFp5rOiesrj6RM+w7yowmDdmSZp2 JUxeFvQ8YGpbqmijOrkpCqrjNyGhmN1A/spFuEm60QJcljlMHxDBB0HpKcyI29x/gr /sJqJmrzA2YCfszFtIe80cGgUYDNIrU7dM7G4LpD5nWmYzUK2pftWMjZin03FuDCsS ExwpqMC++BGn1gX07OYop8/i9SH4d4yvFoShe7LvZ/Rvit6WPELkTDvEuoOBDyleXg 5FOHSjNsIuAWXya4sEFv6qQms/XaOWF1YhDw7nun66I+MPM1M50fNN6pUD2VOT1twu L6P/3s44/a9CA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DC8DC53209; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) From: Joel Granados Date: Fri, 24 Jul 2026 13:10:57 +0200 Subject: [PATCH RFC v2 03/14] nvme: Add track send command Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-jag-cdq-lkml-v2-3-659a7449e849@kernel.org> References: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> In-Reply-To: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jason Gunthorpe Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Joel Granados X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2010; i=joel.granados@kernel.org; h=from:subject:message-id; bh=ymNcVYBztZcQ5ofo+4XnaaZ/MHgRlLgANYtiOPR0lYs=; b=owJ4nAHtARL+kA0DAAoBupfNUreWQU8ByyZiAGpjSFqbSzvQLIQ7SXsSX8M7WzphY1l3jaRBH WKkKTMDEu/ScokBswQAAQoAHRYhBK5HCVcl5jElzssnkLqXzVK3lkFPBQJqY0haAAoJELqXzVK3 lkFPBTkL/1fqXUb73qFKKCECzj/jAZ93Jza+YYmzoEVKXp14byhQqTdjpro5nVBLcTkEOscaua9 7I8WHTBo8rCQDSD7UdmKH04Ydxm+CsHuF9MRLaPaDvOrYo5FRocW7tAbqaK/fDcpN3nuB0ceIR/ uavYPeo3VNBPqb0VnHM7jkgsP0r0e2ILS21J2ruWRumNDwFM5CKwdBjzEK+KF5JevOM3twPfioj WYY6UUiOrwwOe1OEUmrB4rmIDYw2PtdewGl9bmzy8Kro5ialWBWHy9azy79a1UwT5Fd+AXLMOyU TjgRX2u8vuJkwhhBhO9E9h+ZBGTgpnoRO6djoa9UCwBE1a+AsSAMF+8nUDERDIlfZ8RSGWL5Ve/ W7Qh/ePHJKYzD7jG+ko9VhcBHfvEhxPBls3T232KkOhaLxdzkKfGzGGvlf0wd+GAQgfFeQ2nVXK CfH+DrCDBv+YhhPWfGWe4OwXsUGFeg9irZjVhbNZpsJrQN9hOIbJwOStkOFqgl2E6/XndxRCGo4 8E= X-Developer-Key: i=joel.granados@kernel.org; a=openpgp; fpr=F1F8E46D30F0F6C4A45FF4465895FAAC338C6E77 X-Endpoint-Received: by B4 Relay for joel.granados@kernel.org/default with auth_id=239 The Controller Data Queue is started/stopped using this NVMe command. Signed-off-by: Joel Granados --- include/linux/nvme.h | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/include/linux/nvme.h b/include/linux/nvme.h index 1c20ddc06a46066b5ab9acf1dc44b03d941592a9..7cabedce022eda98fff78959668= 2f257203e3b45 100644 --- a/include/linux/nvme.h +++ b/include/linux/nvme.h @@ -1637,6 +1637,7 @@ enum nvme_admin_opcode { nvme_admin_virtual_mgmt =3D 0x1c, nvme_admin_nvme_mi_send =3D 0x1d, nvme_admin_nvme_mi_recv =3D 0x1e, + nvme_admin_track_send =3D 0x3d, nvme_admin_lm_send =3D 0x41, nvme_admin_lm_recv =3D 0x42, nvme_admin_cdq =3D 0x45, @@ -1681,7 +1682,8 @@ enum nvme_admin_opcode { nvme_admin_opcode_name(nvme_admin_security_recv), \ nvme_admin_opcode_name(nvme_admin_sanitize_nvm), \ nvme_admin_opcode_name(nvme_admin_get_lba_status), \ - nvme_admin_opcode_name(nvme_admin_cdq)) + nvme_admin_opcode_name(nvme_admin_cdq), \ + nvme_admin_opcode_name(nvme_admin_track_send)) =20 enum { NVME_QUEUE_PHYS_CONTIG =3D (1 << 0), @@ -1959,6 +1961,28 @@ struct nvme_cdq_command { __u32 rsvd13[3]; }; =20 +// These are the values for select in dword10 +enum nvme_track_send_cmd_mgmt_op { + NVME_TRSND_CMD_MGMT_LOG_USR_DATA =3D 0x0, + NVME_TRSND_CMD_MGMT_TRACK_MEM =3D 0x1 +}; + +// These are the values for Logging action LACT +#define NVME_TRSND_CMD_MGMT_LOG_USR_DATA_START 0x1 +#define NVME_TRSND_CMD_MGMT_LOG_USR_DATA_STOP 0x0 + +struct track_send_cmd { + __u8 opcode; + __u8 flags; + __u16 command_id; + __u32 rsvd1[9]; + __u8 sel; + __u8 rsvd10; + __le16 mos; + __le32 dw11; + __u32 rsvd12[4]; +}; + /* * Fabrics subcommands. */ @@ -2378,6 +2402,7 @@ struct nvme_command { struct nvme_lm_command lm; struct nvme_io_mgmt_recv_cmd imr; struct nvme_cdq_command cdq; + struct track_send_cmd track_send; }; }; =20 --=20 2.50.1 From nobody Fri Jul 24 21:29:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C65714322F5 for ; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891492; cv=none; b=PLu+6wALFAatL+SPutQnDQnnl7wu++6z/SQLGBgUhAW2Fj2LIT+cnKFy4JGV47tHmdDP6OXSnra886NL6KfZuxi+Q02YUAGPmhtduA5948lRyAQttjx52kjoZfP/jt151JDxtMKr0ahiq4RGs98xqzLa5+lYz+pJiRYHyvUrvVA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891492; c=relaxed/simple; bh=ctGGpbIzK8JbaKPapPs/06WW7DP5TK3ylIpHn8zRy8E=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=i84aSp8zDybvqhhEy6RzwauoyZDWbhm2yW8t3FL8Nm2k6RqtdgbKZVRaGKXlxd2TjD5T3srhsK4kGL+rstNs8ujvo7HDWJ3W31oCqoioOzLklagnwEG+zllZvcSdcLIykgnKFloUbRlgvVN9xZMnN4k5YmguJBtvXcrX85+oMpQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ow945mIo; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ow945mIo" Received: by smtp.kernel.org (Postfix) with ESMTPS id 82DB8C4FF6C; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1784891492; bh=ctGGpbIzK8JbaKPapPs/06WW7DP5TK3ylIpHn8zRy8E=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ow945mIo7qPG3DwG9fWGlk0w901LUpGkzW9JgDexi/1K3PYgEC/atYyLxyU85+VSY ZTMTFH58BEnf/NBS+zR0rr92b3aTFx4LtTCMHRCtwzdgmurlzcQ2MlrFNGzrk5wbet esZf8TQ9nk77V2ZlCd0voM/kkyOSLR1QzcSASUu3d/dgUeJ2VFXeJeuuFmCXXh9hq8 60298jIjPkwHt61PHWWsaKBzt5jhoWqadUSE6luQW7LXr9uqgPiJ0CqWva06myDEww r55L4qmF6m4lbpAUdj9byzatHAtAnZO8M10O36XkxE841jkN3ICogNRVgWPGlCMa0E JyVrLIvHOIKKg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B39AC531F9; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) From: Joel Granados Date: Fri, 24 Jul 2026 13:10:58 +0200 Subject: [PATCH RFC v2 04/14] nvme: Add CDQ to xarray and define delete path Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-jag-cdq-lkml-v2-4-659a7449e849@kernel.org> References: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> In-Reply-To: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jason Gunthorpe Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Joel Granados X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=6664; i=joel.granados@kernel.org; h=from:subject:message-id; bh=ctGGpbIzK8JbaKPapPs/06WW7DP5TK3ylIpHn8zRy8E=; b=owJ4nAHtARL+kA0DAAoBupfNUreWQU8ByyZiAGpjSFt/XNfgWgLFYWMGHMQdodhVUC5mgZPEg hYvs+KyEnN0J4kBswQAAQoAHRYhBK5HCVcl5jElzssnkLqXzVK3lkFPBQJqY0hbAAoJELqXzVK3 lkFP09sL/j2jHWSsx+FLMji4B3z+ehsawvgMK2OqHJf/zTD9Mk6USJqqnhYbgCu41vLgxuRV3vT kPnuyJPp/uc41jYAc1FDH05H8uEQ9IE6kAzD0p+FLAnUFNU25brWYJqed1tfVl4/HhH0RhL6qUm pvjFTMTswpQev/aM/vb54A5h88pdALYJy03BEuHDdJCmXHVxSOOfjSPg+jHiQDF8bGN8OImy1tq sALsxUC9Nn9k3sq8UcTKHTQvMeYBGFakrCk3SXSoiJCXWdv4dcK9lgUy5AExU6KZucyEIvt2bSk VqSRHrhlW8f1wQL1NspkmG2q1Jfy0//scfvJmHHkxl5+F54DXVy90qXFSPU7pHL1lKyCz4aQT0s o5kvLlSXQ9nWn/W/LK3nAH/8LjV3ciEGZLnN2wle5aAuCaLFUrODr5qYAeboSrSrdTug4cgQZCf 4NgLJ9QdJ3FVVICdq2MmL8qIXan46M43iq0WMcpZZxTUaGysOhrgeQTUYGoDWlhW5OnnBOJvPL7 OM= X-Developer-Key: i=joel.granados@kernel.org; a=openpgp; fpr=F1F8E46D30F0F6C4A45FF4465895FAAC338C6E77 X-Endpoint-Received: by B4 Relay for joel.granados@kernel.org/default with auth_id=239 Add Controller managed CDQs to a new xarray in nvme_ctrl indexed by the CDQ id returned by the controller. The new cdqs xarray will house pointers to the struct cdq_nvme_queue which contains individual CDQ metadata. Start it off with id and ctrl back pointer; it will grow as features get added. Commands on the delete path: 1. Individual -vs- "All" CDQ delete: - The individual delete path is taken when there is a specific command from user space to "delete" one CDQ - The "all" delete is taken on contrller reset 2. Separation between host and controller delete: - Controller delete takes care of sending and forwarding the result of the nvme CDQ delete command - Host delete memory on the host side The separation between host and controller delete is important in nvme_disable_ctrl as the host should delete the memory but expect the controller to follow NVMe protocol and remove the CDQ on its side. Note that there is no way of adding a CDQ just yet. This commit focuses on the delete path and leaves room for handling the creation in subsequent commits. Signed-off-by: Joel Granados --- drivers/nvme/host/Makefile | 2 +- drivers/nvme/host/cdq.c | 62 ++++++++++++++++++++++++++++++++++++++++++= ++++ drivers/nvme/host/cdq.h | 20 +++++++++++++++ drivers/nvme/host/core.c | 15 ++++++++++- drivers/nvme/host/nvme.h | 1 + 5 files changed, 98 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/host/Makefile b/drivers/nvme/host/Makefile index 6414ec968f99ae69f991cbd61e7ad79554e5041f..cd895082e259a9b0673c7400f6b= a62401f95767f 100644 --- a/drivers/nvme/host/Makefile +++ b/drivers/nvme/host/Makefile @@ -10,7 +10,7 @@ obj-$(CONFIG_NVME_FC) +=3D nvme-fc.o obj-$(CONFIG_NVME_TCP) +=3D nvme-tcp.o obj-$(CONFIG_NVME_APPLE) +=3D nvme-apple.o =20 -nvme-core-y +=3D core.o ioctl.o sysfs.o pr.o +nvme-core-y +=3D core.o ioctl.o sysfs.o pr.o cdq.o nvme-core-$(CONFIG_NVME_VERBOSE_ERRORS) +=3D constants.o nvme-core-$(CONFIG_TRACING) +=3D trace.o nvme-core-$(CONFIG_NVME_MULTIPATH) +=3D multipath.o diff --git a/drivers/nvme/host/cdq.c b/drivers/nvme/host/cdq.c new file mode 100644 index 0000000000000000000000000000000000000000..0e9ac88a054f6630c1eea73ae41= 40f124b6bb31f --- /dev/null +++ b/drivers/nvme/host/cdq.c @@ -0,0 +1,62 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * NVMe Controller Data Queue (CDQ) support. + */ + +#include "nvme.h" +#include "cdq.h" + +static int nvme_submit_delete_cdq_cmd(const struct cdq_nvme_queue *cdq) +{ + struct nvme_command c =3D { + .cdq.opcode =3D nvme_admin_cdq, + .cdq.sel =3D NVME_CDQ_CMD_MGMT_DELETE, + .cdq.dw11.cdqid =3D cpu_to_le16(cdq->id) + }; + + return __nvme_submit_sync_cmd(cdq->ctrl->admin_q, &c, NULL, NULL, 0, NVME= _QID_ANY, 0); +} + +/* Sends a CDQ delete NVMe cmd */ +static void nvme_delete_cdq_ctrl(struct cdq_nvme_queue *cdq) +{ + if (nvme_submit_delete_cdq_cmd(cdq)) + WARN_ONCE(1, "Failed delete CDQ (id: %d)", cdq->id); +} + +/* Does NOT send a CDQ delete NVMe cmd */ +static void nvme_delete_cdq_host(struct cdq_nvme_queue *cdq) +{ + u16 cdq_id =3D cdq->id; + struct nvme_ctrl *ctrl =3D cdq->ctrl; + + xa_erase(&ctrl->cdqs, cdq_id); +} + +void nvme_delete_cdq(struct cdq_nvme_queue *cdq) +{ + nvme_delete_cdq_ctrl(cdq); + nvme_delete_cdq_host(cdq); +} +EXPORT_SYMBOL_GPL(nvme_delete_cdq); + +/* Will NOT send a CDQ delete NVMe cmd. */ +void nvme_delete_cdqs_host(struct nvme_ctrl *ctrl) +{ + struct cdq_nvme_queue *cdq; + unsigned long i; + + xa_for_each(&ctrl->cdqs, i, cdq) + nvme_delete_cdq_host(cdq); +} + +/* Final teardown at device->release: free all CDQs and destroy the xarray= . */ +void nvme_free_cdqs(struct nvme_ctrl *ctrl) +{ + /* + * Delete host side CDQ only. NOT sending delete cmd as + * Ctrl should delete on disable. + */ + nvme_delete_cdqs_host(ctrl); + xa_destroy(&ctrl->cdqs); +} diff --git a/drivers/nvme/host/cdq.h b/drivers/nvme/host/cdq.h new file mode 100644 index 0000000000000000000000000000000000000000..4378f97553ce2e649d8998eb168= f82039dc762b1 --- /dev/null +++ b/drivers/nvme/host/cdq.h @@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * NVMe Controller Data Queue (CDQ) support. + */ + +#ifndef _NVME_CDQ_H +#define _NVME_CDQ_H + +#include "nvme.h" + +struct cdq_nvme_queue { + u16 id; + struct nvme_ctrl *ctrl; +}; + +void nvme_delete_cdq(struct cdq_nvme_queue *cdq); +void nvme_delete_cdqs_host(struct nvme_ctrl *ctrl); +void nvme_free_cdqs(struct nvme_ctrl *ctrl); + +#endif /* _NVME_CDQ_H */ diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index d11858300ad43f3e229eab7f2dd77563075b801e..fbaa2e065ab846e4700cddd7ae9= f30a52e8a2cd9 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -25,6 +25,7 @@ #include =20 #include "nvme.h" +#include "cdq.h" #include "fabrics.h" #include =20 @@ -2665,9 +2666,19 @@ int nvme_disable_ctrl(struct nvme_ctrl *ctrl, bool s= hutdown) } if (ctrl->quirks & NVME_QUIRK_DELAY_BEFORE_CHK_RDY) msleep(NVME_QUIRK_DELAY_AMOUNT); - return nvme_wait_ready(ctrl, NVME_CSTS_RDY, 0, + ret =3D nvme_wait_ready(ctrl, NVME_CSTS_RDY, 0, (NVME_CAP_TIMEOUT(ctrl->cap) + 1) / 2, "reset"); + if (ret) + return ret; + + /* + * Delete host side CDQ only. Purposefully NOT sending delete cmd. + * Ctrl should delete on disable. + */ + nvme_delete_cdqs_host(ctrl); + return ret; } + EXPORT_SYMBOL_GPL(nvme_disable_ctrl); =20 int nvme_enable_ctrl(struct nvme_ctrl *ctrl) @@ -5074,6 +5085,7 @@ static void nvme_free_ctrl(struct device *dev) if (!subsys || ctrl->instance !=3D subsys->instance) ida_free(&nvme_instance_ida, ctrl->instance); nvme_free_cels(ctrl); + nvme_free_cdqs(ctrl); nvme_mpath_uninit(ctrl); cleanup_srcu_struct(&ctrl->srcu); nvme_auth_stop(ctrl); @@ -5120,6 +5132,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct dev= ice *dev, mutex_init(&ctrl->scan_lock); INIT_LIST_HEAD(&ctrl->namespaces); xa_init(&ctrl->cels); + xa_init(&ctrl->cdqs); ctrl->dev =3D dev; ctrl->ops =3D ops; ctrl->quirks =3D quirks; diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 654e5c1b64956c877ff845f0b1f7ee2fa0852b64..7b9d1781816c4bad719584c3d4b= 4f9fdf7b4e3dc 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -466,6 +466,7 @@ struct nvme_ctrl { enum nvme_dctype dctype; =20 u16 awupf; /* 0's based value. */ + struct xarray cdqs; }; =20 static inline enum nvme_ctrl_state nvme_ctrl_state(struct nvme_ctrl *ctrl) --=20 2.50.1 From nobody Fri Jul 24 21:29:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D2BE64322F6 for ; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891492; cv=none; b=oZpGvUhGKKvWSvPGB9zgImqOf5zH+CFm6YC6ZQQ61NWl1Tbqpi1TdKjufAJmRPSk0atTOoVXeepjNETArnEyjaZTjxeWQVTWbhcPfmKQYh4PxRkWLHVjjK3wuv8IrMzLT+rgrxOhdRQPxDyrzBCQZSvP+AsIGq/JlfXRvkLzysU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891492; c=relaxed/simple; bh=BfgjSIHRduyFlHDKxF9kQTlbbOfb/JFgHX9Y+cGfFGE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=s8LExICnGQFrtDw+feRdl/Go/NLQRitbHiYsvU+3Hr/ynelMtr71tvxMCSkZeXhagwfHHcxZRbmm1pK1bDby2zJ++2nY2wnuBwJwIKF3J3rXXxwJTiM9jxmLmctN84EE3TNbExQJvzTk9PAXvAU0HFtWn9O5KbeFia4yXe0Bn6g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NXPxkQCr; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NXPxkQCr" Received: by smtp.kernel.org (Postfix) with ESMTPS id 942FDC4FF74; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1784891492; bh=BfgjSIHRduyFlHDKxF9kQTlbbOfb/JFgHX9Y+cGfFGE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=NXPxkQCrhBD/e1QNwcWn4hsJF91FL4JYL+cpzPBPgeE00S8tQsf2Zq1UMwAImb2pA YTWa8C6rwuhffsGDPETXS8xGowJEJTOvF5SM02Hz0wbxGjNd89EULJ+GwPSx6DK+UN ReGzdDyxgHC7CWrfJTd/bSQq/La/8jcB7j6WKiLGCL29Yia6BzoDiidjE7nIIU+l/0 a+j7KIZ4aa4mb9rRyiXVTiLBgiqsyXm7Ak+D09tA0TtQVyzZYsv1DsB0NorVH+minP ti09R39B0MKMp97PU3kqn7g/sHa4/hEvvyGTnDtDoY60eVqK5bYyzJ3UZpYb8e/4/Q Xjjqs/F0Q70dQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77C6EC53215; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) From: Joel Granados Date: Fri, 24 Jul 2026 13:10:59 +0200 Subject: [PATCH RFC v2 05/14] nvme: Allocate CDQ backing memory from coherent DMA chunks Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-jag-cdq-lkml-v2-5-659a7449e849@kernel.org> References: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> In-Reply-To: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jason Gunthorpe Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Joel Granados X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=7457; i=joel.granados@kernel.org; h=from:subject:message-id; bh=BfgjSIHRduyFlHDKxF9kQTlbbOfb/JFgHX9Y+cGfFGE=; b=owJ4nAHtARL+kA0DAAoBupfNUreWQU8ByyZiAGpjSFsugttGswnUgb/AhoTjEnse+H5+d9fv0 cS+o/S1+KIeu4kBswQAAQoAHRYhBK5HCVcl5jElzssnkLqXzVK3lkFPBQJqY0hbAAoJELqXzVK3 lkFPvCsL/iUj4uWfffVoADCmyBI0xahIlL6NWoEzkz0+bptUmYTkBP7u8eai+BVU7cAHVpHtBiU ScGgsyrsJBNQctK6/3cKCUgjohoTxhZNErEgccM9GmiMqhXRJvQl0sYxlNrb5F+YJLHnOZlxRWE SLNcZSQ7eKWF2TcXt+4UYxh2ZEaoobND5ez+vMhfa/Vc97FNtH9OB+/CPohudh8NclBIaUNb+Uh K/a9bglCLCg4KBhMZ3tY9oCckhZnvMXRN4SGLTmKYXVvk4F1XPwr9NjkADD/QQJTLVl60uiJdkJ I/6a6hoH1ty/ZXXS9ybob6p0mDB5TUfw9/wadwmvIq3YWtY9Og42uw8vnZj1rhZwy9rc3ku9g/u IcOqJOb0zFpF0Yv6o9vcJVG0jV5iYQG/B+Dg/+LHucAa1LYNG8sPe6+Rpw8vEo5Yu+1KQ+Y2dHa rqaQ9wtybLXQvqZ7lWwNjgl4HLZ9svXa+vARcMyP8V8A4634BdTp+fNXgCt/FMeFAmTt0uYZ718 nQ= X-Developer-Key: i=joel.granados@kernel.org; a=openpgp; fpr=F1F8E46D30F0F6C4A45FF4465895FAAC338C6E77 X-Endpoint-Received: by B4 Relay for joel.granados@kernel.org/default with auth_id=239 Back a Controller Data Queue with a set of dma_alloc_coherent() chunks. This mem is portable to non-coherent archs and stays put for the life of the CDQ. Splitting it into fixed-size chunks avoids the MAX_PAGE_ORDER limit on the no-IOMMU path. A single chunk is handed over as PC_CONT; multiple chunks are stitched together with a chained PRP list (PC_DISCONT). Since it is easier to handle 1 chunk (no prp list), try dma_alloc_coherent with the totality of the CDQ size and fallback on the chunk allocation only when it was not possible to have a contiguous allocation. Build the prp list only when there is more than one chunk of backing memory. Signed-off-by: Joel Granados --- drivers/nvme/host/cdq.c | 166 ++++++++++++++++++++++++++++++++++++++++++++= +++- drivers/nvme/host/cdq.h | 27 ++++++++ 2 files changed, 191 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/host/cdq.c b/drivers/nvme/host/cdq.c index 0e9ac88a054f6630c1eea73ae4140f124b6bb31f..73dd6af42055687bec62ad0350c= 67736c9c047c2 100644 --- a/drivers/nvme/host/cdq.c +++ b/drivers/nvme/host/cdq.c @@ -6,6 +6,166 @@ #include "nvme.h" #include "cdq.h" =20 +static inline void nvme_free_cdqmem_chunks(struct cdq_nvme_queue *cdq) +{ + struct device *dev =3D cdq->ctrl->dev; + unsigned int i; + + if (!cdq->chunks) + return; + for (i =3D 0; i < cdq->nr_chunks; i++) { + if (cdq->chunks[i].vaddr) + dma_free_coherent(dev, cdq->chunks[i].size, + cdq->chunks[i].vaddr, + cdq->chunks[i].dma_addr); + } + kfree(cdq->chunks); + cdq->chunks =3D NULL; + cdq->nr_chunks =3D 0; +} + +static inline int nvme_alloc_cdqmem_chunks(struct cdq_nvme_queue *cdq) +{ + struct device *dev =3D cdq->ctrl->dev; + unsigned int i, nr =3D 1; + + /* + * Try to get a single pointer to the whole cdq, it will take + * the iommu path within the dma_alloc_coherent call + */ + cdq->chunks =3D kcalloc(nr, sizeof(*cdq->chunks), GFP_KERNEL); + if (!cdq->chunks) + return -ENOMEM; + + cdq->chunks[0].vaddr =3D dma_alloc_coherent(dev, cdq->size_nbyte, + &cdq->chunks[0].dma_addr, GFP_KERNEL); + if (cdq->chunks[0].vaddr) + goto out; + + /* Fall back to allocating several chunks */ + nr =3D DIV_ROUND_UP(cdq->size_nbyte, NVME_CDQ_CHUNK_SIZE); + + cdq->chunks =3D krealloc_array(cdq->chunks, nr, sizeof(*cdq->chunks), GFP= _KERNEL); + if (!cdq->chunks) + return -ENOMEM; + + for (i =3D 0; i < nr; i++) { + cdq->chunks[i].vaddr =3D dma_alloc_coherent(dev, + NVME_CDQ_CHUNK_SIZE, &cdq->chunks[i].dma_addr, + GFP_KERNEL); + if (!cdq->chunks[i].vaddr) { + nvme_free_cdqmem_chunks(cdq); + return -ENOMEM; + } + cdq->chunks[i].size =3D NVME_CDQ_CHUNK_SIZE; + } + +out: + cdq->nr_chunks =3D nr; + return 0; +} + +static inline void nvme_free_cdqmem_prp_lists(struct cdq_nvme_queue *cdq) +{ + struct device *dev =3D cdq->ctrl->dev; + unsigned int prp_idx; + + for (prp_idx =3D 0; prp_idx < cdq->nr_prp_lists; prp_idx++) + dma_free_coherent(dev, PAGE_SIZE, cdq->prp_lists[prp_idx], + cdq->prp_lists_dma[prp_idx]); + cdq->nr_prp_lists =3D 0; +} + +static inline dma_addr_t nvme_get_cdq_pagedma(struct cdq_nvme_queue *cdq, + unsigned int page_idx) +{ + return cdq->chunks[page_idx / NVME_CDQ_PAGES_PER_CHUNK].dma_addr + + (page_idx % NVME_CDQ_PAGES_PER_CHUNK) * NVME_CTRL_PAGE_SIZE; +} + +static inline int nvme_build_cdqmem_prp_list(struct cdq_nvme_queue *cdq) +{ + struct device *dev =3D cdq->ctrl->dev; + const unsigned int prps_per_page =3D PAGE_SIZE >> 3; + unsigned int total_pages =3D + DIV_ROUND_UP(cdq->size_nbyte, NVME_CTRL_PAGE_SIZE); + dma_addr_t prp_list_dma; + __le64 *prp_list; + unsigned int prp_idx, page_idx; + + prp_list =3D dma_alloc_coherent(dev, PAGE_SIZE, &prp_list_dma, GFP_KERNEL= ); + if (!prp_list) + return -ENOMEM; + cdq->prp_lists[0] =3D prp_list; + cdq->prp_lists_dma[0] =3D prp_list_dma; + cdq->nr_prp_lists =3D 1; + + for (page_idx =3D 0, prp_idx =3D 0; page_idx < total_pages; page_idx++) { + dma_addr_t page_dma =3D nvme_get_cdq_pagedma(cdq, page_idx); + + /* Current prp_list page full with entries still to place: chain. */ + if (prp_idx =3D=3D prps_per_page) { + __le64 *old =3D prp_list; + + if (cdq->nr_prp_lists =3D=3D MAX_NR_CDQ_PRPS) + goto err; + prp_list =3D dma_alloc_coherent(dev, PAGE_SIZE, &prp_list_dma, + GFP_KERNEL); + if (!prp_list) + goto err; + cdq->prp_lists[cdq->nr_prp_lists] =3D prp_list; + cdq->prp_lists_dma[cdq->nr_prp_lists++] =3D prp_list_dma; + + /* Chain from old to new prp_list */ + prp_list[0] =3D old[prps_per_page - 1]; + old[prps_per_page - 1] =3D cpu_to_le64(prp_list_dma); + prp_idx =3D 1; + } + prp_list[prp_idx++] =3D cpu_to_le64(page_dma); + } + return 0; +err: + nvme_free_cdqmem_prp_lists(cdq); + return -ENOMEM; +} + +/* + * Allocate the coherent store and prps for a CDQ. + * Expects cdq with size_nbytes and ctrl set. + */ +static inline int nvme_create_cdq_backing(struct cdq_nvme_queue *cdq) +{ + int ret; + + if (!cdq->size_nbyte) + return -EINVAL; + + ret =3D nvme_alloc_cdqmem_chunks(cdq); + if (ret) + return ret; + + /* We pass cdq->chunks[0].dma_addr when cdq->nr_chunks =3D=3D 1 */ + if (cdq->nr_chunks > 1) { + ret =3D nvme_build_cdqmem_prp_list(cdq); + if (ret) + goto err_chunks; + } + + return 0; + +err_chunks: + nvme_free_cdqmem_chunks(cdq); + + return ret; +} + +/* Release the coherent DMA backing allocated by nvme_create_cdq_backing()= . */ +static inline void nvme_release_cdq_backing(struct cdq_nvme_queue *cdq) +{ + nvme_free_cdqmem_prp_lists(cdq); + nvme_free_cdqmem_chunks(cdq); +} + static int nvme_submit_delete_cdq_cmd(const struct cdq_nvme_queue *cdq) { struct nvme_command c =3D { @@ -27,10 +187,12 @@ static void nvme_delete_cdq_ctrl(struct cdq_nvme_queue= *cdq) /* Does NOT send a CDQ delete NVMe cmd */ static void nvme_delete_cdq_host(struct cdq_nvme_queue *cdq) { - u16 cdq_id =3D cdq->id; struct nvme_ctrl *ctrl =3D cdq->ctrl; =20 - xa_erase(&ctrl->cdqs, cdq_id); + if (xa_erase(&ctrl->cdqs, cdq->id) !=3D cdq) + return; + + nvme_release_cdq_backing(cdq); } =20 void nvme_delete_cdq(struct cdq_nvme_queue *cdq) diff --git a/drivers/nvme/host/cdq.h b/drivers/nvme/host/cdq.h index 4378f97553ce2e649d8998eb168f82039dc762b1..5deaf3705c65250ea9fdc69e2e9= 73987c8f084c1 100644 --- a/drivers/nvme/host/cdq.h +++ b/drivers/nvme/host/cdq.h @@ -8,9 +8,36 @@ =20 #include "nvme.h" =20 +/* + * The CDQ backing is a set of coherent DMA chunks. Chunk size expressed in + * host pages to match dma_alloc_coherency granularity. + */ +#define NVME_CDQ_CHUNK_ORDER 2 +#define NVME_CDQ_CHUNK_SIZE (PAGE_SIZE << NVME_CDQ_CHUNK_ORDER) +#define NVME_CDQ_PAGES_PER_CHUNK (NVME_CDQ_CHUNK_SIZE / NVME_CTRL_PAGE_SIZ= E) + +/* Max PRP List pages we are willing to chain to describe a discontiguous = CDQ. */ +#define MAX_NR_CDQ_PRPS 20 + +struct nvme_cdq_chunk { + void *vaddr; + dma_addr_t dma_addr; + size_t size; +}; + struct cdq_nvme_queue { u16 id; struct nvme_ctrl *ctrl; + u32 size_nbyte; + + /* Coherent backing store. */ + struct nvme_cdq_chunk *chunks; + unsigned int nr_chunks; + + /* PRP List pages describing the chunks to the controller (PC_DISCONT). */ + __le64 *prp_lists[MAX_NR_CDQ_PRPS]; + dma_addr_t prp_lists_dma[MAX_NR_CDQ_PRPS]; + unsigned int nr_prp_lists; }; =20 void nvme_delete_cdq(struct cdq_nvme_queue *cdq); --=20 2.50.1 From nobody Fri Jul 24 21:29:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3C27432307 for ; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891492; cv=none; b=Cqx72Kc6YGuMMRpMPYDEudscK7wQ9vF4io9wINm8teVR6ZtHNAE88s8dgNUqBKDMtaPa+HuoWiYYG5fhW4tMHFfpLoFUES1/qLUSuB+VM/c7nEThBBYrrOFIBWnhdg1WMDWtpEfuSakOdLkvRbVWyq66kif+wwWDOZ5IQt5D9n0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891492; c=relaxed/simple; bh=cuXW3w1fKxELbA7o+sSCBXcLYsqfoxI6yUn985dXTOA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=f0cof0sBhMrvhpMIu0ku992+OlgEBkPrKnvXlXEWn3U4pvINpNTtakqGp3qCv/hHegUskjmnctxDLxiMJd7mNgCBL5k5urhse5/eIq01OsEP5PYv/jSpBJ8zHYkL2nE69UtWOEyxr/3z+QliG8fhAvmU0Y2K02agcnS3lRq9y+0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cNxwOLTV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cNxwOLTV" Received: by smtp.kernel.org (Postfix) with ESMTPS id 9AC95C4FF6E; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1784891492; bh=cuXW3w1fKxELbA7o+sSCBXcLYsqfoxI6yUn985dXTOA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=cNxwOLTVZv7oUuu/vCFfSoAzNqGwUDUgRynYElcUIX14PO8/ZfBU1qp65JC/e1MGt Kmb7vz7RjMRR1C1zwYYyzVM62UFZStsbtYxFPhk+iTQFlNSLys6MfFddRgqRFA3Jyw /0Oz/liGRv5pThtFEuhOyC6ZXP+CEckVDFrhkIN91i4E7RHCmjP8/liPSNU+TofiIk IJTSDMwwbJDjXaTSGptXZg2ik7iRvtCMFewqvrTGf/SSEFJmwfBVNyMclDJfGDoYBH U8u8eEYLZO4q0p9zNjKYh6XsYv8gkhkl5UJetozniAeEclzKbw9ozmY6uhtoPPGTD5 MBA7rz0QOqmKA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84469C531FC; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) From: Joel Granados Date: Fri, 24 Jul 2026 13:11:00 +0200 Subject: [PATCH RFC v2 06/14] nvme: Add file descriptor to read CDQs Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-jag-cdq-lkml-v2-6-659a7449e849@kernel.org> References: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> In-Reply-To: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jason Gunthorpe Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Joel Granados X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=4622; i=joel.granados@kernel.org; h=from:subject:message-id; bh=cuXW3w1fKxELbA7o+sSCBXcLYsqfoxI6yUn985dXTOA=; b=owJ4nAHtARL+kA0DAAoBupfNUreWQU8ByyZiAGpjSFx5KyutJlzAtt7LfqGf+4dOO0DZ/yiHc wmz28VygnJVXokBswQAAQoAHRYhBK5HCVcl5jElzssnkLqXzVK3lkFPBQJqY0hcAAoJELqXzVK3 lkFPSrYL/A8YCfOy/tCi1jiOi5MVYG0dCQ4K5wVNRQmhJgD+9xWG4XlU9NXj4IEt93v0eiInGwC JfKM+vAlAP7A2Co2M8pDJEphJ5jxt++mHhz3//y/NsjkaDQmae3vn6jZE0qDJV/JnAjYBlMMpnO Mys2x3qlg/XWIn/JyhWsVgyFd5cx/NjLoXZErZBewVsNxevfi2PouqS4X6+YaPiHhWex2ZPx9xV B2VsoSlieq3mNlmJ1eH8KMeH/O4jPnJ6NGucFQwqG5FCrEC2gOvwj6gUSdTRy5zlou8DTGOsoI5 vfSkFrpHo2SFiYcvsFKbWxH/s42HHWyvw8dN9Ruy4s6MAXmD7Y1hK+XNgrHld6GH+446pPKfU+s X8NN+xPqx0EW/J4TVMgGer8SqvPlytdIf05vPYZtzsin+5ugsgmr/hNE8sU3MyJUPUe/14Rc/G6 g6kywQBzWURq2eyOY3kb+VnCa6kCQq9Sl5lUzDZ7Wjrh0RmE/mp7Wv3GXN+PL20YNHqK8IWhwe8 aM= X-Developer-Key: i=joel.granados@kernel.org; a=openpgp; fpr=F1F8E46D30F0F6C4A45FF4465895FAAC338C6E77 X-Endpoint-Received: by B4 Relay for joel.granados@kernel.org/default with auth_id=239 CDQs are read using file descriptors provided by nvme_create_cdqfd. Only add the logic to create and release the file descriptor; return "not implemented" for now. The CDQ traversal logic will be added in a future commit. The CDQ memory is now pointed from two different places (the user space FD and the nvme controller struct) and should only be freed when there are no pointers left. Use a kref to ensure that it is in the FD's private_data even if the controller is goes down. Signed-off-by: Joel Granados --- drivers/nvme/host/cdq.c | 71 +++++++++++++++++++++++++++++++++++++++++++++= ++++ drivers/nvme/host/cdq.h | 21 +++++++++++++++ 2 files changed, 92 insertions(+) diff --git a/drivers/nvme/host/cdq.c b/drivers/nvme/host/cdq.c index 73dd6af42055687bec62ad0350c67736c9c047c2..63c0852e0ba17a17227b9a0189b= e5123358a2262 100644 --- a/drivers/nvme/host/cdq.c +++ b/drivers/nvme/host/cdq.c @@ -3,6 +3,9 @@ * NVMe Controller Data Queue (CDQ) support. */ =20 +#include +#include + #include "nvme.h" #include "cdq.h" =20 @@ -151,6 +154,9 @@ static inline int nvme_create_cdq_backing(struct cdq_nv= me_queue *cdq) goto err_chunks; } =20 + /* FIXME: put this on the create_cdq function*/ + kref_init(&cdq->ref); + return 0; =20 err_chunks: @@ -166,6 +172,69 @@ static inline void nvme_release_cdq_backing(struct cdq= _nvme_queue *cdq) nvme_free_cdqmem_chunks(cdq); } =20 +static ssize_t nvme_cdq_fops_read(struct file *filep, char __user *buf, + size_t size_nbyte, loff_t *ppos) +{ + struct cdq_nvme_queue *cdq =3D filep->private_data; + size_t nbytes =3D round_down(size_nbyte, NVME_CDQ_MQ_ENTRY_NRBYTES); + + if (*ppos) + return -ESPIPE; + + if (size_nbyte < NVME_CDQ_MQ_ENTRY_NRBYTES) + return -EINVAL; + + if (nbytes > (cdq->size_nbyte)) + return -EINVAL; + + /* CDQ traversal not implemented yet. */ + return -EOPNOTSUPP; +} + +/* File reference already dropped by the close path, so don't fput() */ +static int nvme_release_cdqfd(struct cdq_nvme_queue *cdq) +{ + nvme_cdq_put(cdq); + return 0; +} + +static int nvme_cdq_fops_release(struct inode *inode, struct file *filep) +{ + return nvme_release_cdqfd(filep->private_data); +} + +static const struct file_operations cdq_fops =3D { + .owner =3D THIS_MODULE, + .open =3D nonseekable_open, + .read =3D nvme_cdq_fops_read, + .release =3D nvme_cdq_fops_release, +}; + +__maybe_unused +static int nvme_create_cdqfd(struct cdq_nvme_queue *cdq, int *cdq_fdno) +{ + int fdno; + struct file *filep; + + filep =3D anon_inode_getfile("[cdq-readfd]", &cdq_fops, cdq, O_RDWR); + if (IS_ERR(filep)) + return PTR_ERR(filep); + + /* cdq is being pionted at by ->private_data. increase ref */ + nvme_cdq_get(cdq); + + fdno =3D get_unused_fd_flags(O_CLOEXEC | O_RDONLY | O_DIRECT); + if (fdno < 0) { + fput(filep); /* nvme_cdq_put through release */ + return fdno; + } + + fd_install(fdno, filep); + *cdq_fdno =3D fdno; + + return 0; +} + static int nvme_submit_delete_cdq_cmd(const struct cdq_nvme_queue *cdq) { struct nvme_command c =3D { @@ -193,6 +262,8 @@ static void nvme_delete_cdq_host(struct cdq_nvme_queue = *cdq) return; =20 nvme_release_cdq_backing(cdq); + + nvme_cdq_put(cdq); } =20 void nvme_delete_cdq(struct cdq_nvme_queue *cdq) diff --git a/drivers/nvme/host/cdq.h b/drivers/nvme/host/cdq.h index 5deaf3705c65250ea9fdc69e2e973987c8f084c1..8c003aa75bd3e4df9a5f8af7ff2= 1d1957dd43923 100644 --- a/drivers/nvme/host/cdq.h +++ b/drivers/nvme/host/cdq.h @@ -8,6 +8,8 @@ =20 #include "nvme.h" =20 +#define NVME_CDQ_MQ_ENTRY_NRBYTES 32 + /* * The CDQ backing is a set of coherent DMA chunks. Chunk size expressed in * host pages to match dma_alloc_coherency granularity. @@ -38,8 +40,27 @@ struct cdq_nvme_queue { __le64 *prp_lists[MAX_NR_CDQ_PRPS]; dma_addr_t prp_lists_dma[MAX_NR_CDQ_PRPS]; unsigned int nr_prp_lists; + + /* Manage refs for read FD and controller xarray */ + struct kref ref; }; =20 +/* Must not touch cdq->ctrl: Ctrl may have been freed */ +static inline void nvme_free_cdq(struct kref *ref) +{ + kfree(container_of(ref, struct cdq_nvme_queue, ref)); +} + +static inline void nvme_cdq_get(struct cdq_nvme_queue *cdq) +{ + kref_get(&cdq->ref); +} + +static inline void nvme_cdq_put(struct cdq_nvme_queue *cdq) +{ + kref_put(&cdq->ref, nvme_free_cdq); +} + void nvme_delete_cdq(struct cdq_nvme_queue *cdq); void nvme_delete_cdqs_host(struct nvme_ctrl *ctrl); void nvme_free_cdqs(struct nvme_ctrl *ctrl); --=20 2.50.1 From nobody Fri Jul 24 21:29:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA118433039 for ; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; cv=none; b=HAo0y3qqf01UPmydR3hRrip7bS9uOP1yCdwxj1pxlh1joq8WinGazYMqgHVRpAdYLpyZGiIw1WFw0OwQmftGM8bKKxD15w6Cf1Jd5ay3/WY3VfA0BuKgxaRoJq++44hVy9LjjeCMdD2cFiEQoIV2Vfm2YUH/bhA2s3zPkuV1bww= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; c=relaxed/simple; bh=OBk+pHhS1EgYqmXtorRqQcAHABFfQD7THUrxKhZ6gqc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=TKSs9agp6cd4IFTWy60j7phSGzf9nZAFoa8ktcol303RSFWkNrrFzRqtw/DEJIp4gBnBqp8SWsveuerOlA7ejVA+n4iguS0cuP3a8tBEdgxBlJ/BLIumjBsgOx1iv3/liDspPM3Tk9kKYtR02Avw0gON/jxqqAO6bn3IiL7JBrU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YYUW9CwR; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YYUW9CwR" Received: by smtp.kernel.org (Postfix) with ESMTPS id A848DC4FF75; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1784891492; bh=OBk+pHhS1EgYqmXtorRqQcAHABFfQD7THUrxKhZ6gqc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=YYUW9CwRTCTPXefWsjQXc8vKhcDI/MjpuPST/2wKQDsHGt8U8d+j73b5X7wjrPkRS 6c4s75gB+gfOfarRVLM/2G0FJo7Uatr6R2XwFbtTUOV6AYsWe/Q4DC5Fj3KnOhUazP p3mDKrCq07/IoBXUOPFhKMI7Jyo1tXyNBBXKdzQKIqTYfPl7vQII0B6sa+lUhclF1h AiSy8ssyDgXOEF23V+hIlgmu68i/syCQ1RR2F09p7vRxpQ6J2VqYfjXSlLRDa8CJ5I OEaeupLArEn25ebfInyNvHSZa38rcLp7zl8poUuWHmblFOGnH3lwHRycI83YJlYvVn mReG6Z0UtPI6w== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 927D8C531FA; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) From: Joel Granados Date: Fri, 24 Jul 2026 13:11:01 +0200 Subject: [PATCH RFC v2 07/14] nvme: Add the create CDQ functionality Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-jag-cdq-lkml-v2-7-659a7449e849@kernel.org> References: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> In-Reply-To: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jason Gunthorpe Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Joel Granados X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5058; i=joel.granados@kernel.org; h=from:subject:message-id; bh=OBk+pHhS1EgYqmXtorRqQcAHABFfQD7THUrxKhZ6gqc=; b=owJ4nAHtARL+kA0DAAoBupfNUreWQU8ByyZiAGpjSFzcopeYI1fdpWY2cOMQOB0l6oBmK7S2p yxrYkxBIAn3GYkBswQAAQoAHRYhBK5HCVcl5jElzssnkLqXzVK3lkFPBQJqY0hcAAoJELqXzVK3 lkFPWxoL/irh/wA6FJ5rtOyyCO8vmaIeVuRLIE1U4zKdetp0Bp+CJrsHgG/5qXRihGwDCNsaOXM zxADKVsK2Umms2vxXiiYndtxXxNCoOW5QasGzfzhBHKCrT5Icsez5On/7BAE2Zpa/VzlPqgBkXR j5aKInfTsu/GvSFDgp3VJxftM0WjWcGAhNBv8io89/qP5FTK6T945LwmP+B6zQJ5FlFre9vkjNh /j0EHALpoZ4Zyqlpi+rVw6apyzb9Oa4S2Z2Xi8++qIHI8SMKR2H1E5vQcP7mNztTP/IuhHF5hj2 GnqOidLiMvxKHhwOWiM6XHTYU0nZdirBHmN3Tih7TDuwDC/d/Un37LJvJ4vbPebA2h8e2jXqnkp +ptz6z+nz2YBKZlpH+vAbj8ARFGHFcLJowk9zMB/uAkI6Rbzm28ehAazGmz8y0Y9FT2bVnoNmgg 7R4J6IbQxXTMcMLsFWvxrSjwncx60nqeqSJKWDlv6gUDeJUu93K2Na8rf7fK8uSph0L+cXyJMP5 PM= X-Developer-Key: i=joel.granados@kernel.org; a=openpgp; fpr=F1F8E46D30F0F6C4A45FF4465895FAAC338C6E77 X-Endpoint-Received: by B4 Relay for joel.granados@kernel.org/default with auth_id=239 The CDQ create allocates the struct cdq_nvme_queue, allocates the memory for the CDQ entries, sends the nvme command to create the CDQ at the controller side, creates the file descriptor for user space and inserts the CDQ struct in the controller cdqs xarray. Note that: * All memory that is to be shared with the controller needs to be setup before calling nvme_submit_create_cdq_cmd. * Initializing the refcount before sending the create cmd to the controller "assigns" the reference to the controller. In other words, that cdq is being used by that controller. * The creation of the cdq file descriptor is at the end because everything needs to be in place for when the user calls read. The FD will be "installed" once nvme_create_cdqfd returns. * nvme_submit_create_cdq_cmd must come before xa_insert as the cdq is inserted to the xarray with the id returned by the controller Signed-off-by: Joel Granados --- drivers/nvme/host/cdq.c | 90 +++++++++++++++++++++++++++++++++++++++++++++= ++-- drivers/nvme/host/cdq.h | 4 ++- 2 files changed, 90 insertions(+), 4 deletions(-) diff --git a/drivers/nvme/host/cdq.c b/drivers/nvme/host/cdq.c index 63c0852e0ba17a17227b9a0189be5123358a2262..b1776637191db5747053bdcf4bd= f97ef44a66566 100644 --- a/drivers/nvme/host/cdq.c +++ b/drivers/nvme/host/cdq.c @@ -154,9 +154,6 @@ static inline int nvme_create_cdq_backing(struct cdq_nv= me_queue *cdq) goto err_chunks; } =20 - /* FIXME: put this on the create_cdq function*/ - kref_init(&cdq->ref); - return 0; =20 err_chunks: @@ -273,6 +270,93 @@ void nvme_delete_cdq(struct cdq_nvme_queue *cdq) } EXPORT_SYMBOL_GPL(nvme_delete_cdq); =20 +static int nvme_submit_create_cdq_cmd(struct cdq_nvme_queue *cdq) +{ + int ret; + union nvme_result result =3D {}; + struct nvme_command c =3D { + .cdq.opcode =3D nvme_admin_cdq, + .cdq.sel =3D NVME_CDQ_CMD_MGMT_CREATE, + .cdq.mos =3D cpu_to_le16(NVME_CDQ_CMD_MGMT_CREATE_MOS_QT_UDMQ), + .cdq.dw11.cqs =3D cpu_to_le16(cdq->mc_id), + .cdq.cdqsize =3D cpu_to_le32(cdq->size_nbyte >> 2) // size is in dwords + }; + + if (cdq->nr_chunks < 2) { + c.cdq.dw11.flags =3D cpu_to_le16(NVME_CDQ_CMD_MGMT_CREATE_PC_CONT); + c.cdq.prp1 =3D cpu_to_le64(cdq->chunks[0].dma_addr); + } else { + c.cdq.dw11.flags =3D cpu_to_le16(NVME_CDQ_CMD_MGMT_CREATE_PC_DISCONT); + c.cdq.prp1 =3D cpu_to_le64(cdq->prp_lists_dma[0]); + } + + ret =3D __nvme_submit_sync_cmd(cdq->ctrl->admin_q, &c, &result, NULL, 0, = NVME_QID_ANY, 0); + if (ret) + return ret; + + cdq->id =3D le16_to_cpu(result.u16); + + return ret; +} + +int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 entry_nr, const u16 = mc_id) +{ + u64 size_nbyte =3D (u64)entry_nr * NVME_CDQ_MQ_ENTRY_NRBYTES; + struct cdq_nvme_queue *cdq =3D NULL; + int ret, cdq_fd; + + /* The backing size and the CDQSIZE field are both u32 (bytes). */ + if (size_nbyte > U32_MAX) + return -EINVAL; + + cdq =3D kzalloc_obj(*cdq); + if (!cdq) + return -ENOMEM; + + cdq->mc_id =3D mc_id; + cdq->ctrl =3D ctrl; + cdq->size_nbyte =3D (u32)size_nbyte; + + ret =3D nvme_create_cdq_backing(cdq); + if (ret) { + kfree(cdq); + return ret; + } + + kref_init(&cdq->ref); + + ret =3D nvme_submit_create_cdq_cmd(cdq); + if (ret) + goto del_cdqmem; + + ret =3D xa_insert(&cdq->ctrl->cdqs, cdq->id, cdq, GFP_KERNEL); + if (ret) + goto del_cmd; + + ret =3D nvme_create_cdqfd(cdq, &cdq_fd); + if (ret) + goto del_xarray; + + return 0; + +del_xarray: + nvme_delete_cdq(cdq); + /*nvme_delete_cdq, has everything */ + return ret; + +del_cmd: + if (nvme_submit_delete_cdq_cmd(cdq)) + WARN_ONCE(1, "Failed delete CDQ (id: %d)", cdq->id); + +del_cdqmem: + /* puts the ref acquired by kref_init */ + nvme_release_cdq_backing(cdq); + nvme_cdq_put(cdq); + + return ret; +} +EXPORT_SYMBOL_GPL(nvme_create_cdq); + /* Will NOT send a CDQ delete NVMe cmd. */ void nvme_delete_cdqs_host(struct nvme_ctrl *ctrl) { diff --git a/drivers/nvme/host/cdq.h b/drivers/nvme/host/cdq.h index 8c003aa75bd3e4df9a5f8af7ff21d1957dd43923..773f93817d498784450eb0c8c9a= b00a95f0657dc 100644 --- a/drivers/nvme/host/cdq.h +++ b/drivers/nvme/host/cdq.h @@ -11,7 +11,7 @@ #define NVME_CDQ_MQ_ENTRY_NRBYTES 32 =20 /* - * The CDQ backing is a set of coherent DMA chunks. Chunk size expressed in + * The CDQ backing is a set of coherent DMA chunks expressed in * host pages to match dma_alloc_coherency granularity. */ #define NVME_CDQ_CHUNK_ORDER 2 @@ -31,6 +31,7 @@ struct cdq_nvme_queue { u16 id; struct nvme_ctrl *ctrl; u32 size_nbyte; + u16 mc_id; // migratable controller id =20 /* Coherent backing store. */ struct nvme_cdq_chunk *chunks; @@ -62,6 +63,7 @@ static inline void nvme_cdq_put(struct cdq_nvme_queue *cd= q) } =20 void nvme_delete_cdq(struct cdq_nvme_queue *cdq); +int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 entry_nr, const u16 = mc_id); void nvme_delete_cdqs_host(struct nvme_ctrl *ctrl); void nvme_free_cdqs(struct nvme_ctrl *ctrl); =20 --=20 2.50.1 From nobody Fri Jul 24 21:29:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F101A433047 for ; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; cv=none; b=bmcaPhWdxoy4/e1qheWv6WyU5UhZS4nI8Vn+HEas4vifOxjDCoq0svwGVoOjdbX0NDrt+wN500IQMblpy0uGA/Si5nt7FrV3cE3oMxI9PVt8/Ml0whGVXOXfEETKJbPAtiBFxwA4UCYlPYNp6vbBoRz0g0BwMxVKPCeNwgg505M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; c=relaxed/simple; bh=ypeS95ojXjYSmQZdAQ3ygCTtEie40Z+4aoiipnBi1C4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=EgB3msdweFlXC/Su2uNK8UV5MgBlAu+/6NnU+QIYbPQnLZsorQ253TGwqfoFeteFUlRZp7PoFnNiZFoWtV40QApmpcudT9Zc9PJ4JqNJZFqBRTpugTRtlPK+9fU+sgtKKyItLSSX1fxAkpI77zvL7bUaZMhyZHqP2L7k0s0Meyc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bEA0zMNh; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bEA0zMNh" Received: by smtp.kernel.org (Postfix) with ESMTPS id B54A3C4FF7B; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1784891492; bh=ypeS95ojXjYSmQZdAQ3ygCTtEie40Z+4aoiipnBi1C4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=bEA0zMNhTCT3WwK5RVubp5AZ5fK4yT30SEjU6iTXZ7NhAnNS0oMiag/XcWpmhBz6r xKkOu046KZVTTDV1XlbEReRthHd9dxH9gBIfrIskgvQQCKS1Uz31NBG8/3t2+ZcXvF 38lpoEgjZIZ1p2aLureifMdPihBMsnWGdnhpZgdOHfD9tD6RRbdKqBiI3YvEgGP7gF 0pIAX2DVdi+nDbA9ljUtrsRqZHgRrv+HSGfJWQNCgibomM12Nj6BfpTx/YpS64C3jT L54RQ9zJsPQh+t5XbYseOeGiqP/vOrR9C0seFCPHjclYdiCEKz1FFi/c0EFeWfbCer jAEmjCx+psvpQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E309C531C9; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) From: Joel Granados Date: Fri, 24 Jul 2026 13:11:02 +0200 Subject: [PATCH RFC v2 08/14] nvme: Submit track send command to start/stop the CDQ Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-jag-cdq-lkml-v2-8-659a7449e849@kernel.org> References: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> In-Reply-To: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jason Gunthorpe Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Joel Granados X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1887; i=joel.granados@kernel.org; h=from:subject:message-id; bh=ypeS95ojXjYSmQZdAQ3ygCTtEie40Z+4aoiipnBi1C4=; b=owJ4nAHtARL+kA0DAAoBupfNUreWQU8ByyZiAGpjSF3qt02lMyoQAxLyN+Ufsbey5R1oBWRge s76TvRR5comzIkBswQAAQoAHRYhBK5HCVcl5jElzssnkLqXzVK3lkFPBQJqY0hdAAoJELqXzVK3 lkFP0IUMAIQJ5olp+m448keWCoxcK9QwftwDHNiRBHzNCtDgZlvYKDrDm6iLr5YfxenmuyuGJZr 89fBbGE2uufTVQhXPijxwU3g34FdeFXFjc554+0Cmx2S6qtrG9rTrv5qfVX/86mCvW5ajbf437r a4U5G5RjiHIk10XR61wxSs56BmjyUlwZepqCdgFtR7qWSX0kO5Lk5oPB8lWRJ/7KsAsRhZrpP/W l/SwnyDhTDLPsHPpJmAjU68RFtnWm4Bw7W5GTRu3FnwWpyLeZkyOirQDmrv6/wFpXabkAp5/Ws3 sLzCFfZjVAVJsH5H7VAFyHHWonvwVeEtpNh8exlpBSUcRP0lKwf4t0yp+7X8YPLL/lIY/9Ut4iS PuPTEHpn6VK5QkdGIbOmgspdhwNSAnhP/ulto/0gECxeiyGayi62dzLQoehaPTFnCJ6ZgILEeBQ PmYgmBAPQ19yGwvHLK5NOoHpP+mEsXn1OHozeElu7ycIj5Z09ckFbC5YGnwDXp2efHZ5R79TeI8 is= X-Developer-Key: i=joel.granados@kernel.org; a=openpgp; fpr=F1F8E46D30F0F6C4A45FF4465895FAAC338C6E77 X-Endpoint-Received: by B4 Relay for joel.granados@kernel.org/default with auth_id=239 Send the start command at the end of Controller Data Queue creation. Send the stop before sending the "delete" message to the controller. Signed-off-by: Joel Granados --- drivers/nvme/host/cdq.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/drivers/nvme/host/cdq.c b/drivers/nvme/host/cdq.c index b1776637191db5747053bdcf4bdf97ef44a66566..0f01b58a9c3145faae83a81627f= a5a93cfc03cf4 100644 --- a/drivers/nvme/host/cdq.c +++ b/drivers/nvme/host/cdq.c @@ -243,9 +243,27 @@ static int nvme_submit_delete_cdq_cmd(const struct cdq= _nvme_queue *cdq) return __nvme_submit_sync_cmd(cdq->ctrl->admin_q, &c, NULL, NULL, 0, NVME= _QID_ANY, 0); } =20 +static int nvme_submit_startstop_cdq_cmd(struct cdq_nvme_queue *cdq, uint = startstop) +{ + struct nvme_command c =3D { + .track_send.opcode =3D nvme_admin_track_send, + .track_send.sel =3D NVME_TRSND_CMD_MGMT_LOG_USR_DATA, + .track_send.mos =3D cpu_to_le16(startstop), + .track_send.dw11 =3D cpu_to_le32((u32)cdq->id) + }; + + return __nvme_submit_sync_cmd(cdq->ctrl->admin_q, &c, NULL, NULL, 0, NVM= E_QID_ANY, 0); +} +#define nvme_start_cdq(cdq) \ + nvme_submit_startstop_cdq_cmd(cdq, NVME_TRSND_CMD_MGMT_LOG_USR_DATA_START) +#define nvme_stop_cdq(cdq) \ + nvme_submit_startstop_cdq_cmd(cdq, NVME_TRSND_CMD_MGMT_LOG_USR_DATA_STOP) + /* Sends a CDQ delete NVMe cmd */ static void nvme_delete_cdq_ctrl(struct cdq_nvme_queue *cdq) { + if (nvme_stop_cdq(cdq)) + WARN_ONCE(1, "Failed to stop CDQ (id: %d)", cdq->id); if (nvme_submit_delete_cdq_cmd(cdq)) WARN_ONCE(1, "Failed delete CDQ (id: %d)", cdq->id); } @@ -337,6 +355,9 @@ int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 e= ntry_nr, const u16 mc_id) if (ret) goto del_xarray; =20 + ret =3D nvme_start_cdq(cdq); + if (ret) + goto del_xarray; return 0; =20 del_xarray: --=20 2.50.1 From nobody Fri Jul 24 21:29:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF3A643303E for ; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; cv=none; b=GiA7PdddAh4tQfw7S7hK0RhGFQPLiqpHqvlKttt6OGctziUUynV48PSpUBm8SXxb/yvITS9igGXZBxIJxF4IvRhMlxxyPTB96WPvn9pkDP6FjKgBNFq+giNN/2CWI+cJYi8KtND4jeChpDlq26O22mtE/Nmszyzj+RrE0jE+UUo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; c=relaxed/simple; bh=TK6sw5sLChDD2H14umlWrgyrMjWvsU5BCPObipz5PyY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=me/cPAGMVsHGfU0HpjN3Plf7XCnXwpDC6GcZg4E0jlPiki2NeZenVW6YXIVmSVlaVFF16sjiTIWGnsnxZSqEgKTD37PsM5xdyxpRhqNXfdhwFEzlvXAKYNDdQWwP0J3YK+WRrI/wglKrOR5cC6Bb9B1g0Op3495KHHXbCwtKvPo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RjxmS4Cq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RjxmS4Cq" Received: by smtp.kernel.org (Postfix) with ESMTPS id BFDE1C4FF7E; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1784891492; bh=TK6sw5sLChDD2H14umlWrgyrMjWvsU5BCPObipz5PyY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=RjxmS4CqdwWiztuwQyQaVTPT+F30dcu5+HHJLTx6NL2eiFaGbR95xKi4ueh4gY9hU +NYCapdlVkMXSJGJtVaKNxL8U+zYmyCcuB6COfyLFkQi+mTXIZAwJRIjbhFXmfsM0V 5jAtj0SIsy5wk2Jbgt7Xf4OAmjlru0CRVTl7Y+VRuiil+aKY+m8SzqCjhE4fsQXkVg tsZqvSPNXoUC9tK7no+OJcyRdMbUzOSzXxT1TQiwyAGpSBCCQyDDnvlqG/2ALj7+y2 HOa/PxS3nadhYm2I/vQwXzC5AW7zfVrzNs0bD1EmVvH2SttUDwB3M4a7GO6WxgGjRf +L7p1nC1XypIQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAEBCC53209; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) From: Joel Granados Date: Fri, 24 Jul 2026 13:11:03 +0200 Subject: [PATCH RFC v2 09/14] nvme: Coordinate CDQ backing mem and CDQ FD Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-jag-cdq-lkml-v2-9-659a7449e849@kernel.org> References: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> In-Reply-To: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jason Gunthorpe Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Joel Granados X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2709; i=joel.granados@kernel.org; h=from:subject:message-id; bh=TK6sw5sLChDD2H14umlWrgyrMjWvsU5BCPObipz5PyY=; b=owJ4nAHsARP+kA0DAAoBupfNUreWQU8ByyZiAGpjSF6CUSvlQHseGsKAdm427Y9qEzQ/VUWdO AR/T7ClZ+UfjokBsgQAAQoAHRYhBK5HCVcl5jElzssnkLqXzVK3lkFPBQJqY0heAAoJELqXzVK3 lkFPx2ML+NaqLg199fgUV85XEXgknviUKpS8KqSiDa4eaaEHsvqN6v4ho19xSXVj0daHjpTZ5Jk p+OMxfcBkhqO1pPq/hdxQqh0t9+Eh+jxtO7XMwcArbCVTT5TU8u3eQd9+a+UBN3y1aMNiiX95V+ ozbooV4le3S0VuMYsHleuBpO/kCqua25ogMJAH10dlYh/1SWqfs3GadVE1pIk2jKbbAOnbVwkN4 nVZclnt5e1edlrwrFRkP25d1SPfPtaOTbQQQzM5AgDWBXLjaiZpWkjyVAJi/dcHdUIVattGsP5V oGcnIm4dnMJU0dXofgsiZMZrZfIEuN/r07KiS9dVNKqkxZdPXCAB6tUAKDrJ8834vIy7NV3e6kX 6kXhYhV5KVJ8MgYfOk65opzSCRwes4LnT342ev+J8uI5zL1XEblM06T7SdbiBYHpDKyxMheTv+o tNSWAjMAT8u9waUneiS6N6+jbDdpuQu3HIzYiPtxz8Zw9tbtRWfU2syBMP8hO7gikstWGUuTPrB A== X-Developer-Key: i=joel.granados@kernel.org; a=openpgp; fpr=F1F8E46D30F0F6C4A45FF4465895FAAC338C6E77 X-Endpoint-Received: by B4 Relay for joel.granados@kernel.org/default with auth_id=239 The user space file descriptor fails if the memory backing the Controller Data Queue (CDQ) is not "valid". This can happen as part of the controller tear-down procedure. Test the valid_mem variable when reading the from the file descriptor. Set on CDQ creation and unset when the CDQ is being deleted on the host side. Signed-off-by: Joel Granados --- drivers/nvme/host/cdq.c | 20 ++++++++++++++++++-- drivers/nvme/host/cdq.h | 3 +++ 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/host/cdq.c b/drivers/nvme/host/cdq.c index 0f01b58a9c3145faae83a81627fa5a93cfc03cf4..7fe08afa0d80db5f3d60993aba0= 5624b5c59b309 100644 --- a/drivers/nvme/host/cdq.c +++ b/drivers/nvme/host/cdq.c @@ -184,6 +184,9 @@ static ssize_t nvme_cdq_fops_read(struct file *filep, c= har __user *buf, if (nbytes > (cdq->size_nbyte)) return -EINVAL; =20 + if (!READ_ONCE(cdq->valid_mem)) + return -EINVAL; + /* CDQ traversal not implemented yet. */ return -EOPNOTSUPP; } @@ -207,7 +210,16 @@ static const struct file_operations cdq_fops =3D { .release =3D nvme_cdq_fops_release, }; =20 -__maybe_unused +/* Should only handle cdq struct and ctrl kref */ +void nvme_free_cdq(struct kref *ref) +{ + struct cdq_nvme_queue *cdq =3D container_of(ref, struct cdq_nvme_queue, r= ef); + + /* Drop the ctrl kref held since creation */ + nvme_put_ctrl(cdq->ctrl); + kfree(cdq); +} + static int nvme_create_cdqfd(struct cdq_nvme_queue *cdq, int *cdq_fdno) { int fdno; @@ -276,8 +288,9 @@ static void nvme_delete_cdq_host(struct cdq_nvme_queue = *cdq) if (xa_erase(&ctrl->cdqs, cdq->id) !=3D cdq) return; =20 - nvme_release_cdq_backing(cdq); + WRITE_ONCE(cdq->valid_mem, false); =20 + nvme_release_cdq_backing(cdq); nvme_cdq_put(cdq); } =20 @@ -342,11 +355,14 @@ int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32= entry_nr, const u16 mc_id) } =20 kref_init(&cdq->ref); + nvme_get_ctrl(cdq->ctrl); =20 ret =3D nvme_submit_create_cdq_cmd(cdq); if (ret) goto del_cdqmem; =20 + WRITE_ONCE(cdq->valid_mem, true); + ret =3D xa_insert(&cdq->ctrl->cdqs, cdq->id, cdq, GFP_KERNEL); if (ret) goto del_cmd; diff --git a/drivers/nvme/host/cdq.h b/drivers/nvme/host/cdq.h index 773f93817d498784450eb0c8c9ab00a95f0657dc..aa9469af0c7ef08b9f8ebf93e46= 8c7f892aee4b1 100644 --- a/drivers/nvme/host/cdq.h +++ b/drivers/nvme/host/cdq.h @@ -42,6 +42,9 @@ struct cdq_nvme_queue { dma_addr_t prp_lists_dma[MAX_NR_CDQ_PRPS]; unsigned int nr_prp_lists; =20 + /* True if mem for chunks and prps is valid */ + bool valid_mem; + /* Manage refs for read FD and controller xarray */ struct kref ref; }; --=20 2.50.1 From nobody Fri Jul 24 21:29:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 02AAA433048 for ; Fri, 24 Jul 2026 11:11:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; cv=none; b=EJyDvMaJjvjbYnjpdruLQXz/iuEXiQ3t15trvvIEVkZpjO6MSgiMSwKTxQOpgPXENZEl74bHsAb8mv1oopDYaH7k6meZ2vrZVyXuYNNBDlwBO8UQ3GTm9ayXrxR3Hlw5EULhPCmDDc4C+VvAJFrIQHumCTJi32yBLUMKZuarq5U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; c=relaxed/simple; bh=DCUAfDxEseZk6ZmmVcrtbVNMa1iqDO1q+qOvgJZt55w=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=V9MvqtCzQdnaDHBGuQYXJ8HUa1j10RzhtIUCdoUh9mqshOLAVBBgXql702TCP9IP6LdvcVy9vjyBnxRli6t7hqiwKjexLIsyechiHwVS6EA+sPaRSjs5ZoNJ8L6buOByykbd0E7l478rovxCJVx+XNEzyITwmfGReli9hIZ2kUE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HI15W8z9; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HI15W8z9" Received: by smtp.kernel.org (Postfix) with ESMTPS id CC5DDC5383A; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1784891492; bh=DCUAfDxEseZk6ZmmVcrtbVNMa1iqDO1q+qOvgJZt55w=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=HI15W8z9H2F/ItIkgJzoiEQ2O5YH3W5nyujbrAmdQE4/I/3MgSjwSAit88cw1tEOr 8FB5FsST/OWB/XH8aNqK76B5kxAw8UiWlKzlp6HtGFyEzesyuReCQ757DSlNwUeKCy mxU61txk+cHVlV5BIqh2Bh+QXroNhUk8ABnlftoQgrfuUSK71wiBQNevdkdbXJxr6X 2+dZzKoqdQMHko6omwh5o6A3adTvpMWDGltSmAwM4kLvTKjuFJwgEclwRFf1RN8zZN aA4KH4Oejma+BYBgVDeqAAqjQaw1W5+LMDq4zO7h8lRE6ps6v/H+IN5wM2x1dUgQn3 avR7lUVxS3Grg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7D22C531F9; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) From: Joel Granados Date: Fri, 24 Jul 2026 13:11:04 +0200 Subject: [PATCH RFC v2 10/14] nvme: Pin a ctrl ref to every CDQ Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-jag-cdq-lkml-v2-10-659a7449e849@kernel.org> References: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> In-Reply-To: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jason Gunthorpe Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Joel Granados X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2007; i=joel.granados@kernel.org; h=from:subject:message-id; bh=DCUAfDxEseZk6ZmmVcrtbVNMa1iqDO1q+qOvgJZt55w=; b=owJ4nAHtARL+kA0DAAoBupfNUreWQU8ByyZiAGpjSF69HTOAZxtOOJS4e3Kf5yJRhPvY7UVMU E3e6K8IECOrbYkBswQAAQoAHRYhBK5HCVcl5jElzssnkLqXzVK3lkFPBQJqY0heAAoJELqXzVK3 lkFPR+oL/iwl4Zmoe0nvHf9iQioW99Y9emfyJqBLzf61u2qRPuNce3fo9FMLlJoIjugVUpeiwQ7 iqfw9mfKGkgl7bEpt2xT2VDwdvly9QbmFd90pevNlhv3jbAK2VH3XOuB4HVm3BhbIvkvpa5OE8o IOz4tjHFTN3y8K0MErx03GXxi1ERGWPtTC72S2egNte+1mztCRcc+CnUN1UDeXweD2nuaKCgdPu bzi9AomZq1JBPlXUDCx3s/BPWSDdZ/t1WvSrFUucagCeIasbSQK1RGD5N97yUxXA6qIZ5kds6aa vs0KwLOzlPSFLxJAPg7M4RRWodw18ecSjBDoJcLeESwZ+RM8ONj8MUFVDQyfrwtFRDQwoyOChSM W6MUNsa/E/dNDGGA9u3QFhYZ55deAt62+nyXtAowbQNt8aWLfR83fjzQblU2DBpNNOKcBtsMg/Y 7JXA6WfjwaDtgE66VSlRmavOJCQ0DLkOSomyUU6iq4XrLFOiZAV+7Ej39vOxyAyexQBdwt/kUh5 8g= X-Developer-Key: i=joel.granados@kernel.org; a=openpgp; fpr=F1F8E46D30F0F6C4A45FF4465895FAAC338C6E77 X-Endpoint-Received: by B4 Relay for joel.granados@kernel.org/default with auth_id=239 Make sure that the migration manager controller pointed by the CDQ does not get freed unless all CDQs are terminated by user space. Signed-off-by: Joel Granados --- drivers/nvme/host/cdq.h | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/drivers/nvme/host/cdq.h b/drivers/nvme/host/cdq.h index aa9469af0c7ef08b9f8ebf93e468c7f892aee4b1..a8c714ab5fada4325463013c2ff= b23a117cd8423 100644 --- a/drivers/nvme/host/cdq.h +++ b/drivers/nvme/host/cdq.h @@ -29,7 +29,7 @@ struct nvme_cdq_chunk { =20 struct cdq_nvme_queue { u16 id; - struct nvme_ctrl *ctrl; + struct nvme_ctrl *ctrl; // this is kref'ed for the life of the CDQ u32 size_nbyte; u16 mc_id; // migratable controller id =20 @@ -49,11 +49,20 @@ struct cdq_nvme_queue { struct kref ref; }; =20 -/* Must not touch cdq->ctrl: Ctrl may have been freed */ -static inline void nvme_free_cdq(struct kref *ref) -{ - kfree(container_of(ref, struct cdq_nvme_queue, ref)); -} +/* + * nvme_free_cdq: + * - Is called with the cdq struct has no more krefs. + * - Will free (cdq) and decrease ctrl kref. + */ +void nvme_free_cdq(struct kref *ref); +/* nvme_cdq_delete: + * - Sends a nvme delete cmd to the controller + * - removes (cdq) from controller xarray + * - frees the backing CDQ mem + * - Calls nvme_free_cdq if there are no more refs + */ +void nvme_delete_cdq(struct cdq_nvme_queue *cdq); +int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 entry_nr, const u16 = mc_id); =20 static inline void nvme_cdq_get(struct cdq_nvme_queue *cdq) { @@ -65,8 +74,6 @@ static inline void nvme_cdq_put(struct cdq_nvme_queue *cd= q) kref_put(&cdq->ref, nvme_free_cdq); } =20 -void nvme_delete_cdq(struct cdq_nvme_queue *cdq); -int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 entry_nr, const u16 = mc_id); void nvme_delete_cdqs_host(struct nvme_ctrl *ctrl); void nvme_free_cdqs(struct nvme_ctrl *ctrl); =20 --=20 2.50.1 From nobody Fri Jul 24 21:29:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 03F4A43304B for ; Fri, 24 Jul 2026 11:11:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; cv=none; b=q1HEC0SXzzgB4Uz9a1D9ZMSbMwPlxK0OgBCzBrK8PdM5c9GCYU0Do/odKg3EHB8Y8lEvM75AIKPEy/NYoDJ0Zs+McVAkBItF9H5g8arIpnKuXZrHG7XrEQoKB/pTVBZhkg3KUeqoT5l0K71TzAOe9KFxXox9RSJ4YhXNGsQoKP8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; c=relaxed/simple; bh=Hl9wyVfReowmH9lQXOTj9hunvFlCPgM9Uuy4AsqCz3I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=BCCZFKzzBUqM6hI8yUob8l9TBLiCBkVauj565UWoeChX9fyp1ECygsrXPo+6sHlrVJ3bhI6GVBJucsw6IzZEcHKoHl6xaH5o/Xqe0/LuMM8OPiUqUsNngFf06kSdp+zezCNQvmK553+qOwICE3V9H0OxkAyuyq7M26EQ34miPEQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=u5PmhNDy; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="u5PmhNDy" Received: by smtp.kernel.org (Postfix) with ESMTPS id DA75AC4FF61; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1784891492; bh=Hl9wyVfReowmH9lQXOTj9hunvFlCPgM9Uuy4AsqCz3I=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=u5PmhNDy62FU/PIUuw/jJ1O/ciA0R62JR06BjM31bi1UKrv1AuDEXUvggnMwTenUj vRQ9wNcbxuy3ewT4eux4w78z6T4FoUiCienbXubjq+yR7LTccO/6DmBsBDBLPQ1DGV 9J920IQmy+J9TFI7OkrfdRZalDiFGGWri+su+vk/2yU6jrGwE/brUa3LE78FIz1Rlt 0wnRJD2Oil62WuIjGRp7tqduiBdohc9tUSAc6wf+nmixyHM1ddXEPRdhiLwxAvZ7cE y7miuoVMNin8UkANKD+7drEK/pZhLDUHMJHtspztnF2MXMd6oP0xqoVJGrkrDiXbn0 MJqvQKacuR6eg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4C06C53215; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) From: Joel Granados Date: Fri, 24 Jul 2026 13:11:05 +0200 Subject: [PATCH RFC v2 11/14] nvme: Implement CDQ traversal on the read path Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-jag-cdq-lkml-v2-11-659a7449e849@kernel.org> References: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> In-Reply-To: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jason Gunthorpe Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Joel Granados X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=4810; i=joel.granados@kernel.org; h=from:subject:message-id; bh=Hl9wyVfReowmH9lQXOTj9hunvFlCPgM9Uuy4AsqCz3I=; b=owJ4nAHtARL+kA0DAAoBupfNUreWQU8ByyZiAGpjSF+n2iXlrweqzc/nxEwB/z4f0UK8MBU+j NCK/boeFLJJjYkBswQAAQoAHRYhBK5HCVcl5jElzssnkLqXzVK3lkFPBQJqY0hfAAoJELqXzVK3 lkFPUhsMAIGYljanPcOwQ6j/9z4kH80xLUK3aACtckcxSC/qwqXnuS1jRQ/wfkFNlGm7mHLaLmG soKtFi6Cqblp/e8GuLUJqKYK5APRpeoSw8jyNNVEGP/9W+ddXMDjeS29pAXKzw3+OLrzk6E70hd 4BG+b5iBTlhF/R5MpKCguGYIHIy7JgGH45natCdyWbEItJcYCsTijxW1vYXNzs0Qtn5T+9d1n8E P8iGo60G2+RoGKfqGCMQ4cAdbRQgWwYmrzOs7w9rQEHgqcMqeu6cUtb8GWCXarUnKZpVqontwfW yB2C6wgfTLS01ethQcQUpleFB/9btqTf9fBJMMyK6MF9A9mSjLQ/GJAq+/nqGK1Gc6lT0K+R8MV av9LZLbMOkwXtnOqt7PJCSPNc+JSkH6EmBqB2Oxqh/GF72ohYT2WDyAbCQ92d6o6JYfM1x9yxEs kRTx2fNi3KWNnH7kkLnqw2+QKGyYA2M0eMgy+P+O2KTTnc2sxgMn3aRTNXZH4jevDHI9I6bkAq7 HY= X-Developer-Key: i=joel.granados@kernel.org; a=openpgp; fpr=F1F8E46D30F0F6C4A45FF4465895FAAC338C6E77 X-Endpoint-Received: by B4 Relay for joel.granados@kernel.org/default with auth_id=239 Replace the initial -EOPNOTSUPP stub with a call to nvme_traversecopy_cdq which copies new controller entries to the user space buffer, forwards and wraps host_head around when it reaches the end of the queue. Signed-off-by: Joel Granados --- drivers/nvme/host/cdq.c | 73 +++++++++++++++++++++++++++++++++++++++++++++= ++-- drivers/nvme/host/cdq.h | 7 +++++ 2 files changed, 78 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/host/cdq.c b/drivers/nvme/host/cdq.c index 7fe08afa0d80db5f3d60993aba05624b5c59b309..36107cf2fe11ff212d5b02dd551= 3fe02e3ae8f81 100644 --- a/drivers/nvme/host/cdq.c +++ b/drivers/nvme/host/cdq.c @@ -5,6 +5,7 @@ =20 #include #include +#include =20 #include "nvme.h" #include "cdq.h" @@ -169,6 +170,75 @@ static inline void nvme_release_cdq_backing(struct cdq= _nvme_queue *cdq) nvme_free_cdqmem_chunks(cdq); } =20 + +static inline void *nvme_get_cdq_entryvaddr(struct cdq_nvme_queue *cdq, + unsigned int entry_idx) +{ + return cdq->chunks[entry_idx / NVME_CDQ_MQ_ENTRY_PER_CHUNK].vaddr + + (entry_idx % NVME_CDQ_MQ_ENTRY_PER_CHUNK) * NVME_CDQ_MQ_ENTRY_NRBYTES; +} + +/* Advance cdq->host_head at most nrbytes and return actual advanced bytes= */ +static size_t nvme_advance_cdq(struct cdq_nvme_queue *cdq, size_t max_nrby= te) +{ + size_t target_nbyte =3D 0; + void *entry; + u8 phase_bit; + u32 cdq_nrentry =3D cdq->size_nbyte / NVME_CDQ_MQ_ENTRY_NRBYTES; + + while (target_nbyte < max_nrbyte) { + entry =3D nvme_get_cdq_entryvaddr(cdq, cdq->host_head); + phase_bit =3D (*(u8 *)(entry + NVME_CDQ_MQ_PHASE_OFFSET) & NVME_CDQ_MQ_P= HASE_MASK); + + if (phase_bit =3D=3D cdq->phase_bit) + break; + + cdq->host_head =3D (cdq->host_head + 1) % cdq_nrentry; + target_nbyte +=3D NVME_CDQ_MQ_ENTRY_NRBYTES; + if (unlikely(cdq->host_head =3D=3D 0)) + cdq->phase_bit =3D ~cdq->phase_bit & NVME_CDQ_MQ_PHASE_MASK; + } + + return target_nbyte; +} + +static ssize_t nvme_traversecopy_cdq(struct cdq_nvme_queue *cdq, size_t ma= x_nrbyte, + void *priv_data) +{ + char __user *to_buf =3D priv_data; + void *from_buf; + u32 init_host_head =3D cdq->host_head; + u8 init_phase_bit =3D cdq->phase_bit; + + size_t target_nbyte =3D nvme_advance_cdq(cdq, max_nrbyte); + size_t copied_nbyte =3D 0, chunks_idx, entry_idx, tx_nbytes; + + if (target_nbyte =3D=3D 0) + goto out; + + for (chunks_idx =3D init_host_head / NVME_CDQ_MQ_ENTRY_PER_CHUNK, + entry_idx =3D init_host_head % NVME_CDQ_MQ_ENTRY_PER_CHUNK; + copied_nbyte < target_nbyte; + chunks_idx =3D (chunks_idx + 1) % cdq->nr_chunks, entry_idx =3D 0) { + from_buf =3D cdq->chunks[chunks_idx].vaddr + + (entry_idx * NVME_CDQ_MQ_ENTRY_NRBYTES); + tx_nbytes =3D min(target_nbyte - copied_nbyte, + NVME_CDQ_CHUNK_SIZE - (entry_idx * NVME_CDQ_MQ_ENTRY_NRBYTES)); + if (copy_to_user(to_buf, from_buf, tx_nbytes)) + goto err_out; + copied_nbyte +=3D tx_nbytes; + to_buf +=3D tx_nbytes; + } + +out: + return copied_nbyte; + +err_out: + cdq->host_head =3D init_host_head; + cdq->phase_bit =3D init_phase_bit; + return -EFAULT; +} + static ssize_t nvme_cdq_fops_read(struct file *filep, char __user *buf, size_t size_nbyte, loff_t *ppos) { @@ -187,8 +257,7 @@ static ssize_t nvme_cdq_fops_read(struct file *filep, c= har __user *buf, if (!READ_ONCE(cdq->valid_mem)) return -EINVAL; =20 - /* CDQ traversal not implemented yet. */ - return -EOPNOTSUPP; + return nvme_traversecopy_cdq(cdq, nbytes, buf); } =20 /* File reference already dropped by the close path, so don't fput() */ diff --git a/drivers/nvme/host/cdq.h b/drivers/nvme/host/cdq.h index a8c714ab5fada4325463013c2ffb23a117cd8423..96f208264c6144d26aa4ffc3485= 4bb16d3f19af9 100644 --- a/drivers/nvme/host/cdq.h +++ b/drivers/nvme/host/cdq.h @@ -9,6 +9,8 @@ #include "nvme.h" =20 #define NVME_CDQ_MQ_ENTRY_NRBYTES 32 +#define NVME_CDQ_MQ_PHASE_MASK 0x1 +#define NVME_CDQ_MQ_PHASE_OFFSET (NVME_CDQ_MQ_ENTRY_NRBYTES - 1) =20 /* * The CDQ backing is a set of coherent DMA chunks expressed in @@ -17,6 +19,7 @@ #define NVME_CDQ_CHUNK_ORDER 2 #define NVME_CDQ_CHUNK_SIZE (PAGE_SIZE << NVME_CDQ_CHUNK_ORDER) #define NVME_CDQ_PAGES_PER_CHUNK (NVME_CDQ_CHUNK_SIZE / NVME_CTRL_PAGE_SIZ= E) +#define NVME_CDQ_MQ_ENTRY_PER_CHUNK (NVME_CDQ_CHUNK_SIZE / NVME_CDQ_MQ_ENT= RY_NRBYTES) =20 /* Max PRP List pages we are willing to chain to describe a discontiguous = CDQ. */ #define MAX_NR_CDQ_PRPS 20 @@ -45,6 +48,10 @@ struct cdq_nvme_queue { /* True if mem for chunks and prps is valid */ bool valid_mem; =20 + /* How far the CDQ was consumed by the host */ + u32 host_head; + u8 phase_bit; + /* Manage refs for read FD and controller xarray */ struct kref ref; }; --=20 2.50.1 From nobody Fri Jul 24 21:29:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 11396433057 for ; Fri, 24 Jul 2026 11:11:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; cv=none; b=prL1KFGkkeKQe3ieLCcr7CG/ZXMenuJ3Pd2uRcH50Fv1DwMueJ7cLBUKX9IlIG9ZwY983S9+TQsBl1l6eVfDIMwu3byBe6qjWWxSaN6ivIur2/0GCD8zhO6u4VpmBB+ktEXrxFz5ufsGbqSLUkb05lt4TDj8z4Kl9N9RSzON3pw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; c=relaxed/simple; bh=p2OZjMW1aIwGnbexC/PZCKu+1prCQGnCZ5ZAUVdjkWM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=DPDD1uWumMbckpc3bWEiDVeVLiPQy4A2rUOao6NToO8VQFkPWlN/VSOqUePKKtHHLnrCoexTM41djXhH28H7QEC/amfYzrVzz0Y4qBetQ/00WyUpCKmYzM4eRg+I8b097d7+63uMCp3IjdldVbBh/6aNdrNjHziGA5UsBECiMPA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=O64zicd/; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="O64zicd/" Received: by smtp.kernel.org (Postfix) with ESMTPS id E742CC5383D; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1784891492; bh=p2OZjMW1aIwGnbexC/PZCKu+1prCQGnCZ5ZAUVdjkWM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=O64zicd/NFG24Y7Y19t1jDelW6jHmJ6eQYnfw4vPdCOCg9US/xULnVfw8zQ3LLwKK vtLyUplDVSDJosDLADoLDlhklvJq4OlZO+rTtmltvbgjCB9iukZ/clfeSbYmLGFlFm ZioSOz2KqQQnM9G3y8ClWwxWGqAlla4ACf4HpHKTaRoJeX4MoBxk9A2nz4o0aXxsaI vVV8AkL/jnufH9nYwlKk+9XHwa7YVvvxu/9LUlo1xjOzSY/gIhO6Wfx0ZV44Ku3ZUa /MsVoSGfWOMomnM2wMCInzUyF/kxg//Yrim4m6QGvlb6Fagvrbc9skT1f6FtsvVriH bqVxtxV+bX1XA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D22FBC531FC; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) From: Joel Granados Date: Fri, 24 Jul 2026 13:11:06 +0200 Subject: [PATCH RFC v2 12/14] nvme: Update the controller with the CDQ set-feature cmd Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-jag-cdq-lkml-v2-12-659a7449e849@kernel.org> References: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> In-Reply-To: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jason Gunthorpe Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Joel Granados X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5127; i=joel.granados@kernel.org; h=from:subject:message-id; bh=p2OZjMW1aIwGnbexC/PZCKu+1prCQGnCZ5ZAUVdjkWM=; b=owJ4nAHtARL+kA0DAAoBupfNUreWQU8ByyZiAGpjSF+IZ0GpbnntyoJIT9QK1csf6j1fLmAe5 qy2sotJya/cdokBswQAAQoAHRYhBK5HCVcl5jElzssnkLqXzVK3lkFPBQJqY0hfAAoJELqXzVK3 lkFPSp4L/2bx6gV2MXX8pyIuo2xg6uRpOW37BzoC4T+J9pMFwKwByXw6/4Sp4vZdME1lhwDOwzd ge22GnNrV4/y+++108ouaTkBnR4O8R66V5cLDkcZie7bw+PRP60UNfeQm6vzSh5hwR5eIUz2S7v oib/aPqaKEwGZN8W9TefSiUwKtkXEAPrU3DdyJP5EmsoCzoNTV2u8p7h2tgS2h473vUXs5V0mlM UAHR7wBb0YLts28vM2MdNkLBJK12oTndVnigpeB9TLTLRRjHTfAA0a/XPuewao4oC/iCYHFOyAd SzeQ1VORlDKxR4DRVheK4fJsQTWc1Yuj6U8muvePBZxiwn7ZVsgze+0K15rG93eM3+rU9qgbmzI a/coe+Auwt82waFN8gLUaw9bERzGVIznwtdQUVw1gA2xCgmRlGjZAcDKSd2O3SwozAW7ckXtbca ITZ8idANAgvOoQ26nlsOIAb8PP4pYmUDUAXOcmUaG/zDWFokvJIexGTBhB+BSbntkKIf8ikH2k8 Ic= X-Developer-Key: i=joel.granados@kernel.org; a=openpgp; fpr=F1F8E46D30F0F6C4A45FF4465895FAAC338C6E77 X-Endpoint-Received: by B4 Relay for joel.granados@kernel.org/default with auth_id=239 After advancing the queue head, send the new head position to the controller using an NVMe set feature command. Avoid unnecessary wait on the read by not waiting for command completion. sf_lock serializes reads that finish before the in-flight set feature has completed; in these cases the cmd will be resent after completion of the previous one. Also, re-send the set feature command right after returning and seeing that the host head has actually moved forward since the last time the command was sent. Signed-off-by: Joel Granados --- drivers/nvme/host/cdq.c | 92 +++++++++++++++++++++++++++++++++++++++++++++= ++++ drivers/nvme/host/cdq.h | 14 ++++++++ 2 files changed, 106 insertions(+) diff --git a/drivers/nvme/host/cdq.c b/drivers/nvme/host/cdq.c index 36107cf2fe11ff212d5b02dd5513fe02e3ae8f81..8e59c16ddf63df4977e1bd869be= 1e3a7ad19b9c5 100644 --- a/drivers/nvme/host/cdq.c +++ b/drivers/nvme/host/cdq.c @@ -202,6 +202,90 @@ static size_t nvme_advance_cdq(struct cdq_nvme_queue *= cdq, size_t max_nrbyte) return target_nbyte; } =20 +static void nvme_submit_sfcmd_cdq(struct cdq_nvme_queue *cdq); + +/* updates cntl_head, exits sf_inflight and re-arms if needed */ +static enum rq_end_io_ret nvme_endio_sfcmd_cdq(struct request *rq, + blk_status_t status, + const struct io_comp_batch *iob) +{ + struct cdq_nvme_queue *cdq =3D rq->end_io_data; + unsigned long flags; + bool rearm =3D false; + + WRITE_ONCE(cdq->cntl_head, cdq->sent_head); + blk_mq_free_request(rq); + + spin_lock_irqsave(&cdq->sf_lock, flags); + cdq->sf_inflight =3D false; + /* cntl_head was just written above, plain read under the lock. */ + if (READ_ONCE(cdq->host_head) !=3D cdq->cntl_head) { + cdq->sf_inflight =3D true; + rearm =3D true; + } + spin_unlock_irqrestore(&cdq->sf_lock, flags); + + /* Re-arm (takes its own ref) before dropping ours, so cdq stays alive. */ + if (rearm) + nvme_submit_sfcmd_cdq(cdq); + + nvme_cdq_put(cdq); + return RQ_END_IO_NONE; +} + +/* Inform ctrl that head as advance by sending a CDQ set-feature command*/ +static void nvme_submit_sfcmd_cdq(struct cdq_nvme_queue *cdq) +{ + struct nvme_command c =3D { }; + struct request *rq; + unsigned long flags; + u32 head =3D READ_ONCE(cdq->host_head); + + c.features.opcode =3D nvme_admin_set_features; + c.features.fid =3D cpu_to_le32(NVME_FEAT_CDQ); + c.features.dword11 =3D cpu_to_le32(cdq->id & NVME_FEAT_CDQ_ID_MASK); + c.features.dword12 =3D cpu_to_le32(head); + + rq =3D blk_mq_alloc_request(cdq->ctrl->admin_q, nvme_req_op(&c), + BLK_MQ_REQ_NOWAIT); + if (IS_ERR(rq)) { + /* + * No admin tag right now and we cannot sleep. Drop the slot; the + * next read() will re-arm. + */ + spin_lock_irqsave(&cdq->sf_lock, flags); + cdq->sf_inflight =3D false; + spin_unlock_irqrestore(&cdq->sf_lock, flags); + return; + } + + cdq->sent_head =3D head; + nvme_init_request(rq, &c); + rq->end_io =3D nvme_endio_sfcmd_cdq; + rq->end_io_data =3D cdq; + + /* Pin cdq for the lifetime of the command */ + nvme_cdq_get(cdq); + blk_execute_rq_nowait(rq, false); +} + +/* submit set feature if none are in-flight */ +static void nvme_kick_cdq(struct cdq_nvme_queue *cdq) +{ + unsigned long flags; + bool submit =3D false; + + spin_lock_irqsave(&cdq->sf_lock, flags); + if (!cdq->sf_inflight) { + cdq->sf_inflight =3D true; + submit =3D true; + } + spin_unlock_irqrestore(&cdq->sf_lock, flags); + + if (submit) + nvme_submit_sfcmd_cdq(cdq); +} + static ssize_t nvme_traversecopy_cdq(struct cdq_nvme_queue *cdq, size_t ma= x_nrbyte, void *priv_data) { @@ -231,6 +315,13 @@ static ssize_t nvme_traversecopy_cdq(struct cdq_nvme_q= ueue *cdq, size_t max_nrby } =20 out: + /* + * host_head advanced past consumed entries; tell the controller its head + * can move up. Decoupled from this read: the set-feature admin + * round-trip must not delay the data path. + */ + if (copied_nbyte) + nvme_kick_cdq(cdq); return copied_nbyte; =20 err_out: @@ -416,6 +507,7 @@ int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 e= ntry_nr, const u16 mc_id) cdq->mc_id =3D mc_id; cdq->ctrl =3D ctrl; cdq->size_nbyte =3D (u32)size_nbyte; + spin_lock_init(&cdq->sf_lock); =20 ret =3D nvme_create_cdq_backing(cdq); if (ret) { diff --git a/drivers/nvme/host/cdq.h b/drivers/nvme/host/cdq.h index 96f208264c6144d26aa4ffc34854bb16d3f19af9..893aeefb2dfc2c25f390e43eb50= 8067b34dce31f 100644 --- a/drivers/nvme/host/cdq.h +++ b/drivers/nvme/host/cdq.h @@ -50,8 +50,22 @@ struct cdq_nvme_queue { =20 /* How far the CDQ was consumed by the host */ u32 host_head; + + /* + * Value sent by the in-flight set-feature cmd + * Differs from cntl_head until set-feature cmd completes + */ + u32 sent_head; + + /* Last acked CDQ head update. Trails host_head.*/ + u32 cntl_head; + u8 phase_bit; =20 + /* sf_* controlls if the feature set cmd is done or is still inflight */ + spinlock_t sf_lock; + bool sf_inflight; + /* Manage refs for read FD and controller xarray */ struct kref ref; }; --=20 2.50.1 From nobody Fri Jul 24 21:29:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1D28A42E8E8 for ; Fri, 24 Jul 2026 11:11:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; cv=none; b=X/XnBwFRoK75j1c6Pbxbo1EI7+vFebgw/fuMhgdg/PT9nZPpK0zDfwOoqshHq7eZwvgsiQuSKmTNu1po/VsmRC4/QV2DaQOtwAfrqGLhlZL76mKyXXN1MaXO4OHSFGrIOcF9yiKgkuY7CFlWuzxAlQzCXTapQR7BGiFDkCHSwCE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; c=relaxed/simple; bh=6pPy4EUhqfqpoPvpDTe9oapGjYosFOX+XzrMx0vub4M=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=eh2DBR3I51UgxgozDhqMKPs4pohHx0UgTrCwFUYqheLobQzFRcX81FFfvfHqqtj7Soqqe7rwlAUO94Tx8AcrVkaPfsqyTr6um1PqUvqxcX89KgrSrpHazqCxl2uRUIu84N8GXikL9qR7oiXy8Opi8GAtkrUw4Zg1gY002cgh6pQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=aFJo/O4N; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="aFJo/O4N" Received: by smtp.kernel.org (Postfix) with ESMTPS id F39D6C55DDE; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1784891493; bh=6pPy4EUhqfqpoPvpDTe9oapGjYosFOX+XzrMx0vub4M=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=aFJo/O4N2SsTZgHMwMZ+lSZsVufyIcGWL4W5zjts+gl1vM4ZErzk5d4pdXHtOW0Un emF/MePL79NQae2Wniv3pxlVDOoBF5qtI6VJ6U25vmryIiIkJaWnnkczoP0uJI+2LH LQGQXRQKxWxpdhzCLAJ/5fZtVZx7MNFBK0pp5mV0jXdkYU35iRCVf64l4pbfecuckv U5tn28DENQRdFybTDqBpfhIdK7kDqvpUa2iQVLp1q+4rPoRorqA1aFjOb3g8bfBWG2 wL/BeDFAlyC1bKry2UxyK9b/DPi0pdyZHxwptaMnasd4pqAxdrTdX/6/IXamlKetFx pQ2GGmyEkBOqw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE628C531FA; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) From: Joel Granados Date: Fri, 24 Jul 2026 13:11:07 +0200 Subject: [PATCH RFC v2 13/14] nvme: Use eventfd for CDQ Tail pointer triggers Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-jag-cdq-lkml-v2-13-659a7449e849@kernel.org> References: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> In-Reply-To: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jason Gunthorpe Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Joel Granados X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=8352; i=joel.granados@kernel.org; h=from:subject:message-id; bh=6pPy4EUhqfqpoPvpDTe9oapGjYosFOX+XzrMx0vub4M=; b=owJ4nAHtARL+kA0DAAoBupfNUreWQU8ByyZiAGpjSGDfdJ5RLXRYxe/CcpVME8VxdzG0J5xor lIAlNP+JtTSPIkBswQAAQoAHRYhBK5HCVcl5jElzssnkLqXzVK3lkFPBQJqY0hgAAoJELqXzVK3 lkFPeqAL/2jnxMo9aPeJBUk707BZFahn7rX9tVW/ymdR/0cxmh9WnLWVWh4GVbn1X8ITKFGbi4d t+KEJm7Om95n0DVRw9Fpukzrb5tn8xf27tTRARleHfEkQ5tU2V3k9LTjIuyDTDkDthFK0w6gH/1 Or7U4ugNqnc+SqsuyN9522ZZ4ITSTWPQvBwwpbBTIb3/NebWt/e+mWuJictWkkxCBELgx7KpIhl 2++T2M/La9QzYdpRAS4XG7cncnuYlljIdxQhrhrgzukihuOjvNx0+iQV7P1FUgOTa3bSUNSZOL/ eDYNSBIdJYJhGZqZENj6vYzBZRo2RRq7w0qL+ibmRGKxZO8BAYfiZnflTVV3YAgApL6NBjCiy/u hDzaniyMsvwIii/oaZjnyVYP2OCEEAUxOgPvnb/qc8CCDnlXPK0YYCrSP+eDjuY7QvS0SEC+5Kp 5wTODTbhVTP6zp+Fhnep2dPNwRSZsZy2AYWeH4bR+WSHXy1UlQZgasNuROzPFJdl2jvUYMXS0r5 tM= X-Developer-Key: i=joel.granados@kernel.org; a=openpgp; fpr=F1F8E46D30F0F6C4A45FF4465895FAAC338C6E77 X-Endpoint-Received: by B4 Relay for joel.granados@kernel.org/default with auth_id=239 When an eventfd file descriptor is passed to the nvme_create_cdq function, add the eventfd context to the cdq struct and increase its reference. Route the AEN one shot event triggered by the controller to the eventfd signal. Set the tail pointer trigger (TPT) as soon as an empty CDQ is encountered. The TPT offset sent to the controller is in pending_tpt which has the last value assigned to it. For the time being it is set to +1 on a "zero" read. Note: if a set-feature is already in flight on an empty read, the arm lingers in pending_tpt until the next submitting kick. Signed-off-by: Joel Granados --- drivers/nvme/host/cdq.c | 81 ++++++++++++++++++++++++++++++++++++++++++++= +--- drivers/nvme/host/cdq.h | 9 +++++- drivers/nvme/host/core.c | 3 +- include/linux/nvme.h | 1 + 4 files changed, 88 insertions(+), 6 deletions(-) diff --git a/drivers/nvme/host/cdq.c b/drivers/nvme/host/cdq.c index 8e59c16ddf63df4977e1bd869be1e3a7ad19b9c5..f4aabf0a8817058a11e6c30dcea= 9554076861327 100644 --- a/drivers/nvme/host/cdq.c +++ b/drivers/nvme/host/cdq.c @@ -4,6 +4,7 @@ */ =20 #include +#include #include #include =20 @@ -240,10 +241,26 @@ static void nvme_submit_sfcmd_cdq(struct cdq_nvme_que= ue *cdq) struct request *rq; unsigned long flags; u32 head =3D READ_ONCE(cdq->host_head); + u32 tpt =3D READ_ONCE(cdq->pending_tpt); + u32 dword11 =3D cdq->id & NVME_FEAT_CDQ_ID_MASK; + u32 cdq_nrentry =3D cdq->size_nbyte / NVME_CDQ_MQ_ENTRY_NRBYTES; =20 c.features.opcode =3D nvme_admin_set_features; c.features.fid =3D cpu_to_le32(NVME_FEAT_CDQ); - c.features.dword11 =3D cpu_to_le32(cdq->id & NVME_FEAT_CDQ_ID_MASK); + + if (unlikely(tpt !=3D 0)) { + /* + * FIXME: There is a small chance that the sent tpt will have + * already been handled by the time this command completes. If + * we find this to be true in the CDQ, we need to send a + * subsequent feature_id to disable the tail pointer trigger. + * section 5.1.25.1.23 nvme base spec. + */ + dword11 |=3D NVME_FEAT_CDQ_ETPT_MASK; + c.features.dword13 =3D cpu_to_le32((head + tpt) % cdq_nrentry); + } + + c.features.dword11 =3D cpu_to_le32(dword11); c.features.dword12 =3D cpu_to_le32(head); =20 rq =3D blk_mq_alloc_request(cdq->ctrl->admin_q, nvme_req_op(&c), @@ -260,6 +277,7 @@ static void nvme_submit_sfcmd_cdq(struct cdq_nvme_queue= *cdq) } =20 cdq->sent_head =3D head; + WRITE_ONCE(cdq->pending_tpt, 0); nvme_init_request(rq, &c); rq->end_io =3D nvme_endio_sfcmd_cdq; rq->end_io_data =3D cdq; @@ -320,8 +338,13 @@ static ssize_t nvme_traversecopy_cdq(struct cdq_nvme_q= ueue *cdq, size_t max_nrby * can move up. Decoupled from this read: the set-feature admin * round-trip must not delay the data path. */ - if (copied_nbyte) + if (copied_nbyte) { nvme_kick_cdq(cdq); + } else if (cdq->tpt_efd_ctx) { + /* Controller will one-shot AEN when more entries are added */ + WRITE_ONCE(cdq->pending_tpt, 1); + nvme_kick_cdq(cdq); + } return copied_nbyte; =20 err_out: @@ -404,6 +427,31 @@ static int nvme_create_cdqfd(struct cdq_nvme_queue *cd= q, int *cdq_fdno) return 0; } =20 +static void nvme_put_cdq_tpt(struct cdq_nvme_queue *cdq) +{ + if (cdq->tpt_efd_ctx) + eventfd_ctx_put(cdq->tpt_efd_ctx); + cdq->tpt_efd_ctx =3D NULL; +} + +static int nvme_get_cdq_tpt(struct cdq_nvme_queue *cdq, const int tpt_fd) +{ + struct eventfd_ctx *tmp; + + if (tpt_fd <=3D 0) + return 0; + + /* put the old one */ + nvme_put_cdq_tpt(cdq); + + tmp =3D eventfd_ctx_fdget(tpt_fd); + if (IS_ERR(tmp)) + return -EINVAL; + + cdq->tpt_efd_ctx =3D tmp; + return 0; +} + static int nvme_submit_delete_cdq_cmd(const struct cdq_nvme_queue *cdq) { struct nvme_command c =3D { @@ -450,6 +498,8 @@ static void nvme_delete_cdq_host(struct cdq_nvme_queue = *cdq) =20 WRITE_ONCE(cdq->valid_mem, false); =20 + nvme_put_cdq_tpt(cdq); + nvme_release_cdq_backing(cdq); nvme_cdq_put(cdq); } @@ -461,6 +511,22 @@ void nvme_delete_cdq(struct cdq_nvme_queue *cdq) } EXPORT_SYMBOL_GPL(nvme_delete_cdq); =20 +int nvme_handle_cdq_aen_tpevent(struct nvme_ctrl *ctrl, u32 event_param) +{ + u16 cdq_id =3D event_param & NVME_FEAT_CDQ_ID_MASK; + struct cdq_nvme_queue *cdq; + + cdq =3D xa_load(&ctrl->cdqs, cdq_id); + if (xa_is_err(cdq)) + return xa_err(cdq); + if (!cdq->tpt_efd_ctx) + return -EINVAL; + + eventfd_signal(cdq->tpt_efd_ctx); + + return 0; +} + static int nvme_submit_create_cdq_cmd(struct cdq_nvme_queue *cdq) { int ret; @@ -490,7 +556,7 @@ static int nvme_submit_create_cdq_cmd(struct cdq_nvme_q= ueue *cdq) return ret; } =20 -int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 entry_nr, const u16 = mc_id) +int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 entry_nr, const u16 = mc_id, const int tpt_fd) { u64 size_nbyte =3D (u64)entry_nr * NVME_CDQ_MQ_ENTRY_NRBYTES; struct cdq_nvme_queue *cdq =3D NULL; @@ -518,10 +584,14 @@ int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32= entry_nr, const u16 mc_id) kref_init(&cdq->ref); nvme_get_ctrl(cdq->ctrl); =20 - ret =3D nvme_submit_create_cdq_cmd(cdq); + ret =3D nvme_get_cdq_tpt(cdq, tpt_fd); if (ret) goto del_cdqmem; =20 + ret =3D nvme_submit_create_cdq_cmd(cdq); + if (ret) + goto put_tpt; + WRITE_ONCE(cdq->valid_mem, true); =20 ret =3D xa_insert(&cdq->ctrl->cdqs, cdq->id, cdq, GFP_KERNEL); @@ -546,6 +616,9 @@ int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 e= ntry_nr, const u16 mc_id) if (nvme_submit_delete_cdq_cmd(cdq)) WARN_ONCE(1, "Failed delete CDQ (id: %d)", cdq->id); =20 +put_tpt: + nvme_put_cdq_tpt(cdq); + del_cdqmem: /* puts the ref acquired by kref_init */ nvme_release_cdq_backing(cdq); diff --git a/drivers/nvme/host/cdq.h b/drivers/nvme/host/cdq.h index 893aeefb2dfc2c25f390e43eb508067b34dce31f..2958c6b92b1312930414b846669= 0aa7c0eb57754 100644 --- a/drivers/nvme/host/cdq.h +++ b/drivers/nvme/host/cdq.h @@ -60,6 +60,9 @@ struct cdq_nvme_queue { /* Last acked CDQ head update. Trails host_head.*/ u32 cntl_head; =20 + /* ETPT offset to arm on next set-feature send, 0 =3D none */ + u32 pending_tpt; + u8 phase_bit; =20 /* sf_* controlls if the feature set cmd is done or is still inflight */ @@ -68,6 +71,9 @@ struct cdq_nvme_queue { =20 /* Manage refs for read FD and controller xarray */ struct kref ref; + + /* Has a value if user setup an event fd for AEN tpt events */ + struct eventfd_ctx *tpt_efd_ctx; }; =20 /* @@ -83,7 +89,7 @@ void nvme_free_cdq(struct kref *ref); * - Calls nvme_free_cdq if there are no more refs */ void nvme_delete_cdq(struct cdq_nvme_queue *cdq); -int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 entry_nr, const u16 = mc_id); +int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 entry_nr, const u16 = mc_id, int tpt_fd); =20 static inline void nvme_cdq_get(struct cdq_nvme_queue *cdq) { @@ -97,5 +103,6 @@ static inline void nvme_cdq_put(struct cdq_nvme_queue *c= dq) =20 void nvme_delete_cdqs_host(struct nvme_ctrl *ctrl); void nvme_free_cdqs(struct nvme_ctrl *ctrl); +int nvme_handle_cdq_aen_tpevent(struct nvme_ctrl *ctrl, u32 event_param); =20 #endif /* _NVME_CDQ_H */ diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index fbaa2e065ab846e4700cddd7ae9f30a52e8a2cd9..bf6daeb0176fa69f1609280a8c7= db89dadf43161 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -4765,7 +4765,8 @@ static bool nvme_handle_aen_oneshot(struct nvme_ctrl = *ctrl, u32 result, u32 even =20 switch (aer_subtype) { case NVME_AER_ONE_SHOT_CDQ_TAIL_PTR: - WARN_ONCE(1, "CDQ Tail Pointer one shot event ignored"); + if (nvme_handle_cdq_aen_tpevent(ctrl, event_param)) + WARN_ONCE(1, "Error handling CDQ AEN oneshot"); break; case NVME_AER_ONE_SHOT_CDQ_FULL: WARN_ONCE(1, "CDQ Full Error one shot event ignored"); diff --git a/include/linux/nvme.h b/include/linux/nvme.h index 7cabedce022eda98fff789596682f257203e3b45..3c100fae1a4bd2a1f43fe326468= 0214194d2d262 100644 --- a/include/linux/nvme.h +++ b/include/linux/nvme.h @@ -1746,6 +1746,7 @@ enum { =20 enum { NVME_FEAT_CDQ_ID_MASK =3D GENMASK(15, 0), + NVME_FEAT_CDQ_ETPT_MASK =3D GENMASK(31, 31), }; =20 struct nvme_supported_log { --=20 2.50.1 From nobody Fri Jul 24 21:29:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 29E7E4334A4 for ; Fri, 24 Jul 2026 11:11:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; cv=none; b=Ujg83ViyW5mU8qu0PvdhTcQ7BPLz9VTAigql1GBomdabCQO8fE4Kpzsr/klmEiZ7sE8idNCVdM/IZqxJLXDLdRfBWZTQJYOz4Lcau2H5hNTX1RikhmTSfo4pALzMfYN99FuiillaPxCoB3Z5Xu4rAIcR//AgXKRFGeU1MfgWbRk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784891493; c=relaxed/simple; bh=d1zkJDIbXXjPHU7K9NaiA+mfZPVXne7PNdWSFhuUYSQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=IGfkjK/TNexXbsCDCIrj+C/QJvxcPU4jd7wqkUWibNxzpap5xDe3dD79ImVsTqY74LSCzRdPez/eMWzWvyuFZvBCSmeXDc9Q9BoIwpnZ4ByFim1DyNdl72P93qkJCWXTt+gAhWn1RIztN8R+pqJrRlynt1RV4GITXs/RPwW1zM0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lmepWaRP; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lmepWaRP" Received: by smtp.kernel.org (Postfix) with ESMTPS id 0CC16C55DE1; Fri, 24 Jul 2026 11:11:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1784891493; bh=d1zkJDIbXXjPHU7K9NaiA+mfZPVXne7PNdWSFhuUYSQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=lmepWaRP4XPTDeonZJ/TvU2SItRT+ZCx6SjRshs+qwP3CP1TdoQbIJQtwVZorhYug lMVOblRNGgnKVUmA/VsHVEuK9jgSkxihAYr+utCgMjossNnZxMg5h0/lYq3Py7wJ2a 8Mp7f/cdYNiiMQQPN0ll4+GVaoj+Yr4GkQkyB2wSpXPcT0ZRdRBYLD+bjGIyelw0WT fdULNuDX5b05gxyMNxfwixrr+Ci3z56gvMQ9FW5Kd6QB4DDWWzTD2h4ej8KcKw+oJ5 6N0uZA5VHpqCqlSgdWniyrv5QNShF4+NbxKOKq4ljlSTn1QtbJuHa8m9mhRhfKolqC h/NKJcxDc64aQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBE52C53209; Fri, 24 Jul 2026 11:11:32 +0000 (UTC) From: Joel Granados Date: Fri, 24 Jul 2026 13:11:08 +0200 Subject: [PATCH RFC v2 14/14] nvme: Add NVME_IOCTL_CDQ to create/delete migration CDQs Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260724-jag-cdq-lkml-v2-14-659a7449e849@kernel.org> References: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> In-Reply-To: <20260724-jag-cdq-lkml-v2-0-659a7449e849@kernel.org> To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jason Gunthorpe Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Joel Granados X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=6659; i=joel.granados@kernel.org; h=from:subject:message-id; bh=d1zkJDIbXXjPHU7K9NaiA+mfZPVXne7PNdWSFhuUYSQ=; b=owJ4nAHtARL+kA0DAAoBupfNUreWQU8ByyZiAGpjSGG4UvFbI8wiFtfmB5bgwXKTltOYOGg8w ForyT5Mgi3g+YkBswQAAQoAHRYhBK5HCVcl5jElzssnkLqXzVK3lkFPBQJqY0hhAAoJELqXzVK3 lkFP+/4L/1ZQCUenB+UJVFl3tXLpC7+xLkOUDCyPOmo4OeclITNiPo2Ffz2+XiKMOaWnD9dKfG/ 3HAZprtR4WJkQvRjoCtnrHF4EOGu55YM/v39CWHfeaEM1v3BemGsYXwhyVU7TMrCBzQ8Nmk8+Nw 2XzHII9ExTo4uQe+NXjoYKtu21ibDjlHftMZXWmCRFw+t9t5zMWQQe+ACTihRl46DTgJSoXoLgO xd56VzVB7tINK7TH9+VJIGAl9GrjvvKCHrITEotkRhl6uBZO2I2c9bsYuRS+qRQ1P0em/Gj67n4 Bg52FP+aPpSBj65FuWjRcdrQzsvBgVnp0ydtkYctM0B6nh3TRHh69hvvfQmzl6BRnHPdBdTnuti unVg6DD1DJT05oOOm0gLti36b2hyCl4tdw19AgCkEEwhp20polFUR5Qu23ZhyJW0/LLVEQyZeoE 0taGb608txEiKKFPjzgRffi9v2ZVGEtPLmrJVvTGr9VL+UcZSTwFw2TlgBqBlSidm3cghAcbSs5 fE= X-Developer-Key: i=joel.granados@kernel.org; a=openpgp; fpr=F1F8E46D30F0F6C4A45FF4465895FAAC338C6E77 X-Endpoint-Received: by B4 Relay for joel.granados@kernel.org/default with auth_id=239 Controller Data Queue (CDQ) creation and deletion executed for the migratable controller id defined in mc_id. When an eventfd file descriptor is passed in tpt_fd, the AEN tail pointer trigger event is routed as an eventfd signal. When tpt_fd is <=3D 0, the routing is turned off. Signed-off-by: Joel Granados --- drivers/nvme/host/cdq.c | 23 ++++++++++++++++++--- drivers/nvme/host/cdq.h | 4 +++- drivers/nvme/host/ioctl.c | 44 +++++++++++++++++++++++++++++++++++++= +++- include/uapi/linux/nvme_ioctl.h | 22 +++++++++++++++++++++ 4 files changed, 88 insertions(+), 5 deletions(-) diff --git a/drivers/nvme/host/cdq.c b/drivers/nvme/host/cdq.c index f4aabf0a8817058a11e6c30dcea9554076861327..ac10c8d48864615f53da232a547= cd5da72630ed1 100644 --- a/drivers/nvme/host/cdq.c +++ b/drivers/nvme/host/cdq.c @@ -511,6 +511,22 @@ void nvme_delete_cdq(struct cdq_nvme_queue *cdq) } EXPORT_SYMBOL_GPL(nvme_delete_cdq); =20 +int nvme_delete_cdq_mcid(struct nvme_ctrl *ctrl, u16 mc_id) +{ + struct cdq_nvme_queue *cdq; + unsigned long i; + + xa_for_each(&ctrl->cdqs, i, cdq) { + if (cdq->mc_id =3D=3D mc_id) { + nvme_delete_cdq(cdq); + return 0; + } + } + + return -ENOENT; +} +EXPORT_SYMBOL_GPL(nvme_delete_cdq_mcid); + int nvme_handle_cdq_aen_tpevent(struct nvme_ctrl *ctrl, u32 event_param) { u16 cdq_id =3D event_param & NVME_FEAT_CDQ_ID_MASK; @@ -556,11 +572,12 @@ static int nvme_submit_create_cdq_cmd(struct cdq_nvme= _queue *cdq) return ret; } =20 -int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 entry_nr, const u16 = mc_id, const int tpt_fd) +int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 entry_nr, const u16 = mc_id, + const int tpt_fd, int *cdq_fd) { u64 size_nbyte =3D (u64)entry_nr * NVME_CDQ_MQ_ENTRY_NRBYTES; struct cdq_nvme_queue *cdq =3D NULL; - int ret, cdq_fd; + int ret; =20 /* The backing size and the CDQSIZE field are both u32 (bytes). */ if (size_nbyte > U32_MAX) @@ -598,7 +615,7 @@ int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 e= ntry_nr, const u16 mc_id, if (ret) goto del_cmd; =20 - ret =3D nvme_create_cdqfd(cdq, &cdq_fd); + ret =3D nvme_create_cdqfd(cdq, cdq_fd); if (ret) goto del_xarray; =20 diff --git a/drivers/nvme/host/cdq.h b/drivers/nvme/host/cdq.h index 2958c6b92b1312930414b8466690aa7c0eb57754..06a6224a6583dd2ec7463c62f93= 81e0edcb6718f 100644 --- a/drivers/nvme/host/cdq.h +++ b/drivers/nvme/host/cdq.h @@ -89,7 +89,9 @@ void nvme_free_cdq(struct kref *ref); * - Calls nvme_free_cdq if there are no more refs */ void nvme_delete_cdq(struct cdq_nvme_queue *cdq); -int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 entry_nr, const u16 = mc_id, int tpt_fd); +int nvme_delete_cdq_mcid(struct nvme_ctrl *ctrl, u16 mc_id); +int nvme_create_cdq(struct nvme_ctrl *ctrl, const u32 entry_nr, const u16 = mc_id, + int tpt_fd, int *cdq_fd); =20 static inline void nvme_cdq_get(struct cdq_nvme_queue *cdq) { diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index 8844bbd395159e544218db413e066cae6c24b2f1..c289a220cb18b39bef4edbd0a8d= aaa4b46f8b4d9 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -8,6 +8,7 @@ #include #include #include "nvme.h" +#include "cdq.h" =20 enum { NVME_IOCTL_VEC =3D (1 << 0), @@ -538,9 +539,46 @@ static int nvme_uring_cmd_io(struct nvme_ctrl *ctrl, s= truct nvme_ns *ns, return ret; } =20 +static int nvme_user_cdq(struct nvme_ctrl *ctrl, void __user *argp) +{ + struct nvme_cdq_cmd cmd; + u32 entry_nr; + int ret, cdq_fd; + + if (copy_from_user(&cmd, argp, sizeof(cmd))) + return -EFAULT; + + /* A zero size requests deletion of the CDQ created for this mc_id. */ + if (cmd.size_nbyte =3D=3D 0) + return nvme_delete_cdq_mcid(ctrl, cmd.mc_id); + + if (cmd.size_nbyte % NVME_CDQ_MQ_ENTRY_NRBYTES) + return -EINVAL; + + entry_nr =3D cmd.size_nbyte / NVME_CDQ_MQ_ENTRY_NRBYTES; + + ret =3D nvme_create_cdq(ctrl, entry_nr, cmd.mc_id, cmd.tpt_fd, &cdq_fd); + if (ret) + goto err_out; + + cmd.cdq_fd =3D cdq_fd; + if (copy_to_user(argp, &cmd, sizeof(cmd))) { + ret =3D -EFAULT; + goto err_out; + } + + return 0; + +err_out: + nvme_delete_cdq_mcid(ctrl, cmd.mc_id); + return ret; + +} + static bool is_ctrl_ioctl(unsigned int cmd) { - if (cmd =3D=3D NVME_IOCTL_ADMIN_CMD || cmd =3D=3D NVME_IOCTL_ADMIN64_CMD) + if (cmd =3D=3D NVME_IOCTL_ADMIN_CMD || cmd =3D=3D NVME_IOCTL_ADMIN64_CMD = || + cmd =3D=3D NVME_IOCTL_CDQ) return true; if (is_sed_ioctl(cmd)) return true; @@ -555,6 +593,8 @@ static int nvme_ctrl_ioctl(struct nvme_ctrl *ctrl, unsi= gned int cmd, return nvme_user_cmd(ctrl, NULL, argp, 0, open_for_write); case NVME_IOCTL_ADMIN64_CMD: return nvme_user_cmd64(ctrl, NULL, argp, 0, open_for_write); + case NVME_IOCTL_CDQ: + return nvme_user_cdq(ctrl, argp); default: return sed_ioctl(ctrl->opal_dev, cmd, argp); } @@ -873,6 +913,8 @@ long nvme_dev_ioctl(struct file *file, unsigned int cmd, return -EACCES; nvme_queue_scan(ctrl); return 0; + case NVME_IOCTL_CDQ: + return nvme_user_cdq(ctrl, argp); default: return -ENOTTY; } diff --git a/include/uapi/linux/nvme_ioctl.h b/include/uapi/linux/nvme_ioct= l.h index 2f76cba6716637baff53e167a6141b68420d75c3..b3fbab48a903bfb10521ca44d08= e61fd2b37bbbd 100644 --- a/include/uapi/linux/nvme_ioctl.h +++ b/include/uapi/linux/nvme_ioctl.h @@ -92,6 +92,27 @@ struct nvme_uring_cmd { __u32 rsvd2; }; =20 +struct nvme_cdq_cmd { + /* Migratable controller id (filled by user space). */ + __u16 mc_id; + + /* + * CDQ size in bytes: (number of entries) * 32. Must be a multiple of + * 32. A size of 0 requests deletion of the CDQ created for mc_id. + */ + __u32 size_nbyte; + + /* + * Tail Pointer Trigger eventfd file descriptor, passed when creating + * the CDQ. A value <=3D 0 means there is no eventfd and AERs are not + * forwarded. + */ + __s32 tpt_fd; + + /* Returned by the kernel: the CDQ file descriptor. */ + __s32 cdq_fd; +}; + #define nvme_admin_cmd nvme_passthru_cmd =20 #define NVME_IOCTL_ID _IO('N', 0x40) @@ -104,6 +125,7 @@ struct nvme_uring_cmd { #define NVME_IOCTL_ADMIN64_CMD _IOWR('N', 0x47, struct nvme_passthru_cmd64) #define NVME_IOCTL_IO64_CMD _IOWR('N', 0x48, struct nvme_passthru_cmd64) #define NVME_IOCTL_IO64_CMD_VEC _IOWR('N', 0x49, struct nvme_passthru_cmd6= 4) +#define NVME_IOCTL_CDQ _IOWR('N', 0x50, struct nvme_cdq_cmd) =20 /* io_uring async commands: */ #define NVME_URING_CMD_IO _IOWR('N', 0x80, struct nvme_uring_cmd) --=20 2.50.1