From nobody Sat Oct 4 15:52:57 2025 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8B132221714 for ; Fri, 3 Oct 2025 04:32:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759465972; cv=none; b=sUSAtakmuYwgOo3dfYxFDhTHknL5nJxDgrHsPpkf1Rs57qIdy5XCXsyvCk62a8ICqD6GBC0rAEV4FlqiE5XUjJ9ngIZNTD5acF5Oha7mQoC9q7L+zyrN75T2E6hb4UpcV6nVwPdzT6Qx3i36RyHQw2QtO8TzZg0eaVPr7A/Kfhc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759465972; c=relaxed/simple; bh=tq8hWUi9WFbvZCSJuGUlfmGb2RK2ZeIth2EWUddFyYo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Q+Ke8z1qqz3InK2MhXDGXry59C/eL5RXOVbF9oir3F8oON9xkvX+MAl3Ou0hm9lNF2+n/bmP+SRlFVw879mqUIcAdNj+fz2Z1t+r5+k8RPq5dLeN7e+rgIupmN5GHmCaMfCf4oE2RpLixHmYehxYJHfDDPMHjjAjqBJGeHuVmUE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=PU6sEbzS; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PU6sEbzS" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-27c369f898fso18966875ad.3 for ; Thu, 02 Oct 2025 21:32:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759465969; x=1760070769; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p/RhfwOg9G04zj7blr/LVSDCaR4a6pcdo5HXaRarZaQ=; b=PU6sEbzSwxWd7ic9h+HnsfdLmQOPUayx7qnzML9Eyk1HOPdKYMWnbildjcbxrEGM/A u/KKNKehqokS+cbO9Xkw9H20ommbNDQr+MlDTEJQw27aGzQdJKWNtHCU9Dox5D+OnESh 8IQe3C0x7/mEESHS1lI64c9V3eTUahQJ0HOrn/nVINvXWrUHe3M+4W4C0r6mO6eRTXUj AJPcMeS4B13yK7zXYydlfqDUb211wn5jK4xpcJ5r5QFm5Tw4qNJJpp8/g8Got3iHUZ/e tvXswUVwU6Qp6Q/E1r6wrYbpJ1+WxoUaqSUTDywJUrLhi7exfQSHdwrGYFp07d69Rvlx j09Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759465969; x=1760070769; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p/RhfwOg9G04zj7blr/LVSDCaR4a6pcdo5HXaRarZaQ=; b=ZB7l8nX3xjaVFud3QmikmkrEuYYmkYo4JJvNvKJa8Gve6wgywsFwP0ytxe3d29L4BC 7U+BBDK25AHJZv7z4d4iMemJvsYSURFyrkiRcsJTVkpGOv14jBjn7s4FQz++zpa2MNk9 5V30NaWuvq/5z8oQhxxe4VcUotCo+LMQCPJX91m2mYFVXJ5Lq2r1H64vHAOVWuUYbSUR Rt8xyvyZkMWcD0EdEh+1BGvkhYR2e+8DJV+VHFsFcRV3m1dg4nla8PIS4Ae58HqK188H 2ThRoKHv1eod7Qwg1M0c82OcSwvVe08JFyZqNafsvHjAUGGDti9UPn3LduYdAXug+xU7 SGCw== X-Forwarded-Encrypted: i=1; AJvYcCUiKekozHv66Tl7FgdxL1IQ2x/G3i/WzGTACgm53X+J/LJFwoJMKUH/4A0+FAi6qkGXe1VjGYAJ0hVZLQ8=@vger.kernel.org X-Gm-Message-State: AOJu0YzZkqAFKP7kzWpqj3O3mCXtNdYHsMbqZAZGMKBV15yQzefCtWBm BBQohCiK1KBalacwqvdW3xiCfOTgijaQ3yW3gyWCXha1DVbZc2Y/sxqF X-Gm-Gg: ASbGncus8en7csrLK9uMX/Bmbd9b4o5+e1hjYqI/e2VQ2MDhaSffdYl/SDFklbxoVho 7wGD5k31vExhevv+67ThDDhPws/1r69cDvDiJx9H+XnVMCUJXabvmKILcKUhf71yN3DlJTieJbJ t58ri+U+kVQ5T+xJsAG5KjkrUG65KunKhZFGsPSrzFpbwcRMvjXiYABB/PBiRDujF4jBUKpT0D0 aM7HAUGn8H/2r1NsslnS4kmKlwOMdUOIAJM84ws30RSnmXS7IXTIXE+DGZ+uQT7BJWUuDyobF0B HoNZYCGw0C8BB5HfdHUOg84Q0bJXbSDkc1RehHFzGUyzNt3cqXXwlQWMmTJNAL0LXPK3le9Ej58 /N61/+59b61MZcJflIB9VPg1xs8rKaGn8M/i1mCyV6d39UCmvAovn6T24OU3M0+ioJop15RmWgt q9OnhWiMjMcR83GsoyFa9MDx6rqqmBOdyAkvByLSI9p6cHR3D8d2Cz X-Google-Smtp-Source: AGHT+IHqijcQ2mML6mQfax0bd7bg5pqc9YIkhbyYMavM8Tyb3Jp75ZIWGmYxOuPUT9g3dT0PGrahEA== X-Received: by 2002:a17:903:fa6:b0:25f:45d9:6592 with SMTP id d9443c01a7336-28e9a65c0e9mr15641845ad.48.1759465969449; Thu, 02 Oct 2025 21:32:49 -0700 (PDT) Received: from toolbx.alistair23.me (2403-580b-97e8-0-82ce-f179-8a79-69f4.ip6.aussiebb.net. [2403:580b:97e8:0:82ce:f179:8a79:69f4]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-339a701c457sm6528233a91.23.2025.10.02.21.32.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Oct 2025 21:32:48 -0700 (PDT) From: alistair23@gmail.com X-Google-Original-From: alistair.francis@wdc.com To: chuck.lever@oracle.com, hare@kernel.org, kernel-tls-handshake@lists.linux.dev, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-nfs@vger.kernel.org Cc: kbusch@kernel.org, axboe@kernel.dk, hch@lst.de, sagi@grimberg.me, kch@nvidia.com, hare@suse.de, alistair23@gmail.com, Alistair Francis Subject: [PATCH v3 7/8] nvmet-tcp: Support KeyUpdate Date: Fri, 3 Oct 2025 14:31:38 +1000 Message-ID: <20251003043140.1341958-8-alistair.francis@wdc.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251003043140.1341958-1-alistair.francis@wdc.com> References: <20251003043140.1341958-1-alistair.francis@wdc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alistair Francis If the nvmet_tcp_try_recv() function return EKEYEXPIRED or if we receive a KeyUpdate handshake type then the underlying TLS keys need to be updated. If the NVMe Host (TLS client) initiates a KeyUpdate this patch will allow the NVMe layer to process the KeyUpdate request and forward the request to userspace. Userspace must then update the key to keep the connection alive. This patch allows us to handle the NVMe host sending a KeyUpdate request without aborting the connection. At this time we don't support initiating a KeyUpdate. Link: https://datatracker.ietf.org/doc/html/rfc8446#section-4.6.3 Signed-off-by: Alistair Francis --- v3: - Use a write lock for sk_user_data - Fix build with CONFIG_NVME_TARGET_TCP_TLS disabled - Remove unused variable v2: - Use a helper function for KeyUpdates - Ensure keep alive timer is stopped - Wait for TLS KeyUpdate to complete drivers/nvme/target/tcp.c | 90 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 85 insertions(+), 5 deletions(-) diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c index bee0355195f5..fd59dd3ca632 100644 --- a/drivers/nvme/target/tcp.c +++ b/drivers/nvme/target/tcp.c @@ -175,6 +175,7 @@ struct nvmet_tcp_queue { =20 /* TLS state */ key_serial_t tls_pskid; + key_serial_t user_session_id; struct delayed_work tls_handshake_tmo_work; =20 unsigned long poll_end; @@ -186,6 +187,8 @@ struct nvmet_tcp_queue { struct sockaddr_storage sockaddr_peer; struct work_struct release_work; =20 + struct completion tls_complete; + int idx; struct list_head queue_list; =20 @@ -836,6 +839,11 @@ static int nvmet_tcp_try_send_one(struct nvmet_tcp_que= ue *queue, return 1; } =20 +#ifdef CONFIG_NVME_TARGET_TCP_TLS +static int nvmet_tcp_try_peek_pdu(struct nvmet_tcp_queue *queue); +static void nvmet_tcp_tls_handshake_timeout(struct work_struct *w); +#endif + static int nvmet_tcp_try_send(struct nvmet_tcp_queue *queue, int budget, int *sends) { @@ -844,6 +852,13 @@ static int nvmet_tcp_try_send(struct nvmet_tcp_queue *= queue, for (i =3D 0; i < budget; i++) { ret =3D nvmet_tcp_try_send_one(queue, i =3D=3D budget - 1); if (unlikely(ret < 0)) { +#ifdef CONFIG_NVME_TARGET_TCP_TLS + if (ret =3D=3D -EKEYEXPIRED && + queue->state !=3D NVMET_TCP_Q_DISCONNECTING && + queue->state !=3D NVMET_TCP_Q_TLS_HANDSHAKE) { + goto done; + } +#endif nvmet_tcp_socket_error(queue, ret); goto done; } else if (ret =3D=3D 0) { @@ -1110,6 +1125,45 @@ static inline bool nvmet_tcp_pdu_valid(u8 type) return false; } =20 +#ifdef CONFIG_NVME_TARGET_TCP_TLS +static int update_tls_keys(struct nvmet_tcp_queue *queue) +{ + int ret; + + cancel_work(&queue->io_work); + queue->state =3D NVMET_TCP_Q_TLS_HANDSHAKE; + + /* Restore the default callbacks before starting upcall */ + write_lock_bh(&queue->sock->sk->sk_callback_lock); + queue->sock->sk->sk_data_ready =3D queue->data_ready; + queue->sock->sk->sk_state_change =3D queue->state_change; + queue->sock->sk->sk_write_space =3D queue->write_space; + queue->sock->sk->sk_user_data =3D NULL; + write_unlock_bh(&queue->sock->sk->sk_callback_lock); + + nvmet_stop_keep_alive_timer(queue->nvme_sq.ctrl); + + INIT_DELAYED_WORK(&queue->tls_handshake_tmo_work, + nvmet_tcp_tls_handshake_timeout); + + ret =3D nvmet_tcp_tls_handshake(queue, HANDSHAKE_KEY_UPDATE_TYPE_RECEIVED= ); + + if (ret < 0) + return ret; + + ret =3D wait_for_completion_interruptible_timeout(&queue->tls_complete, 1= 0 * HZ); + + if (ret <=3D 0) { + tls_handshake_cancel(queue->sock->sk); + return ret; + } + + queue->state =3D NVMET_TCP_Q_LIVE; + + return ret; +} +#endif + static int nvmet_tcp_tls_record_ok(struct nvmet_tcp_queue *queue, struct msghdr *msg, char *cbuf) { @@ -1135,6 +1189,9 @@ static int nvmet_tcp_tls_record_ok(struct nvmet_tcp_q= ueue *queue, ret =3D -EAGAIN; } break; + case TLS_RECORD_TYPE_HANDSHAKE: + ret =3D -EAGAIN; + break; default: /* discard this record type */ pr_err("queue %d: TLS record %d unhandled\n", @@ -1344,6 +1401,13 @@ static int nvmet_tcp_try_recv(struct nvmet_tcp_queue= *queue, for (i =3D 0; i < budget; i++) { ret =3D nvmet_tcp_try_recv_one(queue); if (unlikely(ret < 0)) { +#ifdef CONFIG_NVME_TARGET_TCP_TLS + if (ret =3D=3D -EKEYEXPIRED && + queue->state !=3D NVMET_TCP_Q_DISCONNECTING && + queue->state !=3D NVMET_TCP_Q_TLS_HANDSHAKE) { + goto done; + } +#endif nvmet_tcp_socket_error(queue, ret); goto done; } else if (ret =3D=3D 0) { @@ -1408,14 +1472,26 @@ static void nvmet_tcp_io_work(struct work_struct *w) ret =3D nvmet_tcp_try_recv(queue, NVMET_TCP_RECV_BUDGET, &ops); if (ret > 0) pending =3D true; - else if (ret < 0) - return; + else if (ret < 0) { +#ifdef CONFIG_NVME_TARGET_TCP_TLS + if (ret =3D=3D -EKEYEXPIRED) + update_tls_keys(queue); + else +#endif + return; + } =20 ret =3D nvmet_tcp_try_send(queue, NVMET_TCP_SEND_BUDGET, &ops); if (ret > 0) pending =3D true; - else if (ret < 0) - return; + else if (ret < 0) { +#ifdef CONFIG_NVME_TARGET_TCP_TLS + if (ret =3D=3D -EKEYEXPIRED) + update_tls_keys(queue); + else +#endif + return; + } =20 } while (pending && ops < NVMET_TCP_IO_WORK_BUDGET); =20 @@ -1798,6 +1874,7 @@ static void nvmet_tcp_tls_handshake_done(void *data, = int status, } if (!status) { queue->tls_pskid =3D peerid; + queue->user_session_id =3D user_session_id; queue->state =3D NVMET_TCP_Q_CONNECTING; } else queue->state =3D NVMET_TCP_Q_FAILED; @@ -1813,6 +1890,7 @@ static void nvmet_tcp_tls_handshake_done(void *data, = int status, else nvmet_tcp_set_queue_sock(queue); kref_put(&queue->kref, nvmet_tcp_release_queue); + complete(&queue->tls_complete); } =20 static void nvmet_tcp_tls_handshake_timeout(struct work_struct *w) @@ -1843,7 +1921,7 @@ static int nvmet_tcp_tls_handshake(struct nvmet_tcp_q= ueue *queue, int ret =3D -EOPNOTSUPP; struct tls_handshake_args args; =20 - if (queue->state !=3D NVMET_TCP_Q_TLS_HANDSHAKE) { + if (queue->state !=3D NVMET_TCP_Q_TLS_HANDSHAKE && !keyupdate) { pr_warn("cannot start TLS in state %d\n", queue->state); return -EINVAL; } @@ -1856,7 +1934,9 @@ static int nvmet_tcp_tls_handshake(struct nvmet_tcp_q= ueue *queue, args.ta_data =3D queue; args.ta_keyring =3D key_serial(queue->port->nport->keyring); args.ta_timeout_ms =3D tls_handshake_timeout * 1000; + args.user_session_id =3D queue->user_session_id; =20 + init_completion(&queue->tls_complete); ret =3D tls_server_hello_psk(&args, GFP_KERNEL, keyupdate); if (ret) { kref_put(&queue->kref, nvmet_tcp_release_queue); --=20 2.51.0