Documentation/netlink/specs/handshake.yaml | 20 +- Documentation/networking/tls-handshake.rst | 2 + drivers/nvme/host/tcp.c | 150 ++++++++++++-- drivers/nvme/target/tcp.c | 216 ++++++++++++++------- include/net/handshake.h | 12 +- include/uapi/linux/handshake.h | 14 ++ net/handshake/genl.c | 5 +- net/handshake/request.c | 18 ++ net/handshake/tlshd.c | 96 ++++++++- net/sunrpc/svcsock.c | 4 +- net/sunrpc/xprtsock.c | 4 +- 11 files changed, 455 insertions(+), 86 deletions(-)
From: Alistair Francis <alistair.francis@wdc.com> The TLS 1.3 specification allows the TLS client or server to send a KeyUpdate. This is generally used when the sequence is about to overflow or after a certain amount of bytes have been encrypted. The TLS spec doesn't mandate the conditions though, so a KeyUpdate can be sent by the TLS client or server at any time. This includes when running NVMe-OF over a TLS 1.3 connection. As such Linux should be able to handle a KeyUpdate event, as the other NVMe side could initiate a KeyUpdate. Upcoming WD NVMe-TCP hardware controllers implement TLS support and send KeyUpdate requests. This series builds on top of the existing TLS EKEYEXPIRED work, which already detects a KeyUpdate request. We can now pass that information up to the NVMe layer (target and host) and then pass it up to userspace. Userspace (ktls-utils) will need to save the connection state in the keyring during the initial handshake. The kernel then provides the key serial back to userspace when handling a KeyUpdate. Userspace can use this to restore the connection information and then update the keys, this final process is similar to the initial handshake. This series depends on the recvmsg() kernel patch: https://lore.kernel.org/linux-nvme/2cbe1350-0bf5-4487-be33-1d317cb73acf@suse.de/T/#mf56283228ae6c93e37dfbf1c0f6263910217cd80 ktls-utils (tlshd) userspace patches are available at: https://lore.kernel.org/kernel-tls-handshake/CAKmqyKNpFhPtM8HAkgRMKQA8_N7AgoeqaSTe2=0spPnb+Oz2ng@mail.gmail.com/T/#mb277f5c998282666d0f41cc02f4abf516fcc4e9c Link: https://datatracker.ietf.org/doc/html/rfc8446#section-4.6.3 Based-on: 2cbe1350-0bf5-4487-be33-1d317cb73acf@suse.de v4: - Don't stop the keep-alive timer - Remove any support for sending a KeyUpdate - Add tls_client_keyupdate_psk()' and 'tls_server_keyupdate_psk()' - Code cleanups - Change order of patches v3: - Rebase on the recvmsg() workflow patch - Add debugfs support for the host - Don't cancel an ongoing request - Ensure a request is destructed on completion v2: - Change "key-serial" to "session-id" - Fix reported build failures - Drop tls_clear_err() function - Stop keep alive timer during KeyUpdate - Drop handshake message decoding in the NVMe layer Alistair Francis (7): net/handshake: Store the key serial number on completion net/handshake: Define handshake_sk_destruct_req net/handshake: Ensure the request is destructed on completion net/handshake: Support KeyUpdate message types nvme-tcp: Support KeyUpdate nvme-tcp: Allow userspace to trigger a KeyUpdate with debugfs nvmet-tcp: Support KeyUpdate Documentation/netlink/specs/handshake.yaml | 20 +- Documentation/networking/tls-handshake.rst | 2 + drivers/nvme/host/tcp.c | 150 ++++++++++++-- drivers/nvme/target/tcp.c | 216 ++++++++++++++------- include/net/handshake.h | 12 +- include/uapi/linux/handshake.h | 14 ++ net/handshake/genl.c | 5 +- net/handshake/request.c | 18 ++ net/handshake/tlshd.c | 96 ++++++++- net/sunrpc/svcsock.c | 4 +- net/sunrpc/xprtsock.c | 4 +- 11 files changed, 455 insertions(+), 86 deletions(-) -- 2.51.0
On 10/17/25 06:23, alistair23@gmail.com wrote: > From: Alistair Francis <alistair.francis@wdc.com> > > The TLS 1.3 specification allows the TLS client or server to send a > KeyUpdate. This is generally used when the sequence is about to > overflow or after a certain amount of bytes have been encrypted. > > The TLS spec doesn't mandate the conditions though, so a KeyUpdate > can be sent by the TLS client or server at any time. This includes > when running NVMe-OF over a TLS 1.3 connection. > > As such Linux should be able to handle a KeyUpdate event, as the > other NVMe side could initiate a KeyUpdate. > > Upcoming WD NVMe-TCP hardware controllers implement TLS support > and send KeyUpdate requests. > > This series builds on top of the existing TLS EKEYEXPIRED work, > which already detects a KeyUpdate request. We can now pass that > information up to the NVMe layer (target and host) and then pass > it up to userspace. > > Userspace (ktls-utils) will need to save the connection state > in the keyring during the initial handshake. The kernel then > provides the key serial back to userspace when handling a > KeyUpdate. Userspace can use this to restore the connection > information and then update the keys, this final process > is similar to the initial handshake. > I am rather sceptical at the current tlshd implementation. At which place do you update the sending keys? I'm only seeing a call to 'gnutls_handhake_update_receiving_key()'. But I haven't found the matching function 'gnutls_handshake_update_sending_key()' in current gnutls. So how does updating of the sending keys work? Cheers, Hannes -- Dr. Hannes Reinecke Kernel Storage Architect hare@suse.de +49 911 74053 688 SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
On Tue, Oct 21, 2025 at 3:46 AM Hannes Reinecke <hare@suse.de> wrote: > > On 10/17/25 06:23, alistair23@gmail.com wrote: > > From: Alistair Francis <alistair.francis@wdc.com> > > > > The TLS 1.3 specification allows the TLS client or server to send a > > KeyUpdate. This is generally used when the sequence is about to > > overflow or after a certain amount of bytes have been encrypted. > > > > The TLS spec doesn't mandate the conditions though, so a KeyUpdate > > can be sent by the TLS client or server at any time. This includes > > when running NVMe-OF over a TLS 1.3 connection. > > > > As such Linux should be able to handle a KeyUpdate event, as the > > other NVMe side could initiate a KeyUpdate. > > > > Upcoming WD NVMe-TCP hardware controllers implement TLS support > > and send KeyUpdate requests. > > > > This series builds on top of the existing TLS EKEYEXPIRED work, > > which already detects a KeyUpdate request. We can now pass that > > information up to the NVMe layer (target and host) and then pass > > it up to userspace. > > > > Userspace (ktls-utils) will need to save the connection state > > in the keyring during the initial handshake. The kernel then > > provides the key serial back to userspace when handling a > > KeyUpdate. Userspace can use this to restore the connection > > information and then update the keys, this final process > > is similar to the initial handshake. > > > > I am rather sceptical at the current tlshd implementation. > At which place do you update the sending keys? The sending keys are updated as part of gnutls_session_key_update(). gnutls_session_key_update() calls update_sending_key() which updates the sending keys. The idea is that when the sequence number is about to overflow the kernel will request userspace to update the sending keys via the HANDSHAKE_KEY_UPDATE_TYPE_SEND key_update_type. Userspace updates the keys and initiates a KeyUpdate. > I'm only seeing a call to 'gnutls_handhake_update_receiving_key()'. > > But I haven't found the matching function > 'gnutls_handshake_update_sending_key()' in current gnutls. > So how does updating of the sending keys work? gnutls_session_key_update() calls update_sending_key() which updates the sending keys. When updating the sending keys we want to send a KeyUpdate request, which is why it's a different flow. Alistair > > Cheers, > > Hannes > -- > Dr. Hannes Reinecke Kernel Storage Architect > hare@suse.de +49 911 74053 688 > SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg > HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
On 10/21/25 03:01, Alistair Francis wrote: > On Tue, Oct 21, 2025 at 3:46 AM Hannes Reinecke <hare@suse.de> wrote: >> >> On 10/17/25 06:23, alistair23@gmail.com wrote: >>> From: Alistair Francis <alistair.francis@wdc.com> >>> >>> The TLS 1.3 specification allows the TLS client or server to send a >>> KeyUpdate. This is generally used when the sequence is about to >>> overflow or after a certain amount of bytes have been encrypted. >>> >>> The TLS spec doesn't mandate the conditions though, so a KeyUpdate >>> can be sent by the TLS client or server at any time. This includes >>> when running NVMe-OF over a TLS 1.3 connection. >>> >>> As such Linux should be able to handle a KeyUpdate event, as the >>> other NVMe side could initiate a KeyUpdate. >>> >>> Upcoming WD NVMe-TCP hardware controllers implement TLS support >>> and send KeyUpdate requests. >>> >>> This series builds on top of the existing TLS EKEYEXPIRED work, >>> which already detects a KeyUpdate request. We can now pass that >>> information up to the NVMe layer (target and host) and then pass >>> it up to userspace. >>> >>> Userspace (ktls-utils) will need to save the connection state >>> in the keyring during the initial handshake. The kernel then >>> provides the key serial back to userspace when handling a >>> KeyUpdate. Userspace can use this to restore the connection >>> information and then update the keys, this final process >>> is similar to the initial handshake. >>> >> >> I am rather sceptical at the current tlshd implementation. >> At which place do you update the sending keys? > > The sending keys are updated as part of gnutls_session_key_update(). > > gnutls_session_key_update() calls update_sending_key() which updates > the sending keys. > > The idea is that when the sequence number is about to overflow the > kernel will request userspace to update the sending keys via the > HANDSHAKE_KEY_UPDATE_TYPE_SEND key_update_type. Userspace updates the > keys and initiates a KeyUpdate. > That's also what the spec says. But in order to do that we would need to get hold of the sequence number, which currently is internal to gnutls. Can we extract it from the session information? And can we display it in sysfs, to give users information whether a KeyUpdate had happened? Cheers, Hannes -- Dr. Hannes Reinecke Kernel Storage Architect hare@suse.de +49 911 74053 688 SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
On Tue, Oct 21, 2025 at 4:40 PM Hannes Reinecke <hare@suse.de> wrote: > > On 10/21/25 03:01, Alistair Francis wrote: > > On Tue, Oct 21, 2025 at 3:46 AM Hannes Reinecke <hare@suse.de> wrote: > >> > >> On 10/17/25 06:23, alistair23@gmail.com wrote: > >>> From: Alistair Francis <alistair.francis@wdc.com> > >>> > >>> The TLS 1.3 specification allows the TLS client or server to send a > >>> KeyUpdate. This is generally used when the sequence is about to > >>> overflow or after a certain amount of bytes have been encrypted. > >>> > >>> The TLS spec doesn't mandate the conditions though, so a KeyUpdate > >>> can be sent by the TLS client or server at any time. This includes > >>> when running NVMe-OF over a TLS 1.3 connection. > >>> > >>> As such Linux should be able to handle a KeyUpdate event, as the > >>> other NVMe side could initiate a KeyUpdate. > >>> > >>> Upcoming WD NVMe-TCP hardware controllers implement TLS support > >>> and send KeyUpdate requests. > >>> > >>> This series builds on top of the existing TLS EKEYEXPIRED work, > >>> which already detects a KeyUpdate request. We can now pass that > >>> information up to the NVMe layer (target and host) and then pass > >>> it up to userspace. > >>> > >>> Userspace (ktls-utils) will need to save the connection state > >>> in the keyring during the initial handshake. The kernel then > >>> provides the key serial back to userspace when handling a > >>> KeyUpdate. Userspace can use this to restore the connection > >>> information and then update the keys, this final process > >>> is similar to the initial handshake. > >>> > >> > >> I am rather sceptical at the current tlshd implementation. > >> At which place do you update the sending keys? > > > > The sending keys are updated as part of gnutls_session_key_update(). > > > > gnutls_session_key_update() calls update_sending_key() which updates > > the sending keys. > > > > The idea is that when the sequence number is about to overflow the > > kernel will request userspace to update the sending keys via the > > HANDSHAKE_KEY_UPDATE_TYPE_SEND key_update_type. Userspace updates the > > keys and initiates a KeyUpdate. > > > That's also what the spec says. > But in order to do that we would need to get hold of the sequence > number, which currently is internal to gnutls. The sequence number is in the kernel. After the handshake the kernel takes over the TLS connection, so it's up to the kernel to detect a sequence number overflow. My sending KeyUpdate patches do this, but they aren't included in this series. > Can we extract it from the session information? gnutls can export the sequence number, but as above it doesn't actually know the correct value (after the handshake). > And can we display it in sysfs, to give users information > whether a KeyUpdate had happened? I don't think that's a good idea. Writing the sequence number to sysfs seems like extra overhead in the TLS fast path. On top of that I don't see why userspace cares or what it can do with the number Alistair > > Cheers, > > Hannes > -- > Dr. Hannes Reinecke Kernel Storage Architect > hare@suse.de +49 911 74053 688 > SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg > HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
© 2016 - 2026 Red Hat, Inc.