From nobody Sat Feb 7 18:16:08 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4507320311; Sun, 11 May 2025 00:41:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924090; cv=none; b=VkqsbpRSWUpEVPCxrjlvi02wUjyq28eqPcoUl9VJ5oACaOaFl6GxZ8voZN1l0FJpntn0rFkKUu1eLZC1TmiTEkCI6SbGomSfaq2TzmPPpx4IF+5WE7/0iG5hLWA95sbykmln6twcVBOZ/xOASe3vUPnXUCZxGCLmRAuZsQRMqaU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924090; c=relaxed/simple; bh=y+V5sqoQHeK8LApRzwTSNGThMks43WhP9Yn3FOEf8Eo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=b231eocWF+fW1hVvAEodyPi6jT3JOx1gMpnCStHQ6dq5gx3uuEUpmKbUP9PnNdEC9AlJM6Zk9/Bpy6bcDQtC/gN+i+0FmxixCuvw1yhha2pI5It7Jz1NXVs+zZqcEpxeKQq4vlHsiyGxzSI2uNuf+NoTC1Wg8kJp8LjqcoVxQkI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=l0y1n+I7; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="l0y1n+I7" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A479BC4CEE7; Sun, 11 May 2025 00:41:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746924089; bh=y+V5sqoQHeK8LApRzwTSNGThMks43WhP9Yn3FOEf8Eo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=l0y1n+I7LS4gxWUEW9UGJENJf8q05kPt14AA+vOjXvPSld21FBsxDTrngmhW6ds9N tNT/l2LtTU/Xb2/E1IMyJa1O1Ik3Tq22x9etKzp4elyOyULegjzrqxh8LqsSveZmoE 5aJWy990axz8FFX4bpgK333gBTiL63N4H+JBq2aP9gLpKve61soSVwMvbQBHeHilEe E06Q51UbJiuMdrNEc2Ccg2Y5ES8nPgRT2hg7KiZViZ1OS4qbHLPIDxNR0xzOz9qssQ x4w8YtgvS0wTrAiRvj8S5qUevItSur4VK5ulS/MjwAi195cZHdwXXxlyElSlFI6JOQ IOI1ZpNS9qCaw== From: Eric Biggers To: netdev@vger.kernel.org Cc: linux-nvme@lists.infradead.org, linux-sctp@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, Daniel Borkmann , Marcelo Ricardo Leitner , Sagi Grimberg , Ard Biesheuvel Subject: [PATCH net-next 01/10] net: introduce CONFIG_NET_CRC32C Date: Sat, 10 May 2025 17:41:01 -0700 Message-ID: <20250511004110.145171-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250511004110.145171-1-ebiggers@kernel.org> References: <20250511004110.145171-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Add a hidden kconfig symbol NET_CRC32C that will group together the functions that calculate CRC32C checksums of packets, so that these don't have to be built into NET-enabled kernels that don't need them. Make skb_crc32c_csum_help() (which is called only when IP_SCTP is enabled) conditional on this symbol, and make IP_SCTP select it. Signed-off-by: Eric Biggers --- net/Kconfig | 4 ++++ net/core/dev.c | 2 ++ net/sctp/Kconfig | 1 + 3 files changed, 7 insertions(+) diff --git a/net/Kconfig b/net/Kconfig index 202cc595e5e6..5b71a52987d3 100644 --- a/net/Kconfig +++ b/net/Kconfig @@ -73,10 +73,14 @@ config NET_DEVMEM depends on PAGE_POOL =20 config NET_SHAPER bool =20 +config NET_CRC32C + bool + select CRC32 + menu "Networking options" =20 source "net/packet/Kconfig" source "net/unix/Kconfig" source "net/tls/Kconfig" diff --git a/net/core/dev.c b/net/core/dev.c index c9013632296f..51606b2d17bb 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -3592,10 +3592,11 @@ int skb_checksum_help(struct sk_buff *skb) out: return ret; } EXPORT_SYMBOL(skb_checksum_help); =20 +#ifdef CONFIG_NET_CRC32C int skb_crc32c_csum_help(struct sk_buff *skb) { __le32 crc32c_csum; int ret =3D 0, offset, start; =20 @@ -3631,10 +3632,11 @@ int skb_crc32c_csum_help(struct sk_buff *skb) skb_reset_csum_not_inet(skb); out: return ret; } EXPORT_SYMBOL(skb_crc32c_csum_help); +#endif /* CONFIG_NET_CRC32C */ =20 __be16 skb_network_protocol(struct sk_buff *skb, int *depth) { __be16 type =3D skb->protocol; =20 diff --git a/net/sctp/Kconfig b/net/sctp/Kconfig index d18a72df3654..3669ba351856 100644 --- a/net/sctp/Kconfig +++ b/net/sctp/Kconfig @@ -9,10 +9,11 @@ menuconfig IP_SCTP depends on IPV6 || IPV6=3Dn select CRC32 select CRYPTO select CRYPTO_HMAC select CRYPTO_SHA1 + select NET_CRC32C select NET_UDP_TUNNEL help Stream Control Transmission Protocol =20 From RFC 2960 . base-commit: 0b28182c73a3d013bcabbb890dc1070a8388f55a --=20 2.49.0 From nobody Sat Feb 7 18:16:08 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8166761FFE; Sun, 11 May 2025 00:41:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924091; cv=none; b=QRTWgjbWYwZdiQ5aIMk2Iy3ah3MsBoKw/YGYx4ikD1C4ZnvVLklN7IfOo9EEP9J+izSyvzZ8WI0vxLBx0B5BORZUlwjgPz+sxS24G/gbjZFHwyVruG6pPt8KIk62/IOJbHOt3jxMJitgW4ptvAJDtIXxchKZLQW4ExY4ZgTFEYU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924091; c=relaxed/simple; bh=Er5gc2hUENA4gCdJhtwjHn21KMcLdvxvQg5p2KS53YU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qw7Afkv+ULMIzIW+WzmNkmm3Fm9Em6RrDIpBLaOBvNiG2BMPyMclUnj2Px3hUkvxe9kIdk9de0KZfndtyz7fdS8ENttvKkjsguCNCFmW73MX5fLFv6iTd73wmf+Znmd51fMDjYjqyhM0jjstX+GAi+YuQD59TgwgZYvflMEOwlo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=tscoOcsq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="tscoOcsq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E7173C4CEF2; Sun, 11 May 2025 00:41:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746924090; bh=Er5gc2hUENA4gCdJhtwjHn21KMcLdvxvQg5p2KS53YU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tscoOcsq7YtK4+n3cVHEr3de+HfaNOpYztWa9Umb5OukdiQO7MTKvPIKjnN3YiL8G Truwd0tO3gJoUvEuYwi2LlghvslZ4Gcuykja8KiO6PmNBLguFwNrZRfagm/7Ug5eJY 0zfccwrKcbUVJf3A32FZHOFMJUiFs0EIuKOYRX0X1nDP79Y33AjhOkWVTWj9MQIH21 vkIpbidhbGefydXm8iPEJEUgnwAFMJKNAijRvfCxh8brkpqOLXyx2a7yYKx4Qyo+q5 eS+IEz1otWLeACSjUsfy6vKdtFAuq8ZjW38lfLWMpZcNOKLIIqlpFWQ9B465Lvte5V xf/NuyBFuaUWA== From: Eric Biggers To: netdev@vger.kernel.org Cc: linux-nvme@lists.infradead.org, linux-sctp@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, Daniel Borkmann , Marcelo Ricardo Leitner , Sagi Grimberg , Ard Biesheuvel Subject: [PATCH net-next 02/10] net: add skb_crc32c() Date: Sat, 10 May 2025 17:41:02 -0700 Message-ID: <20250511004110.145171-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250511004110.145171-1-ebiggers@kernel.org> References: <20250511004110.145171-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Add skb_crc32c(), which calculates the CRC32C of a sk_buff. It will replace __skb_checksum(), which unnecessarily supports arbitrary checksums. Compared to __skb_checksum(), skb_crc32c(): - Uses the correct type for CRC32C values (u32, not __wsum). - Does not require the caller to provide a skb_checksum_ops struct. - Is faster because it does not use indirect calls and does not use the very slow crc32c_combine(). According to commit 2817a336d4d5 ("net: skb_checksum: allow custom update/combine for walking skb") which added __skb_checksum(), the original motivation for the abstraction layer was to avoid code duplication for CRC32C and other checksums in the future. However: - No additional checksums showed up after CRC32C. __skb_checksum() is only used with the "regular" net checksum and CRC32C. - Indirect calls are expensive. Commit 2544af0344ba ("net: avoid indirect calls in L4 checksum calculation") worked around this using the INDIRECT_CALL_1 macro. But that only avoided the indirect call for the net checksum, and at the cost of an extra branch. - The checksums use different types (__wsum and u32), causing casts to be needed. - It made the checksums of fragments be combined (rather than chained) for both checksums, despite this being highly counterproductive for CRC32C due to how slow crc32c_combine() is. This can clearly be seen in commit 4c2f24549644 ("sctp: linearize early if it's not GSO") which tried to work around this performance bug. With a dedicated function for each checksum, we can instead just use the proper strategy for each checksum. Signed-off-by: Eric Biggers --- include/linux/skbuff.h | 1 + net/core/skbuff.c | 73 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 74 insertions(+) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index f3e72be6f634..33b33bb18aa6 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -4192,10 +4192,11 @@ extern const struct skb_checksum_ops *crc32c_csum_s= tub __read_mostly; =20 __wsum __skb_checksum(const struct sk_buff *skb, int offset, int len, __wsum csum, const struct skb_checksum_ops *ops); __wsum skb_checksum(const struct sk_buff *skb, int offset, int len, __wsum csum); +u32 skb_crc32c(const struct sk_buff *skb, int offset, int len, u32 crc); =20 static inline void * __must_check __skb_header_pointer(const struct sk_buff *skb, int offset, int len, const void *data, int hlen, void *buffer) { diff --git a/net/core/skbuff.c b/net/core/skbuff.c index d73ad79fe739..b9900cc16a24 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -62,10 +62,11 @@ #include #include #include #include #include +#include =20 #include #include #include #include @@ -3626,10 +3627,82 @@ __wsum skb_copy_and_csum_bits(const struct sk_buff = *skb, int offset, BUG_ON(len); return csum; } EXPORT_SYMBOL(skb_copy_and_csum_bits); =20 +#ifdef CONFIG_NET_CRC32C +u32 skb_crc32c(const struct sk_buff *skb, int offset, int len, u32 crc) +{ + int start =3D skb_headlen(skb); + int i, copy =3D start - offset; + struct sk_buff *frag_iter; + + if (copy > 0) { + copy =3D min(copy, len); + crc =3D crc32c(crc, skb->data + offset, copy); + len -=3D copy; + if (len =3D=3D 0) + return crc; + offset +=3D copy; + } + + if (WARN_ON_ONCE(!skb_frags_readable(skb))) + return 0; + + for (i =3D 0; i < skb_shinfo(skb)->nr_frags; i++) { + int end; + skb_frag_t *frag =3D &skb_shinfo(skb)->frags[i]; + + WARN_ON(start > offset + len); + + end =3D start + skb_frag_size(frag); + copy =3D end - offset; + if (copy > 0) { + u32 p_off, p_len, copied; + struct page *p; + u8 *vaddr; + + copy =3D min(copy, len); + skb_frag_foreach_page(frag, + skb_frag_off(frag) + offset - start, + copy, p, p_off, p_len, copied) { + vaddr =3D kmap_atomic(p); + crc =3D crc32c(crc, vaddr + p_off, p_len); + kunmap_atomic(vaddr); + } + len -=3D copy; + if (len =3D=3D 0) + return crc; + offset +=3D copy; + } + start =3D end; + } + + skb_walk_frags(skb, frag_iter) { + int end; + + WARN_ON(start > offset + len); + + end =3D start + frag_iter->len; + copy =3D end - offset; + if (copy > 0) { + copy =3D min(copy, len); + crc =3D skb_crc32c(frag_iter, offset - start, copy, crc); + len -=3D copy; + if (len =3D=3D 0) + return crc; + offset +=3D copy; + } + start =3D end; + } + BUG_ON(len); + + return crc; +} +EXPORT_SYMBOL(skb_crc32c); +#endif /* CONFIG_NET_CRC32C */ + __sum16 __skb_checksum_complete_head(struct sk_buff *skb, int len) { __sum16 sum; =20 sum =3D csum_fold(skb_checksum(skb, 0, len, skb->csum)); --=20 2.49.0 From nobody Sat Feb 7 18:16:08 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 979A01F92E; Sun, 11 May 2025 00:41:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924092; cv=none; b=PpaYoLnFsAYhHj7Cb1ONuL7y0h0gNNp/AY1OLBskSrjFF76GJlf3Q4sF0EnSFT67/PJN+aLurevXsSRF2K/CMjBydClX473rY7FKqdtOZN5ihU83Zs1AMzm4eWvQqZGYFwnUTA8fmOr9EZwjMW8AMfW6YMQ/+TjcbyArVrrqez4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924092; c=relaxed/simple; bh=dQ3McGhqu8bezBUWJWiJB1iEsbKs9gFTYUnJ6baxEuo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BdfE6+iecNaj1Tuhgu+ErELaqHsFuP+f88UL3/wSQohnNiwEKMdyfrPlG9wSE/c57eQdI/h+U5AzQSv9qHIkaZLj+SQs0DmVadjTu9cKWIr5nJQV15fFe1bL+769tIgX7WrKLOVK31khGbWZueVezyDhrOaPm1e6v0goFyN1ZTs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=sa7T7Oxn; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="sa7T7Oxn" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F3EDC4CEEF; Sun, 11 May 2025 00:41:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746924092; bh=dQ3McGhqu8bezBUWJWiJB1iEsbKs9gFTYUnJ6baxEuo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sa7T7OxnnKAOtnHbW67Yh3SN6VEFAghG5iuSecCjaRjWK2H5ZFnqd3ruXmX1WCyPj lMGthlluCIWNDUYdw8q/r4ByWc+TU+nj/oVY0GvbUT8XmBr/Gh8OYwcGF1QzHPlXFI RDaackghdGL9dFfrsqWe9NGfMk//qkLqMk1/UGCTqlIXPt7H3DpmtQqS/4uT2bZlr8 rR1BreDqMu+yp9bSA0rtgY/IDR9iPUBpoNV5AhN/RaYk9lk3ajZJWWQq0N3VIdBV17 YFQd37j3IMwjX3/cZkWZvCi4PNEJ5rwtSSo72l6hVio6ufFY3Rk+EXMMvad4l7chNY 964mQbVB+pA5Q== From: Eric Biggers To: netdev@vger.kernel.org Cc: linux-nvme@lists.infradead.org, linux-sctp@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, Daniel Borkmann , Marcelo Ricardo Leitner , Sagi Grimberg , Ard Biesheuvel Subject: [PATCH net-next 03/10] net: use skb_crc32c() in skb_crc32c_csum_help() Date: Sat, 10 May 2025 17:41:03 -0700 Message-ID: <20250511004110.145171-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250511004110.145171-1-ebiggers@kernel.org> References: <20250511004110.145171-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Instead of calling __skb_checksum() with a skb_checksum_ops struct that does CRC32C, just call the new function skb_crc32c(). This is faster and simpler. Signed-off-by: Eric Biggers --- net/core/dev.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/net/core/dev.c b/net/core/dev.c index 51606b2d17bb..1eda2501f07b 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -3595,11 +3595,11 @@ int skb_checksum_help(struct sk_buff *skb) EXPORT_SYMBOL(skb_checksum_help); =20 #ifdef CONFIG_NET_CRC32C int skb_crc32c_csum_help(struct sk_buff *skb) { - __le32 crc32c_csum; + u32 crc; int ret =3D 0, offset, start; =20 if (skb->ip_summed !=3D CHECKSUM_PARTIAL) goto out; =20 @@ -3623,14 +3623,12 @@ int skb_crc32c_csum_help(struct sk_buff *skb) =20 ret =3D skb_ensure_writable(skb, offset + sizeof(__le32)); if (ret) goto out; =20 - crc32c_csum =3D cpu_to_le32(~__skb_checksum(skb, start, - skb->len - start, ~(__u32)0, - crc32c_csum_stub)); - *(__le32 *)(skb->data + offset) =3D crc32c_csum; + crc =3D ~skb_crc32c(skb, start, skb->len - start, ~0); + *(__le32 *)(skb->data + offset) =3D cpu_to_le32(crc); skb_reset_csum_not_inet(skb); out: return ret; } EXPORT_SYMBOL(skb_crc32c_csum_help); --=20 2.49.0 From nobody Sat Feb 7 18:16:08 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB1E5155322; Sun, 11 May 2025 00:41:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924093; cv=none; b=Dw3n3w+WB9rM+lYT8XKZxQJv+yVtoyHGCFdp3eHYMA+l49qGVSiscTJjvSB27AH4pJ3Bq047EirF32lC4fu+Nw0d5PT3crCjAvYujvBk9ryAbr87qatBaFmjURppqZvVnrZIeDs/72V6+eZ8nqEdaxXuig5G7kcQjLBz88dXzbo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924093; c=relaxed/simple; bh=IwN0x6l2VckNHXEEglHCnVHt/6hl+HXCt3jiyh026og=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Y6XrF7vyZuHxtmigfKj2ItOditNYl4MUAlw8vEy3py+Bf3SxO7RQ2raO4Dk2H2nPYLM97/txMDPDICoUSyt24IibkZUH/PfVMohSLRpW6dZEONWZHxd9wUuWzMjes8W9Rs/s/OBk8Xf6Y++8QolBEc/k4UvXW9u7F0KlkJuoYxk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=SJemVLpI; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SJemVLpI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4CADEC4CEE2; Sun, 11 May 2025 00:41:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746924093; bh=IwN0x6l2VckNHXEEglHCnVHt/6hl+HXCt3jiyh026og=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SJemVLpI1SUn8x/itxDunpJJSTPHgSrvU7okEl2jVOzcEmKGHFuJEYaek3TfkG/37 7H5LtY6qDtcPgB/uPA94WF00BzSm3cPQkSIZAy76OQ5GwjdtwAd9vD9+QFAYMruF/E 6PjPCY/oTm2bcITyxO5cSMwJ+s7cVi1XdM9EU76ax+ODQ8j1XEClds82ensWXTDuXz JsTPrFfin6XR5bt+LvJM5U2KYo5VBaAcNqRayfiOHqTmm0VzSpIxr/jI4nAPw0n3zo oXkQVutteDPHb4aVsup7jD9XMotu4BmggbwVlH3qVsiLcrJmnhOD0beWJaWp5T2TXV /ztmrEfidVbJg== From: Eric Biggers To: netdev@vger.kernel.org Cc: linux-nvme@lists.infradead.org, linux-sctp@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, Daniel Borkmann , Marcelo Ricardo Leitner , Sagi Grimberg , Ard Biesheuvel Subject: [PATCH net-next 04/10] RDMA/siw: use skb_crc32c() instead of __skb_checksum() Date: Sat, 10 May 2025 17:41:04 -0700 Message-ID: <20250511004110.145171-5-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250511004110.145171-1-ebiggers@kernel.org> References: <20250511004110.145171-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Instead of calling __skb_checksum() with a skb_checksum_ops struct that does CRC32C, just call the new function skb_crc32c(). This is faster and simpler. Signed-off-by: Eric Biggers --- drivers/infiniband/sw/siw/Kconfig | 1 + drivers/infiniband/sw/siw/siw.h | 22 +--------------------- 2 files changed, 2 insertions(+), 21 deletions(-) diff --git a/drivers/infiniband/sw/siw/Kconfig b/drivers/infiniband/sw/siw/= Kconfig index ae4a953e2a03..186f182b80e7 100644 --- a/drivers/infiniband/sw/siw/Kconfig +++ b/drivers/infiniband/sw/siw/Kconfig @@ -1,10 +1,11 @@ config RDMA_SIW tristate "Software RDMA over TCP/IP (iWARP) driver" depends on INET && INFINIBAND depends on INFINIBAND_VIRT_DMA select CRC32 + select NET_CRC32C help This driver implements the iWARP RDMA transport over the Linux TCP/IP network stack. It enables a system with a standard Ethernet adapter to interoperate with a iWARP adapter or with another system running the SIW driver. diff --git a/drivers/infiniband/sw/siw/siw.h b/drivers/infiniband/sw/siw/si= w.h index 385067e07faf..d9e5a2e4c471 100644 --- a/drivers/infiniband/sw/siw/siw.h +++ b/drivers/infiniband/sw/siw/siw.h @@ -691,33 +691,13 @@ static inline void siw_crc_oneshot(const void *data, = size_t len, u8 out[4]) siw_crc_init(&crc); siw_crc_update(&crc, data, len); return siw_crc_final(&crc, out); } =20 -static inline __wsum siw_csum_update(const void *buff, int len, __wsum sum) -{ - return (__force __wsum)crc32c((__force __u32)sum, buff, len); -} - -static inline __wsum siw_csum_combine(__wsum csum, __wsum csum2, int offse= t, - int len) -{ - return (__force __wsum)crc32c_combine((__force __u32)csum, - (__force __u32)csum2, len); -} - static inline void siw_crc_skb(struct siw_rx_stream *srx, unsigned int len) { - const struct skb_checksum_ops siw_cs_ops =3D { - .update =3D siw_csum_update, - .combine =3D siw_csum_combine, - }; - __wsum crc =3D (__force __wsum)srx->mpa_crc; - - crc =3D __skb_checksum(srx->skb, srx->skb_offset, len, crc, - &siw_cs_ops); - srx->mpa_crc =3D (__force u32)crc; + srx->mpa_crc =3D skb_crc32c(srx->skb, srx->skb_offset, len, srx->mpa_crc); } =20 #define siw_dbg(ibdev, fmt, ...) = \ ibdev_dbg(ibdev, "%s: " fmt, __func__, ##__VA_ARGS__) =20 --=20 2.49.0 From nobody Sat Feb 7 18:16:08 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC89D179BF; Sun, 11 May 2025 00:41:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924096; cv=none; b=sHsVPR76S+a+jQwVk91FRnYEX+OwjPJSUdNozNQwlRlChgFewS1gnFIRN6mcBfqHrj8PUYAUy2LWC375PNiHigl6iPE46H61IC0E4/iDQMcVZKh1Nm3pG7oF3JcA5zKJKkabbyTJZbeKFKTAti9v1xzNTn+uHXVJnEV/4O328UE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924096; c=relaxed/simple; bh=rzxlJ+REpOzn2hxjAiUOHuapxr7sRcF5AgoLopDe6KQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OrfusT1qe0YH6f7u94HSbpaoQ3BG5hulC52Wdku+eaTEbN0FaXwkkMw4ZhyMjP3hd86T1JyBcJvsUf+l+Q9uM0Uao6Axv/Y86PpyuBll2ivvgbPZ2snAx1p7w9xRzCVRYCTpMcwpe+bQIjPkzzXN3W92znvZIeCpcZ4wNNmnT4k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rm/YLJLE; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rm/YLJLE" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6B472C4CEE7; Sun, 11 May 2025 00:41:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746924094; bh=rzxlJ+REpOzn2hxjAiUOHuapxr7sRcF5AgoLopDe6KQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rm/YLJLE66m5WNVbp4MovT3aGm3bwEndEL51q8a2/boVKXK6nYdKZg/vVQFQPKleJ l/e30U3014fXykXrZAK8jxbJXcO/IFBp/aO5zaW34w01OvvqOE9nn9qtjsuhAkBoLr KWFW+hv0HpWlJgsIdqR8vFsplS6Td3hKtaN3oaw7NWBsPLn7kEMc5vC1p7fS3WK9mA eNKHlUbYel1pBeiUIJ+VwhaQy2a7E5/DpVmiKdbpBvdqp8Z6rMZqxc0yVOJXSfFXi+ Nec2eKKlsISTaovL4hCDhNNmMPKRpxIJ343FGc9JCrhCcHfF4wKShEW/P36xp/EL7g R2t/fWbZO9oUA== From: Eric Biggers To: netdev@vger.kernel.org Cc: linux-nvme@lists.infradead.org, linux-sctp@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, Daniel Borkmann , Marcelo Ricardo Leitner , Sagi Grimberg , Ard Biesheuvel Subject: [PATCH net-next 05/10] sctp: use skb_crc32c() instead of __skb_checksum() Date: Sat, 10 May 2025 17:41:05 -0700 Message-ID: <20250511004110.145171-6-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250511004110.145171-1-ebiggers@kernel.org> References: <20250511004110.145171-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Make sctp_compute_cksum() just use the new function skb_crc32c(), instead of calling __skb_checksum() with a skb_checksum_ops struct that does CRC32C. This is faster and simpler. Signed-off-by: Eric Biggers --- include/net/sctp/checksum.h | 29 +++-------------------------- net/netfilter/Kconfig | 4 ++-- net/netfilter/ipvs/Kconfig | 2 +- net/openvswitch/Kconfig | 2 +- net/sched/Kconfig | 2 +- net/sctp/Kconfig | 1 - net/sctp/offload.c | 1 - 7 files changed, 8 insertions(+), 33 deletions(-) diff --git a/include/net/sctp/checksum.h b/include/net/sctp/checksum.h index 291465c25810..654d37ec0402 100644 --- a/include/net/sctp/checksum.h +++ b/include/net/sctp/checksum.h @@ -13,51 +13,28 @@ * * Written or modified by: * Dinakaran Joseph * Jon Grimm * Sridhar Samudrala - * - * Rewritten to use libcrc32c by: * Vlad Yasevich */ =20 #ifndef __sctp_checksum_h__ #define __sctp_checksum_h__ =20 #include #include -#include -#include - -static inline __wsum sctp_csum_update(const void *buff, int len, __wsum su= m) -{ - return (__force __wsum)crc32c((__force __u32)sum, buff, len); -} - -static inline __wsum sctp_csum_combine(__wsum csum, __wsum csum2, - int offset, int len) -{ - return (__force __wsum)crc32c_combine((__force __u32)csum, - (__force __u32)csum2, len); -} - -static const struct skb_checksum_ops sctp_csum_ops =3D { - .update =3D sctp_csum_update, - .combine =3D sctp_csum_combine, -}; =20 static inline __le32 sctp_compute_cksum(const struct sk_buff *skb, unsigned int offset) { struct sctphdr *sh =3D (struct sctphdr *)(skb->data + offset); __le32 old =3D sh->checksum; - __wsum new; + u32 new; =20 sh->checksum =3D 0; - new =3D ~__skb_checksum(skb, offset, skb->len - offset, ~(__wsum)0, - &sctp_csum_ops); + new =3D ~skb_crc32c(skb, offset, skb->len - offset, ~0); sh->checksum =3D old; - - return cpu_to_le32((__force __u32)new); + return cpu_to_le32(new); } =20 #endif /* __sctp_checksum_h__ */ diff --git a/net/netfilter/Kconfig b/net/netfilter/Kconfig index 3b2183fc7e56..2560416218d0 100644 --- a/net/netfilter/Kconfig +++ b/net/netfilter/Kconfig @@ -210,11 +210,11 @@ config NF_CT_PROTO_GRE =20 config NF_CT_PROTO_SCTP bool 'SCTP protocol connection tracking support' depends on NETFILTER_ADVANCED default y - select CRC32 + select NET_CRC32C help With this option enabled, the layer 3 independent connection tracking code will be able to do state tracking on SCTP connections. =20 If unsure, say Y. @@ -473,11 +473,11 @@ config NETFILTER_SYNPROXY =20 endif # NF_CONNTRACK =20 config NF_TABLES select NETFILTER_NETLINK - select CRC32 + select NET_CRC32C tristate "Netfilter nf_tables support" help nftables is the new packet classification framework that intends to replace the existing {ip,ip6,arp,eb}_tables infrastructure. It provides a pseudo-state machine with an extensible instruction-set diff --git a/net/netfilter/ipvs/Kconfig b/net/netfilter/ipvs/Kconfig index 8c5b1fe12d07..c203252e856d 100644 --- a/net/netfilter/ipvs/Kconfig +++ b/net/netfilter/ipvs/Kconfig @@ -103,11 +103,11 @@ config IP_VS_PROTO_AH This option enables support for load balancing AH (Authentication Header) transport protocol. Say Y if unsure. =20 config IP_VS_PROTO_SCTP bool "SCTP load balancing support" - select CRC32 + select NET_CRC32C help This option enables support for load balancing SCTP transport protocol. Say Y if unsure. =20 comment "IPVS scheduler" diff --git a/net/openvswitch/Kconfig b/net/openvswitch/Kconfig index 5481bd561eb4..e6aaee92dba4 100644 --- a/net/openvswitch/Kconfig +++ b/net/openvswitch/Kconfig @@ -9,12 +9,12 @@ config OPENVSWITCH depends on !NF_CONNTRACK || \ (NF_CONNTRACK && ((!NF_DEFRAG_IPV6 || NF_DEFRAG_IPV6) && \ (!NF_NAT || NF_NAT) && \ (!NETFILTER_CONNCOUNT || NETFILTER_CONNCOUNT))) depends on PSAMPLE || !PSAMPLE - select CRC32 select MPLS + select NET_CRC32C select NET_MPLS_GSO select DST_CACHE select NET_NSH select NF_CONNTRACK_OVS if NF_CONNTRACK select NF_NAT_OVS if NF_NAT diff --git a/net/sched/Kconfig b/net/sched/Kconfig index 9f0b3f943fca..ad914d2b2e22 100644 --- a/net/sched/Kconfig +++ b/net/sched/Kconfig @@ -794,11 +794,11 @@ config NET_ACT_SKBEDIT module will be called act_skbedit. =20 config NET_ACT_CSUM tristate "Checksum Updating" depends on NET_CLS_ACT && INET - select CRC32 + select NET_CRC32C help Say Y here to update some common checksum after some direct packet alterations. =20 To compile this code as a module, choose M here: the diff --git a/net/sctp/Kconfig b/net/sctp/Kconfig index 3669ba351856..24d5a35ce894 100644 --- a/net/sctp/Kconfig +++ b/net/sctp/Kconfig @@ -5,11 +5,10 @@ =20 menuconfig IP_SCTP tristate "The SCTP Protocol" depends on INET depends on IPV6 || IPV6=3Dn - select CRC32 select CRYPTO select CRYPTO_HMAC select CRYPTO_SHA1 select NET_CRC32C select NET_UDP_TUNNEL diff --git a/net/sctp/offload.c b/net/sctp/offload.c index 502095173d88..e6f863c031b4 100644 --- a/net/sctp/offload.c +++ b/net/sctp/offload.c @@ -109,11 +109,10 @@ int __init sctp_offload_init(void) =20 ret =3D inet6_add_offload(&sctp6_offload, IPPROTO_SCTP); if (ret) goto ipv4; =20 - crc32c_csum_stub =3D &sctp_csum_ops; return ret; =20 ipv4: inet_del_offload(&sctp_offload, IPPROTO_SCTP); out: --=20 2.49.0 From nobody Sat Feb 7 18:16:08 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1DD961922C4; Sun, 11 May 2025 00:41:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924096; cv=none; b=L6uEZLsZzg/ftHvbWQ+iMJgf0P3QWRcEgpd4239xW20ZwklvomZbyIzan/m4zmH3uEdBBqxoeJWfCPs8HCizcIhboKk2oN+vQtv9zqyG46xqIR1XRbiN+O8Q7pep7Ea6fsmruXvNGPPJL3fLH6KV5E9Y3f1S+FfsUsY6f4qQzrg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924096; c=relaxed/simple; bh=/A9NtJJ0pcYSzi7Q5tbR5gfyPyk7AS4gT0Xn3zsrRXA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=d2KgJ7S6Y7EcI3pkhQgadH3d804VslCoQZUiGnzLIuVJ5WEWLZBbdZuMn97cvZvotfQcVOpxdheGsBdf5NE+FoKlWdrVjFwBXCihufit8TeaYuCDDrDfd5GoYps101Up8WmRsD8BFf3QB49+5OjB5/84TIPceDIuiGY2VPKeLRI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dQ45jumq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dQ45jumq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A9761C4CEEF; Sun, 11 May 2025 00:41:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746924095; bh=/A9NtJJ0pcYSzi7Q5tbR5gfyPyk7AS4gT0Xn3zsrRXA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dQ45jumq+j1KKVo/UHlGTJd20e4DFKDf4HDb7zNpC1qGye2USU/ef8NjVPsV0T14A M4jug2r/Yq+VOS1fGQCxzyXAYoyoevhEV9K80h9QVcttRSXM+iMcoZJkQbp9lQCszx ITOq1tQ3BQYVL+VjDTaZbvKR9A2dg1fYJ5d3vWpy6ATm10LTVbR11DIvSHr7zk/FXp +7hSbi+HClEtL8cFGC8eYxB7glmb8eNbM/ZqywcVrB8xrRphB55w2ygtQI4t75sOh3 GFOjOLIPmoXEpUY7A2BNrk9RdobQ6OX+wMc9mDAWhkfGgZVv65FSbPiSRRJoeuttck 6zPsnrLDxUwaw== From: Eric Biggers To: netdev@vger.kernel.org Cc: linux-nvme@lists.infradead.org, linux-sctp@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, Daniel Borkmann , Marcelo Ricardo Leitner , Sagi Grimberg , Ard Biesheuvel Subject: [PATCH net-next 06/10] net: fold __skb_checksum() into skb_checksum() Date: Sat, 10 May 2025 17:41:06 -0700 Message-ID: <20250511004110.145171-7-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250511004110.145171-1-ebiggers@kernel.org> References: <20250511004110.145171-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Now that the only remaining caller of __skb_checksum() is skb_checksum(), fold __skb_checksum() into skb_checksum(). This makes struct skb_checksum_ops unnecessary, so remove that too and simply do the "regular" net checksum. It also makes the wrapper functions csum_partial_ext() and csum_block_add_ext() unnecessary, so remove those too and just use the underlying functions. Signed-off-by: Eric Biggers --- include/linux/skbuff.h | 9 ------- include/net/checksum.h | 12 --------- net/core/skbuff.c | 59 +++++------------------------------------- 3 files changed, 7 insertions(+), 73 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 33b33bb18aa6..eac5c8dde4a5 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -4181,19 +4181,10 @@ static inline int memcpy_from_msg(void *data, struc= t msghdr *msg, int len) static inline int memcpy_to_msg(struct msghdr *msg, void *data, int len) { return copy_to_iter(data, len, &msg->msg_iter) =3D=3D len ? 0 : -EFAULT; } =20 -struct skb_checksum_ops { - __wsum (*update)(const void *mem, int len, __wsum wsum); - __wsum (*combine)(__wsum csum, __wsum csum2, int offset, int len); -}; - -extern const struct skb_checksum_ops *crc32c_csum_stub __read_mostly; - -__wsum __skb_checksum(const struct sk_buff *skb, int offset, int len, - __wsum csum, const struct skb_checksum_ops *ops); __wsum skb_checksum(const struct sk_buff *skb, int offset, int len, __wsum csum); u32 skb_crc32c(const struct sk_buff *skb, int offset, int len, u32 crc); =20 static inline void * __must_check diff --git a/include/net/checksum.h b/include/net/checksum.h index 243f972267b8..e57986b173f8 100644 --- a/include/net/checksum.h +++ b/include/net/checksum.h @@ -96,16 +96,10 @@ static __always_inline __wsum csum_block_add(__wsum csum, __wsum csum2, int offset) { return csum_add(csum, csum_shift(csum2, offset)); } =20 -static __always_inline __wsum -csum_block_add_ext(__wsum csum, __wsum csum2, int offset, int len) -{ - return csum_block_add(csum, csum2, offset); -} - static __always_inline __wsum csum_block_sub(__wsum csum, __wsum csum2, int offset) { return csum_block_add(csum, ~csum2, offset); } @@ -113,16 +107,10 @@ csum_block_sub(__wsum csum, __wsum csum2, int offset) static __always_inline __wsum csum_unfold(__sum16 n) { return (__force __wsum)n; } =20 -static __always_inline -__wsum csum_partial_ext(const void *buff, int len, __wsum sum) -{ - return csum_partial(buff, len, sum); -} - #define CSUM_MANGLED_0 ((__force __sum16)0xffff) =20 static __always_inline void csum_replace_by_diff(__sum16 *sum, __wsum diff) { *sum =3D csum_fold(csum_add(diff, ~csum_unfold(*sum))); diff --git a/net/core/skbuff.c b/net/core/skbuff.c index b9900cc16a24..bfa892ccc7f7 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -3438,24 +3438,22 @@ int skb_store_bits(struct sk_buff *skb, int offset,= const void *from, int len) return -EFAULT; } EXPORT_SYMBOL(skb_store_bits); =20 /* Checksum skb data. */ -__wsum __skb_checksum(const struct sk_buff *skb, int offset, int len, - __wsum csum, const struct skb_checksum_ops *ops) +__wsum skb_checksum(const struct sk_buff *skb, int offset, int len, __wsum= csum) { int start =3D skb_headlen(skb); int i, copy =3D start - offset; struct sk_buff *frag_iter; int pos =3D 0; =20 /* Checksum header. */ if (copy > 0) { if (copy > len) copy =3D len; - csum =3D INDIRECT_CALL_1(ops->update, csum_partial_ext, - skb->data + offset, copy, csum); + csum =3D csum_partial(skb->data + offset, copy, csum); if ((len -=3D copy) =3D=3D 0) return csum; offset +=3D copy; pos =3D copy; } @@ -3481,17 +3479,13 @@ __wsum __skb_checksum(const struct sk_buff *skb, in= t offset, int len, =20 skb_frag_foreach_page(frag, skb_frag_off(frag) + offset - start, copy, p, p_off, p_len, copied) { vaddr =3D kmap_atomic(p); - csum2 =3D INDIRECT_CALL_1(ops->update, - csum_partial_ext, - vaddr + p_off, p_len, 0); + csum2 =3D csum_partial(vaddr + p_off, p_len, 0); kunmap_atomic(vaddr); - csum =3D INDIRECT_CALL_1(ops->combine, - csum_block_add_ext, csum, - csum2, pos, p_len); + csum =3D csum_block_add(csum, csum2, pos); pos +=3D p_len; } =20 if (!(len -=3D copy)) return csum; @@ -3508,14 +3502,13 @@ __wsum __skb_checksum(const struct sk_buff *skb, in= t offset, int len, end =3D start + frag_iter->len; if ((copy =3D end - offset) > 0) { __wsum csum2; if (copy > len) copy =3D len; - csum2 =3D __skb_checksum(frag_iter, offset - start, - copy, 0, ops); - csum =3D INDIRECT_CALL_1(ops->combine, csum_block_add_ext, - csum, csum2, pos, copy); + csum2 =3D skb_checksum(frag_iter, offset - start, copy, + 0); + csum =3D csum_block_add(csum, csum2, pos); if ((len -=3D copy) =3D=3D 0) return csum; offset +=3D copy; pos +=3D copy; } @@ -3523,22 +3516,10 @@ __wsum __skb_checksum(const struct sk_buff *skb, in= t offset, int len, } BUG_ON(len); =20 return csum; } -EXPORT_SYMBOL(__skb_checksum); - -__wsum skb_checksum(const struct sk_buff *skb, int offset, - int len, __wsum csum) -{ - const struct skb_checksum_ops ops =3D { - .update =3D csum_partial_ext, - .combine =3D csum_block_add_ext, - }; - - return __skb_checksum(skb, offset, len, csum, &ops); -} EXPORT_SYMBOL(skb_checksum); =20 /* Both of above in one bottle. */ =20 __wsum skb_copy_and_csum_bits(const struct sk_buff *skb, int offset, @@ -3758,36 +3739,10 @@ __sum16 __skb_checksum_complete(struct sk_buff *skb) =20 return sum; } EXPORT_SYMBOL(__skb_checksum_complete); =20 -static __wsum warn_crc32c_csum_update(const void *buff, int len, __wsum su= m) -{ - net_warn_ratelimited( - "%s: attempt to compute crc32c without libcrc32c.ko\n", - __func__); - return 0; -} - -static __wsum warn_crc32c_csum_combine(__wsum csum, __wsum csum2, - int offset, int len) -{ - net_warn_ratelimited( - "%s: attempt to compute crc32c without libcrc32c.ko\n", - __func__); - return 0; -} - -static const struct skb_checksum_ops default_crc32c_ops =3D { - .update =3D warn_crc32c_csum_update, - .combine =3D warn_crc32c_csum_combine, -}; - -const struct skb_checksum_ops *crc32c_csum_stub __read_mostly =3D - &default_crc32c_ops; -EXPORT_SYMBOL(crc32c_csum_stub); - /** * skb_zerocopy_headlen - Calculate headroom needed for skb_zerocopy() * @from: source buffer * * Calculates the amount of linear headroom needed in the 'to' skb passed --=20 2.49.0 From nobody Sat Feb 7 18:16:08 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B0D319E97C; Sun, 11 May 2025 00:41:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924097; cv=none; b=L40DIQaoszsNHsDdBn9yQ2t/VXq8VR/Cldst0dlVgol0IiiyeV6TJqx4KVozE9WLFLvtn0yH5bWGCufAV6I03n6O9Ny4Zzr3EuG0Bk+1nAxMQQtUUhx9j8nZutOACfd40Fqyu2Reum95Ouqti9INgQYm4sQP5blyZpXxdI57cd8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924097; c=relaxed/simple; bh=lrrEmehbwxWEKGD3wwCwtRu2pJjck1nWjCeLNo9TZWc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CYOxqduEkSj18pEJQmAgi5GH+V69CW2vABEDPPeHUoaebEv4/8bZAcLDUZV9kE+0CahZzNkwdXwPGHxn8xFPaF64yqbZWY/kXhl8kbmLHsR4XfTq+guTT2KRYbwUpvHr7WmuIjMx8SFphdRfT76ZnZc1fDSDrf+KLahUmeyhZmc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=tZ+G6GKy; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="tZ+G6GKy" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DADD6C4CEE2; Sun, 11 May 2025 00:41:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746924096; bh=lrrEmehbwxWEKGD3wwCwtRu2pJjck1nWjCeLNo9TZWc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tZ+G6GKyaMO9jwCJkNQPgPKy07uW683dXPqFPHLVjvns+beQP1B0lSyu4y/UlOKcN WNCifKV9Fi4CLPKSzJRkKGLS37q1CTr/mM43gJ3d9OPluIDjdjEJzwZ8VkXlZko3lP HUMQlrCAs1PFxceMg5nNJ0lV15q6d7+B7dl28oao79TYJOp3BtI1BFesBcTtSiz0EO Dj5sGtUbGJmnZdskWImOYgulYYil/93TlT7okbsQ6nNg+414oguMpdfzsweIyep4E6 PAUauigCUbb9KjtfcfhvdVlEWXBmzmmnFtu6/RMSV8LOgHxRn2G9RYhRLltKhn++wt XximtRg5csL+g== From: Eric Biggers To: netdev@vger.kernel.org Cc: linux-nvme@lists.infradead.org, linux-sctp@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, Daniel Borkmann , Marcelo Ricardo Leitner , Sagi Grimberg , Ard Biesheuvel Subject: [PATCH net-next 07/10] lib/crc32: remove unused support for CRC32C combination Date: Sat, 10 May 2025 17:41:07 -0700 Message-ID: <20250511004110.145171-8-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250511004110.145171-1-ebiggers@kernel.org> References: <20250511004110.145171-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers crc32c_combine() and crc32c_shift() are no longer used (except by the KUnit test that tests them), and their current implementation is very slow. Remove them. Signed-off-by: Eric Biggers --- include/linux/crc32.h | 23 ----------------------- lib/crc32.c | 6 ------ lib/tests/crc_kunit.c | 6 ------ 3 files changed, 35 deletions(-) diff --git a/include/linux/crc32.h b/include/linux/crc32.h index 69c2e8bb3782..7f7d0be8a0ac 100644 --- a/include/linux/crc32.h +++ b/include/linux/crc32.h @@ -74,33 +74,10 @@ u32 crc32_le_shift(u32 crc, size_t len); static inline u32 crc32_le_combine(u32 crc1, u32 crc2, size_t len2) { return crc32_le_shift(crc1, len2) ^ crc2; } =20 -u32 crc32c_shift(u32 crc, size_t len); - -/** - * crc32c_combine - Combine two crc32c check values into one. For two sequ= ences - * of bytes, seq1 and seq2 with lengths len1 and len2, crc32c() - * check values were calculated for each, crc1 and crc2. - * - * @crc1: crc32c of the first block - * @crc2: crc32c of the second block - * @len2: length of the second block - * - * Return: The crc32c() check value of seq1 and seq2 concatenated, requiri= ng - * only crc1, crc2, and len2. Note: If seq_full denotes the concatenated - * memory area of seq1 with seq2, and crc_full the crc32c() value of - * seq_full, then crc_full =3D=3D crc32c_combine(crc1, crc2, len2) when - * crc_full was seeded with the same initializer as crc1, and crc2 seed - * was 0. See also crc_combine_test(). - */ -static inline u32 crc32c_combine(u32 crc1, u32 crc2, size_t len2) -{ - return crc32c_shift(crc1, len2) ^ crc2; -} - #define crc32(seed, data, length) crc32_le(seed, (unsigned char const *)(= data), length) =20 /* * Helpers for hash table generation of ethernet nics: * diff --git a/lib/crc32.c b/lib/crc32.c index fddd424ff224..ade48bbb0083 100644 --- a/lib/crc32.c +++ b/lib/crc32.c @@ -117,16 +117,10 @@ u32 crc32_le_shift(u32 crc, size_t len) { return crc32_generic_shift(crc, len, CRC32_POLY_LE); } EXPORT_SYMBOL(crc32_le_shift); =20 -u32 crc32c_shift(u32 crc, size_t len) -{ - return crc32_generic_shift(crc, len, CRC32C_POLY_LE); -} -EXPORT_SYMBOL(crc32c_shift); - u32 crc32_be_base(u32 crc, const u8 *p, size_t len) { while (len--) crc =3D (crc << 8) ^ crc32table_be[(crc >> 24) ^ *p++]; return crc; diff --git a/lib/tests/crc_kunit.c b/lib/tests/crc_kunit.c index 585c48b65cef..064c2d581557 100644 --- a/lib/tests/crc_kunit.c +++ b/lib/tests/crc_kunit.c @@ -389,21 +389,15 @@ static void crc32_be_benchmark(struct kunit *test) static u64 crc32c_wrapper(u64 crc, const u8 *p, size_t len) { return crc32c(crc, p, len); } =20 -static u64 crc32c_combine_wrapper(u64 crc1, u64 crc2, size_t len2) -{ - return crc32c_combine(crc1, crc2, len2); -} - static const struct crc_variant crc_variant_crc32c =3D { .bits =3D 32, .le =3D true, .poly =3D 0x82f63b78, .func =3D crc32c_wrapper, - .combine_func =3D crc32c_combine_wrapper, }; =20 static void crc32c_test(struct kunit *test) { crc_test(test, &crc_variant_crc32c); --=20 2.49.0 From nobody Sat Feb 7 18:16:08 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A4771A5B85; Sun, 11 May 2025 00:41:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924098; cv=none; b=XokIHpypDLhjEFY5A/iuRUvg5rFfBxIA40FrvuZUsVWUQ/ivbwDtodOqaapN5vM2xLVD2RUMCaoOQ9W+0sHbFmRdAoiaJTKfdZmtUEOFD6+TzIkf60pF63ComhxyqRkXdNLlz1/R3UGjAb8sic4pukaEKAiQ6pc3mm7YrWa1rT0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924098; c=relaxed/simple; bh=LyWew4r7C8/s4G5rA7UFXiI52nXjAIqNl5KwWhIs7mU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pKhkSTuxe/subqwJ6MaMtVIpOaVreaLbdgOEEgcxD+6Du+ClaEy2g2gG27uj/2wIJwaCVwGL7OhTzk/LxhPxcDUWzVY3oxWriPeDdb3DrzbLOnaBN5ptak32Tc246YxC7cl5nb6FfJL5FrlUeJ9k53MNHlrygH3kWLLjRRE5RIo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=oV1P3UPx; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="oV1P3UPx" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 180BBC4CEEE; Sun, 11 May 2025 00:41:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746924097; bh=LyWew4r7C8/s4G5rA7UFXiI52nXjAIqNl5KwWhIs7mU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oV1P3UPxoSVSGXQHgQoCgxQxJfnGE9kH5epMt277/kKhANGjbgtMM1DFdm4rIsSS4 TnqojLLfFjobtF4Q71jGxB0fWr/mtuVa8bD54vddcKJX3zFoYSvOFokidXlrJSQxBd N8KAuZAcGjDhA53zHmLcWxQ419JOB0h5aM46AgVj/JXtv85HaFzbPl952PqFtWtcCG qOP47npu4FmQUYlvlZnrcatvjepKQFdB3TsehGMLv2FgrV+8Dd6g1AYGbFPaV+uZpX CWOCT2fkIgnWxKg2w3yg/yEvNkNvuXOo2ZiY8ekqJVAOxhmnz63kyMtqLjRGHgnLAD EfxeNQc950JnQ== From: Eric Biggers To: netdev@vger.kernel.org Cc: linux-nvme@lists.infradead.org, linux-sctp@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, Daniel Borkmann , Marcelo Ricardo Leitner , Sagi Grimberg , Ard Biesheuvel Subject: [PATCH net-next 08/10] net: add skb_copy_and_crc32c_datagram_iter() Date: Sat, 10 May 2025 17:41:08 -0700 Message-ID: <20250511004110.145171-9-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250511004110.145171-1-ebiggers@kernel.org> References: <20250511004110.145171-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Since skb_copy_and_hash_datagram_iter() is used only with CRC32C, the crypto_ahash abstraction provides no value. Add skb_copy_and_crc32c_datagram_iter() which just calls crc32c() directly. This is faster and simpler. It also doesn't have the weird dependency issue where skb_copy_and_hash_datagram_iter() depends on CONFIG_CRYPTO_HASH=3Dy without that being expressed explicitly in the kconfig (presumably because it was too heavyweight for NET to select). The new function is conditional on the hidden boolean symbol NET_CRC32C, which selects CRC32. So it gets compiled only when something that actually needs CRC32C packet checksums is enabled, it has no implicit dependency, and it doesn't depend on the heavyweight crypto layer. Signed-off-by: Eric Biggers --- include/linux/skbuff.h | 2 ++ net/core/datagram.c | 31 +++++++++++++++++++++++++++++++ 2 files changed, 33 insertions(+) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index eac5c8dde4a5..0027f2977c02 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -4126,10 +4126,12 @@ static inline int skb_copy_datagram_msg(const struc= t sk_buff *from, int offset, int skb_copy_and_csum_datagram_msg(struct sk_buff *skb, int hlen, struct msghdr *msg); int skb_copy_and_hash_datagram_iter(const struct sk_buff *skb, int offset, struct iov_iter *to, int len, struct ahash_request *hash); +int skb_copy_and_crc32c_datagram_iter(const struct sk_buff *skb, int offse= t, + struct iov_iter *to, int len, u32 *crcp); int skb_copy_datagram_from_iter(struct sk_buff *skb, int offset, struct iov_iter *from, int len); int zerocopy_sg_from_iter(struct sk_buff *skb, struct iov_iter *frm); void skb_free_datagram(struct sock *sk, struct sk_buff *skb); int skb_kill_datagram(struct sock *sk, struct sk_buff *skb, unsigned int f= lags); diff --git a/net/core/datagram.c b/net/core/datagram.c index f0634f0cb834..47a6eef83162 100644 --- a/net/core/datagram.c +++ b/net/core/datagram.c @@ -50,10 +50,11 @@ #include #include #include #include #include +#include =20 #include #include =20 #include @@ -515,10 +516,40 @@ int skb_copy_and_hash_datagram_iter(const struct sk_b= uff *skb, int offset, return __skb_datagram_iter(skb, offset, to, len, true, hash_and_copy_to_iter, hash); } EXPORT_SYMBOL(skb_copy_and_hash_datagram_iter); =20 +#ifdef CONFIG_NET_CRC32C +static size_t crc32c_and_copy_to_iter(const void *addr, size_t bytes, + void *_crcp, struct iov_iter *i) +{ + u32 *crcp =3D _crcp; + size_t copied; + + copied =3D copy_to_iter(addr, bytes, i); + *crcp =3D crc32c(*crcp, addr, copied); + return copied; +} + +/** + * skb_copy_and_crc32c_datagram_iter - Copy datagram to an iovec iterator + * and update a CRC32C value. + * @skb: buffer to copy + * @offset: offset in the buffer to start copying from + * @to: iovec iterator to copy to + * @len: amount of data to copy from buffer to iovec + * @crcp: pointer to CRC32C value to update + */ +int skb_copy_and_crc32c_datagram_iter(const struct sk_buff *skb, int offse= t, + struct iov_iter *to, int len, u32 *crcp) +{ + return __skb_datagram_iter(skb, offset, to, len, true, + crc32c_and_copy_to_iter, crcp); +} +EXPORT_SYMBOL(skb_copy_and_crc32c_datagram_iter); +#endif /* CONFIG_NET_CRC32C */ + static size_t simple_copy_to_iter(const void *addr, size_t bytes, void *data __always_unused, struct iov_iter *i) { return copy_to_iter(addr, bytes, i); } --=20 2.49.0 From nobody Sat Feb 7 18:16:08 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A5D421BD9F0; Sun, 11 May 2025 00:41:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924099; cv=none; b=rzxQL+a7neDMlbsakvWrPiWdhPMfWwoaMr3clDO9mRUfoZEMOJ5Hqxo21pPEzpLXaxeIRxHAzYFEOPIsRpqBM6TcMF4od8L9KMKz0BEf5NKs4JZOwnF4xR/t+EOEHu8VCOxlGPDfhgfI01p9GVVWxdlQmtjyxzKRaZlyK0+APXg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924099; c=relaxed/simple; bh=HPMz7ejNYuNrnD+iwsN2clYkCQXnZS4VOH8SAts9RC8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DJldOwG0l2E6BQMrofTy+vyMeUPNH/XPwiDj73lQDIZ5c6UxVcpgeqaLUTd5Iavy7T3nBMOnJa6Sc699/ktYnXWUjlqzrq5hhixPXF+wwYvc8GSUNWr0HqDY+U2upy4BFHh+ThD+jHdluSsNyUzMgVO+TXCOck7+yLF+TIUMGg4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HA77n87U; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HA77n87U" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 43B0BC4CEEF; Sun, 11 May 2025 00:41:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746924099; bh=HPMz7ejNYuNrnD+iwsN2clYkCQXnZS4VOH8SAts9RC8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HA77n87UoGXdI+7Mcn8u0/HUQHKkjysdd7/lrecl49Yv/n6klzurimndF7ps8e9cS BeHyfCJ+2PdBKNsWeit1pB6euxfG/0W3UT8BTNvUBEkn/riZKfr5fa3wdxkJCqp7As 7O+iAtwLgIZCvm+8x3TYx87wjo9b52yK6XdPGtLfVVc/tyYIDvVcTAMcctgTMYNrIo ZewEZvNWRTa4JNAToYp7krAdRNAqC2HBpY7EPIq0ynY7rlV+ibROWF+4MeGENoKAw3 VR4QymxDRS1TVP1gzWJZKnwYA+AtvHBXmrv9Tcggmg33q9L+3feaRCAJV3u4DI+sJ9 tkbkKg2MJOWpA== From: Eric Biggers To: netdev@vger.kernel.org Cc: linux-nvme@lists.infradead.org, linux-sctp@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, Daniel Borkmann , Marcelo Ricardo Leitner , Sagi Grimberg , Ard Biesheuvel Subject: [PATCH net-next 09/10] nvme-tcp: use crc32c() and skb_copy_and_crc32c_datagram_iter() Date: Sat, 10 May 2025 17:41:09 -0700 Message-ID: <20250511004110.145171-10-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250511004110.145171-1-ebiggers@kernel.org> References: <20250511004110.145171-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Now that the crc32c() library function directly takes advantage of architecture-specific optimizations and there also now exists a function skb_copy_and_crc32c_datagram_iter(), it is unnecessary to go through the crypto_ahash API. Just use those functions. This is much simpler, and it also improves performance due to eliminating the crypto API overhead. Signed-off-by: Eric Biggers Reviewed-by: Christoph Hellwig Reviewed-by: Sagi Grimberg --- drivers/nvme/host/Kconfig | 4 +- drivers/nvme/host/tcp.c | 118 ++++++++++++-------------------------- 2 files changed, 39 insertions(+), 83 deletions(-) diff --git a/drivers/nvme/host/Kconfig b/drivers/nvme/host/Kconfig index 4d64b6935bb9..7dca58f0a237 100644 --- a/drivers/nvme/host/Kconfig +++ b/drivers/nvme/host/Kconfig @@ -82,13 +82,13 @@ config NVME_FC =20 config NVME_TCP tristate "NVM Express over Fabrics TCP host driver" depends on INET depends on BLOCK + select CRC32 + select NET_CRC32C select NVME_FABRICS - select CRYPTO - select CRYPTO_CRC32C help This provides support for the NVMe over Fabrics protocol using the TCP transport. This allows you to use remote block devices exported using the NVMe protocol set. =20 diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c index aba365f97cf6..d32e168036ec 100644 --- a/drivers/nvme/host/tcp.c +++ b/drivers/nvme/host/tcp.c @@ -6,19 +6,19 @@ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt #include #include #include #include +#include #include #include #include #include #include #include #include #include -#include #include #include =20 #include "nvme.h" #include "fabrics.h" @@ -166,12 +166,12 @@ struct nvme_tcp_queue { bool rd_enabled; =20 bool hdr_digest; bool data_digest; bool tls_enabled; - struct ahash_request *rcv_hash; - struct ahash_request *snd_hash; + u32 rcv_crc; + u32 snd_crc; __le32 exp_ddgst; __le32 recv_ddgst; struct completion tls_complete; int tls_err; struct page_frag_cache pf_cache; @@ -454,36 +454,40 @@ nvme_tcp_fetch_request(struct nvme_tcp_queue *queue) =20 list_del(&req->entry); return req; } =20 -static inline void nvme_tcp_ddgst_final(struct ahash_request *hash, - __le32 *dgst) +static inline void nvme_tcp_ddgst_init(u32 *crcp) { - ahash_request_set_crypt(hash, NULL, (u8 *)dgst, 0); - crypto_ahash_final(hash); + *crcp =3D ~0; } =20 -static inline void nvme_tcp_ddgst_update(struct ahash_request *hash, - struct page *page, off_t off, size_t len) +static inline void nvme_tcp_ddgst_update(u32 *crcp, + struct page *page, size_t off, size_t len) { - struct scatterlist sg; + page +=3D off / PAGE_SIZE; + off %=3D PAGE_SIZE; + while (len) { + const void *vaddr =3D kmap_local_page(page); + size_t n =3D min(len, (size_t)PAGE_SIZE - off); =20 - sg_init_table(&sg, 1); - sg_set_page(&sg, page, len, off); - ahash_request_set_crypt(hash, &sg, NULL, len); - crypto_ahash_update(hash); + *crcp =3D crc32c(*crcp, vaddr + off, n); + kunmap_local(vaddr); + page++; + off =3D 0; + len -=3D n; + } } =20 -static inline void nvme_tcp_hdgst(struct ahash_request *hash, - void *pdu, size_t len) +static inline void nvme_tcp_ddgst_final(u32 *crcp, __le32 *ddgst) { - struct scatterlist sg; + *ddgst =3D cpu_to_le32(~*crcp); +} =20 - sg_init_one(&sg, pdu, len); - ahash_request_set_crypt(hash, &sg, pdu + len, len); - crypto_ahash_digest(hash); +static inline void nvme_tcp_hdgst(void *pdu, size_t len) +{ + put_unaligned_le32(~crc32c(~0, pdu, len), pdu + len); } =20 static int nvme_tcp_verify_hdgst(struct nvme_tcp_queue *queue, void *pdu, size_t pdu_len) { @@ -497,11 +501,11 @@ static int nvme_tcp_verify_hdgst(struct nvme_tcp_queu= e *queue, nvme_tcp_queue_id(queue)); return -EPROTO; } =20 recv_digest =3D *(__le32 *)(pdu + hdr->hlen); - nvme_tcp_hdgst(queue->rcv_hash, pdu, pdu_len); + nvme_tcp_hdgst(pdu, pdu_len); exp_digest =3D *(__le32 *)(pdu + hdr->hlen); if (recv_digest !=3D exp_digest) { dev_err(queue->ctrl->ctrl.device, "header digest error: recv %#x expected %#x\n", le32_to_cpu(recv_digest), le32_to_cpu(exp_digest)); @@ -524,11 +528,11 @@ static int nvme_tcp_check_ddgst(struct nvme_tcp_queue= *queue, void *pdu) dev_err(queue->ctrl->ctrl.device, "queue %d: data digest flag is cleared\n", nvme_tcp_queue_id(queue)); return -EPROTO; } - crypto_ahash_init(queue->rcv_hash); + nvme_tcp_ddgst_init(&queue->rcv_crc); =20 return 0; } =20 static void nvme_tcp_exit_request(struct blk_mq_tag_set *set, @@ -924,12 +928,12 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *= queue, struct sk_buff *skb, /* we can read only from what is left in this bio */ recv_len =3D min_t(size_t, recv_len, iov_iter_count(&req->iter)); =20 if (queue->data_digest) - ret =3D skb_copy_and_hash_datagram_iter(skb, *offset, - &req->iter, recv_len, queue->rcv_hash); + ret =3D skb_copy_and_crc32c_datagram_iter(skb, *offset, + &req->iter, recv_len, &queue->rcv_crc); else ret =3D skb_copy_datagram_iter(skb, *offset, &req->iter, recv_len); if (ret) { dev_err(queue->ctrl->ctrl.device, @@ -943,11 +947,12 @@ static int nvme_tcp_recv_data(struct nvme_tcp_queue *= queue, struct sk_buff *skb, queue->data_remaining -=3D recv_len; } =20 if (!queue->data_remaining) { if (queue->data_digest) { - nvme_tcp_ddgst_final(queue->rcv_hash, &queue->exp_ddgst); + nvme_tcp_ddgst_final(&queue->rcv_crc, + &queue->exp_ddgst); queue->ddgst_remaining =3D NVME_TCP_DIGEST_LENGTH; } else { if (pdu->hdr.flags & NVME_TCP_F_DATA_SUCCESS) { nvme_tcp_end_request(rq, le16_to_cpu(req->status)); @@ -1145,11 +1150,11 @@ static int nvme_tcp_try_send_data(struct nvme_tcp_r= equest *req) ret =3D sock_sendmsg(queue->sock, &msg); if (ret <=3D 0) return ret; =20 if (queue->data_digest) - nvme_tcp_ddgst_update(queue->snd_hash, page, + nvme_tcp_ddgst_update(&queue->snd_crc, page, offset, ret); =20 /* * update the request iterator except for the last payload send * in the request where we don't want to modify it as we may @@ -1159,11 +1164,11 @@ static int nvme_tcp_try_send_data(struct nvme_tcp_r= equest *req) nvme_tcp_advance_req(req, ret); =20 /* fully successful last send in current PDU */ if (last && ret =3D=3D len) { if (queue->data_digest) { - nvme_tcp_ddgst_final(queue->snd_hash, + nvme_tcp_ddgst_final(&queue->snd_crc, &req->ddgst); req->state =3D NVME_TCP_SEND_DDGST; req->offset =3D 0; } else { if (h2cdata_left) @@ -1192,11 +1197,11 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tc= p_request *req) msg.msg_flags |=3D MSG_MORE; else msg.msg_flags |=3D MSG_EOR; =20 if (queue->hdr_digest && !req->offset) - nvme_tcp_hdgst(queue->snd_hash, pdu, sizeof(*pdu)); + nvme_tcp_hdgst(pdu, sizeof(*pdu)); =20 bvec_set_virt(&bvec, (void *)pdu + req->offset, len); iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, len); ret =3D sock_sendmsg(queue->sock, &msg); if (unlikely(ret <=3D 0)) @@ -1205,11 +1210,11 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tc= p_request *req) len -=3D ret; if (!len) { if (inline_data) { req->state =3D NVME_TCP_SEND_DATA; if (queue->data_digest) - crypto_ahash_init(queue->snd_hash); + nvme_tcp_ddgst_init(&queue->snd_crc); } else { nvme_tcp_done_send_req(queue); } return 1; } @@ -1227,11 +1232,11 @@ static int nvme_tcp_try_send_data_pdu(struct nvme_t= cp_request *req) u8 hdgst =3D nvme_tcp_hdgst_len(queue); int len =3D sizeof(*pdu) - req->offset + hdgst; int ret; =20 if (queue->hdr_digest && !req->offset) - nvme_tcp_hdgst(queue->snd_hash, pdu, sizeof(*pdu)); + nvme_tcp_hdgst(pdu, sizeof(*pdu)); =20 if (!req->h2cdata_left) msg.msg_flags |=3D MSG_SPLICE_PAGES; =20 bvec_set_virt(&bvec, (void *)pdu + req->offset, len); @@ -1242,11 +1247,11 @@ static int nvme_tcp_try_send_data_pdu(struct nvme_t= cp_request *req) =20 len -=3D ret; if (!len) { req->state =3D NVME_TCP_SEND_DATA; if (queue->data_digest) - crypto_ahash_init(queue->snd_hash); + nvme_tcp_ddgst_init(&queue->snd_crc); return 1; } req->offset +=3D ret; =20 return -EAGAIN; @@ -1382,45 +1387,10 @@ static void nvme_tcp_io_work(struct work_struct *w) } while (!time_after(jiffies, deadline)); /* quota is exhausted */ =20 queue_work_on(queue->io_cpu, nvme_tcp_wq, &queue->io_work); } =20 -static void nvme_tcp_free_crypto(struct nvme_tcp_queue *queue) -{ - struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(queue->rcv_hash); - - ahash_request_free(queue->rcv_hash); - ahash_request_free(queue->snd_hash); - crypto_free_ahash(tfm); -} - -static int nvme_tcp_alloc_crypto(struct nvme_tcp_queue *queue) -{ - struct crypto_ahash *tfm; - - tfm =3D crypto_alloc_ahash("crc32c", 0, CRYPTO_ALG_ASYNC); - if (IS_ERR(tfm)) - return PTR_ERR(tfm); - - queue->snd_hash =3D ahash_request_alloc(tfm, GFP_KERNEL); - if (!queue->snd_hash) - goto free_tfm; - ahash_request_set_callback(queue->snd_hash, 0, NULL, NULL); - - queue->rcv_hash =3D ahash_request_alloc(tfm, GFP_KERNEL); - if (!queue->rcv_hash) - goto free_snd_hash; - ahash_request_set_callback(queue->rcv_hash, 0, NULL, NULL); - - return 0; -free_snd_hash: - ahash_request_free(queue->snd_hash); -free_tfm: - crypto_free_ahash(tfm); - return -ENOMEM; -} - static void nvme_tcp_free_async_req(struct nvme_tcp_ctrl *ctrl) { struct nvme_tcp_request *async =3D &ctrl->async_req; =20 page_frag_free(async->pdu); @@ -1449,13 +1419,10 @@ static void nvme_tcp_free_queue(struct nvme_ctrl *n= ctrl, int qid) unsigned int noreclaim_flag; =20 if (!test_and_clear_bit(NVME_TCP_Q_ALLOCATED, &queue->flags)) return; =20 - if (queue->hdr_digest || queue->data_digest) - nvme_tcp_free_crypto(queue); - page_frag_cache_drain(&queue->pf_cache); =20 noreclaim_flag =3D memalloc_noreclaim_save(); /* ->sock will be released by fput() */ fput(queue->sock->file); @@ -1865,25 +1832,17 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *n= ctrl, int qid, } } =20 queue->hdr_digest =3D nctrl->opts->hdr_digest; queue->data_digest =3D nctrl->opts->data_digest; - if (queue->hdr_digest || queue->data_digest) { - ret =3D nvme_tcp_alloc_crypto(queue); - if (ret) { - dev_err(nctrl->device, - "failed to allocate queue %d crypto\n", qid); - goto err_sock; - } - } =20 rcv_pdu_size =3D sizeof(struct nvme_tcp_rsp_pdu) + nvme_tcp_hdgst_len(queue); queue->pdu =3D kmalloc(rcv_pdu_size, GFP_KERNEL); if (!queue->pdu) { ret =3D -ENOMEM; - goto err_crypto; + goto err_sock; } =20 dev_dbg(nctrl->device, "connecting queue %d\n", nvme_tcp_queue_id(queue)); =20 @@ -1912,13 +1871,10 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *n= ctrl, int qid, =20 err_init_connect: kernel_sock_shutdown(queue->sock, SHUT_RDWR); err_rcv_pdu: kfree(queue->pdu); -err_crypto: - if (queue->hdr_digest || queue->data_digest) - nvme_tcp_free_crypto(queue); err_sock: /* ->sock will be released by fput() */ fput(queue->sock->file); queue->sock =3D NULL; err_destroy_mutex: --=20 2.49.0 From nobody Sat Feb 7 18:16:08 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 641521D5CEA; Sun, 11 May 2025 00:41:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924100; cv=none; b=r/6bnBbgkWuaxqmaoQoQEXLJM/FU1DjZMp4VQWyRVbB9+plvKIlUbXZdpnCOdUJxMQdWDlqNr1nOQZc3k2qc58lulGyCwdgBg6r1Vb5Y6GywTuysjKLQvtj0fshzqLILlZfYACotvjYAwnjqx1XKbi/y9h85m919gtUtCcqDVHo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746924100; c=relaxed/simple; bh=1M/Uxj/GlzhwSrBWvWBpAp/iDXzpGWDqYvfOyMyRoL4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pIwBYzN1AFi5rP+szGhrwM50ZLt2qU/Icspw8rgQFxkCcHeVIsJzrQt0zT1FhN0b2B4IiKLYHsc386UGvrkPdfNmfkBQAOo2ahP1wOth7LbfK4QYFK0YiACVk9m/gQnT69O2BU3IqM9AHwZnuht0sGmcnC2kcNQd0u+hQfAAtE0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NZOX/xwM; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NZOX/xwM" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 631B4C4CEEE; Sun, 11 May 2025 00:41:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746924100; bh=1M/Uxj/GlzhwSrBWvWBpAp/iDXzpGWDqYvfOyMyRoL4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NZOX/xwMxCnXrfMk9nmIM7VO+YI5DMYHUfROWZ/IvYpTGiJ3pS0j87VESZ6xW8kOs A5uDYUAkFqYXzdpzyeng1AI6bchZZ6NmZKvS5t+WQWxNXX9a2yGCpeNKC8dB71eEyX GpBvoEOACOz2AHtm9Y+hBGYNuikSYitAsXa0DpuuiD7xNb3RoKA2jsehoXky/JTvYR 9ptSMpk7mJu4rh3cg/h/r5l042QmusHMbSP5ulzjQMAZBc+UgSwtrcAPi714mInHtX pi7Sqjxuu46+6wuIByHBS6WbVlsJnhDjcQ9CjSpFgfjY3DB5BlAaLJ3tFUyehKvot8 258DbM8dcgFoA== From: Eric Biggers To: netdev@vger.kernel.org Cc: linux-nvme@lists.infradead.org, linux-sctp@vger.kernel.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, Daniel Borkmann , Marcelo Ricardo Leitner , Sagi Grimberg , Ard Biesheuvel Subject: [PATCH net-next 10/10] net: remove skb_copy_and_hash_datagram_iter() Date: Sat, 10 May 2025 17:41:10 -0700 Message-ID: <20250511004110.145171-11-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250511004110.145171-1-ebiggers@kernel.org> References: <20250511004110.145171-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Now that skb_copy_and_hash_datagram_iter() is no longer used, remove it. Signed-off-by: Eric Biggers --- include/linux/skbuff.h | 4 ---- net/core/datagram.c | 37 ------------------------------------- 2 files changed, 41 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 0027f2977c02..98cecd3a2fb8 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -272,11 +272,10 @@ /* return minimum truesize of one skb containing X bytes of data */ #define SKB_TRUESIZE(X) ((X) + \ SKB_DATA_ALIGN(sizeof(struct sk_buff)) + \ SKB_DATA_ALIGN(sizeof(struct skb_shared_info))) =20 -struct ahash_request; struct net_device; struct scatterlist; struct pipe_inode_info; struct iov_iter; struct napi_struct; @@ -4123,13 +4122,10 @@ static inline int skb_copy_datagram_msg(const struc= t sk_buff *from, int offset, { return skb_copy_datagram_iter(from, offset, &msg->msg_iter, size); } int skb_copy_and_csum_datagram_msg(struct sk_buff *skb, int hlen, struct msghdr *msg); -int skb_copy_and_hash_datagram_iter(const struct sk_buff *skb, int offset, - struct iov_iter *to, int len, - struct ahash_request *hash); int skb_copy_and_crc32c_datagram_iter(const struct sk_buff *skb, int offse= t, struct iov_iter *to, int len, u32 *crcp); int skb_copy_datagram_from_iter(struct sk_buff *skb, int offset, struct iov_iter *from, int len); int zerocopy_sg_from_iter(struct sk_buff *skb, struct iov_iter *frm); diff --git a/net/core/datagram.c b/net/core/datagram.c index 47a6eef83162..b83731f11fad 100644 --- a/net/core/datagram.c +++ b/net/core/datagram.c @@ -60,11 +60,10 @@ #include #include #include #include #include -#include =20 /* * Is a socket 'connection oriented' ? */ static inline int connection_based(struct sock *sk) @@ -480,46 +479,10 @@ static int __skb_datagram_iter(const struct sk_buff *= skb, int offset, goto fault; =20 return 0; } =20 -static size_t hash_and_copy_to_iter(const void *addr, size_t bytes, void *= hashp, - struct iov_iter *i) -{ -#ifdef CONFIG_CRYPTO_HASH - struct ahash_request *hash =3D hashp; - struct scatterlist sg; - size_t copied; - - copied =3D copy_to_iter(addr, bytes, i); - sg_init_one(&sg, addr, copied); - ahash_request_set_crypt(hash, &sg, NULL, copied); - crypto_ahash_update(hash); - return copied; -#else - return 0; -#endif -} - -/** - * skb_copy_and_hash_datagram_iter - Copy datagram to an iovec iterator - * and update a hash. - * @skb: buffer to copy - * @offset: offset in the buffer to start copying from - * @to: iovec iterator to copy to - * @len: amount of data to copy from buffer to iovec - * @hash: hash request to update - */ -int skb_copy_and_hash_datagram_iter(const struct sk_buff *skb, int offset, - struct iov_iter *to, int len, - struct ahash_request *hash) -{ - return __skb_datagram_iter(skb, offset, to, len, true, - hash_and_copy_to_iter, hash); -} -EXPORT_SYMBOL(skb_copy_and_hash_datagram_iter); - #ifdef CONFIG_NET_CRC32C static size_t crc32c_and_copy_to_iter(const void *addr, size_t bytes, void *_crcp, struct iov_iter *i) { u32 *crcp =3D _crcp; --=20 2.49.0