From nobody Fri Dec 19 07:31:42 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F595217660; Sat, 7 Jun 2025 20:07:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326849; cv=none; b=EAgQtJijrCg+qUEt2qGonKqtcuP/HZgrJ/M2oYizlYlJ4OMVVi3781mq4hwe0UevsBddsoc8k2DHu6+sZ/PyVn2NUpEpIaI1FuSKOmS33iQrUyPJr4GZygMrujDP4a6t6oezhoxu9DqtF5PqmIDTdsB22e6Ro7H653/Su8Mxe/M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326849; c=relaxed/simple; bh=ul+IYG3QWn7TMt6jwp439C6v/yOYv8veAHOVkfUu8l4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tcYLmnFmTApYAmmguwONd2kcZdNv97kmertX9oqsji6wQZW03a64yCB8ae8HlWTr1/r/6cdvxPxJhrX63e3G4AvqPuQeNJEAc6IKdbiSiWVybT3EoDijceXqO4efzmxjK1LX2EDeEGqgn1Wm3Que1JED7V9K9VUpUoAqsWXwlJw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Tl0d//6C; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Tl0d//6C" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4418FC4CEF0; Sat, 7 Jun 2025 20:07:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1749326848; bh=ul+IYG3QWn7TMt6jwp439C6v/yOYv8veAHOVkfUu8l4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Tl0d//6CPdPlG3/4K4xGeIeOptjkdCK+LvF1RDYSDJB4TuDkHyzyAFAy+0o8jdYQD ZWO9MtQqxMp+mvgQFYw51Ub/8YitCHjeMaLDttKZ6869jmFQlDCdxOwq5Ju93rJEA1 gfLxWzTirOtlQ++xq8LkB5J2fLk0Ueme2p96jnVzKOeDYv0NZTetnjR2Ea8BfpIKFu lR8dnWC/fbq55jPpeIpU9aLLrGT805No7GgA6KRHsqhF0iSJg0+TCkvEdxXYgM54th KGHrYoe4ZCXLd20Ai4X/asdmD4ikmulcOEmMj3hcnUyXWd6JcdP1uwfwEni1zc6C7y TT61g4vhWl6BQ== From: Eric Biggers To: linux-kernel@vger.kernel.org Cc: linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Linus Torvalds Subject: [PATCH v2 01/12] lib/crc: move files into lib/crc/ Date: Sat, 7 Jun 2025 13:04:43 -0700 Message-ID: <20250607200454.73587-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250607200454.73587-1-ebiggers@kernel.org> References: <20250607200454.73587-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Move all CRC files in lib/ into a subdirectory lib/crc/ to keep them from cluttering up the main lib/ directory. Signed-off-by: Eric Biggers Reviewed-by: Martin K. Petersen --- Documentation/core-api/kernel-api.rst | 14 ++-- MAINTAINERS | 3 +- include/linux/crc64.h | 3 - lib/Kconfig | 87 +------------------- lib/Kconfig.debug | 21 ----- lib/Makefile | 32 +------- lib/crc/.gitignore | 5 ++ lib/crc/Kconfig | 111 ++++++++++++++++++++++++++ lib/crc/Makefile | 32 ++++++++ lib/{ =3D> crc}/crc-ccitt.c | 3 - lib/{ =3D> crc}/crc-itu-t.c | 0 lib/{ =3D> crc}/crc-t10dif.c | 0 lib/{ =3D> crc}/crc16.c | 0 lib/{ =3D> crc}/crc32.c | 0 lib/{ =3D> crc}/crc4.c | 0 lib/{ =3D> crc}/crc64.c | 0 lib/{ =3D> crc}/crc7.c | 0 lib/{ =3D> crc}/crc8.c | 0 lib/{ =3D> crc}/gen_crc32table.c | 4 +- lib/{ =3D> crc}/gen_crc64table.c | 11 +-- lib/crc/tests/Makefile | 2 + lib/{ =3D> crc}/tests/crc_kunit.c | 0 lib/tests/Makefile | 1 - 23 files changed, 164 insertions(+), 165 deletions(-) create mode 100644 lib/crc/.gitignore create mode 100644 lib/crc/Kconfig create mode 100644 lib/crc/Makefile rename lib/{ =3D> crc}/crc-ccitt.c (98%) rename lib/{ =3D> crc}/crc-itu-t.c (100%) rename lib/{ =3D> crc}/crc-t10dif.c (100%) rename lib/{ =3D> crc}/crc16.c (100%) rename lib/{ =3D> crc}/crc32.c (100%) rename lib/{ =3D> crc}/crc4.c (100%) rename lib/{ =3D> crc}/crc64.c (100%) rename lib/{ =3D> crc}/crc7.c (100%) rename lib/{ =3D> crc}/crc8.c (100%) rename lib/{ =3D> crc}/gen_crc32table.c (95%) rename lib/{ =3D> crc}/gen_crc64table.c (81%) create mode 100644 lib/crc/tests/Makefile rename lib/{ =3D> crc}/tests/crc_kunit.c (100%) diff --git a/Documentation/core-api/kernel-api.rst b/Documentation/core-api= /kernel-api.rst index ae92a2571388a..c4642d9f13a9c 100644 --- a/Documentation/core-api/kernel-api.rst +++ b/Documentation/core-api/kernel-api.rst @@ -134,28 +134,28 @@ Arithmetic Overflow Checking :internal: =20 CRC Functions ------------- =20 -.. kernel-doc:: lib/crc4.c +.. kernel-doc:: lib/crc/crc4.c :export: =20 -.. kernel-doc:: lib/crc7.c +.. kernel-doc:: lib/crc/crc7.c :export: =20 -.. kernel-doc:: lib/crc8.c +.. kernel-doc:: lib/crc/crc8.c :export: =20 -.. kernel-doc:: lib/crc16.c +.. kernel-doc:: lib/crc/crc16.c :export: =20 -.. kernel-doc:: lib/crc32.c +.. kernel-doc:: lib/crc/crc32.c =20 -.. kernel-doc:: lib/crc-ccitt.c +.. kernel-doc:: lib/crc/crc-ccitt.c :export: =20 -.. kernel-doc:: lib/crc-itu-t.c +.. kernel-doc:: lib/crc/crc-itu-t.c :export: =20 Base 2 log and power Functions ------------------------------ =20 diff --git a/MAINTAINERS b/MAINTAINERS index 44c70071acb08..05ff204b016c5 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -6358,12 +6358,11 @@ L: linux-crypto@vger.kernel.org S: Maintained T: git https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git = crc-next F: Documentation/staging/crc* F: arch/*/lib/crc* F: include/linux/crc* -F: lib/crc* -F: lib/tests/crc_kunit.c +F: lib/crc/ F: scripts/gen-crc-consts.py =20 CREATIVE SB0540 M: Bastien Nocera L: linux-input@vger.kernel.org diff --git a/include/linux/crc64.h b/include/linux/crc64.h index 41de30b907dff..b6aa290a79312 100644 --- a/include/linux/crc64.h +++ b/include/linux/crc64.h @@ -1,9 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 */ -/* - * See lib/crc64.c for the related specification and polynomial arithmetic. - */ #ifndef _LINUX_CRC64_H #define _LINUX_CRC64_H =20 #include =20 diff --git a/lib/Kconfig b/lib/Kconfig index 6c1b8f1842678..3db88ab0d2b4c 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -134,98 +134,13 @@ config TRACE_MMIO_ACCESS depends on TRACING && ARCH_HAVE_TRACE_MMIO_ACCESS help Create tracepoints for MMIO read/write operations. These trace events can be used for logging all MMIO read/write operations. =20 +source "lib/crc/Kconfig" source "lib/crypto/Kconfig" =20 -config CRC_CCITT - tristate - help - The CRC-CCITT library functions. Select this if your module uses any - of the functions from . - -config CRC16 - tristate - help - The CRC16 library functions. Select this if your module uses any of - the functions from . - -config CRC_T10DIF - tristate - help - The CRC-T10DIF library functions. Select this if your module uses - any of the functions from . - -config ARCH_HAS_CRC_T10DIF - bool - -config CRC_T10DIF_ARCH - tristate - default CRC_T10DIF if ARCH_HAS_CRC_T10DIF && CRC_OPTIMIZATIONS - -config CRC_ITU_T - tristate - help - The CRC-ITU-T library functions. Select this if your module uses - any of the functions from . - -config CRC32 - tristate - select BITREVERSE - help - The CRC32 library functions. Select this if your module uses any of - the functions from or . - -config ARCH_HAS_CRC32 - bool - -config CRC32_ARCH - tristate - default CRC32 if ARCH_HAS_CRC32 && CRC_OPTIMIZATIONS - -config CRC64 - tristate - help - The CRC64 library functions. Select this if your module uses any of - the functions from . - -config ARCH_HAS_CRC64 - bool - -config CRC64_ARCH - tristate - default CRC64 if ARCH_HAS_CRC64 && CRC_OPTIMIZATIONS - -config CRC4 - tristate - help - The CRC4 library functions. Select this if your module uses any of - the functions from . - -config CRC7 - tristate - help - The CRC7 library functions. Select this if your module uses any of - the functions from . - -config CRC8 - tristate - help - The CRC8 library functions. Select this if your module uses any of - the functions from . - -config CRC_OPTIMIZATIONS - bool "Enable optimized CRC implementations" if EXPERT - default y - help - Disabling this option reduces code size slightly by disabling the - architecture-optimized implementations of any CRC variants that are - enabled. CRC checksumming performance may get much slower. - - Keep this enabled unless you're really trying to minimize the size of - the kernel. =20 config XXHASH tristate =20 config AUDIT_GENERIC diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index ebe33181b6e6e..3fda96761adbc 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -2899,31 +2899,10 @@ config HW_BREAKPOINT_KUNIT_TEST help Tests for hw_breakpoint constraints accounting. =20 If unsure, say N. =20 -config CRC_KUNIT_TEST - tristate "KUnit tests for CRC functions" if !KUNIT_ALL_TESTS - depends on KUNIT - default KUNIT_ALL_TESTS - select CRC7 - select CRC16 - select CRC_T10DIF - select CRC32 - select CRC64 - help - Unit tests for the CRC library functions. - - This is intended to help people writing architecture-specific - optimized versions. If unsure, say N. - -config CRC_BENCHMARK - bool "Benchmark for the CRC functions" - depends on CRC_KUNIT_TEST - help - Include benchmarks in the KUnit test suite for the CRC functions. - config SIPHASH_KUNIT_TEST tristate "Perform selftest on siphash functions" if !KUNIT_ALL_TESTS depends on KUNIT default KUNIT_ALL_TESTS help diff --git a/lib/Makefile b/lib/Makefile index c38582f187dd8..14a5928bb57f5 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -120,11 +120,11 @@ CFLAGS_kobject_uevent.o +=3D -DDEBUG endif =20 obj-$(CONFIG_DEBUG_INFO_REDUCED) +=3D debug_info.o CFLAGS_debug_info.o +=3D $(call cc-option, -femit-struct-debug-detailed=3D= any) =20 -obj-y +=3D math/ crypto/ tests/ vdso/ +obj-y +=3D math/ crc/ crypto/ tests/ vdso/ =20 obj-$(CONFIG_GENERIC_IOMAP) +=3D iomap.o obj-$(CONFIG_HAS_IOMEM) +=3D iomap_copy.o devres.o obj-$(CONFIG_CHECK_SIGNATURE) +=3D check_signature.o obj-$(CONFIG_DEBUG_LOCKING_API_SELFTESTS) +=3D locking-selftest.o @@ -146,19 +146,10 @@ obj-$(CONFIG_DEBUG_OBJECTS) +=3D debugobjects.o =20 obj-$(CONFIG_BITREVERSE) +=3D bitrev.o obj-$(CONFIG_LINEAR_RANGES) +=3D linear_ranges.o obj-$(CONFIG_PACKING) +=3D packing.o obj-$(CONFIG_PACKING_KUNIT_TEST) +=3D packing_test.o -obj-$(CONFIG_CRC_CCITT) +=3D crc-ccitt.o -obj-$(CONFIG_CRC16) +=3D crc16.o -obj-$(CONFIG_CRC_T10DIF)+=3D crc-t10dif.o -obj-$(CONFIG_CRC_ITU_T) +=3D crc-itu-t.o -obj-$(CONFIG_CRC32) +=3D crc32.o -obj-$(CONFIG_CRC64) +=3D crc64.o -obj-$(CONFIG_CRC4) +=3D crc4.o -obj-$(CONFIG_CRC7) +=3D crc7.o -obj-$(CONFIG_CRC8) +=3D crc8.o obj-$(CONFIG_XXHASH) +=3D xxhash.o obj-$(CONFIG_GENERIC_ALLOCATOR) +=3D genalloc.o =20 obj-$(CONFIG_842_COMPRESS) +=3D 842/ obj-$(CONFIG_842_DECOMPRESS) +=3D 842/ @@ -292,31 +283,10 @@ obj-$(CONFIG_PERCPU_TEST) +=3D percpu_test.o obj-$(CONFIG_ASN1) +=3D asn1_decoder.o obj-$(CONFIG_ASN1_ENCODER) +=3D asn1_encoder.o =20 obj-$(CONFIG_FONT_SUPPORT) +=3D fonts/ =20 -hostprogs :=3D gen_crc32table -hostprogs +=3D gen_crc64table -clean-files :=3D crc32table.h -clean-files +=3D crc64table.h - -$(obj)/crc32.o: $(obj)/crc32table.h - -quiet_cmd_crc32 =3D GEN $@ - cmd_crc32 =3D $< > $@ - -$(obj)/crc32table.h: $(obj)/gen_crc32table - $(call cmd,crc32) - -$(obj)/crc64.o: $(obj)/crc64table.h - -quiet_cmd_crc64 =3D GEN $@ - cmd_crc64 =3D $< > $@ - -$(obj)/crc64table.h: $(obj)/gen_crc64table - $(call cmd,crc64) - # # Build a fast OID lookip registry from include/linux/oid_registry.h # obj-$(CONFIG_OID_REGISTRY) +=3D oid_registry.o =20 diff --git a/lib/crc/.gitignore b/lib/crc/.gitignore new file mode 100644 index 0000000000000..a9e48103c9fb1 --- /dev/null +++ b/lib/crc/.gitignore @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0-only +/crc32table.h +/crc64table.h +/gen_crc32table +/gen_crc64table diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig new file mode 100644 index 0000000000000..e0e7168b74c75 --- /dev/null +++ b/lib/crc/Kconfig @@ -0,0 +1,111 @@ +# SPDX-License-Identifier: GPL-2.0-only + +# Kconfig for the kernel's cyclic redundancy check (CRC) library code + +config CRC4 + tristate + help + The CRC4 library functions. Select this if your module uses any of + the functions from . + +config CRC7 + tristate + help + The CRC7 library functions. Select this if your module uses any of + the functions from . + +config CRC8 + tristate + help + The CRC8 library functions. Select this if your module uses any of + the functions from . + +config CRC16 + tristate + help + The CRC16 library functions. Select this if your module uses any of + the functions from . + +config CRC_CCITT + tristate + help + The CRC-CCITT library functions. Select this if your module uses any + of the functions from . + +config CRC_ITU_T + tristate + help + The CRC-ITU-T library functions. Select this if your module uses + any of the functions from . + +config CRC_T10DIF + tristate + help + The CRC-T10DIF library functions. Select this if your module uses + any of the functions from . + +config ARCH_HAS_CRC_T10DIF + bool + +config CRC_T10DIF_ARCH + tristate + default CRC_T10DIF if ARCH_HAS_CRC_T10DIF && CRC_OPTIMIZATIONS + +config CRC32 + tristate + select BITREVERSE + help + The CRC32 library functions. Select this if your module uses any of + the functions from or . + +config ARCH_HAS_CRC32 + bool + +config CRC32_ARCH + tristate + default CRC32 if ARCH_HAS_CRC32 && CRC_OPTIMIZATIONS + +config CRC64 + tristate + help + The CRC64 library functions. Select this if your module uses any of + the functions from . + +config ARCH_HAS_CRC64 + bool + +config CRC64_ARCH + tristate + default CRC64 if ARCH_HAS_CRC64 && CRC_OPTIMIZATIONS + +config CRC_OPTIMIZATIONS + bool "Enable optimized CRC implementations" if EXPERT + default y + help + Disabling this option reduces code size slightly by disabling the + architecture-optimized implementations of any CRC variants that are + enabled. CRC checksumming performance may get much slower. + + Keep this enabled unless you're really trying to minimize the size of + the kernel. + +config CRC_KUNIT_TEST + tristate "KUnit tests for CRC functions" if !KUNIT_ALL_TESTS + depends on KUNIT + default KUNIT_ALL_TESTS + select CRC7 + select CRC16 + select CRC_T10DIF + select CRC32 + select CRC64 + help + Unit tests for the CRC library functions. + + This is intended to help people writing architecture-specific + optimized versions. If unsure, say N. + +config CRC_BENCHMARK + bool "Benchmark for the CRC functions" + depends on CRC_KUNIT_TEST + help + Include benchmarks in the KUnit test suite for the CRC functions. diff --git a/lib/crc/Makefile b/lib/crc/Makefile new file mode 100644 index 0000000000000..ff4c30dda4528 --- /dev/null +++ b/lib/crc/Makefile @@ -0,0 +1,32 @@ +# SPDX-License-Identifier: GPL-2.0-only + +# Makefile for the kernel's cyclic redundancy check (CRC) library code + +obj-$(CONFIG_CRC4) +=3D crc4.o +obj-$(CONFIG_CRC7) +=3D crc7.o +obj-$(CONFIG_CRC8) +=3D crc8.o +obj-$(CONFIG_CRC16) +=3D crc16.o +obj-$(CONFIG_CRC_CCITT) +=3D crc-ccitt.o +obj-$(CONFIG_CRC_ITU_T) +=3D crc-itu-t.o +obj-$(CONFIG_CRC_T10DIF) +=3D crc-t10dif.o +obj-$(CONFIG_CRC32) +=3D crc32.o +obj-$(CONFIG_CRC64) +=3D crc64.o +obj-y +=3D tests/ + +hostprogs :=3D gen_crc32table gen_crc64table +clean-files :=3D crc32table.h crc64table.h + +$(obj)/crc32.o: $(obj)/crc32table.h +$(obj)/crc64.o: $(obj)/crc64table.h + +quiet_cmd_crc32 =3D GEN $@ + cmd_crc32 =3D $< > $@ + +quiet_cmd_crc64 =3D GEN $@ + cmd_crc64 =3D $< > $@ + +$(obj)/crc32table.h: $(obj)/gen_crc32table + $(call cmd,crc32) + +$(obj)/crc64table.h: $(obj)/gen_crc64table + $(call cmd,crc64) diff --git a/lib/crc-ccitt.c b/lib/crc/crc-ccitt.c similarity index 98% rename from lib/crc-ccitt.c rename to lib/crc/crc-ccitt.c index 9cddf35d3b66e..8d2bc419230b3 100644 --- a/lib/crc-ccitt.c +++ b/lib/crc/crc-ccitt.c @@ -1,9 +1,6 @@ // SPDX-License-Identifier: GPL-2.0-only -/* - * linux/lib/crc-ccitt.c - */ =20 #include #include #include =20 diff --git a/lib/crc-itu-t.c b/lib/crc/crc-itu-t.c similarity index 100% rename from lib/crc-itu-t.c rename to lib/crc/crc-itu-t.c diff --git a/lib/crc-t10dif.c b/lib/crc/crc-t10dif.c similarity index 100% rename from lib/crc-t10dif.c rename to lib/crc/crc-t10dif.c diff --git a/lib/crc16.c b/lib/crc/crc16.c similarity index 100% rename from lib/crc16.c rename to lib/crc/crc16.c diff --git a/lib/crc32.c b/lib/crc/crc32.c similarity index 100% rename from lib/crc32.c rename to lib/crc/crc32.c diff --git a/lib/crc4.c b/lib/crc/crc4.c similarity index 100% rename from lib/crc4.c rename to lib/crc/crc4.c diff --git a/lib/crc64.c b/lib/crc/crc64.c similarity index 100% rename from lib/crc64.c rename to lib/crc/crc64.c diff --git a/lib/crc7.c b/lib/crc/crc7.c similarity index 100% rename from lib/crc7.c rename to lib/crc/crc7.c diff --git a/lib/crc8.c b/lib/crc/crc8.c similarity index 100% rename from lib/crc8.c rename to lib/crc/crc8.c diff --git a/lib/gen_crc32table.c b/lib/crc/gen_crc32table.c similarity index 95% rename from lib/gen_crc32table.c rename to lib/crc/gen_crc32table.c index 6d03425b849e4..9a7f31658e355 100644 --- a/lib/gen_crc32table.c +++ b/lib/crc/gen_crc32table.c @@ -1,9 +1,9 @@ // SPDX-License-Identifier: GPL-2.0 #include -#include "../include/linux/crc32poly.h" -#include "../include/generated/autoconf.h" +#include "../../include/linux/crc32poly.h" +#include "../../include/generated/autoconf.h" #include =20 static uint32_t crc32table_le[256]; static uint32_t crc32table_be[256]; static uint32_t crc32ctable_le[256]; diff --git a/lib/gen_crc64table.c b/lib/crc/gen_crc64table.c similarity index 81% rename from lib/gen_crc64table.c rename to lib/crc/gen_crc64table.c index e05a4230a0a08..f2be9f62bab71 100644 --- a/lib/gen_crc64table.c +++ b/lib/crc/gen_crc64table.c @@ -1,16 +1,9 @@ // SPDX-License-Identifier: GPL-2.0 /* - * Generate lookup table for the table-driven CRC64 calculation. - * - * gen_crc64table is executed in kernel build time and generates - * lib/crc64table.h. This header is included by lib/crc64.c for - * the table-driven CRC64 calculation. - * - * See lib/crc64.c for more information about which specification - * and polynomial arithmetic that gen_crc64table.c follows to - * generate the lookup table. + * This host program runs at kernel build time and generates the lookup ta= bles + * used by the generic CRC64 code. * * Copyright 2018 SUSE Linux. * Author: Coly Li */ #include diff --git a/lib/crc/tests/Makefile b/lib/crc/tests/Makefile new file mode 100644 index 0000000000000..65f63c318ef5e --- /dev/null +++ b/lib/crc/tests/Makefile @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_CRC_KUNIT_TEST) +=3D crc_kunit.o diff --git a/lib/tests/crc_kunit.c b/lib/crc/tests/crc_kunit.c similarity index 100% rename from lib/tests/crc_kunit.c rename to lib/crc/tests/crc_kunit.c diff --git a/lib/tests/Makefile b/lib/tests/Makefile index 56d6450144828..741d3ac2cba25 100644 --- a/lib/tests/Makefile +++ b/lib/tests/Makefile @@ -8,11 +8,10 @@ obj-$(CONFIG_BITFIELD_KUNIT) +=3D bitfield_kunit.o obj-$(CONFIG_BITS_TEST) +=3D test_bits.o obj-$(CONFIG_BLACKHOLE_DEV_KUNIT_TEST) +=3D blackhole_dev_kunit.o obj-$(CONFIG_CHECKSUM_KUNIT) +=3D checksum_kunit.o obj-$(CONFIG_CMDLINE_KUNIT_TEST) +=3D cmdline_kunit.o obj-$(CONFIG_CPUMASK_KUNIT_TEST) +=3D cpumask_kunit.o -obj-$(CONFIG_CRC_KUNIT_TEST) +=3D crc_kunit.o CFLAGS_fortify_kunit.o +=3D $(call cc-disable-warning, unsequenced) CFLAGS_fortify_kunit.o +=3D $(call cc-disable-warning, stringop-overread) CFLAGS_fortify_kunit.o +=3D $(call cc-disable-warning, stringop-truncation) CFLAGS_fortify_kunit.o +=3D $(DISABLE_STRUCTLEAK_PLUGIN) obj-$(CONFIG_FORTIFY_KUNIT_TEST) +=3D fortify_kunit.o --=20 2.49.0 From nobody Fri Dec 19 07:31:42 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7991218821; Sat, 7 Jun 2025 20:07:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326849; cv=none; b=JIfWYoKCBddwnCEuc3Nd5eXeeCVVEvvCBcypij4hrjvUdfdyUotpaVCndBjjY7V0lZq+j4fpqV+Ig+7iUwT1TZXZcjl9dtSEMCgulDKHkACa67LC7+uOZVnT2Fq0gKkSwZCC51MObShiSLTdxi5lX5Wj568nlTm5rNLbYC8sEiM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326849; c=relaxed/simple; bh=yLUp/jnRDPRJYe2ElHSUe7azlBkiNfeDfNdSKo75+q8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=b8hITkiMIFeh3VJ0CX5/a8aSX/sTM88uyGYLWxtiKj9EMd5HAC3sBhueRemT8eekGoiYnYAlVqosbRVDmCt38JkZRoVo32v944YLl6U0oJOvdcRB33jIJCYXh53v+54wZ+kIgVjpAKjoBUXnq1tKDPk651uzobHsfgEXpwnespE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=f1FhhuAi; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="f1FhhuAi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D8B85C4CEF4; Sat, 7 Jun 2025 20:07:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1749326849; bh=yLUp/jnRDPRJYe2ElHSUe7azlBkiNfeDfNdSKo75+q8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=f1FhhuAiTrZ3n1+2RxOIsHAMGSJ/VjobP7NJfQNN0j/NBFbwNgGFUyusJmRjkKbXS sqWh1Vqgcop8SjWbs6LXUgNLslH9vJAaJtRSlUYVblMBS0RiVdEhJomtVaMvtV8ZOa PEHblLWQdMquqQZGguOsB3PGpaG1D6iMt3eqwjzhDYP/ljmb10RAveSRuslbu0TNpU aXvLzJN+nKuiGXJjGUMUMH4iyFPeV5RkG3XLHj627tTQZ05KelNWJFE8NxxO5DTTAN /pAmDXiq2dwYEPh4cjU29Xt59sFxxmxlJaZ8E/hzV8XcP6IZjjcVaxjEm3WyRFm4Z6 8pchieYF/5jfg== From: Eric Biggers To: linux-kernel@vger.kernel.org Cc: linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Linus Torvalds Subject: [PATCH v2 02/12] lib/crc: prepare for arch-optimized code in subdirs of lib/crc/ Date: Sat, 7 Jun 2025 13:04:44 -0700 Message-ID: <20250607200454.73587-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250607200454.73587-1-ebiggers@kernel.org> References: <20250607200454.73587-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Rework how lib/crc/ supports arch-optimized code. First, instead of the arch-optimized CRC code being in arch/$(SRCARCH)/lib/, it will now be in lib/crc/$(SRCARCH)/. Second, the API functions (e.g. crc32c()), arch-optimized functions (e.g. crc32c_arch()), and generic functions (e.g. crc32c_base()) will now be part of a single module for each CRC type, allowing better inlining and dead code elimination. The second change is made possible by the first. As an example, consider CONFIG_CRC32=3Dm on x86. We'll now have just crc32.ko instead of both crc32-x86.ko and crc32.ko. The two modules were already coupled together and always both got loaded together via direct symbol dependency, so the separation provided no benefit. Note: later I'd like to apply the same design to lib/crypto/ too, where often the API functions are out-of-line so this will work even better. In those cases, for each algorithm we currently have 3 modules all coupled together, e.g. libsha256.ko, libsha256-generic.ko, and sha256-x86.ko. We should have just one, inline things properly, and rely on the compiler's dead code elimination to decide the inclusion of the generic code instead of manually setting it via kconfig. Having arch-specific code outside arch/ was somewhat controversial when Zinc proposed it back in 2018. But I don't think the concerns are warranted. It's better from a technical perspective, as it enables the improvements mentioned above. This model is already successfully used in other places in the kernel such as lib/raid6/. The community of each architecture still remains free to work on the code, even if it's not in arch/. At the time there was also a desire to put the library code in the same files as the old-school crypto API, but that was a mistake; now that the library is separate, that's no longer a constraint either. Signed-off-by: Eric Biggers Reviewed-by: Martin K. Petersen --- Documentation/core-api/kernel-api.rst | 2 +- MAINTAINERS | 1 - include/linux/crc-t10dif.h | 10 +-- include/linux/crc32.h | 30 +-------- include/linux/crc64.h | 19 +----- lib/crc/Kconfig | 12 ++-- lib/crc/Makefile | 20 +++++- lib/crc/{crc-t10dif.c =3D> crc-t10dif-main.c} | 37 ++++++++--- lib/crc/{crc32.c =3D> crc32-main.c} | 69 +++++++++++++++++---- lib/crc/{crc64.c =3D> crc64-main.c} | 47 +++++++++++--- 10 files changed, 158 insertions(+), 89 deletions(-) rename lib/crc/{crc-t10dif.c =3D> crc-t10dif-main.c} (78%) rename lib/crc/{crc32.c =3D> crc32-main.c} (58%) rename lib/crc/{crc64.c =3D> crc64-main.c} (66%) diff --git a/Documentation/core-api/kernel-api.rst b/Documentation/core-api= /kernel-api.rst index c4642d9f13a9c..9c8370891a39b 100644 --- a/Documentation/core-api/kernel-api.rst +++ b/Documentation/core-api/kernel-api.rst @@ -146,11 +146,11 @@ CRC Functions :export: =20 .. kernel-doc:: lib/crc/crc16.c :export: =20 -.. kernel-doc:: lib/crc/crc32.c +.. kernel-doc:: lib/crc/crc32-main.c =20 .. kernel-doc:: lib/crc/crc-ccitt.c :export: =20 .. kernel-doc:: lib/crc/crc-itu-t.c diff --git a/MAINTAINERS b/MAINTAINERS index 05ff204b016c5..a729d80594557 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -6356,11 +6356,10 @@ M: Eric Biggers R: Ard Biesheuvel L: linux-crypto@vger.kernel.org S: Maintained T: git https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git = crc-next F: Documentation/staging/crc* -F: arch/*/lib/crc* F: include/linux/crc* F: lib/crc/ F: scripts/gen-crc-consts.py =20 CREATIVE SB0540 diff --git a/include/linux/crc-t10dif.h b/include/linux/crc-t10dif.h index a559fdff3f7e2..ecc8bc2dd7f4c 100644 --- a/include/linux/crc-t10dif.h +++ b/include/linux/crc-t10dif.h @@ -2,19 +2,11 @@ #ifndef _LINUX_CRC_T10DIF_H #define _LINUX_CRC_T10DIF_H =20 #include =20 -u16 crc_t10dif_arch(u16 crc, const u8 *p, size_t len); -u16 crc_t10dif_generic(u16 crc, const u8 *p, size_t len); - -static inline u16 crc_t10dif_update(u16 crc, const u8 *p, size_t len) -{ - if (IS_ENABLED(CONFIG_CRC_T10DIF_ARCH)) - return crc_t10dif_arch(crc, p, len); - return crc_t10dif_generic(crc, p, len); -} +u16 crc_t10dif_update(u16 crc, const u8 *p, size_t len); =20 static inline u16 crc_t10dif(const u8 *p, size_t len) { return crc_t10dif_update(0, p, len); } diff --git a/include/linux/crc32.h b/include/linux/crc32.h index 36bbc0405aa04..22dbe7144eb44 100644 --- a/include/linux/crc32.h +++ b/include/linux/crc32.h @@ -3,37 +3,13 @@ #define _LINUX_CRC32_H =20 #include #include =20 -u32 crc32_le_arch(u32 crc, const u8 *p, size_t len); -u32 crc32_le_base(u32 crc, const u8 *p, size_t len); -u32 crc32_be_arch(u32 crc, const u8 *p, size_t len); -u32 crc32_be_base(u32 crc, const u8 *p, size_t len); -u32 crc32c_arch(u32 crc, const u8 *p, size_t len); -u32 crc32c_base(u32 crc, const u8 *p, size_t len); - -static inline u32 crc32_le(u32 crc, const void *p, size_t len) -{ - if (IS_ENABLED(CONFIG_CRC32_ARCH)) - return crc32_le_arch(crc, p, len); - return crc32_le_base(crc, p, len); -} - -static inline u32 crc32_be(u32 crc, const void *p, size_t len) -{ - if (IS_ENABLED(CONFIG_CRC32_ARCH)) - return crc32_be_arch(crc, p, len); - return crc32_be_base(crc, p, len); -} - -static inline u32 crc32c(u32 crc, const void *p, size_t len) -{ - if (IS_ENABLED(CONFIG_CRC32_ARCH)) - return crc32c_arch(crc, p, len); - return crc32c_base(crc, p, len); -} +u32 crc32_le(u32 crc, const void *p, size_t len); +u32 crc32_be(u32 crc, const void *p, size_t len); +u32 crc32c(u32 crc, const void *p, size_t len); =20 /* * crc32_optimizations() returns flags that indicate which CRC32 library * functions are using architecture-specific optimizations. Unlike * IS_ENABLED(CONFIG_CRC32_ARCH) it takes into account the different CRC32 diff --git a/include/linux/crc64.h b/include/linux/crc64.h index b6aa290a79312..fc0c06ab1993c 100644 --- a/include/linux/crc64.h +++ b/include/linux/crc64.h @@ -2,28 +2,18 @@ #ifndef _LINUX_CRC64_H #define _LINUX_CRC64_H =20 #include =20 -u64 crc64_be_arch(u64 crc, const u8 *p, size_t len); -u64 crc64_be_generic(u64 crc, const u8 *p, size_t len); -u64 crc64_nvme_arch(u64 crc, const u8 *p, size_t len); -u64 crc64_nvme_generic(u64 crc, const u8 *p, size_t len); - /** * crc64_be - Calculate bitwise big-endian ECMA-182 CRC64 * @crc: seed value for computation. 0 or (u64)~0 for a new CRC calculatio= n, * or the previous crc64 value if computing incrementally. * @p: pointer to buffer over which CRC64 is run * @len: length of buffer @p */ -static inline u64 crc64_be(u64 crc, const void *p, size_t len) -{ - if (IS_ENABLED(CONFIG_CRC64_ARCH)) - return crc64_be_arch(crc, p, len); - return crc64_be_generic(crc, p, len); -} +u64 crc64_be(u64 crc, const void *p, size_t len); =20 /** * crc64_nvme - Calculate CRC64-NVME * @crc: seed value for computation. 0 for a new CRC calculation, or the * previous crc64 value if computing incrementally. @@ -31,13 +21,8 @@ static inline u64 crc64_be(u64 crc, const void *p, size_= t len) * @len: length of buffer @p * * This computes the CRC64 defined in the NVME NVM Command Set Specificati= on, * *including the bitwise inversion at the beginning and end*. */ -static inline u64 crc64_nvme(u64 crc, const void *p, size_t len) -{ - if (IS_ENABLED(CONFIG_CRC64_ARCH)) - return ~crc64_nvme_arch(~crc, p, len); - return ~crc64_nvme_generic(~crc, p, len); -} +u64 crc64_nvme(u64 crc, const void *p, size_t len); =20 #endif /* _LINUX_CRC64_H */ diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig index e0e7168b74c75..e049f01a4d2c8 100644 --- a/lib/crc/Kconfig +++ b/lib/crc/Kconfig @@ -46,12 +46,12 @@ config CRC_T10DIF =20 config ARCH_HAS_CRC_T10DIF bool =20 config CRC_T10DIF_ARCH - tristate - default CRC_T10DIF if ARCH_HAS_CRC_T10DIF && CRC_OPTIMIZATIONS + bool + depends on CRC_T10DIF && CRC_OPTIMIZATIONS =20 config CRC32 tristate select BITREVERSE help @@ -60,12 +60,12 @@ config CRC32 =20 config ARCH_HAS_CRC32 bool =20 config CRC32_ARCH - tristate - default CRC32 if ARCH_HAS_CRC32 && CRC_OPTIMIZATIONS + bool + depends on CRC32 && CRC_OPTIMIZATIONS =20 config CRC64 tristate help The CRC64 library functions. Select this if your module uses any of @@ -73,12 +73,12 @@ config CRC64 =20 config ARCH_HAS_CRC64 bool =20 config CRC64_ARCH - tristate - default CRC64 if ARCH_HAS_CRC64 && CRC_OPTIMIZATIONS + bool + depends on CRC64 && CRC_OPTIMIZATIONS =20 config CRC_OPTIMIZATIONS bool "Enable optimized CRC implementations" if EXPERT default y help diff --git a/lib/crc/Makefile b/lib/crc/Makefile index ff4c30dda4528..926edc3b035f6 100644 --- a/lib/crc/Makefile +++ b/lib/crc/Makefile @@ -6,20 +6,36 @@ obj-$(CONFIG_CRC4) +=3D crc4.o obj-$(CONFIG_CRC7) +=3D crc7.o obj-$(CONFIG_CRC8) +=3D crc8.o obj-$(CONFIG_CRC16) +=3D crc16.o obj-$(CONFIG_CRC_CCITT) +=3D crc-ccitt.o obj-$(CONFIG_CRC_ITU_T) +=3D crc-itu-t.o + obj-$(CONFIG_CRC_T10DIF) +=3D crc-t10dif.o +crc-t10dif-y :=3D crc-t10dif-main.o +ifeq ($(CONFIG_CRC_T10DIF_ARCH),y) +CFLAGS_crc-t10dif-main.o +=3D -I$(src)/$(SRCARCH) +endif + obj-$(CONFIG_CRC32) +=3D crc32.o +crc32-y :=3D crc32-main.o +ifeq ($(CONFIG_CRC32_ARCH),y) +CFLAGS_crc32-main.o +=3D -I$(src)/$(SRCARCH) +endif + obj-$(CONFIG_CRC64) +=3D crc64.o +crc64-y :=3D crc64-main.o +ifeq ($(CONFIG_CRC64_ARCH),y) +CFLAGS_crc64-main.o +=3D -I$(src)/$(SRCARCH) +endif + obj-y +=3D tests/ =20 hostprogs :=3D gen_crc32table gen_crc64table clean-files :=3D crc32table.h crc64table.h =20 -$(obj)/crc32.o: $(obj)/crc32table.h -$(obj)/crc64.o: $(obj)/crc64table.h +$(obj)/crc32-main.o: $(obj)/crc32table.h +$(obj)/crc64-main.o: $(obj)/crc64table.h =20 quiet_cmd_crc32 =3D GEN $@ cmd_crc32 =3D $< > $@ =20 quiet_cmd_crc64 =3D GEN $@ diff --git a/lib/crc/crc-t10dif.c b/lib/crc/crc-t10dif-main.c similarity index 78% rename from lib/crc/crc-t10dif.c rename to lib/crc/crc-t10dif-main.c index 311c2ab829f15..08f1f1042a712 100644 --- a/lib/crc/crc-t10dif.c +++ b/lib/crc/crc-t10dif-main.c @@ -48,18 +48,41 @@ static const u16 t10_dif_crc_table[256] =3D { 0xA415, 0x2FA2, 0x38CC, 0xB37B, 0x1610, 0x9DA7, 0x8AC9, 0x017E, 0x1F65, 0x94D2, 0x83BC, 0x080B, 0xAD60, 0x26D7, 0x31B9, 0xBA0E, 0xF0D8, 0x7B6F, 0x6C01, 0xE7B6, 0x42DD, 0xC96A, 0xDE04, 0x55B3 }; =20 -u16 crc_t10dif_generic(u16 crc, const u8 *p, size_t len) +static inline __maybe_unused u16 +crc_t10dif_generic(u16 crc, const u8 *p, size_t len) { - size_t i; + while (len--) + crc =3D (crc << 8) ^ t10_dif_crc_table[(crc >> 8) ^ *p++]; + return crc; +} =20 - for (i =3D 0; i < len; i++) - crc =3D (crc << 8) ^ t10_dif_crc_table[(crc >> 8) ^ p[i]]; +#ifdef CONFIG_CRC_T10DIF_ARCH +#include "crc-t10dif.h" /* $(SRCARCH)/crc-t10dif.h */ +#else +#define crc_t10dif_arch crc_t10dif_generic +#endif =20 - return crc; +u16 crc_t10dif_update(u16 crc, const u8 *p, size_t len) +{ + return crc_t10dif_arch(crc, p, len); +} +EXPORT_SYMBOL(crc_t10dif_update); + +#ifdef crc_t10dif_mod_init_arch +static int __init crc_t10dif_mod_init(void) +{ + crc_t10dif_mod_init_arch(); + return 0; +} +subsys_initcall(crc_t10dif_mod_init); + +static void __exit crc_t10dif_mod_exit(void) +{ } -EXPORT_SYMBOL(crc_t10dif_generic); +module_exit(crc_t10dif_mod_exit); +#endif =20 -MODULE_DESCRIPTION("T10 DIF CRC calculation"); +MODULE_DESCRIPTION("CRC-T10DIF library functions"); MODULE_LICENSE("GPL"); diff --git a/lib/crc/crc32.c b/lib/crc/crc32-main.c similarity index 58% rename from lib/crc/crc32.c rename to lib/crc/crc32-main.c index 6811b37df2aad..b86ee66075d0e 100644 --- a/lib/crc/crc32.c +++ b/lib/crc/crc32-main.c @@ -28,32 +28,77 @@ #include #include =20 #include "crc32table.h" =20 -MODULE_AUTHOR("Matt Domsch "); -MODULE_DESCRIPTION("Various CRC32 calculations"); -MODULE_LICENSE("GPL"); - -u32 crc32_le_base(u32 crc, const u8 *p, size_t len) +static inline __maybe_unused u32 +crc32_le_base(u32 crc, const u8 *p, size_t len) { while (len--) crc =3D (crc >> 8) ^ crc32table_le[(crc & 255) ^ *p++]; return crc; } -EXPORT_SYMBOL(crc32_le_base); =20 -u32 crc32c_base(u32 crc, const u8 *p, size_t len) +static inline __maybe_unused u32 +crc32_be_base(u32 crc, const u8 *p, size_t len) { while (len--) - crc =3D (crc >> 8) ^ crc32ctable_le[(crc & 255) ^ *p++]; + crc =3D (crc << 8) ^ crc32table_be[(crc >> 24) ^ *p++]; return crc; } -EXPORT_SYMBOL(crc32c_base); =20 -u32 crc32_be_base(u32 crc, const u8 *p, size_t len) +static inline __maybe_unused u32 +crc32c_base(u32 crc, const u8 *p, size_t len) { while (len--) - crc =3D (crc << 8) ^ crc32table_be[(crc >> 24) ^ *p++]; + crc =3D (crc >> 8) ^ crc32ctable_le[(crc & 255) ^ *p++]; return crc; } -EXPORT_SYMBOL(crc32_be_base); + +#ifdef CONFIG_CRC32_ARCH +#include "crc32.h" /* $(SRCARCH)/crc32.h */ + +u32 crc32_optimizations(void) +{ + return crc32_optimizations_arch(); +} +EXPORT_SYMBOL(crc32_optimizations); +#else +#define crc32_le_arch crc32_le_base +#define crc32_be_arch crc32_be_base +#define crc32c_arch crc32c_base +#endif + +u32 crc32_le(u32 crc, const void *p, size_t len) +{ + return crc32_le_arch(crc, p, len); +} +EXPORT_SYMBOL(crc32_le); + +u32 crc32_be(u32 crc, const void *p, size_t len) +{ + return crc32_be_arch(crc, p, len); +} +EXPORT_SYMBOL(crc32_be); + +u32 crc32c(u32 crc, const void *p, size_t len) +{ + return crc32c_arch(crc, p, len); +} +EXPORT_SYMBOL(crc32c); + +#ifdef crc32_mod_init_arch +static int __init crc32_mod_init(void) +{ + crc32_mod_init_arch(); + return 0; +} +subsys_initcall(crc32_mod_init); + +static void __exit crc32_mod_exit(void) +{ +} +module_exit(crc32_mod_exit); +#endif + +MODULE_DESCRIPTION("CRC32 library functions"); +MODULE_LICENSE("GPL"); diff --git a/lib/crc/crc64.c b/lib/crc/crc64-main.c similarity index 66% rename from lib/crc/crc64.c rename to lib/crc/crc64-main.c index 5b1b17057f0ae..e4a6d879e84c3 100644 --- a/lib/crc/crc64.c +++ b/lib/crc/crc64-main.c @@ -36,23 +36,56 @@ #include #include #include #include "crc64table.h" =20 -MODULE_DESCRIPTION("CRC64 calculations"); -MODULE_LICENSE("GPL v2"); - -u64 crc64_be_generic(u64 crc, const u8 *p, size_t len) +static inline __maybe_unused u64 +crc64_be_generic(u64 crc, const u8 *p, size_t len) { while (len--) crc =3D (crc << 8) ^ crc64table[(crc >> 56) ^ *p++]; return crc; } -EXPORT_SYMBOL_GPL(crc64_be_generic); =20 -u64 crc64_nvme_generic(u64 crc, const u8 *p, size_t len) +static inline __maybe_unused u64 +crc64_nvme_generic(u64 crc, const u8 *p, size_t len) { while (len--) crc =3D (crc >> 8) ^ crc64nvmetable[(crc & 0xff) ^ *p++]; return crc; } -EXPORT_SYMBOL_GPL(crc64_nvme_generic); + +#ifdef CONFIG_CRC64_ARCH +#include "crc64.h" /* $(SRCARCH)/crc64.h */ +#else +#define crc64_be_arch crc64_be_generic +#define crc64_nvme_arch crc64_nvme_generic +#endif + +u64 crc64_be(u64 crc, const void *p, size_t len) +{ + return crc64_be_arch(crc, p, len); +} +EXPORT_SYMBOL_GPL(crc64_be); + +u64 crc64_nvme(u64 crc, const void *p, size_t len) +{ + return ~crc64_nvme_arch(~crc, p, len); +} +EXPORT_SYMBOL_GPL(crc64_nvme); + +#ifdef crc64_mod_init_arch +static int __init crc64_mod_init(void) +{ + crc64_mod_init_arch(); + return 0; +} +subsys_initcall(crc64_mod_init); + +static void __exit crc64_mod_exit(void) +{ +} +module_exit(crc64_mod_exit); +#endif + +MODULE_DESCRIPTION("CRC64 library functions"); +MODULE_LICENSE("GPL"); --=20 2.49.0 From nobody Fri Dec 19 07:31:42 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C0F3218EBF; Sat, 7 Jun 2025 20:07:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326850; cv=none; b=GxI9oGz+6tGCWS1KP/NrvbRcdON1Lt1d96XeP/jsAlcpOfZGiFc/tLCgAJrIgRKr+VvqAWXwi4B9uM28VmNF2u6coTaZQ0AJ0+qU2UZRWS/AtiAUopLV1zYoprj5BrNQVEN9xMp2Q1QyRhX2DZ5QlNenNGb1MMSGUBKqedyTGs8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326850; c=relaxed/simple; bh=sDKX0MxyKMIL+HZ/mXu69f6T4IOkE11+BdTbwDYdB/Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uF4+M+zQ5fi/0kv2b38EbKhIJizzBKD+vuo+lsuqMtqA5ISdgN9bvLPa6ZiLHaOwpgFjnEW7Ain8k7cIoLuOBmkayvbirNJIFOsnatOULs+tQTyiWHTfW+CptRaIQceiUT7fg9PbtT1uFIXT5iCy+RzP3NnjwXGghli3Q+SX78Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nTfXrhqp; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nTfXrhqp" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 678B4C4CEF6; Sat, 7 Jun 2025 20:07:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1749326849; bh=sDKX0MxyKMIL+HZ/mXu69f6T4IOkE11+BdTbwDYdB/Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nTfXrhqpFlnkZTgQrji+Og+HrxqlZiXD94N/b4vQ4dcwfjOCpaHzuq+5nzA7kNFMw OEkYSRG2dd58HHKjnA0EraCPfAMqk20IUxbiQ7x1g/INvmbiLTdkEDxU1m+ah+5Qto egViG3MIi9m5LmMoD4YIIE3SdjKdp+ub+HsGVVbNB/B31RASYhvKw77kJYT7ef2cdN yPF5nOTYwYDW6kfgpV+XLEM55tulrzlsOCOYqF/2lxDtF54k3ooNGvsw53az5iuumM zQ0Ac4aUcCCpjyrpej4Rrb7nWEjEdvHdysCxxHsXX5R0pnjiJ+d2cMkS9WrvlmTlz5 tQ7NM6IqVseJA== From: Eric Biggers To: linux-kernel@vger.kernel.org Cc: linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Linus Torvalds Subject: [PATCH v2 03/12] lib/crc/arm: migrate arm-optimized CRC code into lib/crc/ Date: Sat, 7 Jun 2025 13:04:45 -0700 Message-ID: <20250607200454.73587-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250607200454.73587-1-ebiggers@kernel.org> References: <20250607200454.73587-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Move the arm-optimized CRC code from arch/arm/lib/crc* into its new location in lib/crc/arm/, and wire it up in the new way. This new way of organizing the CRC code eliminates the need to artificially split the code for each CRC variant into separate arch and generic modules, enabling better inlining and dead code elimination. For more details, see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/". Signed-off-by: Eric Biggers Reviewed-by: Martin K. Petersen --- arch/arm/Kconfig | 2 - arch/arm/lib/Makefile | 6 --- lib/crc/Kconfig | 2 + lib/crc/Makefile | 2 + .../arm/lib =3D> lib/crc/arm}/crc-t10dif-core.S | 0 .../crc-t10dif.c =3D> lib/crc/arm/crc-t10dif.h | 23 ++--------- {arch/arm/lib =3D> lib/crc/arm}/crc32-core.S | 0 arch/arm/lib/crc32.c =3D> lib/crc/arm/crc32.h | 38 ++++--------------- 8 files changed, 15 insertions(+), 58 deletions(-) rename {arch/arm/lib =3D> lib/crc/arm}/crc-t10dif-core.S (100%) rename arch/arm/lib/crc-t10dif.c =3D> lib/crc/arm/crc-t10dif.h (70%) rename {arch/arm/lib =3D> lib/crc/arm}/crc32-core.S (100%) rename arch/arm/lib/crc32.c =3D> lib/crc/arm/crc32.h (69%) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 3072731fe09c5..c531b49aa98ea 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -6,12 +6,10 @@ config ARM select ARCH_CORRECT_STACKTRACE_ON_KRETPROBE if HAVE_KRETPROBES && FRAME_P= OINTER && !ARM_UNWIND select ARCH_HAS_BINFMT_FLAT select ARCH_HAS_CACHE_LINE_SIZE if OF select ARCH_HAS_CPU_CACHE_ALIASING select ARCH_HAS_CPU_FINALIZE_INIT if MMU - select ARCH_HAS_CRC32 if KERNEL_MODE_NEON - select ARCH_HAS_CRC_T10DIF if KERNEL_MODE_NEON select ARCH_HAS_CURRENT_STACK_POINTER select ARCH_HAS_DEBUG_VIRTUAL if MMU select ARCH_HAS_DMA_ALLOC if MMU select ARCH_HAS_DMA_OPS select ARCH_HAS_DMA_WRITE_COMBINE if !ARM_DMA_MEM_BUFFERABLE diff --git a/arch/arm/lib/Makefile b/arch/arm/lib/Makefile index 91ea0e29107af..d1b9ea2023648 100644 --- a/arch/arm/lib/Makefile +++ b/arch/arm/lib/Makefile @@ -45,11 +45,5 @@ ifeq ($(CONFIG_KERNEL_MODE_NEON),y) CFLAGS_xor-neon.o +=3D $(CC_FLAGS_FPU) obj-$(CONFIG_XOR_BLOCKS) +=3D xor-neon.o endif =20 obj-$(CONFIG_FUNCTION_ERROR_INJECTION) +=3D error-inject.o - -obj-$(CONFIG_CRC32_ARCH) +=3D crc32-arm.o -crc32-arm-y :=3D crc32.o crc32-core.o - -obj-$(CONFIG_CRC_T10DIF_ARCH) +=3D crc-t10dif-arm.o -crc-t10dif-arm-y :=3D crc-t10dif.o crc-t10dif-core.o diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig index e049f01a4d2c8..edd1b99098003 100644 --- a/lib/crc/Kconfig +++ b/lib/crc/Kconfig @@ -48,10 +48,11 @@ config ARCH_HAS_CRC_T10DIF bool =20 config CRC_T10DIF_ARCH bool depends on CRC_T10DIF && CRC_OPTIMIZATIONS + default y if ARM && KERNEL_MODE_NEON =20 config CRC32 tristate select BITREVERSE help @@ -62,10 +63,11 @@ config ARCH_HAS_CRC32 bool =20 config CRC32_ARCH bool depends on CRC32 && CRC_OPTIMIZATIONS + default y if ARM && KERNEL_MODE_NEON =20 config CRC64 tristate help The CRC64 library functions. Select this if your module uses any of diff --git a/lib/crc/Makefile b/lib/crc/Makefile index 926edc3b035f6..c72d351be6cb8 100644 --- a/lib/crc/Makefile +++ b/lib/crc/Makefile @@ -11,16 +11,18 @@ obj-$(CONFIG_CRC_ITU_T) +=3D crc-itu-t.o =20 obj-$(CONFIG_CRC_T10DIF) +=3D crc-t10dif.o crc-t10dif-y :=3D crc-t10dif-main.o ifeq ($(CONFIG_CRC_T10DIF_ARCH),y) CFLAGS_crc-t10dif-main.o +=3D -I$(src)/$(SRCARCH) +crc-t10dif-$(CONFIG_ARM) +=3D arm/crc-t10dif-core.o endif =20 obj-$(CONFIG_CRC32) +=3D crc32.o crc32-y :=3D crc32-main.o ifeq ($(CONFIG_CRC32_ARCH),y) CFLAGS_crc32-main.o +=3D -I$(src)/$(SRCARCH) +crc32-$(CONFIG_ARM) +=3D arm/crc32-core.o endif =20 obj-$(CONFIG_CRC64) +=3D crc64.o crc64-y :=3D crc64-main.o ifeq ($(CONFIG_CRC64_ARCH),y) diff --git a/arch/arm/lib/crc-t10dif-core.S b/lib/crc/arm/crc-t10dif-core.S similarity index 100% rename from arch/arm/lib/crc-t10dif-core.S rename to lib/crc/arm/crc-t10dif-core.S diff --git a/arch/arm/lib/crc-t10dif.c b/lib/crc/arm/crc-t10dif.h similarity index 70% rename from arch/arm/lib/crc-t10dif.c rename to lib/crc/arm/crc-t10dif.h index 1093f8ec13b0b..2edf7e9681d05 100644 --- a/arch/arm/lib/crc-t10dif.c +++ b/lib/crc/arm/crc-t10dif.h @@ -3,16 +3,10 @@ * Accelerated CRC-T10DIF using ARM NEON and Crypto Extensions instructions * * Copyright (C) 2016 Linaro Ltd */ =20 -#include -#include -#include -#include -#include - #include =20 #include #include =20 @@ -23,11 +17,11 @@ static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_pmu= ll); =20 asmlinkage u16 crc_t10dif_pmull64(u16 init_crc, const u8 *buf, size_t len); asmlinkage void crc_t10dif_pmull8(u16 init_crc, const u8 *buf, size_t len, u8 out[16]); =20 -u16 crc_t10dif_arch(u16 crc, const u8 *data, size_t length) +static inline u16 crc_t10dif_arch(u16 crc, const u8 *data, size_t length) { if (length >=3D CRC_T10DIF_PMULL_CHUNK_SIZE) { if (static_branch_likely(&have_pmull)) { if (crypto_simd_usable()) { kernel_neon_begin(); @@ -47,26 +41,15 @@ u16 crc_t10dif_arch(u16 crc, const u8 *data, size_t len= gth) return crc_t10dif_generic(0, buf, sizeof(buf)); } } return crc_t10dif_generic(crc, data, length); } -EXPORT_SYMBOL(crc_t10dif_arch); =20 -static int __init crc_t10dif_arm_init(void) +#define crc_t10dif_mod_init_arch crc_t10dif_mod_init_arch +static inline void crc_t10dif_mod_init_arch(void) { if (elf_hwcap & HWCAP_NEON) { static_branch_enable(&have_neon); if (elf_hwcap2 & HWCAP2_PMULL) static_branch_enable(&have_pmull); } - return 0; } -subsys_initcall(crc_t10dif_arm_init); - -static void __exit crc_t10dif_arm_exit(void) -{ -} -module_exit(crc_t10dif_arm_exit); - -MODULE_AUTHOR("Ard Biesheuvel "); -MODULE_DESCRIPTION("Accelerated CRC-T10DIF using ARM NEON and Crypto Exten= sions"); -MODULE_LICENSE("GPL v2"); diff --git a/arch/arm/lib/crc32-core.S b/lib/crc/arm/crc32-core.S similarity index 100% rename from arch/arm/lib/crc32-core.S rename to lib/crc/arm/crc32-core.S diff --git a/arch/arm/lib/crc32.c b/lib/crc/arm/crc32.h similarity index 69% rename from arch/arm/lib/crc32.c rename to lib/crc/arm/crc32.h index f2bef8849c7c3..018007e162a2b 100644 --- a/arch/arm/lib/crc32.c +++ b/lib/crc/arm/crc32.h @@ -4,15 +4,10 @@ * * Copyright (C) 2016 Linaro Ltd */ =20 #include -#include -#include -#include -#include -#include =20 #include =20 #include #include @@ -27,18 +22,18 @@ asmlinkage u32 crc32_pmull_le(const u8 buf[], u32 len, = u32 init_crc); asmlinkage u32 crc32_armv8_le(u32 init_crc, const u8 buf[], u32 len); =20 asmlinkage u32 crc32c_pmull_le(const u8 buf[], u32 len, u32 init_crc); asmlinkage u32 crc32c_armv8_le(u32 init_crc, const u8 buf[], u32 len); =20 -static u32 crc32_le_scalar(u32 crc, const u8 *p, size_t len) +static inline u32 crc32_le_scalar(u32 crc, const u8 *p, size_t len) { if (static_branch_likely(&have_crc32)) return crc32_armv8_le(crc, p, len); return crc32_le_base(crc, p, len); } =20 -u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) +static inline u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) { if (len >=3D PMULL_MIN_LEN + 15 && static_branch_likely(&have_pmull) && crypto_simd_usable()) { size_t n =3D -(uintptr_t)p & 15; =20 @@ -55,20 +50,19 @@ u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) p +=3D n; len -=3D n; } return crc32_le_scalar(crc, p, len); } -EXPORT_SYMBOL(crc32_le_arch); =20 -static u32 crc32c_scalar(u32 crc, const u8 *p, size_t len) +static inline u32 crc32c_scalar(u32 crc, const u8 *p, size_t len) { if (static_branch_likely(&have_crc32)) return crc32c_armv8_le(crc, p, len); return crc32c_base(crc, p, len); } =20 -u32 crc32c_arch(u32 crc, const u8 *p, size_t len) +static inline u32 crc32c_arch(u32 crc, const u8 *p, size_t len) { if (len >=3D PMULL_MIN_LEN + 15 && static_branch_likely(&have_pmull) && crypto_simd_usable()) { size_t n =3D -(uintptr_t)p & 15; =20 @@ -85,39 +79,23 @@ u32 crc32c_arch(u32 crc, const u8 *p, size_t len) p +=3D n; len -=3D n; } return crc32c_scalar(crc, p, len); } -EXPORT_SYMBOL(crc32c_arch); =20 -u32 crc32_be_arch(u32 crc, const u8 *p, size_t len) -{ - return crc32_be_base(crc, p, len); -} -EXPORT_SYMBOL(crc32_be_arch); +#define crc32_be_arch crc32_be_base /* not implemented on this arch */ =20 -static int __init crc32_arm_init(void) +#define crc32_mod_init_arch crc32_mod_init_arch +static inline void crc32_mod_init_arch(void) { if (elf_hwcap2 & HWCAP2_CRC32) static_branch_enable(&have_crc32); if (elf_hwcap2 & HWCAP2_PMULL) static_branch_enable(&have_pmull); - return 0; } -subsys_initcall(crc32_arm_init); =20 -static void __exit crc32_arm_exit(void) -{ -} -module_exit(crc32_arm_exit); - -u32 crc32_optimizations(void) +static inline u32 crc32_optimizations_arch(void) { if (elf_hwcap2 & (HWCAP2_CRC32 | HWCAP2_PMULL)) return CRC32_LE_OPTIMIZATION | CRC32C_OPTIMIZATION; return 0; } -EXPORT_SYMBOL(crc32_optimizations); - -MODULE_AUTHOR("Ard Biesheuvel "); -MODULE_DESCRIPTION("Accelerated CRC32(C) using ARM CRC, NEON and Crypto Ex= tensions"); -MODULE_LICENSE("GPL v2"); --=20 2.49.0 From nobody Fri Dec 19 07:31:42 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C474321A928; Sat, 7 Jun 2025 20:07:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326850; cv=none; b=BGWOd0y1YBgQhIYfAagiwRlyFiwjuMiJrn8STGHxhprgb3GZBP+4FaDoNbOTY4QSp2oKMU5DbNGLq/48pGUxSC/w3MlLyx7ZmE1+feldkCnzwTajq9DEHS02FZSAOJas8WWnDNluCzo6zQnx/WylGGGXJgVlJshmCrFvkZ2rp+A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326850; c=relaxed/simple; bh=WdPZKPTFqdPtFVY3vxcYLn4JpXpcQFxK5xoZxw4oN6M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PW74lS86T7jg4ZRi3qhnXy9nZ0mBtDQ91Lg/tQYLrucjxdNHUrsvnqsyY8Z6o81ytHiuk+HttcxMHxlEyC5OOV/I+GVamtxIew+UpJlfk1m9kBnFvUg0J5vp13IBElmq9CRm5Ep88q5zI1JnZy6SchoZQYWsxT+aLTdjk9ABQjY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rYwbw18n; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rYwbw18n" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E608CC4CEF3; Sat, 7 Jun 2025 20:07:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1749326850; bh=WdPZKPTFqdPtFVY3vxcYLn4JpXpcQFxK5xoZxw4oN6M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rYwbw18nEbKgn50Q7dJjhLaOiApwovsGIwvol4hlE3ERmUUlQJY+VNQ/CnX2+rGSM c0YqjT0Th4xYfjnGQHFz/Sw5Z/ikWRamSc3unwHRfBVwFIjAhxW3i0m6tSQxaDEnvp xHg6+or6mz0ts6zU8NMMPKf7RzLSPbJyLmi0l614IUJ6MicSw9Gk14Y0i5HWuP/W93 x70mCt46NarWsMV8FOFk6YVKK1eOo5CxPH15DKrwBazQXQw1ckbrMhaNtdI9VZxxvX SHiVYR/cJZROszwFVf64VoMxta5X2NJsNqIhT7u7ipAeU4j0/7fkWcqESF/PWSm+3L SdKejv+mALrJg== From: Eric Biggers To: linux-kernel@vger.kernel.org Cc: linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Linus Torvalds Subject: [PATCH v2 04/12] lib/crc/arm64: migrate arm64-optimized CRC code into lib/crc/ Date: Sat, 7 Jun 2025 13:04:46 -0700 Message-ID: <20250607200454.73587-5-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250607200454.73587-1-ebiggers@kernel.org> References: <20250607200454.73587-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Move the arm64-optimized CRC code from arch/arm64/lib/crc* into its new location in lib/crc/arm64/, and wire it up in the new way. This new way of organizing the CRC code eliminates the need to artificially split the code for each CRC variant into separate arch and generic modules, enabling better inlining and dead code elimination. For more details, see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/". Signed-off-by: Eric Biggers Reviewed-by: Martin K. Petersen --- arch/arm64/Kconfig | 2 -- arch/arm64/lib/Makefile | 6 ----- lib/crc/Kconfig | 2 ++ lib/crc/Makefile | 2 ++ .../lib =3D> lib/crc/arm64}/crc-t10dif-core.S | 0 .../crc/arm64/crc-t10dif.h | 22 +++---------------- .../arm64/lib =3D> lib/crc/arm64}/crc32-core.S | 0 .../lib/crc32.c =3D> lib/crc/arm64/crc32.h | 19 ++++------------ 8 files changed, 11 insertions(+), 42 deletions(-) rename {arch/arm64/lib =3D> lib/crc/arm64}/crc-t10dif-core.S (100%) rename arch/arm64/lib/crc-t10dif.c =3D> lib/crc/arm64/crc-t10dif.h (70%) rename {arch/arm64/lib =3D> lib/crc/arm64}/crc32-core.S (100%) rename arch/arm64/lib/crc32.c =3D> lib/crc/arm64/crc32.h (81%) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 55fc331af3371..cbeac225903f7 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -19,12 +19,10 @@ config ARM64 select ARCH_ENABLE_MEMORY_HOTREMOVE select ARCH_ENABLE_SPLIT_PMD_PTLOCK if PGTABLE_LEVELS > 2 select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE select ARCH_HAS_CACHE_LINE_SIZE select ARCH_HAS_CC_PLATFORM - select ARCH_HAS_CRC32 - select ARCH_HAS_CRC_T10DIF if KERNEL_MODE_NEON select ARCH_HAS_CURRENT_STACK_POINTER select ARCH_HAS_DEBUG_VIRTUAL select ARCH_HAS_DEBUG_VM_PGTABLE select ARCH_HAS_DMA_OPS if XEN select ARCH_HAS_DMA_PREP_COHERENT diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile index 027bfa9689c6a..9b255d9332479 100644 --- a/arch/arm64/lib/Makefile +++ b/arch/arm64/lib/Makefile @@ -14,16 +14,10 @@ CFLAGS_xor-neon.o +=3D $(CC_FLAGS_FPU) CFLAGS_REMOVE_xor-neon.o +=3D $(CC_FLAGS_NO_FPU) endif =20 lib-$(CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE) +=3D uaccess_flushcache.o =20 -obj-$(CONFIG_CRC32_ARCH) +=3D crc32-arm64.o -crc32-arm64-y :=3D crc32.o crc32-core.o - -obj-$(CONFIG_CRC_T10DIF_ARCH) +=3D crc-t10dif-arm64.o -crc-t10dif-arm64-y :=3D crc-t10dif.o crc-t10dif-core.o - obj-$(CONFIG_FUNCTION_ERROR_INJECTION) +=3D error-inject.o =20 obj-$(CONFIG_ARM64_MTE) +=3D mte.o =20 obj-$(CONFIG_KASAN_SW_TAGS) +=3D kasan_sw_tags.o diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig index edd1b99098003..63edb487daff8 100644 --- a/lib/crc/Kconfig +++ b/lib/crc/Kconfig @@ -49,10 +49,11 @@ config ARCH_HAS_CRC_T10DIF =20 config CRC_T10DIF_ARCH bool depends on CRC_T10DIF && CRC_OPTIMIZATIONS default y if ARM && KERNEL_MODE_NEON + default y if ARM64 && KERNEL_MODE_NEON =20 config CRC32 tristate select BITREVERSE help @@ -64,10 +65,11 @@ config ARCH_HAS_CRC32 =20 config CRC32_ARCH bool depends on CRC32 && CRC_OPTIMIZATIONS default y if ARM && KERNEL_MODE_NEON + default y if ARM64 =20 config CRC64 tristate help The CRC64 library functions. Select this if your module uses any of diff --git a/lib/crc/Makefile b/lib/crc/Makefile index c72d351be6cb8..8adff4ae1ba63 100644 --- a/lib/crc/Makefile +++ b/lib/crc/Makefile @@ -12,17 +12,19 @@ obj-$(CONFIG_CRC_ITU_T) +=3D crc-itu-t.o obj-$(CONFIG_CRC_T10DIF) +=3D crc-t10dif.o crc-t10dif-y :=3D crc-t10dif-main.o ifeq ($(CONFIG_CRC_T10DIF_ARCH),y) CFLAGS_crc-t10dif-main.o +=3D -I$(src)/$(SRCARCH) crc-t10dif-$(CONFIG_ARM) +=3D arm/crc-t10dif-core.o +crc-t10dif-$(CONFIG_ARM64) +=3D arm64/crc-t10dif-core.o endif =20 obj-$(CONFIG_CRC32) +=3D crc32.o crc32-y :=3D crc32-main.o ifeq ($(CONFIG_CRC32_ARCH),y) CFLAGS_crc32-main.o +=3D -I$(src)/$(SRCARCH) crc32-$(CONFIG_ARM) +=3D arm/crc32-core.o +crc32-$(CONFIG_ARM64) +=3D arm64/crc32-core.o endif =20 obj-$(CONFIG_CRC64) +=3D crc64.o crc64-y :=3D crc64-main.o ifeq ($(CONFIG_CRC64_ARCH),y) diff --git a/arch/arm64/lib/crc-t10dif-core.S b/lib/crc/arm64/crc-t10dif-co= re.S similarity index 100% rename from arch/arm64/lib/crc-t10dif-core.S rename to lib/crc/arm64/crc-t10dif-core.S diff --git a/arch/arm64/lib/crc-t10dif.c b/lib/crc/arm64/crc-t10dif.h similarity index 70% rename from arch/arm64/lib/crc-t10dif.c rename to lib/crc/arm64/crc-t10dif.h index c2ffe4fdb59d1..c4521a7f1ee9b 100644 --- a/arch/arm64/lib/crc-t10dif.c +++ b/lib/crc/arm64/crc-t10dif.h @@ -4,15 +4,10 @@ * * Copyright (C) 2016 - 2017 Linaro Ltd */ =20 #include -#include -#include -#include -#include -#include =20 #include =20 #include #include @@ -24,11 +19,11 @@ static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_pmu= ll); =20 asmlinkage void crc_t10dif_pmull_p8(u16 init_crc, const u8 *buf, size_t le= n, u8 out[16]); asmlinkage u16 crc_t10dif_pmull_p64(u16 init_crc, const u8 *buf, size_t le= n); =20 -u16 crc_t10dif_arch(u16 crc, const u8 *data, size_t length) +static inline u16 crc_t10dif_arch(u16 crc, const u8 *data, size_t length) { if (length >=3D CRC_T10DIF_PMULL_CHUNK_SIZE) { if (static_branch_likely(&have_pmull)) { if (crypto_simd_usable()) { kernel_neon_begin(); @@ -48,26 +43,15 @@ u16 crc_t10dif_arch(u16 crc, const u8 *data, size_t len= gth) return crc_t10dif_generic(0, buf, sizeof(buf)); } } return crc_t10dif_generic(crc, data, length); } -EXPORT_SYMBOL(crc_t10dif_arch); =20 -static int __init crc_t10dif_arm64_init(void) +#define crc_t10dif_mod_init_arch crc_t10dif_mod_init_arch +static inline void crc_t10dif_mod_init_arch(void) { if (cpu_have_named_feature(ASIMD)) { static_branch_enable(&have_asimd); if (cpu_have_named_feature(PMULL)) static_branch_enable(&have_pmull); } - return 0; } -subsys_initcall(crc_t10dif_arm64_init); - -static void __exit crc_t10dif_arm64_exit(void) -{ -} -module_exit(crc_t10dif_arm64_exit); - -MODULE_AUTHOR("Ard Biesheuvel "); -MODULE_DESCRIPTION("CRC-T10DIF using arm64 NEON and Crypto Extensions"); -MODULE_LICENSE("GPL v2"); diff --git a/arch/arm64/lib/crc32-core.S b/lib/crc/arm64/crc32-core.S similarity index 100% rename from arch/arm64/lib/crc32-core.S rename to lib/crc/arm64/crc32-core.S diff --git a/arch/arm64/lib/crc32.c b/lib/crc/arm64/crc32.h similarity index 81% rename from arch/arm64/lib/crc32.c rename to lib/crc/arm64/crc32.h index ed3acd71178f8..6e5dec45f05d2 100644 --- a/arch/arm64/lib/crc32.c +++ b/lib/crc/arm64/crc32.h @@ -1,11 +1,7 @@ // SPDX-License-Identifier: GPL-2.0-only =20 -#include -#include -#include - #include #include #include #include =20 @@ -20,11 +16,11 @@ asmlinkage u32 crc32_be_arm64(u32 crc, unsigned char co= nst *p, size_t len); =20 asmlinkage u32 crc32_le_arm64_4way(u32 crc, unsigned char const *p, size_t= len); asmlinkage u32 crc32c_le_arm64_4way(u32 crc, unsigned char const *p, size_= t len); asmlinkage u32 crc32_be_arm64_4way(u32 crc, unsigned char const *p, size_t= len); =20 -u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) +static inline u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) { if (!alternative_has_cap_likely(ARM64_HAS_CRC32)) return crc32_le_base(crc, p, len); =20 if (len >=3D min_len && cpu_have_named_feature(PMULL) && crypto_simd_usab= le()) { @@ -39,13 +35,12 @@ u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) return crc; } =20 return crc32_le_arm64(crc, p, len); } -EXPORT_SYMBOL(crc32_le_arch); =20 -u32 crc32c_arch(u32 crc, const u8 *p, size_t len) +static inline u32 crc32c_arch(u32 crc, const u8 *p, size_t len) { if (!alternative_has_cap_likely(ARM64_HAS_CRC32)) return crc32c_base(crc, p, len); =20 if (len >=3D min_len && cpu_have_named_feature(PMULL) && crypto_simd_usab= le()) { @@ -60,13 +55,12 @@ u32 crc32c_arch(u32 crc, const u8 *p, size_t len) return crc; } =20 return crc32c_le_arm64(crc, p, len); } -EXPORT_SYMBOL(crc32c_arch); =20 -u32 crc32_be_arch(u32 crc, const u8 *p, size_t len) +static inline u32 crc32_be_arch(u32 crc, const u8 *p, size_t len) { if (!alternative_has_cap_likely(ARM64_HAS_CRC32)) return crc32_be_base(crc, p, len); =20 if (len >=3D min_len && cpu_have_named_feature(PMULL) && crypto_simd_usab= le()) { @@ -81,19 +75,14 @@ u32 crc32_be_arch(u32 crc, const u8 *p, size_t len) return crc; } =20 return crc32_be_arm64(crc, p, len); } -EXPORT_SYMBOL(crc32_be_arch); =20 -u32 crc32_optimizations(void) +static inline u32 crc32_optimizations_arch(void) { if (alternative_has_cap_likely(ARM64_HAS_CRC32)) return CRC32_LE_OPTIMIZATION | CRC32_BE_OPTIMIZATION | CRC32C_OPTIMIZATION; return 0; } -EXPORT_SYMBOL(crc32_optimizations); - -MODULE_LICENSE("GPL"); -MODULE_DESCRIPTION("arm64-optimized CRC32 functions"); --=20 2.49.0 From nobody Fri Dec 19 07:31:42 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 38EA021B9E0; Sat, 7 Jun 2025 20:07:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326852; cv=none; b=pA+UTcqcqsUcUflnDUi91y1/7e9B6xjonPM3ajeaUA8m7lYchbz0FngnW4AarG8Wkpzr+pY3q08ALhZMUZfyj/g0O8KqCO7bkEQfLD2d8UuWTk9MaUHjEmmwtpnT4KrYFv2fHY4+r05nwAb2j1/Pqbggu5ImRRKWnOe07SiTrsw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326852; c=relaxed/simple; bh=zY3fvpUbE+/4lThsCnGfT2NxFkXv0wD4oYk8K5EmW5s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dTt7T4o3F47ZaY7dpEcO3nUUdDKgBPC7dhFHdV7n9f92J/a9oVAeoDBol0sawULoPobhiZYqdU+la5v2VE7YsDQbrJ4oF3zyz9bv0r6ZBb7kOZiU8jG4+/9TzguP6be8eWdnYVoT8+S4CTuM+gJFg3gyx3yHivatcpZS5Gy4Lzg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Chu95TKq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Chu95TKq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 718C0C4CEF0; Sat, 7 Jun 2025 20:07:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1749326850; bh=zY3fvpUbE+/4lThsCnGfT2NxFkXv0wD4oYk8K5EmW5s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Chu95TKqPmDS98OhYY5Ac8BbqnX5apkd7PAsCrFcuiOOi6LR+p0TFPzxMjsPY/PBq p/1Jy8JRQ+XCj0wdcVjt+41xJP78UoKVawE5RR2SKK2RzkUBS0W0LJKy7KBBSJohLi Q6faTyM18RMmPfHyck7zmT9eKoGB1seXkb7EhfyczxWRMffU50h4KIAf034eG732E4 34vYkNT6nw+3TipBEJ7+fzMGPWM2pbIgR5lSjvU4I5fW4tJV2csqi4G22VpKWVRauo yaHESgRIaOrVsNKdbICEnpBylxlg1NhiNCVLjeKF48QN4yRNzdUD/eZrqQphdsTwTY WpEFv90CsX4/g== From: Eric Biggers To: linux-kernel@vger.kernel.org Cc: linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Linus Torvalds Subject: [PATCH v2 05/12] lib/crc/loongarch: migrate loongarch-optimized CRC code into lib/crc/ Date: Sat, 7 Jun 2025 13:04:47 -0700 Message-ID: <20250607200454.73587-6-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250607200454.73587-1-ebiggers@kernel.org> References: <20250607200454.73587-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Move the loongarch-optimized CRC code from arch/loongarch/lib/crc* into its new location in lib/crc/loongarch/, and wire it up in the new way. This new way of organizing the CRC code eliminates the need to artificially split the code for each CRC variant into separate arch and generic modules, enabling better inlining and dead code elimination. For more details, see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/". Signed-off-by: Eric Biggers Reviewed-by: Martin K. Petersen --- arch/loongarch/Kconfig | 1 - arch/loongarch/lib/Makefile | 2 -- lib/crc/Kconfig | 1 + .../crc/loongarch/crc32.h | 32 ++++--------------- 4 files changed, 7 insertions(+), 29 deletions(-) rename arch/loongarch/lib/crc32-loongarch.c =3D> lib/crc/loongarch/crc32.h= (71%) diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 1a2cf012b8f2f..1b19893a6bdf0 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -13,11 +13,10 @@ config LOONGARCH select ARCH_ENABLE_MEMORY_HOTPLUG select ARCH_ENABLE_MEMORY_HOTREMOVE select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI select ARCH_HAS_CPU_FINALIZE_INIT - select ARCH_HAS_CRC32 select ARCH_HAS_CURRENT_STACK_POINTER select ARCH_HAS_DEBUG_VM_PGTABLE select ARCH_HAS_FAST_MULTIPLIER select ARCH_HAS_FORTIFY_SOURCE select ARCH_HAS_KCOV diff --git a/arch/loongarch/lib/Makefile b/arch/loongarch/lib/Makefile index fae77809048b8..ccea3bbd43531 100644 --- a/arch/loongarch/lib/Makefile +++ b/arch/loongarch/lib/Makefile @@ -9,7 +9,5 @@ lib-y +=3D delay.o memset.o memcpy.o memmove.o \ obj-$(CONFIG_ARCH_SUPPORTS_INT128) +=3D tishift.o =20 obj-$(CONFIG_CPU_HAS_LSX) +=3D xor_simd.o xor_simd_glue.o =20 obj-$(CONFIG_FUNCTION_ERROR_INJECTION) +=3D error-inject.o - -obj-$(CONFIG_CRC32_ARCH) +=3D crc32-loongarch.o diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig index 63edb487daff8..c1629f07768f9 100644 --- a/lib/crc/Kconfig +++ b/lib/crc/Kconfig @@ -66,10 +66,11 @@ config ARCH_HAS_CRC32 config CRC32_ARCH bool depends on CRC32 && CRC_OPTIMIZATIONS default y if ARM && KERNEL_MODE_NEON default y if ARM64 + default y if LOONGARCH =20 config CRC64 tristate help The CRC64 library functions. Select this if your module uses any of diff --git a/arch/loongarch/lib/crc32-loongarch.c b/lib/crc/loongarch/crc32= .h similarity index 71% rename from arch/loongarch/lib/crc32-loongarch.c rename to lib/crc/loongarch/crc32.h index b37cd8537b459..6de5c96594afc 100644 --- a/arch/loongarch/lib/crc32-loongarch.c +++ b/lib/crc/loongarch/crc32.h @@ -8,12 +8,10 @@ * Copyright (C) 2018 MIPS Tech, LLC * Copyright (C) 2020-2023 Loongson Technology Corporation Limited */ =20 #include -#include -#include #include =20 #define _CRC32(crc, value, size, type) \ do { \ __asm__ __volatile__( \ @@ -26,11 +24,11 @@ do { \ #define CRC32(crc, value, size) _CRC32(crc, value, size, crc) #define CRC32C(crc, value, size) _CRC32(crc, value, size, crcc) =20 static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_crc32); =20 -u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) +static inline u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) { if (!static_branch_likely(&have_crc32)) return crc32_le_base(crc, p, len); =20 while (len >=3D sizeof(u64)) { @@ -61,13 +59,12 @@ u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) CRC32(crc, value, b); } =20 return crc; } -EXPORT_SYMBOL(crc32_le_arch); =20 -u32 crc32c_arch(u32 crc, const u8 *p, size_t len) +static inline u32 crc32c_arch(u32 crc, const u8 *p, size_t len) { if (!static_branch_likely(&have_crc32)) return crc32c_base(crc, p, len); =20 while (len >=3D sizeof(u64)) { @@ -98,38 +95,21 @@ u32 crc32c_arch(u32 crc, const u8 *p, size_t len) CRC32C(crc, value, b); } =20 return crc; } -EXPORT_SYMBOL(crc32c_arch); =20 -u32 crc32_be_arch(u32 crc, const u8 *p, size_t len) -{ - return crc32_be_base(crc, p, len); -} -EXPORT_SYMBOL(crc32_be_arch); +#define crc32_be_arch crc32_be_base /* not implemented on this arch */ =20 -static int __init crc32_loongarch_init(void) +#define crc32_mod_init_arch crc32_mod_init_arch +static inline void crc32_mod_init_arch(void) { if (cpu_has_crc32) static_branch_enable(&have_crc32); - return 0; } -subsys_initcall(crc32_loongarch_init); =20 -static void __exit crc32_loongarch_exit(void) -{ -} -module_exit(crc32_loongarch_exit); - -u32 crc32_optimizations(void) +static inline u32 crc32_optimizations_arch(void) { if (static_key_enabled(&have_crc32)) return CRC32_LE_OPTIMIZATION | CRC32C_OPTIMIZATION; return 0; } -EXPORT_SYMBOL(crc32_optimizations); - -MODULE_AUTHOR("Min Zhou "); -MODULE_AUTHOR("Huacai Chen "); -MODULE_DESCRIPTION("CRC32 and CRC32C using LoongArch crc* instructions"); -MODULE_LICENSE("GPL v2"); --=20 2.49.0 From nobody Fri Dec 19 07:31:42 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7197321ADC6; Sat, 7 Jun 2025 20:07:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326851; cv=none; b=C4HDVROEWgqcQZ7B6WvMXzCUPTKcvdfnmNMJalwgIxGGoA8rWg/Jptd7H8WtKWwyd+vAywKR052G5UNXHOz4YSPlr3ZAMTbjKgJ/oNPxQzIoWdJd/+8KyKRdNY2C3dsNslPSON5iaqmk1aL9Wsb6STeuLOn4oyU0OhuZm0Xb3Yc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326851; c=relaxed/simple; bh=1/FX0gdNnrILQ8ZGkWaTdKqNzEwmy3fTfx34tGwHQsw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YuphAa+eEA+0IO3FeAyHhUuc885uppb4jQeUmQFMStwKbTYJUR8Q2VvS8v3a5WaDaUslU4f8ZRNhReISIyZS3sD9EexHWzI0XxowYFThdokCStGeSQ7zzNOU6CEqRurCG7dJFqS2wBrg9GFF/pziCJWH46sCdvzlssj6CUFaX0Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ARR+iKGr; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ARR+iKGr" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F0353C4CEF7; Sat, 7 Jun 2025 20:07:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1749326851; bh=1/FX0gdNnrILQ8ZGkWaTdKqNzEwmy3fTfx34tGwHQsw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ARR+iKGrG+gAid+aDf6WQrQsQFpDxmsrmuoXyxau9e/Yt42BjtKcN1ST4zDXz2XsB sXWsh2italtMvrkfA7Aw9KhOC7ceGu5ulDfHXBs2S9VScYnFqNDmIFZTg3tzdkb9BX r6CuvNJgZfTcsPqiJZrueYI08M04Y03dPIUEqPvpkYNgxjTnu/RYNLewzPfrMi8pZB /hzgAURQQ8GA3rbrGzQl3BWr4uDcAzrxu84F3hAOiJM6xseNbi1f6BAE9z14ArgoOp 34rkuBw8bcsaG7YiEiuDnbMwSYwe0c4Qmr9xWGuPJHTXIgH9klSsIbNJCCtzOkiDsv TzdI7DNVBiIkg== From: Eric Biggers To: linux-kernel@vger.kernel.org Cc: linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Linus Torvalds Subject: [PATCH v2 06/12] lib/crc/mips: migrate mips-optimized CRC code into lib/crc/ Date: Sat, 7 Jun 2025 13:04:48 -0700 Message-ID: <20250607200454.73587-7-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250607200454.73587-1-ebiggers@kernel.org> References: <20250607200454.73587-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Move the mips-optimized CRC code from arch/mips/lib/crc* into its new location in lib/crc/mips/, and wire it up in the new way. This new way of organizing the CRC code eliminates the need to artificially split the code for each CRC variant into separate arch and generic modules, enabling better inlining and dead code elimination. For more details, see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/". Signed-off-by: Eric Biggers Reviewed-by: Martin K. Petersen --- arch/mips/Kconfig | 1 - arch/mips/lib/Makefile | 2 -- lib/crc/Kconfig | 1 + .../lib/crc32-mips.c =3D> lib/crc/mips/crc32.h | 33 ++++--------------- 4 files changed, 7 insertions(+), 30 deletions(-) rename arch/mips/lib/crc32-mips.c =3D> lib/crc/mips/crc32.h (82%) diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig index 1e48184ecf1ec..934eb961bd0dd 100644 --- a/arch/mips/Kconfig +++ b/arch/mips/Kconfig @@ -2022,11 +2022,10 @@ config CPU_MIPSR5 select MIPS_SPRAM =20 config CPU_MIPSR6 bool default y if CPU_MIPS32_R6 || CPU_MIPS64_R6 - select ARCH_HAS_CRC32 select CPU_HAS_RIXI select CPU_HAS_DIEI if !CPU_DIEI_BROKEN select HAVE_ARCH_BITREVERSE select MIPS_ASID_BITS_VARIABLE select MIPS_SPRAM diff --git a/arch/mips/lib/Makefile b/arch/mips/lib/Makefile index 9d75845ef78e1..8c40ffb09c420 100644 --- a/arch/mips/lib/Makefile +++ b/arch/mips/lib/Makefile @@ -14,9 +14,7 @@ obj-$(CONFIG_PCI) +=3D iomap-pci.o lib-$(CONFIG_GENERIC_CSUM) :=3D $(filter-out csum_partial.o, $(lib-y)) =20 obj-$(CONFIG_CPU_GENERIC_DUMP_TLB) +=3D dump_tlb.o obj-$(CONFIG_CPU_R3000) +=3D r3k_dump_tlb.o =20 -obj-$(CONFIG_CRC32_ARCH) +=3D crc32-mips.o - # libgcc-style stuff needed in the kernel obj-y +=3D bswapsi.o bswapdi.o multi3.o diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig index c1629f07768f9..3f534fbfba951 100644 --- a/lib/crc/Kconfig +++ b/lib/crc/Kconfig @@ -67,10 +67,11 @@ config CRC32_ARCH bool depends on CRC32 && CRC_OPTIMIZATIONS default y if ARM && KERNEL_MODE_NEON default y if ARM64 default y if LOONGARCH + default y if MIPS && CPU_MIPSR6 =20 config CRC64 tristate help The CRC64 library functions. Select this if your module uses any of diff --git a/arch/mips/lib/crc32-mips.c b/lib/crc/mips/crc32.h similarity index 82% rename from arch/mips/lib/crc32-mips.c rename to lib/crc/mips/crc32.h index 45e4d2c9fbf54..11cb272c63a69 100644 --- a/arch/mips/lib/crc32-mips.c +++ b/lib/crc/mips/crc32.h @@ -7,14 +7,10 @@ * Copyright (C) 2014 Linaro Ltd * Copyright (C) 2018 MIPS Tech, LLC */ =20 #include -#include -#include -#include -#include #include #include =20 #ifndef TOOLCHAIN_SUPPORTS_CRC #define _ASM_SET_CRC(OP, SZ, TYPE) \ @@ -62,11 +58,11 @@ do { \ #define CRC32C(crc, value, size) \ _CRC32(crc, value, size, crc32c) =20 static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_crc32); =20 -u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) +static inline u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) { if (!static_branch_likely(&have_crc32)) return crc32_le_base(crc, p, len); =20 if (IS_ENABLED(CONFIG_64BIT)) { @@ -104,13 +100,12 @@ u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) CRC32(crc, value, b); } =20 return crc; } -EXPORT_SYMBOL(crc32_le_arch); =20 -u32 crc32c_arch(u32 crc, const u8 *p, size_t len) +static inline u32 crc32c_arch(u32 crc, const u8 *p, size_t len) { if (!static_branch_likely(&have_crc32)) return crc32c_base(crc, p, len); =20 if (IS_ENABLED(CONFIG_64BIT)) { @@ -147,37 +142,21 @@ u32 crc32c_arch(u32 crc, const u8 *p, size_t len) =20 CRC32C(crc, value, b); } return crc; } -EXPORT_SYMBOL(crc32c_arch); =20 -u32 crc32_be_arch(u32 crc, const u8 *p, size_t len) -{ - return crc32_be_base(crc, p, len); -} -EXPORT_SYMBOL(crc32_be_arch); +#define crc32_be_arch crc32_be_base /* not implemented on this arch */ =20 -static int __init crc32_mips_init(void) +#define crc32_mod_init_arch crc32_mod_init_arch +static inline void crc32_mod_init_arch(void) { if (cpu_have_feature(cpu_feature(MIPS_CRC32))) static_branch_enable(&have_crc32); - return 0; } -subsys_initcall(crc32_mips_init); =20 -static void __exit crc32_mips_exit(void) -{ -} -module_exit(crc32_mips_exit); - -u32 crc32_optimizations(void) +static inline u32 crc32_optimizations_arch(void) { if (static_key_enabled(&have_crc32)) return CRC32_LE_OPTIMIZATION | CRC32C_OPTIMIZATION; return 0; } -EXPORT_SYMBOL(crc32_optimizations); - -MODULE_AUTHOR("Marcin Nowakowski To: linux-kernel@vger.kernel.org Cc: linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Linus Torvalds Subject: [PATCH v2 07/12] lib/crc/powerpc: migrate powerpc-optimized CRC code into lib/crc/ Date: Sat, 7 Jun 2025 13:04:49 -0700 Message-ID: <20250607200454.73587-8-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250607200454.73587-1-ebiggers@kernel.org> References: <20250607200454.73587-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Move the powerpc-optimized CRC code from arch/powerpc/lib/crc* into its new location in lib/crc/powerpc/, and wire it up in the new way. This new way of organizing the CRC code eliminates the need to artificially split the code for each CRC variant into separate arch and generic modules, enabling better inlining and dead code elimination. For more details, see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/". Signed-off-by: Eric Biggers Reviewed-by: Martin K. Petersen --- arch/powerpc/Kconfig | 2 - arch/powerpc/lib/Makefile | 6 --- lib/crc/Kconfig | 2 + lib/crc/Makefile | 2 + .../crc/powerpc/crc-t10dif.h | 20 ++-------- .../crc/powerpc}/crc-vpmsum-template.S | 0 .../lib/crc32.c =3D> lib/crc/powerpc/crc32.h | 38 ++++--------------- .../crc/powerpc}/crc32c-vpmsum_asm.S | 0 .../crc/powerpc}/crct10dif-vpmsum_asm.S | 0 9 files changed, 14 insertions(+), 56 deletions(-) rename arch/powerpc/lib/crc-t10dif.c =3D> lib/crc/powerpc/crc-t10dif.h (75= %) rename {arch/powerpc/lib =3D> lib/crc/powerpc}/crc-vpmsum-template.S (100%) rename arch/powerpc/lib/crc32.c =3D> lib/crc/powerpc/crc32.h (64%) rename {arch/powerpc/lib =3D> lib/crc/powerpc}/crc32c-vpmsum_asm.S (100%) rename {arch/powerpc/lib =3D> lib/crc/powerpc}/crct10dif-vpmsum_asm.S (100= %) diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index c3e0cc83f1205..45b4fa7b9b02f 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -125,12 +125,10 @@ config PPC select ARCH_DISABLE_KASAN_INLINE if PPC_RADIX_MMU select ARCH_DMA_DEFAULT_COHERENT if !NOT_COHERENT_CACHE select ARCH_ENABLE_MEMORY_HOTPLUG select ARCH_ENABLE_MEMORY_HOTREMOVE select ARCH_HAS_COPY_MC if PPC64 - select ARCH_HAS_CRC32 if PPC64 && ALTIVEC - select ARCH_HAS_CRC_T10DIF if PPC64 && ALTIVEC select ARCH_HAS_CURRENT_STACK_POINTER select ARCH_HAS_DEBUG_VIRTUAL select ARCH_HAS_DEBUG_VM_PGTABLE select ARCH_HAS_DEBUG_WX if STRICT_KERNEL_RWX select ARCH_HAS_DEVMEM_IS_ALLOWED diff --git a/arch/powerpc/lib/Makefile b/arch/powerpc/lib/Makefile index 481f968e42c7b..59de2e2232df6 100644 --- a/arch/powerpc/lib/Makefile +++ b/arch/powerpc/lib/Makefile @@ -78,12 +78,6 @@ obj-$(CONFIG_FTR_FIXUP_SELFTEST) +=3D feature-fixups-tes= t.o obj-$(CONFIG_ALTIVEC) +=3D xor_vmx.o xor_vmx_glue.o CFLAGS_xor_vmx.o +=3D -mhard-float -maltivec $(call cc-option,-mabi=3Dalti= vec) # Enable CFLAGS_xor_vmx.o +=3D -isystem $(shell $(CC) -print-file-name=3Dinclude) =20 -obj-$(CONFIG_CRC32_ARCH) +=3D crc32-powerpc.o -crc32-powerpc-y :=3D crc32.o crc32c-vpmsum_asm.o - -obj-$(CONFIG_CRC_T10DIF_ARCH) +=3D crc-t10dif-powerpc.o -crc-t10dif-powerpc-y :=3D crc-t10dif.o crct10dif-vpmsum_asm.o - obj-$(CONFIG_PPC64) +=3D $(obj64-y) diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig index 3f534fbfba951..c3f9091ee512e 100644 --- a/lib/crc/Kconfig +++ b/lib/crc/Kconfig @@ -50,10 +50,11 @@ config ARCH_HAS_CRC_T10DIF config CRC_T10DIF_ARCH bool depends on CRC_T10DIF && CRC_OPTIMIZATIONS default y if ARM && KERNEL_MODE_NEON default y if ARM64 && KERNEL_MODE_NEON + default y if PPC64 && ALTIVEC =20 config CRC32 tristate select BITREVERSE help @@ -68,10 +69,11 @@ config CRC32_ARCH depends on CRC32 && CRC_OPTIMIZATIONS default y if ARM && KERNEL_MODE_NEON default y if ARM64 default y if LOONGARCH default y if MIPS && CPU_MIPSR6 + default y if PPC64 && ALTIVEC =20 config CRC64 tristate help The CRC64 library functions. Select this if your module uses any of diff --git a/lib/crc/Makefile b/lib/crc/Makefile index 8adff4ae1ba63..555fd3fb6d197 100644 --- a/lib/crc/Makefile +++ b/lib/crc/Makefile @@ -13,18 +13,20 @@ obj-$(CONFIG_CRC_T10DIF) +=3D crc-t10dif.o crc-t10dif-y :=3D crc-t10dif-main.o ifeq ($(CONFIG_CRC_T10DIF_ARCH),y) CFLAGS_crc-t10dif-main.o +=3D -I$(src)/$(SRCARCH) crc-t10dif-$(CONFIG_ARM) +=3D arm/crc-t10dif-core.o crc-t10dif-$(CONFIG_ARM64) +=3D arm64/crc-t10dif-core.o +crc-t10dif-$(CONFIG_PPC) +=3D powerpc/crct10dif-vpmsum_asm.o endif =20 obj-$(CONFIG_CRC32) +=3D crc32.o crc32-y :=3D crc32-main.o ifeq ($(CONFIG_CRC32_ARCH),y) CFLAGS_crc32-main.o +=3D -I$(src)/$(SRCARCH) crc32-$(CONFIG_ARM) +=3D arm/crc32-core.o crc32-$(CONFIG_ARM64) +=3D arm64/crc32-core.o +crc32-$(CONFIG_PPC) +=3D powerpc/crc32c-vpmsum_asm.o endif =20 obj-$(CONFIG_CRC64) +=3D crc64.o crc64-y :=3D crc64-main.o ifeq ($(CONFIG_CRC64_ARCH),y) diff --git a/arch/powerpc/lib/crc-t10dif.c b/lib/crc/powerpc/crc-t10dif.h similarity index 75% rename from arch/powerpc/lib/crc-t10dif.c rename to lib/crc/powerpc/crc-t10dif.h index be23ded3a9df6..59e16804a6eae 100644 --- a/arch/powerpc/lib/crc-t10dif.c +++ b/lib/crc/powerpc/crc-t10dif.h @@ -7,14 +7,11 @@ */ =20 #include #include #include -#include #include -#include -#include #include #include =20 #define VMX_ALIGN 16 #define VMX_ALIGN_MASK (VMX_ALIGN-1) @@ -23,11 +20,11 @@ =20 static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_vec_crypto); =20 u32 __crct10dif_vpmsum(u32 crc, unsigned char const *p, size_t len); =20 -u16 crc_t10dif_arch(u16 crci, const u8 *p, size_t len) +static inline u16 crc_t10dif_arch(u16 crci, const u8 *p, size_t len) { unsigned int prealign; unsigned int tail; u32 crc =3D crci; =20 @@ -60,24 +57,13 @@ u16 crc_t10dif_arch(u16 crci, const u8 *p, size_t len) crc =3D crc_t10dif_generic(crc, p, tail); } =20 return crc & 0xffff; } -EXPORT_SYMBOL(crc_t10dif_arch); =20 -static int __init crc_t10dif_powerpc_init(void) +#define crc_t10dif_mod_init_arch crc_t10dif_mod_init_arch +static inline void crc_t10dif_mod_init_arch(void) { if (cpu_has_feature(CPU_FTR_ARCH_207S) && (cur_cpu_spec->cpu_user_features2 & PPC_FEATURE2_VEC_CRYPTO)) static_branch_enable(&have_vec_crypto); - return 0; } -subsys_initcall(crc_t10dif_powerpc_init); - -static void __exit crc_t10dif_powerpc_exit(void) -{ -} -module_exit(crc_t10dif_powerpc_exit); - -MODULE_AUTHOR("Daniel Axtens "); -MODULE_DESCRIPTION("CRCT10DIF using vector polynomial multiply-sum instruc= tions"); -MODULE_LICENSE("GPL"); diff --git a/arch/powerpc/lib/crc-vpmsum-template.S b/lib/crc/powerpc/crc-v= pmsum-template.S similarity index 100% rename from arch/powerpc/lib/crc-vpmsum-template.S rename to lib/crc/powerpc/crc-vpmsum-template.S diff --git a/arch/powerpc/lib/crc32.c b/lib/crc/powerpc/crc32.h similarity index 64% rename from arch/powerpc/lib/crc32.c rename to lib/crc/powerpc/crc32.h index 0d9befb6e7b83..811cc2e6ed24d 100644 --- a/arch/powerpc/lib/crc32.c +++ b/lib/crc/powerpc/crc32.h @@ -1,32 +1,26 @@ // SPDX-License-Identifier: GPL-2.0-only #include #include #include -#include #include -#include -#include #include #include =20 #define VMX_ALIGN 16 #define VMX_ALIGN_MASK (VMX_ALIGN-1) =20 #define VECTOR_BREAKPOINT 512 =20 static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_vec_crypto); =20 -u32 __crc32c_vpmsum(u32 crc, const u8 *p, size_t len); +#define crc32_le_arch crc32_le_base /* not implemented on this arch */ +#define crc32_be_arch crc32_be_base /* not implemented on this arch */ =20 -u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) -{ - return crc32_le_base(crc, p, len); -} -EXPORT_SYMBOL(crc32_le_arch); +u32 __crc32c_vpmsum(u32 crc, const u8 *p, size_t len); =20 -u32 crc32c_arch(u32 crc, const u8 *p, size_t len) +static inline u32 crc32c_arch(u32 crc, const u8 *p, size_t len) { unsigned int prealign; unsigned int tail; =20 if (len < (VECTOR_BREAKPOINT + VMX_ALIGN) || @@ -56,38 +50,20 @@ u32 crc32c_arch(u32 crc, const u8 *p, size_t len) crc =3D crc32c_base(crc, p, tail); } =20 return crc; } -EXPORT_SYMBOL(crc32c_arch); - -u32 crc32_be_arch(u32 crc, const u8 *p, size_t len) -{ - return crc32_be_base(crc, p, len); -} -EXPORT_SYMBOL(crc32_be_arch); =20 -static int __init crc32_powerpc_init(void) +#define crc32_mod_init_arch crc32_mod_init_arch +static inline void crc32_mod_init_arch(void) { if (cpu_has_feature(CPU_FTR_ARCH_207S) && (cur_cpu_spec->cpu_user_features2 & PPC_FEATURE2_VEC_CRYPTO)) static_branch_enable(&have_vec_crypto); - return 0; -} -subsys_initcall(crc32_powerpc_init); - -static void __exit crc32_powerpc_exit(void) -{ } -module_exit(crc32_powerpc_exit); =20 -u32 crc32_optimizations(void) +static inline u32 crc32_optimizations_arch(void) { if (static_key_enabled(&have_vec_crypto)) return CRC32C_OPTIMIZATION; return 0; } -EXPORT_SYMBOL(crc32_optimizations); - -MODULE_AUTHOR("Anton Blanchard "); -MODULE_DESCRIPTION("CRC32C using vector polynomial multiply-sum instructio= ns"); -MODULE_LICENSE("GPL"); diff --git a/arch/powerpc/lib/crc32c-vpmsum_asm.S b/lib/crc/powerpc/crc32c-= vpmsum_asm.S similarity index 100% rename from arch/powerpc/lib/crc32c-vpmsum_asm.S rename to lib/crc/powerpc/crc32c-vpmsum_asm.S diff --git a/arch/powerpc/lib/crct10dif-vpmsum_asm.S b/lib/crc/powerpc/crct= 10dif-vpmsum_asm.S similarity index 100% rename from arch/powerpc/lib/crct10dif-vpmsum_asm.S rename to lib/crc/powerpc/crct10dif-vpmsum_asm.S --=20 2.49.0 From nobody Fri Dec 19 07:31:42 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA58621C9F6; Sat, 7 Jun 2025 20:07:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326853; cv=none; b=cClZlyS0c45NPf53DLSqdhUP+qVPYwwAFnDzyLuyDQgJBpxBwmQ5PJwN/tVqH0ymqNrFS/WKV99OQ8weUiq+tN2qjllJkrgcHJkRIN9WT0w/PYtuUX/EIbL62F9dl8sfBE+9/pEKOIEafSozQkc6wOxYkgBQsoP+J13CxLHUIpY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326853; c=relaxed/simple; bh=UhWtk6TkzpgWbad69ugt2GMpM8wI/AOjLVW/1+GRCZM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aWAUNOKkSiQLo4vOT5Fn1lE2erMvmaSxCMHz/4hziDfJedlXxE6ZhNuJmfT9ej9JqIRM/7iLMMUs8lFo0yxkHpRRKBn4lLkKlAzC4Lp7amgYkHYGotk7I2JLyFZFPQMVrT80sLGti875f7wHJ3Bn7vZcP2jyLpM7sZ0mgwCQK1s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pTj4CUvS; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pTj4CUvS" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 081C2C4CEFE; Sat, 7 Jun 2025 20:07:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1749326852; bh=UhWtk6TkzpgWbad69ugt2GMpM8wI/AOjLVW/1+GRCZM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pTj4CUvSbhajPr41bWro0qvHnYDm1wwoVE3HzOEhFvE9Wg8gJs54AypdZVwebNAc+ o7wZ6Rh7vv1xZwENmbDBbEl+75t8pskCZwchC38d1qwpq65rU2eHcmy72uzoZxLy/K u3pUA71njZJv5rj6gFTaoiTE0XxXyBWHhpfGFFPBcVnl1WE7jhM8VphGM9/TB9mTEm P2vg+92ZfulqYd8eoyrQZBgDFysfjDyS0Uzw0TTNvPjuEVdoBdqam0nhCguCyBifTk wAMgc4MWzDADFR3rCo5N4HsYK7IcJ9evOG4lUyf1WV5NgsZl8U9B6EkO0nqsbLhjjq YhCKW0G1L4U5g== From: Eric Biggers To: linux-kernel@vger.kernel.org Cc: linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Linus Torvalds Subject: [PATCH v2 08/12] lib/crc/riscv: migrate riscv-optimized CRC code into lib/crc/ Date: Sat, 7 Jun 2025 13:04:50 -0700 Message-ID: <20250607200454.73587-9-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250607200454.73587-1-ebiggers@kernel.org> References: <20250607200454.73587-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Move the riscv-optimized CRC code from arch/riscv/lib/crc* into its new location in lib/crc/riscv/, and wire it up in the new way. This new way of organizing the CRC code eliminates the need to artificially split the code for each CRC variant into separate arch and generic modules, enabling better inlining and dead code elimination. For more details, see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/". Signed-off-by: Eric Biggers Reviewed-by: Martin K. Petersen --- arch/riscv/Kconfig | 3 --- arch/riscv/lib/Makefile | 6 ------ lib/crc/Kconfig | 3 +++ lib/crc/Makefile | 3 +++ .../lib =3D> lib/crc/riscv}/crc-clmul-consts.h | 0 .../lib =3D> lib/crc/riscv}/crc-clmul-template.h | 0 {arch/riscv/lib =3D> lib/crc/riscv}/crc-clmul.h | 0 .../crc-t10dif.c =3D> lib/crc/riscv/crc-t10dif.h | 8 +------- {arch/riscv/lib =3D> lib/crc/riscv}/crc16_msb.c | 0 arch/riscv/lib/crc32.c =3D> lib/crc/riscv/crc32.h | 17 ++++------------- {arch/riscv/lib =3D> lib/crc/riscv}/crc32_lsb.c | 0 {arch/riscv/lib =3D> lib/crc/riscv}/crc32_msb.c | 0 arch/riscv/lib/crc64.c =3D> lib/crc/riscv/crc64.h | 11 ++--------- {arch/riscv/lib =3D> lib/crc/riscv}/crc64_lsb.c | 0 {arch/riscv/lib =3D> lib/crc/riscv}/crc64_msb.c | 0 15 files changed, 13 insertions(+), 38 deletions(-) rename {arch/riscv/lib =3D> lib/crc/riscv}/crc-clmul-consts.h (100%) rename {arch/riscv/lib =3D> lib/crc/riscv}/crc-clmul-template.h (100%) rename {arch/riscv/lib =3D> lib/crc/riscv}/crc-clmul.h (100%) rename arch/riscv/lib/crc-t10dif.c =3D> lib/crc/riscv/crc-t10dif.h (62%) rename {arch/riscv/lib =3D> lib/crc/riscv}/crc16_msb.c (100%) rename arch/riscv/lib/crc32.c =3D> lib/crc/riscv/crc32.h (66%) rename {arch/riscv/lib =3D> lib/crc/riscv}/crc32_lsb.c (100%) rename {arch/riscv/lib =3D> lib/crc/riscv}/crc32_msb.c (100%) rename arch/riscv/lib/crc64.c =3D> lib/crc/riscv/crc64.h (65%) rename {arch/riscv/lib =3D> lib/crc/riscv}/crc64_lsb.c (100%) rename {arch/riscv/lib =3D> lib/crc/riscv}/crc64_msb.c (100%) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 36061f4732b74..d963d8faf2aeb 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -22,13 +22,10 @@ config RISCV select ARCH_ENABLE_MEMORY_HOTPLUG if SPARSEMEM_VMEMMAP select ARCH_ENABLE_MEMORY_HOTREMOVE if MEMORY_HOTPLUG select ARCH_ENABLE_SPLIT_PMD_PTLOCK if PGTABLE_LEVELS > 2 select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE select ARCH_HAS_BINFMT_FLAT - select ARCH_HAS_CRC32 if RISCV_ISA_ZBC - select ARCH_HAS_CRC64 if 64BIT && RISCV_ISA_ZBC - select ARCH_HAS_CRC_T10DIF if RISCV_ISA_ZBC select ARCH_HAS_CURRENT_STACK_POINTER select ARCH_HAS_DEBUG_VIRTUAL if MMU select ARCH_HAS_DEBUG_VM_PGTABLE select ARCH_HAS_DEBUG_WX select ARCH_HAS_FAST_MULTIPLIER diff --git a/arch/riscv/lib/Makefile b/arch/riscv/lib/Makefile index 0baec92d2f55b..a4f4b48ed3a47 100644 --- a/arch/riscv/lib/Makefile +++ b/arch/riscv/lib/Makefile @@ -14,14 +14,8 @@ ifeq ($(CONFIG_MMU), y) lib-$(CONFIG_RISCV_ISA_V) +=3D uaccess_vector.o endif lib-$(CONFIG_MMU) +=3D uaccess.o lib-$(CONFIG_64BIT) +=3D tishift.o lib-$(CONFIG_RISCV_ISA_ZICBOZ) +=3D clear_page.o -obj-$(CONFIG_CRC32_ARCH) +=3D crc32-riscv.o -crc32-riscv-y :=3D crc32.o crc32_msb.o crc32_lsb.o -obj-$(CONFIG_CRC64_ARCH) +=3D crc64-riscv.o -crc64-riscv-y :=3D crc64.o crc64_msb.o crc64_lsb.o -obj-$(CONFIG_CRC_T10DIF_ARCH) +=3D crc-t10dif-riscv.o -crc-t10dif-riscv-y :=3D crc-t10dif.o crc16_msb.o obj-$(CONFIG_FUNCTION_ERROR_INJECTION) +=3D error-inject.o lib-$(CONFIG_RISCV_ISA_V) +=3D xor.o lib-$(CONFIG_RISCV_ISA_V) +=3D riscv_v_helpers.o diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig index c3f9091ee512e..d0c293b1a4182 100644 --- a/lib/crc/Kconfig +++ b/lib/crc/Kconfig @@ -51,10 +51,11 @@ config CRC_T10DIF_ARCH bool depends on CRC_T10DIF && CRC_OPTIMIZATIONS default y if ARM && KERNEL_MODE_NEON default y if ARM64 && KERNEL_MODE_NEON default y if PPC64 && ALTIVEC + default y if RISCV && RISCV_ISA_ZBC =20 config CRC32 tristate select BITREVERSE help @@ -70,10 +71,11 @@ config CRC32_ARCH default y if ARM && KERNEL_MODE_NEON default y if ARM64 default y if LOONGARCH default y if MIPS && CPU_MIPSR6 default y if PPC64 && ALTIVEC + default y if RISCV && RISCV_ISA_ZBC =20 config CRC64 tristate help The CRC64 library functions. Select this if your module uses any of @@ -83,10 +85,11 @@ config ARCH_HAS_CRC64 bool =20 config CRC64_ARCH bool depends on CRC64 && CRC_OPTIMIZATIONS + default y if RISCV && RISCV_ISA_ZBC && 64BIT =20 config CRC_OPTIMIZATIONS bool "Enable optimized CRC implementations" if EXPERT default y help diff --git a/lib/crc/Makefile b/lib/crc/Makefile index 555fd3fb6d197..190f889adf556 100644 --- a/lib/crc/Makefile +++ b/lib/crc/Makefile @@ -14,25 +14,28 @@ crc-t10dif-y :=3D crc-t10dif-main.o ifeq ($(CONFIG_CRC_T10DIF_ARCH),y) CFLAGS_crc-t10dif-main.o +=3D -I$(src)/$(SRCARCH) crc-t10dif-$(CONFIG_ARM) +=3D arm/crc-t10dif-core.o crc-t10dif-$(CONFIG_ARM64) +=3D arm64/crc-t10dif-core.o crc-t10dif-$(CONFIG_PPC) +=3D powerpc/crct10dif-vpmsum_asm.o +crc-t10dif-$(CONFIG_RISCV) +=3D riscv/crc16_msb.o endif =20 obj-$(CONFIG_CRC32) +=3D crc32.o crc32-y :=3D crc32-main.o ifeq ($(CONFIG_CRC32_ARCH),y) CFLAGS_crc32-main.o +=3D -I$(src)/$(SRCARCH) crc32-$(CONFIG_ARM) +=3D arm/crc32-core.o crc32-$(CONFIG_ARM64) +=3D arm64/crc32-core.o crc32-$(CONFIG_PPC) +=3D powerpc/crc32c-vpmsum_asm.o +crc32-$(CONFIG_RISCV) +=3D riscv/crc32_lsb.o riscv/crc32_msb.o endif =20 obj-$(CONFIG_CRC64) +=3D crc64.o crc64-y :=3D crc64-main.o ifeq ($(CONFIG_CRC64_ARCH),y) CFLAGS_crc64-main.o +=3D -I$(src)/$(SRCARCH) +crc64-$(CONFIG_RISCV) +=3D riscv/crc64_lsb.o riscv/crc64_msb.o endif =20 obj-y +=3D tests/ =20 hostprogs :=3D gen_crc32table gen_crc64table diff --git a/arch/riscv/lib/crc-clmul-consts.h b/lib/crc/riscv/crc-clmul-co= nsts.h similarity index 100% rename from arch/riscv/lib/crc-clmul-consts.h rename to lib/crc/riscv/crc-clmul-consts.h diff --git a/arch/riscv/lib/crc-clmul-template.h b/lib/crc/riscv/crc-clmul-= template.h similarity index 100% rename from arch/riscv/lib/crc-clmul-template.h rename to lib/crc/riscv/crc-clmul-template.h diff --git a/arch/riscv/lib/crc-clmul.h b/lib/crc/riscv/crc-clmul.h similarity index 100% rename from arch/riscv/lib/crc-clmul.h rename to lib/crc/riscv/crc-clmul.h diff --git a/arch/riscv/lib/crc-t10dif.c b/lib/crc/riscv/crc-t10dif.h similarity index 62% rename from arch/riscv/lib/crc-t10dif.c rename to lib/crc/riscv/crc-t10dif.h index e6b0051ccd86c..cd6136cbfda1c 100644 --- a/arch/riscv/lib/crc-t10dif.c +++ b/lib/crc/riscv/crc-t10dif.h @@ -5,20 +5,14 @@ * Copyright 2025 Google LLC */ =20 #include #include -#include -#include =20 #include "crc-clmul.h" =20 -u16 crc_t10dif_arch(u16 crc, const u8 *p, size_t len) +static inline u16 crc_t10dif_arch(u16 crc, const u8 *p, size_t len) { if (riscv_has_extension_likely(RISCV_ISA_EXT_ZBC)) return crc16_msb_clmul(crc, p, len, &crc16_msb_0x8bb7_consts); return crc_t10dif_generic(crc, p, len); } -EXPORT_SYMBOL(crc_t10dif_arch); - -MODULE_DESCRIPTION("RISC-V optimized CRC-T10DIF function"); -MODULE_LICENSE("GPL"); diff --git a/arch/riscv/lib/crc16_msb.c b/lib/crc/riscv/crc16_msb.c similarity index 100% rename from arch/riscv/lib/crc16_msb.c rename to lib/crc/riscv/crc16_msb.c diff --git a/arch/riscv/lib/crc32.c b/lib/crc/riscv/crc32.h similarity index 66% rename from arch/riscv/lib/crc32.c rename to lib/crc/riscv/crc32.h index a3188b7d9c403..3ec6eee98afa8 100644 --- a/arch/riscv/lib/crc32.c +++ b/lib/crc/riscv/crc32.h @@ -5,49 +5,40 @@ * Copyright 2025 Google LLC */ =20 #include #include -#include -#include =20 #include "crc-clmul.h" =20 -u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) +static inline u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) { if (riscv_has_extension_likely(RISCV_ISA_EXT_ZBC)) return crc32_lsb_clmul(crc, p, len, &crc32_lsb_0xedb88320_consts); return crc32_le_base(crc, p, len); } -EXPORT_SYMBOL(crc32_le_arch); =20 -u32 crc32_be_arch(u32 crc, const u8 *p, size_t len) +static inline u32 crc32_be_arch(u32 crc, const u8 *p, size_t len) { if (riscv_has_extension_likely(RISCV_ISA_EXT_ZBC)) return crc32_msb_clmul(crc, p, len, &crc32_msb_0x04c11db7_consts); return crc32_be_base(crc, p, len); } -EXPORT_SYMBOL(crc32_be_arch); =20 -u32 crc32c_arch(u32 crc, const u8 *p, size_t len) +static inline u32 crc32c_arch(u32 crc, const u8 *p, size_t len) { if (riscv_has_extension_likely(RISCV_ISA_EXT_ZBC)) return crc32_lsb_clmul(crc, p, len, &crc32_lsb_0x82f63b78_consts); return crc32c_base(crc, p, len); } -EXPORT_SYMBOL(crc32c_arch); =20 -u32 crc32_optimizations(void) +static inline u32 crc32_optimizations_arch(void) { if (riscv_has_extension_likely(RISCV_ISA_EXT_ZBC)) return CRC32_LE_OPTIMIZATION | CRC32_BE_OPTIMIZATION | CRC32C_OPTIMIZATION; return 0; } -EXPORT_SYMBOL(crc32_optimizations); - -MODULE_DESCRIPTION("RISC-V optimized CRC32 functions"); -MODULE_LICENSE("GPL"); diff --git a/arch/riscv/lib/crc32_lsb.c b/lib/crc/riscv/crc32_lsb.c similarity index 100% rename from arch/riscv/lib/crc32_lsb.c rename to lib/crc/riscv/crc32_lsb.c diff --git a/arch/riscv/lib/crc32_msb.c b/lib/crc/riscv/crc32_msb.c similarity index 100% rename from arch/riscv/lib/crc32_msb.c rename to lib/crc/riscv/crc32_msb.c diff --git a/arch/riscv/lib/crc64.c b/lib/crc/riscv/crc64.h similarity index 65% rename from arch/riscv/lib/crc64.c rename to lib/crc/riscv/crc64.h index f0015a27836a4..a1b7873fde579 100644 --- a/arch/riscv/lib/crc64.c +++ b/lib/crc/riscv/crc64.h @@ -5,30 +5,23 @@ * Copyright 2025 Google LLC */ =20 #include #include -#include -#include =20 #include "crc-clmul.h" =20 -u64 crc64_be_arch(u64 crc, const u8 *p, size_t len) +static inline u64 crc64_be_arch(u64 crc, const u8 *p, size_t len) { if (riscv_has_extension_likely(RISCV_ISA_EXT_ZBC)) return crc64_msb_clmul(crc, p, len, &crc64_msb_0x42f0e1eba9ea3693_consts); return crc64_be_generic(crc, p, len); } -EXPORT_SYMBOL(crc64_be_arch); =20 -u64 crc64_nvme_arch(u64 crc, const u8 *p, size_t len) +static inline u64 crc64_nvme_arch(u64 crc, const u8 *p, size_t len) { if (riscv_has_extension_likely(RISCV_ISA_EXT_ZBC)) return crc64_lsb_clmul(crc, p, len, &crc64_lsb_0x9a6c9329ac4bc9b5_consts); return crc64_nvme_generic(crc, p, len); } -EXPORT_SYMBOL(crc64_nvme_arch); - -MODULE_DESCRIPTION("RISC-V optimized CRC64 functions"); -MODULE_LICENSE("GPL"); diff --git a/arch/riscv/lib/crc64_lsb.c b/lib/crc/riscv/crc64_lsb.c similarity index 100% rename from arch/riscv/lib/crc64_lsb.c rename to lib/crc/riscv/crc64_lsb.c diff --git a/arch/riscv/lib/crc64_msb.c b/lib/crc/riscv/crc64_msb.c similarity index 100% rename from arch/riscv/lib/crc64_msb.c rename to lib/crc/riscv/crc64_msb.c --=20 2.49.0 From nobody Fri Dec 19 07:31:42 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6091A21CC5F; Sat, 7 Jun 2025 20:07:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326853; cv=none; b=iaaK6fv+DBd/ZFJ3hW0mBW4XR5DZlkXuBcW5WQhOjiIMUv+E4j7UQIOwbzBPe8v49zeO1jRcGtF/Py7dX6O5uJm/1NX5WRs7YzYz8kN0WGiA7f8xApIDaIbVzHqhZmNvLL5E33yHW6eh3tUPLE9dNFyopftDK1HaHwlhtOtk/o8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326853; c=relaxed/simple; bh=167zq5zi7wdD08Zx3i/mgZI+CHCjTsXkRCx5v5LY1DE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RZ52QW8XYbtvNPO64QKlJD1/6tyAw1tl7fptI1twf0UWirUW4boH83PhnTIMRd7hhQW+diE9KE0zLKkSQ08NAXFmfF0q4ObICpejmT3VTGjAPTZ8Xo6kNjVilaxLM1Ok6Bugr3TKJ6NxQbK01A3Ok7AJPV+Mvc53/og7bwXm3hc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cK97FH/u; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cK97FH/u" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87A05C4AF09; Sat, 7 Jun 2025 20:07:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1749326852; bh=167zq5zi7wdD08Zx3i/mgZI+CHCjTsXkRCx5v5LY1DE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cK97FH/u1ZexrL4xPo108kbPSyiyqv5D8XHaqv+mjS2WIeFt3z02kKZQoPSDq/X9G neVztBJCZmrBoT35lHT7/4k1mw3IU50u8eK2I8rCFsAO3s5eNyH+UgVyYXahnJYGwP TiPrvKloWZbvdMpC4iYVrFLfzw9UqgZm3B60tJ2p04gkZ1fhmhjAqppFmzZj0+0I9k fZrdwvK3bggFG+eTMFGHLb/guQtWXCbVKytVYD912FXa4Gfgsp4iP8TdQzvl2cNx9/ p7CAFgRZSOtmmJ1+fxlmzhOAzyN+XY2FyOZvJtvXu7z3ScuzRfpONDpwE/CuTrEy5Y iwR5ANOjA/+Mw== From: Eric Biggers To: linux-kernel@vger.kernel.org Cc: linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Linus Torvalds Subject: [PATCH v2 09/12] lib/crc/s390: migrate s390-optimized CRC code into lib/crc/ Date: Sat, 7 Jun 2025 13:04:51 -0700 Message-ID: <20250607200454.73587-10-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250607200454.73587-1-ebiggers@kernel.org> References: <20250607200454.73587-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Move the s390-optimized CRC code from arch/s390/lib/crc* into its new location in lib/crc/s390/, and wire it up in the new way. This new way of organizing the CRC code eliminates the need to artificially split the code for each CRC variant into separate arch and generic modules, enabling better inlining and dead code elimination. For more details, see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/". Signed-off-by: Eric Biggers Reviewed-by: Martin K. Petersen --- arch/s390/Kconfig | 1 - arch/s390/lib/Makefile | 3 --- lib/crc/Kconfig | 1 + lib/crc/Makefile | 1 + {arch/s390/lib =3D> lib/crc/s390}/crc32-vx.h | 0 arch/s390/lib/crc32.c =3D> lib/crc/s390/crc32.h | 16 +++------------- {arch/s390/lib =3D> lib/crc/s390}/crc32be-vx.c | 0 {arch/s390/lib =3D> lib/crc/s390}/crc32le-vx.c | 0 8 files changed, 5 insertions(+), 17 deletions(-) rename {arch/s390/lib =3D> lib/crc/s390}/crc32-vx.h (100%) rename arch/s390/lib/crc32.c =3D> lib/crc/s390/crc32.h (81%) rename {arch/s390/lib =3D> lib/crc/s390}/crc32be-vx.c (100%) rename {arch/s390/lib =3D> lib/crc/s390}/crc32le-vx.c (100%) diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig index 0c16dc443e2f6..22b90f6aa1a09 100644 --- a/arch/s390/Kconfig +++ b/arch/s390/Kconfig @@ -73,11 +73,10 @@ config S390 select ARCH_CORRECT_STACKTRACE_ON_KRETPROBE select ARCH_ENABLE_MEMORY_HOTPLUG if SPARSEMEM select ARCH_ENABLE_MEMORY_HOTREMOVE select ARCH_ENABLE_SPLIT_PMD_PTLOCK if PGTABLE_LEVELS > 2 select ARCH_HAS_CPU_FINALIZE_INIT - select ARCH_HAS_CRC32 select ARCH_HAS_CURRENT_STACK_POINTER select ARCH_HAS_DEBUG_VIRTUAL select ARCH_HAS_DEBUG_VM_PGTABLE select ARCH_HAS_DEBUG_WX select ARCH_HAS_DEVMEM_IS_ALLOWED diff --git a/arch/s390/lib/Makefile b/arch/s390/lib/Makefile index cd35cdbfa8713..7c8583d46eca1 100644 --- a/arch/s390/lib/Makefile +++ b/arch/s390/lib/Makefile @@ -23,8 +23,5 @@ obj-$(CONFIG_S390_MODULES_SANITY_TEST) +=3D test_modules.o obj-$(CONFIG_S390_MODULES_SANITY_TEST_HELPERS) +=3D test_modules_helpers.o =20 lib-$(CONFIG_FUNCTION_ERROR_INJECTION) +=3D error-inject.o =20 obj-$(CONFIG_EXPOLINE_EXTERN) +=3D expoline.o - -obj-$(CONFIG_CRC32_ARCH) +=3D crc32-s390.o -crc32-s390-y :=3D crc32.o crc32le-vx.o crc32be-vx.o diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig index d0c293b1a4182..1b69a8bef4a85 100644 --- a/lib/crc/Kconfig +++ b/lib/crc/Kconfig @@ -72,10 +72,11 @@ config CRC32_ARCH default y if ARM64 default y if LOONGARCH default y if MIPS && CPU_MIPSR6 default y if PPC64 && ALTIVEC default y if RISCV && RISCV_ISA_ZBC + default y if S390 =20 config CRC64 tristate help The CRC64 library functions. Select this if your module uses any of diff --git a/lib/crc/Makefile b/lib/crc/Makefile index 190f889adf556..bec58266251f8 100644 --- a/lib/crc/Makefile +++ b/lib/crc/Makefile @@ -25,10 +25,11 @@ ifeq ($(CONFIG_CRC32_ARCH),y) CFLAGS_crc32-main.o +=3D -I$(src)/$(SRCARCH) crc32-$(CONFIG_ARM) +=3D arm/crc32-core.o crc32-$(CONFIG_ARM64) +=3D arm64/crc32-core.o crc32-$(CONFIG_PPC) +=3D powerpc/crc32c-vpmsum_asm.o crc32-$(CONFIG_RISCV) +=3D riscv/crc32_lsb.o riscv/crc32_msb.o +crc32-$(CONFIG_S390) +=3D s390/crc32le-vx.o s390/crc32be-vx.o endif =20 obj-$(CONFIG_CRC64) +=3D crc64.o crc64-y :=3D crc64-main.o ifeq ($(CONFIG_CRC64_ARCH),y) diff --git a/arch/s390/lib/crc32-vx.h b/lib/crc/s390/crc32-vx.h similarity index 100% rename from arch/s390/lib/crc32-vx.h rename to lib/crc/s390/crc32-vx.h diff --git a/arch/s390/lib/crc32.c b/lib/crc/s390/crc32.h similarity index 81% rename from arch/s390/lib/crc32.c rename to lib/crc/s390/crc32.h index 3c4b344417c11..59c8983d428be 100644 --- a/arch/s390/lib/crc32.c +++ b/lib/crc/s390/crc32.h @@ -3,16 +3,12 @@ * CRC-32 implemented with the z/Architecture Vector Extension Facility. * * Copyright IBM Corp. 2015 * Author(s): Hendrik Brueckner */ -#define KMSG_COMPONENT "crc32-vx" -#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt =20 -#include #include -#include #include #include "crc32-vx.h" =20 #define VX_MIN_LEN 64 #define VX_ALIGNMENT 16L @@ -25,11 +21,11 @@ * on the message buffer, the hardware-accelerated or software implementat= ion * is used. Note that the message buffer is aligned to improve fetch * operations of VECTOR LOAD MULTIPLE instructions. */ #define DEFINE_CRC32_VX(___fname, ___crc32_vx, ___crc32_sw) \ - u32 ___fname(u32 crc, const u8 *data, size_t datalen) \ + static inline u32 ___fname(u32 crc, const u8 *data, size_t datalen) \ { \ unsigned long prealign, aligned, remaining; \ DECLARE_KERNEL_FPU_ONSTACK16(vxstate); \ \ if (datalen < VX_MIN_LEN + VX_ALIGN_MASK || !cpu_has_vx()) \ @@ -52,26 +48,20 @@ \ if (remaining) \ crc =3D ___crc32_sw(crc, data + aligned, remaining); \ \ return crc; \ - } \ - EXPORT_SYMBOL(___fname); + } =20 DEFINE_CRC32_VX(crc32_le_arch, crc32_le_vgfm_16, crc32_le_base) DEFINE_CRC32_VX(crc32_be_arch, crc32_be_vgfm_16, crc32_be_base) DEFINE_CRC32_VX(crc32c_arch, crc32c_le_vgfm_16, crc32c_base) =20 -u32 crc32_optimizations(void) +static inline u32 crc32_optimizations_arch(void) { if (cpu_has_vx()) { return CRC32_LE_OPTIMIZATION | CRC32_BE_OPTIMIZATION | CRC32C_OPTIMIZATION; } return 0; } -EXPORT_SYMBOL(crc32_optimizations); - -MODULE_AUTHOR("Hendrik Brueckner "); -MODULE_DESCRIPTION("CRC-32 algorithms using z/Architecture Vector Extensio= n Facility"); -MODULE_LICENSE("GPL"); diff --git a/arch/s390/lib/crc32be-vx.c b/lib/crc/s390/crc32be-vx.c similarity index 100% rename from arch/s390/lib/crc32be-vx.c rename to lib/crc/s390/crc32be-vx.c diff --git a/arch/s390/lib/crc32le-vx.c b/lib/crc/s390/crc32le-vx.c similarity index 100% rename from arch/s390/lib/crc32le-vx.c rename to lib/crc/s390/crc32le-vx.c --=20 2.49.0 From nobody Fri Dec 19 07:31:42 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C38521CC7F; Sat, 7 Jun 2025 20:07:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326853; cv=none; b=FGqIi1YS6JJF8hkAIRr0ymbTdtTmlmA9SnG4VOjv5EmrGbOu3j9ECu0nqKF4bhLR09c8J8koKuUWsigdBLdxbUQQsxK24wkw1jzD8LyeL/aAkKYujw1R4c0PeUFTusBY4981yKraIG+WuvFq3bW20iq8d9KXDr33RVSWwajQeSo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326853; c=relaxed/simple; bh=ml00Eu6L3MCivT8VcjfT2RgfiZJS9znV2CRfUv6dB7E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ir3ui3SN3I9ZqcEKKMg+OL7zpG8YhxDco0cWHqq1bqXXurY8nYEK5iXSeoY4ANq8+pZdC611fCFTPIDMNZO4zD2nE1Ef9Ym9W6gGiVDZ5iNCJs3OVEVwxeAKR3Xbk7minAMzSif9m78t+vwd1K2Kkg4IIxiULEsiYhqy1eOCyjk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Hf6I+Ges; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Hf6I+Ges" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 10C30C4CEE4; Sat, 7 Jun 2025 20:07:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1749326853; bh=ml00Eu6L3MCivT8VcjfT2RgfiZJS9znV2CRfUv6dB7E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Hf6I+GessssiRha4MpA6RFI+58URioJcA8hkf18voXD1e9otmRrh6ut6f37cDXBp+ t02pRBAP/ZlXisnR/Q2kSIylUzzyohdF4xc+dYSaCvPizsqHiblpXpH3M/AcB4DwZm 1gsBgk8eLNYcmlAGFZLxK2WApGsMmbHKpUBzmx0pNNo3RVKGFzhTcZ7ZCx//ZzZdDI hLEhV+ur56mvrGVIPD3EuNCPvrKW9CQHIaAhExDe1uBKmjOUWdJvccInA7Py7CALlr ISUzxfUqOE8bsLzp6szFM0WEjwnQYDirYRxe83hqmv+e8gbpJV65fpLADWeNNXS6YC lNZw0bHLVTzlA== From: Eric Biggers To: linux-kernel@vger.kernel.org Cc: linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Linus Torvalds Subject: [PATCH v2 10/12] lib/crc/sparc: migrate sparc-optimized CRC code into lib/crc/ Date: Sat, 7 Jun 2025 13:04:52 -0700 Message-ID: <20250607200454.73587-11-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250607200454.73587-1-ebiggers@kernel.org> References: <20250607200454.73587-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Move the sparc-optimized CRC code from arch/sparc/lib/crc* into its new location in lib/crc/sparc/, and wire it up in the new way. This new way of organizing the CRC code eliminates the need to artificially split the code for each CRC variant into separate arch and generic modules, enabling better inlining and dead code elimination. For more details, see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/". Signed-off-by: Eric Biggers Reviewed-by: Martin K. Petersen --- arch/sparc/Kconfig | 1 - arch/sparc/lib/Makefile | 2 - lib/crc/Kconfig | 1 + lib/crc/Makefile | 1 + .../lib/crc32.c =3D> lib/crc/sparc/crc32.h | 42 ++++--------------- .../sparc/lib =3D> lib/crc/sparc}/crc32c_asm.S | 0 6 files changed, 10 insertions(+), 37 deletions(-) rename arch/sparc/lib/crc32.c =3D> lib/crc/sparc/crc32.h (60%) rename {arch/sparc/lib =3D> lib/crc/sparc}/crc32c_asm.S (100%) diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig index 0f88123925a4f..dcfdb7f1dae97 100644 --- a/arch/sparc/Kconfig +++ b/arch/sparc/Kconfig @@ -108,11 +108,10 @@ config SPARC64 select ARCH_HAS_GIGANTIC_PAGE select HAVE_SOFTIRQ_ON_OWN_STACK select HAVE_SETUP_PER_CPU_AREA select NEED_PER_CPU_EMBED_FIRST_CHUNK select NEED_PER_CPU_PAGE_FIRST_CHUNK - select ARCH_HAS_CRC32 =20 config ARCH_PROC_KCORE_TEXT def_bool y =20 config CPU_BIG_ENDIAN diff --git a/arch/sparc/lib/Makefile b/arch/sparc/lib/Makefile index 5cf9781d68b40..2d6c3c5352734 100644 --- a/arch/sparc/lib/Makefile +++ b/arch/sparc/lib/Makefile @@ -52,7 +52,5 @@ lib-$(CONFIG_SPARC64) +=3D copy_in_user.o memmove.o lib-$(CONFIG_SPARC64) +=3D mcount.o ipcsum.o xor.o hweight.o ffs.o =20 obj-$(CONFIG_SPARC64) +=3D iomap.o obj-$(CONFIG_SPARC32) +=3D atomic32.o obj-$(CONFIG_SPARC64) +=3D PeeCeeI.o -obj-$(CONFIG_CRC32_ARCH) +=3D crc32-sparc.o -crc32-sparc-y :=3D crc32.o crc32c_asm.o diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig index 1b69a8bef4a85..af4fa857a81f8 100644 --- a/lib/crc/Kconfig +++ b/lib/crc/Kconfig @@ -73,10 +73,11 @@ config CRC32_ARCH default y if LOONGARCH default y if MIPS && CPU_MIPSR6 default y if PPC64 && ALTIVEC default y if RISCV && RISCV_ISA_ZBC default y if S390 + default y if SPARC64 =20 config CRC64 tristate help The CRC64 library functions. Select this if your module uses any of diff --git a/lib/crc/Makefile b/lib/crc/Makefile index bec58266251f8..81e176db0947c 100644 --- a/lib/crc/Makefile +++ b/lib/crc/Makefile @@ -26,10 +26,11 @@ CFLAGS_crc32-main.o +=3D -I$(src)/$(SRCARCH) crc32-$(CONFIG_ARM) +=3D arm/crc32-core.o crc32-$(CONFIG_ARM64) +=3D arm64/crc32-core.o crc32-$(CONFIG_PPC) +=3D powerpc/crc32c-vpmsum_asm.o crc32-$(CONFIG_RISCV) +=3D riscv/crc32_lsb.o riscv/crc32_msb.o crc32-$(CONFIG_S390) +=3D s390/crc32le-vx.o s390/crc32be-vx.o +crc32-$(CONFIG_SPARC) +=3D sparc/crc32c_asm.o endif =20 obj-$(CONFIG_CRC64) +=3D crc64.o crc64-y :=3D crc64-main.o ifeq ($(CONFIG_CRC64_ARCH),y) diff --git a/arch/sparc/lib/crc32.c b/lib/crc/sparc/crc32.h similarity index 60% rename from arch/sparc/lib/crc32.c rename to lib/crc/sparc/crc32.h index 40d4720a42a1b..60f2765ac0157 100644 --- a/arch/sparc/lib/crc32.c +++ b/lib/crc/sparc/crc32.h @@ -6,30 +6,21 @@ * Copyright (C) 2008 Intel Corporation * Authors: Austin Zhang * Kent Liu */ =20 -#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt - -#include -#include -#include -#include #include #include =20 static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_crc32c_opcode); =20 -u32 crc32_le_arch(u32 crc, const u8 *data, size_t len) -{ - return crc32_le_base(crc, data, len); -} -EXPORT_SYMBOL(crc32_le_arch); +#define crc32_le_arch crc32_le_base /* not implemented on this arch */ +#define crc32_be_arch crc32_be_base /* not implemented on this arch */ =20 void crc32c_sparc64(u32 *crcp, const u64 *data, size_t len); =20 -u32 crc32c_arch(u32 crc, const u8 *data, size_t len) +static inline u32 crc32c_arch(u32 crc, const u8 *data, size_t len) { size_t n =3D -(uintptr_t)data & 7; =20 if (!static_branch_likely(&have_crc32c_opcode)) return crc32c_base(crc, data, len); @@ -49,45 +40,28 @@ u32 crc32c_arch(u32 crc, const u8 *data, size_t len) } if (len) crc =3D crc32c_base(crc, data, len); return crc; } -EXPORT_SYMBOL(crc32c_arch); - -u32 crc32_be_arch(u32 crc, const u8 *data, size_t len) -{ - return crc32_be_base(crc, data, len); -} -EXPORT_SYMBOL(crc32_be_arch); =20 -static int __init crc32_sparc_init(void) +#define crc32_mod_init_arch crc32_mod_init_arch +static inline void crc32_mod_init_arch(void) { unsigned long cfr; =20 if (!(sparc64_elf_hwcap & HWCAP_SPARC_CRYPTO)) - return 0; + return; =20 __asm__ __volatile__("rd %%asr26, %0" : "=3Dr" (cfr)); if (!(cfr & CFR_CRC32C)) - return 0; + return; =20 static_branch_enable(&have_crc32c_opcode); pr_info("Using sparc64 crc32c opcode optimized CRC32C implementation\n"); - return 0; } -subsys_initcall(crc32_sparc_init); =20 -static void __exit crc32_sparc_exit(void) -{ -} -module_exit(crc32_sparc_exit); - -u32 crc32_optimizations(void) +static inline u32 crc32_optimizations_arch(void) { if (static_key_enabled(&have_crc32c_opcode)) return CRC32C_OPTIMIZATION; return 0; } -EXPORT_SYMBOL(crc32_optimizations); - -MODULE_LICENSE("GPL"); -MODULE_DESCRIPTION("CRC32c (Castagnoli), sparc64 crc32c opcode accelerated= "); diff --git a/arch/sparc/lib/crc32c_asm.S b/lib/crc/sparc/crc32c_asm.S similarity index 100% rename from arch/sparc/lib/crc32c_asm.S rename to lib/crc/sparc/crc32c_asm.S --=20 2.49.0 From nobody Fri Dec 19 07:31:42 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 82DA921FF38; Sat, 7 Jun 2025 20:07:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326854; cv=none; b=Le3bREux5RSKrGH4WrBzCFdgBWhDG9xXBo6t3jzyzXyqA1ahpoKAIMnc/PBlswCXRnfbkeieG6H/GE13HzOobMSDJIljU3IYgJ3LtiNV9L3tIVx+Dyy5Wc76t/AD2jA8bZcCJl9kzkQbW5UUi3xAm3aaHLyX2kKS3hTsTyTdHEk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326854; c=relaxed/simple; bh=jIlzvyOH3lz8ceAH2y7iElk7+J1Jw8+v3KjhKiaWPsc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GgYGfrpdOTKZ0e9aBYthtAbXCSWf8I/9OOwxld3f6vZoqlChYdv4o1kOkscve/JMAhYACWGP4iPD2Ess+hZ3NMn87sFYkQgKO7TRileJY1hZuNb3HVKoZTU3vJzNdr8rZy0SR/t0RMKDBPj35gIbopUI4JOfFZsenOovw1ZDsUc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mLj0TS3w; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mLj0TS3w" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9269EC4CEF1; Sat, 7 Jun 2025 20:07:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1749326854; bh=jIlzvyOH3lz8ceAH2y7iElk7+J1Jw8+v3KjhKiaWPsc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mLj0TS3wOnKV3XdlHRKqAtQfuh56XhiYerch2IoCBZDySahPA9vBdGkFovPasy7BB ri7jwBaWZ3G3CjFkKNWJ5CjTweFueFDBA0HO5kf47VkD0JsVJzkQ1eoA2jSA6YrdVE HdVGUu/5vklpc5Pvo6rMK9nTdMtQ0LYgXKCwNmOCw+qci2pUAl9ZLtUYeSP2fgMMn0 NbKuJqU2PgJIMSub5oqyFfbIg6yKXMiTedYAcxHj68USQFujMAxP02FbSWO4Hps5mY EPUaKyVJbYPOTZfokOSvEdI6dM7valKr/PA4vnSyIGDJ7mosNmRw1MzfhqSninIrPL FnJPsPRJfImZg== From: Eric Biggers To: linux-kernel@vger.kernel.org Cc: linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Linus Torvalds Subject: [PATCH v2 11/12] lib/crc/x86: migrate x86-optimized CRC code into lib/crc/ Date: Sat, 7 Jun 2025 13:04:53 -0700 Message-ID: <20250607200454.73587-12-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250607200454.73587-1-ebiggers@kernel.org> References: <20250607200454.73587-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers Move the x86-optimized CRC code from arch/x86/lib/crc* into its new location in lib/crc/x86/, and wire it up in the new way. This new way of organizing the CRC code eliminates the need to artificially split the code for each CRC variant into separate arch and generic modules, enabling better inlining and dead code elimination. For more details, see "lib/crc: prepare for arch-optimized code in subdirs of lib/crc/". Signed-off-by: Eric Biggers Reviewed-by: Martin K. Petersen --- arch/x86/Kconfig | 3 -- arch/x86/lib/Makefile | 10 ------- lib/crc/Kconfig | 3 ++ lib/crc/Makefile | 4 +++ .../lib =3D> lib/crc/x86}/crc-pclmul-consts.h | 0 .../lib =3D> lib/crc/x86}/crc-pclmul-template.S | 0 .../lib =3D> lib/crc/x86}/crc-pclmul-template.h | 0 .../crc-t10dif.c =3D> lib/crc/x86/crc-t10dif.h | 18 ++--------- .../lib =3D> lib/crc/x86}/crc16-msb-pclmul.S | 0 {arch/x86/lib =3D> lib/crc/x86}/crc32-pclmul.S | 0 arch/x86/lib/crc32.c =3D> lib/crc/x86/crc32.h | 30 ++++--------------- {arch/x86/lib =3D> lib/crc/x86}/crc32c-3way.S | 0 {arch/x86/lib =3D> lib/crc/x86}/crc64-pclmul.S | 0 arch/x86/lib/crc64.c =3D> lib/crc/x86/crc64.h | 21 +++---------- 14 files changed, 20 insertions(+), 69 deletions(-) rename {arch/x86/lib =3D> lib/crc/x86}/crc-pclmul-consts.h (100%) rename {arch/x86/lib =3D> lib/crc/x86}/crc-pclmul-template.S (100%) rename {arch/x86/lib =3D> lib/crc/x86}/crc-pclmul-template.h (100%) rename arch/x86/lib/crc-t10dif.c =3D> lib/crc/x86/crc-t10dif.h (56%) rename {arch/x86/lib =3D> lib/crc/x86}/crc16-msb-pclmul.S (100%) rename {arch/x86/lib =3D> lib/crc/x86}/crc32-pclmul.S (100%) rename arch/x86/lib/crc32.c =3D> lib/crc/x86/crc32.h (76%) rename {arch/x86/lib =3D> lib/crc/x86}/crc32c-3way.S (100%) rename {arch/x86/lib =3D> lib/crc/x86}/crc64-pclmul.S (100%) rename arch/x86/lib/crc64.c =3D> lib/crc/x86/crc64.h (61%) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 340e5468980e0..6181f414f87ec 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -77,13 +77,10 @@ config X86 select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI select ARCH_HAS_CACHE_LINE_SIZE select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION select ARCH_HAS_CPU_FINALIZE_INIT select ARCH_HAS_CPU_PASID if IOMMU_SVA - select ARCH_HAS_CRC32 - select ARCH_HAS_CRC64 if X86_64 - select ARCH_HAS_CRC_T10DIF select ARCH_HAS_CURRENT_STACK_POINTER select ARCH_HAS_DEBUG_VIRTUAL select ARCH_HAS_DEBUG_VM_PGTABLE if !X86_PAE select ARCH_HAS_DEVMEM_IS_ALLOWED select ARCH_HAS_DMA_OPS if GART_IOMMU || XEN diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile index 4fa5c4e1ba8a0..dc5ee2a6938c4 100644 --- a/arch/x86/lib/Makefile +++ b/arch/x86/lib/Makefile @@ -38,20 +38,10 @@ lib-$(CONFIG_ARCH_HAS_COPY_MC) +=3D copy_mc.o copy_mc_6= 4.o lib-$(CONFIG_INSTRUCTION_DECODER) +=3D insn.o inat.o insn-eval.o lib-$(CONFIG_RANDOMIZE_BASE) +=3D kaslr.o lib-$(CONFIG_FUNCTION_ERROR_INJECTION) +=3D error-inject.o lib-$(CONFIG_MITIGATION_RETPOLINE) +=3D retpoline.o =20 -obj-$(CONFIG_CRC32_ARCH) +=3D crc32-x86.o -crc32-x86-y :=3D crc32.o crc32-pclmul.o -crc32-x86-$(CONFIG_64BIT) +=3D crc32c-3way.o - -obj-$(CONFIG_CRC64_ARCH) +=3D crc64-x86.o -crc64-x86-y :=3D crc64.o crc64-pclmul.o - -obj-$(CONFIG_CRC_T10DIF_ARCH) +=3D crc-t10dif-x86.o -crc-t10dif-x86-y :=3D crc-t10dif.o crc16-msb-pclmul.o - obj-y +=3D msr.o msr-reg.o msr-reg-export.o hweight.o obj-y +=3D iomem.o =20 ifeq ($(CONFIG_X86_32),y) obj-y +=3D atomic64_32.o diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig index af4fa857a81f8..de4b2182ae7ff 100644 --- a/lib/crc/Kconfig +++ b/lib/crc/Kconfig @@ -52,10 +52,11 @@ config CRC_T10DIF_ARCH depends on CRC_T10DIF && CRC_OPTIMIZATIONS default y if ARM && KERNEL_MODE_NEON default y if ARM64 && KERNEL_MODE_NEON default y if PPC64 && ALTIVEC default y if RISCV && RISCV_ISA_ZBC + default y if X86 =20 config CRC32 tristate select BITREVERSE help @@ -74,10 +75,11 @@ config CRC32_ARCH default y if MIPS && CPU_MIPSR6 default y if PPC64 && ALTIVEC default y if RISCV && RISCV_ISA_ZBC default y if S390 default y if SPARC64 + default y if X86 =20 config CRC64 tristate help The CRC64 library functions. Select this if your module uses any of @@ -88,10 +90,11 @@ config ARCH_HAS_CRC64 =20 config CRC64_ARCH bool depends on CRC64 && CRC_OPTIMIZATIONS default y if RISCV && RISCV_ISA_ZBC && 64BIT + default y if X86_64 =20 config CRC_OPTIMIZATIONS bool "Enable optimized CRC implementations" if EXPERT default y help diff --git a/lib/crc/Makefile b/lib/crc/Makefile index 81e176db0947c..7543ad295ab6f 100644 --- a/lib/crc/Makefile +++ b/lib/crc/Makefile @@ -15,10 +15,11 @@ ifeq ($(CONFIG_CRC_T10DIF_ARCH),y) CFLAGS_crc-t10dif-main.o +=3D -I$(src)/$(SRCARCH) crc-t10dif-$(CONFIG_ARM) +=3D arm/crc-t10dif-core.o crc-t10dif-$(CONFIG_ARM64) +=3D arm64/crc-t10dif-core.o crc-t10dif-$(CONFIG_PPC) +=3D powerpc/crct10dif-vpmsum_asm.o crc-t10dif-$(CONFIG_RISCV) +=3D riscv/crc16_msb.o +crc-t10dif-$(CONFIG_X86) +=3D x86/crc16-msb-pclmul.o endif =20 obj-$(CONFIG_CRC32) +=3D crc32.o crc32-y :=3D crc32-main.o ifeq ($(CONFIG_CRC32_ARCH),y) @@ -27,17 +28,20 @@ crc32-$(CONFIG_ARM) +=3D arm/crc32-core.o crc32-$(CONFIG_ARM64) +=3D arm64/crc32-core.o crc32-$(CONFIG_PPC) +=3D powerpc/crc32c-vpmsum_asm.o crc32-$(CONFIG_RISCV) +=3D riscv/crc32_lsb.o riscv/crc32_msb.o crc32-$(CONFIG_S390) +=3D s390/crc32le-vx.o s390/crc32be-vx.o crc32-$(CONFIG_SPARC) +=3D sparc/crc32c_asm.o +crc32-$(CONFIG_X86) +=3D x86/crc32-pclmul.o +crc32-$(CONFIG_X86_64) +=3D x86/crc32c-3way.o endif =20 obj-$(CONFIG_CRC64) +=3D crc64.o crc64-y :=3D crc64-main.o ifeq ($(CONFIG_CRC64_ARCH),y) CFLAGS_crc64-main.o +=3D -I$(src)/$(SRCARCH) crc64-$(CONFIG_RISCV) +=3D riscv/crc64_lsb.o riscv/crc64_msb.o +crc64-$(CONFIG_X86) +=3D x86/crc64-pclmul.o endif =20 obj-y +=3D tests/ =20 hostprogs :=3D gen_crc32table gen_crc64table diff --git a/arch/x86/lib/crc-pclmul-consts.h b/lib/crc/x86/crc-pclmul-cons= ts.h similarity index 100% rename from arch/x86/lib/crc-pclmul-consts.h rename to lib/crc/x86/crc-pclmul-consts.h diff --git a/arch/x86/lib/crc-pclmul-template.S b/lib/crc/x86/crc-pclmul-te= mplate.S similarity index 100% rename from arch/x86/lib/crc-pclmul-template.S rename to lib/crc/x86/crc-pclmul-template.S diff --git a/arch/x86/lib/crc-pclmul-template.h b/lib/crc/x86/crc-pclmul-te= mplate.h similarity index 100% rename from arch/x86/lib/crc-pclmul-template.h rename to lib/crc/x86/crc-pclmul-template.h diff --git a/arch/x86/lib/crc-t10dif.c b/lib/crc/x86/crc-t10dif.h similarity index 56% rename from arch/x86/lib/crc-t10dif.c rename to lib/crc/x86/crc-t10dif.h index db7ce59c31ace..eb1f23db4daa1 100644 --- a/arch/x86/lib/crc-t10dif.c +++ b/lib/crc/x86/crc-t10dif.h @@ -3,38 +3,26 @@ * CRC-T10DIF using [V]PCLMULQDQ instructions * * Copyright 2024 Google LLC */ =20 -#include -#include #include "crc-pclmul-template.h" =20 static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_pclmulqdq); =20 DECLARE_CRC_PCLMUL_FUNCS(crc16_msb, u16); =20 -u16 crc_t10dif_arch(u16 crc, const u8 *p, size_t len) +static inline u16 crc_t10dif_arch(u16 crc, const u8 *p, size_t len) { CRC_PCLMUL(crc, p, len, crc16_msb, crc16_msb_0x8bb7_consts, have_pclmulqdq); return crc_t10dif_generic(crc, p, len); } -EXPORT_SYMBOL(crc_t10dif_arch); =20 -static int __init crc_t10dif_x86_init(void) +#define crc_t10dif_mod_init_arch crc_t10dif_mod_init_arch +static inline void crc_t10dif_mod_init_arch(void) { if (boot_cpu_has(X86_FEATURE_PCLMULQDQ)) { static_branch_enable(&have_pclmulqdq); INIT_CRC_PCLMUL(crc16_msb); } - return 0; } -subsys_initcall(crc_t10dif_x86_init); - -static void __exit crc_t10dif_x86_exit(void) -{ -} -module_exit(crc_t10dif_x86_exit); - -MODULE_DESCRIPTION("CRC-T10DIF using [V]PCLMULQDQ instructions"); -MODULE_LICENSE("GPL"); diff --git a/arch/x86/lib/crc16-msb-pclmul.S b/lib/crc/x86/crc16-msb-pclmul= .S similarity index 100% rename from arch/x86/lib/crc16-msb-pclmul.S rename to lib/crc/x86/crc16-msb-pclmul.S diff --git a/arch/x86/lib/crc32-pclmul.S b/lib/crc/x86/crc32-pclmul.S similarity index 100% rename from arch/x86/lib/crc32-pclmul.S rename to lib/crc/x86/crc32-pclmul.S diff --git a/arch/x86/lib/crc32.c b/lib/crc/x86/crc32.h similarity index 76% rename from arch/x86/lib/crc32.c rename to lib/crc/x86/crc32.h index d09343e2cea93..28451d5769c3a 100644 --- a/arch/x86/lib/crc32.c +++ b/lib/crc/x86/crc32.h @@ -5,26 +5,23 @@ * Copyright (C) 2008 Intel Corporation * Copyright 2012 Xyratex Technology Limited * Copyright 2024 Google LLC */ =20 -#include -#include #include "crc-pclmul-template.h" =20 static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_crc32); static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_pclmulqdq); =20 DECLARE_CRC_PCLMUL_FUNCS(crc32_lsb, u32); =20 -u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) +static inline u32 crc32_le_arch(u32 crc, const u8 *p, size_t len) { CRC_PCLMUL(crc, p, len, crc32_lsb, crc32_lsb_0xedb88320_consts, have_pclmulqdq); return crc32_le_base(crc, p, len); } -EXPORT_SYMBOL(crc32_le_arch); =20 #ifdef CONFIG_X86_64 #define CRC32_INST "crc32q %1, %q0" #else #define CRC32_INST "crc32l %1, %0" @@ -36,11 +33,11 @@ EXPORT_SYMBOL(crc32_le_arch); */ #define CRC32C_PCLMUL_BREAKEVEN 512 =20 asmlinkage u32 crc32c_x86_3way(u32 crc, const u8 *buffer, size_t len); =20 -u32 crc32c_arch(u32 crc, const u8 *p, size_t len) +static inline u32 crc32c_arch(u32 crc, const u8 *p, size_t len) { size_t num_longs; =20 if (!static_branch_likely(&have_crc32)) return crc32c_base(crc, p, len); @@ -68,44 +65,29 @@ u32 crc32c_arch(u32 crc, const u8 *p, size_t len) if (len & 1) asm("crc32b %1, %0" : "+r" (crc) : ASM_INPUT_RM (*p)); =20 return crc; } -EXPORT_SYMBOL(crc32c_arch); =20 -u32 crc32_be_arch(u32 crc, const u8 *p, size_t len) -{ - return crc32_be_base(crc, p, len); -} -EXPORT_SYMBOL(crc32_be_arch); +#define crc32_be_arch crc32_be_base /* not implemented on this arch */ =20 -static int __init crc32_x86_init(void) +#define crc32_mod_init_arch crc32_mod_init_arch +static inline void crc32_mod_init_arch(void) { if (boot_cpu_has(X86_FEATURE_XMM4_2)) static_branch_enable(&have_crc32); if (boot_cpu_has(X86_FEATURE_PCLMULQDQ)) { static_branch_enable(&have_pclmulqdq); INIT_CRC_PCLMUL(crc32_lsb); } - return 0; -} -subsys_initcall(crc32_x86_init); - -static void __exit crc32_x86_exit(void) -{ } -module_exit(crc32_x86_exit); =20 -u32 crc32_optimizations(void) +static inline u32 crc32_optimizations_arch(void) { u32 optimizations =3D 0; =20 if (static_key_enabled(&have_crc32)) optimizations |=3D CRC32C_OPTIMIZATION; if (static_key_enabled(&have_pclmulqdq)) optimizations |=3D CRC32_LE_OPTIMIZATION; return optimizations; } -EXPORT_SYMBOL(crc32_optimizations); - -MODULE_DESCRIPTION("x86-optimized CRC32 functions"); -MODULE_LICENSE("GPL"); diff --git a/arch/x86/lib/crc32c-3way.S b/lib/crc/x86/crc32c-3way.S similarity index 100% rename from arch/x86/lib/crc32c-3way.S rename to lib/crc/x86/crc32c-3way.S diff --git a/arch/x86/lib/crc64-pclmul.S b/lib/crc/x86/crc64-pclmul.S similarity index 100% rename from arch/x86/lib/crc64-pclmul.S rename to lib/crc/x86/crc64-pclmul.S diff --git a/arch/x86/lib/crc64.c b/lib/crc/x86/crc64.h similarity index 61% rename from arch/x86/lib/crc64.c rename to lib/crc/x86/crc64.h index 351a09f5813e2..54aca3a9475c9 100644 --- a/arch/x86/lib/crc64.c +++ b/lib/crc/x86/crc64.h @@ -3,48 +3,35 @@ * CRC64 using [V]PCLMULQDQ instructions * * Copyright 2025 Google LLC */ =20 -#include -#include #include "crc-pclmul-template.h" =20 static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_pclmulqdq); =20 DECLARE_CRC_PCLMUL_FUNCS(crc64_msb, u64); DECLARE_CRC_PCLMUL_FUNCS(crc64_lsb, u64); =20 -u64 crc64_be_arch(u64 crc, const u8 *p, size_t len) +static inline u64 crc64_be_arch(u64 crc, const u8 *p, size_t len) { CRC_PCLMUL(crc, p, len, crc64_msb, crc64_msb_0x42f0e1eba9ea3693_consts, have_pclmulqdq); return crc64_be_generic(crc, p, len); } -EXPORT_SYMBOL_GPL(crc64_be_arch); =20 -u64 crc64_nvme_arch(u64 crc, const u8 *p, size_t len) +static inline u64 crc64_nvme_arch(u64 crc, const u8 *p, size_t len) { CRC_PCLMUL(crc, p, len, crc64_lsb, crc64_lsb_0x9a6c9329ac4bc9b5_consts, have_pclmulqdq); return crc64_nvme_generic(crc, p, len); } -EXPORT_SYMBOL_GPL(crc64_nvme_arch); =20 -static int __init crc64_x86_init(void) +#define crc64_mod_init_arch crc64_mod_init_arch +static inline void crc64_mod_init_arch(void) { if (boot_cpu_has(X86_FEATURE_PCLMULQDQ)) { static_branch_enable(&have_pclmulqdq); INIT_CRC_PCLMUL(crc64_msb); INIT_CRC_PCLMUL(crc64_lsb); } - return 0; } -subsys_initcall(crc64_x86_init); - -static void __exit crc64_x86_exit(void) -{ -} -module_exit(crc64_x86_exit); - -MODULE_DESCRIPTION("CRC64 using [V]PCLMULQDQ instructions"); -MODULE_LICENSE("GPL"); --=20 2.49.0 From nobody Fri Dec 19 07:31:42 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95CD021FF3D; Sat, 7 Jun 2025 20:07:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326854; cv=none; b=oJYwXHAi4CV7uQLUdu8p7PSmILQy96wboUnQBE4umnMByR6F2+a/CG7Yc/Y0giYpuUtT5O2Eq9sICMOSDGb5jFgplksZq1lGg9fmHqSAbxb0ImwKbbECPuVliVBjclKBKLWyX6zmeP5LuHBjlEdsGXLIWBx6ru/ce7icTzJkftM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749326854; c=relaxed/simple; bh=ZDBXUza5mYKN0G+LEgQAmbkZEIUs7iZW79q0Idl2Apk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=f8bMCWEHJi3GaHGjHZ91WvkCVuer56u7SvzmFekeRp4gfWBCLYuFycACl2sT//O0uGPr1YJD1nULl74cta1UbZl8z1qZLf/3yY3ZgqCZoBjIJuyZaR1mhzKHwWPWZ98ab1oml2DE6D+RMNMpm6+YjNEK80JMSVdQvz8nhXyOSjk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ajEhG5/8; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ajEhG5/8" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1E744C4CEE4; Sat, 7 Jun 2025 20:07:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1749326854; bh=ZDBXUza5mYKN0G+LEgQAmbkZEIUs7iZW79q0Idl2Apk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ajEhG5/8HVrfaad1n+slUIYvzE1FkTzDHcl7TWi7cGlPQnDlA04WdKniPoxYQOMYE NFPprqJK5IW0ur0fiG3+y7GcPTyQuszc7j+8AfGYwPfJ+iPIu+ZTxAE0RYDH2G8FKm BJJAZ7QkJFPdzFyT6dT+BjT8tLQ1siQLlN12X59hwHDp/q3vgGfG3SBu0/JsI+taKk yOMBHSm3WUeEOXPURUAoEtz4nwBTplDDJdpMj/2dInexP90CK4TPNO12gzd+vmDwg1 x3S8BGN0sPA3MXigsw27iqzjq/8mJ/jiCprDcU9qVKaUU6hKLM3CwKWLsWamrSVvTN 6UALswZ1OpPdQ== From: Eric Biggers To: linux-kernel@vger.kernel.org Cc: linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org, linux-arch@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Linus Torvalds Subject: [PATCH v2 12/12] lib/crc: remove ARCH_HAS_* kconfig symbols Date: Sat, 7 Jun 2025 13:04:54 -0700 Message-ID: <20250607200454.73587-13-ebiggers@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250607200454.73587-1-ebiggers@kernel.org> References: <20250607200454.73587-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Eric Biggers These symbols are no longer used, so remove them. Signed-off-by: Eric Biggers Reviewed-by: Martin K. Petersen --- lib/crc/Kconfig | 9 --------- 1 file changed, 9 deletions(-) diff --git a/lib/crc/Kconfig b/lib/crc/Kconfig index de4b2182ae7ff..5858b3acc6630 100644 --- a/lib/crc/Kconfig +++ b/lib/crc/Kconfig @@ -42,13 +42,10 @@ config CRC_T10DIF tristate help The CRC-T10DIF library functions. Select this if your module uses any of the functions from . =20 -config ARCH_HAS_CRC_T10DIF - bool - config CRC_T10DIF_ARCH bool depends on CRC_T10DIF && CRC_OPTIMIZATIONS default y if ARM && KERNEL_MODE_NEON default y if ARM64 && KERNEL_MODE_NEON @@ -61,13 +58,10 @@ config CRC32 select BITREVERSE help The CRC32 library functions. Select this if your module uses any of the functions from or . =20 -config ARCH_HAS_CRC32 - bool - config CRC32_ARCH bool depends on CRC32 && CRC_OPTIMIZATIONS default y if ARM && KERNEL_MODE_NEON default y if ARM64 @@ -83,13 +77,10 @@ config CRC64 tristate help The CRC64 library functions. Select this if your module uses any of the functions from . =20 -config ARCH_HAS_CRC64 - bool - config CRC64_ARCH bool depends on CRC64 && CRC_OPTIMIZATIONS default y if RISCV && RISCV_ISA_ZBC && 64BIT default y if X86_64 --=20 2.49.0