[PATCH v2 00/17] SHA-512 library functions

Eric Biggers posted 17 patches 3 months, 3 weeks ago
There is a newer version of this series
arch/arm/configs/exynos_defconfig             |   1 -
arch/arm/configs/milbeaut_m10v_defconfig      |   1 -
arch/arm/configs/multi_v7_defconfig           |   1 -
arch/arm/configs/omap2plus_defconfig          |   1 -
arch/arm/configs/pxa_defconfig                |   1 -
arch/arm/crypto/Kconfig                       |  10 -
arch/arm/crypto/Makefile                      |  15 -
arch/arm/crypto/sha512-glue.c                 | 110 ---
arch/arm/crypto/sha512-neon-glue.c            |  75 --
arch/arm/crypto/sha512.h                      |   3 -
arch/arm64/configs/defconfig                  |   1 -
arch/arm64/crypto/Kconfig                     |  19 -
arch/arm64/crypto/Makefile                    |  14 -
arch/arm64/crypto/sha512-ce-glue.c            |  96 ---
arch/arm64/crypto/sha512-glue.c               |  83 ---
arch/mips/cavium-octeon/crypto/Makefile       |   1 -
.../mips/cavium-octeon/crypto/octeon-crypto.c |   3 +-
arch/mips/cavium-octeon/crypto/octeon-md5.c   |   3 +-
arch/mips/cavium-octeon/crypto/octeon-sha1.c  |   3 +-
.../mips/cavium-octeon/crypto/octeon-sha256.c |   3 +-
.../mips/cavium-octeon/crypto/octeon-sha512.c | 167 -----
arch/mips/configs/cavium_octeon_defconfig     |   1 -
arch/mips/crypto/Kconfig                      |  10 -
.../asm/octeon/crypto.h}                      |   0
arch/riscv/crypto/Kconfig                     |  11 -
arch/riscv/crypto/Makefile                    |   3 -
arch/riscv/crypto/sha512-riscv64-glue.c       | 124 ----
arch/s390/configs/debug_defconfig             |   1 -
arch/s390/configs/defconfig                   |   1 -
arch/s390/crypto/Kconfig                      |  10 -
arch/s390/crypto/Makefile                     |   1 -
arch/s390/crypto/sha512_s390.c                | 151 ----
arch/sparc/crypto/Kconfig                     |  10 -
arch/sparc/crypto/Makefile                    |   2 -
arch/sparc/crypto/sha512_glue.c               | 122 ----
arch/x86/crypto/Kconfig                       |  13 -
arch/x86/crypto/Makefile                      |   3 -
arch/x86/crypto/sha512_ssse3_glue.c           | 322 ---------
crypto/Kconfig                                |   4 +-
crypto/Makefile                               |   2 +-
crypto/sha512.c                               | 338 +++++++++
crypto/sha512_generic.c                       | 217 ------
crypto/testmgr.c                              |  16 +
drivers/crypto/starfive/jh7110-hash.c         |   8 +-
include/crypto/sha2.h                         | 350 +++++++++
include/crypto/sha512_base.h                  | 120 ----
lib/crypto/Kconfig                            |  20 +
lib/crypto/Makefile                           |  38 +
lib/crypto/arm/.gitignore                     |   2 +
.../crypto => lib/crypto/arm}/sha512-armv4.pl |   0
lib/crypto/arm/sha512.h                       |  38 +
lib/crypto/arm64/.gitignore                   |   2 +
.../crypto/arm64}/sha512-ce-core.S            |  10 +-
lib/crypto/arm64/sha512.h                     |  46 ++
lib/crypto/mips/sha512.h                      |  74 ++
.../riscv}/sha512-riscv64-zvknhb-zvkb.S       |   4 +-
lib/crypto/riscv/sha512.h                     |  41 ++
lib/crypto/s390/sha512.h                      |  28 +
lib/crypto/sha512.c                           | 400 +++++++++++
lib/crypto/sparc/sha512.h                     |  42 ++
.../crypto => lib/crypto/sparc}/sha512_asm.S  |   0
lib/crypto/tests/Kconfig                      |  24 +
lib/crypto/tests/Makefile                     |   6 +
lib/crypto/tests/hash-test-template.h         | 512 ++++++++++++++
lib/crypto/tests/sha224-testvecs.h            | 223 ++++++
lib/crypto/tests/sha224_kunit.c               |  50 ++
lib/crypto/tests/sha256-testvecs.h            | 223 ++++++
lib/crypto/tests/sha256_kunit.c               |  39 ++
lib/crypto/tests/sha384-testvecs.h            | 566 +++++++++++++++
lib/crypto/tests/sha384_kunit.c               |  48 ++
lib/crypto/tests/sha512-testvecs.h            | 662 ++++++++++++++++++
lib/crypto/tests/sha512_kunit.c               |  48 ++
.../crypto/x86}/sha512-avx-asm.S              |  11 +-
.../crypto/x86}/sha512-avx2-asm.S             |  11 +-
.../crypto/x86}/sha512-ssse3-asm.S            |  12 +-
lib/crypto/x86/sha512.h                       |  54 ++
scripts/crypto/gen-hash-testvecs.py           |  83 +++
77 files changed, 4012 insertions(+), 1756 deletions(-)
delete mode 100644 arch/arm/crypto/sha512-glue.c
delete mode 100644 arch/arm/crypto/sha512-neon-glue.c
delete mode 100644 arch/arm/crypto/sha512.h
delete mode 100644 arch/arm64/crypto/sha512-ce-glue.c
delete mode 100644 arch/arm64/crypto/sha512-glue.c
delete mode 100644 arch/mips/cavium-octeon/crypto/octeon-sha512.c
rename arch/mips/{cavium-octeon/crypto/octeon-crypto.h => include/asm/octeon/crypto.h} (100%)
delete mode 100644 arch/riscv/crypto/sha512-riscv64-glue.c
delete mode 100644 arch/s390/crypto/sha512_s390.c
delete mode 100644 arch/sparc/crypto/sha512_glue.c
delete mode 100644 arch/x86/crypto/sha512_ssse3_glue.c
create mode 100644 crypto/sha512.c
delete mode 100644 crypto/sha512_generic.c
delete mode 100644 include/crypto/sha512_base.h
create mode 100644 lib/crypto/arm/.gitignore
rename {arch/arm/crypto => lib/crypto/arm}/sha512-armv4.pl (100%)
create mode 100644 lib/crypto/arm/sha512.h
create mode 100644 lib/crypto/arm64/.gitignore
rename {arch/arm64/crypto => lib/crypto/arm64}/sha512-ce-core.S (97%)
create mode 100644 lib/crypto/arm64/sha512.h
create mode 100644 lib/crypto/mips/sha512.h
rename {arch/riscv/crypto => lib/crypto/riscv}/sha512-riscv64-zvknhb-zvkb.S (98%)
create mode 100644 lib/crypto/riscv/sha512.h
create mode 100644 lib/crypto/s390/sha512.h
create mode 100644 lib/crypto/sha512.c
create mode 100644 lib/crypto/sparc/sha512.h
rename {arch/sparc/crypto => lib/crypto/sparc}/sha512_asm.S (100%)
create mode 100644 lib/crypto/tests/Kconfig
create mode 100644 lib/crypto/tests/Makefile
create mode 100644 lib/crypto/tests/hash-test-template.h
create mode 100644 lib/crypto/tests/sha224-testvecs.h
create mode 100644 lib/crypto/tests/sha224_kunit.c
create mode 100644 lib/crypto/tests/sha256-testvecs.h
create mode 100644 lib/crypto/tests/sha256_kunit.c
create mode 100644 lib/crypto/tests/sha384-testvecs.h
create mode 100644 lib/crypto/tests/sha384_kunit.c
create mode 100644 lib/crypto/tests/sha512-testvecs.h
create mode 100644 lib/crypto/tests/sha512_kunit.c
rename {arch/x86/crypto => lib/crypto/x86}/sha512-avx-asm.S (97%)
rename {arch/x86/crypto => lib/crypto/x86}/sha512-avx2-asm.S (98%)
rename {arch/x86/crypto => lib/crypto/x86}/sha512-ssse3-asm.S (97%)
create mode 100644 lib/crypto/x86/sha512.h
create mode 100755 scripts/crypto/gen-hash-testvecs.py
[PATCH v2 00/17] SHA-512 library functions
Posted by Eric Biggers 3 months, 3 weeks ago
This series applies to v6.16-rc1 and is targeting the libcrypto-next
tree.  It is also available at:

    git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git sha512-lib-v2

This series adds support for SHA-384, SHA-512, HMAC-SHA384, and
HMAC-SHA512 to lib/crypto/.  The new functions take advantage of the
kernel's existing architecture-optimized implementations of the SHA-512
compression function.  The new functions are fully tested using KUnit.

To avoid duplicating all arch-optimized implementations of the SHA-512
compression function (~3000 lines of code total), they are moved into
lib/crypto/ rather than copied.  To make the "sha384", "sha512",
"hmac(sha384)", and "hmac(sha512)" crypto_shash algorithms in the
old-school crypto API continue to be properly optimized after that, they
are reimplemented on top of lib/crypto/, which is straightforward.

The following lists some of the design choices and conventions that I've
followed in more detail.  Where these differ from the code or APIs for
other algorithms (e.g., SHA-256 in some cases), I'd like to do it this
way going forward and plan to fix up the other algorithms accordingly:

- APIs are fully documented with kerneldoc comments.

- APIs cannot fail, and return void.

- APIs work in all contexts.  This doesn't mean that they *should* be
  called in all contexts, but rather they always just work as expected.

- Tests are KUnit tests, and they are fairly thorough (more thorough
  than crypto/testmgr.c) and also optionally include benchmarks.

- Architecture-optimized code is integrated the same way I'm doing it
  for lib/crc/: it's in subdirectories lib/crypto/$(SRCARCH), it's
  enabled by default, and it's inlined into the same module as the
  generic code.  This solves a number of problems; for more details, see
  https://lore.kernel.org/r/20250607200454.73587-1-ebiggers@kernel.org

- HMAC support is a first-class citizen.

- APIs handle zeroization, when applicable.

- Message contexts are *_ctx instead of *_state.  It's shorter, avoids
  ambiguity with the compression function state, and matches OpenSSL.

- Length arguments are size_t, are in bytes, are named len or *_len, and
  immediately follow the corresponding buffer.  "Object" being operated
  on is first argument; outputs otherwise follow inputs.

- The structures for different algorithms use different types, which
  prevents usage errors where functions are mixed up between algorithms.

- The compression function state is strongly typed, not a plain array.

Changed in v2:
- Added "crypto: sha512 - use same state format as legacy drivers"
- Fixed build on user-mode Linux
- Fixed W=1 build warning by adding <linux/export.h>
- Optimized __sha512_final() and __hmac_sha512_final() slightly

Eric Biggers (17):
  crypto: sha512 - rename conflicting symbols
  lib/crypto/sha512: add support for SHA-384 and SHA-512
  lib/crypto/sha512: add HMAC-SHA384 and HMAC-SHA512 support
  lib/crypto/sha512: add KUnit tests for SHA-384 and SHA-512
  lib/crypto/sha256: add KUnit tests for SHA-224 and SHA-256
  crypto: riscv/sha512 - stop depending on sha512_generic_block_fn
  crypto: sha512 - replace sha512_generic with wrapper around SHA-512
    library
  crypto: sha512 - use same state format as legacy drivers
  lib/crypto/sha512: migrate arm-optimized SHA-512 code to library
  lib/crypto/sha512: migrate arm64-optimized SHA-512 code to library
  mips: cavium-octeon: move octeon-crypto.h into asm directory
  lib/crypto/sha512: migrate mips-optimized SHA-512 code to library
  lib/crypto/sha512: migrate riscv-optimized SHA-512 code to library
  lib/crypto/sha512: migrate s390-optimized SHA-512 code to library
  lib/crypto/sha512: migrate sparc-optimized SHA-512 code to library
  lib/crypto/sha512: migrate x86-optimized SHA-512 code to library
  crypto: sha512 - remove sha512_base.h

 arch/arm/configs/exynos_defconfig             |   1 -
 arch/arm/configs/milbeaut_m10v_defconfig      |   1 -
 arch/arm/configs/multi_v7_defconfig           |   1 -
 arch/arm/configs/omap2plus_defconfig          |   1 -
 arch/arm/configs/pxa_defconfig                |   1 -
 arch/arm/crypto/Kconfig                       |  10 -
 arch/arm/crypto/Makefile                      |  15 -
 arch/arm/crypto/sha512-glue.c                 | 110 ---
 arch/arm/crypto/sha512-neon-glue.c            |  75 --
 arch/arm/crypto/sha512.h                      |   3 -
 arch/arm64/configs/defconfig                  |   1 -
 arch/arm64/crypto/Kconfig                     |  19 -
 arch/arm64/crypto/Makefile                    |  14 -
 arch/arm64/crypto/sha512-ce-glue.c            |  96 ---
 arch/arm64/crypto/sha512-glue.c               |  83 ---
 arch/mips/cavium-octeon/crypto/Makefile       |   1 -
 .../mips/cavium-octeon/crypto/octeon-crypto.c |   3 +-
 arch/mips/cavium-octeon/crypto/octeon-md5.c   |   3 +-
 arch/mips/cavium-octeon/crypto/octeon-sha1.c  |   3 +-
 .../mips/cavium-octeon/crypto/octeon-sha256.c |   3 +-
 .../mips/cavium-octeon/crypto/octeon-sha512.c | 167 -----
 arch/mips/configs/cavium_octeon_defconfig     |   1 -
 arch/mips/crypto/Kconfig                      |  10 -
 .../asm/octeon/crypto.h}                      |   0
 arch/riscv/crypto/Kconfig                     |  11 -
 arch/riscv/crypto/Makefile                    |   3 -
 arch/riscv/crypto/sha512-riscv64-glue.c       | 124 ----
 arch/s390/configs/debug_defconfig             |   1 -
 arch/s390/configs/defconfig                   |   1 -
 arch/s390/crypto/Kconfig                      |  10 -
 arch/s390/crypto/Makefile                     |   1 -
 arch/s390/crypto/sha512_s390.c                | 151 ----
 arch/sparc/crypto/Kconfig                     |  10 -
 arch/sparc/crypto/Makefile                    |   2 -
 arch/sparc/crypto/sha512_glue.c               | 122 ----
 arch/x86/crypto/Kconfig                       |  13 -
 arch/x86/crypto/Makefile                      |   3 -
 arch/x86/crypto/sha512_ssse3_glue.c           | 322 ---------
 crypto/Kconfig                                |   4 +-
 crypto/Makefile                               |   2 +-
 crypto/sha512.c                               | 338 +++++++++
 crypto/sha512_generic.c                       | 217 ------
 crypto/testmgr.c                              |  16 +
 drivers/crypto/starfive/jh7110-hash.c         |   8 +-
 include/crypto/sha2.h                         | 350 +++++++++
 include/crypto/sha512_base.h                  | 120 ----
 lib/crypto/Kconfig                            |  20 +
 lib/crypto/Makefile                           |  38 +
 lib/crypto/arm/.gitignore                     |   2 +
 .../crypto => lib/crypto/arm}/sha512-armv4.pl |   0
 lib/crypto/arm/sha512.h                       |  38 +
 lib/crypto/arm64/.gitignore                   |   2 +
 .../crypto/arm64}/sha512-ce-core.S            |  10 +-
 lib/crypto/arm64/sha512.h                     |  46 ++
 lib/crypto/mips/sha512.h                      |  74 ++
 .../riscv}/sha512-riscv64-zvknhb-zvkb.S       |   4 +-
 lib/crypto/riscv/sha512.h                     |  41 ++
 lib/crypto/s390/sha512.h                      |  28 +
 lib/crypto/sha512.c                           | 400 +++++++++++
 lib/crypto/sparc/sha512.h                     |  42 ++
 .../crypto => lib/crypto/sparc}/sha512_asm.S  |   0
 lib/crypto/tests/Kconfig                      |  24 +
 lib/crypto/tests/Makefile                     |   6 +
 lib/crypto/tests/hash-test-template.h         | 512 ++++++++++++++
 lib/crypto/tests/sha224-testvecs.h            | 223 ++++++
 lib/crypto/tests/sha224_kunit.c               |  50 ++
 lib/crypto/tests/sha256-testvecs.h            | 223 ++++++
 lib/crypto/tests/sha256_kunit.c               |  39 ++
 lib/crypto/tests/sha384-testvecs.h            | 566 +++++++++++++++
 lib/crypto/tests/sha384_kunit.c               |  48 ++
 lib/crypto/tests/sha512-testvecs.h            | 662 ++++++++++++++++++
 lib/crypto/tests/sha512_kunit.c               |  48 ++
 .../crypto/x86}/sha512-avx-asm.S              |  11 +-
 .../crypto/x86}/sha512-avx2-asm.S             |  11 +-
 .../crypto/x86}/sha512-ssse3-asm.S            |  12 +-
 lib/crypto/x86/sha512.h                       |  54 ++
 scripts/crypto/gen-hash-testvecs.py           |  83 +++
 77 files changed, 4012 insertions(+), 1756 deletions(-)
 delete mode 100644 arch/arm/crypto/sha512-glue.c
 delete mode 100644 arch/arm/crypto/sha512-neon-glue.c
 delete mode 100644 arch/arm/crypto/sha512.h
 delete mode 100644 arch/arm64/crypto/sha512-ce-glue.c
 delete mode 100644 arch/arm64/crypto/sha512-glue.c
 delete mode 100644 arch/mips/cavium-octeon/crypto/octeon-sha512.c
 rename arch/mips/{cavium-octeon/crypto/octeon-crypto.h => include/asm/octeon/crypto.h} (100%)
 delete mode 100644 arch/riscv/crypto/sha512-riscv64-glue.c
 delete mode 100644 arch/s390/crypto/sha512_s390.c
 delete mode 100644 arch/sparc/crypto/sha512_glue.c
 delete mode 100644 arch/x86/crypto/sha512_ssse3_glue.c
 create mode 100644 crypto/sha512.c
 delete mode 100644 crypto/sha512_generic.c
 delete mode 100644 include/crypto/sha512_base.h
 create mode 100644 lib/crypto/arm/.gitignore
 rename {arch/arm/crypto => lib/crypto/arm}/sha512-armv4.pl (100%)
 create mode 100644 lib/crypto/arm/sha512.h
 create mode 100644 lib/crypto/arm64/.gitignore
 rename {arch/arm64/crypto => lib/crypto/arm64}/sha512-ce-core.S (97%)
 create mode 100644 lib/crypto/arm64/sha512.h
 create mode 100644 lib/crypto/mips/sha512.h
 rename {arch/riscv/crypto => lib/crypto/riscv}/sha512-riscv64-zvknhb-zvkb.S (98%)
 create mode 100644 lib/crypto/riscv/sha512.h
 create mode 100644 lib/crypto/s390/sha512.h
 create mode 100644 lib/crypto/sha512.c
 create mode 100644 lib/crypto/sparc/sha512.h
 rename {arch/sparc/crypto => lib/crypto/sparc}/sha512_asm.S (100%)
 create mode 100644 lib/crypto/tests/Kconfig
 create mode 100644 lib/crypto/tests/Makefile
 create mode 100644 lib/crypto/tests/hash-test-template.h
 create mode 100644 lib/crypto/tests/sha224-testvecs.h
 create mode 100644 lib/crypto/tests/sha224_kunit.c
 create mode 100644 lib/crypto/tests/sha256-testvecs.h
 create mode 100644 lib/crypto/tests/sha256_kunit.c
 create mode 100644 lib/crypto/tests/sha384-testvecs.h
 create mode 100644 lib/crypto/tests/sha384_kunit.c
 create mode 100644 lib/crypto/tests/sha512-testvecs.h
 create mode 100644 lib/crypto/tests/sha512_kunit.c
 rename {arch/x86/crypto => lib/crypto/x86}/sha512-avx-asm.S (97%)
 rename {arch/x86/crypto => lib/crypto/x86}/sha512-avx2-asm.S (98%)
 rename {arch/x86/crypto => lib/crypto/x86}/sha512-ssse3-asm.S (97%)
 create mode 100644 lib/crypto/x86/sha512.h
 create mode 100755 scripts/crypto/gen-hash-testvecs.py


base-commit: 19272b37aa4f83ca52bdf9c16d5d81bdd1354494
-- 
2.49.0
Re: [PATCH v2 00/17] SHA-512 library functions
Posted by Ard Biesheuvel 3 months, 2 weeks ago
On Mon, 16 Jun 2025 at 03:41, Eric Biggers <ebiggers@kernel.org> wrote:
>
> This series applies to v6.16-rc1 and is targeting the libcrypto-next
> tree.  It is also available at:
>
>     git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git sha512-lib-v2
>
> This series adds support for SHA-384, SHA-512, HMAC-SHA384, and
> HMAC-SHA512 to lib/crypto/.  The new functions take advantage of the
> kernel's existing architecture-optimized implementations of the SHA-512
> compression function.  The new functions are fully tested using KUnit.
>
> To avoid duplicating all arch-optimized implementations of the SHA-512
> compression function (~3000 lines of code total), they are moved into
> lib/crypto/ rather than copied.  To make the "sha384", "sha512",
> "hmac(sha384)", and "hmac(sha512)" crypto_shash algorithms in the
> old-school crypto API continue to be properly optimized after that, they
> are reimplemented on top of lib/crypto/, which is straightforward.
>
> The following lists some of the design choices and conventions that I've
> followed in more detail.  Where these differ from the code or APIs for
> other algorithms (e.g., SHA-256 in some cases), I'd like to do it this
> way going forward and plan to fix up the other algorithms accordingly:
>
> - APIs are fully documented with kerneldoc comments.
>
> - APIs cannot fail, and return void.
>
> - APIs work in all contexts.  This doesn't mean that they *should* be
>   called in all contexts, but rather they always just work as expected.
>
> - Tests are KUnit tests, and they are fairly thorough (more thorough
>   than crypto/testmgr.c) and also optionally include benchmarks.
>
> - Architecture-optimized code is integrated the same way I'm doing it
>   for lib/crc/: it's in subdirectories lib/crypto/$(SRCARCH), it's
>   enabled by default, and it's inlined into the same module as the
>   generic code.  This solves a number of problems; for more details, see
>   https://lore.kernel.org/r/20250607200454.73587-1-ebiggers@kernel.org
>
> - HMAC support is a first-class citizen.
>
> - APIs handle zeroization, when applicable.
>
> - Message contexts are *_ctx instead of *_state.  It's shorter, avoids
>   ambiguity with the compression function state, and matches OpenSSL.
>
> - Length arguments are size_t, are in bytes, are named len or *_len, and
>   immediately follow the corresponding buffer.  "Object" being operated
>   on is first argument; outputs otherwise follow inputs.
>
> - The structures for different algorithms use different types, which
>   prevents usage errors where functions are mixed up between algorithms.
>
> - The compression function state is strongly typed, not a plain array.
>
> Changed in v2:
> - Added "crypto: sha512 - use same state format as legacy drivers"
> - Fixed build on user-mode Linux
> - Fixed W=1 build warning by adding <linux/export.h>
> - Optimized __sha512_final() and __hmac_sha512_final() slightly
>
> Eric Biggers (17):
>   crypto: sha512 - rename conflicting symbols
>   lib/crypto/sha512: add support for SHA-384 and SHA-512
>   lib/crypto/sha512: add HMAC-SHA384 and HMAC-SHA512 support
>   lib/crypto/sha512: add KUnit tests for SHA-384 and SHA-512
>   lib/crypto/sha256: add KUnit tests for SHA-224 and SHA-256
>   crypto: riscv/sha512 - stop depending on sha512_generic_block_fn
>   crypto: sha512 - replace sha512_generic with wrapper around SHA-512
>     library
>   crypto: sha512 - use same state format as legacy drivers
>   lib/crypto/sha512: migrate arm-optimized SHA-512 code to library
>   lib/crypto/sha512: migrate arm64-optimized SHA-512 code to library
>   mips: cavium-octeon: move octeon-crypto.h into asm directory
>   lib/crypto/sha512: migrate mips-optimized SHA-512 code to library
>   lib/crypto/sha512: migrate riscv-optimized SHA-512 code to library
>   lib/crypto/sha512: migrate s390-optimized SHA-512 code to library
>   lib/crypto/sha512: migrate sparc-optimized SHA-512 code to library
>   lib/crypto/sha512: migrate x86-optimized SHA-512 code to library
>   crypto: sha512 - remove sha512_base.h
>

For the series,

Acked-by: Ard Biesheuvel <ardb@kernel.org>
Re: [PATCH v2 00/17] SHA-512 library functions
Posted by Eric Biggers 3 months, 2 weeks ago
On Sun, Jun 15, 2025 at 06:40:02PM -0700, Eric Biggers wrote:
> This series applies to v6.16-rc1 and is targeting the libcrypto-next
> tree.  It is also available at:
> 
>     git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git sha512-lib-v2
> 
> This series adds support for SHA-384, SHA-512, HMAC-SHA384, and
> HMAC-SHA512 to lib/crypto/.  The new functions take advantage of the
> kernel's existing architecture-optimized implementations of the SHA-512
> compression function.  The new functions are fully tested using KUnit.

FYI, applied to
https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git/log/?h=libcrypto-next

Additional reviews and acks would be greatly appreciated, though!

As per Linus's request, I reordered the KUnit tests (patches 4-5) to be last.
I'll try to keep them last as additional non-test patches get applied.

I also adjusted commit titles slightly to match what I'm planning to use for
lib/crypto and lib/crc going forwards:
    
        lib/(crypto|crc): ((algorithm|arch|arch/algorithm):)? Subject

    e.g.

        lib/crypto: sha512: Add KUnit tests for SHA-384 and SHA-512
        lib/crypto: x86: Move arch/x86/lib/crypto/ into lib/crypto/
        lib/crypto: x86/sha512: Migrate optimized SHA-512 code to library
        lib/crypto: Explicitly include <linux/export.h>

- Eric
Re: [PATCH v2 00/17] SHA-512 library functions
Posted by Eric Biggers 3 months, 3 weeks ago
On Sun, Jun 15, 2025 at 06:40:02PM -0700, Eric Biggers wrote:
> - Tests are KUnit tests, and they are fairly thorough (more thorough
>   than crypto/testmgr.c) and also optionally include benchmarks.

An additional note on testing: I have scripts that build the kernel for all the
arches that have arch-specific code in lib/crc/ or lib/crypto/, launch them in
QEMU with various -cpu options, and gather the results of the tests and any
other issues like warns or panics.

I'll get it into a sharable form at some point.

As far as the coverage of the arch-specific code in this specific patchset goes,
I've verified that my testing strategy covers all sha512_blocks() code paths,
including fallbacks, on arm, arm64, s390, riscv, and x86.

The two incomplete ones are mips and sparc, where I cannot test their optimized
code paths in sha512_blocks() because QEMU does not support it.

Still, I don't expect any issues.  That code is ultimately doing the same thing
as it was before for SHA-512 block processing, just integrated in a simpler way.

FWIW, my policy going forward is that any new arch-specific code in lib/crc/ or
lib/crypto/ *MUST* come with QEMU support so that it can be tested.  It's only
migration of existing code (usually from arch/*/crypto/) like this where I may
tolerate not being able to test it; that code gets "grandfathered in"...

- Eric
Re: [PATCH v2 00/17] SHA-512 library functions
Posted by Linus Torvalds 3 months, 3 weeks ago
On Mon, 16 Jun 2025 at 23:05, Eric Biggers <ebiggers@kernel.org> wrote:
>
> An additional note on testing: [..]

Talking about testing - when adding test scripts, can you do it as a
separate thing entirely - either before or after?

As it is, this code movement is hard to judge just because the stats
are all messed up with new tests:

 77 files changed, 4012 insertions(+), 1756 deletions(-)

that's 2k+ new lines of code that pretty much entirely hides the
question of "did this code movement result in a simpler / same / more
complex end result".

So in general, I'd really prefer big re-organizations to be *separate*
from new code changes.

It's just a pain to review renaming when it's mixed up with other
changes - whether renaming variables or whole files.

And that's as true on an individual commit level (we try to avoid
renaming things *and* making other changes in one go) as it is on a
pull request level.

If I see a pull request that only adds new tests, it's a no-brainer.

If I see a pull request that only re-organizes the code and the
diffstat just just renames with some small updates for new locations,
it's a no-brainer.

If I see a pull request that does both, it's a pain in the arse,
because then I need to start to look into individual commits and go
"which does what".

            Linus
Re: [PATCH v2 00/17] SHA-512 library functions
Posted by Eric Biggers 3 months, 3 weeks ago
On Tue, Jun 17, 2025 at 11:29:15AM -0700, Linus Torvalds wrote:
> On Mon, 16 Jun 2025 at 23:05, Eric Biggers <ebiggers@kernel.org> wrote:
> >
> > An additional note on testing: [..]
> 
> Talking about testing - when adding test scripts, can you do it as a
> separate thing entirely - either before or after?
> 
> As it is, this code movement is hard to judge just because the stats
> are all messed up with new tests:
> 
>  77 files changed, 4012 insertions(+), 1756 deletions(-)
> 
> that's 2k+ new lines of code that pretty much entirely hides the
> question of "did this code movement result in a simpler / same / more
> complex end result".
> 
> So in general, I'd really prefer big re-organizations to be *separate*
> from new code changes.
> 
> It's just a pain to review renaming when it's mixed up with other
> changes - whether renaming variables or whole files.
> 
> And that's as true on an individual commit level (we try to avoid
> renaming things *and* making other changes in one go) as it is on a
> pull request level.
> 
> If I see a pull request that only adds new tests, it's a no-brainer.
> 
> If I see a pull request that only re-organizes the code and the
> diffstat just just renames with some small updates for new locations,
> it's a no-brainer.
> 
> If I see a pull request that does both, it's a pain in the arse,
> because then I need to start to look into individual commits and go
> "which does what".

The tests are already in their own patches: patches 4 and 5.  Yes, this patchset
has a negative diffstat once you subtract them.

(And most of the positive diffstat in the tests is generated test vectors.  The
rest is mostly code that I'll reuse later in tests for other hash functions; it
just gets blamed to this one because it's the first one.)

I'd really prefer to keep testing as a first-class citizen.  Tests should be
contributed along with the code itself.

And the point of this patchset is not just "code movement", but rather adding a
library API, including documentation *and tests*.

Later, I'll be converting various in-kernel users to use that API, just as I've
been doing for crc32 and sha256.  It's already been shown that the
"librarification" works well and is much simpler than the old-school Crypto API.

If you really want patches 4-5 in a separate patchset that's based on this one,
I can do that.  Ultimately, the result would be the same.

I think your request for there to be a separate "tests" pull request is a
non-starter, though.  That would imply separate git trees, and we then wouldn't
be able to land tests at the same time as the code itself.

- Eric
Re: [PATCH v2 00/17] SHA-512 library functions
Posted by Linus Torvalds 3 months, 3 weeks ago
On Tue, 17 Jun 2025 at 12:22, Eric Biggers <ebiggers@kernel.org> wrote:
>>
> The tests are already in their own patches: patches 4 and 5.  Yes, this patchset
> has a negative diffstat once you subtract them.

Yes, the patches were separate, but my point stands.

Let me repeat that part of the email since you seem to have missed it:

> If I see a pull request that only adds new tests, it's a no-brainer.
>
> If I see a pull request that only re-organizes the code and the
> diffstat just just renames with some small updates for new locations,
> it's a no-brainer.
>
> If I see a pull request that does both, it's a pain in the arse,
> because then I need to start to look into individual commits and go
> "which does what".

IOW, I really prefer pull requests to do clearly separate things too
when we're talking re-organization. Or at the very least spell things
out *very* clearly.

Otherwise I have to waste time just to go split things out _anyway_.

            Linus
Re: [PATCH v2 00/17] SHA-512 library functions
Posted by Eric Biggers 3 months, 3 weeks ago
On Tue, Jun 17, 2025 at 12:43:54PM -0700, Linus Torvalds wrote:
> On Tue, 17 Jun 2025 at 12:22, Eric Biggers <ebiggers@kernel.org> wrote:
> >>
> > The tests are already in their own patches: patches 4 and 5.  Yes, this patchset
> > has a negative diffstat once you subtract them.
> 
> Yes, the patches were separate, but my point stands.
> 
> Let me repeat that part of the email since you seem to have missed it:
> 
> > If I see a pull request that only adds new tests, it's a no-brainer.
> >
> > If I see a pull request that only re-organizes the code and the
> > diffstat just just renames with some small updates for new locations,
> > it's a no-brainer.
> >
> > If I see a pull request that does both, it's a pain in the arse,
> > because then I need to start to look into individual commits and go
> > "which does what".
> 
> IOW, I really prefer pull requests to do clearly separate things too
> when we're talking re-organization. Or at the very least spell things
> out *very* clearly.
> 
> Otherwise I have to waste time just to go split things out _anyway_.

Again, the tests depend on the code they test being added first.

I could do two pull requests, the first with all non-test code and the second
with all test code, where the second depends on the first, i.e. it will have the
last commit of the first as its base commit.  Is that what you want?  Or are you
misunderstanding and thinking the tests are independent and apply to current
mainline?

- Eric
Re: [PATCH v2 00/17] SHA-512 library functions
Posted by Linus Torvalds 3 months, 3 weeks ago
On Tue, 17 Jun 2025 at 12:59, Eric Biggers <ebiggers@kernel.org> wrote:
>
> Again, the tests depend on the code they test being added first.

Sure, and that's fine. We have lots of "this depends on that".

> I could do two pull requests, the first with all non-test code and the second
> with all test code, where the second depends on the first, i.e. it will have the
> last commit of the first as its base commit.  Is that what you want?

Yes.

Or if one single pull request, split out the diffstat with the
explanation (that's the "Or at the very least spell things out *very*
clearly" option). But two separate pull requests would actually be my
preference.

          Linus
Re: [PATCH v2 00/17] SHA-512 library functions
Posted by Eric Biggers 3 months, 3 weeks ago
On Tue, Jun 17, 2025 at 01:08:14PM -0700, Linus Torvalds wrote:
> On Tue, 17 Jun 2025 at 12:59, Eric Biggers <ebiggers@kernel.org> wrote:
> >
> > Again, the tests depend on the code they test being added first.
> 
> Sure, and that's fine. We have lots of "this depends on that".
> 
> > I could do two pull requests, the first with all non-test code and the second
> > with all test code, where the second depends on the first, i.e. it will have the
> > last commit of the first as its base commit.  Is that what you want?
> 
> Yes.
> 
> Or if one single pull request, split out the diffstat with the
> explanation (that's the "Or at the very least spell things out *very*
> clearly" option). But two separate pull requests would actually be my
> preference.
> 
>           Linus

Okay.  For now I'll keep the test commits last and plan for a separate pull
request with them, based on the first.  I fear I'll quickly run into
interdependencies, in which case I'll need to fall back to "one pull request and
spell things out very clearly".  But I'll try it.

Just so it's clear, this is the diffstat of this patchset broken down by
non-test code (patches 1-3 and 6-17) and tests (4-5):

    Non-test:
         arch/arm/configs/exynos_defconfig                  |   1 -
         arch/arm/configs/milbeaut_m10v_defconfig           |   1 -
         arch/arm/configs/multi_v7_defconfig                |   1 -
         arch/arm/configs/omap2plus_defconfig               |   1 -
         arch/arm/configs/pxa_defconfig                     |   1 -
         arch/arm/crypto/Kconfig                            |  10 -
         arch/arm/crypto/Makefile                           |  15 -
         arch/arm/crypto/sha512-glue.c                      | 110 ------
         arch/arm/crypto/sha512-neon-glue.c                 |  75 ----
         arch/arm/crypto/sha512.h                           |   3 -
         arch/arm64/configs/defconfig                       |   1 -
         arch/arm64/crypto/Kconfig                          |  19 -
         arch/arm64/crypto/Makefile                         |  14 -
         arch/arm64/crypto/sha512-ce-glue.c                 |  96 -----
         arch/arm64/crypto/sha512-glue.c                    |  83 -----
         arch/mips/cavium-octeon/crypto/Makefile            |   1 -
         arch/mips/cavium-octeon/crypto/octeon-crypto.c     |   3 +-
         arch/mips/cavium-octeon/crypto/octeon-md5.c        |   3 +-
         arch/mips/cavium-octeon/crypto/octeon-sha1.c       |   3 +-
         arch/mips/cavium-octeon/crypto/octeon-sha256.c     |   3 +-
         arch/mips/cavium-octeon/crypto/octeon-sha512.c     | 167 ---------
         arch/mips/configs/cavium_octeon_defconfig          |   1 -
         arch/mips/crypto/Kconfig                           |  10 -
         .../asm/octeon/crypto.h}                           |   0
         arch/riscv/crypto/Kconfig                          |  11 -
         arch/riscv/crypto/Makefile                         |   3 -
         arch/riscv/crypto/sha512-riscv64-glue.c            | 124 -------
         arch/s390/configs/debug_defconfig                  |   1 -
         arch/s390/configs/defconfig                        |   1 -
         arch/s390/crypto/Kconfig                           |  10 -
         arch/s390/crypto/Makefile                          |   1 -
         arch/s390/crypto/sha512_s390.c                     | 151 --------
         arch/sparc/crypto/Kconfig                          |  10 -
         arch/sparc/crypto/Makefile                         |   2 -
         arch/sparc/crypto/sha512_glue.c                    | 122 -------
         arch/x86/crypto/Kconfig                            |  13 -
         arch/x86/crypto/Makefile                           |   3 -
         arch/x86/crypto/sha512_ssse3_glue.c                | 322 -----------------
         crypto/Kconfig                                     |   4 +-
         crypto/Makefile                                    |   2 +-
         crypto/sha512.c                                    | 338 +++++++++++++++++
         crypto/sha512_generic.c                            | 217 -----------
         crypto/testmgr.c                                   |  16 +
         drivers/crypto/starfive/jh7110-hash.c              |   8 +-
         include/crypto/sha2.h                              | 350 ++++++++++++++++++
         include/crypto/sha512_base.h                       | 120 -------
         lib/crypto/Kconfig                                 |  18 +
         lib/crypto/Makefile                                |  36 ++
         lib/crypto/arm/.gitignore                          |   2 +
         .../arm/crypto => lib/crypto/arm}/sha512-armv4.pl  |   0
         lib/crypto/arm/sha512.h                            |  38 ++
         lib/crypto/arm64/.gitignore                        |   2 +
         .../crypto => lib/crypto/arm64}/sha512-ce-core.S   |  10 +-
         lib/crypto/arm64/sha512.h                          |  46 +++
         lib/crypto/mips/sha512.h                           |  74 ++++
         .../crypto/riscv}/sha512-riscv64-zvknhb-zvkb.S     |   4 +-
         lib/crypto/riscv/sha512.h                          |  41 +++
         lib/crypto/s390/sha512.h                           |  28 ++
         lib/crypto/sha512.c                                | 400 +++++++++++++++++++++
         lib/crypto/sparc/sha512.h                          |  42 +++
         .../sparc/crypto => lib/crypto/sparc}/sha512_asm.S |   0
         .../x86/crypto => lib/crypto/x86}/sha512-avx-asm.S |  11 +-
         .../crypto => lib/crypto/x86}/sha512-avx2-asm.S    |  11 +-
         .../crypto => lib/crypto/x86}/sha512-ssse3-asm.S   |  12 +-
         lib/crypto/x86/sha512.h                            |  54 +++
         65 files changed, 1524 insertions(+), 1756 deletions(-)

    Test:
         lib/crypto/Kconfig                    |   2 +
         lib/crypto/Makefile                   |   2 +
         lib/crypto/tests/Kconfig              |  24 ++
         lib/crypto/tests/Makefile             |   6 +
         lib/crypto/tests/hash-test-template.h | 512 ++++++++++++++++++++++++++
         lib/crypto/tests/sha224-testvecs.h    | 223 ++++++++++++
         lib/crypto/tests/sha224_kunit.c       |  50 +++
         lib/crypto/tests/sha256-testvecs.h    | 223 ++++++++++++
         lib/crypto/tests/sha256_kunit.c       |  39 ++
         lib/crypto/tests/sha384-testvecs.h    | 566 +++++++++++++++++++++++++++++
         lib/crypto/tests/sha384_kunit.c       |  48 +++
         lib/crypto/tests/sha512-testvecs.h    | 662 ++++++++++++++++++++++++++++++++++
         lib/crypto/tests/sha512_kunit.c       |  48 +++
         scripts/crypto/gen-hash-testvecs.py   |  83 +++++
         14 files changed, 2488 insertions(+)

Note that the non-test part includes kerneldoc comments.  I'll assume you aren't
going to insist on those being in a separate "documentation" pull request...

If I need to resend this patchset for another reason, I'll also split it into
two patchsets, one depending on the other.  But I'm not planning to resend it
purely to do that split and with no other changes, as that seems a bit silly.

- Eric
Re: [PATCH v2 00/17] SHA-512 library functions
Posted by Linus Torvalds 3 months, 3 weeks ago
On Tue, 17 Jun 2025 at 13:37, Eric Biggers <ebiggers@kernel.org> wrote:
>
> Okay.  For now I'll keep the test commits last and plan for a separate pull
> request with them, based on the first.  I fear I'll quickly run into
> interdependencies, in which case I'll need to fall back to "one pull request and
> spell things out very clearly".  But I'll try it.

Thanks.

Note that this "split it out" is really _only_ for when there's big
code movement and re-organization like this - it's certainly not a
general thing.

So you don't need to feel like I'm going to ask you to jump through
hoops in general for normal crypto library updates, this is really
only for these kinds of initial "big code movement" things.

> Just so it's clear, this is the diffstat of this patchset broken down by
> non-test code (patches 1-3 and 6-17) and tests (4-5):
>
>     Non-test:
>          65 files changed, 1524 insertions(+), 1756 deletions(-)
>
>     Test:
>          14 files changed, 2488 insertions(+)

Looks good. That's the kind of diffstat that makes me happy to pull:
the first one removes move code than it adds, and the second one very
clearly just adds tests.

So yes, this is the kind of thing that makes my life easy..

> Note that the non-test part includes kerneldoc comments.  I'll assume you aren't
> going to insist on those being in a separate "documentation" pull request...

Naah, they're relatively tiny, and don't skew the diffstat in huge ways.

             Linus