[PATCH v2 0/6] Optimize fsverity using 2-way interleaved SHA-256 hashing

Eric Biggers posted 6 patches 2 weeks, 2 days ago
fs/verity/enable.c              |  12 +-
fs/verity/fsverity_private.h    |   2 +-
fs/verity/hash_algs.c           |   3 +-
fs/verity/verify.c              | 175 ++++++++++++---
include/crypto/sha2.h           |  28 +++
lib/crypto/arm64/sha256-ce.S    | 284 +++++++++++++++++++++++-
lib/crypto/arm64/sha256.h       |  37 ++++
lib/crypto/sha256.c             |  71 +++++-
lib/crypto/tests/sha256_kunit.c | 184 ++++++++++++++++
lib/crypto/x86/sha256-ni-asm.S  | 368 ++++++++++++++++++++++++++++++++
lib/crypto/x86/sha256.h         |  39 ++++
11 files changed, 1147 insertions(+), 56 deletions(-)
[PATCH v2 0/6] Optimize fsverity using 2-way interleaved SHA-256 hashing
Posted by Eric Biggers 2 weeks, 2 days ago
This series is targeting libcrypto-next.  It can also be retrieved from:

    git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git sha256_finup_2x-v2

This series adds support for 2-way interleaved SHA-256 hashing to
lib/crypto/, implements it for arm64 and x86_64, and makes fsverity use
it.  This significantly improves fsverity performance on many CPUs.

Later patches will make dm-verity use this optimization as well.

Changed in v2:
- Made the new arm64 assembly compatible with CONFIG_CPU_BIG_ENDIAN=y.
- Omitted sha256_finup_2x() from pre-boot environments.
- Made alloc_guarded_buf() assert that the allocation succeeded.
- Minor tweaks to comments and whitespace.

Eric Biggers (6):
  lib/crypto: sha256: Add support for 2-way interleaved hashing
  lib/crypto: arm64/sha256: Add support for 2-way interleaved hashing
  lib/crypto: x86/sha256: Add support for 2-way interleaved hashing
  lib/crypto: tests: Add tests and benchmark for sha256_finup_2x()
  fsverity: Remove inode parameter from fsverity_hash_block()
  fsverity: Use 2-way interleaved SHA-256 hashing when supported

 fs/verity/enable.c              |  12 +-
 fs/verity/fsverity_private.h    |   2 +-
 fs/verity/hash_algs.c           |   3 +-
 fs/verity/verify.c              | 175 ++++++++++++---
 include/crypto/sha2.h           |  28 +++
 lib/crypto/arm64/sha256-ce.S    | 284 +++++++++++++++++++++++-
 lib/crypto/arm64/sha256.h       |  37 ++++
 lib/crypto/sha256.c             |  71 +++++-
 lib/crypto/tests/sha256_kunit.c | 184 ++++++++++++++++
 lib/crypto/x86/sha256-ni-asm.S  | 368 ++++++++++++++++++++++++++++++++
 lib/crypto/x86/sha256.h         |  39 ++++
 11 files changed, 1147 insertions(+), 56 deletions(-)

base-commit: 54e7bb6ade8acd3fb1c486c9f3e2c0dfdc18f84e
-- 
2.51.0
Re: [PATCH v2 0/6] Optimize fsverity using 2-way interleaved SHA-256 hashing
Posted by Eric Biggers 2 weeks ago
On Mon, Sep 15, 2025 at 11:08:13AM -0500, Eric Biggers wrote:
> This series is targeting libcrypto-next.  It can also be retrieved from:
> 
>     git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git sha256_finup_2x-v2
> 
> This series adds support for 2-way interleaved SHA-256 hashing to
> lib/crypto/, implements it for arm64 and x86_64, and makes fsverity use
> it.  This significantly improves fsverity performance on many CPUs.
> 
> Later patches will make dm-verity use this optimization as well.
> 
> Changed in v2:
> - Made the new arm64 assembly compatible with CONFIG_CPU_BIG_ENDIAN=y.
> - Omitted sha256_finup_2x() from pre-boot environments.
> - Made alloc_guarded_buf() assert that the allocation succeeded.
> - Minor tweaks to comments and whitespace.
> 
> Eric Biggers (6):
>   lib/crypto: sha256: Add support for 2-way interleaved hashing
>   lib/crypto: arm64/sha256: Add support for 2-way interleaved hashing
>   lib/crypto: x86/sha256: Add support for 2-way interleaved hashing
>   lib/crypto: tests: Add tests and benchmark for sha256_finup_2x()
>   fsverity: Remove inode parameter from fsverity_hash_block()
>   fsverity: Use 2-way interleaved SHA-256 hashing when supported
> 
>  fs/verity/enable.c              |  12 +-
>  fs/verity/fsverity_private.h    |   2 +-
>  fs/verity/hash_algs.c           |   3 +-
>  fs/verity/verify.c              | 175 ++++++++++++---
>  include/crypto/sha2.h           |  28 +++
>  lib/crypto/arm64/sha256-ce.S    | 284 +++++++++++++++++++++++-
>  lib/crypto/arm64/sha256.h       |  37 ++++
>  lib/crypto/sha256.c             |  71 +++++-
>  lib/crypto/tests/sha256_kunit.c | 184 ++++++++++++++++
>  lib/crypto/x86/sha256-ni-asm.S  | 368 ++++++++++++++++++++++++++++++++
>  lib/crypto/x86/sha256.h         |  39 ++++
>  11 files changed, 1147 insertions(+), 56 deletions(-)

FYI, applied to https://git.kernel.org/pub/scm/fs/fsverity/linux.git/log/?h=for-next

I decided to use the fsverity tree instead of the libcrypto one.  There
are no dependencies on other libcrypto changes for 6.18, and this makes
it easier to do a separate pull request.

Also, as always, reviews and acks would be appreciated!  Note that I
dropped the reviews and acks that were on the original crypto_shash
version from earlier this year, due to changes in the patches.  The
high-level idea is still the same, though.  If people could
re-review/ack this latest version, that would be great.  Thanks,

- Eric
Re: [PATCH v2 0/6] Optimize fsverity using 2-way interleaved SHA-256 hashing
Posted by Ard Biesheuvel 2 weeks ago
On Wed, 17 Sept 2025 at 17:35, Eric Biggers <ebiggers@kernel.org> wrote:
>
> On Mon, Sep 15, 2025 at 11:08:13AM -0500, Eric Biggers wrote:
> > This series is targeting libcrypto-next.  It can also be retrieved from:
> >
> >     git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git sha256_finup_2x-v2
> >
> > This series adds support for 2-way interleaved SHA-256 hashing to
> > lib/crypto/, implements it for arm64 and x86_64, and makes fsverity use
> > it.  This significantly improves fsverity performance on many CPUs.
> >
> > Later patches will make dm-verity use this optimization as well.
> >
> > Changed in v2:
> > - Made the new arm64 assembly compatible with CONFIG_CPU_BIG_ENDIAN=y.
> > - Omitted sha256_finup_2x() from pre-boot environments.
> > - Made alloc_guarded_buf() assert that the allocation succeeded.
> > - Minor tweaks to comments and whitespace.
> >
> > Eric Biggers (6):
> >   lib/crypto: sha256: Add support for 2-way interleaved hashing
> >   lib/crypto: arm64/sha256: Add support for 2-way interleaved hashing
> >   lib/crypto: x86/sha256: Add support for 2-way interleaved hashing
> >   lib/crypto: tests: Add tests and benchmark for sha256_finup_2x()
> >   fsverity: Remove inode parameter from fsverity_hash_block()
> >   fsverity: Use 2-way interleaved SHA-256 hashing when supported
> >
> >  fs/verity/enable.c              |  12 +-
> >  fs/verity/fsverity_private.h    |   2 +-
> >  fs/verity/hash_algs.c           |   3 +-
> >  fs/verity/verify.c              | 175 ++++++++++++---
> >  include/crypto/sha2.h           |  28 +++
> >  lib/crypto/arm64/sha256-ce.S    | 284 +++++++++++++++++++++++-
> >  lib/crypto/arm64/sha256.h       |  37 ++++
> >  lib/crypto/sha256.c             |  71 +++++-
> >  lib/crypto/tests/sha256_kunit.c | 184 ++++++++++++++++
> >  lib/crypto/x86/sha256-ni-asm.S  | 368 ++++++++++++++++++++++++++++++++
> >  lib/crypto/x86/sha256.h         |  39 ++++
> >  11 files changed, 1147 insertions(+), 56 deletions(-)
>
> FYI, applied to https://git.kernel.org/pub/scm/fs/fsverity/linux.git/log/?h=for-next
>
> I decided to use the fsverity tree instead of the libcrypto one.  There
> are no dependencies on other libcrypto changes for 6.18, and this makes
> it easier to do a separate pull request.
>
> Also, as always, reviews and acks would be appreciated!  Note that I
> dropped the reviews and acks that were on the original crypto_shash
> version from earlier this year, due to changes in the patches.  The
> high-level idea is still the same, though.  If people could
> re-review/ack this latest version, that would be great.  Thanks,
>

I looked at this the other day but forgot to reply.

Reviewed-by: Ard Biesheuvel <ardb@kernel.org>

Happy to see that this finally landed!