[PATCH 00/11] xz: Updates to license, filters, and compression options

Lasse Collin posted 11 patches 1 year, 10 months ago
There is a newer version of this series
Documentation/staging/xz.rst    | 130 ++++++++---------------
LICENSES/deprecated/0BSD        |  23 ++++
MAINTAINERS                     |  14 +++
include/linux/decompress/unxz.h |   5 +-
include/linux/xz.h              |   5 +-
init/Kconfig                    |   5 +-
lib/decompress_unxz.c           |  39 ++++---
lib/xz/Kconfig                  |  13 ++-
lib/xz/xz_crc32.c               |   7 +-
lib/xz/xz_dec_bcj.c             | 183 ++++++++++++++++++++++++++++++--
lib/xz/xz_dec_lzma2.c           |   5 +-
lib/xz/xz_dec_stream.c          |   5 +-
lib/xz/xz_dec_syms.c            |  16 +--
lib/xz/xz_dec_test.c            |  12 +--
lib/xz/xz_lzma2.h               |   5 +-
lib/xz/xz_private.h             |  20 ++--
lib/xz/xz_stream.h              |   7 +-
scripts/Makefile.lib            |  13 ++-
scripts/xz_wrap.sh              | 157 +++++++++++++++++++++++++--
19 files changed, 487 insertions(+), 177 deletions(-)
create mode 100644 LICENSES/deprecated/0BSD
[PATCH 00/11] xz: Updates to license, filters, and compression options
Posted by Lasse Collin 1 year, 10 months ago
XZ Embedded, the upstream project, switched from public domain to the
BSD Zero Clause License (0BSD). Now matching SPDX license identifiers
can be added.

The new ARM64 and RISC-V filters can be used by Squashfs.

Account for the default threading change made in the xz command line
tool version 5.6.0. Tweak kernel compression options for archs that
support XZ compressed kernel.

Documentation was revised. There are minor cleanups too.

Lasse Collin (11):
  MAINTAINERS: Add XZ Embedded maintainers
  LICENSES: Add 0BSD license text
  xz: Switch from public domain to BSD Zero Clause License (0BSD)
  xz: Documentation/staging/xz.rst: Revise thoroughly
  xz: Fix comments and coding style
  xz: Cleanup CRC32 edits from 2018
  xz: Optimize for-loop conditions in the BCJ decoders
  xz: Add ARM64 BCJ filter
  xz: Add RISC-V BCJ filter
  xz: Use 128 MiB dictionary and force single-threaded mode
  xz: Adjust arch-specific options for better kernel compression

 Documentation/staging/xz.rst    | 130 ++++++++---------------
 LICENSES/deprecated/0BSD        |  23 ++++
 MAINTAINERS                     |  14 +++
 include/linux/decompress/unxz.h |   5 +-
 include/linux/xz.h              |   5 +-
 init/Kconfig                    |   5 +-
 lib/decompress_unxz.c           |  39 ++++---
 lib/xz/Kconfig                  |  13 ++-
 lib/xz/xz_crc32.c               |   7 +-
 lib/xz/xz_dec_bcj.c             | 183 ++++++++++++++++++++++++++++++--
 lib/xz/xz_dec_lzma2.c           |   5 +-
 lib/xz/xz_dec_stream.c          |   5 +-
 lib/xz/xz_dec_syms.c            |  16 +--
 lib/xz/xz_dec_test.c            |  12 +--
 lib/xz/xz_lzma2.h               |   5 +-
 lib/xz/xz_private.h             |  20 ++--
 lib/xz/xz_stream.h              |   7 +-
 scripts/Makefile.lib            |  13 ++-
 scripts/xz_wrap.sh              | 157 +++++++++++++++++++++++++--
 19 files changed, 487 insertions(+), 177 deletions(-)
 create mode 100644 LICENSES/deprecated/0BSD

-- 
2.44.0
Re: [PATCH 00/11] xz: Updates to license, filters, and compression options
Posted by Kees Cook 1 year, 10 months ago
On Wed, Mar 20, 2024 at 08:38:33PM +0200, Lasse Collin wrote:
> XZ Embedded, the upstream project, switched from public domain to the
> BSD Zero Clause License (0BSD). Now matching SPDX license identifiers
> can be added.
> 
> The new ARM64 and RISC-V filters can be used by Squashfs.
> 
> Account for the default threading change made in the xz command line
> tool version 5.6.0. Tweak kernel compression options for archs that
> support XZ compressed kernel.
> 
> Documentation was revised. There are minor cleanups too.
> 
> Lasse Collin (11):
>   MAINTAINERS: Add XZ Embedded maintainers
>   LICENSES: Add 0BSD license text
>   xz: Switch from public domain to BSD Zero Clause License (0BSD)
>   xz: Documentation/staging/xz.rst: Revise thoroughly
>   xz: Fix comments and coding style
>   xz: Cleanup CRC32 edits from 2018
>   xz: Optimize for-loop conditions in the BCJ decoders
>   xz: Add ARM64 BCJ filter
>   xz: Add RISC-V BCJ filter
>   xz: Use 128 MiB dictionary and force single-threaded mode
>   xz: Adjust arch-specific options for better kernel compression
> 
>  Documentation/staging/xz.rst    | 130 ++++++++---------------
>  LICENSES/deprecated/0BSD        |  23 ++++
>  MAINTAINERS                     |  14 +++
>  include/linux/decompress/unxz.h |   5 +-
>  include/linux/xz.h              |   5 +-
>  init/Kconfig                    |   5 +-
>  lib/decompress_unxz.c           |  39 ++++---
>  lib/xz/Kconfig                  |  13 ++-
>  lib/xz/xz_crc32.c               |   7 +-
>  lib/xz/xz_dec_bcj.c             | 183 ++++++++++++++++++++++++++++++--
>  lib/xz/xz_dec_lzma2.c           |   5 +-
>  lib/xz/xz_dec_stream.c          |   5 +-
>  lib/xz/xz_dec_syms.c            |  16 +--
>  lib/xz/xz_dec_test.c            |  12 +--
>  lib/xz/xz_lzma2.h               |   5 +-
>  lib/xz/xz_private.h             |  20 ++--
>  lib/xz/xz_stream.h              |   7 +-
>  scripts/Makefile.lib            |  13 ++-
>  scripts/xz_wrap.sh              | 157 +++++++++++++++++++++++++--
>  19 files changed, 487 insertions(+), 177 deletions(-)
>  create mode 100644 LICENSES/deprecated/0BSD

Andrew (and anyone else), please do not take this code right now.

Until the backdooring of upstream xz[1] is fully understood, we should not
accept any code from Jia Tan, Lasse Collin, or any other folks associated
with tukaani.org. It appears the domain, or at least credentials
associated with Jia Tan, have been used to create an obfuscated ssh
server backdoor via the xz upstream releases since at least 5.6.0.
Without extensive analysis, we should not take any associated code.
It may be worth doing some retrospective analysis of past contributions
as well...

Lasse, are you able to comment about what is going on here?

-Kees

[1] https://www.openwall.com/lists/oss-security/2024/03/29/4

-- 
Kees Cook
Re: [tech-board] [PATCH 00/11] xz: Updates to license, filters, and compression options
Posted by Jonathan Corbet 1 year, 10 months ago
"Kees Cook" <keescook@chromium.org> writes:

> On Wed, Mar 20, 2024 at 08:38:33PM +0200, Lasse Collin wrote:
>> XZ Embedded, the upstream project, switched from public domain to the
>> BSD Zero Clause License (0BSD). Now matching SPDX license identifiers
>> can be added.
>> 
>> The new ARM64 and RISC-V filters can be used by Squashfs.
>> 
>> Account for the default threading change made in the xz command line
>> tool version 5.6.0. Tweak kernel compression options for archs that
>> support XZ compressed kernel.
>> 
>> Documentation was revised. There are minor cleanups too.
>> 
>> Lasse Collin (11):
>>   MAINTAINERS: Add XZ Embedded maintainers
>>   LICENSES: Add 0BSD license text
>>   xz: Switch from public domain to BSD Zero Clause License (0BSD)
>>   xz: Documentation/staging/xz.rst: Revise thoroughly
>>   xz: Fix comments and coding style
>>   xz: Cleanup CRC32 edits from 2018
>>   xz: Optimize for-loop conditions in the BCJ decoders
>>   xz: Add ARM64 BCJ filter
>>   xz: Add RISC-V BCJ filter
>>   xz: Use 128 MiB dictionary and force single-threaded mode
>>   xz: Adjust arch-specific options for better kernel compression
>> 
>>  Documentation/staging/xz.rst    | 130 ++++++++---------------
>>  LICENSES/deprecated/0BSD        |  23 ++++
>>  MAINTAINERS                     |  14 +++
>>  include/linux/decompress/unxz.h |   5 +-
>>  include/linux/xz.h              |   5 +-
>>  init/Kconfig                    |   5 +-
>>  lib/decompress_unxz.c           |  39 ++++---
>>  lib/xz/Kconfig                  |  13 ++-
>>  lib/xz/xz_crc32.c               |   7 +-
>>  lib/xz/xz_dec_bcj.c             | 183 ++++++++++++++++++++++++++++++--
>>  lib/xz/xz_dec_lzma2.c           |   5 +-
>>  lib/xz/xz_dec_stream.c          |   5 +-
>>  lib/xz/xz_dec_syms.c            |  16 +--
>>  lib/xz/xz_dec_test.c            |  12 +--
>>  lib/xz/xz_lzma2.h               |   5 +-
>>  lib/xz/xz_private.h             |  20 ++--
>>  lib/xz/xz_stream.h              |   7 +-
>>  scripts/Makefile.lib            |  13 ++-
>>  scripts/xz_wrap.sh              | 157 +++++++++++++++++++++++++--
>>  19 files changed, 487 insertions(+), 177 deletions(-)
>>  create mode 100644 LICENSES/deprecated/0BSD
>
> Andrew (and anyone else), please do not take this code right now.
>
> Until the backdooring of upstream xz[1] is fully understood, we should not
> accept any code from Jia Tan, Lasse Collin, or any other folks associated
> with tukaani.org. It appears the domain, or at least credentials
> associated with Jia Tan, have been used to create an obfuscated ssh
> server backdoor via the xz upstream releases since at least 5.6.0.
> Without extensive analysis, we should not take any associated code.
> It may be worth doing some retrospective analysis of past contributions
> as well...
>
> Lasse, are you able to comment about what is going on here?

FWIW, it looks like this series has been in linux-next for a few days.
Maybe it needs to come out, for now at least?

jon
Re: [tech-board] [PATCH 00/11] xz: Updates to license, filters, and compression options
Posted by Andrew Morton 1 year, 10 months ago
On Fri, 29 Mar 2024 14:51:41 -0600 Jonathan Corbet <corbet@lwn.net> wrote:

> > Andrew (and anyone else), please do not take this code right now.
> >
> > Until the backdooring of upstream xz[1] is fully understood, we should not
> > accept any code from Jia Tan, Lasse Collin, or any other folks associated
> > with tukaani.org. It appears the domain, or at least credentials
> > associated with Jia Tan, have been used to create an obfuscated ssh
> > server backdoor via the xz upstream releases since at least 5.6.0.
> > Without extensive analysis, we should not take any associated code.
> > It may be worth doing some retrospective analysis of past contributions
> > as well...
> >
> > Lasse, are you able to comment about what is going on here?
> 
> FWIW, it looks like this series has been in linux-next for a few days.
> Maybe it needs to come out, for now at least?

Yes, I have removed that series.
Re: [tech-board] [PATCH 00/11] xz: Updates to license, filters, and compression options
Posted by Lasse Collin 1 year, 10 months ago
On 2024-03-29 Andrew Morton wrote:
> On Fri, 29 Mar 2024 14:51:41 -0600 Jonathan Corbet <corbet@lwn.net>
> wrote:
> 
> > > Andrew (and anyone else), please do not take this code right now.
> > >
> > > Until the backdooring of upstream xz[1] is fully understood, we
> > > should not accept any code from Jia Tan, Lasse Collin, or any
> > > other folks associated with tukaani.org. It appears the domain,
> > > or at least credentials associated with Jia Tan, have been used
> > > to create an obfuscated ssh server backdoor via the xz upstream
> > > releases since at least 5.6.0. Without extensive analysis, we
> > > should not take any associated code. It may be worth doing some
> > > retrospective analysis of past contributions as well...
> > >
> > > Lasse, are you able to comment about what is going on here?  
> > 
> > FWIW, it looks like this series has been in linux-next for a few
> > days. Maybe it needs to come out, for now at least?  
> 
> Yes, I have removed that series.

Thank you. None of these patches are urgent. I'm on a holiday and only
happened to look at my emails and it seems to be a major mess.

My proper investigation efforts likely start in the first days of
April. That is, I currently know only a few facts which alone are bad
enough.

Info will be updated here: https://tukaani.org/xz-backdoor/

-- 
Lasse Collin
Re: [tech-board] [PATCH 00/11] xz: Updates to license, filters, and compression options
Posted by Kees Cook 1 year, 10 months ago
On Sat, Mar 30, 2024 at 02:48:48PM +0200, Lasse Collin wrote:
> On 2024-03-29 Andrew Morton wrote:
> > On Fri, 29 Mar 2024 14:51:41 -0600 Jonathan Corbet <corbet@lwn.net>
> > wrote:
> > 
> > > > Andrew (and anyone else), please do not take this code right now.
> > > >
> > > > Until the backdooring of upstream xz[1] is fully understood, we
> > > > should not accept any code from Jia Tan, Lasse Collin, or any
> > > > other folks associated with tukaani.org. It appears the domain,
> > > > or at least credentials associated with Jia Tan, have been used
> > > > to create an obfuscated ssh server backdoor via the xz upstream
> > > > releases since at least 5.6.0. Without extensive analysis, we
> > > > should not take any associated code. It may be worth doing some
> > > > retrospective analysis of past contributions as well...
> > > >
> > > > Lasse, are you able to comment about what is going on here?  
> > > 
> > > FWIW, it looks like this series has been in linux-next for a few
> > > days. Maybe it needs to come out, for now at least?  
> > 
> > Yes, I have removed that series.
> 
> Thank you. None of these patches are urgent. I'm on a holiday and only
> happened to look at my emails and it seems to be a major mess.
> 
> My proper investigation efforts likely start in the first days of
> April. That is, I currently know only a few facts which alone are bad
> enough.
> 
> Info will be updated here: https://tukaani.org/xz-backdoor/

Thanks; I appreciate the update! And I'm sorry this has happened to you.
We'll all know more in the coming days. (I, too, am on vacation, wheee.)

-Kees

-- 
Kees Cook
Re: [PATCH 00/11] xz: Updates to license, filters, and compression options
Posted by Kees Cook 1 year, 10 months ago
On Fri, Mar 29, 2024 at 02:51:41PM -0600, Jonathan Corbet wrote:
> "Kees Cook" <keescook@chromium.org> writes:
> 
> > On Wed, Mar 20, 2024 at 08:38:33PM +0200, Lasse Collin wrote:
> >> XZ Embedded, the upstream project, switched from public domain to the
> >> BSD Zero Clause License (0BSD). Now matching SPDX license identifiers
> >> can be added.
> >> 
> >> The new ARM64 and RISC-V filters can be used by Squashfs.
> >> 
> >> Account for the default threading change made in the xz command line
> >> tool version 5.6.0. Tweak kernel compression options for archs that
> >> support XZ compressed kernel.
> >> 
> >> Documentation was revised. There are minor cleanups too.
> >> 
> >> Lasse Collin (11):
> >>   MAINTAINERS: Add XZ Embedded maintainers
> >>   LICENSES: Add 0BSD license text
> >>   xz: Switch from public domain to BSD Zero Clause License (0BSD)
> >>   xz: Documentation/staging/xz.rst: Revise thoroughly
> >>   xz: Fix comments and coding style
> >>   xz: Cleanup CRC32 edits from 2018
> >>   xz: Optimize for-loop conditions in the BCJ decoders
> >>   xz: Add ARM64 BCJ filter
> >>   xz: Add RISC-V BCJ filter
> >>   xz: Use 128 MiB dictionary and force single-threaded mode
> >>   xz: Adjust arch-specific options for better kernel compression
> >> 
> >>  Documentation/staging/xz.rst    | 130 ++++++++---------------
> >>  LICENSES/deprecated/0BSD        |  23 ++++
> >>  MAINTAINERS                     |  14 +++
> >>  include/linux/decompress/unxz.h |   5 +-
> >>  include/linux/xz.h              |   5 +-
> >>  init/Kconfig                    |   5 +-
> >>  lib/decompress_unxz.c           |  39 ++++---
> >>  lib/xz/Kconfig                  |  13 ++-
> >>  lib/xz/xz_crc32.c               |   7 +-
> >>  lib/xz/xz_dec_bcj.c             | 183 ++++++++++++++++++++++++++++++--
> >>  lib/xz/xz_dec_lzma2.c           |   5 +-
> >>  lib/xz/xz_dec_stream.c          |   5 +-
> >>  lib/xz/xz_dec_syms.c            |  16 +--
> >>  lib/xz/xz_dec_test.c            |  12 +--
> >>  lib/xz/xz_lzma2.h               |   5 +-
> >>  lib/xz/xz_private.h             |  20 ++--
> >>  lib/xz/xz_stream.h              |   7 +-
> >>  scripts/Makefile.lib            |  13 ++-
> >>  scripts/xz_wrap.sh              | 157 +++++++++++++++++++++++++--
> >>  19 files changed, 487 insertions(+), 177 deletions(-)
> >>  create mode 100644 LICENSES/deprecated/0BSD
> >
> > Andrew (and anyone else), please do not take this code right now.
> >
> > Until the backdooring of upstream xz[1] is fully understood, we should not
> > accept any code from Jia Tan, Lasse Collin, or any other folks associated
> > with tukaani.org. It appears the domain, or at least credentials
> > associated with Jia Tan, have been used to create an obfuscated ssh
> > server backdoor via the xz upstream releases since at least 5.6.0.
> > Without extensive analysis, we should not take any associated code.
> > It may be worth doing some retrospective analysis of past contributions
> > as well...
> >
> > Lasse, are you able to comment about what is going on here?
> 
> FWIW, it looks like this series has been in linux-next for a few days.
> Maybe it needs to come out, for now at least?

Yes, for sure. Andrew, just so it's explicitly clear: please remove this
series from -mm for now, until the situation is better understood.

Thanks!

-Kees

-- 
Kees Cook
Re: [PATCH 00/11] xz: Updates to license, filters, and compression options
Posted by Jonathan Bennett 1 year, 10 months ago
Just in case it hasn't been brought to the kernel's attention, the xz 
project has been compromised in its upstream repo, and versions 5.6.0 
and 5.6.1 attempt to load a backdoor into SSH. I suggest any patches 
associated with Lasse Collin, Jia Tan, or tukaani.org  be held until 
that matter is fully resolved. And all their previous work needs to be 
re-examined with this in mind.

--Jonathan Bennett

On 3/20/24 1:38 PM, Lasse Collin wrote:
> XZ Embedded, the upstream project, switched from public domain to the
> BSD Zero Clause License (0BSD). Now matching SPDX license identifiers
> can be added.
>
> The new ARM64 and RISC-V filters can be used by Squashfs.
>
> Account for the default threading change made in the xz command line
> tool version 5.6.0. Tweak kernel compression options for archs that
> support XZ compressed kernel.
>
> Documentation was revised. There are minor cleanups too.
>
> Lasse Collin (11):
>    MAINTAINERS: Add XZ Embedded maintainers
>    LICENSES: Add 0BSD license text
>    xz: Switch from public domain to BSD Zero Clause License (0BSD)
>    xz: Documentation/staging/xz.rst: Revise thoroughly
>    xz: Fix comments and coding style
>    xz: Cleanup CRC32 edits from 2018
>    xz: Optimize for-loop conditions in the BCJ decoders
>    xz: Add ARM64 BCJ filter
>    xz: Add RISC-V BCJ filter
>    xz: Use 128 MiB dictionary and force single-threaded mode
>    xz: Adjust arch-specific options for better kernel compression
>
>   Documentation/staging/xz.rst    | 130 ++++++++---------------
>   LICENSES/deprecated/0BSD        |  23 ++++
>   MAINTAINERS                     |  14 +++
>   include/linux/decompress/unxz.h |   5 +-
>   include/linux/xz.h              |   5 +-
>   init/Kconfig                    |   5 +-
>   lib/decompress_unxz.c           |  39 ++++---
>   lib/xz/Kconfig                  |  13 ++-
>   lib/xz/xz_crc32.c               |   7 +-
>   lib/xz/xz_dec_bcj.c             | 183 ++++++++++++++++++++++++++++++--
>   lib/xz/xz_dec_lzma2.c           |   5 +-
>   lib/xz/xz_dec_stream.c          |   5 +-
>   lib/xz/xz_dec_syms.c            |  16 +--
>   lib/xz/xz_dec_test.c            |  12 +--
>   lib/xz/xz_lzma2.h               |   5 +-
>   lib/xz/xz_private.h             |  20 ++--
>   lib/xz/xz_stream.h              |   7 +-
>   scripts/Makefile.lib            |  13 ++-
>   scripts/xz_wrap.sh              | 157 +++++++++++++++++++++++++--
>   19 files changed, 487 insertions(+), 177 deletions(-)
>   create mode 100644 LICENSES/deprecated/0BSD
>