From nobody Wed May 8 13:32:38 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1611069366; cv=none; d=zohomail.com; s=zohoarc; b=Ki+CCabG3ry25fa2eTXOFgWzZRnHZv8XbB20X087sNGfX/AXEHj6UZ3DDzp8u7lWzhrUBMZDQCPuuNJj6DfKtEZ7AeODbwDYXxBSfNHZnI9kEpiy2BafR6oqmJ8xByOUM+mrPsFq5D/rtGY/QGcqWF5miuuOAB2dH0AZf3KaLIs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1611069366; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=aJuXIvTUUf3hQsUT0NGc7MLb9kDPJKb761kcw2wx+GY=; b=MamRW3/WX60Dsh/ryrZEhzdMCmVbdtIHl3XP4UQo/hQHfXjOjo7WpAZdBJmqubx/qX0bwyDTI7BPVYwOoTNqVd4g3UaQBUVAUrCkt6T/udRw1goVELUeowIhIb6P59FsOV1aLoFtKQUSz65hRP+ZooGIjmqxF2CoFIfzUWnNhLw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1611069366417797.5931920212853; Tue, 19 Jan 2021 07:16:06 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.70477.126500 (Exim 4.92) (envelope-from ) id 1l1sjE-00046Z-7i; Tue, 19 Jan 2021 15:15:32 +0000 Received: by outflank-mailman (output) from mailman id 70477.126500; Tue, 19 Jan 2021 15:15:32 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l1sjE-00046S-4A; Tue, 19 Jan 2021 15:15:32 +0000 Received: by outflank-mailman (input) for mailman id 70477; Tue, 19 Jan 2021 15:15:30 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l1sjC-00046L-GT for xen-devel@lists.xenproject.org; Tue, 19 Jan 2021 15:15:30 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 3ee8900e-3687-438b-a12f-fec38a0f20b1; Tue, 19 Jan 2021 15:15:27 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 335DCAE6E; Tue, 19 Jan 2021 15:15:26 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 3ee8900e-3687-438b-a12f-fec38a0f20b1 X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1611069326; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aJuXIvTUUf3hQsUT0NGc7MLb9kDPJKb761kcw2wx+GY=; b=rG0XwOD80W+WjSBebRi100qCoADUcDBuzAiMhbZwbXe7vhEfIJAFR3JeN+O0blNqAJJ6vE r8Idf8H4tTsxVWb+iUV8ZOtEqDqpJpD0MCGJe56gsctxlWCr6ED0D4vCtRAvu6fMAT+oJY w8VQeLsnJUcMdi111DHHA/RU7CtBfEQ= Subject: [PATCH v2 1/5] libxenguest: support zstd compressed kernels From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Stefano Stabellini , Wei Liu References: Message-ID: <403d985c-b4c0-d8d1-4f42-7e1fe6ae3ee0@suse.com> Date: Tue, 19 Jan 2021 16:15:25 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" This follows the logic used for other decompression methods utilizing an external library, albeit here we can't ignore the 32-bit size field appended to the compressed image - its presence causes decompression to fail. Leverage the field instead to allocate the output buffer in one go, i.e. without incrementally realloc()ing. Note that, where possible, instead of #ifdef-ing xen/*.h inclusions, they get removed. Signed-off-by: Jan Beulich Acked-by: Wei Liu --- Note for committer: As an alternative to patching tools/configure here, autoconf may want re-running. --- v2: New. --- a/tools/configure +++ b/tools/configure @@ -643,6 +643,8 @@ PTHREAD_CFLAGS EXTFS_LIBS system_aio zlib +libzstd_LIBS +libzstd_CFLAGS FETCHER FTP FALSE @@ -857,6 +859,8 @@ glib_CFLAGS glib_LIBS pixman_CFLAGS pixman_LIBS +libzstd_CFLAGS +libzstd_LIBS LIBNL3_CFLAGS LIBNL3_LIBS SYSTEMD_CFLAGS @@ -1605,6 +1609,10 @@ Some influential environment variables: pixman_CFLAGS C compiler flags for pixman, overriding pkg-config pixman_LIBS linker flags for pixman, overriding pkg-config + libzstd_CFLAGS + C compiler flags for libzstd, overriding pkg-config + libzstd_LIBS + linker flags for libzstd, overriding pkg-config LIBNL3_CFLAGS C compiler flags for LIBNL3, overriding pkg-config LIBNL3_LIBS linker flags for LIBNL3, overriding pkg-config @@ -8744,6 +8752,97 @@ fi =20 =20 =20 +pkg_failed=3Dno +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$libzstd_CFLAGS"; then + pkg_cv_libzstd_CFLAGS=3D"$libzstd_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --pri= nt-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=3D$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? =3D $ac_status" >&5 + test $ac_status =3D 0; }; then + pkg_cv_libzstd_CFLAGS=3D`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" !=3D "x0" && pkg_failed=3Dyes +else + pkg_failed=3Dyes +fi + else + pkg_failed=3Duntried +fi +if test -n "$libzstd_LIBS"; then + pkg_cv_libzstd_LIBS=3D"$libzstd_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --pri= nt-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=3D$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? =3D $ac_status" >&5 + test $ac_status =3D 0; }; then + pkg_cv_libzstd_LIBS=3D`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" !=3D "x0" && pkg_failed=3Dyes +else + pkg_failed=3Dyes +fi + else + pkg_failed=3Duntried +fi + + + +if test $pkg_failed =3D yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=3Dyes +else + _pkg_short_errors_supported=3Dno +fi + if test $_pkg_short_errors_supported =3D yes; then + libzstd_PKG_ERRORS=3D`$PKG_CONFIG --short-errors --print-errors -= -cflags --libs "libzstd" 2>&1` + else + libzstd_PKG_ERRORS=3D`$PKG_CONFIG --print-errors --cflags --libs = "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$libzstd_PKG_ERRORS" >&5 + + as_fn_error $? "Package requirements (libzstd) were not met: + +$libzstd_PKG_ERRORS + +Consider adjusting the PKG_CONFIG_PATH environment variable if you +installed software in a non-standard prefix. + +Alternatively, you may set the environment variables libzstd_CFLAGS +and libzstd_LIBS to avoid the need to call pkg-config. +See the pkg-config man page for more details." "$LINENO" 5 +elif test $pkg_failed =3D untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 +$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} +as_fn_error $? "The pkg-config script could not be found or is too old. M= ake sure it +is in your PATH or set the PKG_CONFIG environment variable to the full +path to pkg-config. + +Alternatively, you may set the environment variables libzstd_CFLAGS +and libzstd_LIBS to avoid the need to call pkg-config. +See the pkg-config man page for more details. + +To get pkg-config, see . +See \`config.log' for more details" "$LINENO" 5; } +else + libzstd_CFLAGS=3D$pkg_cv_libzstd_CFLAGS + libzstd_LIBS=3D$pkg_cv_libzstd_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + zlib=3D"$zlib -DHAVE_ZSTD $libzstd_CFLAGS $libzstd_LIBS" +fi + =20 =20 ac_fn_c_check_header_mongrel "$LINENO" "ext2fs/ext2fs.h" "ac_cv_header_ext= 2fs_ext2fs_h" "$ac_includes_default" --- a/tools/configure.ac +++ b/tools/configure.ac @@ -414,6 +414,7 @@ AC_CHECK_LIB([lzma], [lzma_stream_decode AC_CHECK_HEADER([lzo/lzo1x.h], [ AC_CHECK_LIB([lzo2], [lzo1x_decompress], [zlib=3D"$zlib -DHAVE_LZO1X -llzo= 2"]) ]) +PKG_CHECK_MODULES([libzstd], [libzstd], [zlib=3D"$zlib -DHAVE_ZSTD $libzst= d_CFLAGS $libzstd_LIBS"]) AC_SUBST(zlib) AC_SUBST(system_aio) AX_CHECK_EXTFS --- a/tools/libs/guest/Makefile +++ b/tools/libs/guest/Makefile @@ -64,6 +64,7 @@ SRCS-y +=3D xg_dom_decompr SRCS-y +=3D xg_dom_decompress_unsafe_lzma.c SRCS-y +=3D xg_dom_decompress_unsafe_lzo1x.c SRCS-y +=3D xg_dom_decompress_unsafe_xz.c +SRCS-y +=3D xg_dom_decompress_unsafe_zstd.c endif =20 -include $(XEN_TARGET_ARCH)/Makefile --- a/tools/libs/guest/xg_dom_bzimageloader.c +++ b/tools/libs/guest/xg_dom_bzimageloader.c @@ -589,6 +589,85 @@ static int xc_try_lzo1x_decode( =20 #endif =20 +#if defined(HAVE_ZSTD) + +#include + +static int xc_try_zstd_decode( + struct xc_dom_image *dom, void **blob, size_t *size) +{ + size_t outsize, insize, actual; + unsigned char *outbuf; + + /* Magic, descriptor byte, and trailing size field. */ + if ( *size <=3D 9 ) + { + DOMPRINTF("ZSTD: insufficient input data"); + return -1; + } + + insize =3D *size - 4; + outsize =3D *(uint32_t *)(*blob + insize); + + if ( xc_dom_kernel_check_size(dom, outsize) ) + { + DOMPRINTF("ZSTD: output too large"); + return -1; + } + + outbuf =3D malloc(outsize); + if ( !outbuf ) + { + DOMPRINTF("ZSTD: failed to alloc memory"); + return -1; + } + + actual =3D ZSTD_decompress(outbuf, outsize, *blob, insize); + + if ( ZSTD_isError(actual) ) + { + DOMPRINTF("ZSTD: error: %s", ZSTD_getErrorName(actual)); + free(outbuf); + return -1; + } + + if ( actual !=3D outsize ) + { + DOMPRINTF("ZSTD: got 0x%zx bytes instead of 0x%zx", + actual, outsize); + free(outbuf); + return -1; + } + + if ( xc_dom_register_external(dom, outbuf, outsize) ) + { + DOMPRINTF("ZSTD: error registering stream output"); + free(outbuf); + return -1; + } + + DOMPRINTF("%s: ZSTD decompress OK, 0x%zx -> 0x%zx", + __FUNCTION__, insize, outsize); + + *blob =3D outbuf; + *size =3D outsize; + + return 0; +} + +#else /* !defined(HAVE_ZSTD) */ + +static int xc_try_zstd_decode( + struct xc_dom_image *dom, void **blob, size_t *size) +{ + xc_dom_panic(dom->xch, XC_INTERNAL_ERROR, + "%s: ZSTD decompress support unavailable\n", + __FUNCTION__); + return -1; +} + +#endif + #else /* __MINIOS__ */ =20 int xc_try_bzip2_decode(struct xc_dom_image *dom, void **blob, size_t *siz= e); @@ -735,6 +814,17 @@ static int xc_dom_probe_bzimage_kernel(s __FUNCTION__); return -EINVAL; } + } + else if ( check_magic(dom, "\x28\xb5\x2f\xfd", 4) ) + { + ret =3D xc_try_zstd_decode(dom, &dom->kernel_blob, &dom->kernel_si= ze); + if ( ret < 0 ) + { + xc_dom_panic(dom->xch, XC_INVALID_KERNEL, + "%s unable to ZSTD decompress kernel", + __FUNCTION__); + return -EINVAL; + } } else if ( check_magic(dom, "\135\000", 2) ) { --- /dev/null +++ b/tools/libs/guest/xg_dom_decompress_unsafe_zstd.c @@ -0,0 +1,45 @@ +#include +#include +#include +#include +#include +#include + +#include "xg_private.h" +#include "xg_dom_decompress_unsafe.h" + +typedef uint8_t u8; + +typedef uint16_t __u16; +typedef uint32_t __u32; +typedef uint64_t __u64; + +typedef uint16_t __le16; +typedef uint32_t __le32; +typedef uint64_t __le64; + +typedef uint16_t __be16; +typedef uint32_t __be32; +typedef uint64_t __be64; + +#define __attribute_const__ +#define __force +#define always_inline +#define noinline + +#undef ERROR + +#define __BYTEORDER_HAS_U64__ +#define __TYPES_H__ /* xen/types.h guard */ +#include "../../xen/include/xen/byteorder/little_endian.h" +#define __ASM_UNALIGNED_H__ /* asm/unaligned.h guard */ +#include "../../xen/include/xen/unaligned.h" +#include "../../xen/include/xen/xxhash.h" +#include "../../xen/lib/xxhash64.c" +#include "../../xen/common/unzstd.c" + +int xc_try_zstd_decode( + struct xc_dom_image *dom, void **blob, size_t *size) +{ + return xc_dom_decompress_unsafe(unzstd, dom, blob, size); +} --- a/tools/libs/guest/xg_dom_decompress_unsafe.h +++ b/tools/libs/guest/xg_dom_decompress_unsafe.h @@ -16,3 +16,5 @@ int xc_try_lzo1x_decode(struct xc_dom_im __attribute__((visibility("internal"))); int xc_try_xz_decode(struct xc_dom_image *dom, void **blob, size_t *size) __attribute__((visibility("internal"))); +int xc_try_zstd_decode(struct xc_dom_image *dom, void **blob, size_t *size) + __attribute__((visibility("internal"))); --- a/xen/common/zstd/decompress.c +++ b/xen/common/zstd/decompress.c @@ -33,7 +33,6 @@ #include "huf.h" #include "mem.h" /* low level memory routines */ #include "zstd_internal.h" -#include /* memcpy, memmove, memset */ =20 #define ZSTD_PREFETCH(ptr) __builtin_prefetch(ptr, 0, 0) =20 @@ -99,9 +98,12 @@ struct ZSTD_DCtx_s { BYTE headerBuffer[ZSTD_FRAMEHEADERSIZE_MAX]; }; /* typedef'd to ZSTD_DCtx within "zstd.h" */ =20 -size_t INIT ZSTD_DCtxWorkspaceBound(void) { return ZSTD_ALIGN(sizeof(ZSTD_= stack)) + ZSTD_ALIGN(sizeof(ZSTD_DCtx)); } +STATIC size_t INIT ZSTD_DCtxWorkspaceBound(void) +{ + return ZSTD_ALIGN(sizeof(ZSTD_stack)) + ZSTD_ALIGN(sizeof(ZSTD_DCtx)); +} =20 -size_t INIT ZSTD_decompressBegin(ZSTD_DCtx *dctx) +STATIC size_t INIT ZSTD_decompressBegin(ZSTD_DCtx *dctx) { dctx->expected =3D ZSTD_frameHeaderSize_prefix; dctx->stage =3D ZSTDds_getFrameHeaderSize; @@ -121,7 +123,7 @@ size_t INIT ZSTD_decompressBegin(ZSTD_DC return 0; } =20 -ZSTD_DCtx *INIT ZSTD_createDCtx_advanced(ZSTD_customMem customMem) +STATIC ZSTD_DCtx *INIT ZSTD_createDCtx_advanced(ZSTD_customMem customMem) { ZSTD_DCtx *dctx; =20 @@ -136,7 +138,7 @@ ZSTD_DCtx *INIT ZSTD_createDCtx_advanced return dctx; } =20 -ZSTD_DCtx *INIT ZSTD_initDCtx(void *workspace, size_t workspaceSize) +STATIC ZSTD_DCtx *INIT ZSTD_initDCtx(void *workspace, size_t workspaceSize) { ZSTD_customMem const stackMem =3D ZSTD_initStack(workspace, workspaceSize= ); return ZSTD_createDCtx_advanced(stackMem); @@ -150,11 +152,13 @@ size_t INIT ZSTD_freeDCtx(ZSTD_DCtx *dct return 0; /* reserved as a potential error code in the future */ } =20 +#ifdef BUILD_DEAD_CODE void INIT ZSTD_copyDCtx(ZSTD_DCtx *dstDCtx, const ZSTD_DCtx *srcDCtx) { size_t const workSpaceSize =3D (ZSTD_BLOCKSIZE_ABSOLUTEMAX + WILDCOPY_OVE= RLENGTH) + ZSTD_frameHeaderSize_max; memcpy(dstDCtx, srcDCtx, sizeof(ZSTD_DCtx) - workSpaceSize); /* no need t= o copy workspace */ } +#endif =20 STATIC size_t ZSTD_findFrameCompressedSize(const void *src, size_t srcSize= ); STATIC size_t ZSTD_decompressBegin_usingDict(ZSTD_DCtx *dctx, const void *= dict, @@ -166,6 +170,7 @@ static void ZSTD_refDDict(ZSTD_DCtx *dst * Decompression section ***************************************************************/ =20 +#ifdef BUILD_DEAD_CODE /*! ZSTD_isFrame() : * Tells if the content of `buffer` starts with a valid Frame Identifier. * Note : Frame Identifier is 4 bytes. If `size < 4`, @return will always= be 0. @@ -184,6 +189,7 @@ unsigned INIT ZSTD_isFrame(const void *b } return 0; } +#endif =20 /** ZSTD_frameHeaderSize() : * srcSize must be >=3D ZSTD_frameHeaderSize_prefix. @@ -206,7 +212,7 @@ static size_t INIT ZSTD_frameHeaderSize( * @return : 0, `fparamsPtr` is correctly filled, * >0, `srcSize` is too small, result is expected `srcSize`, * or an error code, which can be tested using ZSTD_isError() */ -size_t INIT ZSTD_getFrameParams(ZSTD_frameParams *fparamsPtr, const void *= src, size_t srcSize) +STATIC size_t INIT ZSTD_getFrameParams(ZSTD_frameParams *fparamsPtr, const= void *src, size_t srcSize) { const BYTE *ip =3D (const BYTE *)src; =20 @@ -291,6 +297,7 @@ size_t INIT ZSTD_getFrameParams(ZSTD_fra return 0; } =20 +#ifdef BUILD_DEAD_CODE /** ZSTD_getFrameContentSize() : * compatible with legacy mode * @return : decompressed size of the single frame pointed to be `src` if= known, otherwise @@ -367,6 +374,7 @@ unsigned long long INIT ZSTD_findDecompr return totalDstSize; } } +#endif /* BUILD_DEAD_CODE */ =20 /** ZSTD_decodeFrameHeader() : * `headerSize` must be the size provided by ZSTD_frameHeaderSize(). @@ -393,7 +401,7 @@ typedef struct { =20 /*! ZSTD_getcBlockSize() : * Provides the size of compressed block from block header `src` */ -size_t INIT ZSTD_getcBlockSize(const void *src, size_t srcSize, blockPrope= rties_t *bpPtr) +STATIC size_t INIT ZSTD_getcBlockSize(const void *src, size_t srcSize, blo= ckProperties_t *bpPtr) { if (srcSize < ZSTD_blockHeaderSize) return ERROR(srcSize_wrong); @@ -431,7 +439,7 @@ static size_t INIT ZSTD_setRleBlock(void =20 /*! ZSTD_decodeLiteralsBlock() : @return : nb of bytes read from src (< srcSize ) */ -size_t INIT ZSTD_decodeLiteralsBlock(ZSTD_DCtx *dctx, const void *src, siz= e_t srcSize) /* note : srcSize < BLOCKSIZE */ +STATIC size_t INIT ZSTD_decodeLiteralsBlock(ZSTD_DCtx *dctx, const void *s= rc, size_t srcSize) /* note : srcSize < BLOCKSIZE */ { if (srcSize < MIN_CBLOCK_SIZE) return ERROR(corruption_detected); @@ -795,7 +803,7 @@ static size_t INIT ZSTD_buildSeqTable(FS } } =20 -size_t INIT ZSTD_decodeSeqHeaders(ZSTD_DCtx *dctx, int *nbSeqPtr, const vo= id *src, size_t srcSize) +STATIC size_t INIT ZSTD_decodeSeqHeaders(ZSTD_DCtx *dctx, int *nbSeqPtr, c= onst void *src, size_t srcSize) { const BYTE *const istart =3D (const BYTE *const)src; const BYTE *const iend =3D istart + srcSize; @@ -1481,6 +1489,7 @@ static void INIT ZSTD_checkContinuity(ZS } } =20 +#ifdef BUILD_DEAD_CODE size_t INIT ZSTD_decompressBlock(ZSTD_DCtx *dctx, void *dst, size_t dstCap= acity, const void *src, size_t srcSize) { size_t dSize; @@ -1498,8 +1507,9 @@ size_t INIT ZSTD_insertBlock(ZSTD_DCtx * dctx->previousDstEnd =3D (const char *)blockStart + blockSize; return blockSize; } +#endif /* BUILD_DEAD_CODE */ =20 -size_t INIT ZSTD_generateNxBytes(void *dst, size_t dstCapacity, BYTE byte,= size_t length) +STATIC size_t INIT ZSTD_generateNxBytes(void *dst, size_t dstCapacity, BYT= E byte, size_t length) { if (length > dstCapacity) return ERROR(dstSize_tooSmall); @@ -1512,7 +1522,7 @@ size_t INIT ZSTD_generateNxBytes(void *d * `src` must point to the start of a ZSTD frame, ZSTD legacy frame, or s= kippable frame * `srcSize` must be at least as large as the frame contained * @return : the compressed size of the frame starting at `src` */ -size_t INIT ZSTD_findFrameCompressedSize(const void *src, size_t srcSize) +STATIC size_t INIT ZSTD_findFrameCompressedSize(const void *src, size_t sr= cSize) { if (srcSize >=3D ZSTD_skippableHeaderSize && (ZSTD_readLE32(src) & 0xFFFF= FFF0U) =3D=3D ZSTD_MAGIC_SKIPPABLE_START) { return ZSTD_skippableHeaderSize + ZSTD_readLE32((const BYTE *)src + 4); @@ -1709,12 +1719,12 @@ static size_t INIT ZSTD_decompressMultiF return (BYTE *)dst - (BYTE *)dststart; } =20 -size_t INIT ZSTD_decompress_usingDict(ZSTD_DCtx *dctx, void *dst, size_t d= stCapacity, const void *src, size_t srcSize, const void *dict, size_t dictS= ize) +STATIC size_t INIT ZSTD_decompress_usingDict(ZSTD_DCtx *dctx, void *dst, s= ize_t dstCapacity, const void *src, size_t srcSize, const void *dict, size_= t dictSize) { return ZSTD_decompressMultiFrame(dctx, dst, dstCapacity, src, srcSize, di= ct, dictSize, NULL); } =20 -size_t INIT ZSTD_decompressDCtx(ZSTD_DCtx *dctx, void *dst, size_t dstCapa= city, const void *src, size_t srcSize) +STATIC size_t INIT ZSTD_decompressDCtx(ZSTD_DCtx *dctx, void *dst, size_t = dstCapacity, const void *src, size_t srcSize) { return ZSTD_decompress_usingDict(dctx, dst, dstCapacity, src, srcSize, NU= LL, 0); } @@ -1723,9 +1733,12 @@ size_t INIT ZSTD_decompressDCtx(ZSTD_DCt * Advanced Streaming Decompression API * Bufferless and synchronous ****************************************/ -size_t INIT ZSTD_nextSrcSizeToDecompress(ZSTD_DCtx *dctx) { return dctx->e= xpected; } +STATIC size_t INIT ZSTD_nextSrcSizeToDecompress(ZSTD_DCtx *dctx) +{ + return dctx->expected; +} =20 -ZSTD_nextInputType_e INIT ZSTD_nextInputType(ZSTD_DCtx *dctx) +STATIC ZSTD_nextInputType_e INIT ZSTD_nextInputType(ZSTD_DCtx *dctx) { switch (dctx->stage) { default: /* should not happen */ @@ -1745,7 +1758,7 @@ int INIT ZSTD_isSkipFrame(ZSTD_DCtx *dct /** ZSTD_decompressContinue() : * @return : nb of bytes generated into `dst` (necessarily <=3D `dstCapac= ity) * or an error code, which can be tested using ZSTD_isError() */ -size_t INIT ZSTD_decompressContinue(ZSTD_DCtx *dctx, void *dst, size_t dst= Capacity, const void *src, size_t srcSize) +STATIC size_t INIT ZSTD_decompressContinue(ZSTD_DCtx *dctx, void *dst, siz= e_t dstCapacity, const void *src, size_t srcSize) { /* Sanity check */ if (srcSize !=3D dctx->expected) @@ -1971,7 +1984,7 @@ static size_t INIT ZSTD_decompress_inser return ZSTD_refDictContent(dctx, dict, dictSize); } =20 -size_t INIT ZSTD_decompressBegin_usingDict(ZSTD_DCtx *dctx, const void *di= ct, size_t dictSize) +STATIC size_t INIT ZSTD_decompressBegin_usingDict(ZSTD_DCtx *dctx, const v= oid *dict, size_t dictSize) { CHECK_F(ZSTD_decompressBegin(dctx)); if (dict && dictSize) @@ -1991,7 +2004,9 @@ struct ZSTD_DDict_s { ZSTD_customMem cMem; }; /* typedef'd to ZSTD_DDict within "zstd.h" */ =20 +#ifdef BUILD_DEAD_CODE size_t INIT ZSTD_DDictWorkspaceBound(void) { return ZSTD_ALIGN(sizeof(ZSTD= _stack)) + ZSTD_ALIGN(sizeof(ZSTD_DDict)); } +#endif =20 static const void *INIT ZSTD_DDictDictContent(const ZSTD_DDict *ddict) { r= eturn ddict->dictContent; } =20 @@ -2023,6 +2038,7 @@ static void INIT ZSTD_refDDict(ZSTD_DCtx } } =20 +#ifdef BUILD_DEAD_CODE static size_t INIT ZSTD_loadEntropy_inDDict(ZSTD_DDict *ddict) { ddict->dictID =3D 0; @@ -2090,6 +2106,7 @@ ZSTD_DDict *INIT ZSTD_initDDict(const vo ZSTD_customMem const stackMem =3D ZSTD_initStack(workspace, workspaceSize= ); return ZSTD_createDDict_advanced(dict, dictSize, 1, stackMem); } +#endif /* BUILD_DEAD_CODE */ =20 size_t INIT ZSTD_freeDDict(ZSTD_DDict *ddict) { @@ -2103,6 +2120,7 @@ size_t INIT ZSTD_freeDDict(ZSTD_DDict *d } } =20 +#ifdef BUILD_DEAD_CODE /*! ZSTD_getDictID_fromDict() : * Provides the dictID stored within dictionary. * if @return =3D=3D 0, the dictionary is not conformant with Zstandard s= pecification. @@ -2145,11 +2163,12 @@ unsigned INIT ZSTD_getDictID_fromFrame(c return 0; return zfp.dictID; } +#endif /* BUILD_DEAD_CODE */ =20 /*! ZSTD_decompress_usingDDict() : * Decompression using a pre-digested Dictionary * Use dictionary without significant overhead. */ -size_t INIT ZSTD_decompress_usingDDict(ZSTD_DCtx *dctx, void *dst, size_t = dstCapacity, const void *src, size_t srcSize, const ZSTD_DDict *ddict) +STATIC size_t INIT ZSTD_decompress_usingDDict(ZSTD_DCtx *dctx, void *dst, = size_t dstCapacity, const void *src, size_t srcSize, const ZSTD_DDict *ddic= t) { /* pass content and size in case legacy frames are encountered */ return ZSTD_decompressMultiFrame(dctx, dst, dstCapacity, src, srcSize, NU= LL, 0, ddict); @@ -2186,7 +2205,7 @@ struct ZSTD_DStream_s { U32 hostageByte; }; /* typedef'd to ZSTD_DStream within "zstd.h" */ =20 -size_t INIT ZSTD_DStreamWorkspaceBound(size_t maxWindowSize) +STATIC size_t INIT ZSTD_DStreamWorkspaceBound(size_t maxWindowSize) { size_t const blockSize =3D MIN(maxWindowSize, ZSTD_BLOCKSIZE_ABSOLUTEMAX); size_t const inBuffSize =3D blockSize; @@ -2216,7 +2235,7 @@ static ZSTD_DStream *INIT ZSTD_createDSt return zds; } =20 -ZSTD_DStream *INIT ZSTD_initDStream(size_t maxWindowSize, void *workspace,= size_t workspaceSize) +STATIC ZSTD_DStream *INIT ZSTD_initDStream(size_t maxWindowSize, void *wor= kspace, size_t workspaceSize) { ZSTD_customMem const stackMem =3D ZSTD_initStack(workspace, workspaceSize= ); ZSTD_DStream *zds =3D ZSTD_createDStream_advanced(stackMem); @@ -2249,6 +2268,7 @@ ZSTD_DStream *INIT ZSTD_initDStream(size return zds; } =20 +#ifdef BUILD_DEAD_CODE ZSTD_DStream *INIT ZSTD_initDStream_usingDDict(size_t maxWindowSize, const= ZSTD_DDict *ddict, void *workspace, size_t workspaceSize) { ZSTD_DStream *zds =3D ZSTD_initDStream(maxWindowSize, workspace, workspac= eSize); @@ -2257,6 +2277,7 @@ ZSTD_DStream *INIT ZSTD_initDStream_usin } return zds; } +#endif =20 size_t INIT ZSTD_freeDStream(ZSTD_DStream *zds) { @@ -2279,10 +2300,12 @@ size_t INIT ZSTD_freeDStream(ZSTD_DStrea =20 /* *** Initialization *** */ =20 +#ifdef BUILD_DEAD_CODE size_t INIT ZSTD_DStreamInSize(void) { return ZSTD_BLOCKSIZE_ABSOLUTEMAX += ZSTD_blockHeaderSize; } size_t INIT ZSTD_DStreamOutSize(void) { return ZSTD_BLOCKSIZE_ABSOLUTEMAX;= } +#endif =20 -size_t INIT ZSTD_resetDStream(ZSTD_DStream *zds) +STATIC size_t INIT ZSTD_resetDStream(ZSTD_DStream *zds) { zds->stage =3D zdss_loadHeader; zds->lhSize =3D zds->inPos =3D zds->outStart =3D zds->outEnd =3D 0; @@ -2300,7 +2323,7 @@ ZSTD_STATIC size_t INIT ZSTD_limitCopy(v return length; } =20 -size_t INIT ZSTD_decompressStream(ZSTD_DStream *zds, ZSTD_outBuffer *outpu= t, ZSTD_inBuffer *input) +STATIC size_t INIT ZSTD_decompressStream(ZSTD_DStream *zds, ZSTD_outBuffer= *output, ZSTD_inBuffer *input) { const char *const istart =3D (const char *)(input->src) + input->pos; const char *const iend =3D (const char *)(input->src) + input->size; --- a/xen/common/zstd/error_private.h +++ b/xen/common/zstd/error_private.h @@ -19,11 +19,6 @@ #ifndef ERROR_H_MODULE #define ERROR_H_MODULE =20 -/* **************************************** -* Dependencies -******************************************/ -#include /* size_t */ - /** * enum ZSTD_ErrorCode - zstd error codes * --- a/xen/common/zstd/fse.h +++ b/xen/common/zstd/fse.h @@ -41,11 +41,6 @@ #define FSE_H =20 /*-***************************************** -* Dependencies -******************************************/ -#include /* size_t, ptrdiff_t */ - -/*-***************************************** * FSE_PUBLIC_API : control library symbols visibility ******************************************/ #define FSE_PUBLIC_API --- a/xen/common/zstd/fse_decompress.c +++ b/xen/common/zstd/fse_decompress.c @@ -48,8 +48,6 @@ #include "bitstream.h" #include "fse.h" #include "zstd_internal.h" -#include -#include /* memcpy, memset */ =20 /* ************************************************************** * Error Management --- a/xen/common/zstd/huf.h +++ b/xen/common/zstd/huf.h @@ -40,9 +40,6 @@ #ifndef HUF_H_298734234 #define HUF_H_298734234 =20 -/* *** Dependencies *** */ -#include /* size_t */ - /* *** Tool functions *** */ #define HUF_BLOCKSIZE_MAX (128 * 1024) /**< maximum input size for a singl= e block compressed with HUF_compress */ size_t HUF_compressBound(size_t size); /**< maximum compressed size (worst= case) */ --- a/xen/common/zstd/huf_decompress.c +++ b/xen/common/zstd/huf_decompress.c @@ -48,8 +48,6 @@ #include "bitstream.h" /* BIT_* */ #include "fse.h" /* header compression */ #include "huf.h" -#include -#include /* memcpy, memset */ =20 /* ************************************************************** * Error Management --- a/xen/common/zstd/mem.h +++ b/xen/common/zstd/mem.h @@ -20,9 +20,11 @@ /*-**************************************** * Dependencies ******************************************/ +#ifdef __XEN__ #include /* memcpy */ #include /* size_t, ptrdiff_t */ #include +#endif =20 /*-**************************************** * Compiler specifics --- a/xen/common/zstd/zstd_internal.h +++ b/xen/common/zstd/zstd_internal.h @@ -28,8 +28,10 @@ ***************************************/ #include "error_private.h" #include "mem.h" +#ifdef __XEN__ #include #include +#endif =20 #define ALIGN(x, a) ((x + (a) - 1) & ~((a) - 1)) #define PTR_ALIGN(p, a) ((typeof(p))ALIGN((unsigned long)(p), (a))) @@ -95,8 +97,10 @@ typedef struct ZSTD_DStream_s ZSTD_DStre /*-************************************* * shared macros ***************************************/ +#ifndef MIN #define MIN(a, b) ((a) < (b) ? (a) : (b)) #define MAX(a, b) ((a) > (b) ? (a) : (b)) +#endif #define CHECK_F(f) \ { \ size_t const errcod =3D f; \ --- a/xen/include/xen/unaligned.h +++ b/xen/include/xen/unaligned.h @@ -10,8 +10,10 @@ #ifndef __XEN_UNALIGNED_H__ #define __XEN_UNALIGNED_H__ =20 +#ifdef __XEN__ #include #include +#endif =20 #define get_unaligned(p) (*(p)) #define put_unaligned(val, p) (*(p) =3D (val)) --- a/xen/lib/xxhash64.c +++ b/xen/lib/xxhash64.c @@ -38,11 +38,13 @@ * - xxHash source repository: https://github.com/Cyan4973/xxHash */ =20 +#ifdef __XEN__ #include #include #include #include #include +#endif =20 /*-************************************* * Macros From nobody Wed May 8 13:32:38 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1611243650; cv=none; d=zohomail.com; s=zohoarc; b=BrBJ4B5NONnsfJ84ZI282Ghkflt8JKTwBUBe/7R0o0SyZ/xdXlnc/XqQUqkqnd3PfryExJOpm98fO04RsHZpHsSudNrKMjMtntUh5I7v0xZ/cpN4oglF3AxlD+Eo4isbSbMUhZx7XuUtdJr2WdWOxCS5dBEpupYTeiucZfjl2XA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1611243650; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Eeg10pjA6Ab2MZUm332DPNbVDecp/aJf14wUlLEOE90=; b=n5TVe3oqxbMsHq7jaKPfCVHi4u/B15Mwahntdd2cu8mTL9rSStYOmUWHVAdRhzlQHZMRBCNI1OvKvEuGhwfALyIUVPe5hgKslhh5oeKCM/S+B7Yl2BfovY90py+asooB9qD8EM3i1dlqagAGmoDdEov9h7xqfsfmTiKAdFaCI40= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1611243650408587.2710886767295; Thu, 21 Jan 2021 07:40:50 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.72214.129848 (Exim 4.92) (envelope-from ) id 1l2c4R-0000FX-EF; Thu, 21 Jan 2021 15:40:27 +0000 Received: by outflank-mailman (output) from mailman id 72214.129848; Thu, 21 Jan 2021 15:40:27 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l2c4R-0000FQ-BA; Thu, 21 Jan 2021 15:40:27 +0000 Received: by outflank-mailman (input) for mailman id 72214; Thu, 21 Jan 2021 15:40:25 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l2c4P-0000FL-68 for xen-devel@lists.xenproject.org; Thu, 21 Jan 2021 15:40:25 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id f8559c27-7323-4040-b416-839b1b2f9157; Thu, 21 Jan 2021 15:40:22 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id AFC52AC97; Thu, 21 Jan 2021 15:40:21 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f8559c27-7323-4040-b416-839b1b2f9157 X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1611243621; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Eeg10pjA6Ab2MZUm332DPNbVDecp/aJf14wUlLEOE90=; b=IrM4KLCGrMi3YIR27e8frZWSLVWkuSB7cPoVkTI9sqYnvIivTe1qcufxtRIil8xAzE36Y1 NNR+lYpnTXXab1vLGM/tgzmBMSyiGJpTSJOaxid07pRvFy78zPTalHAI40MAjOQy5LPazN snDdynXswNPXEot5u4PDdThqJtxl3K4= Subject: [PATCH v2.5 1/5] libxenguest: support zstd compressed kernels From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Stefano Stabellini , Wei Liu , M A Young References: Message-ID: Date: Thu, 21 Jan 2021 16:40:20 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" This follows the logic used for other decompression methods utilizing an external library, albeit here we can't ignore the 32-bit size field appended to the compressed image - its presence causes decompression to fail. Leverage the field instead to allocate the output buffer in one go, i.e. without incrementally realloc()ing. As far as configure.ac goes, I'm pretty sure there is a better (more "standard") way of using PKG_CHECK_MODULES(). The construct also gets put next to the other decompression library checks, albeit I think they all ought to be x86-specific (e.g. placed in the existing case block a few lines down). Note that, where possible, instead of #ifdef-ing xen/*.h inclusions, they get removed. Signed-off-by: Jan Beulich Acked-by: Wei Liu --- Note for committer: As an alternative to patching tools/configure here, autoconf may want re-running. --- v2.5: Don't make libzstd a hard dependency. Adjust ./README. v2: New. --- a/README +++ b/README @@ -84,6 +84,8 @@ disabled at compile time: * 16-bit x86 assembler, loader and compiler for qemu-traditional / rom= bios (dev86 rpm or bin86 & bcc debs) * Development install of liblzma for rombios + * Development install of libbz2, liblzma, liblzo2, and libzstd for DomU + kernel decompression. =20 Second, you need to acquire a suitable kernel for use in domain 0. If possible you should use a kernel provided by your OS distributor. If --- a/tools/configure +++ b/tools/configure @@ -643,6 +643,8 @@ PTHREAD_CFLAGS EXTFS_LIBS system_aio zlib +libzstd_LIBS +libzstd_CFLAGS FETCHER FTP FALSE @@ -857,6 +859,8 @@ glib_CFLAGS glib_LIBS pixman_CFLAGS pixman_LIBS +libzstd_CFLAGS +libzstd_LIBS LIBNL3_CFLAGS LIBNL3_LIBS SYSTEMD_CFLAGS @@ -1605,6 +1609,10 @@ Some influential environment variables: pixman_CFLAGS C compiler flags for pixman, overriding pkg-config pixman_LIBS linker flags for pixman, overriding pkg-config + libzstd_CFLAGS + C compiler flags for libzstd, overriding pkg-config + libzstd_LIBS + linker flags for libzstd, overriding pkg-config LIBNL3_CFLAGS C compiler flags for LIBNL3, overriding pkg-config LIBNL3_LIBS linker flags for LIBNL3, overriding pkg-config @@ -8744,6 +8752,84 @@ fi =20 =20 =20 +pkg_failed=3Dno +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$libzstd_CFLAGS"; then + pkg_cv_libzstd_CFLAGS=3D"$libzstd_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --pri= nt-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=3D$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? =3D $ac_status" >&5 + test $ac_status =3D 0; }; then + pkg_cv_libzstd_CFLAGS=3D`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" !=3D "x0" && pkg_failed=3Dyes +else + pkg_failed=3Dyes +fi + else + pkg_failed=3Duntried +fi +if test -n "$libzstd_LIBS"; then + pkg_cv_libzstd_LIBS=3D"$libzstd_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --pri= nt-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=3D$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? =3D $ac_status" >&5 + test $ac_status =3D 0; }; then + pkg_cv_libzstd_LIBS=3D`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" !=3D "x0" && pkg_failed=3Dyes +else + pkg_failed=3Dyes +fi + else + pkg_failed=3Duntried +fi + + + +if test $pkg_failed =3D yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=3Dyes +else + _pkg_short_errors_supported=3Dno +fi + if test $_pkg_short_errors_supported =3D yes; then + libzstd_PKG_ERRORS=3D`$PKG_CONFIG --short-errors --print-errors -= -cflags --libs "libzstd" 2>&1` + else + libzstd_PKG_ERRORS=3D`$PKG_CONFIG --print-errors --cflags --libs = "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$libzstd_PKG_ERRORS" >&5 + + true +elif test $pkg_failed =3D untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + true +else + libzstd_CFLAGS=3D$pkg_cv_libzstd_CFLAGS + libzstd_LIBS=3D$pkg_cv_libzstd_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + true +fi +if test -z "$libzstd_PKG_ERRORS" +then + zlib=3D"$zlib -DHAVE_ZSTD $libzstd_CFLAGS $libzstd_LIBS" +else + { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: \"disabling zstd dec= ompression: $libzstd_PKG_ERRORS\"" >&5 +$as_echo "$as_me: WARNING: \"disabling zstd decompression: $libzstd_PKG_ER= RORS\"" >&2;} +fi + =20 =20 ac_fn_c_check_header_mongrel "$LINENO" "ext2fs/ext2fs.h" "ac_cv_header_ext= 2fs_ext2fs_h" "$ac_includes_default" --- a/tools/configure.ac +++ b/tools/configure.ac @@ -414,6 +414,13 @@ AC_CHECK_LIB([lzma], [lzma_stream_decode AC_CHECK_HEADER([lzo/lzo1x.h], [ AC_CHECK_LIB([lzo2], [lzo1x_decompress], [zlib=3D"$zlib -DHAVE_LZO1X -llzo= 2"]) ]) +PKG_CHECK_MODULES([libzstd], [libzstd], [true], [true]) +if test -z "$libzstd_PKG_ERRORS" +then + zlib=3D"$zlib -DHAVE_ZSTD $libzstd_CFLAGS $libzstd_LIBS" +else + AC_MSG_WARN("disabling zstd decompression: $libzstd_PKG_ERRORS") +fi AC_SUBST(zlib) AC_SUBST(system_aio) AX_CHECK_EXTFS --- a/tools/libs/guest/Makefile +++ b/tools/libs/guest/Makefile @@ -64,6 +64,7 @@ SRCS-y +=3D xg_dom_decompr SRCS-y +=3D xg_dom_decompress_unsafe_lzma.c SRCS-y +=3D xg_dom_decompress_unsafe_lzo1x.c SRCS-y +=3D xg_dom_decompress_unsafe_xz.c +SRCS-y +=3D xg_dom_decompress_unsafe_zstd.c endif =20 -include $(XEN_TARGET_ARCH)/Makefile --- a/tools/libs/guest/xg_dom_bzimageloader.c +++ b/tools/libs/guest/xg_dom_bzimageloader.c @@ -589,6 +589,85 @@ static int xc_try_lzo1x_decode( =20 #endif =20 +#if defined(HAVE_ZSTD) + +#include + +static int xc_try_zstd_decode( + struct xc_dom_image *dom, void **blob, size_t *size) +{ + size_t outsize, insize, actual; + unsigned char *outbuf; + + /* Magic, descriptor byte, and trailing size field. */ + if ( *size <=3D 9 ) + { + DOMPRINTF("ZSTD: insufficient input data"); + return -1; + } + + insize =3D *size - 4; + outsize =3D *(uint32_t *)(*blob + insize); + + if ( xc_dom_kernel_check_size(dom, outsize) ) + { + DOMPRINTF("ZSTD: output too large"); + return -1; + } + + outbuf =3D malloc(outsize); + if ( !outbuf ) + { + DOMPRINTF("ZSTD: failed to alloc memory"); + return -1; + } + + actual =3D ZSTD_decompress(outbuf, outsize, *blob, insize); + + if ( ZSTD_isError(actual) ) + { + DOMPRINTF("ZSTD: error: %s", ZSTD_getErrorName(actual)); + free(outbuf); + return -1; + } + + if ( actual !=3D outsize ) + { + DOMPRINTF("ZSTD: got 0x%zx bytes instead of 0x%zx", + actual, outsize); + free(outbuf); + return -1; + } + + if ( xc_dom_register_external(dom, outbuf, outsize) ) + { + DOMPRINTF("ZSTD: error registering stream output"); + free(outbuf); + return -1; + } + + DOMPRINTF("%s: ZSTD decompress OK, 0x%zx -> 0x%zx", + __FUNCTION__, insize, outsize); + + *blob =3D outbuf; + *size =3D outsize; + + return 0; +} + +#else /* !defined(HAVE_ZSTD) */ + +static int xc_try_zstd_decode( + struct xc_dom_image *dom, void **blob, size_t *size) +{ + xc_dom_panic(dom->xch, XC_INTERNAL_ERROR, + "%s: ZSTD decompress support unavailable\n", + __FUNCTION__); + return -1; +} + +#endif + #else /* __MINIOS__ */ =20 int xc_try_bzip2_decode(struct xc_dom_image *dom, void **blob, size_t *siz= e); @@ -735,6 +814,17 @@ static int xc_dom_probe_bzimage_kernel(s __FUNCTION__); return -EINVAL; } + } + else if ( check_magic(dom, "\x28\xb5\x2f\xfd", 4) ) + { + ret =3D xc_try_zstd_decode(dom, &dom->kernel_blob, &dom->kernel_si= ze); + if ( ret < 0 ) + { + xc_dom_panic(dom->xch, XC_INVALID_KERNEL, + "%s unable to ZSTD decompress kernel", + __FUNCTION__); + return -EINVAL; + } } else if ( check_magic(dom, "\135\000", 2) ) { --- /dev/null +++ b/tools/libs/guest/xg_dom_decompress_unsafe_zstd.c @@ -0,0 +1,45 @@ +#include +#include +#include +#include +#include +#include + +#include "xg_private.h" +#include "xg_dom_decompress_unsafe.h" + +typedef uint8_t u8; + +typedef uint16_t __u16; +typedef uint32_t __u32; +typedef uint64_t __u64; + +typedef uint16_t __le16; +typedef uint32_t __le32; +typedef uint64_t __le64; + +typedef uint16_t __be16; +typedef uint32_t __be32; +typedef uint64_t __be64; + +#define __attribute_const__ +#define __force +#define always_inline +#define noinline + +#undef ERROR + +#define __BYTEORDER_HAS_U64__ +#define __TYPES_H__ /* xen/types.h guard */ +#include "../../xen/include/xen/byteorder/little_endian.h" +#define __ASM_UNALIGNED_H__ /* asm/unaligned.h guard */ +#include "../../xen/include/xen/unaligned.h" +#include "../../xen/include/xen/xxhash.h" +#include "../../xen/lib/xxhash64.c" +#include "../../xen/common/unzstd.c" + +int xc_try_zstd_decode( + struct xc_dom_image *dom, void **blob, size_t *size) +{ + return xc_dom_decompress_unsafe(unzstd, dom, blob, size); +} --- a/tools/libs/guest/xg_dom_decompress_unsafe.h +++ b/tools/libs/guest/xg_dom_decompress_unsafe.h @@ -16,3 +16,5 @@ int xc_try_lzo1x_decode(struct xc_dom_im __attribute__((visibility("internal"))); int xc_try_xz_decode(struct xc_dom_image *dom, void **blob, size_t *size) __attribute__((visibility("internal"))); +int xc_try_zstd_decode(struct xc_dom_image *dom, void **blob, size_t *size) + __attribute__((visibility("internal"))); --- a/xen/common/zstd/decompress.c +++ b/xen/common/zstd/decompress.c @@ -33,7 +33,6 @@ #include "huf.h" #include "mem.h" /* low level memory routines */ #include "zstd_internal.h" -#include /* memcpy, memmove, memset */ =20 #define ZSTD_PREFETCH(ptr) __builtin_prefetch(ptr, 0, 0) =20 @@ -99,9 +98,12 @@ struct ZSTD_DCtx_s { BYTE headerBuffer[ZSTD_FRAMEHEADERSIZE_MAX]; }; /* typedef'd to ZSTD_DCtx within "zstd.h" */ =20 -size_t INIT ZSTD_DCtxWorkspaceBound(void) { return ZSTD_ALIGN(sizeof(ZSTD_= stack)) + ZSTD_ALIGN(sizeof(ZSTD_DCtx)); } +STATIC size_t INIT ZSTD_DCtxWorkspaceBound(void) +{ + return ZSTD_ALIGN(sizeof(ZSTD_stack)) + ZSTD_ALIGN(sizeof(ZSTD_DCtx)); +} =20 -size_t INIT ZSTD_decompressBegin(ZSTD_DCtx *dctx) +STATIC size_t INIT ZSTD_decompressBegin(ZSTD_DCtx *dctx) { dctx->expected =3D ZSTD_frameHeaderSize_prefix; dctx->stage =3D ZSTDds_getFrameHeaderSize; @@ -121,7 +123,7 @@ size_t INIT ZSTD_decompressBegin(ZSTD_DC return 0; } =20 -ZSTD_DCtx *INIT ZSTD_createDCtx_advanced(ZSTD_customMem customMem) +STATIC ZSTD_DCtx *INIT ZSTD_createDCtx_advanced(ZSTD_customMem customMem) { ZSTD_DCtx *dctx; =20 @@ -136,7 +138,7 @@ ZSTD_DCtx *INIT ZSTD_createDCtx_advanced return dctx; } =20 -ZSTD_DCtx *INIT ZSTD_initDCtx(void *workspace, size_t workspaceSize) +STATIC ZSTD_DCtx *INIT ZSTD_initDCtx(void *workspace, size_t workspaceSize) { ZSTD_customMem const stackMem =3D ZSTD_initStack(workspace, workspaceSize= ); return ZSTD_createDCtx_advanced(stackMem); @@ -150,11 +152,13 @@ size_t INIT ZSTD_freeDCtx(ZSTD_DCtx *dct return 0; /* reserved as a potential error code in the future */ } =20 +#ifdef BUILD_DEAD_CODE void INIT ZSTD_copyDCtx(ZSTD_DCtx *dstDCtx, const ZSTD_DCtx *srcDCtx) { size_t const workSpaceSize =3D (ZSTD_BLOCKSIZE_ABSOLUTEMAX + WILDCOPY_OVE= RLENGTH) + ZSTD_frameHeaderSize_max; memcpy(dstDCtx, srcDCtx, sizeof(ZSTD_DCtx) - workSpaceSize); /* no need t= o copy workspace */ } +#endif =20 STATIC size_t ZSTD_findFrameCompressedSize(const void *src, size_t srcSize= ); STATIC size_t ZSTD_decompressBegin_usingDict(ZSTD_DCtx *dctx, const void *= dict, @@ -166,6 +170,7 @@ static void ZSTD_refDDict(ZSTD_DCtx *dst * Decompression section ***************************************************************/ =20 +#ifdef BUILD_DEAD_CODE /*! ZSTD_isFrame() : * Tells if the content of `buffer` starts with a valid Frame Identifier. * Note : Frame Identifier is 4 bytes. If `size < 4`, @return will always= be 0. @@ -184,6 +189,7 @@ unsigned INIT ZSTD_isFrame(const void *b } return 0; } +#endif =20 /** ZSTD_frameHeaderSize() : * srcSize must be >=3D ZSTD_frameHeaderSize_prefix. @@ -206,7 +212,7 @@ static size_t INIT ZSTD_frameHeaderSize( * @return : 0, `fparamsPtr` is correctly filled, * >0, `srcSize` is too small, result is expected `srcSize`, * or an error code, which can be tested using ZSTD_isError() */ -size_t INIT ZSTD_getFrameParams(ZSTD_frameParams *fparamsPtr, const void *= src, size_t srcSize) +STATIC size_t INIT ZSTD_getFrameParams(ZSTD_frameParams *fparamsPtr, const= void *src, size_t srcSize) { const BYTE *ip =3D (const BYTE *)src; =20 @@ -291,6 +297,7 @@ size_t INIT ZSTD_getFrameParams(ZSTD_fra return 0; } =20 +#ifdef BUILD_DEAD_CODE /** ZSTD_getFrameContentSize() : * compatible with legacy mode * @return : decompressed size of the single frame pointed to be `src` if= known, otherwise @@ -367,6 +374,7 @@ unsigned long long INIT ZSTD_findDecompr return totalDstSize; } } +#endif /* BUILD_DEAD_CODE */ =20 /** ZSTD_decodeFrameHeader() : * `headerSize` must be the size provided by ZSTD_frameHeaderSize(). @@ -393,7 +401,7 @@ typedef struct { =20 /*! ZSTD_getcBlockSize() : * Provides the size of compressed block from block header `src` */ -size_t INIT ZSTD_getcBlockSize(const void *src, size_t srcSize, blockPrope= rties_t *bpPtr) +STATIC size_t INIT ZSTD_getcBlockSize(const void *src, size_t srcSize, blo= ckProperties_t *bpPtr) { if (srcSize < ZSTD_blockHeaderSize) return ERROR(srcSize_wrong); @@ -431,7 +439,7 @@ static size_t INIT ZSTD_setRleBlock(void =20 /*! ZSTD_decodeLiteralsBlock() : @return : nb of bytes read from src (< srcSize ) */ -size_t INIT ZSTD_decodeLiteralsBlock(ZSTD_DCtx *dctx, const void *src, siz= e_t srcSize) /* note : srcSize < BLOCKSIZE */ +STATIC size_t INIT ZSTD_decodeLiteralsBlock(ZSTD_DCtx *dctx, const void *s= rc, size_t srcSize) /* note : srcSize < BLOCKSIZE */ { if (srcSize < MIN_CBLOCK_SIZE) return ERROR(corruption_detected); @@ -795,7 +803,7 @@ static size_t INIT ZSTD_buildSeqTable(FS } } =20 -size_t INIT ZSTD_decodeSeqHeaders(ZSTD_DCtx *dctx, int *nbSeqPtr, const vo= id *src, size_t srcSize) +STATIC size_t INIT ZSTD_decodeSeqHeaders(ZSTD_DCtx *dctx, int *nbSeqPtr, c= onst void *src, size_t srcSize) { const BYTE *const istart =3D (const BYTE *const)src; const BYTE *const iend =3D istart + srcSize; @@ -1481,6 +1489,7 @@ static void INIT ZSTD_checkContinuity(ZS } } =20 +#ifdef BUILD_DEAD_CODE size_t INIT ZSTD_decompressBlock(ZSTD_DCtx *dctx, void *dst, size_t dstCap= acity, const void *src, size_t srcSize) { size_t dSize; @@ -1498,8 +1507,9 @@ size_t INIT ZSTD_insertBlock(ZSTD_DCtx * dctx->previousDstEnd =3D (const char *)blockStart + blockSize; return blockSize; } +#endif /* BUILD_DEAD_CODE */ =20 -size_t INIT ZSTD_generateNxBytes(void *dst, size_t dstCapacity, BYTE byte,= size_t length) +STATIC size_t INIT ZSTD_generateNxBytes(void *dst, size_t dstCapacity, BYT= E byte, size_t length) { if (length > dstCapacity) return ERROR(dstSize_tooSmall); @@ -1512,7 +1522,7 @@ size_t INIT ZSTD_generateNxBytes(void *d * `src` must point to the start of a ZSTD frame, ZSTD legacy frame, or s= kippable frame * `srcSize` must be at least as large as the frame contained * @return : the compressed size of the frame starting at `src` */ -size_t INIT ZSTD_findFrameCompressedSize(const void *src, size_t srcSize) +STATIC size_t INIT ZSTD_findFrameCompressedSize(const void *src, size_t sr= cSize) { if (srcSize >=3D ZSTD_skippableHeaderSize && (ZSTD_readLE32(src) & 0xFFFF= FFF0U) =3D=3D ZSTD_MAGIC_SKIPPABLE_START) { return ZSTD_skippableHeaderSize + ZSTD_readLE32((const BYTE *)src + 4); @@ -1709,12 +1719,12 @@ static size_t INIT ZSTD_decompressMultiF return (BYTE *)dst - (BYTE *)dststart; } =20 -size_t INIT ZSTD_decompress_usingDict(ZSTD_DCtx *dctx, void *dst, size_t d= stCapacity, const void *src, size_t srcSize, const void *dict, size_t dictS= ize) +STATIC size_t INIT ZSTD_decompress_usingDict(ZSTD_DCtx *dctx, void *dst, s= ize_t dstCapacity, const void *src, size_t srcSize, const void *dict, size_= t dictSize) { return ZSTD_decompressMultiFrame(dctx, dst, dstCapacity, src, srcSize, di= ct, dictSize, NULL); } =20 -size_t INIT ZSTD_decompressDCtx(ZSTD_DCtx *dctx, void *dst, size_t dstCapa= city, const void *src, size_t srcSize) +STATIC size_t INIT ZSTD_decompressDCtx(ZSTD_DCtx *dctx, void *dst, size_t = dstCapacity, const void *src, size_t srcSize) { return ZSTD_decompress_usingDict(dctx, dst, dstCapacity, src, srcSize, NU= LL, 0); } @@ -1723,9 +1733,12 @@ size_t INIT ZSTD_decompressDCtx(ZSTD_DCt * Advanced Streaming Decompression API * Bufferless and synchronous ****************************************/ -size_t INIT ZSTD_nextSrcSizeToDecompress(ZSTD_DCtx *dctx) { return dctx->e= xpected; } +STATIC size_t INIT ZSTD_nextSrcSizeToDecompress(ZSTD_DCtx *dctx) +{ + return dctx->expected; +} =20 -ZSTD_nextInputType_e INIT ZSTD_nextInputType(ZSTD_DCtx *dctx) +STATIC ZSTD_nextInputType_e INIT ZSTD_nextInputType(ZSTD_DCtx *dctx) { switch (dctx->stage) { default: /* should not happen */ @@ -1745,7 +1758,7 @@ int INIT ZSTD_isSkipFrame(ZSTD_DCtx *dct /** ZSTD_decompressContinue() : * @return : nb of bytes generated into `dst` (necessarily <=3D `dstCapac= ity) * or an error code, which can be tested using ZSTD_isError() */ -size_t INIT ZSTD_decompressContinue(ZSTD_DCtx *dctx, void *dst, size_t dst= Capacity, const void *src, size_t srcSize) +STATIC size_t INIT ZSTD_decompressContinue(ZSTD_DCtx *dctx, void *dst, siz= e_t dstCapacity, const void *src, size_t srcSize) { /* Sanity check */ if (srcSize !=3D dctx->expected) @@ -1971,7 +1984,7 @@ static size_t INIT ZSTD_decompress_inser return ZSTD_refDictContent(dctx, dict, dictSize); } =20 -size_t INIT ZSTD_decompressBegin_usingDict(ZSTD_DCtx *dctx, const void *di= ct, size_t dictSize) +STATIC size_t INIT ZSTD_decompressBegin_usingDict(ZSTD_DCtx *dctx, const v= oid *dict, size_t dictSize) { CHECK_F(ZSTD_decompressBegin(dctx)); if (dict && dictSize) @@ -1991,7 +2004,9 @@ struct ZSTD_DDict_s { ZSTD_customMem cMem; }; /* typedef'd to ZSTD_DDict within "zstd.h" */ =20 +#ifdef BUILD_DEAD_CODE size_t INIT ZSTD_DDictWorkspaceBound(void) { return ZSTD_ALIGN(sizeof(ZSTD= _stack)) + ZSTD_ALIGN(sizeof(ZSTD_DDict)); } +#endif =20 static const void *INIT ZSTD_DDictDictContent(const ZSTD_DDict *ddict) { r= eturn ddict->dictContent; } =20 @@ -2023,6 +2038,7 @@ static void INIT ZSTD_refDDict(ZSTD_DCtx } } =20 +#ifdef BUILD_DEAD_CODE static size_t INIT ZSTD_loadEntropy_inDDict(ZSTD_DDict *ddict) { ddict->dictID =3D 0; @@ -2090,6 +2106,7 @@ ZSTD_DDict *INIT ZSTD_initDDict(const vo ZSTD_customMem const stackMem =3D ZSTD_initStack(workspace, workspaceSize= ); return ZSTD_createDDict_advanced(dict, dictSize, 1, stackMem); } +#endif /* BUILD_DEAD_CODE */ =20 size_t INIT ZSTD_freeDDict(ZSTD_DDict *ddict) { @@ -2103,6 +2120,7 @@ size_t INIT ZSTD_freeDDict(ZSTD_DDict *d } } =20 +#ifdef BUILD_DEAD_CODE /*! ZSTD_getDictID_fromDict() : * Provides the dictID stored within dictionary. * if @return =3D=3D 0, the dictionary is not conformant with Zstandard s= pecification. @@ -2145,11 +2163,12 @@ unsigned INIT ZSTD_getDictID_fromFrame(c return 0; return zfp.dictID; } +#endif /* BUILD_DEAD_CODE */ =20 /*! ZSTD_decompress_usingDDict() : * Decompression using a pre-digested Dictionary * Use dictionary without significant overhead. */ -size_t INIT ZSTD_decompress_usingDDict(ZSTD_DCtx *dctx, void *dst, size_t = dstCapacity, const void *src, size_t srcSize, const ZSTD_DDict *ddict) +STATIC size_t INIT ZSTD_decompress_usingDDict(ZSTD_DCtx *dctx, void *dst, = size_t dstCapacity, const void *src, size_t srcSize, const ZSTD_DDict *ddic= t) { /* pass content and size in case legacy frames are encountered */ return ZSTD_decompressMultiFrame(dctx, dst, dstCapacity, src, srcSize, NU= LL, 0, ddict); @@ -2186,7 +2205,7 @@ struct ZSTD_DStream_s { U32 hostageByte; }; /* typedef'd to ZSTD_DStream within "zstd.h" */ =20 -size_t INIT ZSTD_DStreamWorkspaceBound(size_t maxWindowSize) +STATIC size_t INIT ZSTD_DStreamWorkspaceBound(size_t maxWindowSize) { size_t const blockSize =3D MIN(maxWindowSize, ZSTD_BLOCKSIZE_ABSOLUTEMAX); size_t const inBuffSize =3D blockSize; @@ -2216,7 +2235,7 @@ static ZSTD_DStream *INIT ZSTD_createDSt return zds; } =20 -ZSTD_DStream *INIT ZSTD_initDStream(size_t maxWindowSize, void *workspace,= size_t workspaceSize) +STATIC ZSTD_DStream *INIT ZSTD_initDStream(size_t maxWindowSize, void *wor= kspace, size_t workspaceSize) { ZSTD_customMem const stackMem =3D ZSTD_initStack(workspace, workspaceSize= ); ZSTD_DStream *zds =3D ZSTD_createDStream_advanced(stackMem); @@ -2249,6 +2268,7 @@ ZSTD_DStream *INIT ZSTD_initDStream(size return zds; } =20 +#ifdef BUILD_DEAD_CODE ZSTD_DStream *INIT ZSTD_initDStream_usingDDict(size_t maxWindowSize, const= ZSTD_DDict *ddict, void *workspace, size_t workspaceSize) { ZSTD_DStream *zds =3D ZSTD_initDStream(maxWindowSize, workspace, workspac= eSize); @@ -2257,6 +2277,7 @@ ZSTD_DStream *INIT ZSTD_initDStream_usin } return zds; } +#endif =20 size_t INIT ZSTD_freeDStream(ZSTD_DStream *zds) { @@ -2279,10 +2300,12 @@ size_t INIT ZSTD_freeDStream(ZSTD_DStrea =20 /* *** Initialization *** */ =20 +#ifdef BUILD_DEAD_CODE size_t INIT ZSTD_DStreamInSize(void) { return ZSTD_BLOCKSIZE_ABSOLUTEMAX += ZSTD_blockHeaderSize; } size_t INIT ZSTD_DStreamOutSize(void) { return ZSTD_BLOCKSIZE_ABSOLUTEMAX;= } +#endif =20 -size_t INIT ZSTD_resetDStream(ZSTD_DStream *zds) +STATIC size_t INIT ZSTD_resetDStream(ZSTD_DStream *zds) { zds->stage =3D zdss_loadHeader; zds->lhSize =3D zds->inPos =3D zds->outStart =3D zds->outEnd =3D 0; @@ -2300,7 +2323,7 @@ ZSTD_STATIC size_t INIT ZSTD_limitCopy(v return length; } =20 -size_t INIT ZSTD_decompressStream(ZSTD_DStream *zds, ZSTD_outBuffer *outpu= t, ZSTD_inBuffer *input) +STATIC size_t INIT ZSTD_decompressStream(ZSTD_DStream *zds, ZSTD_outBuffer= *output, ZSTD_inBuffer *input) { const char *const istart =3D (const char *)(input->src) + input->pos; const char *const iend =3D (const char *)(input->src) + input->size; --- a/xen/common/zstd/error_private.h +++ b/xen/common/zstd/error_private.h @@ -19,11 +19,6 @@ #ifndef ERROR_H_MODULE #define ERROR_H_MODULE =20 -/* **************************************** -* Dependencies -******************************************/ -#include /* size_t */ - /** * enum ZSTD_ErrorCode - zstd error codes * --- a/xen/common/zstd/fse.h +++ b/xen/common/zstd/fse.h @@ -41,11 +41,6 @@ #define FSE_H =20 /*-***************************************** -* Dependencies -******************************************/ -#include /* size_t, ptrdiff_t */ - -/*-***************************************** * FSE_PUBLIC_API : control library symbols visibility ******************************************/ #define FSE_PUBLIC_API --- a/xen/common/zstd/fse_decompress.c +++ b/xen/common/zstd/fse_decompress.c @@ -48,8 +48,6 @@ #include "bitstream.h" #include "fse.h" #include "zstd_internal.h" -#include -#include /* memcpy, memset */ =20 /* ************************************************************** * Error Management --- a/xen/common/zstd/huf.h +++ b/xen/common/zstd/huf.h @@ -40,9 +40,6 @@ #ifndef HUF_H_298734234 #define HUF_H_298734234 =20 -/* *** Dependencies *** */ -#include /* size_t */ - /* *** Tool functions *** */ #define HUF_BLOCKSIZE_MAX (128 * 1024) /**< maximum input size for a singl= e block compressed with HUF_compress */ size_t HUF_compressBound(size_t size); /**< maximum compressed size (worst= case) */ --- a/xen/common/zstd/huf_decompress.c +++ b/xen/common/zstd/huf_decompress.c @@ -48,8 +48,6 @@ #include "bitstream.h" /* BIT_* */ #include "fse.h" /* header compression */ #include "huf.h" -#include -#include /* memcpy, memset */ =20 /* ************************************************************** * Error Management --- a/xen/common/zstd/mem.h +++ b/xen/common/zstd/mem.h @@ -20,9 +20,11 @@ /*-**************************************** * Dependencies ******************************************/ +#ifdef __XEN__ #include /* memcpy */ #include /* size_t, ptrdiff_t */ #include +#endif =20 /*-**************************************** * Compiler specifics --- a/xen/common/zstd/zstd_internal.h +++ b/xen/common/zstd/zstd_internal.h @@ -28,8 +28,10 @@ ***************************************/ #include "error_private.h" #include "mem.h" +#ifdef __XEN__ #include #include +#endif =20 #define ALIGN(x, a) ((x + (a) - 1) & ~((a) - 1)) #define PTR_ALIGN(p, a) ((typeof(p))ALIGN((unsigned long)(p), (a))) @@ -95,8 +97,10 @@ typedef struct ZSTD_DStream_s ZSTD_DStre /*-************************************* * shared macros ***************************************/ +#ifndef MIN #define MIN(a, b) ((a) < (b) ? (a) : (b)) #define MAX(a, b) ((a) > (b) ? (a) : (b)) +#endif #define CHECK_F(f) \ { \ size_t const errcod =3D f; \ --- a/xen/include/xen/unaligned.h +++ b/xen/include/xen/unaligned.h @@ -10,8 +10,10 @@ #ifndef __XEN_UNALIGNED_H__ #define __XEN_UNALIGNED_H__ =20 +#ifdef __XEN__ #include #include +#endif =20 #define get_unaligned(p) (*(p)) #define put_unaligned(val, p) (*(p) =3D (val)) --- a/xen/lib/xxhash64.c +++ b/xen/lib/xxhash64.c @@ -38,11 +38,13 @@ * - xxHash source repository: https://github.com/Cyan4973/xxHash */ =20 +#ifdef __XEN__ #include #include #include #include #include +#endif =20 /*-************************************* * Macros From nobody Wed May 8 13:32:38 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1611069374; cv=none; d=zohomail.com; s=zohoarc; b=AoBNMfkdPtrMeFJ9mae6rM4BXFEhMehV/MY8YcjPS03xDnHkZKp2q7V++x67bQhJnTY+h8bz/UUhvXfoRvcbAZYY5fRLNU/hcCpoy4RK18ipT97kZjd4xqPF4FrGotl50nWkeMFvUb8rC1niWOh/IJvlOfndtUChQQx86eLbgdc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1611069374; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=EZpgyMc4DwSP/R3mbq7ta8DfhXOQugPMtfzDbMFjt/A=; b=ndwbKivaItlgMi38sbhLxEo5gGBhQRKukB0vPS0dEyfxpoNXk2DNDNE7/CVsLMtU2oaFokJN9jFV9jvREZTBRR59mZmZ4qmL9DjXXRLy2DLpo1ur+Rth5qkRfhLym0TDS31cVIasYXSe7fl8hOwP8PiEuc0IaAzwsJUpIxACCFo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1611069374374980.3976460451031; Tue, 19 Jan 2021 07:16:14 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.70479.126512 (Exim 4.92) (envelope-from ) id 1l1sjd-0004CS-N2; Tue, 19 Jan 2021 15:15:57 +0000 Received: by outflank-mailman (output) from mailman id 70479.126512; Tue, 19 Jan 2021 15:15:57 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l1sjd-0004CL-JC; Tue, 19 Jan 2021 15:15:57 +0000 Received: by outflank-mailman (input) for mailman id 70479; Tue, 19 Jan 2021 15:15:56 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l1sjc-0004CC-H1 for xen-devel@lists.xenproject.org; Tue, 19 Jan 2021 15:15:56 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id f84500c6-a716-48c8-ba64-667ac2c36f8b; Tue, 19 Jan 2021 15:15:55 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 4DE48AAAE; Tue, 19 Jan 2021 15:15:54 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f84500c6-a716-48c8-ba64-667ac2c36f8b X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1611069354; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EZpgyMc4DwSP/R3mbq7ta8DfhXOQugPMtfzDbMFjt/A=; b=AkpiedSxK0NSZNyyjeqmgIQLOPh6/Fk/DmCM7/pzaLNcXUeXXYRBTS9kdcyGbxUKfjf84i teFkgkwob+Pz6gxz/TR/vftyQn7GJ7vMHaUOp5J4OTaPXnfwWWJddUvcK6onU/+5y422Dl hmKzF7PYI6evkQeWSw0mVPhnGDz60Gk= Subject: [PATCH v2 2/5] xen/decompress: make helper symbols static From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Stefano Stabellini , Wei Liu References: Message-ID: <80b7e64d-02b3-55c6-1379-4ac9f77d3869@suse.com> Date: Tue, 19 Jan 2021 16:15:53 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" The individual decompression CUs need to only surface their top level functions to other code. Arrange for everything else to be static, to make sure no undue uses of that code exist or will appear without explicitly noticing. (In some cases this also results in code size reduction, but since this is all init-only code this probably doesn't matter very much.) In the LZO case also take the opportunity and convert u8 where lines get touched anyway. The downside is that the top level functions will now be non-static in stubdom builds of libxenguest, but I think that's acceptable. This does require declaring them first, though, as the compiler warns about the lack of declarations. Signed-off-by: Jan Beulich Acked-by: Andrew Cooper Acked-by: Wei Liu --- v2: Fix stubdom build. --- a/tools/libs/guest/xg_dom_decompress_unsafe.h +++ b/tools/libs/guest/xg_dom_decompress_unsafe.h @@ -1,8 +1,12 @@ +#ifdef __MINIOS__ +# include "../../xen/include/xen/decompress.h" +#else typedef int decompress_fn(unsigned char *inbuf, unsigned int len, int (*fill)(void*, unsigned int), int (*flush)(void*, unsigned int), unsigned char *outbuf, unsigned int *posp, void (*error)(const char *x)); +#endif =20 int xc_dom_decompress_unsafe( decompress_fn fn, struct xc_dom_image *dom, void **blob, size_t *size) --- a/xen/common/bunzip2.c +++ b/xen/common/bunzip2.c @@ -665,12 +665,11 @@ static int INIT start_bunzip(struct bunz =20 /* Example usage: decompress src_fd to dst_fd. (Stops at end of bzip2 dat= a, not end of file.) */ -STATIC int INIT bunzip2(unsigned char *buf, unsigned int len, - int(*fill)(void*, unsigned int), - int(*flush)(void*, unsigned int), - unsigned char *outbuf, - unsigned int *pos, - void(*error)(const char *x)) +int INIT bunzip2(unsigned char *buf, unsigned int len, + int(*fill)(void*, unsigned int), + int(*flush)(void*, unsigned int), + unsigned char *outbuf, unsigned int *pos, + void(*error)(const char *x)) { struct bunzip_data *bd; int i =3D -1; --- a/xen/common/decompress.h +++ b/xen/common/decompress.h @@ -7,7 +7,7 @@ #include #include =20 -#define STATIC +#define STATIC static #define INIT __init #define INITDATA __initdata =20 --- a/xen/common/unlz4.c +++ b/xen/common/unlz4.c @@ -22,12 +22,11 @@ #define LZ4_DEFAULT_UNCOMPRESSED_CHUNK_SIZE (8 << 20) #define ARCHIVE_MAGICNUMBER 0x184C2102 =20 -STATIC int INIT unlz4(unsigned char *input, unsigned int in_len, - int (*fill)(void *, unsigned int), - int (*flush)(void *, unsigned int), - unsigned char *output, - unsigned int *posp, - void (*error)(const char *x)) +int INIT unlz4(unsigned char *input, unsigned int in_len, + int (*fill)(void *, unsigned int), + int (*flush)(void *, unsigned int), + unsigned char *output, unsigned int *posp, + void (*error)(const char *x)) { int ret =3D -1; size_t chunksize =3D 0; --- a/xen/common/unlzma.c +++ b/xen/common/unlzma.c @@ -528,13 +528,11 @@ static inline int INIT process_bit1(stru =20 =20 =20 -STATIC int INIT unlzma(unsigned char *buf, unsigned int in_len, - int(*fill)(void*, unsigned int), - int(*flush)(void*, unsigned int), - unsigned char *output, - unsigned int *posp, - void(*error)(const char *x) - ) +int INIT unlzma(unsigned char *buf, unsigned int in_len, + int(*fill)(void*, unsigned int), + int(*flush)(void*, unsigned int), + unsigned char *output, unsigned int *posp, + void(*error)(const char *x)) { struct lzma_header header; int lc, pb, lp; --- a/xen/common/unlzo.c +++ b/xen/common/unlzo.c @@ -114,11 +114,11 @@ static int INIT parse_header(u8 *input, return 1; } =20 -STATIC int INIT unlzo(u8 *input, unsigned int in_len, - int (*fill) (void *, unsigned int), - int (*flush) (void *, unsigned int), - u8 *output, unsigned int *posp, - void (*error) (const char *x)) +int INIT unlzo(unsigned char *input, unsigned int in_len, + int (*fill) (void *, unsigned int), + int (*flush) (void *, unsigned int), + unsigned char *output, unsigned int *posp, + void (*error) (const char *x)) { u8 r =3D 0; int skip =3D 0; --- a/xen/common/unxz.c +++ b/xen/common/unxz.c @@ -157,11 +157,11 @@ * both input and output buffers are available as a single chunk, i.e. when * fill() and flush() won't be used. */ -STATIC int INIT unxz(unsigned char *in, unsigned int in_size, - int (*fill)(void *dest, unsigned int size), - int (*flush)(void *src, unsigned int size), - unsigned char *out, unsigned int *in_used, - void (*error)(const char *x)) +int INIT unxz(unsigned char *in, unsigned int in_size, + int (*fill)(void *dest, unsigned int size), + int (*flush)(void *src, unsigned int size), + unsigned char *out, unsigned int *in_used, + void (*error)(const char *x)) { struct xz_buf b; struct xz_dec *s; --- a/xen/common/unzstd.c +++ b/xen/common/unzstd.c @@ -142,12 +142,11 @@ out: return err; } =20 -STATIC int INIT unzstd(unsigned char *in_buf, unsigned int in_len, - int (*fill)(void*, unsigned int), - int (*flush)(void*, unsigned int), - unsigned char *out_buf, - unsigned int *in_pos, - void (*error)(const char *x)) +int INIT unzstd(unsigned char *in_buf, unsigned int in_len, + int (*fill)(void*, unsigned int), + int (*flush)(void*, unsigned int), + unsigned char *out_buf, unsigned int *in_pos, + void (*error)(const char *x)) { ZSTD_inBuffer in; ZSTD_outBuffer out; From nobody Wed May 8 13:32:38 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1611069417; cv=none; d=zohomail.com; s=zohoarc; b=CFgFtVCNV3Ixw7plS4VSZsoyCUGP/TKapJlIUHLwxz0nQqkY9b54pT1XPnqrLV04+1fV4CZTYD/VtKjBfgFGCXsDbb/ee3AyNnc7Kx/5AJGyL+1y+wIy+2580sf/S+EVosT0gcDoxzx4Jvy72xXg6DxfqT/tLAe+f1wI+Qub/p4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1611069417; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=XzQQDvN2oOpxpLvfBkCEem0nzWZsBYK3ZsdptFa+w+4=; b=NDpzlKBi/80jrChAVUWWA4Xk0FTl39LTkW6HcBGbqsC256B0z1uDy1pvxNfyeMRwAFTh/tDOi6qMo97bhWl0xavLix44GWJhow/ih0cT1Yw1VwLn9esol+X3bI18RHXva0/qK4h+uuMszCXc2gPTjV7o3SfQ3OSPzXwz3gWQ2sw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1611069417590832.2224645884938; Tue, 19 Jan 2021 07:16:57 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.70483.126524 (Exim 4.92) (envelope-from ) id 1l1skJ-0004Jm-0g; Tue, 19 Jan 2021 15:16:39 +0000 Received: by outflank-mailman (output) from mailman id 70483.126524; Tue, 19 Jan 2021 15:16:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l1skI-0004Jf-TU; Tue, 19 Jan 2021 15:16:38 +0000 Received: by outflank-mailman (input) for mailman id 70483; Tue, 19 Jan 2021 15:16:37 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l1skH-0004JW-EF for xen-devel@lists.xenproject.org; Tue, 19 Jan 2021 15:16:37 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 1eb8ccf4-b007-45e3-85a5-a54262a868fc; Tue, 19 Jan 2021 15:16:36 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id A92DAB7BB; Tue, 19 Jan 2021 15:16:35 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1eb8ccf4-b007-45e3-85a5-a54262a868fc X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1611069395; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XzQQDvN2oOpxpLvfBkCEem0nzWZsBYK3ZsdptFa+w+4=; b=RvMlnR4RAjkVMh5shIXBPI5mLQJesN+Oa7NtpuyHo8pQavCKjiL53kxy/GqSO9MgQhaw4c Y1xiNSZFOBAZtgo7m+rak3FC1tvw8hg25NWGNRg1ZzUFrLKW+iWeKpKINCnIyj7psMSp4X di72f3E3bqREdDdJQL8Ku/bBYNbtv4A= Subject: [PATCH v2 3/5] libxenguest: "standardize" LZO kernel decompression code From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Stefano Stabellini , Wei Liu References: Message-ID: <4fb01879-4b9c-e711-ecd0-39b0e4d50878@suse.com> Date: Tue, 19 Jan 2021 16:16:35 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" Add a DOMPRINTF() other methods have, indicating success. To facilitate this, introduce an "outsize" local variable and update *size as well as *blob only once done. The latter then also avoids leaving a pointer to freed memory in dom->kernel_blob in case of a decompression error. Signed-off-by: Jan Beulich Acked-by: Wei Liu --- v2: New. --- a/tools/libs/guest/xg_dom_bzimageloader.c +++ b/tools/libs/guest/xg_dom_bzimageloader.c @@ -409,7 +409,7 @@ static int xc_try_lzo1x_decode( int ret; const unsigned char *cur =3D dom->kernel_blob; unsigned char *out_buf =3D NULL; - size_t left =3D dom->kernel_size; + size_t left =3D dom->kernel_size, outsize; const char *msg; unsigned version; static const unsigned char magic[] =3D { @@ -471,7 +471,7 @@ static int xc_try_lzo1x_decode( cur +=3D ret; left -=3D ret; =20 - for ( *size =3D 0; ; ) + for ( outsize =3D 0; ; ) { lzo_uint src_len, dst_len, out_len; unsigned char *tmp_buf; @@ -484,9 +484,15 @@ static int xc_try_lzo1x_decode( if ( !dst_len ) { msg =3D "Error registering stream output"; - if ( xc_dom_register_external(dom, out_buf, *size) ) + if ( xc_dom_register_external(dom, out_buf, outsize) ) break; =20 + DOMPRINTF("%s: LZO decompress OK, 0x%zx -> 0x%zx", + __FUNCTION__, *size, outsize); + + *blob =3D out_buf; + *size =3D outsize; + return 0; } =20 @@ -508,15 +514,15 @@ static int xc_try_lzo1x_decode( break; =20 msg =3D "Output buffer overflow"; - if ( *size > SIZE_MAX - dst_len ) + if ( outsize > SIZE_MAX - dst_len ) break; =20 msg =3D "Decompressed image too large"; - if ( xc_dom_kernel_check_size(dom, *size + dst_len) ) + if ( xc_dom_kernel_check_size(dom, outsize + dst_len) ) break; =20 msg =3D "Failed to (re)alloc memory"; - tmp_buf =3D realloc(out_buf, *size + dst_len); + tmp_buf =3D realloc(out_buf, outsize + dst_len); if ( tmp_buf =3D=3D NULL ) break; =20 @@ -524,7 +530,7 @@ static int xc_try_lzo1x_decode( out_len =3D dst_len; =20 ret =3D lzo1x_decompress_safe(cur, src_len, - out_buf + *size, &out_len, NULL); + out_buf + outsize, &out_len, NULL); switch ( ret ) { case LZO_E_OK: @@ -532,8 +538,7 @@ static int xc_try_lzo1x_decode( if ( out_len !=3D dst_len ) break; =20 - *blob =3D out_buf; - *size +=3D out_len; + outsize +=3D out_len; cur +=3D src_len; left -=3D src_len; continue; From nobody Wed May 8 13:32:38 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1611069431; cv=none; d=zohomail.com; s=zohoarc; b=a9bwN0EOBGu27d3rLRYPisg2WLcUjeyL9tm0YRpqxOQa3MEOLudhXUmh5BDyJlHz9e0/e2027F/Oj3oPQoQYqQsEKFXHhlN/hnEWBILFb66abpySmS2TB/EtMzfTdcVDLSA0K9ofBit+orNK32zUfcbw70lmWZthK9qy9Ok2S6k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1611069431; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=kJ7Pv7Jlp4SzB+Vj6j3PgkJrnOZngDatZK/keb5brzA=; b=PSXRdGQEK7w3+QCkoKMXdZ+GDkyBE3ZqX68cVgIkZ1Cy1uT7KamrpQufpyhDYpuNOJX1CaMjNJTXjW0bhdaAd2pubUB67SJWQ+givvWKsr6Q+FzqsyIZ3Svb7n3Z+ah0tN8tEN0RmgahSN4tiYurmGEbpYq1y/fCd8n7Z/A6w0g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1611069431885969.4752518529992; Tue, 19 Jan 2021 07:17:11 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.70484.126536 (Exim 4.92) (envelope-from ) id 1l1skb-0004OX-8v; Tue, 19 Jan 2021 15:16:57 +0000 Received: by outflank-mailman (output) from mailman id 70484.126536; Tue, 19 Jan 2021 15:16:57 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l1skb-0004OP-5b; Tue, 19 Jan 2021 15:16:57 +0000 Received: by outflank-mailman (input) for mailman id 70484; Tue, 19 Jan 2021 15:16:55 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l1skZ-0004O9-G3 for xen-devel@lists.xenproject.org; Tue, 19 Jan 2021 15:16:55 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d7cb3896-b73a-4eff-b42d-a121aeae42ce; Tue, 19 Jan 2021 15:16:54 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id D0580B28A; Tue, 19 Jan 2021 15:16:53 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d7cb3896-b73a-4eff-b42d-a121aeae42ce X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1611069414; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kJ7Pv7Jlp4SzB+Vj6j3PgkJrnOZngDatZK/keb5brzA=; b=aESMWS1bCx1OzsfWZ32pOll2X7pfJjyaEXPeVFxov0T+WkPjlIHWxC+j3JFcDCIDkvSJl4 wYxDLNzrEHYB2bGYCd3XR3nldCCt5XD63CuZZzOl4UbB/KVfl9qALhHZzHwajMQdDOYiwC 75omETs/Efqk8gUm12YbvARk6xg9cfs= Subject: [PATCH v2 4/5] libxenguest: drop redundant decompression declarations From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Stefano Stabellini , Wei Liu References: Message-ID: <17147e8a-2c5d-ac96-db16-8db4f4e87bbd@suse.com> Date: Tue, 19 Jan 2021 16:16:53 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" The ones in xg_dom_decompress_unsafe.h suffice. Signed-off-by: Jan Beulich Acked-by: Wei Liu --- v2: New. --- a/tools/libs/guest/xg_dom_bzimageloader.c +++ b/tools/libs/guest/xg_dom_bzimageloader.c @@ -673,13 +673,6 @@ static int xc_try_zstd_decode( =20 #endif =20 -#else /* __MINIOS__ */ - -int xc_try_bzip2_decode(struct xc_dom_image *dom, void **blob, size_t *siz= e); -int xc_try_lzma_decode(struct xc_dom_image *dom, void **blob, size_t *size= ); -int xc_try_lzo1x_decode(struct xc_dom_image *dom, void **blob, size_t *siz= e); -int xc_try_xz_decode(struct xc_dom_image *dom, void **blob, size_t *size); - #endif /* !__MINIOS__ */ =20 struct setup_header { From nobody Wed May 8 13:32:38 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1611069453; cv=none; d=zohomail.com; s=zohoarc; b=mysPCwCSuiIJ7Au4nVNwtn/Yt31Rx3HojYeyzd8a4bsvlNzmP3h5FtNNxUsxG7Rnl4vsvKEb/Y399I7Lb/9BkagTYmCNdid28wZS/wO4n+/LDnRz1l8pTOjwso0wej+mtgyurhLb9v7TZabjzDpxswKWf+GA9ACT9mYMHCuIp8A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1611069453; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=GzQ/TMLxOp954HatIYsp8zXkUc5E4Inz83HkcZgnzwg=; b=M9YNo5X1aMLakSGeeuzoZm18VePN4HMJN5Pgc3jEIttCe+YwS8YW30fCu77zYFNgJCR3N6A2lIvl2lHxSRfrlhEgeEKwko3i02FjZ0tbx6Gx4+GjRze4fBSXO1KWRd/CdGrDwY1wkQsnjYXv9PQ0dmScUZBIdWUcK8+Jc/A7Xfk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1611069453044639.3844735238839; Tue, 19 Jan 2021 07:17:33 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.70488.126547 (Exim 4.92) (envelope-from ) id 1l1sku-0004VR-I1; Tue, 19 Jan 2021 15:17:16 +0000 Received: by outflank-mailman (output) from mailman id 70488.126547; Tue, 19 Jan 2021 15:17:16 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l1sku-0004VK-Ez; Tue, 19 Jan 2021 15:17:16 +0000 Received: by outflank-mailman (input) for mailman id 70488; Tue, 19 Jan 2021 15:17:14 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1l1sks-0004Uw-Pr for xen-devel@lists.xenproject.org; Tue, 19 Jan 2021 15:17:14 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 01b5de42-3fdd-4bab-9cd0-7807e18462aa; Tue, 19 Jan 2021 15:17:13 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 74A25ADD6; Tue, 19 Jan 2021 15:17:12 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 01b5de42-3fdd-4bab-9cd0-7807e18462aa X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1611069432; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GzQ/TMLxOp954HatIYsp8zXkUc5E4Inz83HkcZgnzwg=; b=BUR1zZUKk4eezMmxuGDQoUbMmlnmoobycDRIxFqQGz99LBXpX+eI+IaaiDV7mxR17BqR8s ZVmCZIEHcgBJ+T26Z01nHhzBZRKdo+9VIit+Kz10+tAVtAsjrUSE47lmGGG3d6RCMhUZqx nxcPuInS9wkE3Ygt+bBWxot6hW7r/p4= Subject: [PATCH v2 5/5] libxenguest: simplify kernel decompression From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , George Dunlap , Ian Jackson , Julien Grall , Stefano Stabellini , Wei Liu References: Message-ID: Date: Tue, 19 Jan 2021 16:17:12 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" In all cases the kernel build makes available the uncompressed size in the final 4 bytes of the bzImage payload. Utilize this to avoid repeated realloc()ing of the output buffer. As a side effect this also addresses the previous mistaken return of 0 (success) from xc_try_{bzip2,lzma,xz}_decode() in case xc_dom_register_external() would have failed. As another side effect this also addresses the first error path of _xc_try_lzma_decode() previously bypassing lzma_end(). Signed-off-by: Jan Beulich Acked-by: Wei Liu --- v2: New. --- a/tools/libs/guest/xg_dom_bzimageloader.c +++ b/tools/libs/guest/xg_dom_bzimageloader.c @@ -48,18 +48,16 @@ static int xc_try_bzip2_decode( bz_stream stream; int ret; char *out_buf; - char *tmp_buf; int retval =3D -1; - unsigned int outsize; - uint64_t total; + unsigned int insize, outsize; =20 stream.bzalloc =3D NULL; stream.bzfree =3D NULL; stream.opaque =3D NULL; =20 - if ( dom->kernel_size =3D=3D 0) + if ( *size <=3D 8 ) { - DOMPRINTF("BZIP2: Input is 0 size"); + DOMPRINTF("BZIP2: insufficient input data"); return -1; } =20 @@ -70,22 +68,25 @@ static int xc_try_bzip2_decode( return -1; } =20 - /* sigh. We don't know up-front how much memory we are going to need - * for the output buffer. Allocate the output buffer to be equal - * the input buffer to start, and we'll realloc as needed. - */ - outsize =3D dom->kernel_size; + insize =3D *size - 4; + outsize =3D *(uint32_t *)(*blob + insize); =20 /* - * stream.avail_in and outsize are unsigned int, while kernel_size + * stream.avail_in and insize are unsigned int, while *size * is a size_t. Check we aren't overflowing. */ - if ( outsize !=3D dom->kernel_size ) + if ( insize + 4 !=3D *size ) { DOMPRINTF("BZIP2: Input too large"); goto bzip2_cleanup; } =20 + if ( xc_dom_kernel_check_size(dom, outsize) ) + { + DOMPRINTF("BZIP2: output too large"); + goto bzip2_cleanup; + } + out_buf =3D malloc(outsize); if ( out_buf =3D=3D NULL ) { @@ -94,86 +95,45 @@ static int xc_try_bzip2_decode( } =20 stream.next_in =3D dom->kernel_blob; - stream.avail_in =3D dom->kernel_size; + stream.avail_in =3D insize; =20 stream.next_out =3D out_buf; - stream.avail_out =3D dom->kernel_size; + stream.avail_out =3D outsize; =20 - for ( ; ; ) + ret =3D BZ2_bzDecompress(&stream); + if ( ret =3D=3D BZ_STREAM_END ) + DOMPRINTF("BZIP2: Saw data stream end"); + else if ( ret !=3D BZ_OK ) { - ret =3D BZ2_bzDecompress(&stream); - if ( ret =3D=3D BZ_STREAM_END ) - { - DOMPRINTF("BZIP2: Saw data stream end"); - retval =3D 0; - break; - } - if ( ret !=3D BZ_OK ) - { - DOMPRINTF("BZIP2: error %d", ret); - free(out_buf); - goto bzip2_cleanup; - } + DOMPRINTF("BZIP2: error %d", ret); + free(out_buf); + goto bzip2_cleanup; + } =20 - if ( stream.avail_out =3D=3D 0 ) - { - /* Protect against output buffer overflow */ - if ( outsize > UINT_MAX / 2 ) - { - DOMPRINTF("BZIP2: output buffer overflow"); - free(out_buf); - goto bzip2_cleanup; - } - - if ( xc_dom_kernel_check_size(dom, outsize * 2) ) - { - DOMPRINTF("BZIP2: output too large"); - free(out_buf); - goto bzip2_cleanup; - } - - tmp_buf =3D realloc(out_buf, outsize * 2); - if ( tmp_buf =3D=3D NULL ) - { - DOMPRINTF("BZIP2: Failed to realloc memory"); - free(out_buf); - goto bzip2_cleanup; - } - out_buf =3D tmp_buf; - - stream.next_out =3D out_buf + outsize; - stream.avail_out =3D (outsize * 2) - outsize; - outsize *=3D 2; - } - else if ( stream.avail_in =3D=3D 0 ) - { - /* - * If there is output buffer available then this indicates - * that BZ2_bzDecompress would like more input data to be - * provided. However our complete input buffer is in - * memory and provided upfront so if avail_in is zero this - * actually indicates a truncated input. - */ - DOMPRINTF("BZIP2: not enough input"); - free(out_buf); - goto bzip2_cleanup; - } + if ( stream.total_out_lo32 !=3D outsize || stream.total_out_hi32 ) + { + DOMPRINTF("BZIP2: got 0x%x%08x bytes instead of 0x%09x", + stream.total_out_hi32, stream.total_out_lo32, outsize); + free(out_buf); + goto bzip2_cleanup; } =20 - total =3D (((uint64_t)stream.total_out_hi32) << 32) | stream.total_out= _lo32; + if ( stream.avail_in ) + DOMPRINTF("BZIP2: Warning: %#x unconsumed bytes", stream.avail_in); =20 - if ( xc_dom_register_external(dom, out_buf, total) ) + if ( xc_dom_register_external(dom, out_buf, outsize) ) { DOMPRINTF("BZIP2: Error registering stream output"); free(out_buf); goto bzip2_cleanup; } =20 - DOMPRINTF("%s: BZIP2 decompress OK, 0x%zx -> 0x%lx", - __FUNCTION__, *size, (long unsigned int) total); + DOMPRINTF("%s: BZIP2 decompress OK, 0x%zx -> 0x%x", + __FUNCTION__, *size, outsize); =20 *blob =3D out_buf; - *size =3D total; + *size =3D outsize; + retval =3D 0; =20 bzip2_cleanup: BZ2_bzDecompressEnd(&stream); @@ -205,22 +165,24 @@ static int _xc_try_lzma_decode( lzma_ret ret; lzma_action action =3D LZMA_RUN; unsigned char *out_buf; - unsigned char *tmp_buf; int retval =3D -1; - size_t outsize; - const char *msg; + size_t insize, outsize; =20 - if ( dom->kernel_size =3D=3D 0) + if ( *size < 8 ) { - DOMPRINTF("%s: Input is 0 size", what); - return -1; + DOMPRINTF("%s: insufficient input data", what); + goto lzma_cleanup; + } + + insize =3D *size - 4; + outsize =3D *(uint32_t *)(*blob + insize); + + if ( xc_dom_kernel_check_size(dom, outsize) ) + { + DOMPRINTF("%s: output too large", what); + goto lzma_cleanup; } =20 - /* sigh. We don't know up-front how much memory we are going to need - * for the output buffer. Allocate the output buffer to be equal - * the input buffer to start, and we'll realloc as needed. - */ - outsize =3D dom->kernel_size; out_buf =3D malloc(outsize); if ( out_buf =3D=3D NULL ) { @@ -229,92 +191,68 @@ static int _xc_try_lzma_decode( } =20 stream->next_in =3D dom->kernel_blob; - stream->avail_in =3D dom->kernel_size; + stream->avail_in =3D insize; =20 stream->next_out =3D out_buf; - stream->avail_out =3D dom->kernel_size; + stream->avail_out =3D outsize; =20 - for ( ; ; ) + ret =3D lzma_code(stream, action); + if ( ret =3D=3D LZMA_STREAM_END ) + DOMPRINTF("%s: Saw data stream end", what); + else if ( ret !=3D LZMA_OK ) { - ret =3D lzma_code(stream, action); - if ( ret =3D=3D LZMA_STREAM_END ) + const char *msg; + + switch ( ret ) { - DOMPRINTF("%s: Saw data stream end", what); - retval =3D 0; + case LZMA_MEM_ERROR: + msg =3D strerror(ENOMEM); break; - } - if ( ret !=3D LZMA_OK ) - { - switch ( ret ) - { - case LZMA_MEM_ERROR: - msg =3D strerror(ENOMEM); - break; =20 - case LZMA_MEMLIMIT_ERROR: - msg =3D "Memory usage limit reached"; - break; + case LZMA_MEMLIMIT_ERROR: + msg =3D "Memory usage limit reached"; + break; =20 - case LZMA_FORMAT_ERROR: - msg =3D "File format not recognized"; - break; + case LZMA_FORMAT_ERROR: + msg =3D "File format not recognized"; + break; =20 - case LZMA_OPTIONS_ERROR: - // FIXME: Better message? - msg =3D "Unsupported compression options"; - break; + case LZMA_OPTIONS_ERROR: + // FIXME: Better message? + msg =3D "Unsupported compression options"; + break; =20 - case LZMA_DATA_ERROR: - msg =3D "File is corrupt"; - break; + case LZMA_DATA_ERROR: + msg =3D "File is corrupt"; + break; =20 - case LZMA_BUF_ERROR: - msg =3D "Unexpected end of input"; - break; + case LZMA_BUF_ERROR: + msg =3D "Unexpected end of input"; + break; =20 - default: - msg =3D "Internal program error (bug)"; - break; - } - DOMPRINTF("%s: %s decompression error: %s", - __FUNCTION__, what, msg); - free(out_buf); - goto lzma_cleanup; + default: + msg =3D "Internal program error (bug)"; + break; } =20 - if ( stream->avail_out =3D=3D 0 ) - { - /* Protect against output buffer overflow */ - if ( outsize > SIZE_MAX / 2 ) - { - DOMPRINTF("%s: output buffer overflow", what); - free(out_buf); - goto lzma_cleanup; - } - - if ( xc_dom_kernel_check_size(dom, outsize * 2) ) - { - DOMPRINTF("%s: output too large", what); - free(out_buf); - goto lzma_cleanup; - } - - tmp_buf =3D realloc(out_buf, outsize * 2); - if ( tmp_buf =3D=3D NULL ) - { - DOMPRINTF("%s: Failed to realloc memory", what); - free(out_buf); - goto lzma_cleanup; - } - out_buf =3D tmp_buf; - - stream->next_out =3D out_buf + outsize; - stream->avail_out =3D (outsize * 2) - outsize; - outsize *=3D 2; - } + DOMPRINTF("%s: %s decompression error: %s", + __FUNCTION__, what, msg); + free(out_buf); + goto lzma_cleanup; + } + + if ( stream->total_out !=3D outsize ) + { + DOMPRINTF("%s: got 0x%"PRIx64" bytes instead of 0x%zx", + what, stream->total_out, outsize); + free(out_buf); + goto lzma_cleanup; } =20 - if ( xc_dom_register_external(dom, out_buf, stream->total_out) ) + if ( stream->avail_in ) + DOMPRINTF("%s: Warning: %#zx unconsumed bytes", what, stream->avai= l_in); + + if ( xc_dom_register_external(dom, out_buf, outsize) ) { DOMPRINTF("%s: Error registering stream output", what); free(out_buf); @@ -322,10 +260,11 @@ static int _xc_try_lzma_decode( } =20 DOMPRINTF("%s: %s decompress OK, 0x%zx -> 0x%zx", - __FUNCTION__, what, *size, (size_t)stream->total_out); + __FUNCTION__, what, *size, outsize); =20 *blob =3D out_buf; - *size =3D stream->total_out; + *size =3D outsize; + retval =3D 0; =20 lzma_cleanup: lzma_end(stream); @@ -408,8 +347,8 @@ static int xc_try_lzo1x_decode( { int ret; const unsigned char *cur =3D dom->kernel_blob; - unsigned char *out_buf =3D NULL; - size_t left =3D dom->kernel_size, outsize; + unsigned char *out_buf; + size_t left =3D dom->kernel_size, outsize, outtot; const char *msg; unsigned version; static const unsigned char magic[] =3D { @@ -435,6 +374,15 @@ static int xc_try_lzo1x_decode( return -1; } =20 + left -=3D 4; + outtot =3D *(uint32_t *)(*blob + left); + + if ( xc_dom_kernel_check_size(dom, outtot) ) + { + DOMPRINTF("LZO1x: output too large"); + return -1; + } + /* get version (2bytes), skip library version (2), * 'need to be extracted' version (2) and method (1) */ version =3D lzo_read_16(cur + 9); @@ -471,10 +419,16 @@ static int xc_try_lzo1x_decode( cur +=3D ret; left -=3D ret; =20 + out_buf =3D malloc(outtot); + if ( !out_buf ) + { + DOMPRINTF("LZO1x: failed to alloc memory"); + return -1; + } + for ( outsize =3D 0; ; ) { lzo_uint src_len, dst_len, out_len; - unsigned char *tmp_buf; =20 msg =3D "Short input"; if ( left < 4 ) @@ -483,6 +437,13 @@ static int xc_try_lzo1x_decode( dst_len =3D lzo_read_32(cur); if ( !dst_len ) { + msg =3D "Unexpected output size"; + if ( outsize !=3D outtot ) + break; + + if ( left !=3D 4 ) + DOMPRINTF("LZO1x: Warning: %#zx unconsumed bytes", left - = 4); + msg =3D "Error registering stream output"; if ( xc_dom_register_external(dom, out_buf, outsize) ) break; @@ -514,19 +475,9 @@ static int xc_try_lzo1x_decode( break; =20 msg =3D "Output buffer overflow"; - if ( outsize > SIZE_MAX - dst_len ) - break; - - msg =3D "Decompressed image too large"; - if ( xc_dom_kernel_check_size(dom, outsize + dst_len) ) - break; - - msg =3D "Failed to (re)alloc memory"; - tmp_buf =3D realloc(out_buf, outsize + dst_len); - if ( tmp_buf =3D=3D NULL ) + if ( dst_len > outtot - outsize ) break; =20 - out_buf =3D tmp_buf; out_len =3D dst_len; =20 ret =3D lzo1x_decompress_safe(cur, src_len,