From nobody Mon Feb 9 08:50:53 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+88443+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+88443+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1649193930; cv=none; d=zohomail.com; s=zohoarc; b=SOdPOgLKfg3HUinLtOIOjL35GZEY5iuBMZrz3tXWBQRdIfzEcTADueDKXZlHkYnRbnhWC0kKVGBv6511HC46R2RoLx3VT6gQbnjPLC1a+xkWbzeTqJbEbbhjmLJ1vx3DvAb96TQSccUSjvf2Rl2Nv4RfmJDWGXiWdzfwh1avy7Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1649193930; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Sender:Subject:To; bh=5dU9hpJ3+PrBiAimDw57f6uy0NhO3k3bNgNbXmhxiJU=; b=gtjoWvoJUxs07uU83stfnoZe6ooOf2PrctAewBW0e04K0ghNpXhv62+DWUFWVUVjJ/0E7z3O+exMgXG4HKSJ8x1ReO2G8AcdEoCSbJhQ2ZkZnnZrt//GEAups8R9EU9VIYDvejt8LHldr9M4EW2GmnJS2IscI6GqZmEcPRfMC7c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+88443+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1649193930000382.935200230728; Tue, 5 Apr 2022 14:25:30 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id yz7nYY1788612x8Swl133R9G; Tue, 05 Apr 2022 14:25:29 -0700 X-Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by mx.groups.io with SMTP id smtpd.web10.133.1649188825827398065 for ; Tue, 05 Apr 2022 13:00:26 -0700 X-IronPort-AV: E=McAfee;i="6200,9189,10308"; a="241438054" X-IronPort-AV: E=Sophos;i="5.90,238,1643702400"; d="scan'208";a="241438054" X-Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Apr 2022 13:00:31 -0700 X-IronPort-AV: E=Sophos;i="5.90,238,1643702400"; d="scan'208";a="570061651" X-Received: from plxcad1029.pdx.intel.com ([10.39.237.164]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Apr 2022 13:00:30 -0700 From: artem.khakimov@intel.com To: devel@edk2.groups.io Cc: Artem Khakimov , Bob Feng , Olga Artemeva , Vadim Mikheev , Alexander Larionov Subject: [edk2-devel] [PATCH] BaseTools: Add python version of GenCrc32 Date: Tue, 5 Apr 2022 12:59:31 -0700 Message-Id: <86c94d6a11c45be6173bfd185d87158e08569746.1649188308.git.artem.khakimov@intel.com> MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,artem.khakimov@intel.com X-Gm-Message-State: o5lGCjRJ9WkWdL8jRjfDguphx1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1649193929; bh=xgwfzm1JRt0usmG6zF9tTSUOKsWkqcNrZj46J0tSoNo=; h=Cc:Content-Type:Date:From:Reply-To:Subject:To; b=NIAKEpmR6nA0Ql7s7biqeu6NSqTSnmFFl/zVzl15GlHuOdE1CJYH6oN776qpdvWCwBA EhnmfvQKTs9GF1Izv+PTPjvSP6/qD3FL3vqqOnBWmJukR925/fezyXgRpVNWsNb5Uod1O n0ej4c8d459YcxgLqK9ol5znIewJ5sTdsyw= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1649193930867100002 Content-Type: text/plain; charset="utf-8" From: Artem Khakimov Add GenCrc32 python tool, add tests, switch BinWrappers and BinPipWrappers to GenCrc32.py Cc: Bob Feng Cc: Olga Artemeva Cc: Vadim Mikheev Cc: Alexander Larionov Signed-off-by: Artem Khakimov --- BaseTools/BinPipWrappers/PosixLike/GenCrc32 | 41 ++-- .../BinPipWrappers/WindowsLike/GenCrc32.bat | 3 + BaseTools/BinWrappers/PosixLike/GenCrc32 | 43 ++-- .../BinWrappers/WindowsLike/GenCrc32.bat | 4 + BaseTools/Source/Python/GenCrc32/GenCrc32.py | 185 ++++++++++++++++++ BaseTools/Tests/PythonToolsTests.py | 2 + BaseTools/Tests/TestGenCrc32.py | 64 ++++++ 7 files changed, 284 insertions(+), 58 deletions(-) create mode 100644 BaseTools/BinPipWrappers/WindowsLike/GenCrc32.bat create mode 100644 BaseTools/BinWrappers/WindowsLike/GenCrc32.bat create mode 100644 BaseTools/Source/Python/GenCrc32/GenCrc32.py create mode 100644 BaseTools/Tests/TestGenCrc32.py diff --git a/BaseTools/BinPipWrappers/PosixLike/GenCrc32 b/BaseTools/BinPip= Wrappers/PosixLike/GenCrc32 index 0945d86d92..14e8d8af71 100755 --- a/BaseTools/BinPipWrappers/PosixLike/GenCrc32 +++ b/BaseTools/BinPipWrappers/PosixLike/GenCrc32 @@ -1,29 +1,12 @@ -#!/usr/bin/env bash - -full_cmd=3D${BASH_SOURCE:-$0} # see http://mywiki.wooledge.org/BashFAQ/028= for a discussion of why $0 is not a good choice here -dir=3D$(dirname "$full_cmd") -cmd=3D${full_cmd##*/} - -if [ -n "$WORKSPACE" ] && [ -e "$WORKSPACE/Conf/BaseToolsCBinaries" ] -then - exec "$WORKSPACE/Conf/BaseToolsCBinaries/$cmd" -elif [ -n "$WORKSPACE" ] && [ -e "$EDK_TOOLS_PATH/Source/C" ] -then - if [ ! -e "$EDK_TOOLS_PATH/Source/C/bin/$cmd" ] - then - echo "BaseTools C Tool binary was not found ($cmd)" - echo "You may need to run:" - echo " make -C $EDK_TOOLS_PATH/Source/C" - else - exec "$EDK_TOOLS_PATH/Source/C/bin/$cmd" "$@" - fi -elif [ -e "$dir/../../Source/C/bin/$cmd" ] -then - exec "$dir/../../Source/C/bin/$cmd" "$@" -else - echo "Unable to find the real '$cmd' to run" - echo "This message was printed by" - echo " $0" - exit 127 -fi - +#!/usr/bin/env bash +#python `dirname $0`/RunToolFromSource.py `basename $0` $* + +# If a ${PYTHON_COMMAND} command is available, use it in preference to pyt= hon +if command -v ${PYTHON_COMMAND} >/dev/null 2>&1; then + python_exe=3D${PYTHON_COMMAND} +fi + +full_cmd=3D${BASH_SOURCE:-$0} # see http://mywiki.wooledge.org/BashFAQ/028= for a discussion of why $0 is not a good choice here +cmd=3D${full_cmd##*/} + +exec "${python_exe:-python}" -m edk2basetools.$cmd "$@" diff --git a/BaseTools/BinPipWrappers/WindowsLike/GenCrc32.bat b/BaseTools/= BinPipWrappers/WindowsLike/GenCrc32.bat new file mode 100644 index 0000000000..d347d64844 --- /dev/null +++ b/BaseTools/BinPipWrappers/WindowsLike/GenCrc32.bat @@ -0,0 +1,3 @@ +@setlocal +@set ToolName=3D%~n0% +@%PYTHON_COMMAND% -m edk2basetools.%ToolName%.%ToolName% %* diff --git a/BaseTools/BinWrappers/PosixLike/GenCrc32 b/BaseTools/BinWrappe= rs/PosixLike/GenCrc32 index 0945d86d92..39483a9869 100755 --- a/BaseTools/BinWrappers/PosixLike/GenCrc32 +++ b/BaseTools/BinWrappers/PosixLike/GenCrc32 @@ -1,29 +1,14 @@ -#!/usr/bin/env bash - -full_cmd=3D${BASH_SOURCE:-$0} # see http://mywiki.wooledge.org/BashFAQ/028= for a discussion of why $0 is not a good choice here -dir=3D$(dirname "$full_cmd") -cmd=3D${full_cmd##*/} - -if [ -n "$WORKSPACE" ] && [ -e "$WORKSPACE/Conf/BaseToolsCBinaries" ] -then - exec "$WORKSPACE/Conf/BaseToolsCBinaries/$cmd" -elif [ -n "$WORKSPACE" ] && [ -e "$EDK_TOOLS_PATH/Source/C" ] -then - if [ ! -e "$EDK_TOOLS_PATH/Source/C/bin/$cmd" ] - then - echo "BaseTools C Tool binary was not found ($cmd)" - echo "You may need to run:" - echo " make -C $EDK_TOOLS_PATH/Source/C" - else - exec "$EDK_TOOLS_PATH/Source/C/bin/$cmd" "$@" - fi -elif [ -e "$dir/../../Source/C/bin/$cmd" ] -then - exec "$dir/../../Source/C/bin/$cmd" "$@" -else - echo "Unable to find the real '$cmd' to run" - echo "This message was printed by" - echo " $0" - exit 127 -fi - +#!/usr/bin/env bash +#python `dirname $0`/RunToolFromSource.py `basename $0` $* + +# If a ${PYTHON_COMMAND} command is available, use it in preference to pyt= hon +if command -v ${PYTHON_COMMAND} >/dev/null 2>&1; then + python_exe=3D${PYTHON_COMMAND} +fi + +full_cmd=3D${BASH_SOURCE:-$0} # see http://mywiki.wooledge.org/BashFAQ/028= for a discussion of why $0 is not a good choice here +dir=3D$(dirname "$full_cmd") +cmd=3D${full_cmd##*/} + +export PYTHONPATH=3D"$dir/../../Source/Python${PYTHONPATH:+:"$PYTHONPATH"}" +exec "${python_exe:-python}" -m $cmd.$cmd "$@" diff --git a/BaseTools/BinWrappers/WindowsLike/GenCrc32.bat b/BaseTools/Bin= Wrappers/WindowsLike/GenCrc32.bat new file mode 100644 index 0000000000..f43dba81f1 --- /dev/null +++ b/BaseTools/BinWrappers/WindowsLike/GenCrc32.bat @@ -0,0 +1,4 @@ +@setlocal +@set ToolName=3D%~n0% +@set PYTHONPATH=3D%PYTHONPATH%;%BASE_TOOLS_PATH%\Source\Python +@%PYTHON_COMMAND% -m %ToolName%.%ToolName% %* diff --git a/BaseTools/Source/Python/GenCrc32/GenCrc32.py b/BaseTools/Sourc= e/Python/GenCrc32/GenCrc32.py new file mode 100644 index 0000000000..ef1f2a5b0a --- /dev/null +++ b/BaseTools/Source/Python/GenCrc32/GenCrc32.py @@ -0,0 +1,185 @@ +# +# @file +# Calculate Crc32 value and Verify Crc32 value for input data. +# +# Copyright (c) 2022, Intel Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent +# + +import sys +import argparse +import zlib + +__prog__ =3D 'GenCrc32' +__description__ =3D 'Tool has two modes of operation: Encode and Decode. \ + Encode calculates CRC32 of a file and appends checksum \ + to the beginning of file. Decode checks stored CRC32 \ + and removes it from file.' +__version__ =3D '0.3' + +MSG_INFO =3D 0 +MSG_ERROR =3D 1 +MSG_DEBUG =3D 2 + +STATUS_SUCCESS =3D 0 +STATUS_ERROR =3D 2 + + +def log(args, message_type, message): + if (((args.verbose and + (message_type =3D=3D MSG_INFO or + message_type =3D=3D MSG_ERROR)) or + (args.debug and + (message_type =3D=3D MSG_DEBUG or + message_type =3D=3D MSG_INFO or + message_type =3D=3D MSG_ERROR))) and + not args.quiet): + print("%s: %s" % (__prog__, message)) + + +# replacement for int.from_bytes() +# for compatability with python 2 +def uint32_from_bytes(data): + barray =3D bytearray(data) + return ( + (barray[3] << 24) | + (barray[2] << 16) | + (barray[1] << 8) | + (barray[0] << 0)) + + +# replacement for int.to_bytes() +# for compatability with python 2 +def uint32_to_bytes(uint): + return bytearray([ + (uint >> 0) & 0xff, + (uint >> 8) & 0xff, + (uint >> 16) & 0xff, + (uint >> 24) & 0xff]) + + +def parse_args(): + parser =3D argparse.ArgumentParser(prog=3D__prog__, + description=3D__description__, + conflict_handler=3D'resolve') + parser.add_argument('--version', + action=3D'version', + version=3D__version__, + help=3D'Show version number and exit') + parser.add_argument('--debug', + type=3Dint, + metavar=3D'[0-9]', + choices=3Drange(0, 10), + default=3D0, + help=3D'Output DEBUG statements') + logging =3D parser.add_mutually_exclusive_group(required=3DFalse) + logging.add_argument('-v', + '--verbose', + action=3D'store_true', + help=3D'Print informational statements') + logging.add_argument('-q', + '--quiet', + action=3D'store_true', + help=3D'Returns the exit code, \ + error messages will be displayed') + operation =3D parser.add_mutually_exclusive_group(required=3DTrue) + operation.add_argument('-e', + '--encode', + action=3D'store_true', + help=3D'Calculate CRC32 value for the input fil= e') + operation.add_argument('-d', + '--decode', + action=3D'store_true', + help=3D'Verify CRC32 value for the input file') + parser.add_argument('-o', + '--output', + dest=3D'output_file', + type=3Dstr, + help=3D'Output file name', + required=3DTrue) + parser.add_argument(dest=3D'input_file', + type=3Dargparse.FileType('rb'), + help=3D'Input file name') + return parser.parse_args() + + +def main(): + args =3D parse_args() + log(args, MSG_DEBUG, "Input file name: %s" % args.input_file.name) + log(args, MSG_DEBUG, "Output file name: %s" % args.output_file) + if args.encode: + log(args, MSG_INFO, "Operation: Encode") + if args.decode: + log(args, MSG_INFO, "Operation: Decode") + + # Reading input file to buffer + data =3D bytearray() + header =3D bytearray() + log(args, MSG_INFO, "Reading input file") + try: + if args.decode: + header =3D args.input_file.read(4) + data =3D args.input_file.read() + args.input_file.close() + except MemoryError as e: + log(args, MSG_ERROR, ("Error reading input file, " + "cannot allocate memory")) + sys.exit(STATUS_ERROR) + log(args, MSG_INFO, ("Input file length: %d bytes " + "(%d bytes crc32 header and %d bytes data)") % + ((len(header) + len(data)), len(header), len(data))) + + # Calculating CRC32 + log(args, MSG_INFO, "Calculating CRC32") + data_crc32 =3D zlib.crc32(data) & 0xffffffff + if args.decode: + if len(header) >=3D 4: + header_crc32 =3D uint32_from_bytes(header) + else: + log(args, MSG_ERROR, ("Input file size is smaller " + "than 4 bytes, invalid file format")) + sys.exit(STATUS_ERROR) + log(args, MSG_INFO, "CRC32 in input file header is %s" % + hex(header_crc32)) + log(args, MSG_INFO, "CRC32 of input file data is %s" % + hex(data_crc32)) + + # Perform selected operation by modifying header/data + + # Encoding + if args.encode: + log(args, MSG_DEBUG, "Encoding header") + header =3D uint32_to_bytes(data_crc32) + + # Verification and decoding + if args.decode: + log(args, MSG_INFO, "Verifying checksum") + header =3D bytearray() + if data_crc32 =3D=3D header_crc32: + log(args, MSG_INFO, "Checksum verification succeeded") + else: + log(args, MSG_ERROR, ("Checksum verification failed, " + "CRC32 mismatch")) + sys.exit(STATUS_ERROR) + + # Write output file + log(args, MSG_INFO, "Writing output file") + try: + with open(args.output_file, 'wb') as output_file: + output_file.write(header) + output_file.write(data) + except OSError as e: + log(args, MSG_ERROR, "Error writing output file: %s", + format(e.errno, e.strerror)) + sys.exit(STATUS_ERROR) + log(args, MSG_INFO, ("Output file length: %d bytes " + "(%d bytes crc32 header and %d bytes data)") % + ((len(header) + len(data)), len(header), len(data))) + + # Finish + log(args, MSG_INFO, "Success") + sys.exit(STATUS_SUCCESS) + + +if __name__ =3D=3D "__main__": + main() diff --git a/BaseTools/Tests/PythonToolsTests.py b/BaseTools/Tests/PythonTo= olsTests.py index 05b27ab033..4d0ae89508 100644 --- a/BaseTools/Tests/PythonToolsTests.py +++ b/BaseTools/Tests/PythonToolsTests.py @@ -20,6 +20,8 @@ def TheTestSuite(): suites.append(CheckPythonSyntax.TheTestSuite()) import CheckUnicodeSourceFiles suites.append(CheckUnicodeSourceFiles.TheTestSuite()) + import TestGenCrc32 + suites.append(TestGenCrc32.TheTestSuite()) return unittest.TestSuite(suites) =20 if __name__ =3D=3D '__main__': diff --git a/BaseTools/Tests/TestGenCrc32.py b/BaseTools/Tests/TestGenCrc32= .py new file mode 100644 index 0000000000..937d0b14d5 --- /dev/null +++ b/BaseTools/Tests/TestGenCrc32.py @@ -0,0 +1,64 @@ +# @file +# Unit tests for GenCrc32 utility +# +# Copyright (c) 2008, Intel Corporation. All rights reserved.
+# +# SPDX-License-Identifier: BSD-2-Clause-Patent +# + +import unittest +import TestTools + + +class Tests(TestTools.BaseToolsTest): + + def setUp(self): + TestTools.BaseToolsTest.setUp(self) + self.toolName =3D 'GenCrc32' + + def test_display_help(self): + result =3D self.RunTool( + '--help', + logFile=3D'help' + ) + self.assertTrue(result =3D=3D 0) + + def do_encode_decode(self, data): + path =3D self.GetTmpFilePath('input') + self.WriteTmpFile('input', data) + result =3D self.RunTool( + '--verbose', + '-e', + '-o', self.GetTmpFilePath('output1'), + self.GetTmpFilePath('input'), + ) + self.assertTrue(result =3D=3D 0) + result =3D self.RunTool( + '-d', + '-o', self.GetTmpFilePath('output2'), + self.GetTmpFilePath('output1') + ) + self.assertTrue(result =3D=3D 0) + start =3D self.ReadTmpFile('input') + finish =3D self.ReadTmpFile('output2') + start_equals_finish =3D start =3D=3D finish + if not start_equals_finish: + print('Original data did not match decode(encode(data))') + self.DisplayBinaryData('original data', start) + self.DisplayBinaryData('after encoding', + self.ReadTmpFile('output1')) + self.DisplayBinaryData('after decoding', finish) + self.assertTrue(start_equals_finish) + + def test_encode_decode_various_sizes(self): + for i in [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 100, 1000]: + data =3D self.GetRandomString(i, i) + self.do_encode_decode(data) + self.CleanUpTmpDir() + + +TheTestSuite =3D TestTools.MakeTheTestSuite(locals()) + +if __name__ =3D=3D '__main__': + allTests =3D TheTestSuite() + unittest.TextTestRunner().run(allTests) --=20 2.25.1 -------------------------------------------------------------------- Joint Stock Company Intel A/O Registered legal address: Krylatsky Hills Business Park, 17 Krylatskaya Str., Bldg 4, Moscow 121614, Russian Federation This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#88443): https://edk2.groups.io/g/devel/message/88443 Mute This Topic: https://groups.io/mt/90276517/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-