From nobody Wed Apr 24 14:30:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+70233+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+70233+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1610531783; cv=none; d=zohomail.com; s=zohoarc; b=a42zg6kX++kuTJ9bcnIRc/NILmn862xjDo4F/zT4gED0i0NefJOnFdBETWrkdJN8FjXIOS501M7kgRxiQiqqwJTLqKRA/r6AJwL1I9RzeeRN6VRhCmtVYguznMRboLX9M/fP73bcPKg6kPrbY8EpX6abPa48btJl7dB18hAbY6U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1610531783; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=e9QultEMbAoH5d8K8Ypk7L84/znF1h5OZk1RsfeNHUs=; b=QLiplKTnCXwGy2fk/psO8E0C0FOOGUSKQQEjRGh41awQqJhNfDUN6YExjs/xxJx9CjH894sKL8MM95Ss9gBoNEDQrlP+ppgQ3GkBuTXb3fEjaU2F/Cc3WhWaeDprp51SrZzj4t2//1z7zchihoX4t4WrS2oLcJ3fbwBYZHfWCgk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+70233+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1610531783425107.15865337759737; Wed, 13 Jan 2021 01:56:23 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id JhviYY1788612xmS3e9EM3Oe; Wed, 13 Jan 2021 01:56:22 -0800 X-Received: from mga14.intel.com (mga14.intel.com []) by mx.groups.io with SMTP id smtpd.web11.5943.1610531774284313300 for ; Wed, 13 Jan 2021 01:56:17 -0800 IronPort-SDR: vBwy18wBYto/XGWMZuaKnKu9zyNOM7UAbYy/Qg7ArgPXJp6KC4OxocEDXFiRBKC11bRcBU7D4x ZdRknLFmbBfg== X-IronPort-AV: E=McAfee;i="6000,8403,9862"; a="177403469" X-IronPort-AV: E=Sophos;i="5.79,344,1602572400"; d="scan'208";a="177403469" X-Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jan 2021 01:56:17 -0800 IronPort-SDR: sCXN/MYHlUuZOWAOoInyv9OJIYAzTa8mMaeDLbDarS76H23tR7CE8kVaWY7PqX4G4gjpMgJu4Y mIzvgCT4Oh+w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.79,344,1602572400"; d="scan'208";a="464845012" X-Received: from shwdepsi1121.ccr.corp.intel.com ([10.239.158.32]) by fmsmga001.fm.intel.com with ESMTP; 13 Jan 2021 01:56:15 -0800 From: "Bob Feng" To: devel@edk2.groups.io Cc: Liming Gao , Yuwei Chen Subject: [edk2-devel] [Patch 1/2] BaseTools: Convert Split tool to python Date: Wed, 13 Jan 2021 17:56:08 +0800 Message-Id: <20210113095609.1398-2-bob.c.feng@intel.com> In-Reply-To: <20210113095609.1398-1-bob.c.feng@intel.com> References: <20210113095609.1398-1-bob.c.feng@intel.com> MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,bob.c.feng@intel.com X-Gm-Message-State: Wt42xLbo60hpvhM6SpGwPIUzx1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1610531782; bh=62dVY/oUal7LDA4RuSHki8hAFbmDihZHcE/sTX6j/tQ=; h=Cc:Date:From:Reply-To:Subject:To; b=j8RYfkkji+CEkQwuv1sEqYgMa8JE+CskhoSMzYIWl294SNqFQq4vHIuVxpL3XysQiRN P2nerVHgt9Zjo34NNH/QhhsE9APIgvHWEdr7savSOVPSezqLZyMGquzf6lTQfWxwWZ9bn rmeSTwQ/yi/Uc6fftewbkOOCAiedIrie3Vk= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Type: text/plain; charset="utf-8" There are 2 reasons to convert Split tool from C to Python. 1. We are in the process of moving the Basetools Python code to a separate repository. But there still are many C tools under edk2/BaseTools. To make all Basetools be in the separate repo, we can convert the C tools to Python tools. 2. The original Split tool is very slow. This python tool can reduce 90% time. Signed-off-by: Bob Feng Cc: Liming Gao Cc: Yuwei Chen --- BaseTools/Source/C/Split/Split.c | 466 -------------------- BaseTools/BinWrappers/PosixLike/Split | 31 +- BaseTools/BinWrappers/WindowsLike/Split.bat | 3 + BaseTools/Source/C/GNUmakefile | 1 - BaseTools/Source/C/Makefile | 1 - BaseTools/Source/C/Split/GNUmakefile | 17 - BaseTools/Source/C/Split/Makefile | 16 - BaseTools/Source/Python/Split/Split.py | 203 +++++++++ BaseTools/Source/Python/Split/__init__.py | 10 + 9 files changed, 224 insertions(+), 524 deletions(-) delete mode 100644 BaseTools/Source/C/Split/Split.c create mode 100644 BaseTools/BinWrappers/WindowsLike/Split.bat delete mode 100644 BaseTools/Source/C/Split/GNUmakefile delete mode 100644 BaseTools/Source/C/Split/Makefile create mode 100644 BaseTools/Source/Python/Split/Split.py create mode 100644 BaseTools/Source/Python/Split/__init__.py diff --git a/BaseTools/Source/C/Split/Split.c b/BaseTools/Source/C/Split/Sp= lit.c deleted file mode 100644 index be0ee124bfe0..000000000000 --- a/BaseTools/Source/C/Split/Split.c +++ /dev/null @@ -1,466 +0,0 @@ -/** @file - - Split a file into two pieces at the request offset. - -Copyright (c) 1999 - 2017, Intel Corporation. All rights reserved.
-SPDX-License-Identifier: BSD-2-Clause-Patent - -**/ - -// GC_TODO: fix comment to start with /*++ -#include -#include -#include -#ifdef __GNUC__ -#include -#else -#include -#endif -#include -#include "ParseInf.h" -#include "CommonLib.h" -#include "EfiUtilityMsgs.h" -// -// Utility Name -// -#define UTILITY_NAME "Split" - -// -// Utility version information -// -#define UTILITY_MAJOR_VERSION 1 -#define UTILITY_MINOR_VERSION 0 - -void -Version ( - void - ) -/*++ - -Routine Description: - - Displays the standard utility information to SDTOUT - -Arguments: - - None - -Returns: - - None - ---*/ -{ - printf ("%s Version %d.%d Build %s\n", UTILITY_NAME, UTILITY_MAJOR_VERSI= ON, UTILITY_MINOR_VERSION, __BUILD_VERSION); -} - -void -Usage ( - void - ) -/*++ - -Routine Description: - - GC_TODO: Add function description - -Arguments: - - -Returns: - - GC_TODO: add return values - ---*/ -{ - Version(); - printf ("Copyright (c) 1999-2017 Intel Corporation. All rights reserved.= \n"); - printf ("\n SplitFile creates two Binary files either in the same direc= tory as the current working\n"); - printf (" directory or in the specified directory.\n"); - printf ("\nUsage: \n\ - Split\n\ - -f, --filename inputFile to split\n\ - -s, --split VALUE the number of bytes in the first file\n\ - [-p, --prefix OutputDir]\n\ - [-o, --firstfile Filename1]\n\ - [-t, --secondfile Filename2]\n\ - [-v, --verbose]\n\ - [--version]\n\ - [-q, --quiet disable all messages except fatal errors]\n\ - [-d, --debug[#]\n\ - [-h, --help]\n"); -} - -EFI_STATUS -GetSplitValue ( - IN CONST CHAR8* SplitValueString, - OUT UINT64 *ReturnValue -) -{ - UINT64 len =3D 0; - UINT64 base =3D 1; - UINT64 index =3D 0; - UINT64 number =3D 0; - CHAR8 lastCHAR =3D 0; - EFI_STATUS Status =3D EFI_SUCCESS; - - if (SplitValueString !=3D NULL){ - len =3D strlen(SplitValueString); - } - - if (len =3D=3D 0) { - return EFI_ABORTED; - } - - Status =3D AsciiStringToUint64 (SplitValueString, FALSE, ReturnValue); - if (!EFI_ERROR (Status)) { - return Status; - } - - if (SplitValueString[0] =3D=3D '0' && (SplitValueString[1] =3D=3D 'x' ||= SplitValueString[1] =3D=3D 'X')) { - Status =3D AsciiStringToUint64 (SplitValueString, TRUE, ReturnValue); - if (!EFI_ERROR (Status)) { - return Status; - } - } - - lastCHAR =3D (CHAR8)toupper((int)SplitValueString[len - 1]); - - if (lastCHAR !=3D 'K' && lastCHAR !=3D 'M' && lastCHAR !=3D 'G') { - return STATUS_ERROR; - } - - for (;index < len - 1; ++index) { - if (!isdigit((int)SplitValueString[index])) { - return EFI_ABORTED; - } - } - - number =3D atol (SplitValueString); - if (lastCHAR =3D=3D 'K') - base =3D 1024; - else if (lastCHAR =3D=3D 'M') - base =3D 1024*1024; - else - base =3D 1024*1024*1024; - - *ReturnValue =3D number*base; - - return EFI_SUCCESS; -} - -EFI_STATUS -CountVerboseLevel ( - IN CONST CHAR8* VerboseLevelString, - IN CONST UINT64 Length, - OUT UINT64 *ReturnValue -) -{ - UINT64 i =3D 0; - for (;i < Length; ++i) { - if (VerboseLevelString[i] !=3D 'v' && VerboseLevelString[i] !=3D 'V') { - return EFI_ABORTED; - } - ++(*ReturnValue); - } - - return EFI_SUCCESS; -} - -EFI_STATUS -CreateDir ( - IN OUT CHAR8** FullFileName -) -{ - CHAR8* temp =3D *FullFileName; - CHAR8* start =3D temp; - CHAR8 tempchar; - UINT64 index =3D 0; - - for (;index < strlen(temp); ++index) { - if (temp[index] =3D=3D '\\' || temp[index] =3D=3D '/') { - if (temp[index + 1] !=3D '\0') { - tempchar =3D temp[index + 1]; - temp[index + 1] =3D 0; - if (chdir(start)) { - if (mkdir(start, S_IRWXU | S_IRWXG | S_IRWXO) !=3D 0) { - return EFI_ABORTED; - } - chdir(start); - } - start =3D temp + index + 1; - temp[index] =3D '/'; - temp[index + 1] =3D tempchar; - } - } - } - - return EFI_SUCCESS; -} - -int -main ( - int argc, - char*argv[] - ) -/*++ - -Routine Description: - - GC_TODO: Add function description - -Arguments: - - argc - GC_TODO: add argument description - ] - GC_TODO: add argument description - -Returns: - - GC_TODO: add return values - ---*/ -{ - EFI_STATUS Status =3D EFI_SUCCESS; - INTN ReturnStatus =3D STATUS_SUCCESS; - FILE *In; - CHAR8 *InputFileName =3D NULL; - CHAR8 *OutputDir =3D NULL; - CHAR8 *OutFileName1 =3D NULL; - CHAR8 *OutFileName2 =3D NULL; - UINT64 SplitValue =3D (UINT64) -1; - FILE *Out1 =3D NULL; - FILE *Out2 =3D NULL; - CHAR8 *OutName1 =3D NULL; - CHAR8 *OutName2 =3D NULL; - CHAR8 *CurrentDir =3D NULL; - UINT64 Index; - CHAR8 CharC; - UINT64 DebugLevel =3D 0; - UINT64 VerboseLevel =3D 0; - - SetUtilityName(UTILITY_NAME); - if (argc =3D=3D 1) { - Usage(); - return STATUS_ERROR; - } - - argc --; - argv ++; - - if ((stricmp (argv[0], "-h") =3D=3D 0) || (stricmp (argv[0], "--help") = =3D=3D 0)) { - Usage(); - return STATUS_SUCCESS; - } - - if (stricmp (argv[0], "--version") =3D=3D 0) { - Version(); - return STATUS_SUCCESS; - } - - while (argc > 0) { - if ((stricmp (argv[0], "-p") =3D=3D 0) || (stricmp (argv[0], "--prefix= ") =3D=3D 0)) { - OutputDir =3D argv[1]; - if (OutputDir =3D=3D NULL) { - Warning (NULL, 0, 0, "NO output directory specified.", NULL); - return STATUS_ERROR; - } - argc -=3D 2; - argv +=3D 2; - continue; - } - - if ((stricmp (argv[0], "-f") =3D=3D 0) || (stricmp (argv[0], "--filena= me") =3D=3D 0)) { - InputFileName =3D argv[1]; - if (InputFileName =3D=3D NULL) { - Error (NULL, 0, 0x1001, "NO Input file specified.", NULL); - return STATUS_ERROR; - } - argc -=3D 2; - argv +=3D 2; - continue; - } - - if ((stricmp (argv[0], "-s") =3D=3D 0) || (stricmp (argv[0], "--split"= ) =3D=3D 0)) { - Status =3D GetSplitValue(argv[1], &SplitValue); - if (EFI_ERROR (Status)) { - Error (NULL, 0, 0x1003, "Input split value is not one valid intege= r.", NULL); - return STATUS_ERROR; - } - argc -=3D 2; - argv +=3D 2; - continue; - } - - if ((stricmp (argv[0], "-o") =3D=3D 0) || (stricmp (argv[0], "--firstf= ile") =3D=3D 0)) { - OutFileName1 =3D argv[1]; - if (OutFileName1 =3D=3D NULL) { - Warning (NULL, 0, 0, NULL, "No output file1 specified."); - } - argc -=3D 2; - argv +=3D 2; - continue; - } - - if ((stricmp (argv[0], "-t") =3D=3D 0) || (stricmp (argv[0], "--second= file") =3D=3D 0)) { - OutFileName2 =3D argv[1]; - if (OutFileName2 =3D=3D NULL) { - Warning (NULL, 0, 0, NULL, "No output file2 specified."); - } - argc -=3D 2; - argv +=3D 2; - continue; - } - - if ((stricmp (argv[0], "-q") =3D=3D 0) || (stricmp (argv[0], "--quiet"= ) =3D=3D 0)) { - argc --; - argv ++; - continue; - } - - if ((strlen(argv[0]) >=3D 2 && argv[0][0] =3D=3D '-' && (argv[0][1] = =3D=3D 'v' || argv[0][1] =3D=3D 'V')) || (stricmp (argv[0], "--verbose") = =3D=3D 0)) { - VerboseLevel =3D 1; - if (strlen(argv[0]) > 2) { - Status =3D CountVerboseLevel (&argv[0][2], strlen(argv[0]) - 2, &V= erboseLevel); - if (EFI_ERROR (Status)) { - Error (NULL, 0, 0x1003, NULL, "%s is invalid parameter!", argv[0= ]); - return STATUS_ERROR; - } - } - - argc --; - argv ++; - continue; - } - - if ((stricmp (argv[0], "-d") =3D=3D 0) || (stricmp (argv[0], "--debug"= ) =3D=3D 0)) { - Status =3D AsciiStringToUint64 (argv[1], FALSE, &DebugLevel); - if (EFI_ERROR (Status)) { - Error (NULL, 0, 0x1003, "Input debug level is not one valid integr= ator.", NULL); - return STATUS_ERROR; - } - argc -=3D 2; - argv +=3D 2; - continue; - } - // - // Don't recognize the parameter. - // - Error (NULL, 0, 0x1003, NULL, "%s is invalid parameter!", argv[0]); - return STATUS_ERROR; - } - - if (InputFileName =3D=3D NULL) { - Error (NULL, 0, 0x1001, "NO Input file specified.", NULL); - return STATUS_ERROR; - } - - In =3D fopen (LongFilePath (InputFileName), "rb"); - if (In =3D=3D NULL) { - // ("Unable to open file \"%s\"\n", InputFileName); - Error (InputFileName, 0, 1, "File open failure", NULL); - return STATUS_ERROR; - } - - if (OutFileName1 =3D=3D NULL) { - OutName1 =3D (CHAR8*)malloc(strlen(InputFileName) + 16); - if (OutName1 =3D=3D NULL) { - Warning (NULL, 0, 0, NULL, "Memory Allocation Fail."); - ReturnStatus =3D STATUS_ERROR; - goto Finish; - } - strcpy (OutName1, InputFileName); - strcat (OutName1, "1"); - OutFileName1 =3D OutName1; - - } - if (OutFileName2 =3D=3D NULL) { - OutName2 =3D (CHAR8*)malloc(strlen(InputFileName) + 16); - if (OutName2 =3D=3D NULL) { - Warning (NULL, 0, 0, NULL, "Memory Allocation Fail."); - ReturnStatus =3D STATUS_ERROR; - goto Finish; - } - strcpy (OutName2, InputFileName); - strcat (OutName2, "2"); - OutFileName2 =3D OutName2; - - } - - if (OutputDir !=3D NULL) { - //OutputDirSpecified =3D TRUE; - if (chdir(OutputDir) !=3D 0) { - Warning (NULL, 0, 0, NULL, "Change dir to OutputDir Fail."); - ReturnStatus =3D STATUS_ERROR; - goto Finish; - } - } - - CurrentDir =3D (CHAR8*)getcwd((CHAR8*)0, 0); - if (EFI_ERROR(CreateDir(&OutFileName1))) { - Error (OutFileName1, 0, 5, "Create Dir for File1 Fail.", NULL); - ReturnStatus =3D STATUS_ERROR; - goto Finish; - } - chdir(CurrentDir); - - if (EFI_ERROR(CreateDir(&OutFileName2))) { - Error (OutFileName2, 0, 5, "Create Dir for File2 Fail.", NULL); - ReturnStatus =3D STATUS_ERROR; - goto Finish; - } - chdir(CurrentDir); - free(CurrentDir); - - Out1 =3D fopen (LongFilePath (OutFileName1), "wb"); - if (Out1 =3D=3D NULL) { - // ("Unable to open file \"%s\"\n", OutFileName1); - Error (OutFileName1, 0, 1, "File open failure", NULL); - ReturnStatus =3D STATUS_ERROR; - goto Finish; - } - - Out2 =3D fopen (LongFilePath (OutFileName2), "wb"); - if (Out2 =3D=3D NULL) { - // ("Unable to open file \"%s\"\n", OutFileName2); - Error (OutFileName2, 0, 1, "File open failure", NULL); - ReturnStatus =3D STATUS_ERROR; - goto Finish; - } - - for (Index =3D 0; Index < SplitValue; Index++) { - CharC =3D (CHAR8) fgetc (In); - if (feof (In)) { - break; - } - - fputc (CharC, Out1); - } - - for (;;) { - CharC =3D (CHAR8) fgetc (In); - if (feof (In)) { - break; - } - - fputc (CharC, Out2); - } - -Finish: - if (OutName1 !=3D NULL) { - free(OutName1); - } - if (OutName2 !=3D NULL) { - free(OutName2); - } - if (In !=3D NULL) { - fclose (In); - } - if (Out1 !=3D NULL) { - fclose (Out1); - } - if (Out2 !=3D NULL) { - fclose (Out2); - } - - return ReturnStatus; -} diff --git a/BaseTools/BinWrappers/PosixLike/Split b/BaseTools/BinWrappers/= PosixLike/Split index 0945d86d9209..f3770eed42b4 100755 --- a/BaseTools/BinWrappers/PosixLike/Split +++ b/BaseTools/BinWrappers/PosixLike/Split @@ -1,29 +1,14 @@ #!/usr/bin/env bash +#python `dirname $0`/RunToolFromSource.py `basename $0` $* + +# If a ${PYTHON_COMMAND} command is available, use it in preference to pyt= hon +if command -v ${PYTHON_COMMAND} >/dev/null 2>&1; then + python_exe=3D${PYTHON_COMMAND} +fi =20 full_cmd=3D${BASH_SOURCE:-$0} # see http://mywiki.wooledge.org/BashFAQ/028= for a discussion of why $0 is not a good choice here dir=3D$(dirname "$full_cmd") cmd=3D${full_cmd##*/} =20 -if [ -n "$WORKSPACE" ] && [ -e "$WORKSPACE/Conf/BaseToolsCBinaries" ] -then - exec "$WORKSPACE/Conf/BaseToolsCBinaries/$cmd" -elif [ -n "$WORKSPACE" ] && [ -e "$EDK_TOOLS_PATH/Source/C" ] -then - if [ ! -e "$EDK_TOOLS_PATH/Source/C/bin/$cmd" ] - then - echo "BaseTools C Tool binary was not found ($cmd)" - echo "You may need to run:" - echo " make -C $EDK_TOOLS_PATH/Source/C" - else - exec "$EDK_TOOLS_PATH/Source/C/bin/$cmd" "$@" - fi -elif [ -e "$dir/../../Source/C/bin/$cmd" ] -then - exec "$dir/../../Source/C/bin/$cmd" "$@" -else - echo "Unable to find the real '$cmd' to run" - echo "This message was printed by" - echo " $0" - exit 127 -fi - +export PYTHONPATH=3D"$dir/../../Source/Python${PYTHONPATH:+:"$PYTHONPATH"}" +exec "${python_exe:-python}" "$dir/../../Source/Python/$cmd/$cmd.py" "$@" diff --git a/BaseTools/BinWrappers/WindowsLike/Split.bat b/BaseTools/BinWra= ppers/WindowsLike/Split.bat new file mode 100644 index 000000000000..9616cd893bec --- /dev/null +++ b/BaseTools/BinWrappers/WindowsLike/Split.bat @@ -0,0 +1,3 @@ +@setlocal +@set ToolName=3D%~n0% +@%PYTHON_COMMAND% %BASE_TOOLS_PATH%\Source\Python\%ToolName%\%ToolName%.py= %* diff --git a/BaseTools/Source/C/GNUmakefile b/BaseTools/Source/C/GNUmakefile index 464f43277455..8c191e0c3817 100644 --- a/BaseTools/Source/C/GNUmakefile +++ b/BaseTools/Source/C/GNUmakefile @@ -55,11 +55,10 @@ APPLICATIONS =3D \ GenFv \ GenFw \ GenSec \ GenCrc32 \ LzmaCompress \ - Split \ TianoCompress \ VolInfo \ DevicePath =20 SUBDIRS :=3D $(LIBRARIES) $(APPLICATIONS) diff --git a/BaseTools/Source/C/Makefile b/BaseTools/Source/C/Makefile index e8f8abe59a79..a376d32e220e 100644 --- a/BaseTools/Source/C/Makefile +++ b/BaseTools/Source/C/Makefile @@ -17,11 +17,10 @@ APPLICATIONS =3D \ GenFfs \ GenFv \ GenFw \ GenSec \ LzmaCompress \ - Split \ TianoCompress \ VolInfo \ DevicePath =20 all: libs apps install diff --git a/BaseTools/Source/C/Split/GNUmakefile b/BaseTools/Source/C/Spli= t/GNUmakefile deleted file mode 100644 index b3d4dff51ac1..000000000000 --- a/BaseTools/Source/C/Split/GNUmakefile +++ /dev/null @@ -1,17 +0,0 @@ -## @file -# GNU/Linux makefile for 'Split' module build. -# -# Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.
-# SPDX-License-Identifier: BSD-2-Clause-Patent -# -MAKEROOT ?=3D .. - -APPNAME =3D Split - -OBJECTS =3D Split.o - -include $(MAKEROOT)/Makefiles/app.makefile - -LIBS =3D -lCommon - - diff --git a/BaseTools/Source/C/Split/Makefile b/BaseTools/Source/C/Split/M= akefile deleted file mode 100644 index 19d3e31a7624..000000000000 --- a/BaseTools/Source/C/Split/Makefile +++ /dev/null @@ -1,16 +0,0 @@ -## @file -# Windows makefile for 'Split' module build. -# -# Copyright (c) 2007 - 2010, Intel Corporation. All rights reserved.
-# SPDX-License-Identifier: BSD-2-Clause-Patent -# -!INCLUDE ..\Makefiles\ms.common - -APPNAME =3D Split - -LIBS =3D $(LIB_PATH)\Common.lib - -OBJECTS =3D Split.obj - -!INCLUDE ..\Makefiles\ms.app - diff --git a/BaseTools/Source/Python/Split/Split.py b/BaseTools/Source/Pyth= on/Split/Split.py new file mode 100644 index 000000000000..39d4ee7689d6 --- /dev/null +++ b/BaseTools/Source/Python/Split/Split.py @@ -0,0 +1,203 @@ + +# @file +# Split a file into two pieces at the request offset. +# +# Copyright (c) 2021, Intel Corporation. All rights reserved.
+# +# SPDX-License-Identifier: BSD-2-Clause-Patent +# +## + +# Import Modules +# +import argparse +import os +import io +import shutil +import logging +import sys +import tempfile + +parser =3D argparse.ArgumentParser(description=3D''' +SplitFile creates two Binary files either in the same directory as the cur= rent working directory or in the specified directory. +''') +parser.add_argument("-f", "--filename", dest=3D"inputfile", + required=3DTrue, help=3D"The input file to split tool.= ") +parser.add_argument("-s", "--split", dest=3D"position", + required=3DTrue, help=3D"The number of bytes in the fi= rst file. The valid format are HEX, Decimal and Decimal[KMG].") +parser.add_argument("-p", "--prefix", dest=3D"output", + help=3D"The output folder.") +parser.add_argument("-o", "--firstfile", help=3D"The first file name") +parser.add_argument("-t", "--secondfile", help=3D"The second file name") +parser.add_argument("--version", action=3D"version", version=3D'%(prog)s V= ersion 1.0', + help=3D"Print debug information.") + +group =3D parser.add_mutually_exclusive_group() +group.add_argument("-v", "--verbose", action=3D"store_true", + help=3D"Print debug information.") +group.add_argument("-q", "--quiet", action=3D"store_true", + help=3D"Disable all messages except fatal errors") + +SizeDict =3D { + "K": 1024, + "M": 1024*1024, + "G": 1024*1024*1024 +} + + +def GetPositionValue(position): + ''' + Parse the string of the argument position and return a decimal number. + The valid position formats are + 1. HEX + e.g. 0x1000 or 0X1000 + 2. Decimal + e.g. 100 + 3. Decimal[KMG] + e.g. 100K or 100M or 100G or 100k or 100m or 100g + ''' + logger =3D logging.getLogger('Split') + PosVal =3D 0 + header =3D position[:2].upper() + tailer =3D position[-1].upper() + + try: + if tailer in SizeDict: + PosVal =3D int(position[:-1]) * SizeDict[tailer] + else: + if header =3D=3D "0X": + PosVal =3D int(position, 16) + else: + PosVal =3D int(position) + except Exception as e: + logger.error( + "The parameter %s format is incorrect. The valid format is HEX= , Decimal and Decimal[KMG]." % position) + raise(e) + + return PosVal + + +def getFileSize(filename): + ''' + Read the input file and return the file size. + ''' + logger =3D logging.getLogger('Split') + length =3D 0 + try: + with open(filename, "rb") as fin: + fin.seek(0, io.SEEK_END) + length =3D fin.tell() + except Exception as e: + logger.error("Access file failed: %s", filename) + raise(e) + + return length + + +def splitFile(inputfile, position, outputdir=3DNone, outputfile1=3DNone, o= utputfile2=3DNone): + ''' + Split the inputfile into outputfile1 and outputfile2 from the position. + ''' + logger =3D logging.getLogger('Split') + + inputfile =3D os.path.abspath(inputfile) + workspace =3D os.path.dirname(inputfile) + if not os.path.exists(inputfile): + logger.error("File Not Found: %s" % inputfile) + raise(Exception) + + if outputfile1 and outputfile2 and outputfile1 =3D=3D outputfile2: + logger.error( + "The firstfile and the secondfile can't be the same: %s" % out= putfile1) + raise(Exception) + + if not outputdir: + outputdir =3D workspace + elif not os.path.isabs(outputdir): + outputdir =3D os.path.join(workspace, outputdir) + + # Create dir for the output files + try: + if not outputfile1: + outputfile1 =3D os.path.abspath(os.path.join( + outputdir, "{}1".format(os.path.basename(inputfile)))) + else: + outputfile1 =3D os.path.abspath(os.path.join(outputdir, output= file1)) + outputdir =3D os.path.dirname(outputfile1) + if not os.path.exists(outputdir): + os.makedirs(outputdir) + + if not outputfile2: + outputfile2 =3D os.path.abspath(os.path.join( + outputdir, "{}2".format(os.path.basename(inputfile)))) + else: + outputfile2 =3D os.path.abspath(os.path.join(outputdir, output= file2)) + outputdir =3D os.path.dirname(outputfile2) + if not os.path.exists(outputdir): + os.makedirs(outputdir) + except Exception as e: + logger.error("Can't make dir: %s" % outputdir) + raise(e) + + if position <=3D 0: + if outputfile2 !=3D inputfile: + shutil.copy2(inputfile, outputfile2) + with open(outputfile1, "wb") as fout: + fout.write(b'') + else: + inputfilesize =3D getFileSize(inputfile) + if position >=3D inputfilesize: + if outputfile1 !=3D inputfile: + shutil.copy2(inputfile, outputfile1) + with open(outputfile2, "wb") as fout: + fout.write(b'') + else: + try: + tempdir =3D tempfile.mkdtemp() + tempfile1 =3D os.path.join(tempdir, "file1.bin") + tempfile2 =3D os.path.join(tempdir, "file2.bin") + with open(inputfile, "rb") as fin: + content1 =3D fin.read(position) + with open(tempfile1, "wb") as fout1: + fout1.write(content1) + + content2 =3D fin.read(inputfilesize - position) + with open(tempfile2, "wb") as fout2: + fout2.write(content2) + shutil.copy2(tempfile1, outputfile1) + shutil.copy2(tempfile2, outputfile2) + except Exception as e: + logger.error("Split file failed") + raise(e) + finally: + if os.path.exists(tempdir): + shutil.rmtree(tempdir) + + +def main(): + args =3D parser.parse_args() + status =3D 0 + + logger =3D logging.getLogger('Split') + if args.quiet: + logger.setLevel(logging.CRITICAL) + if args.verbose: + logger.setLevel(logging.DEBUG) + + lh =3D logging.StreamHandler(sys.stdout) + lf =3D logging.Formatter("%(levelname)-8s: %(message)s") + lh.setFormatter(lf) + logger.addHandler(lh) + + try: + position =3D GetPositionValue(args.position) + splitFile(args.inputfile, position, args.output, + args.firstfile, args.secondfile) + except Exception as e: + status =3D 1 + + return status + + +if __name__ =3D=3D "__main__": + exit(main()) diff --git a/BaseTools/Source/Python/Split/__init__.py b/BaseTools/Source/P= ython/Split/__init__.py new file mode 100644 index 000000000000..8f4daf86e84a --- /dev/null +++ b/BaseTools/Source/Python/Split/__init__.py @@ -0,0 +1,10 @@ +# @file +# Split a file into two pieces at the request offset. +# +# Copyright (c) 2021, Intel Corporation. All rights reserved.
+# +# SPDX-License-Identifier: BSD-2-Clause-Patent +# +## + +# Import Modules --=20 2.29.1.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#70233): https://edk2.groups.io/g/devel/message/70233 Mute This Topic: https://groups.io/mt/79647274/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Wed Apr 24 14:30:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+70234+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+70234+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1610531780; cv=none; d=zohomail.com; s=zohoarc; b=RAQqMUsU0xZvjfB5gxwZuLkvO+ZtIbSdRU0SmA8kczzzhdVWkHadFYkFszrgi9hOiDiWlF6X8jlsLLiBk75t0Dsr1oK5RKLwK2HsGXtyQVonGHdN2a4TTB6xLwdGEKOO2issyrbcm+4b/zdDuVD0WAIktUlnOTqU3vKnrm0cSjY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1610531780; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=b0ucxHEiq2wfCfXQc7ABx6HlnnA/0IJwtu+wqPp5mkA=; b=dht/Ys3DZJmrfRfzMybiS+rycHdkJQ/s9aUh1Bfw5FYADSRME40dAcl0DFY1bfKJT7Nb1pqazGIDExy3I8g8uQMEgp6dcK44mj0rH3rm9VFFiDjNjqWU9sNvl6Wjg3CaymBL2OXDKOi8FHTtoTC2MVlPu7maOIRq9NGewwqLJKw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+70234+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 161053178013587.32032259825019; Wed, 13 Jan 2021 01:56:20 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id cqJIYY1788612xvsgbj0xsAQ; Wed, 13 Jan 2021 01:56:19 -0800 X-Received: from mga14.intel.com (mga14.intel.com []) by mx.groups.io with SMTP id smtpd.web11.5943.1610531774284313300 for ; Wed, 13 Jan 2021 01:56:19 -0800 IronPort-SDR: GRZoE3rHgSh5SJ0ESPDV79CpYdHUBZiX25xf7dqRhX4PxXMl0Htk5Ab5WorUBbtv9ZUIZSeiig n16iuF0s548g== X-IronPort-AV: E=McAfee;i="6000,8403,9862"; a="177403477" X-IronPort-AV: E=Sophos;i="5.79,344,1602572400"; d="scan'208";a="177403477" X-Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jan 2021 01:56:19 -0800 IronPort-SDR: z4HVuZMgOgYzkBRdS2Lx5ZdAERqBl2xbMhfDmq8IsAtKyYWDYTTbEcP6lPH9pp5eiePmzqu78g iyunibXG91FA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.79,344,1602572400"; d="scan'208";a="464845019" X-Received: from shwdepsi1121.ccr.corp.intel.com ([10.239.158.32]) by fmsmga001.fm.intel.com with ESMTP; 13 Jan 2021 01:56:18 -0800 From: "Bob Feng" To: devel@edk2.groups.io Cc: Liming Gao , Yuwei Chen Subject: [edk2-devel] [Patch 2/2] BaseTools: Add unittest for Split tool Date: Wed, 13 Jan 2021 17:56:09 +0800 Message-Id: <20210113095609.1398-3-bob.c.feng@intel.com> In-Reply-To: <20210113095609.1398-1-bob.c.feng@intel.com> References: <20210113095609.1398-1-bob.c.feng@intel.com> MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,bob.c.feng@intel.com X-Gm-Message-State: jYqAWYtKLSYGYa7DO92zxga8x1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1610531779; bh=wjvY5777KK+YZZbGNpaQ721PAckDiF8p2ZGzx6nS8Ko=; h=Cc:Date:From:Reply-To:Subject:To; b=F7mETMdLUS6I4mgj8TKO7Djif8iBLxHgf0/JfCc6A89EYoL0o3ms6CNeHQ74dPJ1uh8 ahi1ntRpbRzhNHuo2O75SoyrhdFe9HxKtww4Ity3C6KcaGGtzayBKlIyqB5PI9M8KSBhQ wjSx8lT5AHoEeXOLoWk9vmE8MeDDtMHL66A= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Type: text/plain; charset="utf-8" This patch is to add the unit test for Split python tool Signed-off-by: Bob Feng Cc: Liming Gao Cc: Yuwei Chen --- BaseTools/Source/Python/tests/test_split.py | 101 ++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 BaseTools/Source/Python/tests/test_split.py diff --git a/BaseTools/Source/Python/tests/test_split.py b/BaseTools/Source= /Python/tests/test_split.py new file mode 100644 index 000000000000..0d44946d7010 --- /dev/null +++ b/BaseTools/Source/Python/tests/test_split.py @@ -0,0 +1,101 @@ +import unittest +import tempfile +import os +import shutil +import Split.Split as sp +import struct as st + + +class TestSplit(unittest.TestCase): + def setUp(self): + self.WORKSPACE =3D tempfile.mkdtemp() + self.binary_file =3D os.path.join(self.WORKSPACE, "Binary.bin") + self.create_inputfile() + + def tearDown(self): + if os.path.exists(self.WORKSPACE): + shutil.rmtree(self.WORKSPACE) + + def test_splitFile_position(self): + position =3D [-1, 0, 256, 512, 700, 1024, 2048] + result =3D [(0, 1024), (0, 1024), (256, 768), + (512, 512), (700, 324), (1024, 0), (1024, 0)] + for index, po in enumerate(position): + try: + sp.splitFile(self.binary_file, po) + except Exception as e: + self.assertTrue(False, msg=3D"splitFile function error") + + output1 =3D os.path.join(self.WORKSPACE, "Binary.bin1") + output2 =3D os.path.join(self.WORKSPACE, "Binary.bin2") + with open(output1, "rb") as f1: + size1 =3D len(f1.read()) + with open(output2, "rb") as f2: + size2 =3D len(f2.read()) + + ex_result =3D result[index] + self.assertEqual(size1, ex_result[0]) + self.assertEqual(size2, ex_result[1]) + + def create_inputfile(self): + with open(self.binary_file, "wb") as fout: + for i in range(512): + fout.write(st.pack("