From nobody Sat May 18 23:23:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=none (zoho.com: 198.145.21.10 is neither permitted nor denied by domain of lists.01.org) smtp.mailfrom=edk2-devel-bounces@lists.01.org; dmarc=fail(p=none dis=none) header.from=intel.com Return-Path: Received: from ml01.01.org (ml01.01.org [198.145.21.10]) by mx.zohomail.com with SMTPS id 1526878324359388.8698983350521; Sun, 20 May 2018 21:52:04 -0700 (PDT) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id C0B58207E6371; Sun, 20 May 2018 21:52:02 -0700 (PDT) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 61372207E53E2 for ; Sun, 20 May 2018 21:52:00 -0700 (PDT) Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 20 May 2018 21:52:00 -0700 Received: from shwde7172.ccr.corp.intel.com ([10.239.158.42]) by orsmga005.jf.intel.com with ESMTP; 20 May 2018 21:51:59 -0700 X-Original-To: edk2-devel@lists.01.org Received-SPF: none (zoho.com: 198.145.21.10 is neither permitted nor denied by domain of lists.01.org) client-ip=198.145.21.10; envelope-from=edk2-devel-bounces@lists.01.org; helo=ml01.01.org; Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=134.134.136.31; helo=mga06.intel.com; envelope-from=liming.gao@intel.com; receiver=edk2-devel@lists.01.org X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,425,1520924400"; d="scan'208";a="225852639" From: Liming Gao To: edk2-devel@lists.01.org Date: Mon, 21 May 2018 12:51:41 +0800 Message-Id: <1526878301-13892-1-git-send-email-liming.gao@intel.com> X-Mailer: git-send-email 2.8.0.windows.1 Subject: [edk2] [RFC] Formalize source files to follow DOS format X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Errors-To: edk2-devel-bounces@lists.01.org Sender: "edk2-devel" X-ZohoMail: RSF_4 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" FormatDosFiles.py is added to clean up dos source files. It bases on the rules defined in EDKII C Coding Standards Specification. 5.1.2 Do not use tab characters 5.1.6 Only use CRLF (Carriage Return Line Feed) line endings. 5.1.7 All files must end with CRLF No trailing white space in one line. (To be added in spec) The source files in edk2 project with the below postfix are dos format. .h .c .nasm .nasmb .asm .S .inf .dec .dsc .fdf .uni .asl .aslc .vfr .idf=20 .txt .bat .py The package maintainer can use this script to clean up all files in his=20 package. The prefer way is to create one patch per one package. Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Liming Gao --- BaseTools/Scripts/FormatDosFiles.py | 93 +++++++++++++++++++++++++++++++++= ++++ 1 file changed, 93 insertions(+) create mode 100644 BaseTools/Scripts/FormatDosFiles.py diff --git a/BaseTools/Scripts/FormatDosFiles.py b/BaseTools/Scripts/Format= DosFiles.py new file mode 100644 index 0000000..c3a5476 --- /dev/null +++ b/BaseTools/Scripts/FormatDosFiles.py @@ -0,0 +1,93 @@ +# @file FormatDosFiles.py +# This script format the source files to follow dos style. +# It supports Python2.x and Python3.x both. +# +# Copyright (c) 2018, Intel Corporation. All rights reserved.
+# +# This program and the accompanying materials +# are licensed and made available under the terms and conditions of the B= SD License +# which accompanies this distribution. The full text of the license may = be found at +# http://opensource.org/licenses/bsd-license.php +# +# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, +# WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IM= PLIED. +# + +# +# Import Modules +# +import argparse +import os +import os.path +import re +import sys + +""" +difference of string between python2 and python3: + +there is a large difference of string in python2 and python3. + +in python2,there are two type string,unicode string (unicode type) and 8-b= it string (str type). + us =3D u"abcd", + unicode string,which is internally stored as unicode code point. + s =3D "abcd",s =3D b"abcd",s =3D r"abcd", + all of them are 8-bit string,which is internally stored as bytes. + +in python3,a new type called bytes replace 8-bit string,and str type is re= garded as unicode string. + s =3D "abcd", s =3D u"abcd", s =3D r"abcd", + all of them are str type,which is internally stored unicode code point. + bs =3D b"abcd", + bytes type,which is interally stored as bytes + +in python2 ,the both type string can be mixed use,but in python3 it could = not, +which means the pattern and content in re match should be the same type in= python3. +in function FormatFile,it read file in binary mode so that the content is = bytes type,so the pattern should also be bytes type. +As a result,I add encode() to make it compitable among python2 and python3. + =20 +difference of encode,decode in python2 and python3:=20 +the builtin function str.encode(encoding) and str.decode(encoding) are use= d for convert between 8-bit string and unicode string. + +in python2 + encode convert unicode type to str type.decode vice versa.default encodin= g is ascii. + for example: s =3D us.encode() + but if the us is str type,the code will also work.it will be firstly conv= ert to unicode type, + in this situation,the call equals s =3D us.decode().encode(). + +in python3 + encode convert str type to bytes type,decode vice versa.default encoding = is utf8. + fpr example: + bs =3D s.encode(),only str type has encode method,so that won't be used w= rongly.decode is the same. +=09 +in conclusion: + this code could work the same in python27 and python36 environment as far= as the re pattern satisfy ascii character set. + +""" +def FormatFiles(): + parser =3D argparse.ArgumentParser() + parser.add_argument('path', nargs=3D1, help=3D'The path for files to b= e converted.') + parser.add_argument('extensions', nargs=3D'+', help=3D'File extensions= filter. (Example: .txt .c .h)') + args =3D parser.parse_args() + filelist =3D [] + for dirpath, dirnames, filenames in os.walk(args.path[0]): + for filename in [f for f in filenames if any(f.endswith(ext) for e= xt in args.extensions)]: + filelist.append(os.path.join(dirpath, filename)) + for file in filelist: + fd =3D open(file, 'rb') + content =3D fd.read() + fd.close() + # Convert the line endings to CRLF + content =3D re.sub(r'([^\r])\n'.encode(), r'\1\r\n'.encode(), cont= ent) + content =3D re.sub(r'^\n'.encode(), r'\r\n'.encode(), content, fla= gs =3D re.MULTILINE) + # Add a new empty line if the file is not end with one + content =3D re.sub(r'([^\r\n])$'.encode(), r'\1\r\n'.encode(), con= tent) + # Remove trailing white spaces + content =3D re.sub(r'[ \t]+(\r\n)'.encode(), r'\1'.encode(), conte= nt, flags =3D re.MULTILINE) + # Replace '\t' with two spaces + content =3D re.sub('\t'.encode(), ' '.encode(), content) + fd =3D open(file, 'wb') + fd.write(content) + fd.close() + print(file) + +if __name__ =3D=3D "__main__": + sys.exit(FormatFiles()) \ No newline at end of file --=20 2.8.0.windows.1 _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel