From nobody Thu May 2 15:53:32 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+66562+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+66562+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1603446872; cv=none; d=zohomail.com; s=zohoarc; b=RFGcOMZPXKoDmlP20PUKPpLCJArcJ2WttlYY+OTnSrCTmWCZUTPAm/189a9VaU0E12IMvIU/GznOEEwryQZ/blwp0JvwjpHIaR6ISDCb/5OpZdBTosV0iWWM9/IQvD7hawqbQxrQJBg5mL9cHNxpwK+dIZOFuhB539BmJwKB/X0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1603446872; h=Content-Transfer-Encoding:Cc:Date:From:List-Id:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Sender:Subject:To; bh=kYUVY+waXFNjx2e0p+kC5UHENaHdq4mwf+RbavwCJe8=; b=SQl7WLUwgZ0BpdQuIE/6eb87eme936o9/mxQxICtDg66DPgaFaK6Kc6mjOUglcn3T/MQRuLUvwYZfmmCKaKsU6BVr6NEV0hefUsiP8iMJTcdVkf/kXPY4f1LXkDPmrj1txs72yXTH9Pc4ZHjvojb8suOP6aApsfH90Xul+g9iyw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+66562+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1603446872562765.4929996504985; Fri, 23 Oct 2020 02:54:32 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id ruxZYY1788612xPIpPKE4eed; Fri, 23 Oct 2020 02:54:32 -0700 X-Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mx.groups.io with SMTP id smtpd.web11.3383.1603422047220650603 for ; Thu, 22 Oct 2020 20:00:47 -0700 IronPort-SDR: MD8n95GOTKBAfl0TjR0dJO5nEwgYVYPeVVwG+BY27oOQIv4+lQJe6XwsT2BqexpGLn3SlRy/zj T63gBrtF3OIQ== X-IronPort-AV: E=McAfee;i="6000,8403,9782"; a="154579204" X-IronPort-AV: E=Sophos;i="5.77,404,1596524400"; d="scan'208";a="154579204" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Oct 2020 20:00:46 -0700 IronPort-SDR: OEYH6HQaEFNPDvugIuWTV0ccKFtpmuk0vTJQWDPUjmV00pNZX3lm40AIQY6D7BStJbLcBDwEgr xyvchBiEG4Cw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,404,1596524400"; d="scan'208";a="354304226" X-Received: from bob-desktop.ccr.corp.intel.com ([10.239.49.38]) by fmsmga002.fm.intel.com with ESMTP; 22 Oct 2020 20:00:44 -0700 From: mliang2x To: devel@edk2.groups.io Cc: Mingyue Liang , Bob Feng , Liming Gao , Yuwei Chen Subject: [edk2-devel] [PATCH] BaseTools: Limit command line length. Date: Fri, 23 Oct 2020 11:00:43 +0800 Message-Id: <20201023030043.1047-1-mingyuex.liang@intel.com> MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,mingyuex.liang@intel.com X-Gm-Message-State: tdkOxm33oXtbungtSG9pFNOBx1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1603446872; bh=QwTh20u+aP4unEcNm6GwBqpUrXpZy4kLbGKqHT0haNI=; h=Cc:Date:From:Reply-To:Subject:To; b=KDVm9Qts8wBtHPJRFILuR45V4hJU3q7McpGKEpFn79OEkH/58PUPour4z0Hm6oLTAf0 0P/3rFZj2ZtuSiR3RhNB1qebCy93Gh5QMaehNCljnsLEYZ8yGf9L2dEY84U0rO5enQYW3 seb9qAo0ssqCuFix0xbI98w5sii0IcYg290= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Type: text/plain; charset="utf-8" REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2528 Currently, the command line is too long because the CL command is followed by multiple C files. Therefore, the number of C files can be used to determine whether the command line needs to be written to the file. If the number of C files is greater than one, the command line is directly written to the file. On the contrary, whether to write to the file is determined by whether the length of the command line exceeds the limited length Documents. Signed-off-by: Mingyue Liang Cc: Bob Feng Cc: Liming Gao Cc: Yuwei Chen --- BaseTools/Source/Python/AutoGen/GenMake.py | 45 +++++++++++++++---- .../Source/Python/AutoGen/IncludesAutoGen.py | 13 +++++- 2 files changed, 49 insertions(+), 9 deletions(-) diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/= Python/AutoGen/GenMake.py index 0314d0ea34..0cb97dc18d 100755 --- a/BaseTools/Source/Python/AutoGen/GenMake.py +++ b/BaseTools/Source/Python/AutoGen/GenMake.py @@ -576,7 +576,8 @@ cleanlib: EdkLogger.error("build", AUTOGEN_ERROR, "Nothing to build", ExtraData=3D"[%s]" % str(MyAgo)) =20 - self.ProcessBuildTargetList() + self.ProcessBuildTargetList(MyAgo.OutputDir,ToolsDef) + self.ParserGenerateFfsCmd() =20 # Generate macros used to represent input files @@ -866,7 +867,6 @@ cleanlib: else: break SingleCommandLength +=3D len(Str) - if SingleCommandLength > GlobalData.gCommandMa= xLength: FlagDict[Tool]['Value'] =3D True =20 @@ -890,18 +890,18 @@ cleanlib: break else: break - if self._AutoGenObject.ToolChainFamily =3D=3D 'GCC= ': RespDict[Key] =3D Value.replace('\\', '/') else: RespDict[Key] =3D Value + for Target in BuildTargets: for i, SingleCommand in enumerate(BuildTargets= [Target].Commands): if FlagDict[Flag]['Macro'] in SingleComman= d: BuildTargets[Target].Commands[i] =3D S= ingleCommand.replace('$(INC)', '').replace(FlagDict[Flag]['Macro'], RespMac= ro) return RespDict =20 - def ProcessBuildTargetList(self): + def ProcessBuildTargetList(self, RespFile, ToolsDef): # # Search dependency file list for each source file # @@ -1002,6 +1002,7 @@ cleanlib: self.ObjTargetDict[T.Target.SubDir] =3D set() self.ObjTargetDict[T.Target.SubDir].add(NewFile) for Type in self._AutoGenObject.Targets: + resp_file_number =3D 0 for T in self._AutoGenObject.Targets[Type]: # Generate related macros if needed if T.GenFileListMacro and T.FileListMacro not in self.File= ListMacros: @@ -1043,7 +1044,8 @@ cleanlib: Deps.append("$(%s)" % T.ListFileMacro) =20 if self._AutoGenObject.BuildRuleFamily =3D=3D TAB_COMPILER= _MSFT and Type =3D=3D TAB_C_CODE_FILE: - T, CmdTarget, CmdTargetDict, CmdCppDict =3D self.Parse= rCCodeFile(T, Type, CmdSumDict, CmdTargetDict, CmdCppDict, DependencyDict) + T, CmdTarget, CmdTargetDict, CmdCppDict =3D self.Parse= rCCodeFile(T, Type, CmdSumDict, CmdTargetDict, CmdCppDict, DependencyDict, = RespFile, ToolsDef, resp_file_number) + resp_file_number +=3D 1 TargetDict =3D {"target": self.PlaceMacro(T.Target.Pat= h, self.Macros), "cmd": "\n\t".join(T.Commands),"deps": CCodeDeps} CmdLine =3D self._BUILD_TARGET_TEMPLATE.Replace(Target= Dict).rstrip().replace('\t$(OBJLIST', '$(OBJLIST') if T.Commands: @@ -1060,7 +1062,7 @@ cleanlib: AnnexeTargetDict =3D {"target": self.PlaceMacro(i.= Path, self.Macros), "cmd": "", "deps": self.PlaceMacro(T.Target.Path, self.= Macros)} self.BuildTargetList.append(self._BUILD_TARGET_TEM= PLATE.Replace(AnnexeTargetDict)) =20 - def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict, CmdCppDi= ct, DependencyDict): + def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict, CmdCppDi= ct, DependencyDict, RespFile, ToolsDef, resp_file_number): if not CmdSumDict: for item in self._AutoGenObject.Targets[Type]: CmdSumDict[item.Target.SubDir] =3D item.Target.BaseName @@ -1080,6 +1082,7 @@ cleanlib: CmdCppDict[item.Target.SubDir].append(Path) if T.Commands: CommandList =3D T.Commands[:] + SaveFilePath =3D os.path.join(RespFile, "cc_resp_%s.txt" % res= p_file_number) for Item in CommandList[:]: SingleCommandList =3D Item.split() if len(SingleCommandList) > 0 and self.CheckCCCmd(SingleCo= mmandList): @@ -1087,19 +1090,45 @@ cleanlib: if Temp.startswith('/Fo'): CmdSign =3D '%s%s' % (Temp.rsplit(TAB_SLASH, 1= )[0], TAB_SLASH) break - else: continue + else: + continue if CmdSign not in list(CmdTargetDict.keys()): CmdTargetDict[CmdSign] =3D Item.replace(Temp, CmdS= ign) else: CmdTargetDict[CmdSign] =3D "%s %s" % (CmdTargetDic= t[CmdSign], SingleCommandList[-1]) + Index =3D CommandList.index(Item) CommandList.pop(Index) if SingleCommandList[-1].endswith("%s%s.c" % (TAB_SLAS= H, CmdSumDict[CmdSign[3:].rsplit(TAB_SLASH, 1)[0]])): Cpplist =3D CmdCppDict[T.Target.SubDir] Cpplist.insert(0, '$(OBJLIST_%d): ' % list(self.Ob= jTargetDict.keys()).index(T.Target.SubDir)) - T.Commands[Index] =3D '%s\n\t%s' % (' \\\n\t'.join= (Cpplist), CmdTargetDict[CmdSign]) + cmdtargetlist =3D CmdTargetDict[CmdSign].split(" ") + # get Source files and Save resp file. + c_files =3D [] + cmds =3D [] + if cmdtargetlist: + for item in cmdtargetlist: + if item.startswith('$(') or item.startswit= h('/Fo') or item.startswith('"$('): + cmds.append(item) + if item.endswith('.c'): + c_files.append(item) + c_files.insert(0, " ") + if len(c_files) > 2: + SaveFileOnChange(SaveFilePath," ".join(c_f= iles), False) + T.Commands[Index] =3D '%s\n\t%s $(cc_resp_= %s)' % (' \\\n\t'.join(Cpplist), " ".join(cmds), resp_file_number) + ToolsDef.append("cc_resp_%s =3D @%s" % (re= sp_file_number, SaveFilePath)) + + elif len(CmdTargetDict[CmdSign]) > GlobalData.= gCommandMaxLength and len(c_files) <=3D2: + SaveFileOnChange(SaveFilePath, " ".join(c_= files), False) + T.Commands[Index] =3D '%s\n\t%s $(cc_resp_= %s)' % (' \\\n\t'.join(Cpplist), " ".join(cmds), resp_file_number) + ToolsDef.append("cc_resp_%s =3D @%s" % (re= sp_file_number, SaveFilePath)) + + else: + T.Commands[Index] =3D '%s\n\t%s' % (' \\\n= \t'.join(Cpplist), CmdTargetDict[CmdSign]) + else: T.Commands.pop(Index) + return T, CmdSumDict, CmdTargetDict, CmdCppDict =20 def CheckCCCmd(self, CommandList): diff --git a/BaseTools/Source/Python/AutoGen/IncludesAutoGen.py b/BaseTools= /Source/Python/AutoGen/IncludesAutoGen.py index 720d93395a..9f61d49b3a 100644 --- a/BaseTools/Source/Python/AutoGen/IncludesAutoGen.py +++ b/BaseTools/Source/Python/AutoGen/IncludesAutoGen.py @@ -203,7 +203,18 @@ ${END} cc_options =3D line[len(cc_cmd)+2:].split() else: cc_options =3D line[len(cc_cmd):].split() - SourceFileAbsPathMap =3D {os.path.basename(item):i= tem for item in cc_options if not item.startswith("/") and os.path.exists(i= tem)} + + for item in cc_options: + if not item.startswith("/"): + # if item.startswith("@"): + if item.endswith(".txt") and item.startswi= th("@"): + with open(item[1:], "r") as file: + source_files =3D file.readlines()[= 0].split() + SourceFileAbsPathMap =3D {os.path.= basename(file):file for file in source_files if os.path.exists(file)} + else: + if os.path.exists(item): + SourceFileAbsPathMap.update({os.pa= th.basename(item): item.strip()}) + if line in SourceFileAbsPathMap: current_source =3D line if current_source not in ModuleDepDict: --=20 2.28.0.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#66562): https://edk2.groups.io/g/devel/message/66562 Mute This Topic: https://groups.io/mt/77748878/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-