From nobody Thu May 2 23:09:21 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+58919+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58919+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1588981601; cv=none; d=zohomail.com; s=zohoarc; b=Q3Wm0URERglSFd+es7744MDKHHcsHTYaX3+lB7CA0Lsn+IHrNOyezXyx/fh9+syUA5Aw2XGlWxTX90iFGgS/s1Ss0UIwAyRGBRIREr/UK6+T2kU6f3vsE66zl8oYSKgWdnlH1bCJVBRawo9rFqReK+Ggq2EakGkKxDiuFgWxZMo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1588981601; h=Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=SJwzNWT2hGs6zB/QYCzYgHVRhrrNKnmELE/5zLdMkmw=; b=gas+nNc63x/EVWnCGDuHV8zyjqvMolnV393I52b/7Hx6HF6pQMb+QfSCeC+tBH1+tNCFxC4nZQbpmLTl7Lxnd3Fid/Yt+YprS+/5Xb1f9WZIzJV2bBCMTyxeUHw3xYWyqnPr8piFwvrVi3FJksIUMuZ71U9csTk6lq7Xsbls/qc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58919+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1588981601833683.0513228729749; Fri, 8 May 2020 16:46:41 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id Gdh7YY1788612x2zpWdUwB01; Fri, 08 May 2020 16:46:41 -0700 X-Received: from mga07.intel.com (mga07.intel.com []) by mx.groups.io with SMTP id smtpd.web11.1562.1588981599947016545 for ; Fri, 08 May 2020 16:46:40 -0700 IronPort-SDR: OhL5kw2TEMJSYcvbqT2PrRJbxWLDL1atUODQCPLP/2i9mGXzXGBhAg9mKm8QujT5gXYq/0RW11 ivv3KxoRFFLA== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 May 2020 16:46:39 -0700 IronPort-SDR: ZZJ0Xk/ovTuHBHJH+wnD0TaPHYA3hlOb77/62SdA3nGNVeGGKNHILS4qNs5IqIkwWoaEAGWc/U ld/vG4P3aO+w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,369,1583222400"; d="scan'208";a="264545623" X-Received: from aedesimo-desk.amr.corp.intel.com ([10.212.16.147]) by orsmga006.jf.intel.com with ESMTP; 08 May 2020 16:46:39 -0700 From: "Ashley E Desimone" To: devel@edk2.groups.io Cc: Nate DeSimone , Puja Pandya , Erik Bjorge , Bret Barkelew , Prince Agyeman Subject: [edk2-devel] [edk2-staging/EdkRepo] [PATCH 1/2] EdkRepo: Add verify_single_manifest() Date: Fri, 8 May 2020 16:46:35 -0700 Message-Id: <20200508234636.18268-2-ashley.e.desimone@intel.com> In-Reply-To: <20200508234636.18268-1-ashley.e.desimone@intel.com> References: <20200508234636.18268-1-ashley.e.desimone@intel.com> Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,ashley.e.desimone@intel.com X-Gm-Message-State: Tn4pQNjetsdLeoDGMxYL33kqx1787277AA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1588981601; bh=4tzMy9NcqFNOi6NKUQ7vQg08xKKw6bWIbq/NuxOc3nM=; h=Cc:Date:From:Reply-To:Subject:To; b=BuuhUylGWO+nBC9WiJ7ecYAKOAGfKcPUtNzSkp+JX3pIgTwictV17Xezsgl3XRbSLL1 Y0rf+46Vzk22/KSjazq61lFJ7ksGZNy/VSHy7ScoXtBpdT5XA0Q/jkNknhRt+EtuExNn5 MNMCXP/FuomNSah6ISCIsIig8SWhVDtQLUQ= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Erik Bjorge Cc: Bret Barkelew Cc: Prince Agyeman Reviewed-by: Nate DeSimone --- edkrepo/common/common_repo_functions.py | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/edkrepo/common/common_repo_functions.py b/edkrepo/common/commo= n_repo_functions.py index 485aa26..20dcb8a 100644 --- a/edkrepo/common/common_repo_functions.py +++ b/edkrepo/common/common_repo_functions.py @@ -426,6 +426,17 @@ def verify_manifest_data(global_manifest_directory, co= nfig, verbose=3DFalse, verif if verbose: print_manifest_errors(manifestfile_validation_data) =20 +def verify_single_manifest(cfg_file, manifest_repo, manifest_path, verbose= =3DFalse): + manifest =3D ManifestXml(manifest_path) + print(VERIFY_PROJ.format(manifest.project_info.codename)) + index_path =3D os.path.join(cfg_file.manifest_repo_abs_path(manifest_r= epo), CI_INDEX_FILE_NAME) + proj_val_data =3D validate_manifestfiles(index_path, [manifest_path]) + proj_val_error =3D get_manifest_validation_status(proj_val_data) + if proj_val_error: + if verbose: + print_manifest_errors(proj_val_data) + raise EdkrepoManifestInvalidException(VERIFY_PROJ_FAIL.format(mani= fest.project_info.codename)) + def sort_commits(manifest, workspace_path, max_commits=3DNone): colorama.init() repo_sources_to_log =3D manifest.get_repo_sources(manifest.general_con= fig.current_combo) --=20 2.16.2.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#58919): https://edk2.groups.io/g/devel/message/58919 Mute This Topic: https://groups.io/mt/74086767/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Thu May 2 23:09:21 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+58920+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58920+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1588981602; cv=none; d=zohomail.com; s=zohoarc; b=c9AIC518wf+Ha1c5bi89P1Z04StHvHvPhgTwlfWdk5DrZODAk0GetqVa+uKZCbw6BDPgsCMNWPJj/jOdkcKi7XrNU9qv4YR1mtwLahKu0sDgskShypJ9Vkw+OydrDLrpNOevEkcIHLpgsVLRLYyeGrSKgM7l9UQuepJeSst6bQk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1588981602; h=Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=toCaqAiIODoO0VntujPaHjTDWV1yK6oVzPkOI4cu3tQ=; b=je/cCTB5hEZBlGsLhVx5XUXqTjZ55WSZuz8B2rvthBsWjy7zxa8e079/g2wEXodisEzYEAWXdzuiY2WzLrmx7bFL22cX5jkMNaWSIBjGW5FAnVJPL8ynilPXXefeRF1YL6QaH9bxTU7jDiobNa5GjJfuUgTenqM426rj1QzFVc0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58920+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1588981602251360.79173419216943; Fri, 8 May 2020 16:46:42 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id ufc3YY1788612xayu7X1Xs6g; Fri, 08 May 2020 16:46:41 -0700 X-Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mx.groups.io with SMTP id smtpd.web11.1563.1588981600546055161 for ; Fri, 08 May 2020 16:46:40 -0700 IronPort-SDR: jXlfnBYVnIVbAKF6T7CxdO10y2LUGMh+smkUbxs4GR5Z0aeOWS+0Az7BMX0rnrb4Qzzr4b1tMy i8FDrnMiDLnA== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 May 2020 16:46:40 -0700 IronPort-SDR: ep3cbMfj+7l/rHzQJIodFlucEkl58lJ0aWjHyUwpC2qENvJuO9EEtm4HMbklOfnA9FoTcD5c5m i/fbUXGB6/IQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,369,1583222400"; d="scan'208";a="264545628" X-Received: from aedesimo-desk.amr.corp.intel.com ([10.212.16.147]) by orsmga006.jf.intel.com with ESMTP; 08 May 2020 16:46:39 -0700 From: "Ashley E Desimone" To: devel@edk2.groups.io Cc: Nate DeSimone , Puja Pandya , Erik Bjorge , Bret Barkelew , Prince Agyeman Subject: [edk2-devel] [edk2-staging/EdkRepo] [PATCH 2/2] EdkRepo: Add multiple manifest repository support to clone Date: Fri, 8 May 2020 16:46:36 -0700 Message-Id: <20200508234636.18268-3-ashley.e.desimone@intel.com> In-Reply-To: <20200508234636.18268-1-ashley.e.desimone@intel.com> References: <20200508234636.18268-1-ashley.e.desimone@intel.com> Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,ashley.e.desimone@intel.com X-Gm-Message-State: Wo51ALr6a1Fyj0frzbZqVFUVx1787277AA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1588981601; bh=BAfrnEUKQG24k5CjriIbiEdNCkTHtpe7aj5dS9ioiXs=; h=Cc:Date:From:Reply-To:Subject:To; b=l3qZd3mBQK4Fwn0qamo8UZ4WZWUs+zvzKbCv11CQXAO48y7WLbFln7IVBjl6khh7NIR k8fzqg+5V1TVh7opSbgKwDMRtY++3s6tcOr5GY/RDVKQ8S5QhP/HAhqsRvt2VCRSCgJpK r7WH0vFqI5oTUZmgagMBZe98WAvjraqKBAQ= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Update the clone command to support searching for the project to be cloned in all manifest repositories defined within both the edkrepo.cfg and the edkrepo_user.cfg files. Signed-off-by: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Erik Bjorge Cc: Bret Barkelew Cc: Prince Agyeman Reviewed-by: Nate DeSimone --- edkrepo/commands/clone_command.py | 36 +++++++++++++++++++++--------------- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/edkrepo/commands/clone_command.py b/edkrepo/commands/clone_com= mand.py index cd65fe6..b7f3158 100644 --- a/edkrepo/commands/clone_command.py +++ b/edkrepo/commands/clone_command.py @@ -11,16 +11,18 @@ import os import shutil =20 from edkrepo.commands.edkrepo_command import EdkrepoCommand -from edkrepo.commands.edkrepo_command import SubmoduleSkipArgument +from edkrepo.commands.edkrepo_command import SubmoduleSkipArgument, Source= ManifestRepoArgument import edkrepo.commands.arguments.clone_args as arguments -from edkrepo.common.common_repo_functions import pull_latest_manifest_repo= , clone_repos, sparse_checkout, verify_manifest_data -from edkrepo.common.common_repo_functions import update_editor_config +from edkrepo.common.common_repo_functions import clone_repos, sparse_check= out, verify_single_manifest +from edkrepo.common.common_repo_functions import update_editor_config, com= binations_in_manifest from edkrepo.common.common_repo_functions import write_included_config, wr= ite_conditional_include -from edkrepo.common.common_repo_functions import find_project_in_index, co= mbinations_in_manifest from edkrepo.common.edkrepo_exception import EdkrepoInvalidParametersExcep= tion, EdkrepoManifestInvalidException from edkrepo.common.humble import CLONE_INVALID_WORKSPACE, CLONE_INVALID_P= ROJECT_ARG, CLONE_INVALID_COMBO_ARG from edkrepo.common.humble import SPARSE_CHECKOUT, CLONE_INVALID_LOCAL_ROO= TS from edkrepo.common.workspace_maintenance.workspace_maintenance import cas= e_insensitive_single_match +from edkrepo.common.workspace_maintenance.manifest_repos_maintenance impor= t pull_all_manifest_repos, find_project_in_all_indices +from edkrepo.common.workspace_maintenance.manifest_repos_maintenance impor= t list_available_manifest_repos +from edkrepo.common.workspace_maintenance.humble.manifest_repos_maintenanc= e_humble import PROJ_NOT_IN_REPO, SOURCE_MANIFEST_REPO_NOT_FOUND from edkrepo_manifest_parser.edk_manifest import CiIndexXml, ManifestXml =20 =20 @@ -58,12 +60,14 @@ class CloneCommand(EdkrepoCommand): 'required': False, 'help-text': arguments.NO_SPARSE_HELP}) args.append(SubmoduleSkipArgument) + args.append(SourceManifestRepoArgument) return metadata =20 =20 def run_command(self, args, config): - pull_latest_manifest_repo(args, config) + pull_all_manifest_repos(config['cfg_file'], config['user_cfg_file'= ], False) update_editor_config(config) + name_or_manifest =3D args.ProjectNameOrManifestFile workspace_dir =3D args.Workspace # Check to see if requested workspace exists. If not create it. If= so check for empty @@ -77,17 +81,19 @@ class CloneCommand(EdkrepoCommand): if not os.path.isdir(workspace_dir): os.makedirs(workspace_dir) =20 - # Get path to global manifest file - global_manifest_directory =3D config['cfg_file'].manifest_repo_abs= _local_path - - # Verify the manifest directory at this point. - verify_manifest_data(global_manifest_directory, config, verbose=3D= args.verbose, verify_proj=3Dname_or_manifest) - - # Now try to find the correct manifest file. - index_path =3D os.path.join(global_manifest_directory, 'CiIndex.xm= l') - ci_index_xml =3D CiIndexXml(index_path) + cfg, user_cfg, conflicts =3D list_available_manifest_repos(config[= 'cfg_file'], config['user_cfg_file']) + manifest_repo, source_cfg, global_manifest_path =3D find_project_i= n_all_indices(args.ProjectNameOrManifestFile, + config['= cfg_file'], + config['= user_cfg_file'], + PROJ_NOT= _IN_REPO.format(args.ProjectNameOrManifestFile), + SOURCE_M= ANIFEST_REPO_NOT_FOUND.format(args.ProjectNameOrManifestFile), + args.sou= rce_manifest_repo) =20 - global_manifest_path =3D find_project_in_index(name_or_manifest, c= i_index_xml, global_manifest_directory, CLONE_INVALID_PROJECT_ARG) + # If this manifest is in a defined manifest repository validate th= e manifest within the manifest repo + if manifest_repo in cfg: + verify_single_manifest(config['cfg_file'], manifest_repo, glob= al_manifest_path) + elif manifest_repo in user_cfg: + verify_single_manifest(config['user_cfg_file'], manifest_repo,= global_manifest_path) =20 # Copy project manifest to local manifest dir and rename it Manife= st.xml. local_manifest_dir =3D os.path.join(workspace_dir, "repo") --=20 2.16.2.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#58920): https://edk2.groups.io/g/devel/message/58920 Mute This Topic: https://groups.io/mt/74086768/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-