From nobody Sun Apr 28 09:21:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+58489+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58489+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1588309991; cv=none; d=zohomail.com; s=zohoarc; b=jVVoYzPcJedrgq+QcqtrXsKTivNUBlAko6k1hEp2Dc44rIyqXO3V2t5gaSL0CiKorZGeRa5KxDnUi/dYmyyP7T98jYsQTcRkRa0/+9xKdgiK+mJET874ruEPLPfXV7txjH3vPlPAXkjhlUGtCnY0E+jwwST1D8NhYN0EplQnzzc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1588309991; h=Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=/k17BTZXMnfNTqPZ1HfEp1szQMOLbT7nd1ulUcc3obM=; b=mNLQR+gPC6GQXWut8MwlDPT7KlBe686n0ucpOrzdAcxf/UmYYw252VEO6IZ7KdZF8pUGFyr0WOiXvNjoluNUb0MVnrVssPCuwdBlReNKcxYzHcSli7BYcvW9SzFrHJr/ssAPGALyTrlg55xfwpt93bccQnxILIO+/lEfV8gh2Mk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58489+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1588309991753979.4780462491045; Thu, 30 Apr 2020 22:13:11 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id h8QvYY1788612x8b70MmnNfa; Thu, 30 Apr 2020 22:13:11 -0700 X-Received: from mga01.intel.com (mga01.intel.com []) by mx.groups.io with SMTP id smtpd.web11.4932.1588309989257688147 for ; Thu, 30 Apr 2020 22:13:09 -0700 IronPort-SDR: CfW7dRopnc5DHut0ETT3XJD1yC98Ohn+6FOFKSH2Emli3eyc6deU2PKoNWibgpd9YUqOf36/fg 5LQQ27JpEjrA== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Apr 2020 22:13:08 -0700 IronPort-SDR: +g8RnGUASIWhy6SyW4j5O3qhH/Cznk5nb31XMMhf+B+k1BbUicV0PNRXPgMHYRe8rrGpHqzaGL TpAjhmPGrHRQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,338,1583222400"; d="scan'208";a="283096075" X-Received: from aedesimo-desk.amr.corp.intel.com ([10.212.210.118]) by fmsmga004.fm.intel.com with ESMTP; 30 Apr 2020 22:12:48 -0700 From: "Ashley E Desimone" To: devel@edk2.groups.io Cc: Nate DeSimone , Puja Pandya , Erik Bjorge , Bret Barkelew , Prince Agyeman Subject: [edk2-devel] [edk2-staging/EdkRepo] [PATCH V2 1/8] EdkRepo: Add check for conflicting/duplicated manifest repo definitions Date: Thu, 30 Apr 2020 22:12:32 -0700 Message-Id: <20200501051239.13976-2-ashley.e.desimone@intel.com> In-Reply-To: <20200501051239.13976-1-ashley.e.desimone@intel.com> References: <20200501051239.13976-1-ashley.e.desimone@intel.com> Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,ashley.e.desimone@intel.com X-Gm-Message-State: 3dNKcEpehr8P2xV5VWRktKVlx1787277AA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1588309991; bh=COUg49fKLMS0GnhjrfouEv9E5ZmsLpIH6v+54cWVgl4=; h=Cc:Date:From:Reply-To:Subject:To; b=ZC+yy+gQCDHiQi3f1vRYcKbSzfCI4U4BL74rftsAKUt5WooIGjwLaKsXENeordz9wbK yIE7q+yPmH4qES1p29t+0GJiV6cwkZTOfBUOqZSqpu9TVVCp58wDK+3E1LJHpdefuhLel 4K9ddotpEl2NADDWygv6kmN0Lohuizn80eA= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add a functions to check for conflicting or duplicated manifest repository definitions in the edkrepo.cfg and the edkrepo_user.cfg files. Two manifest repositories definitions are considered conflicting if they have the same name and at least one of URL, branch or local path differ. Two manifest repository definitions are considered duplicates if the name, URL, branch and local path are the same. Signed-off-by: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Erik Bjorge Cc: Bret Barkelew Cc: Prince Agyeman Reviewed-by: Nate DeSimone --- .../manifest_repos_maintenance.py | 25 ++++++++++++++++++= ++++ 1 file changed, 25 insertions(+) diff --git a/edkrepo/common/workspace_maintenance/manifest_repos_maintenanc= e.py b/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py index 6e26d4f..ad6ddbc 100644 --- a/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py +++ b/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py @@ -57,3 +57,28 @@ def pull_single_manifest_repo(url, branch, local_path, r= eset_hard=3DFalse): print (humble.CLONE_SINGLE_MAN_REPO.format(local_path, url)) repo =3D Repo.clone_from(url, local_path, progress=3DGitProgre= ssHandler(), branch=3Dbranch) =20 +def detect_man_repo_conflicts_duplicates(edkrepo_cfg, edkrepo_user_cfg): + ''' + Determines whether there is are conflicting or duplicated manifest + repositories listed in the edkrepo.cfg and the edkrepo_user.cfg. + ''' + conflicts =3D [] + duplicates =3D [] + if not edkrepo_user_cfg.manifest_repo_list: + return conflicts, duplicates + else: + config_repos =3D set(edkrepo_cfg.manifest_repo_list) + user_cfg_repos =3D set(edkrepo_user_cfg.manifest_repo_list) + if config_repos.isdisjoint(user_cfg_repos): + return conflicts, duplicates + else: + for repo in config_repos.intersection(user_cfg_repos): + if edkrepo_cfg.get_manifest_repo_url(repo) !=3D edkrepo_user_c= fg.get_manifest_repo_url(repo): + conflicts.append(repo) + elif edkrepo_cfg.get_manifest_repo_branch(repo) !=3D edkrepo_u= ser_cfg.get_manifest_repo_branch(repo): + conflicts.append(repo) + elif edkrepo_cfg.get_manifest_repo_local_path(repo) !=3D edkre= po_user_cfg.get_manifest_repo_local_path(repo): + conflicts.append(repo) + else: + duplicates.append(repo) + return conflicts, duplicates --=20 2.16.2.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#58489): https://edk2.groups.io/g/devel/message/58489 Mute This Topic: https://groups.io/mt/73391733/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Sun Apr 28 09:21:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+58490+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58490+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1588309990; cv=none; d=zohomail.com; s=zohoarc; b=U/Up2LmWt0D2mgZ3LBiJ6wO/EqTIXqrANvJFV58EhkVffb3Bbc0Avt7y29qlFpWMXYYzOQ3m830Q/4IVUF4biOaly9JlT276hsiIug50KynrYCEquUQK/cWRxDxyC1NPovWc58qrhY67+980jpgVe9pE1qeZOjHlp7k+ciFN7RM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1588309990; h=Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=1jz7Uo8I6H0c3lEgHq92ZmOqKyusLBHcQbQcYiISICw=; b=jwX1nCDe4z5ItRfVH3MSwSM8d/Cp/m+FY4+eMY2PncBmGmDSmR11W2w16c+FprcNgp6hlkCR4DE5GzUXvVEJqEQW+ZAPjU+7drN75HA/U7R9+NqUF5cHW6JkMY5xBpzoQS67vGXA+FrNbfqVWxbr4DuHcUjBGwN97hojEUUAkkE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58490+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 15883099906441022.5070952803366; Thu, 30 Apr 2020 22:13:10 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id SV9QYY1788612xPqgZhCt6eE; Thu, 30 Apr 2020 22:13:10 -0700 X-Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mx.groups.io with SMTP id smtpd.web10.4894.1588309989753910554 for ; Thu, 30 Apr 2020 22:13:09 -0700 IronPort-SDR: 3xxxsGlTMem4z0U5oNu5RKyaHmbk4bbB0RO+RYzc0oRp19H5UN1s1gzJcD/ExZinSF25tc/YEX QSlEl49pbjRA== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Apr 2020 22:13:08 -0700 IronPort-SDR: UIhhpO+PYm0gGh+jAMNQS+HhncjT0BjgIGAZ2YBDDghdgB8ibTflSofc8mZQxGGEucrWkHUkRQ RdSd+0gPZ96A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,338,1583222400"; d="scan'208";a="283096078" X-Received: from aedesimo-desk.amr.corp.intel.com ([10.212.210.118]) by fmsmga004.fm.intel.com with ESMTP; 30 Apr 2020 22:12:48 -0700 From: "Ashley E Desimone" To: devel@edk2.groups.io Cc: Nate DeSimone , Puja Pandya , Erik Bjorge , Bret Barkelew , Prince Agyeman Subject: [edk2-devel] [edk2-staging/EdkRepo] [PATCH V2 2/8] EdkRepo: Add downloading all available manifest repositories Date: Thu, 30 Apr 2020 22:12:33 -0700 Message-Id: <20200501051239.13976-3-ashley.e.desimone@intel.com> In-Reply-To: <20200501051239.13976-1-ashley.e.desimone@intel.com> References: <20200501051239.13976-1-ashley.e.desimone@intel.com> Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,ashley.e.desimone@intel.com X-Gm-Message-State: ZJcjg0ig7PZb0xg9s9vC5SJzx1787277AA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1588309990; bh=mxwOfPcGTSJkRh51zXQB7ElZpNFWBOVHwalJjOnMQ8w=; h=Cc:Date:From:Reply-To:Subject:To; b=tOHLKFJfn5kNq4mm78nbjFaKopxMUWvRe9+WV+ICwWCzOGJRnGgsand8SwVAHo05ddT u/PGXzLDXWI6y6WXth1ZIsyxyD72/wjOyHrtGhuzzsUQXvdaRxE9lBezl3Y6MjkETeM8s dwgCrYDJ5dYpwMt1HrkljURTlm46pi+NuSs= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add a function that will download all available manifest repositories defined in either the edkrepo.cfg or the edkrepo_user.cfg Signed-off-by: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Erik Bjorge Cc: Bret Barkelew Cc: Prince Agyeman Reviewed-by: Nate DeSimone --- .../humble/manifest_repos_maintenance_humble.py | 4 +++ .../manifest_repos_maintenance.py | 38 ++++++++++++++++++= ++++ 2 files changed, 42 insertions(+) diff --git a/edkrepo/common/workspace_maintenance/humble/manifest_repos_mai= ntenance_humble.py b/edkrepo/common/workspace_maintenance/humble/manifest_r= epos_maintenance_humble.py index 440fd8a..e592f19 100644 --- a/edkrepo/common/workspace_maintenance/humble/manifest_repos_maintenanc= e_humble.py +++ b/edkrepo/common/workspace_maintenance/humble/manifest_repos_maintenanc= e_humble.py @@ -21,3 +21,7 @@ SINGLE_MAN_REPO_NOT_CFG_BRANCH =3D ('The current active b= ranch, {}, is not the ' 'specified branch for global manifst rep= ository: {}') SINGLE_MAN_REPO_CHECKOUT_CFG_BRANCH =3D 'Checking out the specified branch= : {} prior to syncing' SINGLE_MAN_REPO_MOVED =3D '{}{}WARNING:{}{} The global manifest repository= has moved. Backing up previous global manifest repository to: {{}}{}\n'.fo= rmat(Style.BRIGHT, Fore.RED, Style.RESET_ALL, Fore.RED, Style.RESET_ALL) +CONFLICT_NO_CLONE =3D ('The definition of global manifest repository, {}, ' + 'in the edkrepo_user.cfg does not match the definitio= n in the edkrepo.cfg. ' + 'This global manifest repository will not be download= ed or updated. ' + 'Resolve the conflict and then re-run the failed oper= ation') diff --git a/edkrepo/common/workspace_maintenance/manifest_repos_maintenanc= e.py b/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py index ad6ddbc..24ad76a 100644 --- a/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py +++ b/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py @@ -57,6 +57,44 @@ def pull_single_manifest_repo(url, branch, local_path, r= eset_hard=3DFalse): print (humble.CLONE_SINGLE_MAN_REPO.format(local_path, url)) repo =3D Repo.clone_from(url, local_path, progress=3DGitProgre= ssHandler(), branch=3Dbranch) =20 +def pull_all_manifest_repos(edkrepo_cfg, edkrepo_user_cfg, reset_hard=3DFa= lse): + ''' + Clones or syncs all global manifest repositories defined in both the + edkrepo_cfg and the edkrepo_user.cfg) + ''' + cfg_man_repos =3D [] + user_cfg_man_repos =3D [] + conflicts, duplicates =3D detect_man_repo_conflicts_duplicates(edkrepo= _cfg, edkrepo_user_cfg) + if not conflicts and not duplicates: + cfg_man_repos.extend(edkrepo_cfg.manifest_repo_list) + user_cfg_man_repos.extend(edkrepo_user_cfg.manifest_repo_list) + elif conflicts: + for conflict in conflicts: + # In the case of a conflict do not pull conflicting repo + print(humble.CONFLICT_NO_CLONE.format(conflict)) + cfg_man_repos.extend(edkrepo_cfg.manifest_repo_list) + cfg_man_repos.remove(conflict) + user_cfg_man_repos.extend(edkrepo_user_cfg.manifest_repo_list) + user_cfg_man_repos.remove(conflict) + elif duplicates: + for duplicate in duplicates: + # the duplicate needs to be ignored in on of the repo lists so= it is + # not cloned/pulled twice + cfg_man_repos.extend(edkrepo_cfg.manifest_repo_list) + user_cfg_man_repos.extend(edkrepo_user_cfg.manifest_repo_list) + user_cfg_man_repos.remove(conflict) + for repo in cfg_man_repos: + pull_single_manifest_repo(edkrepo_cfg.get_manifest_repo_url(repo), + edkrepo_cfg.get_manifest_repo_branch(rep= o), + edkrepo_cfg.get_manifest_repo_local_path= (repo), + reset_hard) + for repo in user_cfg_man_repos: + pull_single_manifest_repo(edkrepo_user_cfg.get_manifest_repo_url(r= epo), + edkrepo_user_cfg.get_manifest_repo_branc= h(repo), + edkrepo_user_cfg.get_manifest_repo_local= _path(repo), + reset_hard) + + def detect_man_repo_conflicts_duplicates(edkrepo_cfg, edkrepo_user_cfg): ''' Determines whether there is are conflicting or duplicated manifest --=20 2.16.2.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#58490): https://edk2.groups.io/g/devel/message/58490 Mute This Topic: https://groups.io/mt/73391734/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Sun Apr 28 09:21:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+58491+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58491+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1588309992; cv=none; d=zohomail.com; s=zohoarc; b=oAML1W8qHn0GUBFNA67iieEw8mX78yj9Zas4qV1aeg6pWZHTb2Dfpjvg3tUP0+8cZrWuCLPIoCTsqJQ2XW1zzpQZOO7/8PIIyNdw3YfcizKjdVT0zTqsGFJcorb6cBn+pfI/WKTEKz1ptS+NswHpZshMTzfG5LgnrJtw1MmgVFQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1588309992; h=Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=e3cuwrcLzZ8IT7uzuk1k/WVaGrtB297sgiRyG0QTxF0=; b=WVvU1NoW7QLEq0gYB6n6VfwheMYqDNCOcuCtk6vsH2FXQEWhRMSMlfvAeLcDTFYW42lgzj+o79K/fu2diAXRlLHfF5lxEXO/vr4rSYDVjOywiJPK4gAf7YHal/eTr6tCqLPR6CMfvbPxFa86O1WyGMsSp3PuNtqeMXfAfjAIruU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58491+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1588309992111377.03891453485244; Thu, 30 Apr 2020 22:13:12 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id wiYfYY1788612xs1KdBKaZUE; Thu, 30 Apr 2020 22:13:11 -0700 X-Received: from mga01.intel.com (mga01.intel.com []) by mx.groups.io with SMTP id smtpd.web11.4932.1588309989257688147 for ; Thu, 30 Apr 2020 22:13:10 -0700 IronPort-SDR: T1ffp/apIv/vReFOhi2GId5ihFvOMBb+9gcu+V4zYMenQrxHiS4dJxhCT1ojvCz803N/AKPJuP ixbcvF7I/R1w== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Apr 2020 22:13:08 -0700 IronPort-SDR: wAcecC9yZGufGBZ1DyTch510uO85Et2pO7GexBI67AkK5JMulTa2U512os5kUrrld/BIUOHyTg EXZmDfwVk7vg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,338,1583222400"; d="scan'208";a="283096081" X-Received: from aedesimo-desk.amr.corp.intel.com ([10.212.210.118]) by fmsmga004.fm.intel.com with ESMTP; 30 Apr 2020 22:12:48 -0700 From: "Ashley E Desimone" To: devel@edk2.groups.io Cc: Nate DeSimone , Puja Pandya , Erik Bjorge , Bret Barkelew , Prince Agyeman Subject: [edk2-devel] [edk2-staging/EdkRepo] [PATCH 3/8] EdkRepo: Add optional field to edkrepo_manifst to track the source manifest repo Date: Thu, 30 Apr 2020 22:12:34 -0700 Message-Id: <20200501051239.13976-4-ashley.e.desimone@intel.com> In-Reply-To: <20200501051239.13976-1-ashley.e.desimone@intel.com> References: <20200501051239.13976-1-ashley.e.desimone@intel.com> Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,ashley.e.desimone@intel.com X-Gm-Message-State: rebuzOP60vLLGN2MCw956WB5x1787277AA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1588309991; bh=hIMddyZl5V3/786f7zcoYfsgECSEV/iNAuywS4gFeQw=; h=Cc:Date:From:Reply-To:Subject:To; b=IXSpnpsAEwcEANBbbulM/GPAgVAoLL7bjXEKFVxplMB3DhyT/0Af7qtgsG1LfjZEVWA uQekfz8uhtIcN+W9MFhQ4U7/rsO0NdgbWmmJKj2z5+iW6SR99BviDlIceAo7wzSmmf6Bf JIOpasDjxHYfr+bT2vTz7GILn7sJFsEGWnk= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add the SourceManifestRepository to the edkrepo manfiest general config for use by edkrepo to track the source manifest repository for the workspac= e. Signed-off-by: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Erik Bjorge Cc: Bret Barkelew Cc: Prince Agyeman Reviewed-by: Nate DeSimone --- edkrepo_manifest_parser/edk_manifest.py | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/edkrepo_manifest_parser/edk_manifest.py b/edkrepo_manifest_par= ser/edk_manifest.py index 080448f..dcf9c29 100644 --- a/edkrepo_manifest_parser/edk_manifest.py +++ b/edkrepo_manifest_parser/edk_manifest.py @@ -21,7 +21,7 @@ import copy # All the namedtuple data structures that consumers of this module will ne= ed. # ProjectInfo =3D namedtuple('ProjectInfo', ['codename', 'description', 'dev= _leads', 'reviewers', 'org', 'short_name']) -GeneralConfig =3D namedtuple('GeneralConfig', ['default_combo', 'current_c= ombo', 'pin_path']) +GeneralConfig =3D namedtuple('GeneralConfig', ['default_combo', 'current_c= ombo', 'pin_path', 'source_man_repo']) RemoteRepo =3D namedtuple('RemoteRepo', ['name', 'url']) RepoHook =3D namedtuple('RepoHook', ['source', 'dest_path', 'dest_file', '= remote_url']) Combination =3D namedtuple('Combination', ['name', 'description']) @@ -406,6 +406,24 @@ class ManifestXml(BaseXmlHelper): self._tree.write(filename) self.__general_config.current_combo =3D combo_name =20 + def write_source_manifest_repo(self, manifest_repo, filename=3DNone): + ''' + Writes the name of the source manifest repository to the + general config sections of the manifest file. + ''' + if filename is None: + filename =3D self._fileref + subroot =3D self._tree.find('GeneralConfig') + if subroot is None: + raise KeyError(GENERAL_CONFIG_MISSING_ERROR) + + element =3D subroot.find('SourceManifestRepository') + if element is None: + element =3D ET.SubElement(subroot, 'SourceManifestRepository') + element.attrib['manifest_repo'] =3D manifest_repo + self._tree.write(filename) + self.__general_config.source_man_repo =3D manifest_repo + def generate_pin_xml(self, description, combo_name, repo_source_list, = filename=3DNone): =20 pin_tree =3D ET.ElementTree(ET.Element('Pin')) @@ -605,10 +623,14 @@ class _GeneralConfig(): self.curr_combo =3D element.find('CurrentClonedCombo').attrib[= 'combination'] except: self.curr_combo =3D None + try: + self.source_man_repo =3D element.find('SourceManifestRepositor= y').attrib['manifest_repo'] + except: + self.source_man_repo =3D None =20 @property def tuple(self): - return GeneralConfig(self.default_combo, self.curr_combo, self.pin= _path) + return GeneralConfig(self.default_combo, self.curr_combo, self.pin= _path, self.source_man_repo) =20 class _RemoteRepo(): def __init__(self, element): --=20 2.16.2.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#58491): https://edk2.groups.io/g/devel/message/58491 Mute This Topic: https://groups.io/mt/73391735/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Sun Apr 28 09:21:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+58492+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58492+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1588309992; cv=none; d=zohomail.com; s=zohoarc; b=bMI/iKqvNCZIdb6QLzo4STGg9nTGiuZI9t4SnSLlkd7h8V7Dlr7tvG7iMDBS2sy9wNliznHFh7NLJNOUnxqOe0CnV+YHeBUlluBnzcscfiLazwKHhcNaTtzuSPIPfEYjjnt7MiZsYKtfq6dCG1WV462w7658/Hlz3eQ+F6Whad0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1588309992; h=Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=GRRQoybmcr1XnRArs/q+QdGaESSEIlwxr73FNwaFpvI=; b=U9Ew5g7U1n73ZA0J9Tj0Wzo0taZxd3GAPFxchCVlvLXwdZjRf+8wYkeyCgJ6WR/Orz3g8/GYY5xvJdTqZp5i7EVt41YfLB0ssf1jPAZvLe0AXJ1FkJqTpRmT9ysW83gMqS6THodVEQWYN2DXGRDDogkkULBGSidooZrt66llcCc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58492+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1588309992525210.03721960733765; Thu, 30 Apr 2020 22:13:12 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id 9x9ZYY1788612xYnOejYtWKU; Thu, 30 Apr 2020 22:13:12 -0700 X-Received: from mga01.intel.com (mga01.intel.com []) by mx.groups.io with SMTP id smtpd.web10.4894.1588309989753910554 for ; Thu, 30 Apr 2020 22:13:10 -0700 IronPort-SDR: EaThAuhfy8/CmvC/sypI9cuKB12tE5eLs5zkRmKDY05IzIIfqH49ovjd41hxonXSNh/HCMP6mh Das2WA+q763Q== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Apr 2020 22:13:08 -0700 IronPort-SDR: GnNLlZj5I4jZj6Zczz5esXYa3Hupu6BzSqDQgWE+dCP3f0uT2ChiUDQGBHfYgGHo9dSrEvlKuA /s5RSmWn5REw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,338,1583222400"; d="scan'208";a="283096084" X-Received: from aedesimo-desk.amr.corp.intel.com ([10.212.210.118]) by fmsmga004.fm.intel.com with ESMTP; 30 Apr 2020 22:12:49 -0700 From: "Ashley E Desimone" To: devel@edk2.groups.io Cc: Nate DeSimone , Puja Pandya , Erik Bjorge , Bret Barkelew , Prince Agyeman Subject: [edk2-devel] [edk2-staging/EdkRepo] [PATCH V2 4/8] EdkRepo: Add list_available_manifest_repos() Date: Thu, 30 Apr 2020 22:12:35 -0700 Message-Id: <20200501051239.13976-5-ashley.e.desimone@intel.com> In-Reply-To: <20200501051239.13976-1-ashley.e.desimone@intel.com> References: <20200501051239.13976-1-ashley.e.desimone@intel.com> Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,ashley.e.desimone@intel.com X-Gm-Message-State: jYaKbBhC7SUqUPWVhYvdIf6Gx1787277AA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1588309992; bh=5BbEKeWnBepSM0L5qy+buxQroYICF36uXYicA7M5e1M=; h=Cc:Date:From:Reply-To:Subject:To; b=AKtIm4dBlFETDlAJnHOw9C02EAEGZFctqCE6TIyv10UYAXvKhc2HKGH/X4IFvGQRmEP kQBwDj85NEU19pIN9tGwFg6kTr4oWMlz7oLafQypVT4lZOPiRiD3R/MdmTYOmkHOjKaSa LawO4fQi9pesgd1YEnjg2RLAV4I2t/n3dAM= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add the ability to calculate a list of available manifest repositories from the contents of the edkrepo.cfg and the edkrepo_user.cfg files. Signed-off-by: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Erik Bjorge Cc: Bret Barkelew Cc: Prince Agyeman Reviewed-by: Nate DeSimone --- .../manifest_repos_maintenance.py | 53 ++++++++++++++----= ---- 1 file changed, 34 insertions(+), 19 deletions(-) diff --git a/edkrepo/common/workspace_maintenance/manifest_repos_maintenanc= e.py b/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py index 24ad76a..4bded46 100644 --- a/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py +++ b/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py @@ -64,25 +64,10 @@ def pull_all_manifest_repos(edkrepo_cfg, edkrepo_user_c= fg, reset_hard=3DFalse): ''' cfg_man_repos =3D [] user_cfg_man_repos =3D [] - conflicts, duplicates =3D detect_man_repo_conflicts_duplicates(edkrepo= _cfg, edkrepo_user_cfg) - if not conflicts and not duplicates: - cfg_man_repos.extend(edkrepo_cfg.manifest_repo_list) - user_cfg_man_repos.extend(edkrepo_user_cfg.manifest_repo_list) - elif conflicts: - for conflict in conflicts: - # In the case of a conflict do not pull conflicting repo - print(humble.CONFLICT_NO_CLONE.format(conflict)) - cfg_man_repos.extend(edkrepo_cfg.manifest_repo_list) - cfg_man_repos.remove(conflict) - user_cfg_man_repos.extend(edkrepo_user_cfg.manifest_repo_list) - user_cfg_man_repos.remove(conflict) - elif duplicates: - for duplicate in duplicates: - # the duplicate needs to be ignored in on of the repo lists so= it is - # not cloned/pulled twice - cfg_man_repos.extend(edkrepo_cfg.manifest_repo_list) - user_cfg_man_repos.extend(edkrepo_user_cfg.manifest_repo_list) - user_cfg_man_repos.remove(conflict) + conflicts =3D [] + cfg_man_repos, user_cfg_man_repos, conflicts =3D list_available_man_re= pos(edkrepo_cfg, edkrepo_user_cfg) + for conflict in conflicts: + print(humble.CONFLICT_NO_CLONE.format(conflict)) for repo in cfg_man_repos: pull_single_manifest_repo(edkrepo_cfg.get_manifest_repo_url(repo), edkrepo_cfg.get_manifest_repo_branch(rep= o), @@ -120,3 +105,33 @@ def detect_man_repo_conflicts_duplicates(edkrepo_cfg, = edkrepo_user_cfg): else: duplicates.append(repo) return conflicts, duplicates + +def list_available_man_repos(edkrepo_cfg, edkrepo_user_cfg): + ''' + Checks for conflicts/duplicates within all manifest repositories defin= ed in + both the edkrepo.cfg and the edkrepo_user.cfg and resturns a list of a= vailable + manifest_repos for each and a list of conflicting manifest repository = entries. + ''' + cfg_man_repos =3D [] + user_cfg_man_repos =3D [] + conflicts, duplicates =3D detect_man_repo_conflicts_duplicates(edkrepo= _cfg, edkrepo_user_cfg) + if not conflicts and not duplicates: + cfg_man_repos.extend(edkrepo_cfg.manifest_repo_list) + user_cfg_man_repos.extend(edkrepo_user_cfg.manifest_repo_list) + elif conflicts: + for conflict in conflicts: + # In the case of a conflict do not pull conflicting repo + cfg_man_repos.extend(edkrepo_cfg.manifest_repo_list) + cfg_man_repos.remove(conflict) + user_cfg_man_repos.extend(edkrepo_user_cfg.manifest_repo_list) + user_cfg_man_repos.remove(conflict) + elif duplicates: + for duplicate in duplicates: + # the duplicate needs to be ignored in on of the repo lists so= it is + # not cloned/pulled twice + cfg_man_repos.extend(edkrepo_cfg.manifest_repo_list) + user_cfg_man_repos.extend(edkrepo_user_cfg.manifest_repo_list) + user_cfg_man_repos.remove(duplicate) + return cfg_man_repos, user_cfg_man_repos, conflicts + + --=20 2.16.2.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#58492): https://edk2.groups.io/g/devel/message/58492 Mute This Topic: https://groups.io/mt/73391736/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Sun Apr 28 09:21:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+58493+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58493+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1588309992; cv=none; d=zohomail.com; s=zohoarc; b=Dh3y2LmO/vlTe8rL5uI7SilGuchjFnSu++uQHzhR0gkm6XK9QEV4lWuvqeQq8GhFoRZNz79JAhEfKrXbbYYVhqmxtUrry2/lq04B8I009mOmV36V86stQMSAx9MnLukFfhjloVrpt5INyfqTqu5iYuhIDF0agl4eahcowtsxplg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1588309992; h=Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=k8eiGOTPCvH3VmZc06Rg7ppdPB8Jnmrh7jbZ9+1Uyi4=; b=Bom3fwniXYEzf7h3DI6FZ7uMSob6Eqod5Oi6lTI9AVMu8yajGRBupFvLf722zcb0ZyX6kC6nFQOUKyQ2r/ZD325LcPfCjHjEOHs15D2vvz1bYb08X6HoXrAgu1WaBRON6KoJzxINPR2KnYp8gRhDu/Ueah9jCOMJHELGIgD7xxY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58493+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1588309992945201.70257932769516; Thu, 30 Apr 2020 22:13:12 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id sXjbYY1788612xKpJUyivmx8; Thu, 30 Apr 2020 22:13:12 -0700 X-Received: from mga01.intel.com (mga01.intel.com []) by mx.groups.io with SMTP id smtpd.web11.4932.1588309989257688147 for ; Thu, 30 Apr 2020 22:13:10 -0700 IronPort-SDR: 6BW0Nz4vybKbZT1bBkxtTHQlGpEroE2ecKKm/LJIlhsFORDIC1mRGaBHEOhXjirJjLGdrvZrjh kGGMnygJHrGg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Apr 2020 22:13:08 -0700 IronPort-SDR: TcAdhGOLX7M+xpAe9U0ChKaFcPARodbmPEFyni4+odYiFNEowLEriNo2UCUbfIIvYItf0qdEj7 ZcTDQIl0GxSA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,338,1583222400"; d="scan'208";a="283096087" X-Received: from aedesimo-desk.amr.corp.intel.com ([10.212.210.118]) by fmsmga004.fm.intel.com with ESMTP; 30 Apr 2020 22:12:49 -0700 From: "Ashley E Desimone" To: devel@edk2.groups.io Cc: Nate DeSimone , Puja Pandya , Erik Bjorge , Bret Barkelew , Prince Agyeman Subject: [edk2-devel] [edk2-staging/EdkRepo] [PATCH 5/8] EdkRepo: Add ability to find projects across all manifest repositories Date: Thu, 30 Apr 2020 22:12:36 -0700 Message-Id: <20200501051239.13976-6-ashley.e.desimone@intel.com> In-Reply-To: <20200501051239.13976-1-ashley.e.desimone@intel.com> References: <20200501051239.13976-1-ashley.e.desimone@intel.com> Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,ashley.e.desimone@intel.com X-Gm-Message-State: HTUe69rX7PEP5m5NFvviJLPsx1787277AA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1588309992; bh=vN4z/i2zF1C1e+9lIUfdkVInhBJj8WcYQ0eH5q+WCkI=; h=Cc:Date:From:Reply-To:Subject:To; b=sbLOtgHgK7oDXPp6a5P4EGJ72l19r1TekDA0AAD0kn69X2feI5coXIZ9vBrjFpsNezs uTXTy72ePl7FdDgQD3qLRs0wviE7OWlCRn1WwDbXX4/9kqx98pmGoNrJm3Da7m4OiUyp8 67bra0nxEc/YvU4GSN5lUioTTQdclCi8T8Y= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add find_project_in_all_indicies() to search for and return a tuple (source repo, source config, path to manifest) if a matching project is found. Add find_project_in_single_index() to find the path to a project within a single manifest repositories index file and return a whether the specified project was found and if so its path within the manifest repository. Signed-off-by: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Erik Bjorge Cc: Bret Barkelew Cc: Prince Agyeman Reviewed-by: Nate DeSimone --- .../manifest_repos_maintenance.py | 70 ++++++++++++++++++= +++- .../workspace_maintenance/workspace_maintenance.py | 17 +++++- 2 files changed, 84 insertions(+), 3 deletions(-) diff --git a/edkrepo/common/workspace_maintenance/manifest_repos_maintenanc= e.py b/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py index 4bded46..9b441ac 100644 --- a/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py +++ b/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py @@ -15,11 +15,12 @@ import git from git import Repo =20 import edkrepo.config.config_factory as cfg -from edkrepo.common.edkrepo_exception import EdkrepoUncommitedChangesExcep= tion +from edkrepo.common.edkrepo_exception import EdkrepoUncommitedChangesExcep= tion, EdkrepoInvalidParametersException from edkrepo.common.progress_handler import GitProgressHandler import edkrepo.common.workspace_maintenance.humble.manifest_repos_maintena= nce_humble as humble from edkrepo.common.workspace_maintenance.workspace_maintenance import gen= erate_name_for_obsolete_backup - +from edkrepo.common.workspace_maintenance.workspace_maintenance import cas= e_insensitive_single_match +from edkrepo_manifest_parser.edk_manifest import CiIndexXml, ManifestXml =20 def pull_single_manifest_repo(url, branch, local_path, reset_hard=3DFalse): ''' @@ -135,3 +136,68 @@ def list_available_man_repos(edkrepo_cfg, edkrepo_user= _cfg): return cfg_man_repos, user_cfg_man_repos, conflicts =20 =20 +def find_project_in_single_index (project, index_file, manifest_dir): + ''' + Finds a project in a single global manifest repositories index file. I= f found + returns (True, path to file) if not returns (False, None) + ''' + global_manifest_path =3D None + try: + proj_name =3D case_insensitive_single_match(project, index_file.pr= oject_list) + except: + proj_name =3D None + if proj_name: + ci_index_xml_rel_path =3D os.path.normpath(index_file.get_project_= xml(proj_name)) + global_manifest_path =3D os.path.join(manifest_dir, ci_index_xml_r= el_path) + return True, global_manifest_path + else: + return False, global_manifest_path + + +def find_project_in_all_indices (project, edkrepo_cfg, edkrepo_user_cfg, e= xcept_msg_man_repo, except_msg_not_found, man_repo=3DNone): + ''' + Finds the project in all manifest repositories listed in the edkrepo.e= fg and + edkrepo_user.cfg. If a project with the same name is found uses man_re= po to select + the correct entry + ''' + cfg_man_repos, user_cfg_man_repos, conflicts =3D list_available_man_re= pos(edkrepo_cfg, edkrepo_user_cfg) + projects =3D {} + for repo in cfg_man_repos: + manifest_dir =3D edkrepo_cfg.manifest_repo_abs_path(repo) + index_file =3D CiIndexXml(os.path.join(manifest_dir, 'CiIndex.xml'= )) + found, man_path =3D find_project_in_single_index(project, index_fi= le, manifest_dir) + if found: + projects[repo] =3D ('edkrepo_cfg', man_path) + for repo in user_cfg_man_repos: + manifest_dir =3D edkrepo_user_cfg.manifest_repo_abs_path(repo) + index_file =3D CiIndexXml(os.path.join(manifest_dir, 'CiIndex.xml'= )) + found, man_path =3D find_project_in_single_index(project, index_fi= le, manifest_dir) + if found: + projects[repo] =3D ('edkrepo_user_cfg', man_path) + if len(projects.keys()) =3D=3D 1: + repo =3D list(projects.keys())[0] + return repo, projects[repo][0], projects[repo][1] + elif len(projects.keys()) > 1 and man_repo: + try: + return man_repo, projects[man_repo][0], projects[man_repo][1] + except KeyError: + raise EdkrepoInvalidParametersException(except_msg_man_repo) + elif os.path.isabs(project): + return None, None, project + elif os.path.isfile(os.path.join(os.getcwd(), project)): + return None, None, os.path.join(os.getcwd(), project) + elif not os.path.dirname(project): + for repo in cfg_man_repos: + if (man_repo and (repo =3D=3D man_repo)) or not man_repo: + for dirpath, dirname, filenames in os.walk(edkrepo_cfg.man= ifest_repo_abs_path(repo)): + if project in filenames: + return repo, 'edkrepo_cfg', os.path.join(dirpath, = project) + for repo in user_cfg_man_repos: + if (man_repo and (repo =3D=3D man_repo)) or not man_repo: + for dirpath, dirname, filenames in os.walk(edkrepo_user_cf= g.manifest_repo_abs_path(repo)): + if project in filenames: + return repo, 'edkrepo_user_cfg', os.path.join(dirp= ath, project) + + + + diff --git a/edkrepo/common/workspace_maintenance/workspace_maintenance.py = b/edkrepo/common/workspace_maintenance/workspace_maintenance.py index 6e20d43..ba62f6d 100644 --- a/edkrepo/common/workspace_maintenance/workspace_maintenance.py +++ b/edkrepo/common/workspace_maintenance/workspace_maintenance.py @@ -10,6 +10,10 @@ ''' Contains shared workspace maintenance functions. ''' =20 import os +import unicodedata + +from edkrepo.common.edkrepo_exception import EdkrepoFoundMultipleException= , EdkrepoNotFoundException +from edkrepo.common.humble import GEN_A_NOT_IN_B, GEN_FOUND_MULT_A_IN_B =20 def generate_name_for_obsolete_backup(absolute_path): if not os.path.exists(absolute_path): @@ -27,4 +31,15 @@ def generate_name_for_obsolete_backup(absolute_path): if not os.path.exists(os.path.join(dir_name, unique_name)): unique_name_found =3D True index +=3D 1 - return unique_name \ No newline at end of file + return unique_name + +def case_insensitive_equal(str1, str2): + return unicodedata.normalize("NFKD", str1.casefold()) =3D=3D unicodeda= ta.normalize("NFKD", str2.casefold()) + +def case_insensitive_single_match(str1, str_list): + matches =3D [x for x in str_list if case_insensitive_equal(str1, x)] + if len(matches) =3D=3D 0: + raise EdkrepoNotFoundException(GEN_A_NOT_IN_B.format(str1, str_lis= t)) + elif len(matches) > 1: + raise EdkrepoFoundMultipleException(GEN_FOUND_MULT_A_IN_B.format(s= tr1, str_list)) + return matches[0] --=20 2.16.2.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#58493): https://edk2.groups.io/g/devel/message/58493 Mute This Topic: https://groups.io/mt/73391737/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Sun Apr 28 09:21:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+58494+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58494+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1588309993; cv=none; d=zohomail.com; s=zohoarc; b=Y9u4E8af3VYAw619oiU35hOArw1SCRIRjomX+xfWGJBQGCy63vQ1sKkjCzGjYB0F42KznzEXR04lZ/XH1Qv1xN9CEjcaEWsAVIi/CivHSEbeU42CHlwRKzjRunfe2V9FwSMt+ltPXmWOOhS5MjG/t0Hl8WLtofMTySr3guXVsc8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1588309993; h=Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=0p3LFhgK1NxpNxClQkR9aWUszlXPwQULPcWLKetJOcY=; b=PsGCfc2Dwthe44v+LiJKPDjcwmAqvpQ16cO7wghpuwHIMeHQBelY98ONgpLf3kaRzE8fy3Y13LWkk9gGTCRuHo/VHGxaKGpHsuysePVg3CvUbMisNNeGPghYeFb8y8suaQqkilOFnB3vYYCv48RzO/Kuc/pZva5XK9mVNzpX+kw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58494+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 15883099934969.60830901814677; Thu, 30 Apr 2020 22:13:13 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id zAw2YY1788612xPa8GsaYRmG; Thu, 30 Apr 2020 22:13:13 -0700 X-Received: from mga01.intel.com (mga01.intel.com []) by mx.groups.io with SMTP id smtpd.web10.4894.1588309989753910554 for ; Thu, 30 Apr 2020 22:13:10 -0700 IronPort-SDR: I3ii7C/q82NKFrdKkMTLYOrNX6JaaFsm56EpK5g4/Y3e8afqavL2LVoQyRNA3/lgiTL3QzKHb0 tjFamv8T0BaQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Apr 2020 22:13:08 -0700 IronPort-SDR: hug5Mq4QWnu9MN6pVr5ybGPFQ7OFyibJc0GiICBwsdu3u0V/qOXprhNoQ2qTYWUu8ckr9XrjsD hJezJkZllABA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,338,1583222400"; d="scan'208";a="283096090" X-Received: from aedesimo-desk.amr.corp.intel.com ([10.212.210.118]) by fmsmga004.fm.intel.com with ESMTP; 30 Apr 2020 22:12:49 -0700 From: "Ashley E Desimone" To: devel@edk2.groups.io Cc: Nate DeSimone , Puja Pandya , Erik Bjorge , Bret Barkelew , Prince Agyeman Subject: [edk2-devel] [edk2-stagin/EdkRepo] [PATCH 6/8] EdkRepo: Add ability to determine the source manifest of a workspace Date: Thu, 30 Apr 2020 22:12:37 -0700 Message-Id: <20200501051239.13976-7-ashley.e.desimone@intel.com> In-Reply-To: <20200501051239.13976-1-ashley.e.desimone@intel.com> References: <20200501051239.13976-1-ashley.e.desimone@intel.com> Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,ashley.e.desimone@intel.com X-Gm-Message-State: xLHK6WwCFEDK6pbP64NGCLyhx1787277AA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1588309993; bh=dFO3PuhLdh9msFH4SNJceUTdmTDd3dkYGJPT+A3lAec=; h=Cc:Date:From:Reply-To:Subject:To; b=aGjWl9m1FmYOaOtx0015L0bR3gCgenzYbVGvEQ4kukncNXrN7CIh2InsPzFCEn6mP1X sCIA0kX0k1oejTX2bjxU+H8gdT9oyX8/ZtGff+5lPsm0amrL8HtlMa8bl6stK6LVWtAge 03uIYx4MHj69ham8WJ8MlYGtoXpUNieCCbA= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add find_source_man_repo() to check if for the source manifest repo is contained in the workspaces project manifest file. If it is not determine the value and write it to the manifest. Signed-off-by: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Erik Bjorge Cc: Bret Barkelew Cc: Prince Agyeman Reviewed-by: Nate DeSimone --- .../humble/manifest_repos_maintenance_humble.py | 2 ++ .../workspace_maintenance/manifest_repos_maintenance.py | 15 +++++++++++= ++++ 2 files changed, 17 insertions(+) diff --git a/edkrepo/common/workspace_maintenance/humble/manifest_repos_mai= ntenance_humble.py b/edkrepo/common/workspace_maintenance/humble/manifest_r= epos_maintenance_humble.py index e592f19..05e76b1 100644 --- a/edkrepo/common/workspace_maintenance/humble/manifest_repos_maintenanc= e_humble.py +++ b/edkrepo/common/workspace_maintenance/humble/manifest_repos_maintenanc= e_humble.py @@ -25,3 +25,5 @@ CONFLICT_NO_CLONE =3D ('The definition of global manifest= repository, {}, ' 'in the edkrepo_user.cfg does not match the definitio= n in the edkrepo.cfg. ' 'This global manifest repository will not be download= ed or updated. ' 'Resolve the conflict and then re-run the failed oper= ation') +SOURCE_MAN_REPO_NOT_FOUND =3D 'Could not determine the source global manif= est repository for project: {}' +PROJ_NOT_IN_REPO =3D 'Project: {} does not exist in any global manifest re= pository' \ No newline at end of file diff --git a/edkrepo/common/workspace_maintenance/manifest_repos_maintenanc= e.py b/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py index 9b441ac..7b3f866 100644 --- a/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py +++ b/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py @@ -199,5 +199,20 @@ def find_project_in_all_indices (project, edkrepo_cfg,= edkrepo_user_cfg, except_ return repo, 'edkrepo_user_cfg', os.path.join(dirp= ath, project) =20 =20 +def find_source_man_repo (project_manifest, edkrepo_cfg, edkrepo_user_cfg): + ''' + Finds the source manifest repo for a given project. + ''' + if project_manifest.general_config.source_man_repo: + return project_manifest.general_config.source_man_repo + else: + src_man_repo, src_config, src_man_path =3D find_project_in_all_ind= ices(project_manifest.project_info.codename, + = edkrepo_cfg, + = edkrepo_user_cfg, + = humble.PROJ_NOT_IN_REPO.format(project_manifest.project_info.codename), + = humble.SOURCE_MAN_REPO_NOT_FOUND.format(project_manifest.project_info.co= dename), + = man_repo=3DNone) + project_manifest.write_source_manifest_repo(src_man_repo) + return src_man_repo =20 =20 =20 --=20 2.16.2.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#58494): https://edk2.groups.io/g/devel/message/58494 Mute This Topic: https://groups.io/mt/73391738/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Sun Apr 28 09:21:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+58495+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58495+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1588309993; cv=none; d=zohomail.com; s=zohoarc; b=eJjNKt94SEjRlIQ3daZFlWIpSG7zV4aSMxpUtXFEGkMjPBZufyAb6J9W9jmByk4yGX6A4cSs8KaAvemz2nlDrZPH0LllRo9pF+WEVEgTYW2+9DI8wTlZMAwwYenrPskVNVwh/QQkJXG/PT6VGT5WAHV8zBO/2U5Fe1QwV4/IfTQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1588309993; h=Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=35C03fM4ERimGNU7OPu9jcrrhn+Nem+u6y6HeC+B3Mk=; b=M3cDs9nXpH27Oc5HJvLwJbO/bNuE17E80V2myMXzpafJJOrEcpnemKJSfrT9kRBH/N/M41w9KFwD27/ef+dylftPCXYWMGeGV4LnfEkmy8wLVMXj0MNHib+9bAUv4xfuliZGRnsFYoHa3t7jltgWrdtXir06SiEq6dakfbP3d6w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58495+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1588309993775482.02604771660367; Thu, 30 Apr 2020 22:13:13 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id VI3LYY1788612xxwIKVQ3VX3; Thu, 30 Apr 2020 22:13:13 -0700 X-Received: from mga01.intel.com (mga01.intel.com []) by mx.groups.io with SMTP id smtpd.web11.4932.1588309989257688147 for ; Thu, 30 Apr 2020 22:13:10 -0700 IronPort-SDR: iUFxk8b66Lt7U5ksV9FB17O1qW28Q4cc7P1UFu1Sk+uz9MLa83UHxo5xFpX9YFit/mAmiRFrxF 0qE5ocHeu7qw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Apr 2020 22:13:08 -0700 IronPort-SDR: YLPQ5pKcggjEHn8XAFhYLOGqk7eX1o2eFfXH18t/kd7EG/DkI8LZmqFOamLHA6LHpyBkygEaMY pIc66Xj7SJoA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,338,1583222400"; d="scan'208";a="283096093" X-Received: from aedesimo-desk.amr.corp.intel.com ([10.212.210.118]) by fmsmga004.fm.intel.com with ESMTP; 30 Apr 2020 22:12:49 -0700 From: "Ashley E Desimone" To: devel@edk2.groups.io Cc: Nate DeSimone , Puja Pandya , Erik Bjorge , Bret Barkelew , Prince Agyeman Subject: [edk2-devel] [edk2-staging/EdkRepo] [PATCH V2 7/8] EdkRepo: Add the ability to pull only the global manifest repository for a given workspace. Date: Thu, 30 Apr 2020 22:12:38 -0700 Message-Id: <20200501051239.13976-8-ashley.e.desimone@intel.com> In-Reply-To: <20200501051239.13976-1-ashley.e.desimone@intel.com> References: <20200501051239.13976-1-ashley.e.desimone@intel.com> Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,ashley.e.desimone@intel.com X-Gm-Message-State: d5V60VeaXtYfb1M4g8b1UtQYx1787277AA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1588309993; bh=jBNtTeW9vEMB7DGZ2iRotqu6G+awa9oAy4tqpKWLdxY=; h=Cc:Date:From:Reply-To:Subject:To; b=FJ4F2+jLkvV5HGfN828VeAJaJkwXqEbpoMShQY1TWzCG9AzFhl12u3USjtTrFOoyv9a uzT5TU6z2EDd7GxwCngcqqcmY7fkzp99L9x8XORHEma96iNNqErIDY7MUi3fIs/srLXcZ 56jUhU24RJKcr7ImFWD7WTAOQ44LaAlTt7M= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add pull_workspace_man_repo() to pull only the global manifest repository affiliated with a single workspace. Signed-off-by: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Erik Bjorge Cc: Bret Barkelew Cc: Prince Agyeman Reviewed-by: Nate DeSimone --- .../manifest_repos_maintenance.py | 23 ++++++++++++++++++= ++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/edkrepo/common/workspace_maintenance/manifest_repos_maintenanc= e.py b/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py index 7b3f866..7a7f946 100644 --- a/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py +++ b/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py @@ -199,7 +199,7 @@ def find_project_in_all_indices (project, edkrepo_cfg, = edkrepo_user_cfg, except_ return repo, 'edkrepo_user_cfg', os.path.join(dirp= ath, project) =20 =20 -def find_source_man_repo (project_manifest, edkrepo_cfg, edkrepo_user_cfg): +def find_source_man_repo(project_manifest, edkrepo_cfg, edkrepo_user_cfg, = man_repo=3DNone): ''' Finds the source manifest repo for a given project. ''' @@ -213,6 +213,25 @@ def find_source_man_repo (project_manifest, edkrepo_cf= g, edkrepo_user_cfg): = humble.SOURCE_MAN_REPO_NOT_FOUND.format(project_manifest.project_info.co= dename), = man_repo=3DNone) project_manifest.write_source_manifest_repo(src_man_repo) - return src_man_repo =20 + return src_man_repo + +def pull_workspace_man_repo(project_manifest, edkrepo_cfg, edkrepo_user_cf= g, man_repo=3DNone, reset_hard=3DFalse): + ''' + Pulls only the global manifest repo for the current workspace. + ''' + src_man_repo =3D find_source_man_repo(project_manifest, edkrepo_cfg, e= dkrepo_user_cfg, man_repo) + config_repos, user_config_repos, conflicts =3D list_available_man_repo= s(edkrepo_cfg, edkrepo_user_cfg) + if src_man_repo in config_repos: + pull_single_manifest_repo(edkrepo_cfg.get_manifest_repo_url(src_ma= n_repo), + edkrepo_cfg.get_manifest_repo_branch(src= _man_repo), + edkrepo_cfg.get_manifest_repo_local_path= (src_man_repo), + reset_hard) + elif src_man_repo in user_config_repos: + pull_single_manifest_repo(edkrepo_user_cfg.get_manifest_repo_url(s= rc_man_repo), + edkrepo_user_cfg.get_manifest_repo_branc= h(src_man_repo), + edkrepo_user_cfg.get_manifest_repo_local= _path(src_man_repo), + reset_hard) + elif src_man_repo in conflicts: + raise EdkrepoInvalidParametersException(humble.CONFLICT_NO_CLONE.f= ormat(src_man_repo)) =20 =20 --=20 2.16.2.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#58495): https://edk2.groups.io/g/devel/message/58495 Mute This Topic: https://groups.io/mt/73391739/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Sun Apr 28 09:21:30 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+58496+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58496+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1588309994; cv=none; d=zohomail.com; s=zohoarc; b=fyXmSaUt+efjwhdvWrSQMto7wzVzCImjdMoxVaPnmd2+Wz1RsAMTbJAXPa4jkxMBRkmjFjVxcWXBK5g4k1NiFGcY849TGnU00vTm+CNLHoyejQLLR/QxNUUc/QFwI/m/NMMvWqwIVcRtV+Lv2W98+zdMh5EvZPL3aoDVzVy7qZU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1588309994; h=Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=ITIefTzCd4+ZQXjfc8dqVNcM9dX3fBCXL2+NklEcJ8U=; b=KzK72s35pFNBIZsk4M4YPeI2fK6KzmqMNK2dx+bfuZ74mJwxxZbuA4V82YKXlRkDNi4Vrb3IWbkDCLCfdNCNWthYKTQ0/M3PJ30/4ZLJM7Qlh8ZX9b+bzmTcemoQZwvceu9zkapPDrTm2c/J+ffaAH4BdWbB7VRfEkHGEC8k8Eo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+58496+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1588309994346549.1659100593749; Thu, 30 Apr 2020 22:13:14 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id dNO1YY1788612xRZUgevfI73; Thu, 30 Apr 2020 22:13:14 -0700 X-Received: from mga01.intel.com (mga01.intel.com []) by mx.groups.io with SMTP id smtpd.web10.4894.1588309989753910554 for ; Thu, 30 Apr 2020 22:13:10 -0700 IronPort-SDR: ZKac3FlQcXuqV0EY54vtIXsp1vqvpfdPLd/nYHmnWvNj04Cvao3fuVAgn0Aua89IR0eKEkKvZW d9YE4z/phIig== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Apr 2020 22:13:08 -0700 IronPort-SDR: yAmijEvaYpvPECHjwwbWj2nHng+PztfLCP+j2W+fyLJ6RlKdU2SWnEWOC97Nrjrkd65C9Vr/Di V57iA97WnPqQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,338,1583222400"; d="scan'208";a="283096096" X-Received: from aedesimo-desk.amr.corp.intel.com ([10.212.210.118]) by fmsmga004.fm.intel.com with ESMTP; 30 Apr 2020 22:12:50 -0700 From: "Ashley E Desimone" To: devel@edk2.groups.io Cc: Nate DeSimone , Puja Pandya , Erik Bjorge , Bret Barkelew , Prince Agyeman Subject: [edk2-devel] [edk2-staging/EdkRepo] [PATCH V2 8/8] EdkRepo: Rename man to manifest Date: Thu, 30 Apr 2020 22:12:39 -0700 Message-Id: <20200501051239.13976-9-ashley.e.desimone@intel.com> In-Reply-To: <20200501051239.13976-1-ashley.e.desimone@intel.com> References: <20200501051239.13976-1-ashley.e.desimone@intel.com> Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,ashley.e.desimone@intel.com X-Gm-Message-State: yEHcVbbyl5U9L9oDWFNlQ5eWx1787277AA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1588309994; bh=PkpN6NAXFCBZ8BifjO5miyx/BCjkkDe38Ir6nV8ansc=; h=Cc:Date:From:Reply-To:Subject:To; b=F3by0099oR1fBbgVTc6g9+5/M9oDqf4ugpupcxLFg2ItepOHuW3BnP2QZoV5KyO44Vt BS2j45mdVNiEqoSqxb7tqhLLn89mN0HLiUekNHI0g/nAMVtkvDGtC5+87somlIfC+bqGb F026UWudZhr6JOyaGzigpi/XtDLUaSTdLkg= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Signed-off-by: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Erik Bjorge Cc: Bret Barkelew Cc: Prince Agyeman Reviewed-by: Nate DeSimone --- .../humble/manifest_repos_maintenance_humble.py | 2 +- .../manifest_repos_maintenance.py | 32 ++++++++++++------= ---- edkrepo/config/tool_config.py | 10 +++++++ edkrepo_manifest_parser/edk_manifest.py | 10 +++---- 4 files changed, 34 insertions(+), 20 deletions(-) create mode 100644 edkrepo/config/tool_config.py diff --git a/edkrepo/common/workspace_maintenance/humble/manifest_repos_mai= ntenance_humble.py b/edkrepo/common/workspace_maintenance/humble/manifest_r= epos_maintenance_humble.py index 05e76b1..290b463 100644 --- a/edkrepo/common/workspace_maintenance/humble/manifest_repos_maintenanc= e_humble.py +++ b/edkrepo/common/workspace_maintenance/humble/manifest_repos_maintenanc= e_humble.py @@ -25,5 +25,5 @@ CONFLICT_NO_CLONE =3D ('The definition of global manifest= repository, {}, ' 'in the edkrepo_user.cfg does not match the definitio= n in the edkrepo.cfg. ' 'This global manifest repository will not be download= ed or updated. ' 'Resolve the conflict and then re-run the failed oper= ation') -SOURCE_MAN_REPO_NOT_FOUND =3D 'Could not determine the source global manif= est repository for project: {}' +SOURCE_MANIFEST_REPO_NOT_FOUND =3D 'Could not determine the source global = manifest repository for project: {}' PROJ_NOT_IN_REPO =3D 'Project: {} does not exist in any global manifest re= pository' \ No newline at end of file diff --git a/edkrepo/common/workspace_maintenance/manifest_repos_maintenanc= e.py b/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py index 7a7f946..2a5a70f 100644 --- a/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py +++ b/edkrepo/common/workspace_maintenance/manifest_repos_maintenance.py @@ -15,7 +15,9 @@ import git from git import Repo =20 import edkrepo.config.config_factory as cfg +from edkrepo.config.tool_config import CI_INDEX_FILE_NAME from edkrepo.common.edkrepo_exception import EdkrepoUncommitedChangesExcep= tion, EdkrepoInvalidParametersException +from edkrepo.common.edkrepo_exception import EdkrepoManifestNotFoundExcept= ion from edkrepo.common.progress_handler import GitProgressHandler import edkrepo.common.workspace_maintenance.humble.manifest_repos_maintena= nce_humble as humble from edkrepo.common.workspace_maintenance.workspace_maintenance import gen= erate_name_for_obsolete_backup @@ -66,7 +68,7 @@ def pull_all_manifest_repos(edkrepo_cfg, edkrepo_user_cfg= , reset_hard=3DFalse): cfg_man_repos =3D [] user_cfg_man_repos =3D [] conflicts =3D [] - cfg_man_repos, user_cfg_man_repos, conflicts =3D list_available_man_re= pos(edkrepo_cfg, edkrepo_user_cfg) + cfg_man_repos, user_cfg_man_repos, conflicts =3D list_available_manife= st_repos(edkrepo_cfg, edkrepo_user_cfg) for conflict in conflicts: print(humble.CONFLICT_NO_CLONE.format(conflict)) for repo in cfg_man_repos: @@ -81,7 +83,7 @@ def pull_all_manifest_repos(edkrepo_cfg, edkrepo_user_cfg= , reset_hard=3DFalse): reset_hard) =20 =20 -def detect_man_repo_conflicts_duplicates(edkrepo_cfg, edkrepo_user_cfg): +def detect_manifest_repo_conflicts_duplicates(edkrepo_cfg, edkrepo_user_cf= g): ''' Determines whether there is are conflicting or duplicated manifest repositories listed in the edkrepo.cfg and the edkrepo_user.cfg. @@ -107,7 +109,7 @@ def detect_man_repo_conflicts_duplicates(edkrepo_cfg, e= dkrepo_user_cfg): duplicates.append(repo) return conflicts, duplicates =20 -def list_available_man_repos(edkrepo_cfg, edkrepo_user_cfg): +def list_available_manifest_repos(edkrepo_cfg, edkrepo_user_cfg): ''' Checks for conflicts/duplicates within all manifest repositories defin= ed in both the edkrepo.cfg and the edkrepo_user.cfg and resturns a list of a= vailable @@ -115,7 +117,7 @@ def list_available_man_repos(edkrepo_cfg, edkrepo_user_= cfg): ''' cfg_man_repos =3D [] user_cfg_man_repos =3D [] - conflicts, duplicates =3D detect_man_repo_conflicts_duplicates(edkrepo= _cfg, edkrepo_user_cfg) + conflicts, duplicates =3D detect_manifest_repo_conflicts_duplicates(ed= krepo_cfg, edkrepo_user_cfg) if not conflicts and not duplicates: cfg_man_repos.extend(edkrepo_cfg.manifest_repo_list) user_cfg_man_repos.extend(edkrepo_user_cfg.manifest_repo_list) @@ -160,17 +162,17 @@ def find_project_in_all_indices (project, edkrepo_cfg= , edkrepo_user_cfg, except_ edkrepo_user.cfg. If a project with the same name is found uses man_re= po to select the correct entry ''' - cfg_man_repos, user_cfg_man_repos, conflicts =3D list_available_man_re= pos(edkrepo_cfg, edkrepo_user_cfg) + cfg_man_repos, user_cfg_man_repos, conflicts =3D list_available_manife= st_repos(edkrepo_cfg, edkrepo_user_cfg) projects =3D {} for repo in cfg_man_repos: manifest_dir =3D edkrepo_cfg.manifest_repo_abs_path(repo) - index_file =3D CiIndexXml(os.path.join(manifest_dir, 'CiIndex.xml'= )) + index_file =3D CiIndexXml(os.path.join(manifest_dir, CI_INDEX_FILE= _NAME)) found, man_path =3D find_project_in_single_index(project, index_fi= le, manifest_dir) if found: projects[repo] =3D ('edkrepo_cfg', man_path) for repo in user_cfg_man_repos: manifest_dir =3D edkrepo_user_cfg.manifest_repo_abs_path(repo) - index_file =3D CiIndexXml(os.path.join(manifest_dir, 'CiIndex.xml'= )) + index_file =3D CiIndexXml(os.path.join(manifest_dir, CI_INDEX_FILE= _NAME)) found, man_path =3D find_project_in_single_index(project, index_fi= le, manifest_dir) if found: projects[repo] =3D ('edkrepo_user_cfg', man_path) @@ -197,30 +199,32 @@ def find_project_in_all_indices (project, edkrepo_cfg= , edkrepo_user_cfg, except_ for dirpath, dirname, filenames in os.walk(edkrepo_user_cf= g.manifest_repo_abs_path(repo)): if project in filenames: return repo, 'edkrepo_user_cfg', os.path.join(dirp= ath, project) + else: + raise EdkrepoManifestNotFoundException(humble.PROJ_NOT_IN_REPO.for= mat(project)) =20 =20 -def find_source_man_repo(project_manifest, edkrepo_cfg, edkrepo_user_cfg, = man_repo=3DNone): +def find_source_manifest_repo(project_manifest, edkrepo_cfg, edkrepo_user_= cfg, man_repo=3DNone): ''' Finds the source manifest repo for a given project. ''' - if project_manifest.general_config.source_man_repo: - return project_manifest.general_config.source_man_repo + if project_manifest.general_config.source_manifest_repo: + return project_manifest.general_config.source_manifest_repo else: src_man_repo, src_config, src_man_path =3D find_project_in_all_ind= ices(project_manifest.project_info.codename, = edkrepo_cfg, = edkrepo_user_cfg, = humble.PROJ_NOT_IN_REPO.format(project_manifest.project_info.codename), - = humble.SOURCE_MAN_REPO_NOT_FOUND.format(project_manifest.project_info.co= dename), + = humble.SOURCE_MANIFEST_REPO_NOT_FOUND.format(project_manifest.project_in= fo.codename), = man_repo=3DNone) project_manifest.write_source_manifest_repo(src_man_repo) return src_man_repo =20 -def pull_workspace_man_repo(project_manifest, edkrepo_cfg, edkrepo_user_cf= g, man_repo=3DNone, reset_hard=3DFalse): +def pull_workspace_manifest_repo(project_manifest, edkrepo_cfg, edkrepo_us= er_cfg, man_repo=3DNone, reset_hard=3DFalse): ''' Pulls only the global manifest repo for the current workspace. ''' - src_man_repo =3D find_source_man_repo(project_manifest, edkrepo_cfg, e= dkrepo_user_cfg, man_repo) - config_repos, user_config_repos, conflicts =3D list_available_man_repo= s(edkrepo_cfg, edkrepo_user_cfg) + src_man_repo =3D find_source_manifest_repo(project_manifest, edkrepo_c= fg, edkrepo_user_cfg, man_repo) + config_repos, user_config_repos, conflicts =3D list_available_manifest= _repos(edkrepo_cfg, edkrepo_user_cfg) if src_man_repo in config_repos: pull_single_manifest_repo(edkrepo_cfg.get_manifest_repo_url(src_ma= n_repo), edkrepo_cfg.get_manifest_repo_branch(src= _man_repo), diff --git a/edkrepo/config/tool_config.py b/edkrepo/config/tool_config.py new file mode 100644 index 0000000..eee1326 --- /dev/null +++ b/edkrepo/config/tool_config.py @@ -0,0 +1,10 @@ +#!/usr/bin/env python3 +# +## @file +# tool)config.py +# +# Copyright (c) 2020, Intel Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent +# + +CI_INDEX_FILE_NAME =3D 'CiIndex.xml' \ No newline at end of file diff --git a/edkrepo_manifest_parser/edk_manifest.py b/edkrepo_manifest_par= ser/edk_manifest.py index dcf9c29..f7118ae 100644 --- a/edkrepo_manifest_parser/edk_manifest.py +++ b/edkrepo_manifest_parser/edk_manifest.py @@ -21,7 +21,7 @@ import copy # All the namedtuple data structures that consumers of this module will ne= ed. # ProjectInfo =3D namedtuple('ProjectInfo', ['codename', 'description', 'dev= _leads', 'reviewers', 'org', 'short_name']) -GeneralConfig =3D namedtuple('GeneralConfig', ['default_combo', 'current_c= ombo', 'pin_path', 'source_man_repo']) +GeneralConfig =3D namedtuple('GeneralConfig', ['default_combo', 'current_c= ombo', 'pin_path', 'source_manifest_repo']) RemoteRepo =3D namedtuple('RemoteRepo', ['name', 'url']) RepoHook =3D namedtuple('RepoHook', ['source', 'dest_path', 'dest_file', '= remote_url']) Combination =3D namedtuple('Combination', ['name', 'description']) @@ -422,7 +422,7 @@ class ManifestXml(BaseXmlHelper): element =3D ET.SubElement(subroot, 'SourceManifestRepository') element.attrib['manifest_repo'] =3D manifest_repo self._tree.write(filename) - self.__general_config.source_man_repo =3D manifest_repo + self.__general_config.source_manifest_repo =3D manifest_repo =20 def generate_pin_xml(self, description, combo_name, repo_source_list, = filename=3DNone): =20 @@ -624,13 +624,13 @@ class _GeneralConfig(): except: self.curr_combo =3D None try: - self.source_man_repo =3D element.find('SourceManifestRepositor= y').attrib['manifest_repo'] + self.source_manifest_repo =3D element.find('SourceManifestRepo= sitory').attrib['manifest_repo'] except: - self.source_man_repo =3D None + self.source_manifest_repo =3D None =20 @property def tuple(self): - return GeneralConfig(self.default_combo, self.curr_combo, self.pin= _path, self.source_man_repo) + return GeneralConfig(self.default_combo, self.curr_combo, self.pin= _path, self.source_manifest_repo) =20 class _RemoteRepo(): def __init__(self, element): --=20 2.16.2.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#58496): https://edk2.groups.io/g/devel/message/58496 Mute This Topic: https://groups.io/mt/73391740/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-