.azurepipelines/templates/pr-gate-build-job.yml | 6 ++++++ 1 file changed, 6 insertions(+)
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2570
OvmfPkg and EmulatorPkg are mostly used by the developers, so add
them to target list.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
---
.azurepipelines/templates/pr-gate-build-job.yml | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
index 61868554d43c..34f03745cc70 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -44,6 +44,12 @@ jobs:
TARGET_SECURITY:
Build.Pkgs: 'SecurityPkg'
Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_OVMF:
+ Build.Pkgs: 'OvmfPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_EMULATOR:
+ Build.Pkgs: 'EmulatorPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
workspace:
clean: all
--
2.18.0.windows.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#56365): https://edk2.groups.io/g/devel/message/56365
Mute This Topic: https://groups.io/mt/72559106/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
There are two parts of this I think should be discussed. 1. Core-Ci for emulator, ovmf, armvirt packages. a. I think there is some value here for those package maintainers. The core ci does spell checks, char encoding checks, lib class declaration, etc. Those seem valuable to help keep the package clean. B. The core ci works on an assumption that the DSC builds every module within the package and for that I think you would want to create new DSC file in those packages that included every module in the package and used NULL libs to resolve as many dependencies as possible. That DSC would be only for Core CI. 2. Platform Ci for emulator, ovmf, armvirt packages. a. I think this is what we want long term. It would compile the platform and ideally do boot testing and checks. b. Platform Ci should scale out to all sorts of platforms (open and closed src) that want to register for CI and are willing to keep their platform current. c. Should be a signal into the PR process. I don't think it should block by policy, but reviewers should use the results to evaluate the impact of the PR. d. I have setup a branch here with Platform CI. 1. Code changes to enable pytool to build of OVMF and EmulatorPkg. https://github.com/spbrogan/edk2/tree/ci-for-ovmf 2. 4 pipelines here: https://dev.azure.com/tianocore/edk2-ci-play/_build?treeState=XEVtdWxhdG9yUGtnJFxPVk1G&view=folders 3. OVMF GCC5: 8 builds. see the matrix here https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=4915&view=results and https://github.com/spbrogan/edk2/blob/ci-for-ovmf/.azurepipelines/platforms/Ovmf/Ubuntu-GCC5.yml#L22. This worked without issue. 4. OVMF VS2019: Same 8 builds. Results here: https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=4916&view=results and matrix file here https://github.com/spbrogan/edk2/blob/ci-for-ovmf/.azurepipelines/platforms/Ovmf/Windows-VS2019.yml#L22. The full build fails 2 of the "flavors". See https://bugzilla.tianocore.org/show_bug.cgi?id=2636 5. EmulatorPkg GCC: 2 builds. https://dev.azure.com/tianocore/edk2-ci-play/_build?definitionId=39&_a=summary IA32 fails so it currently only builds X64 6. EmulatorPkg VS2019: 2 builds https://dev.azure.com/tianocore/edk2-ci-play/_build?definitionId=40&_a=summary IA32 fails so it currently only builds X64. Feedback wanted. Happy to add more if there is some agreement that you like this direction. Also if you have ideas how to run the images and then exit from the shell with a status so we can tell success/failure that would be great. Thanks Sean -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56521): https://edk2.groups.io/g/devel/message/56521 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On 3/27/20 8:29 PM, Sean via Groups.Io wrote: > > Feedback wanted. Happy to add more if there is some agreement that > you like this direction. I see the OVMF build is running for example: /opt/hostedtoolcache/Python/3.8.2/x64/bin/stuart_build -c OvmfPkg/PlatformBuild.py TOOL_CHAIN_TAG=GCC5 TARGET=DEBUG -a X64 Can't it just run "build" (or, OvmfPkg/build.sh) directly without going through another script? I'm not sure the added complexity is worthwhile. -- Rebecca Cran -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56522): https://edk2.groups.io/g/devel/message/56522 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
"Stuart" gives the system a lot of structure and flexibility. It lets us maintain the build process consistently in a cross platform way and allows platforms to use python instead of shell scripting. This takes advantage of our investments in dependency management, optimized submodule management, logging, plugins, etc. OVMF is a pretty simple "platformbuild" but it still get many advantages (from my pov). We have talked about Stuart a few times at design meetings (so you can see some of the slides that have been uploaded) and willing to discuss more if you have specific questions. Thanks Sean -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56523): https://edk2.groups.io/g/devel/message/56523 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
Added support for EmulatorPkg running to UEFI shell. Segmentation fault on GCC / Ubuntu x64. Works on Windows. Could very easily be user error. https://bugzilla.tianocore.org/show_bug.cgi?id=2639 Laszlo - can you point me to any docs, scripts, examples of how i would do the same thing with OVMF (just don't want to re-invent the wheel)? Thanks Sean -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56532): https://edk2.groups.io/g/devel/message/56532 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On Sat, 28 Mar 2020 at 20:29, Sean via Groups.Io <sean.brogan=microsoft.com@groups.io> wrote: > > Added support for EmulatorPkg running to UEFI shell. > Segmentation fault on GCC / Ubuntu x64. Works on Windows. > > Could very easily be user error. > https://bugzilla.tianocore.org/show_bug.cgi?id=2639 > > Laszlo - can you point me to any docs, scripts, examples of how i would do the same thing with OVMF (just don't want to re-invent the wheel)? > Hello Sean, Thanks a lot for taking the time to work on this. IMO, having a CI smoke test that boots, say, an X64 as well as a AARCH64 virtual platform to the UEFI shell is extremely valuable, and given how little time it should take to run this test after doing the build, I would argue for it to be incorporated as a pre-commit check. (Background: the situation is much better than it was, but for a long time, being a maintainer of the ARM pieces in EDK2 was no fun, given how often the build got broken by people working on [and limiting their testing to] X64 platforms) -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56552): https://edk2.groups.io/g/devel/message/56552 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
Ard, I agree. How would we do this with AARCH64? I don't believe azure devops pipelines has a native AARCH64 platform available. I briefly looked over ArmVirtPkg but not sure where to start. OVMF readme only talks about ia32/x64. I could also see a scenario for a self hosted agent that is aarch64 but that would require learning that aspect of devops (on the list but not at the top yet). Anyway look forward to ideas and discussion. Thanks Sean -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56554): https://edk2.groups.io/g/devel/message/56554 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On Sat, 28 Mar 2020 at 22:47, Sean via Groups.Io <sean.brogan=microsoft.com@groups.io> wrote: > > Ard, > I agree. How would we do this with AARCH64? I don't believe azure devops pipelines has a native AARCH64 platform available. I briefly looked over ArmVirtPkg but not sure where to start. OVMF readme only talks about ia32/x64. I could also see a scenario for a self hosted agent that is aarch64 but that would require learning that aspect of devops (on the list but not at the top yet). Anyway look forward to ideas and discussion. > OVMF and ArmVirtQemu are very similar, in this regard: they require a matching version of QEMU: qemu-system-x86_64 for OVMF, and qemu-system-aarch64 for ArmVirtQemu, and I would expect most distros that carry either to carry both. Note that this will not require a native host - QEMU will use emulation if KVM acceleration is not available. *If* a native host is available that supports KVM, running the smoke test with KVM acceleration would definitely be preferred, especially on AARCH64, for reasons I pointed out earlier in this thread: the QEMU emulator is less strict than bare metal, and so issues related to cache maintenance or unaligned accesses to device mappings will only be caught when using KVM. KVM is also significantly faster, although I wouldn't expect that to matter for a boot-to-shell smoke test. I also think that having the EmulatorPkg based smoke test is already a huge step up from the current situation. But for changes to ArmPkg and ArmPlatformPkg, we need something on top of that, given that EmulatorPkg does not cover those at all. -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56556): https://edk2.groups.io/g/devel/message/56556 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
Ard/Laszlo or anyone familiar with QEMU. I read up on the ovmf readme and the qemu wiki but still have a few issues i am hoping for quick/easy answers. 1. How do i programmatically exit the emulator. Seems like uefi shell > reset just reboots. Other ideas? 2. Is there an easy way to map a local file system so that i can setup a startup.nsh? Thanks Sean -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56582): https://edk2.groups.io/g/devel/message/56582 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
> On Mar 29, 2020, at 4:16 PM, Sean via Groups.Io <sean.brogan=microsoft.com@groups.io> wrote: > > Ard/Laszlo or anyone familiar with QEMU. > > I read up on the ovmf readme and the qemu wiki but still have a few issues i am hoping for quick/easy answers. > > 1. How do i programmatically exit the emulator. Seems like uefi shell > reset just reboots. Other ideas? I'd like to know the answer to this too. I've ended up using gdb/lldb to kill it. Can you just kill the process after the test completes from the thread that is processing test results? > 2. Is there an easy way to map a local file system so that i can setup a startup.nsh? > I think you can mount a dir as a FAT32 disk by passing this to QEMU `-drive file=fat:rw:c:\Exchange,format=raw,media=disk` Thanks, Andrew Fish > Thanks > Sean > -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56586): https://edk2.groups.io/g/devel/message/56586 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On Mon, 30 Mar 2020 at 01:16, Sean via Groups.Io <sean.brogan=microsoft.com@groups.io> wrote: > > Ard/Laszlo or anyone familiar with QEMU. > > I read up on the ovmf readme and the qemu wiki but still have a few issues i am hoping for quick/easy answers. > > 1. How do i programmatically exit the emulator. Seems like uefi shell > reset just reboots. Other ideas? 'reset' is supposed to do that. Use 'reset -s' to kill the VM > 2. Is there an easy way to map a local file system so that i can setup a startup.nsh? > As Andrew points out, use '-hda fat:<path>' and it will be exposed to the VM as a FAT formatted block device. -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56601): https://edk2.groups.io/g/devel/message/56601 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On 03/30/20 08:07, Ard Biesheuvel wrote: > On Mon, 30 Mar 2020 at 01:16, Sean via Groups.Io > <sean.brogan=microsoft.com@groups.io> wrote: >> >> Ard/Laszlo or anyone familiar with QEMU. >> >> I read up on the ovmf readme and the qemu wiki but still have a few issues i am hoping for quick/easy answers. >> >> 1. How do i programmatically exit the emulator. Seems like uefi shell > reset just reboots. Other ideas? > > 'reset' is supposed to do that. Use 'reset -s' to kill the VM > >> 2. Is there an easy way to map a local file system so that i can setup a startup.nsh? >> > > As Andrew points out, use '-hda fat:<path>' and it will be exposed to > the VM as a FAT formatted block device. Note that it will not offer (functional, or any) write support. I prefer "mtools" or "guestfish" for formatting and populating a virtual disk image. None of those are easy ways, admittedly -- I'm not offering some examples right now exactly because I have to sit down and think about them every time I need them. :) If there's interest in such commands, I could hack up something, but then please give me some specs, and I'll have to work on these things in the morning (while my brain is still not mush). Thanks Laszlo -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56686): https://edk2.groups.io/g/devel/message/56686 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
Hi Laszlo, I think using the QEMU feature to map a path from the host as a FAT formatted driver has value to provision QEMU with a set of target tests to run. I agree there is no persistent storage of writes to this type of drive. I believe writes are supported if file state is needed during the single boot of QEMU. In order to get test results out of QEMU back to the host, I was considering the use of consoles mapped as named pipes and send all test results out the console device and the host can parse the logs. I have not had good experiences trying to build virtual disks, provision them, map them into QEMU, and remount from host to collect test results. Slow operations and cumbersome to mount/unmount from the host (at least from Windows environments). Mike > -----Original Message----- > From: devel@edk2.groups.io <devel@edk2.groups.io> On > Behalf Of Laszlo Ersek > Sent: Monday, March 30, 2020 2:11 PM > To: devel@edk2.groups.io; ard.biesheuvel@linaro.org; Sean > Brogan <sean.brogan@microsoft.com> > Subject: Re: [edk2-devel] [PATCH] .azurepipelines: Enable > CI for OvmfPkg and EmulatorPkg > > On 03/30/20 08:07, Ard Biesheuvel wrote: > > On Mon, 30 Mar 2020 at 01:16, Sean via Groups.Io > > <sean.brogan=microsoft.com@groups.io> wrote: > >> > >> Ard/Laszlo or anyone familiar with QEMU. > >> > >> I read up on the ovmf readme and the qemu wiki but > still have a few issues i am hoping for quick/easy > answers. > >> > >> 1. How do i programmatically exit the emulator. > Seems like uefi shell > reset just reboots. Other ideas? > > > > 'reset' is supposed to do that. Use 'reset -s' to kill > the VM > > > >> 2. Is there an easy way to map a local file system so > that i can setup a startup.nsh? > >> > > > > As Andrew points out, use '-hda fat:<path>' and it will > be exposed to > > the VM as a FAT formatted block device. > > Note that it will not offer (functional, or any) write > support. > > I prefer "mtools" or "guestfish" for formatting and > populating a virtual > disk image. > > None of those are easy ways, admittedly -- I'm not > offering some > examples right now exactly because I have to sit down and > think about > them every time I need them. :) If there's interest in > such commands, I > could hack up something, but then please give me some > specs, and I'll > have to work on these things in the morning (while my > brain is still not > mush). > > Thanks > Laszlo > > > -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56690): https://edk2.groups.io/g/devel/message/56690 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On 03/30/20 23:29, Kinney, Michael D wrote: > Hi Laszlo, > > I think using the QEMU feature to map a path from the host as > a FAT formatted driver has value to provision QEMU with a set > of target tests to run. I agree there is no persistent storage > of writes to this type of drive. I believe writes are supported > if file state is needed during the single boot of QEMU. > > In order to get test results out of QEMU back to the host, > I was considering the use of consoles mapped as named pipes > and send all test results out the console device and the > host can parse the logs. Yes, this should be viable. > I have not had good experiences trying to build virtual > disks, provision them, map them into QEMU, and remount > from host to collect test results. Slow operations and > cumbersome to mount/unmount from the host (at least from > Windows environments). I must agree. It is slow and somewhat cumbersome with "guestfish" too, on Linux. Guestfish, on the plus side, does not mount the disk with the host kernel, and so it doesn't need root rights. Guestfish launches a separate QEMU instance on top of the virtual disk, booting a dedicated kernel and agent (the "libguestfs appliance") in the guest. The guest appliance communicates with the "guestfish" host side command shell over virtio-serial. So the host kernel never trusts (or even sees) the filesystem structures of the guest disk image; files are exchanged between the "libguestfs appliance" (= guest kernel + agent) and the host-side command shell using a dedicated wire protocol. But, I agree it is somewhat cumbersome and slow, and I have no idea whether it works on Windows hosts. Thanks, Laszlo > > Mike > >> -----Original Message----- >> From: devel@edk2.groups.io <devel@edk2.groups.io> On >> Behalf Of Laszlo Ersek >> Sent: Monday, March 30, 2020 2:11 PM >> To: devel@edk2.groups.io; ard.biesheuvel@linaro.org; Sean >> Brogan <sean.brogan@microsoft.com> >> Subject: Re: [edk2-devel] [PATCH] .azurepipelines: Enable >> CI for OvmfPkg and EmulatorPkg >> >> On 03/30/20 08:07, Ard Biesheuvel wrote: >>> On Mon, 30 Mar 2020 at 01:16, Sean via Groups.Io >>> <sean.brogan=microsoft.com@groups.io> wrote: >>>> >>>> Ard/Laszlo or anyone familiar with QEMU. >>>> >>>> I read up on the ovmf readme and the qemu wiki but >> still have a few issues i am hoping for quick/easy >> answers. >>>> >>>> 1. How do i programmatically exit the emulator. >> Seems like uefi shell > reset just reboots. Other ideas? >>> >>> 'reset' is supposed to do that. Use 'reset -s' to kill >> the VM >>> >>>> 2. Is there an easy way to map a local file system so >> that i can setup a startup.nsh? >>>> >>> >>> As Andrew points out, use '-hda fat:<path>' and it will >> be exposed to >>> the VM as a FAT formatted block device. >> >> Note that it will not offer (functional, or any) write >> support. >> >> I prefer "mtools" or "guestfish" for formatting and >> populating a virtual >> disk image. >> >> None of those are easy ways, admittedly -- I'm not >> offering some >> examples right now exactly because I have to sit down and >> think about >> them every time I need them. :) If there's interest in >> such commands, I >> could hack up something, but then please give me some >> specs, and I'll >> have to work on these things in the morning (while my >> brain is still not >> mush). >> >> Thanks >> Laszlo >> >> >> > -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56758): https://edk2.groups.io/g/devel/message/56758 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
@Rebecca - Agree. I need to package up a newer version. I have treated this as lower priority. Is there a feature you need or just best practices on keeping current? @Ard - Padding in python is easy. I just need to understand the requirements. @Mike - I would like to get to read/write filesystem for Target based tests so that we don't have to reinvent yet another way to collect results (especially one as error prone as sniffing logs). But lets get basic CI with read-only functionality working and checked in first. As it looks like in the last 8 hours another breaking change has been checked in. After rebase to top of edk2 master. NFO - "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.24.28314\bin\Hostx86\x64\link.exe" /OUT:d:\a\1\s\Build\Ovmf3264\RELEASE_VS2019\X64\OvmfPkg\QemuKernelLoaderFsDxe\QemuKernelLoaderFsDxe\DEBUG\QemuKernelLoaderFsDxe.dll /NOLOGO /NODEFAULTLIB /IGNORE:4001 /IGNORE:4281 /IGNORE:4254 /OPT:REF /OPT:ICF=10 /MAP /ALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /Machine:X64 /LTCG /DLL /ENTRY:_ModuleEntryPoint /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DRIVER /MERGE:.rdata=.data @d:\a\1\s\Build\Ovmf3264\RELEASE_VS2019\X64\OvmfPkg\QemuKernelLoaderFsDxe\QemuKernelLoaderFsDxe\OUTPUT\static_library_files.lst INFO - PvScsi.c INFO - Generating code INFO - d:\a\1\s\OvmfPkg\PvScsiDxe\PvScsi.c(459): error C2220: the following warning is treated as an error INFO - d:\a\1\s\OvmfPkg\PvScsiDxe\PvScsi.c(459): warning C4244: '=': conversion from 'const UINT16' to 'UINT8', possible loss of data INFO - Finished generating code INFO - NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.24.28314\bin\Hostx86\x64\cl.exe"' : return code '0x2' INFO - Stop. INFO - "GenFw" -e DXE_DRIVER -o d:\a\1\s\Build\Ovmf3264\RELEASE_VS2019\X64\OvmfPkg\QemuKernelLoaderFsDxe\QemuKernelLoaderFsDxe\OUTPUT\QemuKernelLoaderFsDxe.efi d:\a\1\s\Build\Ovmf3264\RELEASE_VS2019\X64\OvmfPkg\QemuKernelLoaderFsDxe\QemuKernelLoaderFsDxe\DEBUG\QemuKernelLoaderFsDxe.dll INFO - INFO - INFO - build.py... INFO - : error 7000: Failed to execute command INFO - C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.24.28314\bin\Hostx86\x86\nmake.exe /nologo tbuild [d:\a\1\s\Build\Ovmf3264\RELEASE_VS2019\X64\OvmfPkg\PvScsiDxe\PvScsiDxe] INFO - INFO - INFO - build.py... INFO - : error F002: Failed to build module INFO - d:\a\1\s\OvmfPkg\PvScsiDxe\PvScsiDxe.inf [X64, VS2019, RELEASE] INFO - INFO - - Failed - -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56692): https://edk2.groups.io/g/devel/message/56692 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
Sean, I agree a r/w file system with simple access from host and QEMU is a much better choice. When Laszlo gets some time, I would be interested in his recommendations. My negative experience was from a Windows host. Perhaps that are must better virtual disk tools from a Linux host. Mike From: sean.brogan via [] <sean.brogan=microsoft.com@[]> Sent: Monday, March 30, 2020 2:42 PM To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io Subject: Re: [edk2-devel] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg @Rebecca - Agree. I need to package up a newer version. I have treated this as lower priority. Is there a feature you need or just best practices on keeping current? @Ard - Padding in python is easy. I just need to understand the requirements. @Mike - I would like to get to read/write filesystem for Target based tests so that we don't have to reinvent yet another way to collect results (especially one as error prone as sniffing logs). But lets get basic CI with read-only functionality working and checked in first. As it looks like in the last 8 hours another breaking change has been checked in. After rebase to top of edk2 master. NFO - "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.24.28314\bin\Hostx86\x64\link.exe" /OUT:d:\a\1\s\Build\Ovmf3264\RELEASE_VS2019\X64\OvmfPkg\QemuKernelLoaderFsDxe\QemuKernelLoaderFsDxe\DEBUG\QemuKernelLoaderFsDxe.dll /NOLOGO /NODEFAULTLIB /IGNORE:4001 /IGNORE:4281 /IGNORE:4254 /OPT:REF /OPT:ICF=10 /MAP /ALIGN:32 /SECTION:.xdata,D /SECTION:.pdata,D /Machine:X64 /LTCG /DLL /ENTRY:_ModuleEntryPoint /SUBSYSTEM:EFI_BOOT_SERVICE_DRIVER /SAFESEH:NO /BASE:0 /DRIVER /MERGE:.rdata=.data @d:\a\1\s\Build\Ovmf3264\RELEASE_VS2019\X64\OvmfPkg\QemuKernelLoaderFsDxe\QemuKernelLoaderFsDxe\OUTPUT\static_library_files.lst INFO - PvScsi.c INFO - Generating code INFO - d:\a\1\s\OvmfPkg\PvScsiDxe\PvScsi.c(459): error C2220: the following warning is treated as an error INFO - d:\a\1\s\OvmfPkg\PvScsiDxe\PvScsi.c(459): warning C4244: '=': conversion from 'const UINT16' to 'UINT8', possible loss of data INFO - Finished generating code INFO - NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.24.28314\bin\Hostx86\x64\cl.exe"' : return code '0x2' INFO - Stop. INFO - "GenFw" -e DXE_DRIVER -o d:\a\1\s\Build\Ovmf3264\RELEASE_VS2019\X64\OvmfPkg\QemuKernelLoaderFsDxe\QemuKernelLoaderFsDxe\OUTPUT\QemuKernelLoaderFsDxe.efi d:\a\1\s\Build\Ovmf3264\RELEASE_VS2019\X64\OvmfPkg\QemuKernelLoaderFsDxe\QemuKernelLoaderFsDxe\DEBUG\QemuKernelLoaderFsDxe.dll INFO - INFO - INFO - build.py... INFO - : error 7000: Failed to execute command INFO - C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.24.28314\bin\Hostx86\x86\nmake.exe /nologo tbuild [d:\a\1\s\Build\Ovmf3264\RELEASE_VS2019\X64\OvmfPkg\PvScsiDxe\PvScsiDxe] INFO - INFO - INFO - build.py... INFO - : error F002: Failed to build module INFO - d:\a\1\s\OvmfPkg\PvScsiDxe\PvScsiDxe.inf [X64, VS2019, RELEASE] INFO - INFO - - Failed - -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56695): https://edk2.groups.io/g/devel/message/56695 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On 3/30/20 3:42 PM, Sean via Groups.Io wrote: > @Rebecca - Agree. I need to package up a newer version. I have > treated this as lower priority. Is there a feature you need or just > best practices on keeping current? Just best practices around keeping current. I don't think it's a problem just now, but I start getting anxious when I see software being used that's _several_ years old, unsupported and there's a fairly straightforward upgrade path. -- Rebecca Cran -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56694): https://edk2.groups.io/g/devel/message/56694 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On Mon, 30 Mar 2020 at 23:42, Sean via Groups.Io <sean.brogan=microsoft.com@groups.io> wrote: > > @Rebecca - Agree. I need to package up a newer version. I have treated this as lower priority. Is there a feature you need or just best practices on keeping current? > @Ard - Padding in python is easy. I just need to understand the requirements. OK, in that case, please switch to a single -pflash argument from -bios, but pad the QEMU_EFI.fd file to exactly 64 MiB first. I also noticed that you are logging to a file for OVMF - you might want to do the same for ArmVirtQemu, by using "-serial file:" + QemuLogFile instead of 'stdio' -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56693): https://edk2.groups.io/g/devel/message/56693 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
@Ard - pflash change: https://github.com/spbrogan/edk2/pull/12 Logging change - I actually switched OVMF to use stdio since the log is captured either way and now it shows up in the web log output. https://github.com/spbrogan/edk2/pull/13 Do you have instructions for the cmdline for Qemu for Arm? Is it something you would like to run? All the builds can be seen here or clicking the icon in github: https://dev.azure.com/tianocore/edk2-ci-play/_build?view=folders&treeState=XEFybVZpcnRQa2ckXEVtdWxhdG9yUGtnJFxPVk1G -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56722): https://edk2.groups.io/g/devel/message/56722 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On Tue, 31 Mar 2020 at 08:31, Sean via Groups.Io <sean.brogan=microsoft.com@groups.io> wrote: > > @Ard - > pflash change: https://github.com/spbrogan/edk2/pull/12 > > Logging change - I actually switched OVMF to use stdio since the log is captured either way and now it shows up in the web log output. https://github.com/spbrogan/edk2/pull/13 > Even better. > Do you have instructions for the cmdline for Qemu for Arm? Is it something you would like to run? > Not sure I follow. Which command line are we talking about? > All the builds can be seen here or clicking the icon in github: > https://dev.azure.com/tianocore/edk2-ci-play/_build?view=folders&treeState=XEFybVZpcnRQa2ckXEVtdWxhdG9yUGtnJFxPVk1G Thanks again for the effort. This is going to help tremendously. -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56723): https://edk2.groups.io/g/devel/message/56723 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On Mon, Mar 30, 2020 at 11:41 PM, Ard Biesheuvel wrote: > > Not sure I follow. Which command line are we talking about? @Ard - In this Platform CI, ArmVirt is building and running AARCH64 but not ARM 32bit. Would it be valuable to build for ARM too? I prototyped it but want to make sure I am calling QEMU with the parameters you would expect to work with ArmVirtPkg qemu-system-arm -M virt -cpu cortex-a15 -pflash /home/vsts/work/1/s/Build/ArmVirtQemu-ARM/DEBUG_GCC5/FV/QEMU_EFI.fd -m 1024 -net none -serial stdio -drive file=fat:rw:/home/vsts/work/1/s/Build/ArmVirtQemu-ARM/DEBUG_GCC5/VirtualDrive,format=raw,media=disk -display none PR build results can be seen here: https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=5074&view=results PR code change: https://github.com/spbrogan/edk2/pull/14 -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56772): https://edk2.groups.io/g/devel/message/56772 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On 3/31/20 6:26 PM, Sean via Groups.Io wrote: > On Mon, Mar 30, 2020 at 11:41 PM, Ard Biesheuvel wrote: > > Not sure I follow. Which command line are we talking about? > > @Ard - In this Platform CI, ArmVirt is building and running AARCH64 but > not ARM 32bit. Would it be valuable to build for ARM too? > Yes, absolutely. > I prototyped it but want to make sure I am calling QEMU with the > parameters you would expect to work with ArmVirtPkg > > qemu-system-arm -M virt -cpu cortex-a15 -pflash > /home/vsts/work/1/s/Build/ArmVirtQemu-ARM/DEBUG_GCC5/FV/QEMU_EFI.fd -m > 1024 -net none -serial stdio -drive > file=fat:rw:/home/vsts/work/1/s/Build/ArmVirtQemu-ARM/DEBUG_GCC5/VirtualDrive,format=raw,media=disk > -display none > The parameters look fine. Note that you can use the qemu-system-aarch64 binary as well, so no need to install both. > > PR build results can be seen here: > https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=5074&view=results > PR code change: https://github.com/spbrogan/edk2/pull/14 Yep, looking good. The main focus is obviously on X64 and AARCH64, but that only increases the risk that we might regress on IA32 or ARM without anyone noticing. So having both IA32 and ARM is a good thing. -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56773): https://edk2.groups.io/g/devel/message/56773 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
Thanks. I was missing the "-s" and Andrew you solution worked for the filesystem. OVMF: Windows builds and Linux builds and boot to shell. IA32, X64 and IA32x64 - https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=4950&view=results Emulator: Windows and linux builds and boots to shell. https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=4922&view=results https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=4921&view=results Src: Needs commit cleanup but its all here. https://github.com/spbrogan/edk2/tree/ci-for-ovmf Still have a few loose ends on Windows pipeline 1. Windows isn't able to find the QEMU i installed so it is failing to run the emulator. -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56619): https://edk2.groups.io/g/devel/message/56619 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On Mon, 30 Mar 2020 at 11:31, Sean via Groups.Io <sean.brogan=microsoft.com@groups.io> wrote: > > Thanks. I was missing the "-s" and Andrew you solution worked for the filesystem. > > OVMF: Windows builds and Linux builds and boot to shell. IA32, X64 and IA32x64 - https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=4950&view=results > Emulator: Windows and linux builds and boots to shell. https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=4922&view=results https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=4921&view=results > > Src: Needs commit cleanup but its all here. https://github.com/spbrogan/edk2/tree/ci-for-ovmf > > Still have a few loose ends on Windows pipeline > 1. Windows isn't able to find the QEMU i installed so it is failing to run the emulator. Excellent! It should be trivial to extend this to ARM, using TCG emulation. One question though: what happens if the boot does not make it to the shell, and crashes for some reason? The QEMU process will hang, so I'd assume some kind of timeout should be applied? -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56620): https://edk2.groups.io/g/devel/message/56620 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On Mon, Mar 30, 2020 at 02:36 AM, Ard Biesheuvel wrote: > > It should be trivial to extend this to ARM, using TCG emulation. > > One question though: what happens if the boot does not make it to the > shell, and crashes for some reason? The QEMU process will hang, so I'd > assume some kind of timeout should be applied? It has a 1 minute timeout and then the build will kill the process. The bootlog is uploaded in all cases so you can then look at the log. Here is an example where i didn't configure QEMU right: https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=4942&view=logs&j=6fb09fdb-58e7-5b12-0698-873f788bd2e9&t=e63402bd-99b5-5ddd-38f9-868e9402ecc0 And the last lines in the bootlog.txt show MaxCpuCountInitialization: BootCpuCount=1 mMaxCpuCount=1 Q35BoardVerification: no TSEG (SMRAM) on host bridge DID=0x1237; only DID=0x29C0 (Q35) is supported ASSERT /home/vsts/work/1/s/OvmfPkg/PlatformPei/Platform.c(564): ((BOOLEAN)(0==1)) -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56672): https://edk2.groups.io/g/devel/message/56672 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On Mon, 30 Mar 2020 at 19:01, Sean via Groups.Io <sean.brogan=microsoft.com@groups.io> wrote: > > On Mon, Mar 30, 2020 at 02:36 AM, Ard Biesheuvel wrote: > > It should be trivial to extend this to ARM, using TCG emulation. > > One question though: what happens if the boot does not make it to the > shell, and crashes for some reason? The QEMU process will hang, so I'd > assume some kind of timeout should be applied? > > It has a 1 minute timeout and then the build will kill the process. > > The bootlog is uploaded in all cases so you can then look at the log. > Here is an example where i didn't configure QEMU right: https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=4942&view=logs&j=6fb09fdb-58e7-5b12-0698-873f788bd2e9&t=e63402bd-99b5-5ddd-38f9-868e9402ecc0 > > And the last lines in the bootlog.txt show > > MaxCpuCountInitialization: BootCpuCount=1 mMaxCpuCount=1 > Q35BoardVerification: no TSEG (SMRAM) on host bridge DID=0x1237; only DID=0x29C0 (Q35) is supported > ASSERT /home/vsts/work/1/s/OvmfPkg/PlatformPei/Platform.c(564): ((BOOLEAN)(0==1)) > OK, great. So we're all set :-) Is there any way I could contribute ArmVirtQemu to this? Or would it be easier if I provided comments/instructions? -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56673): https://edk2.groups.io/g/devel/message/56673 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On Mon, Mar 30, 2020 at 10:04 AM, Ard Biesheuvel wrote: > > Is there any way I could contribute ArmVirtQemu to this? Or would it > be easier if I provided comments/instructions? Either way. Any instructions you provide would be great. I was going to hack something up for feedback but happy for someone else to do it. Let me know. Basically add the platform and pipeline yml here https://github.com/spbrogan/edk2/tree/ci-for-ovmf/.azurepipelines/platforms Then i have been leveraging PyTools to build and script operations. So something like this. https://github.com/spbrogan/edk2/blob/ci-for-ovmf/OvmfPkg/PlatformBuild.py A little bit of help in this readme. https://github.com/spbrogan/edk2/blob/ci-for-ovmf/OvmfPkg/README-pytools.md -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56674): https://edk2.groups.io/g/devel/message/56674 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On Mon, 30 Mar 2020 at 19:11, Sean via Groups.Io <sean.brogan=microsoft.com@groups.io> wrote: > > On Mon, Mar 30, 2020 at 10:04 AM, Ard Biesheuvel wrote: > > Is there any way I could contribute ArmVirtQemu to this? Or would it > be easier if I provided comments/instructions? > > Either way. > Any instructions you provide would be great. I was going to hack something up for feedback but happy for someone else to do it. Let me know. OK, so the typical invocation would be qemu-system-aarch64 -M virt -cpu cortex-a57 -m 1024 -net none -nographic -bios .../path/to/QEMU_EFI.fd -hda fat:rw:.../path/to/startup.nsh The only complication compared to OVMF is that there is no separate serial port for debug output vs console output, so everything is going to come out of the same pipe, and grep'ing the console output for meaningful strings may easily result in false positives. (-pflash could be used as well, but doesn't really add anything in this case, and QEMU for ARM has a quirk where pflash images must be exactly 64 MB in size) > > Basically add the platform and pipeline yml here > https://github.com/spbrogan/edk2/tree/ci-for-ovmf/.azurepipelines/platforms > > Then i have been leveraging PyTools to build and script operations. > So something like this. https://github.com/spbrogan/edk2/blob/ci-for-ovmf/OvmfPkg/PlatformBuild.py > > A little bit of help in this readme. > https://github.com/spbrogan/edk2/blob/ci-for-ovmf/OvmfPkg/README-pytools.md > Thanks. But can I get these actions to trigger from my branch as well? Or do I need special powers for that? -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56677): https://edk2.groups.io/g/devel/message/56677 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On 03/30/20 19:44, Ard Biesheuvel wrote: > On Mon, 30 Mar 2020 at 19:11, Sean via Groups.Io > <sean.brogan=microsoft.com@groups.io> wrote: >> >> On Mon, Mar 30, 2020 at 10:04 AM, Ard Biesheuvel wrote: >> >> Is there any way I could contribute ArmVirtQemu to this? Or would it >> be easier if I provided comments/instructions? >> >> Either way. >> Any instructions you provide would be great. I was going to hack something up for feedback but happy for someone else to do it. Let me know. > > OK, so the typical invocation would be > > qemu-system-aarch64 -M virt -cpu cortex-a57 -m 1024 -net none > -nographic -bios .../path/to/QEMU_EFI.fd -hda > fat:rw:.../path/to/startup.nsh > > The only complication compared to OVMF is that there is no separate > serial port for debug output vs console output, so everything is going > to come out of the same pipe, and grep'ing the console output for > meaningful strings may easily result in false positives. (-pflash > could be used as well, but doesn't really add anything in this case, > and QEMU for ARM has a quirk where pflash images must be exactly 64 MB > in size) I'm begging you not to introduce instances of "-bios" anywhere near edk2. Please? :) We need to educate QEMU users, and we need to keep our sanity when facing bug reports. Let's not set bad examples anywhere, if we can manage. I think truncate(1) can do what we need for padding, without actually allocating those MBs. Thanks! Laszlo -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56684): https://edk2.groups.io/g/devel/message/56684 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On Mon, 30 Mar 2020 at 22:56, Laszlo Ersek <lersek@redhat.com> wrote: > > On 03/30/20 19:44, Ard Biesheuvel wrote: > > On Mon, 30 Mar 2020 at 19:11, Sean via Groups.Io > > <sean.brogan=microsoft.com@groups.io> wrote: > >> > >> On Mon, Mar 30, 2020 at 10:04 AM, Ard Biesheuvel wrote: > >> > >> Is there any way I could contribute ArmVirtQemu to this? Or would it > >> be easier if I provided comments/instructions? > >> > >> Either way. > >> Any instructions you provide would be great. I was going to hack something up for feedback but happy for someone else to do it. Let me know. > > > > OK, so the typical invocation would be > > > > qemu-system-aarch64 -M virt -cpu cortex-a57 -m 1024 -net none > > -nographic -bios .../path/to/QEMU_EFI.fd -hda > > fat:rw:.../path/to/startup.nsh > > > > The only complication compared to OVMF is that there is no separate > > serial port for debug output vs console output, so everything is going > > to come out of the same pipe, and grep'ing the console output for > > meaningful strings may easily result in false positives. (-pflash > > could be used as well, but doesn't really add anything in this case, > > and QEMU for ARM has a quirk where pflash images must be exactly 64 MB > > in size) > > I'm begging you not to introduce instances of "-bios" anywhere near > edk2. Please? :) We need to educate QEMU users, and we need to keep our > sanity when facing bug reports. Let's not set bad examples anywhere, if > we can manage. > > I think truncate(1) can do what we need for padding, without actually > allocating those MBs. > truncate should work on Linux, but I have no idea how to pad an image to a certain size on Windows, and I think the idea was to enable both? In any case, I don't feel as strongly about this as Laszlo does: even though -bios really shouldn't be used when you are actually installing an OS into the VM, I don't think it is inappropriate for booting into a shell and nothing else. Perhaps we could annotate the scripts in a way that discourages people from adopting it? Or if there is an easy way to make this work on both Windows and Linux using -pflash, I obviously wouldn't mind. I just don't feel it is essential. -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56688): https://edk2.groups.io/g/devel/message/56688 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On 03/30/20 23:22, Ard Biesheuvel wrote: > On Mon, 30 Mar 2020 at 22:56, Laszlo Ersek <lersek@redhat.com> wrote: >> >> On 03/30/20 19:44, Ard Biesheuvel wrote: >>> On Mon, 30 Mar 2020 at 19:11, Sean via Groups.Io >>> <sean.brogan=microsoft.com@groups.io> wrote: >>>> >>>> On Mon, Mar 30, 2020 at 10:04 AM, Ard Biesheuvel wrote: >>>> >>>> Is there any way I could contribute ArmVirtQemu to this? Or would it >>>> be easier if I provided comments/instructions? >>>> >>>> Either way. >>>> Any instructions you provide would be great. I was going to hack something up for feedback but happy for someone else to do it. Let me know. >>> >>> OK, so the typical invocation would be >>> >>> qemu-system-aarch64 -M virt -cpu cortex-a57 -m 1024 -net none >>> -nographic -bios .../path/to/QEMU_EFI.fd -hda >>> fat:rw:.../path/to/startup.nsh >>> >>> The only complication compared to OVMF is that there is no separate >>> serial port for debug output vs console output, so everything is going >>> to come out of the same pipe, and grep'ing the console output for >>> meaningful strings may easily result in false positives. (-pflash >>> could be used as well, but doesn't really add anything in this case, >>> and QEMU for ARM has a quirk where pflash images must be exactly 64 MB >>> in size) >> >> I'm begging you not to introduce instances of "-bios" anywhere near >> edk2. Please? :) We need to educate QEMU users, and we need to keep our >> sanity when facing bug reports. Let's not set bad examples anywhere, if >> we can manage. >> >> I think truncate(1) can do what we need for padding, without actually >> allocating those MBs. >> > > truncate should work on Linux, but I have no idea how to pad an image > to a certain size on Windows, and I think the idea was to enable both? > > In any case, I don't feel as strongly about this as Laszlo does: even > though -bios really shouldn't be used when you are actually installing > an OS into the VM, I don't think it is inappropriate for booting into > a shell and nothing else. Perhaps we could annotate the scripts in a > way that discourages people from adopting it? > > Or if there is an easy way to make this work on both Windows and Linux > using -pflash, I obviously wouldn't mind. I just don't feel it is > essential. OK, thank you. Meta-comment: while I have strong opinions about these matters, I'm 100% aware that I cannot put in basically any actual work. So I'm trying to police myself in that I make suggestions politely and really only as moderate suggestions, not as "demands" or knee-jerk reactions. Unfortunately, I sometimes slip up, and my thoughts hit the list in less than optimal form. I'm really sorry about that! It's a fine line -- I don't want to ignore this topic (esp. when asked for an opinion), but then again I shouldn't present unreasonable "requirements", without the capacity to contribute code. Please bear with me. :) Thanks! Laszlo -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56755): https://edk2.groups.io/g/devel/message/56755 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
Laszlo/Ard - I have cleaned up git history and pushed to https://github.com/spbrogan/edk2 master. If you want you can create a PR against that branch and it should kick OVMF, Emulator, and ArmVirt to build and run. Looking for feedback. @Laszlo - I followed Ard's parameters but feel free to make a PR that changes it to not use "-bios". :) -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56685): https://edk2.groups.io/g/devel/message/56685 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
Platform and Core CI patches are ready for review. I have 14 commits here. https://github.com/spbrogan/edk2/tree/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages This adds support for "Platform CI" for ArmVirtPkg, OvmfPkg, and EmulatorPkg. Each readme has live status and links to the builds as well as details of how to build and run the same way the CI server will. ArmVirt: https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/ArmVirtPkg/README-pytools.md Emulator: https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/EmulatorPkg/README-pytools.md OvmfPkg: https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/OvmfPkg/README-pytools.md It also adds ArmVirtPkg, OvmfPkg, and EmulatorPkg to Core CI just for the code evaluation tests (not compiling). Details of those tests are here: https://github.com/spbrogan/edk2/tree/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/.pytool ArmVirtPkg, OvmfPkg, and EmulatorPkg maintainers please review and let me know what the next step for you package is. This is ready to commit from my perspective and has already caught a few issues in the last couple of weeks. Finally there are a few pending bugs that should get fixed and at least a couple of gaps I have identified. These are not fixed in the above 14 commits. * OvmfPkg * https://bugzilla.tianocore.org/show_bug.cgi?id=2662 -- Blocking Core CI * https://bugzilla.tianocore.org/show_bug.cgi?id=2661 -- Blocking Platform CI * Spell check in audit mode * EmulatorPkg * https://bugzilla.tianocore.org/show_bug.cgi?id=2663 -- Ignore added to Core CI * https://bugzilla.tianocore.org/show_bug.cgi?id=2639 -- Feature in Platform CI turned off * https://bugzilla.tianocore.org/show_bug.cgi?id=2638 -- Feature in Platform CI turned off * Spell check in audit mode * ArmVirtPkg * No builds on Windows or with VS toolchain -- Feature in Platform CI turned off * Other * IASL versions are from 2019 -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56966): https://edk2.groups.io/g/devel/message/56966 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On 4/5/20 8:39 AM, Sean via groups.io wrote: > Platform and Core CI patches are ready for review. I have 14 commits here. > https://github.com/spbrogan/edk2/tree/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages > > This adds support for "Platform CI" for ArmVirtPkg, OvmfPkg, and > EmulatorPkg. > Each readme has live status and links to the builds as well as details > of how to build and run the same way the CI server will. > ArmVirt: > https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/ArmVirtPkg/README-pytools.md > Emulator: > https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/EmulatorPkg/README-pytools.md > OvmfPkg: > https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/OvmfPkg/README-pytools.md > > It also adds ArmVirtPkg, OvmfPkg, and EmulatorPkg to Core CI just for > the code evaluation tests (not compiling). Details of those tests are > here: > https://github.com/spbrogan/edk2/tree/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/.pytool > > ArmVirtPkg, OvmfPkg, and EmulatorPkg maintainers please review and let > me know what the next step for you package is. This is ready to commit > from my perspective and has already caught a few issues in the last > couple of weeks. > Thanks Sean. This is all looking really good. I think this stuff is ready to be sent to the mailing list for wider review. > Finally there are a few pending bugs that should get fixed and at least > a couple of gaps I have identified. These are not fixed in the above 14 > commits. > > 1. OvmfPkg > 1. https://bugzilla.tianocore.org/show_bug.cgi?id=2662 -- Blocking > Core CI > 2. https://bugzilla.tianocore.org/show_bug.cgi?id=2661 -- Blocking > Platform CI > 3. Spell check in audit mode > 2. EmulatorPkg > 1. https://bugzilla.tianocore.org/show_bug.cgi?id=2663 -- Ignore > added to Core CI > 2. https://bugzilla.tianocore.org/show_bug.cgi?id=2639 -- Feature > in Platform CI turned off > 3. https://bugzilla.tianocore.org/show_bug.cgi?id=2638 -- Feature > in Platform CI turned off > 4. Spell check in audit mode > 3. ArmVirtPkg > 1. No builds on Windows or with VS toolchain -- Feature in Platform > CI turned off Leif has looked into this in the past. As I understand it, it is simply a lack of .asm versions of the various assembler source files in ArmLib, ArmMmuLib and the startup code in ArmPlatformPkg. -- Ard. -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56975): https://edk2.groups.io/g/devel/message/56975 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On 04/06/20 12:11, Ard Biesheuvel wrote: > On 4/5/20 8:39 AM, Sean via groups.io wrote: >> Platform and Core CI patches are ready for review. I have 14 commits >> here. >> https://github.com/spbrogan/edk2/tree/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages >> >> >> This adds support for "Platform CI" for ArmVirtPkg, OvmfPkg, and >> EmulatorPkg. >> Each readme has live status and links to the builds as well as details >> of how to build and run the same way the CI server will. >> ArmVirt: >> https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/ArmVirtPkg/README-pytools.md >> >> Emulator: >> https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/EmulatorPkg/README-pytools.md >> >> OvmfPkg: >> https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/OvmfPkg/README-pytools.md >> >> >> It also adds ArmVirtPkg, OvmfPkg, and EmulatorPkg to Core CI just for >> the code evaluation tests (not compiling). Details of those tests are >> here: >> https://github.com/spbrogan/edk2/tree/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/.pytool >> >> >> ArmVirtPkg, OvmfPkg, and EmulatorPkg maintainers please review and let >> me know what the next step for you package is. This is ready to >> commit from my perspective and has already caught a few issues in the >> last couple of weeks. >> > > Thanks Sean. This is all looking really good. I think this stuff is > ready to be sent to the mailing list for wider review. > >> Finally there are a few pending bugs that should get fixed and at >> least a couple of gaps I have identified. These are not fixed in the >> above 14 commits. >> >> 1. OvmfPkg >> 1. https://bugzilla.tianocore.org/show_bug.cgi?id=2662 -- Blocking >> Core CI >> 2. https://bugzilla.tianocore.org/show_bug.cgi?id=2661 -- Blocking Thanks Sean! I've posted a patch for 2662, and commented on 2661. >> Platform CI >> 3. Spell check in audit mode >> 2. EmulatorPkg >> 1. https://bugzilla.tianocore.org/show_bug.cgi?id=2663 -- Ignore >> added to Core CI >> 2. https://bugzilla.tianocore.org/show_bug.cgi?id=2639 -- Feature >> in Platform CI turned off >> 3. https://bugzilla.tianocore.org/show_bug.cgi?id=2638 -- Feature >> in Platform CI turned off >> 4. Spell check in audit mode >> 3. ArmVirtPkg >> 1. No builds on Windows or with VS toolchain -- Feature in Platform >> CI turned off For the ArmVirtPkg "README-pytools.md" file, I have a superficial comment: The "E1000_ENABLE" reference may not be the best example (I understand it's an example) with ArmVirtPkg, as E1000_ENABLE isn't actually used in the ArmVirtPkg DSC/FDF files. Thanks Laszlo > > Leif has looked into this in the past. As I understand it, it is simply > a lack of .asm versions of the various assembler source files in ArmLib, > ArmMmuLib and the startup code in ArmPlatformPkg. > > > -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#57025): https://edk2.groups.io/g/devel/message/57025 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On 3/30/20 3:03 PM, Sean via Groups.Io wrote: > Laszlo/Ard - > > I have cleaned up git history and pushed to > https://github.com/spbrogan/edk2 master. > > If you want you can create a PR against that branch and it should kick > OVMF, Emulator, and ArmVirt to build and run. > > Looking for feedback. The 20190215 release of iasl/acpica-tools is pretty old (https://acpica.org/downloads). Should it be bumped to something more recent? -- Rebecca Cran -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56687): https://edk2.groups.io/g/devel/message/56687 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On Mon, Mar 30, 2020 at 10:44 AM, Ard Biesheuvel wrote: > > Thanks. But can I get these actions to trigger from my branch as well? > Or do I need special powers for that? Right now it requires manually running so it requires "special powers". :) I am happy to get this into a branch that you can PR against and on each PR it will run. Basically you would just submit PRs against the branch in my edk2 fork. I have it running successfully on GCC5 toolchain and AARCH64 https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=4978&view=results I don't see any output from the Qemu boot process even for debug. Am I missing a parameter? qemu-system-aarch64 -M virt -cpu cortex-a57 -bios /home/vsts/work/1/s/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/FV/QEMU_EFI.fd -m 1024 -net none -drive file=fat:rw:/home/vsts/work/1/s/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/VirtualDrive,format=raw,media=disk -display none -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56680): https://edk2.groups.io/g/devel/message/56680 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On Mon, 30 Mar 2020 at 21:07, Sean via Groups.Io <sean.brogan=microsoft.com@groups.io> wrote: > > > > On Mon, Mar 30, 2020 at 10:44 AM, Ard Biesheuvel wrote: > > Thanks. But can I get these actions to trigger from my branch as well? > Or do I need special powers for that? > > Right now it requires manually running so it requires "special powers". :) > I am happy to get this into a branch that you can PR against and on each PR it will run. Basically you would just submit PRs against the branch in my edk2 fork. > > I have it running successfully on GCC5 toolchain and AARCH64 > https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=4978&view=results > > I don't see any output from the Qemu boot process even for debug. Am I missing a parameter? > qemu-system-aarch64 -M virt -cpu cortex-a57 -bios /home/vsts/work/1/s/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/FV/QEMU_EFI.fd -m 1024 -net none -drive file=fat:rw:/home/vsts/work/1/s/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/VirtualDrive,format=raw,media=disk -display none > You need either -nographic or -serial stdio to get the serial output to go to QEMU's stdout. The latter is probably better, since the former gives you the serial stdout multiplexed with the QEMU console, accessible via special control sequences, and the latter only gives you the serial output stream. -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56681): https://edk2.groups.io/g/devel/message/56681 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
Taking a moment to look at this a different way…
Is it expected at some point that we would want to run OvmfPkg-based integration tests as part of a “second-pass” automatic validation (maybe not a PR-gate, since those should be as fast as possible, but a nightly CI)? If so, I think I’d rather see these platforms covered under that pipeline.
If anyone would be interested in pursuing that approach (which would also be more portable to other platforms in edk2-platforms, if they should want automated nightlys at some point), I’d be happy to put a pin in this topic and throw up a prototype of what that might look like.
Thanks!
- Bret
From: Shenglei Zhang<mailto:shenglei.zhang@intel.com>
Sent: Thursday, March 26, 2020 12:04 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>; Liming Gao<mailto:liming.gao@intel.com>
Subject: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
REF: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2570&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C198058e322e7419c15d708d7d153eaff%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637208030668393225&sdata=TkqjqO7Fi%2BN7FPgJ0FlRcD4T59zCu7hfWQRbb%2FO5dKA%3D&reserved=0
OvmfPkg and EmulatorPkg are mostly used by the developers, so add
them to target list.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
---
.azurepipelines/templates/pr-gate-build-job.yml | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
index 61868554d43c..34f03745cc70 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -44,6 +44,12 @@ jobs:
TARGET_SECURITY:
Build.Pkgs: 'SecurityPkg'
Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_OVMF:
+ Build.Pkgs: 'OvmfPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_EMULATOR:
+ Build.Pkgs: 'EmulatorPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
workspace:
clean: all
--
2.18.0.windows.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#56427): https://edk2.groups.io/g/devel/message/56427
Mute This Topic: https://groups.io/mt/72576939/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
Bret,
I would like to see issues with these platforms packages caught pre-commit. If a core package change breaks one of these platforms, the we can potentially prevent an issue in many other platforms.
Mike
From: Bret Barkelew <Bret.Barkelew@microsoft.com>
Sent: Thursday, March 26, 2020 4:27 PM
To: Zhang, Shenglei <shenglei.zhang@intel.com>; devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@microsoft.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Gao, Liming <liming.gao@intel.com>
Subject: RE: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
Taking a moment to look at this a different way…
Is it expected at some point that we would want to run OvmfPkg-based integration tests as part of a “second-pass” automatic validation (maybe not a PR-gate, since those should be as fast as possible, but a nightly CI)? If so, I think I’d rather see these platforms covered under that pipeline.
If anyone would be interested in pursuing that approach (which would also be more portable to other platforms in edk2-platforms, if they should want automated nightlys at some point), I’d be happy to put a pin in this topic and throw up a prototype of what that might look like.
Thanks!
- Bret
From: Shenglei Zhang<mailto:shenglei.zhang@intel.com>
Sent: Thursday, March 26, 2020 12:04 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>; Liming Gao<mailto:liming.gao@intel.com>
Subject: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
REF: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2570&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C198058e322e7419c15d708d7d153eaff%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637208030668393225&sdata=TkqjqO7Fi%2BN7FPgJ0FlRcD4T59zCu7hfWQRbb%2FO5dKA%3D&reserved=0
OvmfPkg and EmulatorPkg are mostly used by the developers, so add
them to target list.
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>
Cc: Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Cc: Liming Gao <liming.gao@intel.com<mailto:liming.gao@intel.com>>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com<mailto:shenglei.zhang@intel.com>>
---
.azurepipelines/templates/pr-gate-build-job.yml | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
index 61868554d43c..34f03745cc70 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -44,6 +44,12 @@ jobs:
TARGET_SECURITY:
Build.Pkgs: 'SecurityPkg'
Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_OVMF:
+ Build.Pkgs: 'OvmfPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_EMULATOR:
+ Build.Pkgs: 'EmulatorPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
workspace:
clean: all
--
2.18.0.windows.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#56428): https://edk2.groups.io/g/devel/message/56428
Mute This Topic: https://groups.io/mt/72576939/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
So just clarifying, the only thing in scope right now is a simple build-test against these packages. There’s no desire to run tests within the Ovmf environment itself?
- Bret
From: Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Sent: Thursday, March 26, 2020 5:00 PM
To: Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Zhang, Shenglei<mailto:shenglei.zhang@intel.com>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Gao, Liming<mailto:liming.gao@intel.com>
Subject: RE: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
Bret,
I would like to see issues with these platforms packages caught pre-commit. If a core package change breaks one of these platforms, the we can potentially prevent an issue in many other platforms.
Mike
From: Bret Barkelew <Bret.Barkelew@microsoft.com>
Sent: Thursday, March 26, 2020 4:27 PM
To: Zhang, Shenglei <shenglei.zhang@intel.com>; devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@microsoft.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Gao, Liming <liming.gao@intel.com>
Subject: RE: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
Taking a moment to look at this a different way…
Is it expected at some point that we would want to run OvmfPkg-based integration tests as part of a “second-pass” automatic validation (maybe not a PR-gate, since those should be as fast as possible, but a nightly CI)? If so, I think I’d rather see these platforms covered under that pipeline.
If anyone would be interested in pursuing that approach (which would also be more portable to other platforms in edk2-platforms, if they should want automated nightlys at some point), I’d be happy to put a pin in this topic and throw up a prototype of what that might look like.
Thanks!
- Bret
From: Shenglei Zhang<mailto:shenglei.zhang@intel.com>
Sent: Thursday, March 26, 2020 12:04 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>; Liming Gao<mailto:liming.gao@intel.com>
Subject: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
REF: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2570&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C198058e322e7419c15d708d7d153eaff%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637208030668393225&sdata=TkqjqO7Fi%2BN7FPgJ0FlRcD4T59zCu7hfWQRbb%2FO5dKA%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2570&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C95bd9871827d4ac0a84708d7d1e1e394%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637208640427947736&sdata=paW9upRWw3kGWHGW7fXE1ecdQ%2FfxfXMt0s%2BYH15llW8%3D&reserved=0>
OvmfPkg and EmulatorPkg are mostly used by the developers, so add
them to target list.
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>
Cc: Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Cc: Liming Gao <liming.gao@intel.com<mailto:liming.gao@intel.com>>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com<mailto:shenglei.zhang@intel.com>>
---
.azurepipelines/templates/pr-gate-build-job.yml | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
index 61868554d43c..34f03745cc70 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -44,6 +44,12 @@ jobs:
TARGET_SECURITY:
Build.Pkgs: 'SecurityPkg'
Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_OVMF:
+ Build.Pkgs: 'OvmfPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_EMULATOR:
+ Build.Pkgs: 'EmulatorPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
workspace:
clean: all
--
2.18.0.windows.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#56430): https://edk2.groups.io/g/devel/message/56430
Mute This Topic: https://groups.io/mt/72576939/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
Bret,
Yes. Let’s start with build testing.
It will be good to boot the shell and run test suites like UEFI SCTs from the shell of OVMF in a CI agent, but that could be a scheduled post-commit task. Booting the shell is fast enough that we could consider it for pre-commit.
Mike
From: Bret Barkelew <Bret.Barkelew@microsoft.com>
Sent: Thursday, March 26, 2020 5:15 PM
To: Kinney, Michael D <michael.d.kinney@intel.com>; Zhang, Shenglei <shenglei.zhang@intel.com>; devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@microsoft.com>; Gao, Liming <liming.gao@intel.com>
Subject: RE: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
So just clarifying, the only thing in scope right now is a simple build-test against these packages. There’s no desire to run tests within the Ovmf environment itself?
- Bret
From: Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Sent: Thursday, March 26, 2020 5:00 PM
To: Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Zhang, Shenglei<mailto:shenglei.zhang@intel.com>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Gao, Liming<mailto:liming.gao@intel.com>
Subject: RE: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
Bret,
I would like to see issues with these platforms packages caught pre-commit. If a core package change breaks one of these platforms, the we can potentially prevent an issue in many other platforms.
Mike
From: Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>
Sent: Thursday, March 26, 2020 4:27 PM
To: Zhang, Shenglei <shenglei.zhang@intel.com<mailto:shenglei.zhang@intel.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>
Subject: RE: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
Taking a moment to look at this a different way…
Is it expected at some point that we would want to run OvmfPkg-based integration tests as part of a “second-pass” automatic validation (maybe not a PR-gate, since those should be as fast as possible, but a nightly CI)? If so, I think I’d rather see these platforms covered under that pipeline.
If anyone would be interested in pursuing that approach (which would also be more portable to other platforms in edk2-platforms, if they should want automated nightlys at some point), I’d be happy to put a pin in this topic and throw up a prototype of what that might look like.
Thanks!
- Bret
From: Shenglei Zhang<mailto:shenglei.zhang@intel.com>
Sent: Thursday, March 26, 2020 12:04 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>; Liming Gao<mailto:liming.gao@intel.com>
Subject: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
REF: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2570&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C198058e322e7419c15d708d7d153eaff%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637208030668393225&sdata=TkqjqO7Fi%2BN7FPgJ0FlRcD4T59zCu7hfWQRbb%2FO5dKA%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2570&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C95bd9871827d4ac0a84708d7d1e1e394%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637208640427947736&sdata=paW9upRWw3kGWHGW7fXE1ecdQ%2FfxfXMt0s%2BYH15llW8%3D&reserved=0>
OvmfPkg and EmulatorPkg are mostly used by the developers, so add
them to target list.
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>
Cc: Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Cc: Liming Gao <liming.gao@intel.com<mailto:liming.gao@intel.com>>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com<mailto:shenglei.zhang@intel.com>>
---
.azurepipelines/templates/pr-gate-build-job.yml | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
index 61868554d43c..34f03745cc70 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -44,6 +44,12 @@ jobs:
TARGET_SECURITY:
Build.Pkgs: 'SecurityPkg'
Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_OVMF:
+ Build.Pkgs: 'OvmfPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_EMULATOR:
+ Build.Pkgs: 'EmulatorPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
workspace:
clean: all
--
2.18.0.windows.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#56449): https://edk2.groups.io/g/devel/message/56449
Mute This Topic: https://groups.io/mt/72576939/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
BZ 2570 request build test only. The boot functionality test can be added later.
Ovmf has three combination for X86, IA32, X64 and IA32X64. Emulator has two combination IA32, and X64. Can they be supported?
Thanks
Liming
From: Kinney, Michael D <michael.d.kinney@intel.com>
Sent: 2020年3月27日 9:59
To: Bret Barkelew <Bret.Barkelew@microsoft.com>; Zhang, Shenglei <shenglei.zhang@intel.com>; devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>; Gao, Liming <liming.gao@intel.com>
Subject: RE: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
Bret,
Yes. Let’s start with build testing.
It will be good to boot the shell and run test suites like UEFI SCTs from the shell of OVMF in a CI agent, but that could be a scheduled post-commit task. Booting the shell is fast enough that we could consider it for pre-commit.
Mike
From: Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>
Sent: Thursday, March 26, 2020 5:15 PM
To: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Zhang, Shenglei <shenglei.zhang@intel.com<mailto:shenglei.zhang@intel.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>
Subject: RE: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
So just clarifying, the only thing in scope right now is a simple build-test against these packages. There’s no desire to run tests within the Ovmf environment itself?
- Bret
From: Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Sent: Thursday, March 26, 2020 5:00 PM
To: Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Zhang, Shenglei<mailto:shenglei.zhang@intel.com>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Gao, Liming<mailto:liming.gao@intel.com>
Subject: RE: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
Bret,
I would like to see issues with these platforms packages caught pre-commit. If a core package change breaks one of these platforms, the we can potentially prevent an issue in many other platforms.
Mike
From: Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>
Sent: Thursday, March 26, 2020 4:27 PM
To: Zhang, Shenglei <shenglei.zhang@intel.com<mailto:shenglei.zhang@intel.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>
Subject: RE: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
Taking a moment to look at this a different way…
Is it expected at some point that we would want to run OvmfPkg-based integration tests as part of a “second-pass” automatic validation (maybe not a PR-gate, since those should be as fast as possible, but a nightly CI)? If so, I think I’d rather see these platforms covered under that pipeline.
If anyone would be interested in pursuing that approach (which would also be more portable to other platforms in edk2-platforms, if they should want automated nightlys at some point), I’d be happy to put a pin in this topic and throw up a prototype of what that might look like.
Thanks!
- Bret
From: Shenglei Zhang<mailto:shenglei.zhang@intel.com>
Sent: Thursday, March 26, 2020 12:04 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>; Liming Gao<mailto:liming.gao@intel.com>
Subject: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
REF: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2570&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C198058e322e7419c15d708d7d153eaff%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637208030668393225&sdata=TkqjqO7Fi%2BN7FPgJ0FlRcD4T59zCu7hfWQRbb%2FO5dKA%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2570&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C95bd9871827d4ac0a84708d7d1e1e394%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637208640427947736&sdata=paW9upRWw3kGWHGW7fXE1ecdQ%2FfxfXMt0s%2BYH15llW8%3D&reserved=0>
OvmfPkg and EmulatorPkg are mostly used by the developers, so add
them to target list.
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>
Cc: Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Cc: Liming Gao <liming.gao@intel.com<mailto:liming.gao@intel.com>>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com<mailto:shenglei.zhang@intel.com>>
---
.azurepipelines/templates/pr-gate-build-job.yml | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
index 61868554d43c..34f03745cc70 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -44,6 +44,12 @@ jobs:
TARGET_SECURITY:
Build.Pkgs: 'SecurityPkg'
Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_OVMF:
+ Build.Pkgs: 'OvmfPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_EMULATOR:
+ Build.Pkgs: 'EmulatorPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
workspace:
clean: all
--
2.18.0.windows.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#56452): https://edk2.groups.io/g/devel/message/56452
Mute This Topic: https://groups.io/mt/72576939/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
So even though the scope for right now is to build the package only, since these are platforms I think they would better align with a platform build pipeline rather than the stuart_ci_build pipeline. Basically we would setup a new pipeline that builds each platform. To start they can just build but I image rather quickly we could add boot to shell capability and then follow that on with a set of tests run from the shell.
A platform pipeline obviously has to compile the package too so it will be covered.
Thanks
Sean
From: Gao, Liming <liming.gao@intel.com>
Sent: Thursday, March 26, 2020 7:04 PM
To: Kinney, Michael D <michael.d.kinney@intel.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Zhang, Shenglei <shenglei.zhang@intel.com>; devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@microsoft.com>
Subject: RE: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
BZ 2570 request build test only. The boot functionality test can be added later.
Ovmf has three combination for X86, IA32, X64 and IA32X64. Emulator has two combination IA32, and X64. Can they be supported?
Thanks
Liming
From: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Sent: 2020年3月27日 9:59
To: Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>; Zhang, Shenglei <shenglei.zhang@intel.com<mailto:shenglei.zhang@intel.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>
Subject: RE: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
Bret,
Yes. Let’s start with build testing.
It will be good to boot the shell and run test suites like UEFI SCTs from the shell of OVMF in a CI agent, but that could be a scheduled post-commit task. Booting the shell is fast enough that we could consider it for pre-commit.
Mike
From: Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>
Sent: Thursday, March 26, 2020 5:15 PM
To: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Zhang, Shenglei <shenglei.zhang@intel.com<mailto:shenglei.zhang@intel.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>
Subject: RE: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
So just clarifying, the only thing in scope right now is a simple build-test against these packages. There’s no desire to run tests within the Ovmf environment itself?
- Bret
From: Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Sent: Thursday, March 26, 2020 5:00 PM
To: Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Zhang, Shenglei<mailto:shenglei.zhang@intel.com>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Gao, Liming<mailto:liming.gao@intel.com>
Subject: RE: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
Bret,
I would like to see issues with these platforms packages caught pre-commit. If a core package change breaks one of these platforms, the we can potentially prevent an issue in many other platforms.
Mike
From: Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>
Sent: Thursday, March 26, 2020 4:27 PM
To: Zhang, Shenglei <shenglei.zhang@intel.com<mailto:shenglei.zhang@intel.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>
Subject: RE: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
Taking a moment to look at this a different way…
Is it expected at some point that we would want to run OvmfPkg-based integration tests as part of a “second-pass” automatic validation (maybe not a PR-gate, since those should be as fast as possible, but a nightly CI)? If so, I think I’d rather see these platforms covered under that pipeline.
If anyone would be interested in pursuing that approach (which would also be more portable to other platforms in edk2-platforms, if they should want automated nightlys at some point), I’d be happy to put a pin in this topic and throw up a prototype of what that might look like.
Thanks!
- Bret
From: Shenglei Zhang<mailto:shenglei.zhang@intel.com>
Sent: Thursday, March 26, 2020 12:04 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Cc: Sean Brogan<mailto:sean.brogan@microsoft.com>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>; Liming Gao<mailto:liming.gao@intel.com>
Subject: [EXTERNAL] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg
REF: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2570&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C198058e322e7419c15d708d7d153eaff%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637208030668393225&sdata=TkqjqO7Fi%2BN7FPgJ0FlRcD4T59zCu7hfWQRbb%2FO5dKA%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2570&data=02%7C01%7Csean.brogan%40microsoft.com%7C3549c265edf045c793e808d7d1f32c19%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637208714654257398&sdata=eoUEhEWOW2HJKoPoO3OajijWQ9rqxaOf%2BO79yllMVuY%3D&reserved=0>
OvmfPkg and EmulatorPkg are mostly used by the developers, so add
them to target list.
Cc: Sean Brogan <sean.brogan@microsoft.com<mailto:sean.brogan@microsoft.com>>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com<mailto:Bret.Barkelew@microsoft.com>>
Cc: Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Cc: Liming Gao <liming.gao@intel.com<mailto:liming.gao@intel.com>>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com<mailto:shenglei.zhang@intel.com>>
---
.azurepipelines/templates/pr-gate-build-job.yml | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
index 61868554d43c..34f03745cc70 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -44,6 +44,12 @@ jobs:
TARGET_SECURITY:
Build.Pkgs: 'SecurityPkg'
Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_OVMF:
+ Build.Pkgs: 'OvmfPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_EMULATOR:
+ Build.Pkgs: 'EmulatorPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
workspace:
clean: all
--
2.18.0.windows.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#56459): https://edk2.groups.io/g/devel/message/56459
Mute This Topic: https://groups.io/mt/72576939/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
(+ Laszlo) On Thu, 26 Mar 2020 at 08:04, Zhang, Shenglei <shenglei.zhang@intel.com> wrote: > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2570 > OvmfPkg and EmulatorPkg are mostly used by the developers, so add > them to target list. > > Cc: Sean Brogan <sean.brogan@microsoft.com> > Cc: Bret Barkelew <Bret.Barkelew@microsoft.com> > Cc: Michael D Kinney <michael.d.kinney@intel.com> > Cc: Liming Gao <liming.gao@intel.com> > Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com> Could we add ArmVirtPkg as well please? Also, which .DSCs are built inside OvmfPkg/ with this change? > --- > .azurepipelines/templates/pr-gate-build-job.yml | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml > index 61868554d43c..34f03745cc70 100644 > --- a/.azurepipelines/templates/pr-gate-build-job.yml > +++ b/.azurepipelines/templates/pr-gate-build-job.yml > @@ -44,6 +44,12 @@ jobs: > TARGET_SECURITY: > Build.Pkgs: 'SecurityPkg' > Build.Targets: 'DEBUG,RELEASE,NO-TARGET' > + TARGET_OVMF: > + Build.Pkgs: 'OvmfPkg' > + Build.Targets: 'DEBUG,RELEASE,NO-TARGET' > + TARGET_EMULATOR: > + Build.Pkgs: 'EmulatorPkg' > + Build.Targets: 'DEBUG,RELEASE,NO-TARGET' > > workspace: > clean: all > -- > 2.18.0.windows.1 > > > > -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56381): https://edk2.groups.io/g/devel/message/56381 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
Hi Mike, > -----Original Message----- > From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org] > Sent: Thursday, March 26, 2020 4:43 PM > To: edk2-devel-groups-io <devel@edk2.groups.io>; Zhang, Shenglei > <shenglei.zhang@intel.com>; Laszlo Ersek <lersek@redhat.com> > Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew > <Bret.Barkelew@microsoft.com>; Kinney, Michael D > <michael.d.kinney@intel.com>; Gao, Liming <liming.gao@intel.com> > Subject: Re: [edk2-devel] [PATCH] .azurepipelines: Enable CI for OvmfPkg > and EmulatorPkg > > (+ Laszlo) > > On Thu, 26 Mar 2020 at 08:04, Zhang, Shenglei <shenglei.zhang@intel.com> > wrote: > > > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2570 > > OvmfPkg and EmulatorPkg are mostly used by the developers, so add > > them to target list. > > > > Cc: Sean Brogan <sean.brogan@microsoft.com> > > Cc: Bret Barkelew <Bret.Barkelew@microsoft.com> > > Cc: Michael D Kinney <michael.d.kinney@intel.com> > > Cc: Liming Gao <liming.gao@intel.com> > > Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com> > > Could we add ArmVirtPkg as well please? Also, which .DSCs are built > inside OvmfPkg/ with this change? Could you help answer this question? I only see interfaces to include package level things. Thanks, Shenglei > > > --- > > .azurepipelines/templates/pr-gate-build-job.yml | 6 ++++++ > > 1 file changed, 6 insertions(+) > > > > diff --git a/.azurepipelines/templates/pr-gate-build-job.yml > b/.azurepipelines/templates/pr-gate-build-job.yml > > index 61868554d43c..34f03745cc70 100644 > > --- a/.azurepipelines/templates/pr-gate-build-job.yml > > +++ b/.azurepipelines/templates/pr-gate-build-job.yml > > @@ -44,6 +44,12 @@ jobs: > > TARGET_SECURITY: > > Build.Pkgs: 'SecurityPkg' > > Build.Targets: 'DEBUG,RELEASE,NO-TARGET' > > + TARGET_OVMF: > > + Build.Pkgs: 'OvmfPkg' > > + Build.Targets: 'DEBUG,RELEASE,NO-TARGET' > > + TARGET_EMULATOR: > > + Build.Pkgs: 'EmulatorPkg' > > + Build.Targets: 'DEBUG,RELEASE,NO-TARGET' > > > > workspace: > > clean: all > > -- > > 2.18.0.windows.1 > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56382): https://edk2.groups.io/g/devel/message/56382 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
(+Phil, +Anthony) On 03/26/20 09:43, Ard Biesheuvel wrote: > (+ Laszlo) > > On Thu, 26 Mar 2020 at 08:04, Zhang, Shenglei <shenglei.zhang@intel.com> wrote: >> >> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2570 >> OvmfPkg and EmulatorPkg are mostly used by the developers, so add >> them to target list. >> >> Cc: Sean Brogan <sean.brogan@microsoft.com> >> Cc: Bret Barkelew <Bret.Barkelew@microsoft.com> >> Cc: Michael D Kinney <michael.d.kinney@intel.com> >> Cc: Liming Gao <liming.gao@intel.com> >> Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com> > > Could we add ArmVirtPkg as well please? Also, which .DSCs are built > inside OvmfPkg/ with this change? Thanks for the CC. (1) Having learned that Sean and probably others do not benefit from OvmfPkg, I do not wish to slow down PR gating for them, by making OVMF builds a mandatory part of CI -- as long as the PR does not have a patch that modifies OvmfPkg itself. (I'm unsure if the above behavior is already ensured by the CI system -- if it is, then I'm sorry about the noise.) Same applies (from my perspective) to ArmVirtPkg. Obviously, I would definitely *like* if ArmVirtPkg / OvmfPkg builds were a constant, mandatory part of CI, but I totally realize it would put an unjustified burden on contributors that do not care about those platforms at all -- especially given that the core package(s) being patched -- such as NetworkPkg -- *are* test-built by CI in isolation anyway. (2) In case the PR does modify OvmfPkg, and so the platform builds are required, I would like the builds to cover more ground: (2a) DEBUG, RELEASE and NOOPT should all be covered, (2b) Ia32, Ia32X64, and X64 should all be covered. (I've CC'd Anthony so he can speak to the "OvmfXen.dsc" build configs.) (2c) For each of the three platforms I mention in (2b), there should be a "bare bones" (default) build, but also a reasonably "full-blown" build. - Bare-bones build: -D FD_SIZE_2MB (This is so that the default (bare-bones) configuration continue to fit in the (arguably legacy) 2MB FD.) - Full-blown build: -D SECURE_BOOT_ENABLE \ -D SMM_REQUIRE \ -D TPM_ENABLE \ -D TPM_CONFIG_ENABLE \ -D NETWORK_TLS_ENABLE \ -D NETWORK_IP6_ENABLE \ -D NETWORK_HTTP_BOOT_ENABLE Note that this will require the OpenSSL submodule to be initialized. This means 3 * 3 * 2 = 18 builds thus far. (3) ArmVirtPkg platforms should be built if either OvmfPkg or ArmVirtPkg is modified. Personally I'd like the ArmVirtQemu platform to be built, but Ard and Anthony will likely ask for more. For ArmVirtQemu, because we have no (i) different FD sizes, nor (b) an SMM stack that makes some drivers strictly exclusive with other drivers, I think we should simply aim at the most comprehensive build: (3a) DEBUG, RELEASE and NOOPT, (3b) Options: -D SECURE_BOOT_ENABLE \ -D TPM2_ENABLE \ -D TPM2_CONFIG_ENABLE \ -D NETWORK_IP6_ENABLE \ -D NETWORK_HTTP_BOOT_ENABLE \ -D NETWORK_TLS_ENABLE This means +3 builds (21 builds in total) from my perspective. Thanks Laszlo -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56497): https://edk2.groups.io/g/devel/message/56497 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On 03/27/20 15:36, Laszlo Ersek wrote: > (+Phil, +Anthony) > > On 03/26/20 09:43, Ard Biesheuvel wrote: >> (+ Laszlo) >> >> On Thu, 26 Mar 2020 at 08:04, Zhang, Shenglei <shenglei.zhang@intel.com> wrote: >>> >>> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2570 >>> OvmfPkg and EmulatorPkg are mostly used by the developers, so add >>> them to target list. >>> >>> Cc: Sean Brogan <sean.brogan@microsoft.com> >>> Cc: Bret Barkelew <Bret.Barkelew@microsoft.com> >>> Cc: Michael D Kinney <michael.d.kinney@intel.com> >>> Cc: Liming Gao <liming.gao@intel.com> >>> Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com> >> >> Could we add ArmVirtPkg as well please? Also, which .DSCs are built >> inside OvmfPkg/ with this change? > > Thanks for the CC. > > (1) Having learned that Sean and probably others do not benefit from > OvmfPkg, I do not wish to slow down PR gating for them, by making OVMF > builds a mandatory part of CI -- as long as the PR does not have a patch > that modifies OvmfPkg itself. > > (I'm unsure if the above behavior is already ensured by the CI system -- > if it is, then I'm sorry about the noise.) > > Same applies (from my perspective) to ArmVirtPkg. > > Obviously, I would definitely *like* if ArmVirtPkg / OvmfPkg builds were > a constant, mandatory part of CI, but I totally realize it would put an > unjustified burden on contributors that do not care about those > platforms at all -- especially given that the core package(s) being > patched -- such as NetworkPkg -- *are* test-built by CI in isolation anyway. > > > (2) In case the PR does modify OvmfPkg, and so the platform builds are > required, I would like the builds to cover more ground: > > (2a) DEBUG, RELEASE and NOOPT should all be covered, > > (2b) Ia32, Ia32X64, and X64 should all be covered. > > (I've CC'd Anthony so he can speak to the "OvmfXen.dsc" build configs.) > > (2c) For each of the three platforms I mention in (2b), there should be > a "bare bones" (default) build, but also a reasonably "full-blown" build. > > - Bare-bones build: > > -D FD_SIZE_2MB > > (This is so that the default (bare-bones) configuration continue to fit > in the (arguably legacy) 2MB FD.) > > - Full-blown build: > > -D SECURE_BOOT_ENABLE \ > -D SMM_REQUIRE \ > -D TPM_ENABLE \ > -D TPM_CONFIG_ENABLE \ > -D NETWORK_TLS_ENABLE \ > -D NETWORK_IP6_ENABLE \ > -D NETWORK_HTTP_BOOT_ENABLE > > Note that this will require the OpenSSL submodule to be initialized. > > This means 3 * 3 * 2 = 18 builds thus far. > > > (3) ArmVirtPkg platforms should be built if either OvmfPkg or ArmVirtPkg > is modified. > > Personally I'd like the ArmVirtQemu platform to be built, but Ard and > Anthony will likely ask for more. > > For ArmVirtQemu, because we have no (i) different FD sizes, nor (b) an > SMM stack that makes some drivers strictly exclusive with other drivers, > I think we should simply aim at the most comprehensive build: > > (3a) DEBUG, RELEASE and NOOPT, > > (3b) Options: > > -D SECURE_BOOT_ENABLE \ > -D TPM2_ENABLE \ > -D TPM2_CONFIG_ENABLE \ > -D NETWORK_IP6_ENABLE \ > -D NETWORK_HTTP_BOOT_ENABLE \ > -D NETWORK_TLS_ENABLE > > This means +3 builds (21 builds in total) from my perspective. ... to be clear: the above is my "wish list". If, at the moment, we can enable OvmfPkg and ArmVirtPkg builds in any shape or form whatsoever, that's already an improvement! So my comments are not to be taken as disagreement with the currently proposed patch. I'm just describing how I believe we should do these builds eventually. Thanks Laszlo -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56499): https://edk2.groups.io/g/devel/message/56499 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On 3/26/20 1:04 AM, Zhang, Shenglei wrote: > Build.Targets: 'DEBUG,RELEASE,NO-TARGET' > + TARGET_OVMF: > + Build.Pkgs: 'OvmfPkg' > + Build.Targets: 'DEBUG,RELEASE,NO-TARGET' > + TARGET_EMULATOR: > + Build.Pkgs: 'EmulatorPkg' > + Build.Targets: 'DEBUG,RELEASE,NO-TARGET' > > workspace: > clean: all Should we build the NOOPT target as well? -- Rebecca Cran -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56366): https://edk2.groups.io/g/devel/message/56366 Mute This Topic: https://groups.io/mt/72559106/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
Currently not necessary unless you have host-based tests in those packages, but it doesn’t hurt. - Bret ________________________________ From: Rebecca Cran <rebecca@bsdio.com> Sent: Thursday, March 26, 2020 12:11:06 AM To: devel@edk2.groups.io <devel@edk2.groups.io>; shenglei.zhang@intel.com <shenglei.zhang@intel.com> Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Liming Gao <liming.gao@intel.com> Subject: [EXTERNAL] Re: [edk2-devel] [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg On 3/26/20 1:04 AM, Zhang, Shenglei wrote: > Build.Targets: 'DEBUG,RELEASE,NO-TARGET' > + TARGET_OVMF: > + Build.Pkgs: 'OvmfPkg' > + Build.Targets: 'DEBUG,RELEASE,NO-TARGET' > + TARGET_EMULATOR: > + Build.Pkgs: 'EmulatorPkg' > + Build.Targets: 'DEBUG,RELEASE,NO-TARGET' > > workspace: > clean: all Should we build the NOOPT target as well? -- Rebecca Cran -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#56376): https://edk2.groups.io/g/devel/message/56376 Mute This Topic: https://groups.io/mt/72559460/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
© 2016 - 2024 Red Hat, Inc.