[PATCH 00/10] edk2: adopt the edk2-stable202008 release

Laszlo Ersek posted 10 patches 3 years, 7 months ago
Only 9 patches received!
Makefile                               |   1 -
pc-bios/README                         |   4 +--
pc-bios/edk2-aarch64-code.fd.bz2       | Bin 1178070 -> 1507722 bytes
pc-bios/edk2-arm-code.fd.bz2           | Bin 1172752 -> 1503187 bytes
pc-bios/edk2-i386-code.fd.bz2          | Bin 1736199 -> 1646741 bytes
pc-bios/edk2-i386-secure-code.fd.bz2   | Bin 1943949 -> 1860546 bytes
pc-bios/edk2-x86_64-code.fd.bz2        | Bin 1717094 -> 1680164 bytes
pc-bios/edk2-x86_64-secure-code.fd.bz2 | Bin 1958037 -> 1912112 bytes
roms/Makefile                          |   1 +
roms/Makefile.edk2                     |  26 ++++++++++++--------
roms/edk2                              |   2 +-
tests/data/acpi/virt/SSDT.memhp        | Bin 736 -> 736 bytes
tests/uefi-test-tools/Makefile         |   1 +
13 files changed, 21 insertions(+), 14 deletions(-)
[PATCH 00/10] edk2: adopt the edk2-stable202008 release
Posted by Laszlo Ersek 3 years, 7 months ago
Ref:    https://bugs.launchpad.net/qemu/+bug/1852196
Repo:   https://github.com/lersek/qemu.git
Branch: edk2stable202008_lp_1852196

This series consumes the following upstream edk2 releases:

  https://github.com/tianocore/edk2/releases/tag/edk2-stable201908
  https://github.com/tianocore/edk2/releases/tag/edk2-stable201911
  https://github.com/tianocore/edk2/releases/tag/edk2-stable202002
  https://github.com/tianocore/edk2/releases/tag/edk2-stable202005
  https://github.com/tianocore/edk2/releases/tag/edk2-stable202008

Worth mentioning (in random order):

- various CVE fixes (see shortlog)
- OpenSSL-1.1.1g
- UEFI HTTPS Boot for ARM/AARCH64
- TPM2 for ARM/AARCH64
- VCPU hotplug with SMI
- support for Linux v5.7+ initrd and mixed mode loading
- Fusion-MPT SCSI driver in OVMF
- VMware PVSCSI driver in OVMF
- PXEv4 / PXEv6 boot possible to disable on the QEMU command line
- SEV-ES support

The IA32 and X64 binaries are now smaller -- the reason is that I buit
them with DevToolSet 9 (gcc-9) on RHEL7, and so this is the first time
they've undergone LTO (with the GCC5 edk2 toolchain settings).

Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>

Thanks,
Laszlo

Laszlo Ersek (10):
  Makefile: remove obsolete edk2 exception from "clean" rule
  roms/efirom, tests/uefi-test-tools: update edk2's own submodules first
  roms/Makefile.edk2: prepare for replacing TPM2*_ENABLE macros
  tests: acpi: tolerate "virt/SSDT.memhp" mismatch temporarily
  roms/edk2: update submodule from edk2-stable201905 to
    edk2-stable202008
  roms/Makefile.edk2: complete replacing TPM2*_ENABLE macros
  roms/Makefile.edk2: enable new ARM/AARCH64 flags up to
    edk2-stable202008
  pc-bios: refresh edk2 build artifacts for edk2-stable202008
  pc-bios: update the README file with edk2-stable202008 information
  tests: acpi: update "virt/SSDT.memhp" for edk2-stable202008

 Makefile                               |   1 -
 pc-bios/README                         |   4 +--
 pc-bios/edk2-aarch64-code.fd.bz2       | Bin 1178070 -> 1507722 bytes
 pc-bios/edk2-arm-code.fd.bz2           | Bin 1172752 -> 1503187 bytes
 pc-bios/edk2-i386-code.fd.bz2          | Bin 1736199 -> 1646741 bytes
 pc-bios/edk2-i386-secure-code.fd.bz2   | Bin 1943949 -> 1860546 bytes
 pc-bios/edk2-x86_64-code.fd.bz2        | Bin 1717094 -> 1680164 bytes
 pc-bios/edk2-x86_64-secure-code.fd.bz2 | Bin 1958037 -> 1912112 bytes
 roms/Makefile                          |   1 +
 roms/Makefile.edk2                     |  26 ++++++++++++--------
 roms/edk2                              |   2 +-
 tests/data/acpi/virt/SSDT.memhp        | Bin 736 -> 736 bytes
 tests/uefi-test-tools/Makefile         |   1 +
 13 files changed, 21 insertions(+), 14 deletions(-)


base-commit: 9d5589bb3feed442ae7ee24d2d882aa0312349a6
-- 
2.19.1.3.g30247aa5d201

Re: [PATCH 00/10] edk2: adopt the edk2-stable202008 release
Posted by Philippe Mathieu-Daudé 3 years, 7 months ago
On 9/8/20 9:29 AM, Laszlo Ersek wrote:
> Ref:    https://bugs.launchpad.net/qemu/+bug/1852196
> Repo:   https://github.com/lersek/qemu.git
> Branch: edk2stable202008_lp_1852196
> 
> This series consumes the following upstream edk2 releases:
> 
>   https://github.com/tianocore/edk2/releases/tag/edk2-stable201908
>   https://github.com/tianocore/edk2/releases/tag/edk2-stable201911
>   https://github.com/tianocore/edk2/releases/tag/edk2-stable202002
>   https://github.com/tianocore/edk2/releases/tag/edk2-stable202005
>   https://github.com/tianocore/edk2/releases/tag/edk2-stable202008
> 
> Worth mentioning (in random order):
> 
> - various CVE fixes (see shortlog)
> - OpenSSL-1.1.1g
> - UEFI HTTPS Boot for ARM/AARCH64
> - TPM2 for ARM/AARCH64
> - VCPU hotplug with SMI
> - support for Linux v5.7+ initrd and mixed mode loading
> - Fusion-MPT SCSI driver in OVMF
> - VMware PVSCSI driver in OVMF
> - PXEv4 / PXEv6 boot possible to disable on the QEMU command line
> - SEV-ES support
> 
> The IA32 and X64 binaries are now smaller -- the reason is that I buit
> them with DevToolSet 9 (gcc-9) on RHEL7, and so this is the first time
> they've undergone LTO (with the GCC5 edk2 toolchain settings).
> 
> Cc: "Michael S. Tsirkin" <mst@redhat.com>
> Cc: Igor Mammedov <imammedo@redhat.com>
> Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
> 
> Thanks,
> Laszlo
> 
> Laszlo Ersek (10):
>   Makefile: remove obsolete edk2 exception from "clean" rule
>   roms/efirom, tests/uefi-test-tools: update edk2's own submodules first
>   roms/Makefile.edk2: prepare for replacing TPM2*_ENABLE macros
>   tests: acpi: tolerate "virt/SSDT.memhp" mismatch temporarily
>   roms/edk2: update submodule from edk2-stable201905 to
>     edk2-stable202008
>   roms/Makefile.edk2: complete replacing TPM2*_ENABLE macros
>   roms/Makefile.edk2: enable new ARM/AARCH64 flags up to
>     edk2-stable202008
>   pc-bios: refresh edk2 build artifacts for edk2-stable202008
>   pc-bios: update the README file with edk2-stable202008 information
>   tests: acpi: update "virt/SSDT.memhp" for edk2-stable202008
> 
>  Makefile                               |   1 -
>  pc-bios/README                         |   4 +--
>  pc-bios/edk2-aarch64-code.fd.bz2       | Bin 1178070 -> 1507722 bytes
>  pc-bios/edk2-arm-code.fd.bz2           | Bin 1172752 -> 1503187 bytes
>  pc-bios/edk2-i386-code.fd.bz2          | Bin 1736199 -> 1646741 bytes
>  pc-bios/edk2-i386-secure-code.fd.bz2   | Bin 1943949 -> 1860546 bytes
>  pc-bios/edk2-x86_64-code.fd.bz2        | Bin 1717094 -> 1680164 bytes
>  pc-bios/edk2-x86_64-secure-code.fd.bz2 | Bin 1958037 -> 1912112 bytes
>  roms/Makefile                          |   1 +
>  roms/Makefile.edk2                     |  26 ++++++++++++--------
>  roms/edk2                              |   2 +-
>  tests/data/acpi/virt/SSDT.memhp        | Bin 736 -> 736 bytes
>  tests/uefi-test-tools/Makefile         |   1 +
>  13 files changed, 21 insertions(+), 14 deletions(-)

Series:
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

And applied to the edk2-next tree.

Thanks,

Phil.


Re: [PATCH 00/10] edk2: adopt the edk2-stable202008 release
Posted by Laszlo Ersek 3 years, 7 months ago
On 09/10/20 21:31, Philippe Mathieu-Daudé wrote:
> On 9/8/20 9:29 AM, Laszlo Ersek wrote:
>> Ref:    https://bugs.launchpad.net/qemu/+bug/1852196
>> Repo:   https://github.com/lersek/qemu.git
>> Branch: edk2stable202008_lp_1852196
>>
>> This series consumes the following upstream edk2 releases:
>>
>>   https://github.com/tianocore/edk2/releases/tag/edk2-stable201908
>>   https://github.com/tianocore/edk2/releases/tag/edk2-stable201911
>>   https://github.com/tianocore/edk2/releases/tag/edk2-stable202002
>>   https://github.com/tianocore/edk2/releases/tag/edk2-stable202005
>>   https://github.com/tianocore/edk2/releases/tag/edk2-stable202008
>>
>> Worth mentioning (in random order):
>>
>> - various CVE fixes (see shortlog)
>> - OpenSSL-1.1.1g
>> - UEFI HTTPS Boot for ARM/AARCH64
>> - TPM2 for ARM/AARCH64
>> - VCPU hotplug with SMI
>> - support for Linux v5.7+ initrd and mixed mode loading
>> - Fusion-MPT SCSI driver in OVMF
>> - VMware PVSCSI driver in OVMF
>> - PXEv4 / PXEv6 boot possible to disable on the QEMU command line
>> - SEV-ES support
>>
>> The IA32 and X64 binaries are now smaller -- the reason is that I buit
>> them with DevToolSet 9 (gcc-9) on RHEL7, and so this is the first time
>> they've undergone LTO (with the GCC5 edk2 toolchain settings).
>>
>> Cc: "Michael S. Tsirkin" <mst@redhat.com>
>> Cc: Igor Mammedov <imammedo@redhat.com>
>> Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
>>
>> Thanks,
>> Laszlo
>>
>> Laszlo Ersek (10):
>>   Makefile: remove obsolete edk2 exception from "clean" rule
>>   roms/efirom, tests/uefi-test-tools: update edk2's own submodules first
>>   roms/Makefile.edk2: prepare for replacing TPM2*_ENABLE macros
>>   tests: acpi: tolerate "virt/SSDT.memhp" mismatch temporarily
>>   roms/edk2: update submodule from edk2-stable201905 to
>>     edk2-stable202008
>>   roms/Makefile.edk2: complete replacing TPM2*_ENABLE macros
>>   roms/Makefile.edk2: enable new ARM/AARCH64 flags up to
>>     edk2-stable202008
>>   pc-bios: refresh edk2 build artifacts for edk2-stable202008
>>   pc-bios: update the README file with edk2-stable202008 information
>>   tests: acpi: update "virt/SSDT.memhp" for edk2-stable202008
>>
>>  Makefile                               |   1 -
>>  pc-bios/README                         |   4 +--
>>  pc-bios/edk2-aarch64-code.fd.bz2       | Bin 1178070 -> 1507722 bytes
>>  pc-bios/edk2-arm-code.fd.bz2           | Bin 1172752 -> 1503187 bytes
>>  pc-bios/edk2-i386-code.fd.bz2          | Bin 1736199 -> 1646741 bytes
>>  pc-bios/edk2-i386-secure-code.fd.bz2   | Bin 1943949 -> 1860546 bytes
>>  pc-bios/edk2-x86_64-code.fd.bz2        | Bin 1717094 -> 1680164 bytes
>>  pc-bios/edk2-x86_64-secure-code.fd.bz2 | Bin 1958037 -> 1912112 bytes
>>  roms/Makefile                          |   1 +
>>  roms/Makefile.edk2                     |  26 ++++++++++++--------
>>  roms/edk2                              |   2 +-
>>  tests/data/acpi/virt/SSDT.memhp        | Bin 736 -> 736 bytes
>>  tests/uefi-test-tools/Makefile         |   1 +
>>  13 files changed, 21 insertions(+), 14 deletions(-)
> 
> Series:
> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> 
> And applied to the edk2-next tree.
> 
> Thanks,
> 
> Phil.
> 

Thank you Phil!
Laszlo


[PATCH 08/10] pc-bios: refresh edk2 build artifacts for edk2-stable202008
Posted by Laszlo Ersek 3 years, 7 months ago
Rebuild the pc-bios/edk2-*.fd.bz2 binaries, based on the edk2-stable202008
release.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugs.launchpad.net/qemu/+bug/1852196
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
---
 pc-bios/edk2-aarch64-code.fd.bz2       | Bin 1178070 -> 1507722 bytes
 pc-bios/edk2-arm-code.fd.bz2           | Bin 1172752 -> 1503187 bytes
 pc-bios/edk2-i386-code.fd.bz2          | Bin 1736199 -> 1646741 bytes
 pc-bios/edk2-i386-secure-code.fd.bz2   | Bin 1943949 -> 1860546 bytes
 pc-bios/edk2-x86_64-code.fd.bz2        | Bin 1717094 -> 1680164 bytes
 pc-bios/edk2-x86_64-secure-code.fd.bz2 | Bin 1958037 -> 1912112 bytes
 6 files changed, 0 insertions(+), 0 deletions(-)

diff --git a/pc-bios/edk2-aarch64-code.fd.bz2 b/pc-bios/edk2-aarch64-code.fd.bz2
index a074085b224f..5bf311464a79 100644
Binary files a/pc-bios/edk2-aarch64-code.fd.bz2 and b/pc-bios/edk2-aarch64-code.fd.bz2 differ
diff --git a/pc-bios/edk2-arm-code.fd.bz2 b/pc-bios/edk2-arm-code.fd.bz2
index 42453cd1f273..7a98069814dc 100644
Binary files a/pc-bios/edk2-arm-code.fd.bz2 and b/pc-bios/edk2-arm-code.fd.bz2 differ
diff --git a/pc-bios/edk2-i386-code.fd.bz2 b/pc-bios/edk2-i386-code.fd.bz2
index 633759688e32..e7b1befe2cfe 100644
Binary files a/pc-bios/edk2-i386-code.fd.bz2 and b/pc-bios/edk2-i386-code.fd.bz2 differ
diff --git a/pc-bios/edk2-i386-secure-code.fd.bz2 b/pc-bios/edk2-i386-secure-code.fd.bz2
index df27bdd2ddbd..b5df5bed3086 100644
Binary files a/pc-bios/edk2-i386-secure-code.fd.bz2 and b/pc-bios/edk2-i386-secure-code.fd.bz2 differ
diff --git a/pc-bios/edk2-x86_64-code.fd.bz2 b/pc-bios/edk2-x86_64-code.fd.bz2
index 0e108fc68a91..e1654d4003b7 100644
Binary files a/pc-bios/edk2-x86_64-code.fd.bz2 and b/pc-bios/edk2-x86_64-code.fd.bz2 differ
diff --git a/pc-bios/edk2-x86_64-secure-code.fd.bz2 b/pc-bios/edk2-x86_64-secure-code.fd.bz2
index 522f8376aabe..767274c38c7f 100644
Binary files a/pc-bios/edk2-x86_64-secure-code.fd.bz2 and b/pc-bios/edk2-x86_64-secure-code.fd.bz2 differ
-- 
2.19.1.3.g30247aa5d201


Re: [PATCH 08/10] pc-bios: refresh edk2 build artifacts for edk2-stable202008
Posted by Philippe Mathieu-Daudé 3 years, 7 months ago
+GitLab team & Gerd (for building firmwares)

On 9/8/20 9:29 AM, Laszlo Ersek wrote:
> Rebuild the pc-bios/edk2-*.fd.bz2 binaries, based on the edk2-stable202008
> release.
> 
> Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
> Ref: https://bugs.launchpad.net/qemu/+bug/1852196
> Signed-off-by: Laszlo Ersek <lersek@redhat.com>
> ---
>  pc-bios/edk2-aarch64-code.fd.bz2       | Bin 1178070 -> 1507722 bytes
>  pc-bios/edk2-arm-code.fd.bz2           | Bin 1172752 -> 1503187 bytes
>  pc-bios/edk2-i386-code.fd.bz2          | Bin 1736199 -> 1646741 bytes
>  pc-bios/edk2-i386-secure-code.fd.bz2   | Bin 1943949 -> 1860546 bytes
>  pc-bios/edk2-x86_64-code.fd.bz2        | Bin 1717094 -> 1680164 bytes
>  pc-bios/edk2-x86_64-secure-code.fd.bz2 | Bin 1958037 -> 1912112 bytes
>  6 files changed, 0 insertions(+), 0 deletions(-)
> 
> diff --git a/pc-bios/edk2-aarch64-code.fd.bz2 b/pc-bios/edk2-aarch64-code.fd.bz2
> index a074085b224f..5bf311464a79 100644
> Binary files a/pc-bios/edk2-aarch64-code.fd.bz2 and b/pc-bios/edk2-aarch64-code.fd.bz2 differ
> diff --git a/pc-bios/edk2-arm-code.fd.bz2 b/pc-bios/edk2-arm-code.fd.bz2
> index 42453cd1f273..7a98069814dc 100644
> Binary files a/pc-bios/edk2-arm-code.fd.bz2 and b/pc-bios/edk2-arm-code.fd.bz2 differ
> diff --git a/pc-bios/edk2-i386-code.fd.bz2 b/pc-bios/edk2-i386-code.fd.bz2
> index 633759688e32..e7b1befe2cfe 100644
> Binary files a/pc-bios/edk2-i386-code.fd.bz2 and b/pc-bios/edk2-i386-code.fd.bz2 differ
> diff --git a/pc-bios/edk2-i386-secure-code.fd.bz2 b/pc-bios/edk2-i386-secure-code.fd.bz2
> index df27bdd2ddbd..b5df5bed3086 100644
> Binary files a/pc-bios/edk2-i386-secure-code.fd.bz2 and b/pc-bios/edk2-i386-secure-code.fd.bz2 differ
> diff --git a/pc-bios/edk2-x86_64-code.fd.bz2 b/pc-bios/edk2-x86_64-code.fd.bz2
> index 0e108fc68a91..e1654d4003b7 100644
> Binary files a/pc-bios/edk2-x86_64-code.fd.bz2 and b/pc-bios/edk2-x86_64-code.fd.bz2 differ
> diff --git a/pc-bios/edk2-x86_64-secure-code.fd.bz2 b/pc-bios/edk2-x86_64-secure-code.fd.bz2
> index 522f8376aabe..767274c38c7f 100644
> Binary files a/pc-bios/edk2-x86_64-secure-code.fd.bz2 and b/pc-bios/edk2-x86_64-secure-code.fd.bz2 differ
> 

Now I remember why I kept that patch on hold.

The CI idea is to have reproducible builds if possible.
When the submodule is updated (or the QEMU scripts containing the
-D defines), it triggers the 'build-edk2' job which produce these
same binaries.
My original idea was to push the tag on GitLab that triggers the
job, then download the produced binaries, test them, then commit
them.

With your series, I get these binaries:
https://gitlab.com/philmd/qemu/-/jobs/731618363/artifacts/browse/pc-bios/

However they differ with yours, for example:

0000 6100: 00 00 00 00 00 00 00 00  00 00 00 00 2F 68 6F 6D  ........
..../hom
0000 6110: 65 2F 6C 61 63 6F 73 2F  73 72 63 2F 75 70 73 74  e/lacos/
src/upst
0000 6120: 72 65 61 6D 2F 71 65 6D  75 2F 72 6F 6D 73 2F 65  ream/qem
u/roms/e
0000 6130: 64 6B 32 2F 42 75 69 6C  64 2F 41 72 6D 56 69 72  dk2/Buil
d/ArmVir
0000 6140: 74 51 65 6D 75 2D 41 41  52 43 48 36 34 2F 44 45  tQemu-AA
RCH64/DE
0000 6150: 42 55 47 5F 47 43 43 35  2F 41 41 52 43 48 36 34  BUG_GCC5
/AARCH64
0000 6160: 2F 41 72 6D 50 6C 61 74  66 6F 72 6D 50 6B 67 2F  /ArmPlat
formPkg/
0000 6170: 50 72 65 50 65 69 43 6F  72 65 2F 50 72 65 50 65  PrePeiCo
re/PrePe
0000 6180: 69 43 6F 72 65 55 6E 69  43 6F 72 65 2F 44 45 42  iCoreUni
Core/DEB
0000 6190: 55 47 2F 41 72 6D 50 6C  61 74 66 6F 72 6D 50 72  UG/ArmPl
atformPr
0000 61A0: 65 50 65 69 43 6F 72 65  2E 64 6C 6C 00 00 00 00  ePeiCore
.dll....

0000 6100: 00 00 00 00 00 00 00 00  00 00 00 00 2F 62 75 69  ........
..../bui
0000 6110: 6C 64 73 2F 70 68 69 6C  6D 64 2F 71 65 6D 75 2F  lds/phil
md/qemu/
0000 6120: 72 6F 6D 73 2F 65 64 6B  32 2F 42 75 69 6C 64 2F  roms/edk
2/Build/
0000 6130: 41 72 6D 56 69 72 74 51  65 6D 75 2D 41 41 52 43  ArmVirtQ
emu-AARC
0000 6140: 48 36 34 2F 44 45 42 55  47 5F 47 43 43 35 2F 41  H64/DEBU
G_GCC5/A
0000 6150: 41 52 43 48 36 34 2F 41  72 6D 50 6C 61 74 66 6F  ARCH64/A
rmPlatfo
0000 6160: 72 6D 50 6B 67 2F 50 72  65 50 65 69 43 6F 72 65  rmPkg/Pr
ePeiCore
0000 6170: 2F 50 72 65 50 65 69 43  6F 72 65 55 6E 69 43 6F  /PrePeiC
oreUniCo
0000 6180: 72 65 2F 44 45 42 55 47  2F 41 72 6D 50 6C 61 74  re/DEBUG
/ArmPlat
0000 6190: 66 6F 72 6D 50 72 65 50  65 69 43 6F 72 65 2E 64  formPreP
eiCore.d
0000 61A0: 6C 6C 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ll......
........

For now this is not a blocker, but we should consider switching to
this workflow at some point (caring about all the files that really
need to be archived, maybe debug symbols etc...).

w.r.t. your patch:
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Regards,

Phil.


Re: [PATCH 08/10] pc-bios: refresh edk2 build artifacts for edk2-stable202008
Posted by Laszlo Ersek 3 years, 7 months ago
On 09/10/20 18:32, Philippe Mathieu-Daudé wrote:
> +GitLab team & Gerd (for building firmwares)
> 
> On 9/8/20 9:29 AM, Laszlo Ersek wrote:
>> Rebuild the pc-bios/edk2-*.fd.bz2 binaries, based on the edk2-stable202008
>> release.
>>
>> Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
>> Ref: https://bugs.launchpad.net/qemu/+bug/1852196
>> Signed-off-by: Laszlo Ersek <lersek@redhat.com>
>> ---
>>  pc-bios/edk2-aarch64-code.fd.bz2       | Bin 1178070 -> 1507722 bytes
>>  pc-bios/edk2-arm-code.fd.bz2           | Bin 1172752 -> 1503187 bytes
>>  pc-bios/edk2-i386-code.fd.bz2          | Bin 1736199 -> 1646741 bytes
>>  pc-bios/edk2-i386-secure-code.fd.bz2   | Bin 1943949 -> 1860546 bytes
>>  pc-bios/edk2-x86_64-code.fd.bz2        | Bin 1717094 -> 1680164 bytes
>>  pc-bios/edk2-x86_64-secure-code.fd.bz2 | Bin 1958037 -> 1912112 bytes
>>  6 files changed, 0 insertions(+), 0 deletions(-)
>>
>> diff --git a/pc-bios/edk2-aarch64-code.fd.bz2 b/pc-bios/edk2-aarch64-code.fd.bz2
>> index a074085b224f..5bf311464a79 100644
>> Binary files a/pc-bios/edk2-aarch64-code.fd.bz2 and b/pc-bios/edk2-aarch64-code.fd.bz2 differ
>> diff --git a/pc-bios/edk2-arm-code.fd.bz2 b/pc-bios/edk2-arm-code.fd.bz2
>> index 42453cd1f273..7a98069814dc 100644
>> Binary files a/pc-bios/edk2-arm-code.fd.bz2 and b/pc-bios/edk2-arm-code.fd.bz2 differ
>> diff --git a/pc-bios/edk2-i386-code.fd.bz2 b/pc-bios/edk2-i386-code.fd.bz2
>> index 633759688e32..e7b1befe2cfe 100644
>> Binary files a/pc-bios/edk2-i386-code.fd.bz2 and b/pc-bios/edk2-i386-code.fd.bz2 differ
>> diff --git a/pc-bios/edk2-i386-secure-code.fd.bz2 b/pc-bios/edk2-i386-secure-code.fd.bz2
>> index df27bdd2ddbd..b5df5bed3086 100644
>> Binary files a/pc-bios/edk2-i386-secure-code.fd.bz2 and b/pc-bios/edk2-i386-secure-code.fd.bz2 differ
>> diff --git a/pc-bios/edk2-x86_64-code.fd.bz2 b/pc-bios/edk2-x86_64-code.fd.bz2
>> index 0e108fc68a91..e1654d4003b7 100644
>> Binary files a/pc-bios/edk2-x86_64-code.fd.bz2 and b/pc-bios/edk2-x86_64-code.fd.bz2 differ
>> diff --git a/pc-bios/edk2-x86_64-secure-code.fd.bz2 b/pc-bios/edk2-x86_64-secure-code.fd.bz2
>> index 522f8376aabe..767274c38c7f 100644
>> Binary files a/pc-bios/edk2-x86_64-secure-code.fd.bz2 and b/pc-bios/edk2-x86_64-secure-code.fd.bz2 differ
>>
> 
> Now I remember why I kept that patch on hold.
> 
> The CI idea is to have reproducible builds if possible.
> When the submodule is updated (or the QEMU scripts containing the
> -D defines), it triggers the 'build-edk2' job which produce these
> same binaries.
> My original idea was to push the tag on GitLab that triggers the
> job, then download the produced binaries, test them, then commit
> them.
> 
> With your series, I get these binaries:
> https://gitlab.com/philmd/qemu/-/jobs/731618363/artifacts/browse/pc-bios/
> 
> However they differ with yours, for example:
> 
> 0000 6100: 00 00 00 00 00 00 00 00  00 00 00 00 2F 68 6F 6D  ........
> ..../hom
> 0000 6110: 65 2F 6C 61 63 6F 73 2F  73 72 63 2F 75 70 73 74  e/lacos/
> src/upst
> 0000 6120: 72 65 61 6D 2F 71 65 6D  75 2F 72 6F 6D 73 2F 65  ream/qem
> u/roms/e
> 0000 6130: 64 6B 32 2F 42 75 69 6C  64 2F 41 72 6D 56 69 72  dk2/Buil
> d/ArmVir
> 0000 6140: 74 51 65 6D 75 2D 41 41  52 43 48 36 34 2F 44 45  tQemu-AA
> RCH64/DE
> 0000 6150: 42 55 47 5F 47 43 43 35  2F 41 41 52 43 48 36 34  BUG_GCC5
> /AARCH64
> 0000 6160: 2F 41 72 6D 50 6C 61 74  66 6F 72 6D 50 6B 67 2F  /ArmPlat
> formPkg/
> 0000 6170: 50 72 65 50 65 69 43 6F  72 65 2F 50 72 65 50 65  PrePeiCo
> re/PrePe
> 0000 6180: 69 43 6F 72 65 55 6E 69  43 6F 72 65 2F 44 45 42  iCoreUni
> Core/DEB
> 0000 6190: 55 47 2F 41 72 6D 50 6C  61 74 66 6F 72 6D 50 72  UG/ArmPl
> atformPr
> 0000 61A0: 65 50 65 69 43 6F 72 65  2E 64 6C 6C 00 00 00 00  ePeiCore
> .dll....
> 
> 0000 6100: 00 00 00 00 00 00 00 00  00 00 00 00 2F 62 75 69  ........
> ..../bui
> 0000 6110: 6C 64 73 2F 70 68 69 6C  6D 64 2F 71 65 6D 75 2F  lds/phil
> md/qemu/
> 0000 6120: 72 6F 6D 73 2F 65 64 6B  32 2F 42 75 69 6C 64 2F  roms/edk
> 2/Build/
> 0000 6130: 41 72 6D 56 69 72 74 51  65 6D 75 2D 41 41 52 43  ArmVirtQ
> emu-AARC
> 0000 6140: 48 36 34 2F 44 45 42 55  47 5F 47 43 43 35 2F 41  H64/DEBU
> G_GCC5/A
> 0000 6150: 41 52 43 48 36 34 2F 41  72 6D 50 6C 61 74 66 6F  ARCH64/A
> rmPlatfo
> 0000 6160: 72 6D 50 6B 67 2F 50 72  65 50 65 69 43 6F 72 65  rmPkg/Pr
> ePeiCore
> 0000 6170: 2F 50 72 65 50 65 69 43  6F 72 65 55 6E 69 43 6F  /PrePeiC
> oreUniCo
> 0000 6180: 72 65 2F 44 45 42 55 47  2F 41 72 6D 50 6C 61 74  re/DEBUG
> /ArmPlat
> 0000 6190: 66 6F 72 6D 50 72 65 50  65 69 43 6F 72 65 2E 64  formPreP
> eiCore.d
> 0000 61A0: 6C 6C 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ll......
> ........
> 
> For now this is not a blocker, but we should consider switching to
> this workflow at some point (caring about all the files that really
> need to be archived, maybe debug symbols etc...).

Yes, I remember the related discussion from last time.

When preparing this patch set, I didn't know where we stood on that, so
I fully expected that this patch could be dropped, and the remotely
built binaries would be merged instead.

But I wasn't sure either way, so I did build the binaries on my end (I
had to do that anyway for actually testing the other patches, and the
resultant binaries too!), and then I just included them in the mailing
list posting / topic branch.

> w.r.t. your patch:
> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Thanks!
Laszlo


Re: [PATCH 08/10] pc-bios: refresh edk2 build artifacts for edk2-stable202008
Posted by Gerd Hoffmann 3 years, 7 months ago
  Hi,

> The CI idea is to have reproducible builds if possible.
> When the submodule is updated (or the QEMU scripts containing the
> -D defines), it triggers the 'build-edk2' job which produce these
> same binaries.
> My original idea was to push the tag on GitLab that triggers the
> job, then download the produced binaries, test them, then commit
> them.

Now with CI in place we maybe should step back and have a look at the
big picture.

Should we simply stop committing firmware binaries into git and provide
them as CI artifacts instead?  Can we build all our firmware in CI,
using the shared gitlab x86 runners and cross compilers?

take care,
  Gerd


Re: [PATCH 08/10] pc-bios: refresh edk2 build artifacts for edk2-stable202008
Posted by Philippe Mathieu-Daudé 3 years, 7 months ago
+Michael & Bruce

On 9/14/20 11:54 AM, Gerd Hoffmann wrote:
>   Hi,
> 
>> The CI idea is to have reproducible builds if possible.
>> When the submodule is updated (or the QEMU scripts containing the
>> -D defines), it triggers the 'build-edk2' job which produce these
>> same binaries.
>> My original idea was to push the tag on GitLab that triggers the
>> job, then download the produced binaries, test them, then commit
>> them.
> 
> Now with CI in place we maybe should step back and have a look at the
> big picture.
> 
> Should we simply stop committing firmware binaries into git and provide
> them as CI artifacts instead?

What I'm not sure about is how to keep the built artifacts forever.
Usually git-forge based projects keep release builds forever.
In our case we are interested in updating them during the development
window, to be well tested on release.

Also we need to modify the source tarball generator script to fetch
the artifacts and include them, isn't it?

>  Can we build all our firmware in CI,
> using the shared gitlab x86 runners and cross compilers?

I'm pretty sure. In case something is missing, adding it shouldn't
be a problem anyway.

Regards,

Phil.


Re: [PATCH 08/10] pc-bios: refresh edk2 build artifacts for edk2-stable202008
Posted by Gerd Hoffmann 3 years, 7 months ago
On Mon, Sep 14, 2020 at 12:53:00PM +0200, Philippe Mathieu-Daudé wrote:
> +Michael & Bruce
> 
> On 9/14/20 11:54 AM, Gerd Hoffmann wrote:
> >   Hi,
> > 
> >> The CI idea is to have reproducible builds if possible.
> >> When the submodule is updated (or the QEMU scripts containing the
> >> -D defines), it triggers the 'build-edk2' job which produce these
> >> same binaries.
> >> My original idea was to push the tag on GitLab that triggers the
> >> job, then download the produced binaries, test them, then commit
> >> them.
> > 
> > Now with CI in place we maybe should step back and have a look at the
> > big picture.
> > 
> > Should we simply stop committing firmware binaries into git and provide
> > them as CI artifacts instead?
> 
> What I'm not sure about is how to keep the built artifacts forever.

Dunno.  gitlab has support for various packages and container images,
but not for tarballs or rpms or raw binaries.  Possibly we could publish
them as gitlab pages.

> Also we need to modify the source tarball generator script to fetch
> the artifacts and include them, isn't it?

An firmware download script certainly makes sense.

Whenever we want continue including the firmware in the tarballs when
we dropped it from the git repo is up for debate ;)

take care,
  Gerd


Re: [PATCH 08/10] pc-bios: refresh edk2 build artifacts for edk2-stable202008
Posted by Daniel P. Berrangé 3 years, 7 months ago
On Thu, Sep 10, 2020 at 06:32:28PM +0200, Philippe Mathieu-Daudé wrote:
> +GitLab team & Gerd (for building firmwares)
> 
> On 9/8/20 9:29 AM, Laszlo Ersek wrote:

> Now I remember why I kept that patch on hold.
> 
> The CI idea is to have reproducible builds if possible.

NB it isn't going to eb truly reproducible because the docker
env we're building in is liable to change each time we run it,
either because we've updated the base OS, or "apt update" pulls
in different content set. At the least, we should likely record
the full set of packages installed in the container, alongside
the blob in qemu.git, so we have a strong record of the toolchain
used.

> When the submodule is updated (or the QEMU scripts containing the
> -D defines), it triggers the 'build-edk2' job which produce these
> same binaries.
> My original idea was to push the tag on GitLab that triggers the
> job, then download the produced binaries, test them, then commit
> them.
> 
> With your series, I get these binaries:
> https://gitlab.com/philmd/qemu/-/jobs/731618363/artifacts/browse/pc-bios/
> 
> However they differ with yours, for example:
> 
> 0000 6100: 00 00 00 00 00 00 00 00  00 00 00 00 2F 68 6F 6D  ........
> ..../hom
> 0000 6110: 65 2F 6C 61 63 6F 73 2F  73 72 63 2F 75 70 73 74  e/lacos/
> src/upst
> 0000 6120: 72 65 61 6D 2F 71 65 6D  75 2F 72 6F 6D 73 2F 65  ream/qem
> u/roms/e
> 0000 6130: 64 6B 32 2F 42 75 69 6C  64 2F 41 72 6D 56 69 72  dk2/Buil
> d/ArmVir
> 0000 6140: 74 51 65 6D 75 2D 41 41  52 43 48 36 34 2F 44 45  tQemu-AA
> RCH64/DE
> 0000 6150: 42 55 47 5F 47 43 43 35  2F 41 41 52 43 48 36 34  BUG_GCC5
> /AARCH64
> 0000 6160: 2F 41 72 6D 50 6C 61 74  66 6F 72 6D 50 6B 67 2F  /ArmPlat
> formPkg/
> 0000 6170: 50 72 65 50 65 69 43 6F  72 65 2F 50 72 65 50 65  PrePeiCo
> re/PrePe
> 0000 6180: 69 43 6F 72 65 55 6E 69  43 6F 72 65 2F 44 45 42  iCoreUni
> Core/DEB
> 0000 6190: 55 47 2F 41 72 6D 50 6C  61 74 66 6F 72 6D 50 72  UG/ArmPl
> atformPr
> 0000 61A0: 65 50 65 69 43 6F 72 65  2E 64 6C 6C 00 00 00 00  ePeiCore
> .dll....
> 
> 0000 6100: 00 00 00 00 00 00 00 00  00 00 00 00 2F 62 75 69  ........
> ..../bui
> 0000 6110: 6C 64 73 2F 70 68 69 6C  6D 64 2F 71 65 6D 75 2F  lds/phil
> md/qemu/
> 0000 6120: 72 6F 6D 73 2F 65 64 6B  32 2F 42 75 69 6C 64 2F  roms/edk
> 2/Build/
> 0000 6130: 41 72 6D 56 69 72 74 51  65 6D 75 2D 41 41 52 43  ArmVirtQ
> emu-AARC
> 0000 6140: 48 36 34 2F 44 45 42 55  47 5F 47 43 43 35 2F 41  H64/DEBU
> G_GCC5/A
> 0000 6150: 41 52 43 48 36 34 2F 41  72 6D 50 6C 61 74 66 6F  ARCH64/A
> rmPlatfo
> 0000 6160: 72 6D 50 6B 67 2F 50 72  65 50 65 69 43 6F 72 65  rmPkg/Pr
> ePeiCore
> 0000 6170: 2F 50 72 65 50 65 69 43  6F 72 65 55 6E 69 43 6F  /PrePeiC
> oreUniCo
> 0000 6180: 72 65 2F 44 45 42 55 47  2F 41 72 6D 50 6C 61 74  re/DEBUG
> /ArmPlat
> 0000 6190: 66 6F 72 6D 50 72 65 50  65 69 43 6F 72 65 2E 64  formPreP
> eiCore.d
> 0000 61A0: 6C 6C 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ll......
> ........
> 
> For now this is not a blocker, but we should consider switching to
> this workflow at some point (caring about all the files that really
> need to be archived, maybe debug symbols etc...).
> 
> w.r.t. your patch:
> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> 
> Regards,
> 
> Phil.
> 
> 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|