[Xen-devel] [PATCH STABLE] tools/firmware: update OVMF Makefile, when necessary

Ian Jackson posted 1 patch 4 years, 11 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/xen tags/patchew/23751.6062.590245.436664@mariner.uk.xensource.com
tools/firmware/ovmf-makefile | 1 +
1 file changed, 1 insertion(+)
[Xen-devel] [PATCH STABLE] tools/firmware: update OVMF Makefile, when necessary
Posted by Ian Jackson 4 years, 11 months ago
On advice from Wei, I am about to push this squashed backport to the
stable trees.  We think this will help fix osstest on those branches
following the upgrade to Debian stretch.

Ian.

From e983e8ae84efd5e43045a3d20a820f13cb4a75bf Mon Sep 17 00:00:00 2001
From: Wei Liu <wei.liu2@citrix.com>
Date: Wed, 28 Nov 2018 17:43:33 +0000
Subject: [PATCH] tools/firmware: update OVMF Makefile, when necessary

[ This is two commits from master aka staging-4.12: ]

OVMF has become dependent on OpenSSL, which is included as a
submodule.  Initialise submodules before building.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
(cherry picked from commit b16281870e06f5f526029a4e69634a16dc38e8e4)

tools: only call git when necessary in OVMF Makefile

Users may choose to export a snapshot of OVMF and build it
with xen.git supplied ovmf-makefile. In that case we don't
need to call `git submodule`.

Fixes b16281870e.

Reported-by: Olaf Hering <olaf@aepfle.de>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
(cherry picked from commit 68292c94a60eab24514ab4a8e4772af24dead807)
---
 tools/firmware/ovmf-makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/firmware/ovmf-makefile b/tools/firmware/ovmf-makefile
index 2838744461..55f9992145 100644
--- a/tools/firmware/ovmf-makefile
+++ b/tools/firmware/ovmf-makefile
@@ -16,6 +16,7 @@ all: build
 
 .PHONY: build
 build:
+	if test -e .git ; then $(GIT) submodule update --init --recursive ; fi
 	OvmfPkg/build.sh -a X64 -b $(TARGET) -n 4
 	cp Build/OvmfX64/$(TARGET)_GCC*/FV/OVMF.fd ovmf.bin
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH STABLE] tools/firmware: update OVMF Makefile, when necessary
Posted by Ian Jackson 4 years, 11 months ago
Ian Jackson writes ("[PATCH STABLE] tools/firmware: update OVMF Makefile, when necessary"):
> On advice from Wei, I am about to push this squashed backport to the
> stable trees.  We think this will help fix osstest on those branches
> following the upgrade to Debian stretch.

Now done, including for staging-4.6.  4.6 is out of security support,
but osstest wants to be able to build 4.6 so that it can test
migration from 4.6 to 4.7, and 4.7 *is* still (just about) in security
support.

I expect the 4.6 push gate to fail and to have to force push it.

It is also possible that we will experience other build fallout.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] Stable trees (4.6 and 4.7), building on stretch, osstest, redux
Posted by Ian Jackson 4 years, 10 months ago
Ian Jackson writes ("[PATCH STABLE] tools/firmware: update OVMF Makefile, when necessary"):
> Now done, including for staging-4.6.  4.6 is out of security support,
> but osstest wants to be able to build 4.6 so that it can test
> migration from 4.6 to 4.7, and 4.7 *is* still (just about) in security
> support.
> 
> I expect the 4.6 push gate to fail and to have to force push it.
> 
> It is also possible that we will experience other build fallout.

I am still struggling with this.  4.7 and 4.6 still do not build.
Right now there are two problems that need thought:


1. That old ipxe is just too badly broken.  I spent a long while
trying to backport compiler fixes but it is totally ridiculous.  IMO
our only sensible option is to update at least osstest's buildsx to a
much newer ipxe.

This could be done by cherry picking
     38ab99b26bf4298a33105ec66f3f6a3f7e05a326
       ipxe: update to newer commit
(which is from xen 4.8 ish) onto our 4.7 and 4.6 branches.

If this is felt too intrusive, then I could somehow make it
conditional and have osstest use it.  This is not entirely trivial
because we have an ad hoc patch application thing.

I'm sort of tempted to have osstest arbitrarily run
   git cherry-pick 38ab99b26bf4298a33105ec66f3f6a3f7e05a326
at some appropriate point...


2. hvmloader fails to build, I think because we need
    7825ae12df1f6d48c4d009cbbdf5a55aff27291b
      errno: introduce EISDIR/EROFS/ENOTEMPTY to the ABI
    03720ea541382a3ca80eaaec2aa11932b03aacaf
      errno: declare aliases using XEN_ERRNO()
    67790205df26e7c3dfeef8b8e64194ebc279220d
      public/errno: Reduce complexity of inclusion
    305e957ffee94fc06c4ba53ef5562f1b8c1c6b02
      hvmloader: use xen/errno.h rather than the host systems errno.h

Is backporting that lot OK ?


There are also some simple backports we need:
   c2a17869d5dcd845d646bf4db122cad73596a2be
     libfsimage: replace deprecated readdir_r() with readdir()
   b9daff9d811285f1e40669bc621c2241793f7a95
     libxl: replace deprecated readdir_r() with readdir()
   668e4edf92fcf7cb929eed221059a3eeb02722c3
     stubdom: make GMP aware that it's being cross-compiled
   2f9eb73c2e2d7fdda8e2586c20f7dbd856002eba
      stubdom: fix stubdom-vtpm build


With all of the above, 4.6 builds again.  I guess 4.7 will too.

Fixing that will also probably enable the 4.8 push gate to pass.  It
is currently blocked because it wants to test 4.7->4.8 migration and
can't build 4.7.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Stable trees (4.6 and 4.7), building on stretch, osstest, redux
Posted by Olaf Hering 4 years, 10 months ago
Am Tue, 28 May 2019 20:59:24 +0100
schrieb Ian Jackson <ian.jackson@citrix.com>:

> I am still struggling with this.  4.7 and 4.6 still do not build.

I maintain the various staging-X.Y branches for various releases,
just for my own entertainment. The remaining build error in SLE_15
(and Tumbleweed) is some asm error in ipxe. It was silently fixed
by rearranging the code, so there was no single commit to cherry-pick.
At some point I lost interest and did not finish the work to make it
build again.


https://build.opensuse.org/package/show/home:olh:xen-4.7/xen

The individual changes:

https://github.com/olafhering/xen/compare/olh-base-staging-4.7...olh-fixes-staging-4.7
https://github.com/olafhering/qemu-xen/compare/olh-base-xen-staging-4.7...olh-fixes-xen-staging-4.7
https://github.com/olafhering/seabios/compare/olh-base-xen-staging-4.7...olh-fixes-xen-staging-4.7
https://github.com/olafhering/edk2/compare/olh-base-xen-staging-4.7...olh-fixes-xen-staging-4.7
https://github.com/olafhering/ipxe/compare/olh-base-xen-staging-4.7...olh-fixes-xen-staging-4.7

Since I last touched it a few months ago, Tumbleweed moved on and will
require additional changes for gcc9. Not sure how stale Stretch is.

To see changes for other staging-X.Y branches, replace '4.7' as needed.

Olaf
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Stable trees (4.6 and 4.7), building on stretch, osstest, redux
Posted by Ian Jackson 4 years, 10 months ago
Ian Jackson writes ("Stable trees (4.6 and 4.7), building on stretch, osstest, redux"):
> I am still struggling with this.  4.7 and 4.6 still do not build.

I have now pushed all of these to 4.6 and 4.7 and it builds for me.
I will kill the current, doomed, 4.6 and 4.7 flights.

> 1. That old ipxe is just too badly broken.  I spent a long while
> trying to backport compiler fixes but it is totally ridiculous.  IMO
> our only sensible option is to update at least osstest's buildsx to a
> much newer ipxe.
> 
> This could be done by cherry picking
>      38ab99b26bf4298a33105ec66f3f6a3f7e05a326
>        ipxe: update to newer commit
> (which is from xen 4.8 ish) onto our 4.7 and 4.6 branches.
> 
> If this is felt too intrusive, then I could somehow make it
> conditional and have osstest use it.  This is not entirely trivial
> because we have an ad hoc patch application thing.
> 
> I'm sort of tempted to have osstest arbitrarily run
>    git cherry-pick 38ab99b26bf4298a33105ec66f3f6a3f7e05a326
> at some appropriate point...

This applied cleanly to both.

> 2. hvmloader fails to build, I think because we need
>     7825ae12df1f6d48c4d009cbbdf5a55aff27291b
>       errno: introduce EISDIR/EROFS/ENOTEMPTY to the ABI
>     03720ea541382a3ca80eaaec2aa11932b03aacaf
>       errno: declare aliases using XEN_ERRNO()
>     67790205df26e7c3dfeef8b8e64194ebc279220d
>       public/errno: Reduce complexity of inclusion
>     305e957ffee94fc06c4ba53ef5562f1b8c1c6b02
>       hvmloader: use xen/errno.h rather than the host systems errno.h
> 
> Is backporting that lot OK ?

These were not needed for 4.7.

> 
> There are also some simple backports we need:
>    c2a17869d5dcd845d646bf4db122cad73596a2be
>      libfsimage: replace deprecated readdir_r() with readdir()
>    b9daff9d811285f1e40669bc621c2241793f7a95
>      libxl: replace deprecated readdir_r() with readdir()

These were not needed for 4.7

>    668e4edf92fcf7cb929eed221059a3eeb02722c3
>      stubdom: make GMP aware that it's being cross-compiled
>    2f9eb73c2e2d7fdda8e2586c20f7dbd856002eba
>       stubdom: fix stubdom-vtpm build

These were needed for 4.6 and 4.7.  The last quoted commit there is
wrong (the quoted commit id is a local one of mine).  It should read:

     7791790c7ab97c85306dce749c6c8eb56d1dc0da
        stubdom: fix stubdom-vtpm build

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Stable trees (4.6 and 4.7), building on stretch, osstest, redux
Posted by Ian Jackson 4 years, 10 months ago
Ian Jackson writes ("Re: Stable trees (4.6 and 4.7), building on stretch, osstest, redux"):
> I have now pushed all of these to 4.6 and 4.7 and it builds for me.
> I will kill the current, doomed, 4.6 and 4.7 flights.

I have now also backported
  5f28de0b0e474e01931b719fa27ca30b8aa446e0
  libxl: compilation warning fix for arm & aarch64

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Stable trees (4.6 and 4.7), building on stretch, osstest, redux
Posted by Jan Beulich 4 years, 10 months ago
>>> On 28.05.19 at 21:59, <ian.jackson@citrix.com> wrote:
> Ian Jackson writes ("[PATCH STABLE] tools/firmware: update OVMF Makefile, 
> when necessary"):
>> Now done, including for staging-4.6.  4.6 is out of security support,
>> but osstest wants to be able to build 4.6 so that it can test
>> migration from 4.6 to 4.7, and 4.7 *is* still (just about) in security
>> support.
>> 
>> I expect the 4.6 push gate to fail and to have to force push it.
>> 
>> It is also possible that we will experience other build fallout.
> 
> I am still struggling with this.  4.7 and 4.6 still do not build.
> Right now there are two problems that need thought:
> 
> 
> 1. That old ipxe is just too badly broken.  I spent a long while
> trying to backport compiler fixes but it is totally ridiculous.  IMO
> our only sensible option is to update at least osstest's buildsx to a
> much newer ipxe.
> 
> This could be done by cherry picking
>      38ab99b26bf4298a33105ec66f3f6a3f7e05a326
>        ipxe: update to newer commit
> (which is from xen 4.8 ish) onto our 4.7 and 4.6 branches.
> 
> If this is felt too intrusive, then I could somehow make it
> conditional and have osstest use it.  This is not entirely trivial
> because we have an ad hoc patch application thing.

Since the new ipxe should have been proven stable enough in
the newer trees, I think cherry-picking said commit should be
fine.

> 2. hvmloader fails to build, I think because we need
>     7825ae12df1f6d48c4d009cbbdf5a55aff27291b
>       errno: introduce EISDIR/EROFS/ENOTEMPTY to the ABI
>     03720ea541382a3ca80eaaec2aa11932b03aacaf
>       errno: declare aliases using XEN_ERRNO()
>     67790205df26e7c3dfeef8b8e64194ebc279220d
>       public/errno: Reduce complexity of inclusion
>     305e957ffee94fc06c4ba53ef5562f1b8c1c6b02
>       hvmloader: use xen/errno.h rather than the host systems errno.h
> 
> Is backporting that lot OK ?

I think so, yes, albeit I'm puzzled how they would end up being
needed.

> There are also some simple backports we need:
>    c2a17869d5dcd845d646bf4db122cad73596a2be
>      libfsimage: replace deprecated readdir_r() with readdir()
>    b9daff9d811285f1e40669bc621c2241793f7a95
>      libxl: replace deprecated readdir_r() with readdir()
>    668e4edf92fcf7cb929eed221059a3eeb02722c3
>      stubdom: make GMP aware that it's being cross-compiled
>    2f9eb73c2e2d7fdda8e2586c20f7dbd856002eba
>       stubdom: fix stubdom-vtpm build
> 
> 
> With all of the above, 4.6 builds again.  I guess 4.7 will too.
> 
> Fixing that will also probably enable the 4.8 push gate to pass.  It
> is currently blocked because it wants to test 4.7->4.8 migration and
> can't build 4.7.

And similarly in turn for 4.9's need to have a building 4.8 baseline,
afaict.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] Stable trees (4.6 and 4.7), building on stretch, osstest, redux
Posted by Ian Jackson 4 years, 10 months ago
Jan Beulich writes ("Re: Stable trees (4.6 and 4.7), building on stretch, osstest, redux"):
> On 28.05.19 at 21:59, <ian.jackson@citrix.com> wrote:
> > 1. That old ipxe is just too badly broken.  I spent a long while
> > trying to backport compiler fixes but it is totally ridiculous.  IMO
> > our only sensible option is to update at least osstest's buildsx to a
> > much newer ipxe.
> > 
> > This could be done by cherry picking
> >      38ab99b26bf4298a33105ec66f3f6a3f7e05a326
> >        ipxe: update to newer commit
> > (which is from xen 4.8 ish) onto our 4.7 and 4.6 branches.
> > 
> > If this is felt too intrusive, then I could somehow make it
> > conditional and have osstest use it.  This is not entirely trivial
> > because we have an ad hoc patch application thing.
> 
> Since the new ipxe should have been proven stable enough in
> the newer trees, I think cherry-picking said commit should be
> fine.

That wasn't what I was expecting you to say :-).  I will go with your
opinion since it is certainly less work.

> > 2. hvmloader fails to build, I think because we need
> >     7825ae12df1f6d48c4d009cbbdf5a55aff27291b
> >       errno: introduce EISDIR/EROFS/ENOTEMPTY to the ABI
> >     03720ea541382a3ca80eaaec2aa11932b03aacaf
> >       errno: declare aliases using XEN_ERRNO()
> >     67790205df26e7c3dfeef8b8e64194ebc279220d
> >       public/errno: Reduce complexity of inclusion
> >     305e957ffee94fc06c4ba53ef5562f1b8c1c6b02
> >       hvmloader: use xen/errno.h rather than the host systems errno.h
> > 
> > Is backporting that lot OK ?
> 
> I think so, yes, albeit I'm puzzled how they would end up being
> needed.

We need the last of these because in stretch Linux the host system's
errno header files were reorganised in a way that was incompatible
with the wrong-headed use of host headers and (only some) host header
directories.

The others are neded because of textual or semantic conflicts.

> > There are also some simple backports we need:
> >    c2a17869d5dcd845d646bf4db122cad73596a2be
> >      libfsimage: replace deprecated readdir_r() with readdir()
> >    b9daff9d811285f1e40669bc621c2241793f7a95
> >      libxl: replace deprecated readdir_r() with readdir()
> >    668e4edf92fcf7cb929eed221059a3eeb02722c3
> >      stubdom: make GMP aware that it's being cross-compiled
> >    2f9eb73c2e2d7fdda8e2586c20f7dbd856002eba
> >       stubdom: fix stubdom-vtpm build
> > 
> > 
> > With all of the above, 4.6 builds again.  I guess 4.7 will too.
> > 
> > Fixing that will also probably enable the 4.8 push gate to pass.  It
> > is currently blocked because it wants to test 4.7->4.8 migration and
> > can't build 4.7.
> 
> And similarly in turn for 4.9's need to have a building 4.8 baseline,
> afaict.

Yes.  We have fixed the 4.8 build but it hasn't passed its push gate
so 4.9 is using the "tested" branch...

OK, I will go ahead with all of the above for 4.6 and 4.7.  Thanks.

Thanks also to Olaf.  Your experience with the ipxe problems was
similar to mine.  I think just upgrading is the best approach.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH STABLE] tools/firmware: update OVMF Makefile, when necessaryf
Posted by Ian Jackson 4 years, 10 months ago
Ian Jackson writes ("[PATCH STABLE] tools/firmware: update OVMF Makefile, when necessary"):
> Now done, including for staging-4.6.  4.6 is out of security support,
> but osstest wants to be able to build 4.6 so that it can test
> migration from 4.6 to 4.7, and 4.7 *is* still (just about) in security
> support.

I have cherry picked another 2 build fixes from 4.7 onto staging-4.6:

  7c8db58d3739c805f4c0f773b65157f306b00c2a
  Fix misleading indentation warnings

  e675332d5d049bbf5ce4cf1924a6414b8035963d
  xenalyze: remove cr3_compare_total

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH STABLE] tools/firmware: update OVMF Makefile, when necessary
Posted by Ian Jackson 4 years, 11 months ago
Ian Jackson writes ("[PATCH STABLE] tools/firmware: update OVMF Makefile, when necessary"):
> Ian Jackson writes ("[PATCH STABLE] tools/firmware: update OVMF Makefile, when necessary"):
> > On advice from Wei, I am about to push this squashed backport to the
> > stable trees.  We think this will help fix osstest on those branches
> > following the upgrade to Debian stretch.
> 
> Now done, including for staging-4.6.  4.6 is out of security support,
> but osstest wants to be able to build 4.6 so that it can test
> migration from 4.6 to 4.7, and 4.7 *is* still (just about) in security
> support.
> 
> I expect the 4.6 push gate to fail and to have to force push it.
> 
> It is also possible that we will experience other build fallout.

In fact, even with this patch, 4.6 does not build because it is
missing a backport of
   ebdba150bff1d914805d60efa576337bbef0c305
   xenalyze: fix misleading indentation.

So I have cherry picked
   ebdba150bff1d914805d60efa576337bbef0c305
which is is the 4.7 backport of that commit, onto staging-4.6.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel