[Qemu-devel] [PATCH 0/2] testing: Build WHPX enabled binaries

Philippe Mathieu-Daudé posted 2 patches 4 years, 7 months ago
Test docker-quick@centos7 passed
Test docker-clang@ubuntu passed
Test docker-mingw@fedora passed
Test checkpatch passed
Test asan passed
Test FreeBSD passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20190919105932.19412-1-philmd@redhat.com
Maintainers: "Alex Bennée" <alex.bennee@linaro.org>, Fam Zheng <fam@euphon.net>, "Philippe Mathieu-Daudé" <philmd@redhat.com>
There is a newer version of this series
.shippable.yml                                |  2 ++
tests/docker/Makefile.include                 |  1 +
.../dockerfiles/fedora-win10sdk-cross.docker  | 21 +++++++++++++++++++
3 files changed, 24 insertions(+)
create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
[Qemu-devel] [PATCH 0/2] testing: Build WHPX enabled binaries
Posted by Philippe Mathieu-Daudé 4 years, 7 months ago
Add a job to cross-build QEMU with WHPX enabled.

Use the Win10SDK headers from the Android Project, as commented
in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html

Based-on: <20190918121101.30690-1-philmd@redhat.com>
https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.html

Philippe Mathieu-Daudé (2):
  tests/docker: Add fedora-win10sdk-cross image
  .shippable.yml: Build WHPX enabled binaries

 .shippable.yml                                |  2 ++
 tests/docker/Makefile.include                 |  1 +
 .../dockerfiles/fedora-win10sdk-cross.docker  | 21 +++++++++++++++++++
 3 files changed, 24 insertions(+)
 create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker

-- 
2.20.1


Re: [Qemu-devel] [PATCH 0/2] testing: Build WHPX enabled binaries
Posted by Thomas Huth 4 years, 7 months ago
On 19/09/2019 12.59, Philippe Mathieu-Daudé wrote:
> Add a job to cross-build QEMU with WHPX enabled.
> 
> Use the Win10SDK headers from the Android Project, as commented
> in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
> 
> Based-on: <20190918121101.30690-1-philmd@redhat.com>
> https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.html
> 
> Philippe Mathieu-Daudé (2):
>   tests/docker: Add fedora-win10sdk-cross image
>   .shippable.yml: Build WHPX enabled binaries
> 
>  .shippable.yml                                |  2 ++
>  tests/docker/Makefile.include                 |  1 +
>  .../dockerfiles/fedora-win10sdk-cross.docker  | 21 +++++++++++++++++++
>  3 files changed, 24 insertions(+)
>  create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker

Don't you want to include a fix for the #include "hw/boards.h" problem
in your series? ... or is it not happening with your builds here?

 Thomas

Re: [Qemu-devel] [PATCH 0/2] testing: Build WHPX enabled binaries
Posted by Philippe Mathieu-Daudé 4 years, 7 months ago
On 9/19/19 1:09 PM, Thomas Huth wrote:
> On 19/09/2019 12.59, Philippe Mathieu-Daudé wrote:
>> Add a job to cross-build QEMU with WHPX enabled.
>>
>> Use the Win10SDK headers from the Android Project, as commented
>> in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
>>
>> Based-on: <20190918121101.30690-1-philmd@redhat.com>
>> https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.html
>>
>> Philippe Mathieu-Daudé (2):
>>   tests/docker: Add fedora-win10sdk-cross image
>>   .shippable.yml: Build WHPX enabled binaries
>>
>>  .shippable.yml                                |  2 ++
>>  tests/docker/Makefile.include                 |  1 +
>>  .../dockerfiles/fedora-win10sdk-cross.docker  | 21 +++++++++++++++++++
>>  3 files changed, 24 insertions(+)
>>  create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
> 
> Don't you want to include a fix for the #include "hw/boards.h" problem
> in your series? ... or is it not happening with your builds here?

Ah Based-on is for Patchew, and this series need to be tested by
Shippable... you are right.
Well, this is a good opportunity to see the job catch the WHPX failure
reported :)

Re: [Qemu-devel] [PATCH 0/2] testing: Build WHPX enabled binaries
Posted by Stefan Weil 4 years, 7 months ago
Am 19.09.2019 um 12:59 schrieb Philippe Mathieu-Daudé:
> Add a job to cross-build QEMU with WHPX enabled.
>
> Use the Win10SDK headers from the Android Project, as commented
> in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
>
> Based-on: <20190918121101.30690-1-philmd@redhat.com>
> https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.html
>
> Philippe Mathieu-Daudé (2):
>   tests/docker: Add fedora-win10sdk-cross image
>   .shippable.yml: Build WHPX enabled binaries
>
>  .shippable.yml                                |  2 ++
>  tests/docker/Makefile.include                 |  1 +
>  .../dockerfiles/fedora-win10sdk-cross.docker  | 21 +++++++++++++++++++
>  3 files changed, 24 insertions(+)
>  create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
>

Please note that the required header files are part of the Win10SDK
which is not published under a free license, so I am afraid that they
cannot be used with QEMU code to produce free binaries.

I have addressed that some time ago, and Justin Terry is still looking
for a solution on the Microsoft side.

Regards,
Stefan Weil



Re: [Qemu-devel] [PATCH 0/2] testing: Build WHPX enabled binaries
Posted by Philippe Mathieu-Daudé 4 years, 7 months ago
On 9/19/19 1:18 PM, Stefan Weil wrote:
> Am 19.09.2019 um 12:59 schrieb Philippe Mathieu-Daudé:
>> Add a job to cross-build QEMU with WHPX enabled.
>>
>> Use the Win10SDK headers from the Android Project, as commented
>> in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
>>
>> Based-on: <20190918121101.30690-1-philmd@redhat.com>
>> https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.html
>>
>> Philippe Mathieu-Daudé (2):
>>   tests/docker: Add fedora-win10sdk-cross image
>>   .shippable.yml: Build WHPX enabled binaries
>>
>>  .shippable.yml                                |  2 ++
>>  tests/docker/Makefile.include                 |  1 +
>>  .../dockerfiles/fedora-win10sdk-cross.docker  | 21 +++++++++++++++++++
>>  3 files changed, 24 insertions(+)
>>  create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
>>
> 
> Please note that the required header files are part of the Win10SDK
> which is not published under a free license, so I am afraid that they
> cannot be used with QEMU code to produce free binaries.

Yes :S

> I have addressed that some time ago, and Justin Terry is still looking
> for a solution on the Microsoft side.

Oh this is a good news, thanks for caring about this issue,
and thanks Justin for looking for a solution!

Trying to understand how WHPX is used, I noticed there are much many
Windows QEMU users than I thought, and it would be nice if we can have
some upstream CI testing to not break the various projects using it.

Regards,

Phil.


Re: [PATCH 0/2] testing: Build WHPX enabled binaries
Posted by Philippe Mathieu-Daudé 3 years, 11 months ago
+launchpad ticket

On 9/19/19 1:26 PM, Philippe Mathieu-Daudé wrote:
> On 9/19/19 1:18 PM, Stefan Weil wrote:
>> Am 19.09.2019 um 12:59 schrieb Philippe Mathieu-Daudé:
>>> Add a job to cross-build QEMU with WHPX enabled.
>>>
>>> Use the Win10SDK headers from the Android Project, as commented
>>> in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
>>>
>>> Based-on: <20190918121101.30690-1-philmd@redhat.com>
>>> https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.html
>>>
>>> Philippe Mathieu-Daudé (2):
>>>    tests/docker: Add fedora-win10sdk-cross image
>>>    .shippable.yml: Build WHPX enabled binaries
>>>
>>>   .shippable.yml                                |  2 ++
>>>   tests/docker/Makefile.include                 |  1 +
>>>   .../dockerfiles/fedora-win10sdk-cross.docker  | 21 +++++++++++++++++++
>>>   3 files changed, 24 insertions(+)
>>>   create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
>>>
>>
>> Please note that the required header files are part of the Win10SDK
>> which is not published under a free license, so I am afraid that they
>> cannot be used with QEMU code to produce free binaries.
> 
> Yes :S
> 
>> I have addressed that some time ago, and Justin Terry is still looking
>> for a solution on the Microsoft side.
> 
> Oh this is a good news, thanks for caring about this issue,
> and thanks Justin for looking for a solution!
> 
> Trying to understand how WHPX is used, I noticed there are much many
> Windows QEMU users than I thought, and it would be nice if we can have
> some upstream CI testing to not break the various projects using it.
> 
> Regards,
> 
> Phil.
> 


Re: [Qemu-devel] [PATCH 0/2] testing: Build WHPX enabled binaries
Posted by Daniel P. Berrangé 4 years, 7 months ago
On Thu, Sep 19, 2019 at 01:18:57PM +0200, Stefan Weil wrote:
> Am 19.09.2019 um 12:59 schrieb Philippe Mathieu-Daudé:
> > Add a job to cross-build QEMU with WHPX enabled.
> >
> > Use the Win10SDK headers from the Android Project, as commented
> > in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
> >
> > Based-on: <20190918121101.30690-1-philmd@redhat.com>
> > https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.html
> >
> > Philippe Mathieu-Daudé (2):
> >   tests/docker: Add fedora-win10sdk-cross image
> >   .shippable.yml: Build WHPX enabled binaries
> >
> >  .shippable.yml                                |  2 ++
> >  tests/docker/Makefile.include                 |  1 +
> >  .../dockerfiles/fedora-win10sdk-cross.docker  | 21 +++++++++++++++++++
> >  3 files changed, 24 insertions(+)
> >  create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
> >
> 
> Please note that the required header files are part of the Win10SDK
> which is not published under a free license, so I am afraid that they
> cannot be used with QEMU code to produce free binaries.

Can you elaborate on why you think that is the case ?

Looking at the 3 header files used, all they contain are struct
definitions and integer constants, fortunately no functional
code in macros or inline functions. As such, it is questionable
whether the headers alone can be considered copyrightable material.
Any compatible implementation that follows the WHPX API/ABI will by
neccessity be the same, modulo code style.

Further, from a GPL POV, these headers should fall under the system
libraries exception, as WHPX is core low level operating system
functionality.

There's a valid legal question as to how to we got access to the
files in the first place. Assuming they are freely distributable
though, given their hosting on Google Android downloads.

So the fact that the headers are not under an open source / free
license doesn't look like it should impact the ability to distribute
the resulting QEMU binaries.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH 0/2] testing: Build WHPX enabled binaries
Posted by Philippe Mathieu-Daudé 4 years, 7 months ago
On 9/19/19 1:41 PM, Daniel P. Berrangé wrote:
> On Thu, Sep 19, 2019 at 01:18:57PM +0200, Stefan Weil wrote:
>> Am 19.09.2019 um 12:59 schrieb Philippe Mathieu-Daudé:
>>> Add a job to cross-build QEMU with WHPX enabled.
>>>
>>> Use the Win10SDK headers from the Android Project, as commented
>>> in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
>>>
>>> Based-on: <20190918121101.30690-1-philmd@redhat.com>
>>> https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.html
>>>
>>> Philippe Mathieu-Daudé (2):
>>>   tests/docker: Add fedora-win10sdk-cross image
>>>   .shippable.yml: Build WHPX enabled binaries
>>>
>>>  .shippable.yml                                |  2 ++
>>>  tests/docker/Makefile.include                 |  1 +
>>>  .../dockerfiles/fedora-win10sdk-cross.docker  | 21 +++++++++++++++++++
>>>  3 files changed, 24 insertions(+)
>>>  create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
>>>
>>
>> Please note that the required header files are part of the Win10SDK
>> which is not published under a free license, so I am afraid that they
>> cannot be used with QEMU code to produce free binaries.
> 
> Can you elaborate on why you think that is the case ?
> 
> Looking at the 3 header files used, all they contain are struct
> definitions and integer constants, fortunately no functional
> code in macros or inline functions. As such, it is questionable
> whether the headers alone can be considered copyrightable material.
> Any compatible implementation that follows the WHPX API/ABI will by
> neccessity be the same, modulo code style.
> 
> Further, from a GPL POV, these headers should fall under the system
> libraries exception, as WHPX is core low level operating system
> functionality.
> 
> There's a valid legal question as to how to we got access to the
> files in the first place. Assuming they are freely distributable
> though, given their hosting on Google Android downloads.
> 
> So the fact that the headers are not under an open source / free
> license doesn't look like it should impact the ability to distribute
> the resulting QEMU binaries.

Are we distributing the resulting binaries?
I don't think so, we only build until linking in a scratch container and
discard its content, we don't have access to the container storage via
Shippable (also Travis). We only use the exit code and the console
output from the build.

We are not distributing those headers neither, as we currently do not
distribute the docker images.

Regards,

Phil.

Re: [Qemu-devel] [PATCH 0/2] testing: Build WHPX enabled binaries
Posted by Daniel P. Berrangé 4 years, 7 months ago
On Thu, Sep 19, 2019 at 01:54:40PM +0200, Philippe Mathieu-Daudé wrote:
> On 9/19/19 1:41 PM, Daniel P. Berrangé wrote:
> > On Thu, Sep 19, 2019 at 01:18:57PM +0200, Stefan Weil wrote:
> >> Am 19.09.2019 um 12:59 schrieb Philippe Mathieu-Daudé:
> >>> Add a job to cross-build QEMU with WHPX enabled.
> >>>
> >>> Use the Win10SDK headers from the Android Project, as commented
> >>> in https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03842.html
> >>>
> >>> Based-on: <20190918121101.30690-1-philmd@redhat.com>
> >>> https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg03844.html
> >>>
> >>> Philippe Mathieu-Daudé (2):
> >>>   tests/docker: Add fedora-win10sdk-cross image
> >>>   .shippable.yml: Build WHPX enabled binaries
> >>>
> >>>  .shippable.yml                                |  2 ++
> >>>  tests/docker/Makefile.include                 |  1 +
> >>>  .../dockerfiles/fedora-win10sdk-cross.docker  | 21 +++++++++++++++++++
> >>>  3 files changed, 24 insertions(+)
> >>>  create mode 100644 tests/docker/dockerfiles/fedora-win10sdk-cross.docker
> >>>
> >>
> >> Please note that the required header files are part of the Win10SDK
> >> which is not published under a free license, so I am afraid that they
> >> cannot be used with QEMU code to produce free binaries.
> > 
> > Can you elaborate on why you think that is the case ?
> > 
> > Looking at the 3 header files used, all they contain are struct
> > definitions and integer constants, fortunately no functional
> > code in macros or inline functions. As such, it is questionable
> > whether the headers alone can be considered copyrightable material.
> > Any compatible implementation that follows the WHPX API/ABI will by
> > neccessity be the same, modulo code style.
> > 
> > Further, from a GPL POV, these headers should fall under the system
> > libraries exception, as WHPX is core low level operating system
> > functionality.
> > 
> > There's a valid legal question as to how to we got access to the
> > files in the first place. Assuming they are freely distributable
> > though, given their hosting on Google Android downloads.
> > 
> > So the fact that the headers are not under an open source / free
> > license doesn't look like it should impact the ability to distribute
> > the resulting QEMU binaries.
> 
> Are we distributing the resulting binaries?
> I don't think so, we only build until linking in a scratch container and
> discard its content, we don't have access to the container storage via
> Shippable (also Travis). We only use the exit code and the console
> output from the build.
> 
> We are not distributing those headers neither, as we currently do not
> distribute the docker images.

No, but I think its none the less useful to consider what impact is
on distribution of binaries. Currently Stefan builds & distributes
the QEMU Windows binaries for each release. I think the ideal would
be for Windows builds to be a core deliverable of the upstram release
process. So this docker CI job can be viewed as a stepping stone to
that point, especially if it is made to run the full NSIS installer
build, not just standalone binary build.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|