[Qemu-devel] [RFC PATCH 0/6] generic way to deprecate machines

Philippe Mathieu-Daudé posted 6 patches 6 years, 4 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20171108022828.7242-1-f4bug@amsat.org
Test checkpatch passed
Test docker passed
Test ppc passed
Test s390x passed
include/hw/boards.h  | 1 +
hw/arm/gumstix.c     | 2 ++
hw/arm/xlnx-zcu102.c | 6 +++++-
hw/i386/pc_piix.c    | 3 +++
hw/ppc/prep.c        | 1 +
vl.c                 | 7 +++++++
6 files changed, 19 insertions(+), 1 deletion(-)
[Qemu-devel] [RFC PATCH 0/6] generic way to deprecate machines
Posted by Philippe Mathieu-Daudé 6 years, 4 months ago
Hi,

This series intends to provide a simple and common way to deprecate
machines between releases. It may be extended to deprecate devices.

I started with the machines list from:

  https://wiki.qemu.org/Features/LegacyRemoval#Deprecated_machines

Comment welcomed :)

Regards,

Phil.

Alistair Francis (1):
  hw/arm: deprecate the EP108 board

Philippe Mathieu-Daudé (5):
  machine: add a deprecated_reason property
  hw/arm: deprecate the Gumstix boards
  hw/ppc: deprecate the PReP machine
  hw/i386: deprecate the pc-0.10 and pc-0.11 machines
  hw/i386: deprecate the "isapc" machine

 include/hw/boards.h  | 1 +
 hw/arm/gumstix.c     | 2 ++
 hw/arm/xlnx-zcu102.c | 6 +++++-
 hw/i386/pc_piix.c    | 3 +++
 hw/ppc/prep.c        | 1 +
 vl.c                 | 7 +++++++
 6 files changed, 19 insertions(+), 1 deletion(-)

-- 
2.15.0


Re: [Qemu-devel] [RFC PATCH 0/6] generic way to deprecate machines
Posted by Daniel P. Berrange 6 years, 4 months ago
On Tue, Nov 07, 2017 at 11:28:22PM -0300, Philippe Mathieu-Daudé wrote:
> Hi,
> 
> This series intends to provide a simple and common way to deprecate
> machines between releases. It may be extended to deprecate devices.
> 
> I started with the machines list from:
> 
>   https://wiki.qemu.org/Features/LegacyRemoval#Deprecated_machines
> 
> Comment welcomed :)
> 
> Regards,
> 
> Phil.
> 
> Alistair Francis (1):
>   hw/arm: deprecate the EP108 board
> 
> Philippe Mathieu-Daudé (5):
>   machine: add a deprecated_reason property
>   hw/arm: deprecate the Gumstix boards
>   hw/ppc: deprecate the PReP machine
>   hw/i386: deprecate the pc-0.10 and pc-0.11 machines
>   hw/i386: deprecate the "isapc" machine
> 
>  include/hw/boards.h  | 1 +
>  hw/arm/gumstix.c     | 2 ++
>  hw/arm/xlnx-zcu102.c | 6 +++++-
>  hw/i386/pc_piix.c    | 3 +++
>  hw/ppc/prep.c        | 1 +
>  vl.c                 | 7 +++++++
>  6 files changed, 19 insertions(+), 1 deletion(-)

Unless listed in the qemu-doc.texi "Deprecated features" appendix,
things are not considered deprecated...

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [RFC PATCH 0/6] generic way to deprecate machines
Posted by Thomas Huth 5 years, 10 months ago
On 08.11.2017 03:28, Philippe Mathieu-Daudé wrote:
> Hi,
> 
> This series intends to provide a simple and common way to deprecate
> machines between releases. It may be extended to deprecate devices.

*ping*

Philippe, I just discovered your findings wrt to the Gumstix machines on
https://wiki.qemu.org/Features/LegacyRemoval#Deprecated_machines ...
sounds reasonable, so I think we should really continue here and mark
those as deprecated. And pc-0.10 and pc-0.11, too.

Do you have some spare time to respin your patch series?

 Thomas

Re: [Qemu-devel] [RFC PATCH 0/6] generic way to deprecate machines
Posted by Philippe Mathieu-Daudé 5 years, 10 months ago
Hi Thomas,

On 05/04/2018 02:15 PM, Thomas Huth wrote:
> On 08.11.2017 03:28, Philippe Mathieu-Daudé wrote:
>>
>> This series intends to provide a simple and common way to deprecate
>> machines between releases. It may be extended to deprecate devices.
> 
> *ping*
> 
> Philippe, I just discovered your findings wrt to the Gumstix machines on
> https://wiki.qemu.org/Features/LegacyRemoval#Deprecated_machines ...
> sounds reasonable, so I think we should really continue here and mark
> those as deprecated. And pc-0.10 and pc-0.11, too.

Ironically I have been using the Gumstix machines quite a lot for the SD
'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
reach the Linux userland since the kernel crashes), and plan to add SD
integration tests via Avocado.

This raises:

- What will happens if I add tests downloading running on their compiled
u-boot
(https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
and the company decides to remove this old directory?
Since sometimes old open-source software are hard to rebuild with recent
compilers, should we consider to use a public storage to keep
open-source (signed) blobs we can use for integration testing?

Avocado has a 'vmimage library' which could be extended, adding support
for binary url + detached gpg signatures from some QEMU maintainers?

(I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
Avocado SuperIO tests, which aren't guaranteed to stay downloadable
forever).

- Maybe I have to volunteer to update the Gumstix section from Orphan
'Odd Fixes' to keep those tests.

> Do you have some spare time to respin your patch series?

I'll make some.

Regards,

Phil.

> 
>  Thomas
>

[Qemu-devel] Publishing binary images for testing (was Re: [RFC PATCH 0/6] generic way to deprecate machines)
Posted by Eduardo Habkost 5 years, 10 months ago
(CCing Cleber and avocado-devel in case they have suggestions)

On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
[...]
> Ironically I have been using the Gumstix machines quite a lot for the SD
> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
> reach the Linux userland since the kernel crashes), and plan to add SD
> integration tests via Avocado.
> 
> This raises:
> 
> - What will happens if I add tests downloading running on their compiled
> u-boot
> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
> and the company decides to remove this old directory?
> Since sometimes old open-source software are hard to rebuild with recent
> compilers, should we consider to use a public storage to keep
> open-source (signed) blobs we can use for integration testing?

I think a maintained repository of images for testing would be
nice to have.  We need to be careful to comply with the license
of the software being distributed, though.

If the images are very small (like u-boot.bin above), it might be
OK to carry them in qemu.git, just like the images in pc-bios.

> 
> Avocado has a 'vmimage library' which could be extended, adding support
> for binary url + detached gpg signatures from some QEMU maintainers?

Requiring a signature makes the binaries hard to replace.  Any
specific reason to suggest gpg signatures instead of just a
(e.g.) sha256 hash?

> 
> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
> forever).

Question for the Avocado folks: how this is normally handled in
avocado/avocado-vt?  Do you maintain a repository for guest
images, or you always point to their original sources?

-- 
Eduardo

Re: [Qemu-devel] Publishing binary images for testing (was Re: [RFC PATCH 0/6] generic way to deprecate machines)
Posted by Cleber Rosa 5 years, 10 months ago

On 05/11/2018 09:55 AM, Eduardo Habkost wrote:
> (CCing Cleber and avocado-devel in case they have suggestions)
> 
> On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
> [...]
>> Ironically I have been using the Gumstix machines quite a lot for the SD
>> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
>> reach the Linux userland since the kernel crashes), and plan to add SD
>> integration tests via Avocado.
>>
>> This raises:
>>
>> - What will happens if I add tests downloading running on their compiled
>> u-boot
>> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
>> and the company decides to remove this old directory?
>> Since sometimes old open-source software are hard to rebuild with recent
>> compilers, should we consider to use a public storage to keep
>> open-source (signed) blobs we can use for integration testing?
> 
> I think a maintained repository of images for testing would be
> nice to have.  We need to be careful to comply with the license
> of the software being distributed, though.
> 
> If the images are very small (like u-boot.bin above), it might be
> OK to carry them in qemu.git, just like the images in pc-bios.
> 
>>
>> Avocado has a 'vmimage library' which could be extended, adding support
>> for binary url + detached gpg signatures from some QEMU maintainers?
> 
> Requiring a signature makes the binaries hard to replace.  Any
> specific reason to suggest gpg signatures instead of just a
> (e.g.) sha256 hash?
> 
>>
>> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
>> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
>> forever).
> 
> Question for the Avocado folks: how this is normally handled in
> avocado/avocado-vt?  Do you maintain a repository for guest
> images, or you always point to their original sources?
> 

For pure Avocado, the vmimage library attempts to fetch, by default, the
latest version of a guest image directly from the original sources.
Say, a Fedora image will be downloaded by default from the Fedora
servers.  Because of that, we don't pay too much attention to the
availability of specific (old?) versions of guest images.

For Avocado-VT, there are the JeOS images[1], which we keep on a test
"assets" directory.  We have a lot of storage/bandwidth availability, so
it can be used for other assets proven to be necessary for tests.

As long as distribution rights and licensing are not issues, we can
definitely use the same server for kernels, u-boot images and what not.

[1] - https://avocado-project.org/data/assets/

-- 
Cleber Rosa
[ Sr Software Engineer - Virtualization Team - Red Hat ]
[ Avocado Test Framework - avocado-framework.github.io ]
[  7ABB 96EB 8B46 B94D 5E0F  E9BB 657E 8D33 A5F2 09F3  ]

Re: [Qemu-devel] Publishing binary images for testing (was Re: [RFC PATCH 0/6] generic way to deprecate machines)
Posted by Alistair Francis 5 years, 10 months ago
On Fri, May 11, 2018 at 7:27 AM, Cleber Rosa <crosa@redhat.com> wrote:
>
>
> On 05/11/2018 09:55 AM, Eduardo Habkost wrote:
>> (CCing Cleber and avocado-devel in case they have suggestions)
>>
>> On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
>> [...]
>>> Ironically I have been using the Gumstix machines quite a lot for the SD
>>> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
>>> reach the Linux userland since the kernel crashes), and plan to add SD
>>> integration tests via Avocado.
>>>
>>> This raises:
>>>
>>> - What will happens if I add tests downloading running on their compiled
>>> u-boot
>>> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
>>> and the company decides to remove this old directory?
>>> Since sometimes old open-source software are hard to rebuild with recent
>>> compilers, should we consider to use a public storage to keep
>>> open-source (signed) blobs we can use for integration testing?
>>
>> I think a maintained repository of images for testing would be
>> nice to have.  We need to be careful to comply with the license
>> of the software being distributed, though.
>>
>> If the images are very small (like u-boot.bin above), it might be
>> OK to carry them in qemu.git, just like the images in pc-bios.
>>
>>>
>>> Avocado has a 'vmimage library' which could be extended, adding support
>>> for binary url + detached gpg signatures from some QEMU maintainers?
>>
>> Requiring a signature makes the binaries hard to replace.  Any
>> specific reason to suggest gpg signatures instead of just a
>> (e.g.) sha256 hash?
>>
>>>
>>> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
>>> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
>>> forever).
>>
>> Question for the Avocado folks: how this is normally handled in
>> avocado/avocado-vt?  Do you maintain a repository for guest
>> images, or you always point to their original sources?
>>
>
> For pure Avocado, the vmimage library attempts to fetch, by default, the
> latest version of a guest image directly from the original sources.
> Say, a Fedora image will be downloaded by default from the Fedora
> servers.  Because of that, we don't pay too much attention to the
> availability of specific (old?) versions of guest images.
>
> For Avocado-VT, there are the JeOS images[1], which we keep on a test
> "assets" directory.  We have a lot of storage/bandwidth availability, so
> it can be used for other assets proven to be necessary for tests.
>
> As long as distribution rights and licensing are not issues, we can
> definitely use the same server for kernels, u-boot images and what not.
>
> [1] - https://avocado-project.org/data/assets/

Is it possible to add something to the landing page at
https://avocado-project.org ?

The Palo Alto Network routers block the avocado-project.org page as
they classify it as blank. Something on the root URL would help fix
this.

Alistair

>
> --
> Cleber Rosa
> [ Sr Software Engineer - Virtualization Team - Red Hat ]
> [ Avocado Test Framework - avocado-framework.github.io ]
> [  7ABB 96EB 8B46 B94D 5E0F  E9BB 657E 8D33 A5F2 09F3  ]
>

Re: [Qemu-devel] Publishing binary images for testing (was Re: [RFC PATCH 0/6] generic way to deprecate machines)
Posted by Cleber Rosa 5 years, 10 months ago

On 05/18/2018 02:16 PM, Alistair Francis wrote:
> On Fri, May 11, 2018 at 7:27 AM, Cleber Rosa <crosa@redhat.com> wrote:
>>
>>
>> On 05/11/2018 09:55 AM, Eduardo Habkost wrote:
>>> (CCing Cleber and avocado-devel in case they have suggestions)
>>>
>>> On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
>>> [...]
>>>> Ironically I have been using the Gumstix machines quite a lot for the SD
>>>> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
>>>> reach the Linux userland since the kernel crashes), and plan to add SD
>>>> integration tests via Avocado.
>>>>
>>>> This raises:
>>>>
>>>> - What will happens if I add tests downloading running on their compiled
>>>> u-boot
>>>> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
>>>> and the company decides to remove this old directory?
>>>> Since sometimes old open-source software are hard to rebuild with recent
>>>> compilers, should we consider to use a public storage to keep
>>>> open-source (signed) blobs we can use for integration testing?
>>>
>>> I think a maintained repository of images for testing would be
>>> nice to have.  We need to be careful to comply with the license
>>> of the software being distributed, though.
>>>
>>> If the images are very small (like u-boot.bin above), it might be
>>> OK to carry them in qemu.git, just like the images in pc-bios.
>>>
>>>>
>>>> Avocado has a 'vmimage library' which could be extended, adding support
>>>> for binary url + detached gpg signatures from some QEMU maintainers?
>>>
>>> Requiring a signature makes the binaries hard to replace.  Any
>>> specific reason to suggest gpg signatures instead of just a
>>> (e.g.) sha256 hash?
>>>
>>>>
>>>> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
>>>> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
>>>> forever).
>>>
>>> Question for the Avocado folks: how this is normally handled in
>>> avocado/avocado-vt?  Do you maintain a repository for guest
>>> images, or you always point to their original sources?
>>>
>>
>> For pure Avocado, the vmimage library attempts to fetch, by default, the
>> latest version of a guest image directly from the original sources.
>> Say, a Fedora image will be downloaded by default from the Fedora
>> servers.  Because of that, we don't pay too much attention to the
>> availability of specific (old?) versions of guest images.
>>
>> For Avocado-VT, there are the JeOS images[1], which we keep on a test
>> "assets" directory.  We have a lot of storage/bandwidth availability, so
>> it can be used for other assets proven to be necessary for tests.
>>
>> As long as distribution rights and licensing are not issues, we can
>> definitely use the same server for kernels, u-boot images and what not.
>>
>> [1] - https://avocado-project.org/data/assets/
> 
> Is it possible to add something to the landing page at
> https://avocado-project.org ?
> 

Done!
- Cleber.

> The Palo Alto Network routers block the avocado-project.org page as
> they classify it as blank. Something on the root URL would help fix
> this.
> 
> Alistair
> 

Re: [Qemu-devel] Publishing binary images for testing (was Re: [RFC PATCH 0/6] generic way to deprecate machines)
Posted by Alistair Francis 5 years, 10 months ago
On Tue, May 22, 2018 at 9:17 AM, Cleber Rosa <crosa@redhat.com> wrote:
>
>
> On 05/18/2018 02:16 PM, Alistair Francis wrote:
>> On Fri, May 11, 2018 at 7:27 AM, Cleber Rosa <crosa@redhat.com> wrote:
>>>
>>>
>>> On 05/11/2018 09:55 AM, Eduardo Habkost wrote:
>>>> (CCing Cleber and avocado-devel in case they have suggestions)
>>>>
>>>> On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
>>>> [...]
>>>>> Ironically I have been using the Gumstix machines quite a lot for the SD
>>>>> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
>>>>> reach the Linux userland since the kernel crashes), and plan to add SD
>>>>> integration tests via Avocado.
>>>>>
>>>>> This raises:
>>>>>
>>>>> - What will happens if I add tests downloading running on their compiled
>>>>> u-boot
>>>>> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
>>>>> and the company decides to remove this old directory?
>>>>> Since sometimes old open-source software are hard to rebuild with recent
>>>>> compilers, should we consider to use a public storage to keep
>>>>> open-source (signed) blobs we can use for integration testing?
>>>>
>>>> I think a maintained repository of images for testing would be
>>>> nice to have.  We need to be careful to comply with the license
>>>> of the software being distributed, though.
>>>>
>>>> If the images are very small (like u-boot.bin above), it might be
>>>> OK to carry them in qemu.git, just like the images in pc-bios.
>>>>
>>>>>
>>>>> Avocado has a 'vmimage library' which could be extended, adding support
>>>>> for binary url + detached gpg signatures from some QEMU maintainers?
>>>>
>>>> Requiring a signature makes the binaries hard to replace.  Any
>>>> specific reason to suggest gpg signatures instead of just a
>>>> (e.g.) sha256 hash?
>>>>
>>>>>
>>>>> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
>>>>> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
>>>>> forever).
>>>>
>>>> Question for the Avocado folks: how this is normally handled in
>>>> avocado/avocado-vt?  Do you maintain a repository for guest
>>>> images, or you always point to their original sources?
>>>>
>>>
>>> For pure Avocado, the vmimage library attempts to fetch, by default, the
>>> latest version of a guest image directly from the original sources.
>>> Say, a Fedora image will be downloaded by default from the Fedora
>>> servers.  Because of that, we don't pay too much attention to the
>>> availability of specific (old?) versions of guest images.
>>>
>>> For Avocado-VT, there are the JeOS images[1], which we keep on a test
>>> "assets" directory.  We have a lot of storage/bandwidth availability, so
>>> it can be used for other assets proven to be necessary for tests.
>>>
>>> As long as distribution rights and licensing are not issues, we can
>>> definitely use the same server for kernels, u-boot images and what not.
>>>
>>> [1] - https://avocado-project.org/data/assets/
>>
>> Is it possible to add something to the landing page at
>> https://avocado-project.org ?
>>
>
> Done!

Awesome! It looks good and this should help everyone behind a proxy.

Alistair

> - Cleber.
>
>> The Palo Alto Network routers block the avocado-project.org page as
>> they classify it as blank. Something on the root URL would help fix
>> this.
>>
>> Alistair
>>

Re: [Qemu-devel] Publishing binary images for testing
Posted by Philippe Mathieu-Daudé 4 years, 9 months ago
Hi Cleber,

On 5/11/18 4:27 PM, Cleber Rosa wrote:
> On 05/11/2018 09:55 AM, Eduardo Habkost wrote:
>> (CCing Cleber and avocado-devel in case they have suggestions)
>>
>> On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
>> [...]
>>> Ironically I have been using the Gumstix machines quite a lot for the SD
>>> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
>>> reach the Linux userland since the kernel crashes), and plan to add SD
>>> integration tests via Avocado.
>>>
>>> This raises:
>>>
>>> - What will happens if I add tests downloading running on their compiled
>>> u-boot
>>> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
>>> and the company decides to remove this old directory?
>>> Since sometimes old open-source software are hard to rebuild with recent
>>> compilers, should we consider to use a public storage to keep
>>> open-source (signed) blobs we can use for integration testing?
>>
>> I think a maintained repository of images for testing would be
>> nice to have.  We need to be careful to comply with the license
>> of the software being distributed, though.
>>
>> If the images are very small (like u-boot.bin above), it might be
>> OK to carry them in qemu.git, just like the images in pc-bios.
>>
>>>
>>> Avocado has a 'vmimage library' which could be extended, adding support
>>> for binary url + detached gpg signatures from some QEMU maintainers?
>>
>> Requiring a signature makes the binaries hard to replace.  Any
>> specific reason to suggest gpg signatures instead of just a
>> (e.g.) sha256 hash?
>>
>>>
>>> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
>>> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
>>> forever).
>>
>> Question for the Avocado folks: how this is normally handled in
>> avocado/avocado-vt?  Do you maintain a repository for guest
>> images, or you always point to their original sources?
>>
> 
> For pure Avocado, the vmimage library attempts to fetch, by default, the
> latest version of a guest image directly from the original sources.
> Say, a Fedora image will be downloaded by default from the Fedora
> servers.  Because of that, we don't pay too much attention to the
> availability of specific (old?) versions of guest images.
> 
> For Avocado-VT, there are the JeOS images[1], which we keep on a test
> "assets" directory.  We have a lot of storage/bandwidth availability, so
> it can be used for other assets proven to be necessary for tests.
> 
> As long as distribution rights and licensing are not issues, we can
> definitely use the same server for kernels, u-boot images and what not.
> 
> [1] - https://avocado-project.org/data/assets/

1/ How do we check for distribution rights?

Is it OK for:
- a Debian/Fedora image
- a compiled Linux kernel (for a Debian/Fedora release)

2/ Who to ask to add files to this assets directory?

3/ Can we use a 'webarchive' directory structure?

Such /site/date/original_site_path/file

4/ What are the chances that this website disappears? :S

(Someone has to pay for it, and the bandwidth...)

Thanks,

Phil.

Re: [Qemu-devel] Publishing binary images for testing
Posted by Philippe Mathieu-Daudé 4 years, 7 months ago
ping? :)

On 6/17/19 7:17 AM, Philippe Mathieu-Daudé wrote:
> Hi Cleber,
> 
> On 5/11/18 4:27 PM, Cleber Rosa wrote:
>> On 05/11/2018 09:55 AM, Eduardo Habkost wrote:
>>> (CCing Cleber and avocado-devel in case they have suggestions)
>>>
>>> On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
>>> [...]
>>>> Ironically I have been using the Gumstix machines quite a lot for the SD
>>>> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
>>>> reach the Linux userland since the kernel crashes), and plan to add SD
>>>> integration tests via Avocado.
>>>>
>>>> This raises:
>>>>
>>>> - What will happens if I add tests downloading running on their compiled
>>>> u-boot
>>>> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
>>>> and the company decides to remove this old directory?
>>>> Since sometimes old open-source software are hard to rebuild with recent
>>>> compilers, should we consider to use a public storage to keep
>>>> open-source (signed) blobs we can use for integration testing?
>>>
>>> I think a maintained repository of images for testing would be
>>> nice to have.  We need to be careful to comply with the license
>>> of the software being distributed, though.
>>>
>>> If the images are very small (like u-boot.bin above), it might be
>>> OK to carry them in qemu.git, just like the images in pc-bios.
>>>
>>>>
>>>> Avocado has a 'vmimage library' which could be extended, adding support
>>>> for binary url + detached gpg signatures from some QEMU maintainers?
>>>
>>> Requiring a signature makes the binaries hard to replace.  Any
>>> specific reason to suggest gpg signatures instead of just a
>>> (e.g.) sha256 hash?
>>>
>>>>
>>>> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
>>>> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
>>>> forever).
>>>
>>> Question for the Avocado folks: how this is normally handled in
>>> avocado/avocado-vt?  Do you maintain a repository for guest
>>> images, or you always point to their original sources?
>>>
>>
>> For pure Avocado, the vmimage library attempts to fetch, by default, the
>> latest version of a guest image directly from the original sources.
>> Say, a Fedora image will be downloaded by default from the Fedora
>> servers.  Because of that, we don't pay too much attention to the
>> availability of specific (old?) versions of guest images.
>>
>> For Avocado-VT, there are the JeOS images[1], which we keep on a test
>> "assets" directory.  We have a lot of storage/bandwidth availability, so
>> it can be used for other assets proven to be necessary for tests.
>>
>> As long as distribution rights and licensing are not issues, we can
>> definitely use the same server for kernels, u-boot images and what not.
>>
>> [1] - https://avocado-project.org/data/assets/
> 
> 1/ How do we check for distribution rights?
> 
> Is it OK for:
> - a Debian/Fedora image
> - a compiled Linux kernel (for a Debian/Fedora release)
> 
> 2/ Who to ask to add files to this assets directory?
> 
> 3/ Can we use a 'webarchive' directory structure?
> 
> Such /site/date/original_site_path/file
> 
> 4/ What are the chances that this website disappears? :S
> 
> (Someone has to pay for it, and the bandwidth...)
> 
> Thanks,
> 
> Phil.

Re: [Qemu-devel] Publishing binary images for testing
Posted by Cleber Rosa 4 years, 7 months ago
On Fri, Aug 16, 2019 at 12:51:06PM +0200, Philippe Mathieu-Daudé wrote:
> ping? :)
> 
> On 6/17/19 7:17 AM, Philippe Mathieu-Daudé wrote:
> > Hi Cleber,
> > 
> > On 5/11/18 4:27 PM, Cleber Rosa wrote:
> >> On 05/11/2018 09:55 AM, Eduardo Habkost wrote:
> >>> (CCing Cleber and avocado-devel in case they have suggestions)
> >>>
> >>> On Tue, May 08, 2018 at 12:47:52PM -0300, Philippe Mathieu-Daudé wrote:
> >>> [...]
> >>>> Ironically I have been using the Gumstix machines quite a lot for the SD
> >>>> 'subsystem' refactor, using the MMC commands in U-Boot (I am unable to
> >>>> reach the Linux userland since the kernel crashes), and plan to add SD
> >>>> integration tests via Avocado.
> >>>>
> >>>> This raises:
> >>>>
> >>>> - What will happens if I add tests downloading running on their compiled
> >>>> u-boot
> >>>> (https://downloads.gumstix.com/images/angstrom/developer/2012-01-22-1750/u-boot.bin)
> >>>> and the company decides to remove this old directory?
> >>>> Since sometimes old open-source software are hard to rebuild with recent
> >>>> compilers, should we consider to use a public storage to keep
> >>>> open-source (signed) blobs we can use for integration testing?
> >>>
> >>> I think a maintained repository of images for testing would be
> >>> nice to have.  We need to be careful to comply with the license
> >>> of the software being distributed, though.
> >>>
> >>> If the images are very small (like u-boot.bin above), it might be
> >>> OK to carry them in qemu.git, just like the images in pc-bios.
> >>>
> >>>>
> >>>> Avocado has a 'vmimage library' which could be extended, adding support
> >>>> for binary url + detached gpg signatures from some QEMU maintainers?

Yes, although I believe a stronger distinction between the images
originally targeted by avocado.utils.vmimage and those other images
should probably exist.  One of the reasons that make me think so is
that the images obtained through vmimage *should* be configurable by
the accompanying avocado.utils.cloudinit library.

> >>>
> >>> Requiring a signature makes the binaries hard to replace.  Any
> >>> specific reason to suggest gpg signatures instead of just a
> >>> (e.g.) sha256 hash?
> >>>
> >>>>
> >>>> (I am also using old Gentoo/Debian packaged HPPA/Alpha Linux kernel for
> >>>> Avocado SuperIO tests, which aren't guaranteed to stay downloadable
> >>>> forever).
> >>>
> >>> Question for the Avocado folks: how this is normally handled in
> >>> avocado/avocado-vt?  Do you maintain a repository for guest
> >>> images, or you always point to their original sources?
> >>>
> >>
> >> For pure Avocado, the vmimage library attempts to fetch, by default, the
> >> latest version of a guest image directly from the original sources.
> >> Say, a Fedora image will be downloaded by default from the Fedora
> >> servers.  Because of that, we don't pay too much attention to the
> >> availability of specific (old?) versions of guest images.
> >>
> >> For Avocado-VT, there are the JeOS images[1], which we keep on a test
> >> "assets" directory.  We have a lot of storage/bandwidth availability, so
> >> it can be used for other assets proven to be necessary for tests.
> >>
> >> As long as distribution rights and licensing are not issues, we can
> >> definitely use the same server for kernels, u-boot images and what not.
> >>
> >> [1] - https://avocado-project.org/data/assets/
> > 
> > 1/ How do we check for distribution rights?
> > 
> > Is it OK for:
> > - a Debian/Fedora image
> > - a compiled Linux kernel (for a Debian/Fedora release)
> >

I think released images are just fine.

FIY, with regards to custom iamges: the Avocado-VT JeOS images are
based on Fedora, and can be recreated with the kickstart files that
are also in-tree.  Making the custom binary file reproducible is,
AFAICT good enough.

> > 2/ Who to ask to add files to this assets directory?
> >

I'd be happy to setup a secure upload mechanism, say, ftp+ssl, so that
"image maintainers" can upload new images.

> > 3/ Can we use a 'webarchive' directory structure?
> > 
> > Such /site/date/original_site_path/file
> >

I don't see why not.

> > 4/ What are the chances that this website disappears? :S
> > 
> > (Someone has to pay for it, and the bandwidth...)

Of course the best scenario would be to rely on some bigger
institution, say an University decides to host this archive of images,
and others decide to mirror them.  Until then, if it's decided to push
forward the use of the "avocado-project.org" site, the chances of it
going away are basically if I fail to have $5/month to keep it
running, so hopefully (I mean it) this won't be an issue :)

And thans for the ping, although I was really in need of a brick thrown
at my head to wake me up on this thread! :)

- Cleber.

> > 
> > Thanks,
> > 
> > Phil.