[PATCH 00/14] hw: define and enforce a standard lifecycle for versioned machines

Daniel P. Berrangé posted 14 patches 2 weeks, 2 days ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20240501182759.2934195-1-berrange@redhat.com
Maintainers: Peter Maydell <peter.maydell@linaro.org>, "Michael S. Tsirkin" <mst@redhat.com>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, Paolo Bonzini <pbonzini@redhat.com>, Richard Henderson <richard.henderson@linaro.org>, Eduardo Habkost <eduardo@habkost.net>, Laurent Vivier <laurent@vivier.eu>, Nicholas Piggin <npiggin@gmail.com>, Daniel Henrique Barboza <danielhb413@gmail.com>, David Gibson <david@gibson.dropbear.id.au>, Harsh Prateek Bora <harshpb@linux.ibm.com>, Thomas Huth <thuth@redhat.com>, Halil Pasic <pasic@linux.ibm.com>, Christian Borntraeger <borntraeger@linux.ibm.com>, Eric Farman <farman@linux.ibm.com>, David Hildenbrand <david@redhat.com>, Ilya Leoshkevich <iii@linux.ibm.com>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, Yanan Wang <wangyanan55@huawei.com>
docs/about/deprecated.rst  |  12 ++
hw/arm/virt.c              |  30 +++--
hw/i386/pc_piix.c          | 252 +++++++++++++++-------------------
hw/i386/pc_q35.c           | 215 +++++++++++++----------------
hw/m68k/virt.c             |  53 +++++---
hw/ppc/spapr.c             |  96 +++++++------
hw/s390x/s390-virtio-ccw.c |  98 ++++++++------
include/hw/boards.h        | 268 +++++++++++++++++++++++++++++++++++++
include/hw/i386/pc.h       |  32 +++++
9 files changed, 666 insertions(+), 390 deletions(-)
[PATCH 00/14] hw: define and enforce a standard lifecycle for versioned machines
Posted by Daniel P. Berrangé 2 weeks, 2 days ago
Thomas proposed a new deprecation and removal policy for versioned
machine types that would see them liable for deletion after 6 years:

  https://lists.nongnu.org/archive/html/qemu-devel/2024-04/msg04683.html

This suggest was met with broad approval, however, I suggested that we
could take it further and actually mark them deprecated sooner, at the
3 year timeframe, and also fully automate the enablement of the runtime
deprecation warning without developer intervention on every release
cycle.

This series implements my suggestions.

The first patch introduces some helper macros and documents a standard
code pattern for defining versioned machine types across targets.

The next 6 patches convert existing targets with versioned machine
types (arm, s390x, ppc, m68k, i386) to use the new helper macros and
code patterns.

A further patch introduces some helper macros for automating the
handling of deprecation and deletion of versioned machine types.

Two more patches then enable the deprecation and deletion logic
across all versioned machines

Finally we do some cleanup and document the new policy.

........a tangent about VERSION file handling.......

One oddity here, is that during the development and release
candidate phases the automatic logic in this series has an off-by-1
error.

This is because when we, for example, add the "9.1" machine type
versions, the VERSION file is still reporting '9.0.50', and then
'9.0.9{1,2,3,4}'.

IOW, during development and in rc candidates, we fail to deprecate
and delete 1 machine type. We should already have deprecated the
6.1 machine types, but the most recently deprecated is 6.0.
This is pretty harmless since the final release does the right
thing.

I wonder, however, whether we would benefit from changing how we
update the VERSION file.

eg instead of re-using the micro digit to indicate a dev or rc
snapshot, represent those explicitly. eg "9.1.0-dev" and
"9.1.0-rc1", "9.1.0-rc2", etc in VERSION.

We don't use the full QEMU_VERSION in the code in all that many
places. It appears in some help messages for command line tools,
and in QMP query-version response, and in a few other misc places.
At a glance it appears all of those places would easily handle a
tagged version.

For release candidates in particular I think it would be saner
to show the user the actual version the release is about to become,
rather than the previous release's version. This would make the
reported version match the rc tarball naming too which would be
nice.

Anyway, this isn't a blocker for this machine type versioning
proposal, just a thought....

Daniel P. Berrangé (14):
  include/hw: add helpers for defining versioned machine types
  hw/arm: convert 'virt' machine definitions to use new macros
  hw/s390x: convert 'ccw' machine definitions to use new macros
  hw/ppc: convert 'spapr' machine definitions to use new macros
  hw/m68k: convert 'virt' machine definitions to use new macros
  hw/i386: convert 'i440fx' machine definitions to use new macros
  hw/i386: convert 'q35' machine definitions to use new macros
  include/hw: add macros for deprecation & removal of versioned machines
  hw: temporarily disable deletion of versioned machine types
  hw: set deprecation info for all versioned machine types
  hw: skip registration of outdated versioned machine types
  hw/ppc: remove obsolete manual deprecation reason string of spapr
    machines
  hw/i386: remove obsolete manual deprecation reason string of i440fx
    machines
  docs: document special exception for machine type deprecation &
    removal

 docs/about/deprecated.rst  |  12 ++
 hw/arm/virt.c              |  30 +++--
 hw/i386/pc_piix.c          | 252 +++++++++++++++-------------------
 hw/i386/pc_q35.c           | 215 +++++++++++++----------------
 hw/m68k/virt.c             |  53 +++++---
 hw/ppc/spapr.c             |  96 +++++++------
 hw/s390x/s390-virtio-ccw.c |  98 ++++++++------
 include/hw/boards.h        | 268 +++++++++++++++++++++++++++++++++++++
 include/hw/i386/pc.h       |  32 +++++
 9 files changed, 666 insertions(+), 390 deletions(-)

-- 
2.43.0


Re: [PATCH 00/14] hw: define and enforce a standard lifecycle for versioned machines
Posted by Cédric Le Goater 2 weeks ago
On 5/1/24 20:27, Daniel P. Berrangé wrote:
> Thomas proposed a new deprecation and removal policy for versioned
> machine types that would see them liable for deletion after 6 years:
> 
>    https://lists.nongnu.org/archive/html/qemu-devel/2024-04/msg04683.html
> 
> This suggest was met with broad approval, however, I suggested that we
> could take it further and actually mark them deprecated sooner, at the
> 3 year timeframe, and also fully automate the enablement of the runtime
> deprecation warning without developer intervention on every release
> cycle.
> 
> This series implements my suggestions.
> 
> The first patch introduces some helper macros and documents a standard
> code pattern for defining versioned machine types across targets.
> 
> The next 6 patches convert existing targets with versioned machine
> types (arm, s390x, ppc, m68k, i386) to use the new helper macros and
> code patterns.
> 
> A further patch introduces some helper macros for automating the
> handling of deprecation and deletion of versioned machine types.
> 
> Two more patches then enable the deprecation and deletion logic
> across all versioned machines
> 
> Finally we do some cleanup and document the new policy.
> 
> ........a tangent about VERSION file handling.......
> 
> One oddity here, is that during the development and release
> candidate phases the automatic logic in this series has an off-by-1
> error.
> 
> This is because when we, for example, add the "9.1" machine type
> versions, the VERSION file is still reporting '9.0.50', and then
> '9.0.9{1,2,3,4}'.
> 
> IOW, during development and in rc candidates, we fail to deprecate
> and delete 1 machine type. We should already have deprecated the
> 6.1 machine types, but the most recently deprecated is 6.0.
> This is pretty harmless since the final release does the right
> thing.
> 
> I wonder, however, whether we would benefit from changing how we
> update the VERSION file.
> 
> eg instead of re-using the micro digit to indicate a dev or rc
> snapshot, represent those explicitly. eg "9.1.0-dev" and
> "9.1.0-rc1", "9.1.0-rc2", etc in VERSION.
> 
> We don't use the full QEMU_VERSION in the code in all that many
> places. It appears in some help messages for command line tools,
> and in QMP query-version response, and in a few other misc places.
> At a glance it appears all of those places would easily handle a
> tagged version.
> 
> For release candidates in particular I think it would be saner
> to show the user the actual version the release is about to become,
> rather than the previous release's version. This would make the
> reported version match the rc tarball naming too which would be
> nice.
> 
> Anyway, this isn't a blocker for this machine type versioning
> proposal, just a thought....

I would agree with such a change. The version numbers always confused
me. AFAICT, only QEMU_VERSION_MICRO would need some massaging. It
shouldn't be too complex.

For the series,

Tested-by: Cédric Le Goater <clg@redhat.com>

Thanks,

C.



> 
> Daniel P. Berrangé (14):
>    include/hw: add helpers for defining versioned machine types
>    hw/arm: convert 'virt' machine definitions to use new macros
>    hw/s390x: convert 'ccw' machine definitions to use new macros
>    hw/ppc: convert 'spapr' machine definitions to use new macros
>    hw/m68k: convert 'virt' machine definitions to use new macros
>    hw/i386: convert 'i440fx' machine definitions to use new macros
>    hw/i386: convert 'q35' machine definitions to use new macros
>    include/hw: add macros for deprecation & removal of versioned machines
>    hw: temporarily disable deletion of versioned machine types
>    hw: set deprecation info for all versioned machine types
>    hw: skip registration of outdated versioned machine types
>    hw/ppc: remove obsolete manual deprecation reason string of spapr
>      machines
>    hw/i386: remove obsolete manual deprecation reason string of i440fx
>      machines
>    docs: document special exception for machine type deprecation &
>      removal
> 
>   docs/about/deprecated.rst  |  12 ++
>   hw/arm/virt.c              |  30 +++--
>   hw/i386/pc_piix.c          | 252 +++++++++++++++-------------------
>   hw/i386/pc_q35.c           | 215 +++++++++++++----------------
>   hw/m68k/virt.c             |  53 +++++---
>   hw/ppc/spapr.c             |  96 +++++++------
>   hw/s390x/s390-virtio-ccw.c |  98 ++++++++------
>   include/hw/boards.h        | 268 +++++++++++++++++++++++++++++++++++++
>   include/hw/i386/pc.h       |  32 +++++
>   9 files changed, 666 insertions(+), 390 deletions(-)
> 


Re: [PATCH 00/14] hw: define and enforce a standard lifecycle for versioned machines
Posted by Peter Maydell 2 weeks ago
On Wed, 1 May 2024 at 19:28, Daniel P. Berrangé <berrange@redhat.com> wrote:
> I wonder, however, whether we would benefit from changing how we
> update the VERSION file.
>
> eg instead of re-using the micro digit to indicate a dev or rc
> snapshot, represent those explicitly. eg "9.1.0-dev" and
> "9.1.0-rc1", "9.1.0-rc2", etc in VERSION.
>
> We don't use the full QEMU_VERSION in the code in all that many
> places. It appears in some help messages for command line tools,
> and in QMP query-version response, and in a few other misc places.
> At a glance it appears all of those places would easily handle a
> tagged version.
>
> For release candidates in particular I think it would be saner
> to show the user the actual version the release is about to become,
> rather than the previous release's version. This would make the
> reported version match the rc tarball naming too which would be
> nice.

I think the theory behind the VERSION file is that we want
to be able to express the version:
 * purely numerically
 * in a strictly ascending order

We expose the assumption of numeric versions in places like
QMP's query-version command, which reports it as a set of ints.

I think there's probably scope for making the "human friendly"
version string be surfaced instead of the strictly-numeric
one in more places, but I worry that breaking the "always
numeric and ascending" rule might have subtle breakage for
us or for downstream uses...

thanks
-- PMM
Re: [PATCH 00/14] hw: define and enforce a standard lifecycle for versioned machines
Posted by Daniel P. Berrangé 2 weeks ago
On Fri, May 03, 2024 at 01:14:27PM +0100, Peter Maydell wrote:
> On Wed, 1 May 2024 at 19:28, Daniel P. Berrangé <berrange@redhat.com> wrote:
> > I wonder, however, whether we would benefit from changing how we
> > update the VERSION file.
> >
> > eg instead of re-using the micro digit to indicate a dev or rc
> > snapshot, represent those explicitly. eg "9.1.0-dev" and
> > "9.1.0-rc1", "9.1.0-rc2", etc in VERSION.
> >
> > We don't use the full QEMU_VERSION in the code in all that many
> > places. It appears in some help messages for command line tools,
> > and in QMP query-version response, and in a few other misc places.
> > At a glance it appears all of those places would easily handle a
> > tagged version.
> >
> > For release candidates in particular I think it would be saner
> > to show the user the actual version the release is about to become,
> > rather than the previous release's version. This would make the
> > reported version match the rc tarball naming too which would be
> > nice.
> 
> I think the theory behind the VERSION file is that we want
> to be able to express the version:
>  * purely numerically
>  * in a strictly ascending order
> 
> We expose the assumption of numeric versions in places like
> QMP's query-version command, which reports it as a set of ints.

Right, we have:

#     -> { "execute": "query-version" }
#     <- {
#           "return":{
#              "qemu":{
#                 "major":0,
#                 "minor":11,
#                 "micro":5
#              },
#              "package":""
#           }
#        }


We could add an extra field to it thus:


#     -> { "execute": "query-version" }
#     <- {
#           "return":{
#              "qemu":{
#                 "major":0,
#                 "minor":11,
#                 "micro":5,
#                 "tag": "rc2"
#              },
#              "package":""
#           }
#        }

arguably we are still in strictly ascending order for the
numeric part, we merely didn't bump the numeric part of
the value at rc releases.

> I think there's probably scope for making the "human friendly"
> version string be surfaced instead of the strictly-numeric
> one in more places, but I worry that breaking the "always
> numeric and ascending" rule might have subtle breakage for
> us or for downstream uses...

Downstream I see no issues. There are already many projects which
use a '-dev' or '-rcNN' suffix for tarballs of unreleased versions,
and distros have guidelines on how they deal with this.

In fact downstream already deals with it for QEMU, because they will
typically set packaging versioning based on the version as seen in
the tarball name, not the different version seen in the VERSION file
(and thus QMP).

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|