[PATCH v1 0/9] query & cache host-recommended CPU model

Collin Walling posted 9 patches 7 months, 2 weeks ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20230911210716.75648-1-walling@linux.ibm.com
src/conf/cpu_conf.c                           |   1 +
src/conf/cpu_conf.h                           |   1 +
src/conf/domain_capabilities.c                |  16 ++
src/conf/domain_capabilities.h                |   1 +
src/conf/schemas/cputypes.rng                 |   1 +
src/conf/schemas/domaincaps.rng               |  24 +++
src/cpu/cpu.c                                 |   2 +
src/cpu/cpu_ppc64.c                           |   2 +
src/qemu/qemu_capabilities.c                  | 144 +++++++++++++++---
src/qemu/qemu_capabilities.h                  |   4 +
src/qemu/qemu_capspriv.h                      |   6 +-
src/qemu/qemu_command.c                       |   5 +
src/qemu/qemu_domain.c                        |   2 +
src/qemu/qemu_driver.c                        |  18 ++-
src/qemu/qemu_monitor.h                       |   1 +
src/qemu/qemu_monitor_json.c                  |   4 +-
src/qemu/qemu_process.c                       |   9 +-
src/qemu/qemu_validate.c                      |   8 +
tests/cputest.c                               |   4 +-
tests/domaincapsdata/qemu_8.1.0.s390x.xml     |  55 +++++++
.../caps_8.1.0_s390x.replies                  |  80 +++++++++-
.../qemucapabilitiesdata/caps_8.1.0_s390x.xml |  55 +++++++
...c-cpu-kvm-ccw-virtio-8.1.s390x-latest.args |  32 ++++
.../s390-host-rec-cpu-kvm-ccw-virtio-8.1.xml  |  17 +++
tests/qemuxml2argvtest.c                      |   2 +
...ec-cpu-kvm-ccw-virtio-8.1.s390x-latest.xml |  25 +++
tests/qemuxml2xmltest.c                       |   1 +
27 files changed, 486 insertions(+), 34 deletions(-)
create mode 100644 tests/qemuxml2argvdata/s390-host-rec-cpu-kvm-ccw-virtio-8.1.s390x-latest.args
create mode 100644 tests/qemuxml2argvdata/s390-host-rec-cpu-kvm-ccw-virtio-8.1.xml
create mode 100644 tests/qemuxml2xmloutdata/s390-host-rec-cpu-kvm-ccw-virtio-8.1.s390x-latest.xml
[PATCH v1 0/9] query & cache host-recommended CPU model
Posted by Collin Walling 7 months, 2 weeks ago
Notes:
https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg02518.html
 - For information regarding the QEMU "static-recommended" (aka host-recommended)
   CPU model, please see the analogous QEMU patches: 

     https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg02518.html

 - Patches include caps added to QEMU 8.1 test files. This is a stand-in for the
   interim and the appropriate caps files will be updated in the future pending 
   QEMU acceptance.

Description:

Allow libvirt to query, cache, and report the "host-recommended" CPU model, which
reports CPU features the hypervisor recommends older CPU models to disable in order
to ensure migration to new-generation machines that will no longer support said
features.

The host-recommended CPU is a retrieved via a query to QEMU that is a combination 
of two parameters: the standard model name "host" and a new model type
"static-recommended". The resulting CPU is similar to the host-model which now
reports deprecated CPU features disabled (e.g. featurex=off). Once this model has
been queried, the libvirt ABI will parse the feature on|off parameter and set the
feature policy as appropriate. The resulting model is cached in the respective QEMU
capabilities file. It can be reported via the virsh domcapabilities command

Two "host models" are now represented: the original "host-model" that reports the
hypervisor-supported features as-is, and the new "host-recommended" that reports
the hypervisor-supported features with any deprecated features paired with the
disabled policy.

Additionally, hypervisor-cpu-baseline has been modified to output the feature
policy and report disabled features. Baseline will also expand the resulting
model with the host-recommended parameters if possible.

The following s390x features are flagged as deprecated and will be dropped
outright on future models: csske, te, cte, bpb.

Examples:

To enable this CPU model, define a guest with the following <cpu> element:

  <cpu mode='host-recommended' match='exact' check='partial'/>

A guest will be generated with a CPU e.g.:

    <cpu mode='custom' match='exact' check='partial'>
      <model fallback='forbid'>gen16a-base</model>
        <feature policy='require' name='nnpa'/>
        <feature policy='require' name='aen'/>
        <feature policy='require' name='cmmnt'/>
        <feature policy='require' name='vxpdeh'/>
        <feature policy='require' name='aefsi'/>
        <feature policy='require' name='diag318'/>
        <feature policy='disable' name='csske'/>
        <feature policy='require' name='mepoch'/>
        <feature policy='require' name='msa9'/>
        <feature policy='require' name='msa8'/>
        <feature policy='require' name='msa7'/>
        <feature policy='require' name='msa6'/>
        <feature policy='require' name='msa5'/>
        <feature policy='require' name='msa4'/>
        <feature policy='require' name='msa3'/>
        <feature policy='require' name='msa2'/>
        <feature policy='require' name='msa1'/>
        <feature policy='require' name='sthyi'/>
        <feature policy='require' name='edat'/>
        <feature policy='require' name='ri'/>
        <feature policy='require' name='deflate'/>
        <feature policy='require' name='edat2'/>
        <feature policy='require' name='etoken'/>
        <feature policy='require' name='vx'/>
        <feature policy='require' name='ipter'/>
        <feature policy='require' name='pai'/>
        <feature policy='require' name='paie'/>
        <feature policy='require' name='mepochptff'/>
        <feature policy='require' name='ap'/>
        <feature policy='require' name='vxeh'/>
        <feature policy='require' name='vxpd'/>
        <feature policy='require' name='esop'/>
        <feature policy='require' name='msa9_pckmo'/>
        <feature policy='require' name='vxeh2'/>
        <feature policy='require' name='esort'/>
        <feature policy='require' name='apqi'/>
        <feature policy='require' name='apft'/>
        <feature policy='require' name='els'/>
        <feature policy='require' name='iep'/>
        <feature policy='require' name='apqci'/>
        <feature policy='disable' name='cte'/>
        <feature policy='require' name='ais'/>
        <feature policy='disable' name='bpb'/>
        <feature policy='require' name='gs'/>
        <feature policy='require' name='ppa15'/>
        <feature policy='require' name='zpci'/>
        <feature policy='require' name='rdp'/>
        <feature policy='require' name='sea_esop2'/>
        <feature policy='require' name='beareh'/>
        <feature policy='disable' name='te'/>
        <feature policy='require' name='cmm'/>
        <feature policy='require' name='vxpdeh2'/>
    </cpu>

An example of the hypervisor-cpu-baseline command which will now
invoke a CPU model expansion with the host-recommended parameters:

    # virsh hypervisor-cpu-baseline host-model.xml 
    <cpu mode='custom' match='exact'>
      <model fallback='forbid'>z14.2-base</model>
      <feature policy='require' name='aen'/>
      <feature policy='require' name='cmmnt'/>
      <feature policy='require' name='aefsi'/>
      <feature policy='require' name='diag318'/>
      <feature policy='disable' name='csske'/>
      <feature policy='require' name='mepoch'/>
      <feature policy='require' name='msa8'/>
      <feature policy='require' name='msa7'/>
      <feature policy='require' name='msa6'/>
      <feature policy='require' name='msa5'/>
      <feature policy='require' name='msa4'/>
      <feature policy='require' name='msa3'/>
      <feature policy='require' name='msa2'/>
      <feature policy='require' name='msa1'/>
      <feature policy='require' name='sthyi'/>
      <feature policy='require' name='edat'/>
      <feature policy='require' name='ri'/>
      <feature policy='require' name='edat2'/>
      <feature policy='require' name='etoken'/>
      <feature policy='require' name='vx'/>
      <feature policy='require' name='ipter'/>
      <feature policy='require' name='mepochptff'/>
      <feature policy='require' name='ap'/>
      <feature policy='require' name='vxeh'/>
      <feature policy='require' name='vxpd'/>
      <feature policy='require' name='esop'/>
      <feature policy='require' name='apqi'/>
      <feature policy='require' name='apft'/>
      <feature policy='require' name='els'/>
      <feature policy='require' name='iep'/>
      <feature policy='require' name='apqci'/>
      <feature policy='disable' name='cte'/>
      <feature policy='require' name='ais'/>
      <feature policy='disable' name='bpb'/>
      <feature policy='require' name='gs'/>
      <feature policy='require' name='ppa15'/>
      <feature policy='require' name='zpci'/>
      <feature policy='require' name='sea_esop2'/>
      <feature policy='disable' name='te'/>
      <feature policy='require' name='cmm'/>
    </cpu>

Collin Walling (9):
  qemu: monitor: enable query for static-recommended CPU model
  qemu: capabilities: add static-recommended capability
  qemu: capabilities: refactor virQEMUCapsLoadHostCPUModelInfo
  qemu: capabilities: query and cache host-recommended CPU model
  cpu: conf: add cpu mode for host-recommended
  domain: capabilities: report host-recommended CPU model
  qemu: process: allow guest to use host-recommended CPU model
  qemu_driver: report feature policy when baselining
  qemu_driver: expand baselined model to static_recommended for s390x

 src/conf/cpu_conf.c                           |   1 +
 src/conf/cpu_conf.h                           |   1 +
 src/conf/domain_capabilities.c                |  16 ++
 src/conf/domain_capabilities.h                |   1 +
 src/conf/schemas/cputypes.rng                 |   1 +
 src/conf/schemas/domaincaps.rng               |  24 +++
 src/cpu/cpu.c                                 |   2 +
 src/cpu/cpu_ppc64.c                           |   2 +
 src/qemu/qemu_capabilities.c                  | 144 +++++++++++++++---
 src/qemu/qemu_capabilities.h                  |   4 +
 src/qemu/qemu_capspriv.h                      |   6 +-
 src/qemu/qemu_command.c                       |   5 +
 src/qemu/qemu_domain.c                        |   2 +
 src/qemu/qemu_driver.c                        |  18 ++-
 src/qemu/qemu_monitor.h                       |   1 +
 src/qemu/qemu_monitor_json.c                  |   4 +-
 src/qemu/qemu_process.c                       |   9 +-
 src/qemu/qemu_validate.c                      |   8 +
 tests/cputest.c                               |   4 +-
 tests/domaincapsdata/qemu_8.1.0.s390x.xml     |  55 +++++++
 .../caps_8.1.0_s390x.replies                  |  80 +++++++++-
 .../qemucapabilitiesdata/caps_8.1.0_s390x.xml |  55 +++++++
 ...c-cpu-kvm-ccw-virtio-8.1.s390x-latest.args |  32 ++++
 .../s390-host-rec-cpu-kvm-ccw-virtio-8.1.xml  |  17 +++
 tests/qemuxml2argvtest.c                      |   2 +
 ...ec-cpu-kvm-ccw-virtio-8.1.s390x-latest.xml |  25 +++
 tests/qemuxml2xmltest.c                       |   1 +
 27 files changed, 486 insertions(+), 34 deletions(-)
 create mode 100644 tests/qemuxml2argvdata/s390-host-rec-cpu-kvm-ccw-virtio-8.1.s390x-latest.args
 create mode 100644 tests/qemuxml2argvdata/s390-host-rec-cpu-kvm-ccw-virtio-8.1.xml
 create mode 100644 tests/qemuxml2xmloutdata/s390-host-rec-cpu-kvm-ccw-virtio-8.1.s390x-latest.xml

-- 
2.41.0
Re: [PATCH v1 0/9] query & cache host-recommended CPU model
Posted by Peter Krempa 7 months, 2 weeks ago
On Mon, Sep 11, 2023 at 17:07:07 -0400, Collin Walling wrote:
> Notes:
> https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg02518.html
>  - For information regarding the QEMU "static-recommended" (aka host-recommended)
>    CPU model, please see the analogous QEMU patches: 
> 
>      https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg02518.html
> 
>  - Patches include caps added to QEMU 8.1 test files. This is a stand-in for the
>    interim and the appropriate caps files will be updated in the future pending 
>    QEMU acceptance.

I went looking for the feature in qemu and couldn't find it, but you
explain it here.

Note that it's acceptable only as a RFC to gather feedback, but must not
be comitted in this form. The capability dumps must represent real qemu
capabilities, and thus hacks where you "backport" something are not
acceptable.

Once the qemu patches get commited upstream, it's okay if you create a
capability dump from the in-progress development cycle. We do that also
for x86_64.
Re: [PATCH v1 0/9] query & cache host-recommended CPU model
Posted by Collin Walling 7 months, 2 weeks ago
On 9/12/23 03:58, Peter Krempa wrote:
> On Mon, Sep 11, 2023 at 17:07:07 -0400, Collin Walling wrote:
>> Notes:
>> https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg02518.html
>>  - For information regarding the QEMU "static-recommended" (aka host-recommended)
>>    CPU model, please see the analogous QEMU patches: 
>>
>>      https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg02518.html
>>
>>  - Patches include caps added to QEMU 8.1 test files. This is a stand-in for the
>>    interim and the appropriate caps files will be updated in the future pending 
>>    QEMU acceptance.
> 
> I went looking for the feature in qemu and couldn't find it, but you
> explain it here.
> 
> Note that it's acceptable only as a RFC to gather feedback, but must not
> be comitted in this form. The capability dumps must represent real qemu
> capabilities, and thus hacks where you "backport" something are not
> acceptable.
> 
> Once the qemu patches get commited upstream, it's okay if you create a
> capability dump from the in-progress development cycle. We do that also
> for x86_64.
> 

Absolutely makes sense. I agree, should have posted as an RFC. Will keep
this noted for the future.

-- 
Regards,
  Collin