New CPU related APIs

[libvirt] [PATCH 00/22] New CPU related APIs

Posted by Jiri Denemark 5 years, 11 months ago

The current virConnectCompareCPU and virConnectBaselineCPU APIs are not
very useful because they ignore what a hypervisor can do on the current
host. This series adds two new APIs which are designed to work with
capabilities of a specific hypervisor to provide usable results.

The third CPU related API virConnectGetCPUModelNames is pretty useless
too, but no new API with similar functionality is needed because domain
capabilities XML already contains the relevant data.

Jiri Denemark (22):
  virsh: Move cpu-{baseline,compare} commands
  virsh: Extract common code from cmdCPU{Compare,Baseline}
  virsh: Enhance documentation of cpu-compare command
  virsh: Enhance documentation of cpu-models command
  Improve documentation of virConnectGetCPUModelNames
  vshExtractCPUDefXML: Accept domain capabilities XML
  qemu_capabilities: Introduce virQEMUCapsCacheLookupDefault
  Introduce virConnectCompareHypervisorCPU public API
  remote: Implement virConnectCompareHypervisorCPU
  virsh: Introduce new hypervisor-cpu-compare command
  qemu: Implement virConnectCompareHypervisorCPU
  Introduce virConnectBaselineHypervisorCPU public API
  remote: Implement virConnectBaselineHypervisorCPU
  virsh: Introduce new hypervisor-cpu-baseline command
  cpu: Rename cpuBaseline as virCPUBaseline
  cpu_x86: Add support for passing guest CPUs to virCPUx86Baseline
  cpu: Add explicit arch parameter for virCPUBaseline
  cpu: Update style in virCPUBaseline
  cpu: Add optional list of allowed features to virCPUBaseline
  qemu_capabilities: Introduce virQEMUCapsGetCPUFeatures
  qemu: Implement virConnectBaselineHypervisorCPU
  news: Mention new CPU related APIs

 docs/news.xml                  |   9 +
 include/libvirt/libvirt-host.h |  15 ++
 src/bhyve/bhyve_driver.c       |   4 +-
 src/cpu/cpu.c                  |  34 ++-
 src/cpu/cpu.h                  |  21 +-
 src/cpu/cpu_arm.c              |  11 +-
 src/cpu/cpu_ppc64.c            |   5 +-
 src/cpu/cpu_x86.c              |  35 ++-
 src/driver-hypervisor.h        |  20 ++
 src/libvirt-host.c             | 161 +++++++++++-
 src/libvirt_private.syms       |   2 +-
 src/libvirt_public.syms        |   6 +
 src/libxl/libxl_driver.c       |   4 +-
 src/qemu/qemu_capabilities.c   | 170 +++++++++++++
 src/qemu/qemu_capabilities.h   |  12 +
 src/qemu/qemu_driver.c         | 241 ++++++++++++------
 src/remote/remote_driver.c     |   4 +-
 src/remote/remote_protocol.x   |  40 ++-
 src/remote_protocol-structs    |  27 ++
 src/test/test_driver.c         |   2 +-
 src/vz/vz_driver.c             |   2 +-
 tests/cputest.c                |   6 +-
 tools/virsh-domain.c           | 223 -----------------
 tools/virsh-host.c             | 441 +++++++++++++++++++++++++++++++++
 tools/virsh.pod                |  90 ++++++-
 25 files changed, 1223 insertions(+), 362 deletions(-)

-- 
2.17.0

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 00/22] New CPU related APIs

Posted by Jiri Denemark 5 years, 11 months ago

On Wed, May 16, 2018 at 10:39:19 +0200, Jiri Denemark wrote:
> The current virConnectCompareCPU and virConnectBaselineCPU APIs are not
> very useful because they ignore what a hypervisor can do on the current
> host. This series adds two new APIs which are designed to work with
> capabilities of a specific hypervisor to provide usable results.
> 
> The third CPU related API virConnectGetCPUModelNames is pretty useless
> too, but no new API with similar functionality is needed because domain
> capabilities XML already contains the relevant data.

Any comments? I'd like to have these new APIs in the upcoming 4.4
release.

Jirka

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 00/22] New CPU related APIs

Posted by Collin Walling 5 years, 11 months ago

On 05/22/2018 05:33 AM, Jiri Denemark wrote:
> On Wed, May 16, 2018 at 10:39:19 +0200, Jiri Denemark wrote:
>> The current virConnectCompareCPU and virConnectBaselineCPU APIs are not
>> very useful because they ignore what a hypervisor can do on the current
>> host. This series adds two new APIs which are designed to work with
>> capabilities of a specific hypervisor to provide usable results.
>>
>> The third CPU related API virConnectGetCPUModelNames is pretty useless
>> too, but no new API with similar functionality is needed because domain
>> capabilities XML already contains the relevant data.
> 
> Any comments? I'd like to have these new APIs in the upcoming 4.4
> release.
> 
> Jirka
> 
> --
> libvir-list mailing list
> libvir-list@redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list
> 

This is on top of my todo for today.

-- 
Respectfully,
- Collin Walling

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 00/22] New CPU related APIs

Posted by Jiri Denemark 5 years, 10 months ago

On Wed, May 16, 2018 at 10:39:19 +0200, Jiri Denemark wrote:
> The current virConnectCompareCPU and virConnectBaselineCPU APIs are not
> very useful because they ignore what a hypervisor can do on the current
> host. This series adds two new APIs which are designed to work with
> capabilities of a specific hypervisor to provide usable results.
> 
> The third CPU related API virConnectGetCPUModelNames is pretty useless
> too, but no new API with similar functionality is needed because domain
> capabilities XML already contains the relevant data.

I made the suggested changes and pushed the series. Thanks for the
reviews.

Jirka

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 00/22] New CPU related APIs

Posted by Chris Venteicher 5 years, 10 months ago

Quoting Jiri Denemark (2018-05-28 09:19:51)
> On Wed, May 16, 2018 at 10:39:19 +0200, Jiri Denemark wrote:
> > The current virConnectCompareCPU and virConnectBaselineCPU APIs are not
> > very useful because they ignore what a hypervisor can do on the current
> > host. This series adds two new APIs which are designed to work with
> > capabilities of a specific hypervisor to provide usable results.
> > 
> > The third CPU related API virConnectGetCPUModelNames is pretty useless
> > too, but no new API with similar functionality is needed because domain
> > capabilities XML already contains the relevant data.
> 
> I made the suggested changes and pushed the series. Thanks for the
> reviews.
> 
> Jirka

Hi Jirka,

FYI I reviewed your patches too and everthing I found that was not already 
identified was nitpick stuff but I do have a something I am wondering about...

The new hypervisor specific compare and baseline commands seem to depend on 
qemuCaps being pre-populated with model data that is specific to a hypervisor 
instance.

How do we make sure the qemuCaps are pre-populated with cpu model data for any 
arbitrary hypervisor (with a different exec path, arch, etc) that we can issue 
the hypervisor compare or baseline commands against?

Is it a case of executing a virsh command to populate qemuCaps for a non-default 
hypervisor prior to issuing the hypervisor compare or baseline commands?

Sorry if a complete noob question or I missed the answer in previous mails.

Chris

> 
> --
> libvir-list mailing list
> libvir-list@redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 00/22] New CPU related APIs

Posted by Collin Walling 5 years, 10 months ago

On 05/28/2018 03:44 PM, Chris Venteicher wrote:
> Quoting Jiri Denemark (2018-05-28 09:19:51)
>> On Wed, May 16, 2018 at 10:39:19 +0200, Jiri Denemark wrote:
>>> The current virConnectCompareCPU and virConnectBaselineCPU APIs are not
>>> very useful because they ignore what a hypervisor can do on the current
>>> host. This series adds two new APIs which are designed to work with
>>> capabilities of a specific hypervisor to provide usable results.
>>>
>>> The third CPU related API virConnectGetCPUModelNames is pretty useless
>>> too, but no new API with similar functionality is needed because domain
>>> capabilities XML already contains the relevant data.
>>
>> I made the suggested changes and pushed the series. Thanks for the
>> reviews.
>>
>> Jirka
> 
> Hi Jirka,
> 
> FYI I reviewed your patches too and everthing I found that was not already 
> identified was nitpick stuff but I do have a something I am wondering about...
> 
> The new hypervisor specific compare and baseline commands seem to depend on 
> qemuCaps being pre-populated with model data that is specific to a hypervisor 
> instance.
> 
> How do we make sure the qemuCaps are pre-populated with cpu model data for any 
> arbitrary hypervisor (with a different exec path, arch, etc) that we can issue 
> the hypervisor compare or baseline commands against?
> 
> Is it a case of executing a virsh command to populate qemuCaps for a non-default 
> hypervisor prior to issuing the hypervisor compare or baseline commands?
> 
> Sorry if a complete noob question or I missed the answer in previous mails.
> 
> Chris
> 

The qemuCaps are generated for each qemu binary upon startup of libvirt and are stored 
in /var/cache/libvirt/qemu/capabilities/

If you take a look at virQEMUCapsInitQMPMonitor in file qemu_capabilities, you'll see a 
bunch of functions with monitor calls that will probe qemu for its capabilities.

Since there can be a lot of different qemu caps that can exist on one system (due to different 
archs, different virttypes, etc), Jiri adds the ability to add parameters to specify which 
qemu capabilities file we want to pull data from.


>>
>> --
>> libvir-list mailing list
>> libvir-list@redhat.com
>> https://www.redhat.com/mailman/listinfo/libvir-list
> 
> --
> libvir-list mailing list
> libvir-list@redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list
> 


-- 
Respectfully,
- Collin Walling

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 00/22] New CPU related APIs

Posted by Jiri Denemark 5 years, 10 months ago

Hi Chris,

> The new hypervisor specific compare and baseline commands seem to depend on 
> qemuCaps being pre-populated with model data that is specific to a hypervisor 
> instance.
> 
> How do we make sure the qemuCaps are pre-populated with cpu model data for any 
> arbitrary hypervisor (with a different exec path, arch, etc) that we can issue 
> the hypervisor compare or baseline commands against?

The cache lookup functions automatically generate qemuCaps if they don't
exist (i.e., someone asked for an emulator binary which is not known to
libvirt yet) or if they need to be refreshed (e.g., the QEMU binary
changed in the meantime). It's a bit hidden behind the generic
virFileCache object which uses the following callbacks to handle QEMU
caps cache:

    virFileCacheHandlers qemuCapsCacheHandlers = {
        .isValid = virQEMUCapsIsValid,
        .newData = virQEMUCapsNewData,
        .loadFile = virQEMUCapsLoadFile,
        .saveFile = virQEMUCapsSaveFile,
        .privFree = virQEMUCapsCachePrivFree,
    };

Jirka

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 00/22] New CPU related APIs

Posted by Chris Venteicher 5 years, 10 months ago

Quoting Jiri Denemark (2018-05-29 09:34:02)
> Hi Chris,
> 
> > The new hypervisor specific compare and baseline commands seem to depend on 
> > qemuCaps being pre-populated with model data that is specific to a hypervisor 
> > instance.
> > 
> > How do we make sure the qemuCaps are pre-populated with cpu model data for any 
> > arbitrary hypervisor (with a different exec path, arch, etc) that we can issue 
> > the hypervisor compare or baseline commands against?
> 
> The cache lookup functions automatically generate qemuCaps if they don't
> exist (i.e., someone asked for an emulator binary which is not known to
> libvirt yet) or if they need to be refreshed (e.g., the QEMU binary
> changed in the meantime). It's a bit hidden behind the generic
> virFileCache object which uses the following callbacks to handle QEMU
> caps cache:
> 
>     virFileCacheHandlers qemuCapsCacheHandlers = {
>         .isValid = virQEMUCapsIsValid,
>         .newData = virQEMUCapsNewData,
>         .loadFile = virQEMUCapsLoadFile,
>         .saveFile = virQEMUCapsSaveFile,
>         .privFree = virQEMUCapsCachePrivFree,
>     };
> 
> Jirka

Got it.  Read through cache of sorts.  Thanks Jirka and Collin.

Chris.


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 00/22] New CPU related APIs

Posted by Collin Walling 5 years, 10 months ago

Code-wise, these patches look good. I'd like to take the time next week
to hook up my comparison patches and test them out properly (unless Chris 
gets to it first, then I'll just apply his patches and run some tests)

On 05/16/2018 04:39 AM, Jiri Denemark wrote:
> The current virConnectCompareCPU and virConnectBaselineCPU APIs are not
> very useful because they ignore what a hypervisor can do on the current
> host. This series adds two new APIs which are designed to work with
> capabilities of a specific hypervisor to provide usable results.
> 
> The third CPU related API virConnectGetCPUModelNames is pretty useless
> too, but no new API with similar functionality is needed because domain
> capabilities XML already contains the relevant data.


-- 
Respectfully,
- Collin Walling

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list