KVM supports the ability to publish the amount of time that VCPUs
were runnable, but not running due to other host threads running
instead, to the guest. The guest scheduler may use that information
when making decisions and the guest may expose it to its userspace
(Linux publishes this information in /proc/stat). This feature is
called "steal time" as it represents the amount of time stolen from
a guest by scheduling out its VCPUs. To enable this feature KVM
userspace must provide a memory region that will be used to publish
the information to the guest. The memory region is typical migratable
region. The GPA of the region is given to KVM through a VCPU device
ioctl interface. This feature is only available for 64-bit hosts
running 64-bit guests.

This series provides the QEMU support of this feature. It will
be enabled by default for 5.1 machine types and later, but may
be disabled with a new CPU property "kvm-steal-time".

While testing migration it was observed that the amount of
steal time as viewed by the guest was getting reset on each
migration. Patch 3/5 of a pvtime fix series posted[*] for KVM
should fix that.

Migration testing:
* virt-5.0 can migrate as usual, no kvm-steal-time enabled
* virt-5.1 can migrate between hosts with steal-time enabled and
  disabled when both hosts support steal-time
* virt-5.1 with steal-time disabled can migrate to a host that
  does not support steal-time
* virt-5.1 with steal-time enabled will cleanly fail when migrating
  to a host that does not support steal-time

[*] https://lists.cs.columbia.edu/pipermail/kvmarm/2020-July/041538.html

Thanks,
drew


Andrew Jones (3):
  hw/arm/virt: Move post cpu realize check into its own function
  hw/arm/virt: Move kvm pmu setup to virt_cpu_post_init
  hw/arm/virt: Implement kvm-steal-time

 docs/system/arm/cpu-features.rst |  11 ++++
 hw/arm/virt.c                    | 103 +++++++++++++++++++++----------
 include/hw/arm/virt.h            |   2 +
 target/arm/cpu.c                 |  10 +++
 target/arm/cpu.h                 |   4 ++
 target/arm/kvm.c                 |  20 ++++++
 target/arm/kvm32.c               |   5 ++
 target/arm/kvm64.c               |  70 +++++++++++++++++++--
 target/arm/kvm_arm.h             |  24 ++++++-
 target/arm/monitor.c             |   2 +-
 tests/qtest/arm-cpu-features.c   |  25 ++++++--
 11 files changed, 232 insertions(+), 44 deletions(-)

-- 
2.25.4

On Sat, 11 Jul 2020 at 11:10, Andrew Jones <drjones@redhat.com> wrote:
>
> KVM supports the ability to publish the amount of time that VCPUs
> were runnable, but not running due to other host threads running
> instead, to the guest. The guest scheduler may use that information
> when making decisions and the guest may expose it to its userspace
> (Linux publishes this information in /proc/stat). This feature is
> called "steal time" as it represents the amount of time stolen from
> a guest by scheduling out its VCPUs. To enable this feature KVM
> userspace must provide a memory region that will be used to publish
> the information to the guest. The memory region is typical migratable
> region. The GPA of the region is given to KVM through a VCPU device
> ioctl interface. This feature is only available for 64-bit hosts
> running 64-bit guests.
>
> This series provides the QEMU support of this feature. It will
> be enabled by default for 5.1 machine types and later, but may
> be disabled with a new CPU property "kvm-steal-time".

Just a note that this has missed 5.1, but I'll review it at
some point for 5.2.

thanks
-- PMM

On Mon, Jul 20, 2020 at 11:16:41AM +0100, Peter Maydell wrote:
> On Sat, 11 Jul 2020 at 11:10, Andrew Jones <drjones@redhat.com> wrote:
> >
> > KVM supports the ability to publish the amount of time that VCPUs
> > were runnable, but not running due to other host threads running
> > instead, to the guest. The guest scheduler may use that information
> > when making decisions and the guest may expose it to its userspace
> > (Linux publishes this information in /proc/stat). This feature is
> > called "steal time" as it represents the amount of time stolen from
> > a guest by scheduling out its VCPUs. To enable this feature KVM
> > userspace must provide a memory region that will be used to publish
> > the information to the guest. The memory region is typical migratable
> > region. The GPA of the region is given to KVM through a VCPU device
> > ioctl interface. This feature is only available for 64-bit hosts
> > running 64-bit guests.
> >
> > This series provides the QEMU support of this feature. It will
> > be enabled by default for 5.1 machine types and later, but may
> > be disabled with a new CPU property "kvm-steal-time".
> 
> Just a note that this has missed 5.1, but I'll review it at
> some point for 5.2.
>

Yup. No problem. I was out on vacation all last week, so there wasn't
a chance for me to follow up until now anyway. Thanks for the review.
I'll get to the responses / respinning tomorrow after I dig out of my
email backlog.

drew