Enable interpretive execution of zPCI instructions + adapter interruption
forwarding for s390x KVM vfio-pci.  This is done by triggering a routine
when the VFIO group is associated with the KVM guest, transmitting to
firmware a special token (GISA designation) to enable that specific guest
for interpretive execution on that zPCI device.  Load/store interpreation
enablement is then controlled by userspace (based upon whether or not a
SHM bit is placed in the virtual function handle).  Adapter Event
Notification interpretation is controlled from userspace via a new KVM
ioctl.

By allowing intepretation of zPCI instructions and firmware delivery of
interrupts to guests, we can reduce the frequency of guest SIE exits for
zPCI.  

From the perspective of guest configuration, you passthrough zPCI devices
in the same manner as before, with intepretation support being used by
default if available in kernel+qemu.

Will follow up with a link the most recent QEMU series.

Changelog v8->v9:
- Rebase on top of 5.19-rc1, adjust ioctl and capability defines
- s/kzdev = 0/kzdev = NULL/ (Alex)
- rename vfio_pci_zdev_open to vfio_pci_zdev_open_device (Jason)
- rename vfio_pci_zdev_release to vfio_pci_zdev_close_device (Jason)
- make vfio_pci_zdev_close_device return void, instead WARN_ON or ignore
  errors in lower level function (kvm_s390_pci_unregister_kvm) (Jason)
- remove notifier accidentally left in struct zpci_dev + associated
  include statment (Jason)
- Remove patch 'KVM: s390: introduce CPU feature for zPCI Interpretation'
  based on discussion in QEMU thread.

Matthew Rosato (21):
  s390/sclp: detect the zPCI load/store interpretation facility
  s390/sclp: detect the AISII facility
  s390/sclp: detect the AENI facility
  s390/sclp: detect the AISI facility
  s390/airq: pass more TPI info to airq handlers
  s390/airq: allow for airq structure that uses an input vector
  s390/pci: externalize the SIC operation controls and routine
  s390/pci: stash associated GISA designation
  s390/pci: stash dtsm and maxstbl
  vfio/pci: introduce CONFIG_VFIO_PCI_ZDEV_KVM
  KVM: s390: pci: add basic kvm_zdev structure
  KVM: s390: pci: do initial setup for AEN interpretation
  KVM: s390: pci: enable host forwarding of Adapter Event Notifications
  KVM: s390: mechanism to enable guest zPCI Interpretation
  KVM: s390: pci: provide routines for enabling/disabling interrupt
    forwarding
  KVM: s390: pci: add routines to start/stop interpretive execution
  vfio-pci/zdev: add open/close device hooks
  vfio-pci/zdev: add function handle to clp base capability
  vfio-pci/zdev: different maxstbl for interpreted devices
  KVM: s390: add KVM_S390_ZPCI_OP to manage guest zPCI devices
  MAINTAINERS: additional files related kvm s390 pci passthrough

 Documentation/virt/kvm/api.rst   |  47 +++
 MAINTAINERS                      |   1 +
 arch/s390/include/asm/airq.h     |   7 +-
 arch/s390/include/asm/kvm_host.h |  23 ++
 arch/s390/include/asm/pci.h      |  11 +
 arch/s390/include/asm/pci_clp.h  |   9 +-
 arch/s390/include/asm/pci_insn.h |  29 +-
 arch/s390/include/asm/sclp.h     |   4 +
 arch/s390/include/asm/tpi.h      |  13 +
 arch/s390/kvm/Makefile           |   1 +
 arch/s390/kvm/interrupt.c        |  96 ++++-
 arch/s390/kvm/kvm-s390.c         |  83 +++-
 arch/s390/kvm/kvm-s390.h         |  10 +
 arch/s390/kvm/pci.c              | 690 +++++++++++++++++++++++++++++++
 arch/s390/kvm/pci.h              |  88 ++++
 arch/s390/pci/pci.c              |  16 +
 arch/s390/pci/pci_clp.c          |   7 +
 arch/s390/pci/pci_insn.c         |   4 +-
 arch/s390/pci/pci_irq.c          |  48 ++-
 drivers/s390/char/sclp_early.c   |   4 +
 drivers/s390/cio/airq.c          |  12 +-
 drivers/s390/cio/qdio_thinint.c  |   6 +-
 drivers/s390/crypto/ap_bus.c     |   9 +-
 drivers/s390/virtio/virtio_ccw.c |   6 +-
 drivers/vfio/pci/Kconfig         |  11 +
 drivers/vfio/pci/Makefile        |   2 +-
 drivers/vfio/pci/vfio_pci_core.c |  10 +-
 drivers/vfio/pci/vfio_pci_zdev.c |  35 +-
 include/linux/sched/user.h       |   3 +-
 include/linux/vfio_pci_core.h    |  12 +-
 include/uapi/linux/kvm.h         |  31 ++
 include/uapi/linux/vfio_zdev.h   |   7 +
 32 files changed, 1279 insertions(+), 56 deletions(-)
 create mode 100644 arch/s390/kvm/pci.c
 create mode 100644 arch/s390/kvm/pci.h

-- 
2.27.0

On 6/6/22 4:33 PM, Matthew Rosato wrote:
> Enable interpretive execution of zPCI instructions + adapter interruption
> forwarding for s390x KVM vfio-pci.  This is done by triggering a routine
> when the VFIO group is associated with the KVM guest, transmitting to
> firmware a special token (GISA designation) to enable that specific guest
> for interpretive execution on that zPCI device.  Load/store interpreation
> enablement is then controlled by userspace (based upon whether or not a
> SHM bit is placed in the virtual function handle).  Adapter Event
> Notification interpretation is controlled from userspace via a new KVM
> ioctl.
> 
> By allowing intepretation of zPCI instructions and firmware delivery of
> interrupts to guests, we can reduce the frequency of guest SIE exits for
> zPCI.
> 
>  From the perspective of guest configuration, you passthrough zPCI devices
> in the same manner as before, with intepretation support being used by
> default if available in kernel+qemu.
> 
> Will follow up with a link the most recent QEMU series.
> 
> Changelog v8->v9:
> - Rebase on top of 5.19-rc1, adjust ioctl and capability defines
> - s/kzdev = 0/kzdev = NULL/ (Alex)
> - rename vfio_pci_zdev_open to vfio_pci_zdev_open_device (Jason)
> - rename vfio_pci_zdev_release to vfio_pci_zdev_close_device (Jason)
> - make vfio_pci_zdev_close_device return void, instead WARN_ON or ignore
>    errors in lower level function (kvm_s390_pci_unregister_kvm) (Jason)
> - remove notifier accidentally left in struct zpci_dev + associated
>    include statment (Jason)
> - Remove patch 'KVM: s390: introduce CPU feature for zPCI Interpretation'
>    based on discussion in QEMU thread.
> 

Ping -- I'm hoping this can make the next merge window, but there are 
still 2 patches left without any review tag (16 & 17).

Am 27.06.22 um 22:57 schrieb Matthew Rosato:
> On 6/6/22 4:33 PM, Matthew Rosato wrote:
>> Enable interpretive execution of zPCI instructions + adapter interruption
>> forwarding for s390x KVM vfio-pci.  This is done by triggering a routine
>> when the VFIO group is associated with the KVM guest, transmitting to
>> firmware a special token (GISA designation) to enable that specific guest
>> for interpretive execution on that zPCI device.  Load/store interpreation
>> enablement is then controlled by userspace (based upon whether or not a
>> SHM bit is placed in the virtual function handle).  Adapter Event
>> Notification interpretation is controlled from userspace via a new KVM
>> ioctl.
>>
>> By allowing intepretation of zPCI instructions and firmware delivery of
>> interrupts to guests, we can reduce the frequency of guest SIE exits for
>> zPCI.
>>
>>  From the perspective of guest configuration, you passthrough zPCI devices
>> in the same manner as before, with intepretation support being used by
>> default if available in kernel+qemu.
>>
>> Will follow up with a link the most recent QEMU series.
>>
>> Changelog v8->v9:
>> - Rebase on top of 5.19-rc1, adjust ioctl and capability defines
>> - s/kzdev = 0/kzdev = NULL/ (Alex)
>> - rename vfio_pci_zdev_open to vfio_pci_zdev_open_device (Jason)
>> - rename vfio_pci_zdev_release to vfio_pci_zdev_close_device (Jason)
>> - make vfio_pci_zdev_close_device return void, instead WARN_ON or ignore
>>    errors in lower level function (kvm_s390_pci_unregister_kvm) (Jason)
>> - remove notifier accidentally left in struct zpci_dev + associated
>>    include statment (Jason)
>> - Remove patch 'KVM: s390: introduce CPU feature for zPCI Interpretation'
>>    based on discussion in QEMU thread.
>>
> 
> Ping -- I'm hoping this can make the next merge window, but there are still 2 patches left without any review tag (16 & 17).

Yes, I will queue this (as is). Ideally you would rebase this on top of kvm/next but I can also do while applying.
Let me know if you want to respin with the Nits from Pierre.

On 6/28/22 8:35 AM, Christian Borntraeger wrote:
> Am 27.06.22 um 22:57 schrieb Matthew Rosato:
>> On 6/6/22 4:33 PM, Matthew Rosato wrote:
>>> Enable interpretive execution of zPCI instructions + adapter 
>>> interruption
>>> forwarding for s390x KVM vfio-pci.  This is done by triggering a routine
>>> when the VFIO group is associated with the KVM guest, transmitting to
>>> firmware a special token (GISA designation) to enable that specific 
>>> guest
>>> for interpretive execution on that zPCI device.  Load/store 
>>> interpreation
>>> enablement is then controlled by userspace (based upon whether or not a
>>> SHM bit is placed in the virtual function handle).  Adapter Event
>>> Notification interpretation is controlled from userspace via a new KVM
>>> ioctl.
>>>
>>> By allowing intepretation of zPCI instructions and firmware delivery of
>>> interrupts to guests, we can reduce the frequency of guest SIE exits for
>>> zPCI.
>>>
>>>  From the perspective of guest configuration, you passthrough zPCI 
>>> devices
>>> in the same manner as before, with intepretation support being used by
>>> default if available in kernel+qemu.
>>>
>>> Will follow up with a link the most recent QEMU series.
>>>
>>> Changelog v8->v9:
>>> - Rebase on top of 5.19-rc1, adjust ioctl and capability defines
>>> - s/kzdev = 0/kzdev = NULL/ (Alex)
>>> - rename vfio_pci_zdev_open to vfio_pci_zdev_open_device (Jason)
>>> - rename vfio_pci_zdev_release to vfio_pci_zdev_close_device (Jason)
>>> - make vfio_pci_zdev_close_device return void, instead WARN_ON or ignore
>>>    errors in lower level function (kvm_s390_pci_unregister_kvm) (Jason)
>>> - remove notifier accidentally left in struct zpci_dev + associated
>>>    include statment (Jason)
>>> - Remove patch 'KVM: s390: introduce CPU feature for zPCI 
>>> Interpretation'
>>>    based on discussion in QEMU thread.
>>>
>>
>> Ping -- I'm hoping this can make the next merge window, but there are 
>> still 2 patches left without any review tag (16 & 17).
> 
> Yes, I will queue this (as is). Ideally you would rebase this on top of 
> kvm/next but I can also do while applying.
> Let me know if you want to respin with the Nits from Pierre.

Ah, sorry -- I assume you mean Paolo's kvm/next?  I tried now and see 
some conflicts with the ioctl patch.

Why don't I rebase on top of kvm/next along with these couple of changes 
from Pierre and send this as a v10 for you to queue.

While at it, there's one other issue to be aware of -- There will also 
be small merge conflicts with a patch that just hit vfio-next, "vfio: 
de-extern-ify function prototypes" - My series already avoids adding 
externs to new prototypes, but adjacent code changes will cause a 
conflict with patches 10 and 17.

Not sure what the best way to proceed there is.

https://lore.kernel.org/kvm/165471414407.203056.474032786990662279.stgit@omen/

Am 28.06.22 um 15:40 schrieb Matthew Rosato:
> On 6/28/22 8:35 AM, Christian Borntraeger wrote:
>> Am 27.06.22 um 22:57 schrieb Matthew Rosato:
>>> On 6/6/22 4:33 PM, Matthew Rosato wrote:
>>>> Enable interpretive execution of zPCI instructions + adapter interruption
>>>> forwarding for s390x KVM vfio-pci.  This is done by triggering a routine
>>>> when the VFIO group is associated with the KVM guest, transmitting to
>>>> firmware a special token (GISA designation) to enable that specific guest
>>>> for interpretive execution on that zPCI device.  Load/store interpreation
>>>> enablement is then controlled by userspace (based upon whether or not a
>>>> SHM bit is placed in the virtual function handle).  Adapter Event
>>>> Notification interpretation is controlled from userspace via a new KVM
>>>> ioctl.
>>>>
>>>> By allowing intepretation of zPCI instructions and firmware delivery of
>>>> interrupts to guests, we can reduce the frequency of guest SIE exits for
>>>> zPCI.
>>>>
>>>>  From the perspective of guest configuration, you passthrough zPCI devices
>>>> in the same manner as before, with intepretation support being used by
>>>> default if available in kernel+qemu.
>>>>
>>>> Will follow up with a link the most recent QEMU series.
>>>>
>>>> Changelog v8->v9:
>>>> - Rebase on top of 5.19-rc1, adjust ioctl and capability defines
>>>> - s/kzdev = 0/kzdev = NULL/ (Alex)
>>>> - rename vfio_pci_zdev_open to vfio_pci_zdev_open_device (Jason)
>>>> - rename vfio_pci_zdev_release to vfio_pci_zdev_close_device (Jason)
>>>> - make vfio_pci_zdev_close_device return void, instead WARN_ON or ignore
>>>>    errors in lower level function (kvm_s390_pci_unregister_kvm) (Jason)
>>>> - remove notifier accidentally left in struct zpci_dev + associated
>>>>    include statment (Jason)
>>>> - Remove patch 'KVM: s390: introduce CPU feature for zPCI Interpretation'
>>>>    based on discussion in QEMU thread.
>>>>
>>>
>>> Ping -- I'm hoping this can make the next merge window, but there are still 2 patches left without any review tag (16 & 17).
>>
>> Yes, I will queue this (as is). Ideally you would rebase this on top of kvm/next but I can also do while applying.
>> Let me know if you want to respin with the Nits from Pierre.
> 
> Ah, sorry -- I assume you mean Paolo's kvm/next?  I tried now and see some conflicts with the ioctl patch.
> 
> Why don't I rebase on top of kvm/next along with these couple of changes from Pierre and send this as a v10 for you to queue.
> 
> While at it, there's one other issue to be aware of -- There will also be small merge conflicts with a patch that just hit vfio-next, "vfio: de-extern-ify function prototypes" - My series already avoids adding externs to new prototypes, but adjacent code changes will cause a conflict with patches 10 and 17.
> 
> Not sure what the best way to proceed there is.
> 
> https://lore.kernel.org/kvm/165471414407.203056.474032786990662279.stgit@omen/

I think Linus can sort out if the conflicts are trivial. As an alternative Alex could carry these patches, but then we have a merge conflict between him and KVM.
Alex/Paolo, shall I do a topic branch that you both can merge?

On Tue, Jun 28, 2022 at 09:40:01AM -0400, Matthew Rosato wrote:
> On 6/28/22 8:35 AM, Christian Borntraeger wrote:
> > Am 27.06.22 um 22:57 schrieb Matthew Rosato:
> > > On 6/6/22 4:33 PM, Matthew Rosato wrote:
> > > > Enable interpretive execution of zPCI instructions + adapter
> > > > interruption
> > > > forwarding for s390x KVM vfio-pci.  This is done by triggering a routine
> > > > when the VFIO group is associated with the KVM guest, transmitting to
> > > > firmware a special token (GISA designation) to enable that
> > > > specific guest
> > > > for interpretive execution on that zPCI device.  Load/store
> > > > interpreation
> > > > enablement is then controlled by userspace (based upon whether or not a
> > > > SHM bit is placed in the virtual function handle).  Adapter Event
> > > > Notification interpretation is controlled from userspace via a new KVM
> > > > ioctl.
> > > > 
> > > > By allowing intepretation of zPCI instructions and firmware delivery of
> > > > interrupts to guests, we can reduce the frequency of guest SIE exits for
> > > > zPCI.
> > > > 
> > > >  From the perspective of guest configuration, you passthrough
> > > > zPCI devices
> > > > in the same manner as before, with intepretation support being used by
> > > > default if available in kernel+qemu.
> > > > 
> > > > Will follow up with a link the most recent QEMU series.
> > > > 
> > > > Changelog v8->v9:
> > > > - Rebase on top of 5.19-rc1, adjust ioctl and capability defines
> > > > - s/kzdev = 0/kzdev = NULL/ (Alex)
> > > > - rename vfio_pci_zdev_open to vfio_pci_zdev_open_device (Jason)
> > > > - rename vfio_pci_zdev_release to vfio_pci_zdev_close_device (Jason)
> > > > - make vfio_pci_zdev_close_device return void, instead WARN_ON or ignore
> > > >    errors in lower level function (kvm_s390_pci_unregister_kvm) (Jason)
> > > > - remove notifier accidentally left in struct zpci_dev + associated
> > > >    include statment (Jason)
> > > > - Remove patch 'KVM: s390: introduce CPU feature for zPCI
> > > > Interpretation'
> > > >    based on discussion in QEMU thread.
> > > > 
> > > 
> > > Ping -- I'm hoping this can make the next merge window, but there
> > > are still 2 patches left without any review tag (16 & 17).
> > 
> > Yes, I will queue this (as is). Ideally you would rebase this on top of
> > kvm/next but I can also do while applying.
> > Let me know if you want to respin with the Nits from Pierre.
> 
> Ah, sorry -- I assume you mean Paolo's kvm/next?  I tried now and see some
> conflicts with the ioctl patch.
> 
> Why don't I rebase on top of kvm/next along with these couple of changes
> from Pierre and send this as a v10 for you to queue.
> 
> While at it, there's one other issue to be aware of -- There will also be
> small merge conflicts with a patch that just hit vfio-next, "vfio:
> de-extern-ify function prototypes" - My series already avoids adding externs
> to new prototypes, but adjacent code changes will cause a conflict with
> patches 10 and 17.
> 
> Not sure what the best way to proceed there is.

You should use a branch based on vfio-next and send a Git PR to Christian
and Alex

Jason

Am 06.06.22 um 22:33 schrieb Matthew Rosato:
> Enable interpretive execution of zPCI instructions + adapter interruption
> forwarding for s390x KVM vfio-pci.  This is done by triggering a routine
> when the VFIO group is associated with the KVM guest, transmitting to
> firmware a special token (GISA designation) to enable that specific guest
> for interpretive execution on that zPCI device.  Load/store interpreation
> enablement is then controlled by userspace (based upon whether or not a
> SHM bit is placed in the virtual function handle).  Adapter Event
> Notification interpretation is controlled from userspace via a new KVM
> ioctl.
> 
> By allowing intepretation of zPCI instructions and firmware delivery of
> interrupts to guests, we can reduce the frequency of guest SIE exits for
> zPCI.
> 
>  From the perspective of guest configuration, you passthrough zPCI devices
> in the same manner as before, with intepretation support being used by
> default if available in kernel+qemu.
> 
> Will follow up with a link the most recent QEMU series.
> 
> Changelog v8->v9:
> - Rebase on top of 5.19-rc1, adjust ioctl and capability defines
> - s/kzdev = 0/kzdev = NULL/ (Alex)
> - rename vfio_pci_zdev_open to vfio_pci_zdev_open_device (Jason)
> - rename vfio_pci_zdev_release to vfio_pci_zdev_close_device (Jason)
> - make vfio_pci_zdev_close_device return void, instead WARN_ON or ignore
>    errors in lower level function (kvm_s390_pci_unregister_kvm) (Jason)
> - remove notifier accidentally left in struct zpci_dev + associated
>    include statment (Jason)
> - Remove patch 'KVM: s390: introduce CPU feature for zPCI Interpretation'
>    based on discussion in QEMU thread.
> 
> Matthew Rosato (21):
>    s390/sclp: detect the zPCI load/store interpretation facility
>    s390/sclp: detect the AISII facility
>    s390/sclp: detect the AENI facility
>    s390/sclp: detect the AISI facility
>    s390/airq: pass more TPI info to airq handlers
>    s390/airq: allow for airq structure that uses an input vector
>    s390/pci: externalize the SIC operation controls and routine
>    s390/pci: stash associated GISA designation
>    s390/pci: stash dtsm and maxstbl
>    vfio/pci: introduce CONFIG_VFIO_PCI_ZDEV_KVM
>    KVM: s390: pci: add basic kvm_zdev structure
>    KVM: s390: pci: do initial setup for AEN interpretation
>    KVM: s390: pci: enable host forwarding of Adapter Event Notifications
>    KVM: s390: mechanism to enable guest zPCI Interpretation
>    KVM: s390: pci: provide routines for enabling/disabling interrupt
>      forwarding
>    KVM: s390: pci: add routines to start/stop interpretive execution
>    vfio-pci/zdev: add open/close device hooks
>    vfio-pci/zdev: add function handle to clp base capability
>    vfio-pci/zdev: different maxstbl for interpreted devices
>    KVM: s390: add KVM_S390_ZPCI_OP to manage guest zPCI devices
>    MAINTAINERS: additional files related kvm s390 pci passthrough
> 
>   Documentation/virt/kvm/api.rst   |  47 +++
>   MAINTAINERS                      |   1 +
>   arch/s390/include/asm/airq.h     |   7 +-
>   arch/s390/include/asm/kvm_host.h |  23 ++
>   arch/s390/include/asm/pci.h      |  11 +
>   arch/s390/include/asm/pci_clp.h  |   9 +-
>   arch/s390/include/asm/pci_insn.h |  29 +-
>   arch/s390/include/asm/sclp.h     |   4 +
>   arch/s390/include/asm/tpi.h      |  13 +
>   arch/s390/kvm/Makefile           |   1 +
>   arch/s390/kvm/interrupt.c        |  96 ++++-
>   arch/s390/kvm/kvm-s390.c         |  83 +++-
>   arch/s390/kvm/kvm-s390.h         |  10 +
>   arch/s390/kvm/pci.c              | 690 +++++++++++++++++++++++++++++++
>   arch/s390/kvm/pci.h              |  88 ++++
>   arch/s390/pci/pci.c              |  16 +
>   arch/s390/pci/pci_clp.c          |   7 +
>   arch/s390/pci/pci_insn.c         |   4 +-
>   arch/s390/pci/pci_irq.c          |  48 ++-
>   drivers/s390/char/sclp_early.c   |   4 +
>   drivers/s390/cio/airq.c          |  12 +-
>   drivers/s390/cio/qdio_thinint.c  |   6 +-
>   drivers/s390/crypto/ap_bus.c     |   9 +-
>   drivers/s390/virtio/virtio_ccw.c |   6 +-
>   drivers/vfio/pci/Kconfig         |  11 +
>   drivers/vfio/pci/Makefile        |   2 +-
>   drivers/vfio/pci/vfio_pci_core.c |  10 +-
>   drivers/vfio/pci/vfio_pci_zdev.c |  35 +-
>   include/linux/sched/user.h       |   3 +-
>   include/linux/vfio_pci_core.h    |  12 +-
>   include/uapi/linux/kvm.h         |  31 ++
>   include/uapi/linux/vfio_zdev.h   |   7 +
>   32 files changed, 1279 insertions(+), 56 deletions(-)
>   create mode 100644 arch/s390/kvm/pci.c
>   create mode 100644 arch/s390/kvm/pci.h

So I pulled this into a topic branch and will merge that into kvms390/next. We can
merge this topic  branch into vfio-next and/or s390-next when the conflicts get
to complicated.

While pulling I fixed up the numbers for the capability to

#define KVM_CAP_S390_ZPCI_OP 221

and the doc number to

4.137 KVM_S390_ZPCI_OP

to minize struggle when doing backports.

On 6/6/22 4:33 PM, Matthew Rosato wrote:
> Enable interpretive execution of zPCI instructions + adapter interruption
> forwarding for s390x KVM vfio-pci.  This is done by triggering a routine
> when the VFIO group is associated with the KVM guest, transmitting to
> firmware a special token (GISA designation) to enable that specific guest
> for interpretive execution on that zPCI device.  Load/store interpreation
> enablement is then controlled by userspace (based upon whether or not a
> SHM bit is placed in the virtual function handle).  Adapter Event
> Notification interpretation is controlled from userspace via a new KVM
> ioctl.
> 
> By allowing intepretation of zPCI instructions and firmware delivery of
> interrupts to guests, we can reduce the frequency of guest SIE exits for
> zPCI.
> 
>  From the perspective of guest configuration, you passthrough zPCI devices
> in the same manner as before, with intepretation support being used by
> default if available in kernel+qemu.
> 
> Will follow up with a link the most recent QEMU series.
> 

Latest QEMU series:

https://lore.kernel.org/kvm/20220606203614.110928-1-mjrosato@linux.ibm.com/