[PATCH 00/12] s390x/pci: zPCI interpretation support

Matthew Rosato posted 12 patches 2 years, 4 months ago
Failed in applying to current master (apply log)
There is a newer version of this series
hw/s390x/s390-pci-bus.c                       | 121 +++++++++-
hw/s390x/s390-pci-inst.c                      | 178 +++++++++++++-
hw/s390x/s390-pci-vfio.c                      | 221 +++++++++++++++++-
include/hw/s390x/s390-pci-bus.h               |  11 +-
include/hw/s390x/s390-pci-clp.h               |   3 +-
include/hw/s390x/s390-pci-inst.h              |   2 +-
include/hw/s390x/s390-pci-vfio.h              |  45 ++++
include/hw/virtio/virtio-gpu-bswap.h          |   1 -
include/standard-headers/asm-x86/kvm_para.h   |   1 +
include/standard-headers/drm/drm_fourcc.h     | 121 +++++++++-
include/standard-headers/linux/ethtool.h      |  31 +++
include/standard-headers/linux/fuse.h         |  15 +-
include/standard-headers/linux/pci_regs.h     |   6 +
include/standard-headers/linux/virtio_gpu.h   |  18 +-
include/standard-headers/linux/virtio_ids.h   |  24 ++
include/standard-headers/linux/virtio_mem.h   |   9 +-
include/standard-headers/linux/virtio_vsock.h |   3 +-
linux-headers/asm-arm64/unistd.h              |   1 +
linux-headers/asm-generic/unistd.h            |  22 +-
linux-headers/asm-mips/unistd_n32.h           |   2 +
linux-headers/asm-mips/unistd_n64.h           |   2 +
linux-headers/asm-mips/unistd_o32.h           |   2 +
linux-headers/asm-powerpc/unistd_32.h         |   2 +
linux-headers/asm-powerpc/unistd_64.h         |   2 +
linux-headers/asm-s390/kvm.h                  |   1 +
linux-headers/asm-s390/unistd_32.h            |   2 +
linux-headers/asm-s390/unistd_64.h            |   2 +
linux-headers/asm-x86/kvm.h                   |   5 +
linux-headers/asm-x86/unistd_32.h             |   3 +
linux-headers/asm-x86/unistd_64.h             |   3 +
linux-headers/asm-x86/unistd_x32.h            |   3 +
linux-headers/linux/kvm.h                     |  41 +++-
linux-headers/linux/vfio.h                    |  22 ++
linux-headers/linux/vfio_zdev.h               |  51 ++++
target/s390x/cpu_features_def.h.inc           |   1 +
target/s390x/gen-features.c                   |   2 +
target/s390x/kvm/kvm.c                        |   1 +
37 files changed, 928 insertions(+), 52 deletions(-)
[PATCH 00/12] s390x/pci: zPCI interpretation support
Posted by Matthew Rosato 2 years, 4 months ago
Note:  The first 3 patches of this series are included as pre-reqs, but
should be pulled via a separate series.  Also, patch 5 is needed to
support 5.16+ linux header-sync and was already done by Paolo but not
merged yet so is thus included here as well. 

For QEMU, the majority of the work in enabling instruction interpretation
is handled via new VFIO ioctls to SET the appropriate interpretation and
interrupt forwarding modes, and to GET the function handle to use for
interpretive execution.  

This series implements these new ioctls, as well as adding a new, optional
'intercept' parameter to zpci to request interpretation support not be used
as well as an 'intassist' parameter to determine whether or not the
firmware assist will be used for interrupt delivery or whether the host
will be responsible for delivering all interrupts.

The ZPCI_INTERP CPU feature is added beginning with the z14 model to
enable this support.

As a consequence of implementing zPCI interpretation, ISM devices now
become eligible for passthrough (but only when zPCI interpretation is
available).

From the perspective of guest configuration, you passthrough zPCI devices
in the same manner as before, with intepretation support being used by
default if available in kernel+qemu.

Associated kernel series:
https://lkml.org/lkml/2021/12/7/1179

Matthew Rosato (11):
  s390x/pci: use a reserved ID for the default PCI group
  s390x/pci: don't use hard-coded dma range in reg_ioat
  s390x/pci: add supported DT information to clp response
  Update linux headers
  target/s390x: add zpci-interp to cpu models
  s390x/pci: enable for load/store intepretation
  s390x/pci: don't fence interpreted devices without MSI-X
  s390x/pci: enable adapter event notification for interpreted devices
  s390x/pci: use I/O Address Translation assist when interpreting
  s390x/pci: use dtsm provided from vfio capabilities for interpreted
    devices
  s390x/pci: let intercept devices have separate PCI groups

Paolo Bonzini (1):
  virtio-gpu: do not byteswap padding

 hw/s390x/s390-pci-bus.c                       | 121 +++++++++-
 hw/s390x/s390-pci-inst.c                      | 178 +++++++++++++-
 hw/s390x/s390-pci-vfio.c                      | 221 +++++++++++++++++-
 include/hw/s390x/s390-pci-bus.h               |  11 +-
 include/hw/s390x/s390-pci-clp.h               |   3 +-
 include/hw/s390x/s390-pci-inst.h              |   2 +-
 include/hw/s390x/s390-pci-vfio.h              |  45 ++++
 include/hw/virtio/virtio-gpu-bswap.h          |   1 -
 include/standard-headers/asm-x86/kvm_para.h   |   1 +
 include/standard-headers/drm/drm_fourcc.h     | 121 +++++++++-
 include/standard-headers/linux/ethtool.h      |  31 +++
 include/standard-headers/linux/fuse.h         |  15 +-
 include/standard-headers/linux/pci_regs.h     |   6 +
 include/standard-headers/linux/virtio_gpu.h   |  18 +-
 include/standard-headers/linux/virtio_ids.h   |  24 ++
 include/standard-headers/linux/virtio_mem.h   |   9 +-
 include/standard-headers/linux/virtio_vsock.h |   3 +-
 linux-headers/asm-arm64/unistd.h              |   1 +
 linux-headers/asm-generic/unistd.h            |  22 +-
 linux-headers/asm-mips/unistd_n32.h           |   2 +
 linux-headers/asm-mips/unistd_n64.h           |   2 +
 linux-headers/asm-mips/unistd_o32.h           |   2 +
 linux-headers/asm-powerpc/unistd_32.h         |   2 +
 linux-headers/asm-powerpc/unistd_64.h         |   2 +
 linux-headers/asm-s390/kvm.h                  |   1 +
 linux-headers/asm-s390/unistd_32.h            |   2 +
 linux-headers/asm-s390/unistd_64.h            |   2 +
 linux-headers/asm-x86/kvm.h                   |   5 +
 linux-headers/asm-x86/unistd_32.h             |   3 +
 linux-headers/asm-x86/unistd_64.h             |   3 +
 linux-headers/asm-x86/unistd_x32.h            |   3 +
 linux-headers/linux/kvm.h                     |  41 +++-
 linux-headers/linux/vfio.h                    |  22 ++
 linux-headers/linux/vfio_zdev.h               |  51 ++++
 target/s390x/cpu_features_def.h.inc           |   1 +
 target/s390x/gen-features.c                   |   2 +
 target/s390x/kvm/kvm.c                        |   1 +
 37 files changed, 928 insertions(+), 52 deletions(-)

-- 
2.27.0


Re: [PATCH 00/12] s390x/pci: zPCI interpretation support
Posted by Pierre Morel 2 years, 4 months ago

On 12/7/21 22:04, Matthew Rosato wrote:
> Note:  The first 3 patches of this series are included as pre-reqs, but
> should be pulled via a separate series.  Also, patch 5 is needed to
> support 5.16+ linux header-sync and was already done by Paolo but not
> merged yet so is thus included here as well.
> 
> For QEMU, the majority of the work in enabling instruction interpretation
> is handled via new VFIO ioctls to SET the appropriate interpretation and
> interrupt forwarding modes, and to GET the function handle to use for
> interpretive execution.
> 
> This series implements these new ioctls, as well as adding a new, optional
> 'intercept' parameter to zpci to request interpretation support not be used
> as well as an 'intassist' parameter to determine whether or not the
> firmware assist will be used for interrupt delivery or whether the host
> will be responsible for delivering all interrupts.

In which circumstances do we have an added value by not using interrupt 
delivered by firmware?


> 
> The ZPCI_INTERP CPU feature is added beginning with the z14 model to
> enable this support.
> 
> As a consequence of implementing zPCI interpretation, ISM devices now
> become eligible for passthrough (but only when zPCI interpretation is
> available).
> 
>  From the perspective of guest configuration, you passthrough zPCI devices
> in the same manner as before, with intepretation support being used by
> default if available in kernel+qemu.
> 
> Associated kernel series:
> https://lkml.org/lkml/2021/12/7/1179
> 
> Matthew Rosato (11):
>    s390x/pci: use a reserved ID for the default PCI group
>    s390x/pci: don't use hard-coded dma range in reg_ioat
>    s390x/pci: add supported DT information to clp response
>    Update linux headers
>    target/s390x: add zpci-interp to cpu models
>    s390x/pci: enable for load/store intepretation
>    s390x/pci: don't fence interpreted devices without MSI-X
>    s390x/pci: enable adapter event notification for interpreted devices
>    s390x/pci: use I/O Address Translation assist when interpreting
>    s390x/pci: use dtsm provided from vfio capabilities for interpreted
>      devices
>    s390x/pci: let intercept devices have separate PCI groups
> 
> Paolo Bonzini (1):
>    virtio-gpu: do not byteswap padding
> 
>   hw/s390x/s390-pci-bus.c                       | 121 +++++++++-
>   hw/s390x/s390-pci-inst.c                      | 178 +++++++++++++-
>   hw/s390x/s390-pci-vfio.c                      | 221 +++++++++++++++++-
>   include/hw/s390x/s390-pci-bus.h               |  11 +-
>   include/hw/s390x/s390-pci-clp.h               |   3 +-
>   include/hw/s390x/s390-pci-inst.h              |   2 +-
>   include/hw/s390x/s390-pci-vfio.h              |  45 ++++
>   include/hw/virtio/virtio-gpu-bswap.h          |   1 -
>   include/standard-headers/asm-x86/kvm_para.h   |   1 +
>   include/standard-headers/drm/drm_fourcc.h     | 121 +++++++++-
>   include/standard-headers/linux/ethtool.h      |  31 +++
>   include/standard-headers/linux/fuse.h         |  15 +-
>   include/standard-headers/linux/pci_regs.h     |   6 +
>   include/standard-headers/linux/virtio_gpu.h   |  18 +-
>   include/standard-headers/linux/virtio_ids.h   |  24 ++
>   include/standard-headers/linux/virtio_mem.h   |   9 +-
>   include/standard-headers/linux/virtio_vsock.h |   3 +-
>   linux-headers/asm-arm64/unistd.h              |   1 +
>   linux-headers/asm-generic/unistd.h            |  22 +-
>   linux-headers/asm-mips/unistd_n32.h           |   2 +
>   linux-headers/asm-mips/unistd_n64.h           |   2 +
>   linux-headers/asm-mips/unistd_o32.h           |   2 +
>   linux-headers/asm-powerpc/unistd_32.h         |   2 +
>   linux-headers/asm-powerpc/unistd_64.h         |   2 +
>   linux-headers/asm-s390/kvm.h                  |   1 +
>   linux-headers/asm-s390/unistd_32.h            |   2 +
>   linux-headers/asm-s390/unistd_64.h            |   2 +
>   linux-headers/asm-x86/kvm.h                   |   5 +
>   linux-headers/asm-x86/unistd_32.h             |   3 +
>   linux-headers/asm-x86/unistd_64.h             |   3 +
>   linux-headers/asm-x86/unistd_x32.h            |   3 +
>   linux-headers/linux/kvm.h                     |  41 +++-
>   linux-headers/linux/vfio.h                    |  22 ++
>   linux-headers/linux/vfio_zdev.h               |  51 ++++
>   target/s390x/cpu_features_def.h.inc           |   1 +
>   target/s390x/gen-features.c                   |   2 +
>   target/s390x/kvm/kvm.c                        |   1 +
>   37 files changed, 928 insertions(+), 52 deletions(-)
> 

-- 
Pierre Morel
IBM Lab Boeblingen

Re: [PATCH 00/12] s390x/pci: zPCI interpretation support
Posted by Matthew Rosato 2 years, 4 months ago
On 12/15/21 2:35 AM, Pierre Morel wrote:
> 
> 
> On 12/7/21 22:04, Matthew Rosato wrote:
>> Note:  The first 3 patches of this series are included as pre-reqs, but
>> should be pulled via a separate series.  Also, patch 5 is needed to
>> support 5.16+ linux header-sync and was already done by Paolo but not
>> merged yet so is thus included here as well.
>>
>> For QEMU, the majority of the work in enabling instruction interpretation
>> is handled via new VFIO ioctls to SET the appropriate interpretation and
>> interrupt forwarding modes, and to GET the function handle to use for
>> interpretive execution.
>>
>> This series implements these new ioctls, as well as adding a new, 
>> optional
>> 'intercept' parameter to zpci to request interpretation support not be 
>> used
>> as well as an 'intassist' parameter to determine whether or not the
>> firmware assist will be used for interrupt delivery or whether the host
>> will be responsible for delivering all interrupts.
> 
> In which circumstances do we have an added value by not using interrupt 
> delivered by firmware?
> 

Disabling it can be a tool to debug and assist in problem determination, 
but that's about the only scenario I can think of where you would 
intentionally want to disable intassist.  Perhaps then it's not worth 
leaving in place.


Re: [PATCH 00/12] s390x/pci: zPCI interpretation support
Posted by Christian Borntraeger 2 years, 4 months ago

Am 15.12.21 um 16:53 schrieb Matthew Rosato:
> On 12/15/21 2:35 AM, Pierre Morel wrote:
>>
>>
>> On 12/7/21 22:04, Matthew Rosato wrote:
>>> Note:  The first 3 patches of this series are included as pre-reqs, but
>>> should be pulled via a separate series.  Also, patch 5 is needed to
>>> support 5.16+ linux header-sync and was already done by Paolo but not
>>> merged yet so is thus included here as well.
>>>
>>> For QEMU, the majority of the work in enabling instruction interpretation
>>> is handled via new VFIO ioctls to SET the appropriate interpretation and
>>> interrupt forwarding modes, and to GET the function handle to use for
>>> interpretive execution.
>>>
>>> This series implements these new ioctls, as well as adding a new, optional
>>> 'intercept' parameter to zpci to request interpretation support not be used
>>> as well as an 'intassist' parameter to determine whether or not the
>>> firmware assist will be used for interrupt delivery or whether the host
>>> will be responsible for delivering all interrupts.
>>
>> In which circumstances do we have an added value by not using interrupt delivered by firmware?
>>
> 
> Disabling it can be a tool to debug and assist in problem determination, but that's about the only scenario I can think of where you would intentionally want to disable intassist.  Perhaps then it's not worth leaving in place.

I would leave it in in case we run into problems. Things like the nomio parameter for the kernel have proven to be useful.