For QEMU, the majority of the work in enabling instruction interpretation
is handled via new VFIO ioctls to SET the appropriate interpretation and
interrupt forwarding modes, and to GET the function handle to use for
interpretive execution.  

This series implements these new ioctls, as well as adding a new, optional
'intercept' parameter to zpci to request interpretation support not be used
as well as an 'intassist' parameter to determine whether or not the
firmware assist will be used for interrupt delivery or whether the host
will be responsible for delivering all interrupts.

The ZPCI_INTERP CPU feature is added beginning with the z14 model to
enable this support.

As a consequence of implementing zPCI interpretation, ISM devices now
become eligible for passthrough (but only when zPCI interpretation is
available).

From the perspective of guest configuration, you passthrough zPCI devices
in the same manner as before, with intepretation support being used by
default if available in kernel+qemu.

Associated kernel series:
https://lore.kernel.org/kvm/20220114203145.242984-1-mjrosato@linux.ibm.com/

Changes v1->v2:

- Update kernel headers sync                                                    
- Drop some pre-req patches that are now merged                                 
- Add some R-bs (Thanks!)                                                       
- fence FEAT_ZPCI_INTERP for QEMU 6.2 and older (Christian)                     
- switch from container_of to VFIO_PCI and drop asserts (Thomas)                
- re-arrange g_autofree so we malloc at time of declaration (Thomas) 

Matthew Rosato (9):
  Update linux headers
  target/s390x: add zpci-interp to cpu models
  fixup: force interp off for QEMU machine 6.2 and older
  s390x/pci: enable for load/store intepretation
  s390x/pci: don't fence interpreted devices without MSI-X
  s390x/pci: enable adapter event notification for interpreted devices
  s390x/pci: use I/O Address Translation assist when interpreting
  s390x/pci: use dtsm provided from vfio capabilities for interpreted
    devices
  s390x/pci: let intercept devices have separate PCI groups

 hw/s390x/s390-pci-bus.c                       | 121 ++++++++++-
 hw/s390x/s390-pci-inst.c                      | 168 ++++++++++++++-
 hw/s390x/s390-pci-vfio.c                      | 204 +++++++++++++++++-
 hw/s390x/s390-virtio-ccw.c                    |   1 +
 include/hw/s390x/s390-pci-bus.h               |   8 +-
 include/hw/s390x/s390-pci-inst.h              |   2 +-
 include/hw/s390x/s390-pci-vfio.h              |  45 ++++
 include/standard-headers/asm-x86/kvm_para.h   |   1 +
 include/standard-headers/drm/drm_fourcc.h     |  11 +
 include/standard-headers/linux/ethtool.h      |   1 +
 include/standard-headers/linux/fuse.h         |  60 +++++-
 include/standard-headers/linux/pci_regs.h     |   4 +
 include/standard-headers/linux/virtio_iommu.h |   8 +-
 linux-headers/asm-mips/unistd_n32.h           |   1 +
 linux-headers/asm-mips/unistd_n64.h           |   1 +
 linux-headers/asm-mips/unistd_o32.h           |   1 +
 linux-headers/asm-powerpc/unistd_32.h         |   1 +
 linux-headers/asm-powerpc/unistd_64.h         |   1 +
 linux-headers/asm-s390/kvm.h                  |   1 +
 linux-headers/asm-s390/unistd_32.h            |   1 +
 linux-headers/asm-s390/unistd_64.h            |   1 +
 linux-headers/linux/kvm.h                     |   1 +
 linux-headers/linux/vfio.h                    |  22 ++
 linux-headers/linux/vfio_zdev.h               |  51 +++++
 target/s390x/cpu_features_def.h.inc           |   1 +
 target/s390x/gen-features.c                   |   2 +
 target/s390x/kvm/kvm.c                        |   1 +
 27 files changed, 693 insertions(+), 27 deletions(-)

-- 
2.27.0

On 14/01/2022 21.38, Matthew Rosato wrote:
> For QEMU, the majority of the work in enabling instruction interpretation
> is handled via new VFIO ioctls to SET the appropriate interpretation and
> interrupt forwarding modes, and to GET the function handle to use for
> interpretive execution.
> 
> This series implements these new ioctls, as well as adding a new, optional
> 'intercept' parameter to zpci to request interpretation support not be used
> as well as an 'intassist' parameter to determine whether or not the
> firmware assist will be used for interrupt delivery or whether the host
> will be responsible for delivering all interrupts.

  Hi Matthew,

would it make sense to create a docs/system/s390x/zpci.rst doc file, too, 
where you could describe such new parameters like 'intassist' and 
'intercept' (or is it 'interp') ? ... otherwise hardly anybody except you 
will know how to use these parameters later...

  Thomas

On 1/17/22 10:23 AM, Thomas Huth wrote:
> On 14/01/2022 21.38, Matthew Rosato wrote:
>> For QEMU, the majority of the work in enabling instruction interpretation
>> is handled via new VFIO ioctls to SET the appropriate interpretation and
>> interrupt forwarding modes, and to GET the function handle to use for
>> interpretive execution.
>>
>> This series implements these new ioctls, as well as adding a new, 
>> optional
>> 'intercept' parameter to zpci to request interpretation support not be 
>> used
>> as well as an 'intassist' parameter to determine whether or not the
>> firmware assist will be used for interrupt delivery or whether the host
>> will be responsible for delivering all interrupts.
> 
>   Hi Matthew,
> 
> would it make sense to create a docs/system/s390x/zpci.rst doc file, 

This is a good idea and probably something that was due for zpci before 
this series even.

> too, where you could describe such new parameters like 'intassist' and 
> 'intercept' (or is it 'interp') ? ... otherwise hardly anybody except 

Oops, 'intercept' was a holdover from a previous version and effectively 
had an inverted meaning.  It is indeed 'interp' with this current series 
(and subject to change again per other thread)

On Fri, Jan 14, 2022 at 03:38:40PM -0500, Matthew Rosato wrote:
> For QEMU, the majority of the work in enabling instruction interpretation
> is handled via new VFIO ioctls to SET the appropriate interpretation and
> interrupt forwarding modes, and to GET the function handle to use for
> interpretive execution.  
> 
> This series implements these new ioctls, as well as adding a new, optional
> 'intercept' parameter to zpci to request interpretation support not be used
> as well as an 'intassist' parameter to determine whether or not the
> firmware assist will be used for interrupt delivery or whether the host
> will be responsible for delivering all interrupts.
> 
> The ZPCI_INTERP CPU feature is added beginning with the z14 model to
> enable this support.
> 
> As a consequence of implementing zPCI interpretation, ISM devices now
> become eligible for passthrough (but only when zPCI interpretation is
> available).
> 
> >From the perspective of guest configuration, you passthrough zPCI devices
> in the same manner as before, with intepretation support being used by
> default if available in kernel+qemu.
> 
> Associated kernel series:
> https://lore.kernel.org/kvm/20220114203145.242984-1-mjrosato@linux.ibm.com/

I took a quick look and don't see much to object to, but
I don't know much about the topic either.

Acked-by: Michael S. Tsirkin <mst@redhat.com>

Pls use the s390 tree for this.

> Changes v1->v2:
> 
> - Update kernel headers sync                                                    
> - Drop some pre-req patches that are now merged                                 
> - Add some R-bs (Thanks!)                                                       
> - fence FEAT_ZPCI_INTERP for QEMU 6.2 and older (Christian)                     
> - switch from container_of to VFIO_PCI and drop asserts (Thomas)                
> - re-arrange g_autofree so we malloc at time of declaration (Thomas) 
> 
> Matthew Rosato (9):
>   Update linux headers
>   target/s390x: add zpci-interp to cpu models
>   fixup: force interp off for QEMU machine 6.2 and older
>   s390x/pci: enable for load/store intepretation
>   s390x/pci: don't fence interpreted devices without MSI-X
>   s390x/pci: enable adapter event notification for interpreted devices
>   s390x/pci: use I/O Address Translation assist when interpreting
>   s390x/pci: use dtsm provided from vfio capabilities for interpreted
>     devices
>   s390x/pci: let intercept devices have separate PCI groups
> 
>  hw/s390x/s390-pci-bus.c                       | 121 ++++++++++-
>  hw/s390x/s390-pci-inst.c                      | 168 ++++++++++++++-
>  hw/s390x/s390-pci-vfio.c                      | 204 +++++++++++++++++-
>  hw/s390x/s390-virtio-ccw.c                    |   1 +
>  include/hw/s390x/s390-pci-bus.h               |   8 +-
>  include/hw/s390x/s390-pci-inst.h              |   2 +-
>  include/hw/s390x/s390-pci-vfio.h              |  45 ++++
>  include/standard-headers/asm-x86/kvm_para.h   |   1 +
>  include/standard-headers/drm/drm_fourcc.h     |  11 +
>  include/standard-headers/linux/ethtool.h      |   1 +
>  include/standard-headers/linux/fuse.h         |  60 +++++-
>  include/standard-headers/linux/pci_regs.h     |   4 +
>  include/standard-headers/linux/virtio_iommu.h |   8 +-
>  linux-headers/asm-mips/unistd_n32.h           |   1 +
>  linux-headers/asm-mips/unistd_n64.h           |   1 +
>  linux-headers/asm-mips/unistd_o32.h           |   1 +
>  linux-headers/asm-powerpc/unistd_32.h         |   1 +
>  linux-headers/asm-powerpc/unistd_64.h         |   1 +
>  linux-headers/asm-s390/kvm.h                  |   1 +
>  linux-headers/asm-s390/unistd_32.h            |   1 +
>  linux-headers/asm-s390/unistd_64.h            |   1 +
>  linux-headers/linux/kvm.h                     |   1 +
>  linux-headers/linux/vfio.h                    |  22 ++
>  linux-headers/linux/vfio_zdev.h               |  51 +++++
>  target/s390x/cpu_features_def.h.inc           |   1 +
>  target/s390x/gen-features.c                   |   2 +
>  target/s390x/kvm/kvm.c                        |   1 +
>  27 files changed, 693 insertions(+), 27 deletions(-)
> 
> -- 
> 2.27.0