[PULL v2 00/32] VFIO updates 2020-10-28 (for QEMU 5.2 soft-freeze)

Alex Williamson posted 32 patches 3 years, 6 months ago
Test checkpatch passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/160390309510.12234.8858324597971641979.stgit@gimli.home
Maintainers: Halil Pasic <pasic@linux.ibm.com>, Thomas Huth <thuth@redhat.com>, Matthew Rosato <mjrosato@linux.ibm.com>, Juan Quintela <quintela@redhat.com>, Cornelia Huck <cohuck@redhat.com>, David Hildenbrand <david@redhat.com>, Markus Armbruster <armbru@redhat.com>, Christian Borntraeger <borntraeger@de.ibm.com>, "Michael S. Tsirkin" <mst@redhat.com>, Alex Williamson <alex.williamson@redhat.com>, Eric Blake <eblake@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, Richard Henderson <rth@twiddle.net>, "Dr. David Alan Gilbert" <dgilbert@redhat.com>
MAINTAINERS                                        |   1 +
hw/s390x/meson.build                               |   1 +
hw/s390x/s390-pci-bus.c                            |  91 +-
hw/s390x/s390-pci-inst.c                           |  78 +-
hw/s390x/s390-pci-vfio.c                           | 276 ++++++
hw/s390x/s390-virtio-ccw.c                         |   2 +-
hw/s390x/trace-events                              |   6 +
hw/vfio/common.c                                   | 508 ++++++++++-
hw/vfio/meson.build                                |   1 +
hw/vfio/migration.c                                | 933 +++++++++++++++++++++
hw/vfio/pci.c                                      |  87 +-
hw/vfio/pci.h                                      |   1 -
hw/vfio/platform.c                                 |   7 +-
hw/vfio/trace-events                               |  21 +
{hw => include/hw}/s390x/s390-pci-bus.h            |  22 +
.../hw/s390x/s390-pci-clp.h                        | 123 +--
include/hw/s390x/s390-pci-inst.h                   | 119 +++
include/hw/s390x/s390-pci-vfio.h                   |  23 +
include/hw/vfio/vfio-common.h                      |  30 +
.../infiniband/hw/vmw_pvrdma/pvrdma_verbs.h        |   2 +-
include/standard-headers/linux/ethtool.h           |   2 +
include/standard-headers/linux/fuse.h              |  50 +-
include/standard-headers/linux/input-event-codes.h |   4 +
include/standard-headers/linux/pci_regs.h          |   6 +-
include/standard-headers/linux/virtio_fs.h         |   3 +
include/standard-headers/linux/virtio_gpu.h        |  19 +
include/standard-headers/linux/virtio_mmio.h       |  11 +
include/standard-headers/linux/virtio_pci.h        |  11 +-
linux-headers/asm-arm64/kvm.h                      |  25 +
linux-headers/asm-arm64/mman.h                     |   1 +
linux-headers/asm-generic/hugetlb_encode.h         |   1 +
linux-headers/asm-generic/unistd.h                 |  18 +-
linux-headers/asm-mips/unistd_n32.h                |   1 +
linux-headers/asm-mips/unistd_n64.h                |   1 +
linux-headers/asm-mips/unistd_o32.h                |   1 +
linux-headers/asm-powerpc/unistd_32.h              |   1 +
linux-headers/asm-powerpc/unistd_64.h              |   1 +
linux-headers/asm-s390/unistd_32.h                 |   1 +
linux-headers/asm-s390/unistd_64.h                 |   1 +
linux-headers/asm-x86/kvm.h                        |  20 +
linux-headers/asm-x86/unistd_32.h                  |   1 +
linux-headers/asm-x86/unistd_64.h                  |   1 +
linux-headers/asm-x86/unistd_x32.h                 |   1 +
linux-headers/linux/kvm.h                          |  19 +
linux-headers/linux/mman.h                         |   1 +
linux-headers/linux/vfio.h                         |  29 +-
linux-headers/linux/vfio_zdev.h                    |  78 ++
migration/migration.c                              |  17 +
monitor/hmp-cmds.c                                 |   6 +
qapi/migration.json                                |  17 +
scripts/update-linux-headers.sh                    |   2 +-
softmmu/memory.c                                   |   2 +-
52 files changed, 2467 insertions(+), 217 deletions(-)
create mode 100644 hw/s390x/s390-pci-vfio.c
create mode 100644 hw/vfio/migration.c
rename {hw => include/hw}/s390x/s390-pci-bus.h (94%)
rename hw/s390x/s390-pci-inst.h => include/hw/s390x/s390-pci-clp.h (59%)
create mode 100644 include/hw/s390x/s390-pci-inst.h
create mode 100644 include/hw/s390x/s390-pci-vfio.h
create mode 100644 linux-headers/linux/vfio_zdev.h
[PULL v2 00/32] VFIO updates 2020-10-28 (for QEMU 5.2 soft-freeze)
Posted by Alex Williamson 3 years, 6 months ago
The following changes since commit 33dc9914eac581dea9bdea35dcda4d542531d66a:

  Revert series: virtiofsd: Announce submounts to the guest (2020-10-28 13:17:32 +0000)

are available in the Git repository at:

  git://github.com/awilliam/qemu-vfio.git tags/vfio-update-20201028.0

for you to fetch changes up to 83d64f2efe383f1f70e180cf1579d3bbe2fbcdf5:

  vfio: fix incorrect print type (2020-10-28 10:30:37 -0600)

----------------------------------------------------------------
VFIO update 2020-10-28

 * Migration support (Kirti Wankhede)
 * s390 DMA limiting (Matthew Rosato)
 * zPCI hardware info (Matthew Rosato)
 * Lock guard (Amey Narkhede)
 * Print fixes (Zhengui li)
 * Warning/build fixes

----------------------------------------------------------------
Amey Narkhede (1):
      hw/vfio: Use lock guard macros

Kirti Wankhede (17):
      vfio: Add function to unmap VFIO region
      vfio: Add vfio_get_object callback to VFIODeviceOps
      vfio: Add save and load functions for VFIO PCI devices
      vfio: Add migration region initialization and finalize function
      vfio: Add VM state change handler to know state of VM
      vfio: Add migration state change notifier
      vfio: Register SaveVMHandlers for VFIO device
      vfio: Add save state functions to SaveVMHandlers
      vfio: Add load state functions to SaveVMHandlers
      memory: Set DIRTY_MEMORY_MIGRATION when IOMMU is enabled
      vfio: Get migration capability flags for container
      vfio: Add function to start and stop dirty pages tracking
      vfio: Add vfio_listener_log_sync to mark dirty pages
      vfio: Dirty page tracking when vIOMMU is enabled
      vfio: Add ioctl to get dirty pages bitmap during dma unmap
      vfio: Make vfio-pci device migration capable
      qapi: Add VFIO devices migration stats in Migration stats

Matthew Rosato (10):
      update-linux-headers: Add vfio_zdev.h
      linux-headers: update against 5.10-rc1
      s390x/pci: Move header files to include/hw/s390x
      vfio: Create shared routine for scanning info capabilities
      vfio: Find DMA available capability
      s390x/pci: Add routine to get the vfio dma available count
      s390x/pci: Honor DMA limits set by vfio
      s390x/pci: clean up s390 PCI groups
      vfio: Add routine for finding VFIO_DEVICE_GET_INFO capabilities
      s390x/pci: get zPCI function info from host

Pierre Morel (3):
      s390x/pci: create a header dedicated to PCI CLP
      s390x/pci: use a PCI Group structure
      s390x/pci: use a PCI Function structure

Zhengui Li (1):
      vfio: fix incorrect print type

 MAINTAINERS                                        |   1 +
 hw/s390x/meson.build                               |   1 +
 hw/s390x/s390-pci-bus.c                            |  91 +-
 hw/s390x/s390-pci-inst.c                           |  78 +-
 hw/s390x/s390-pci-vfio.c                           | 276 ++++++
 hw/s390x/s390-virtio-ccw.c                         |   2 +-
 hw/s390x/trace-events                              |   6 +
 hw/vfio/common.c                                   | 508 ++++++++++-
 hw/vfio/meson.build                                |   1 +
 hw/vfio/migration.c                                | 933 +++++++++++++++++++++
 hw/vfio/pci.c                                      |  87 +-
 hw/vfio/pci.h                                      |   1 -
 hw/vfio/platform.c                                 |   7 +-
 hw/vfio/trace-events                               |  21 +
 {hw => include/hw}/s390x/s390-pci-bus.h            |  22 +
 .../hw/s390x/s390-pci-clp.h                        | 123 +--
 include/hw/s390x/s390-pci-inst.h                   | 119 +++
 include/hw/s390x/s390-pci-vfio.h                   |  23 +
 include/hw/vfio/vfio-common.h                      |  30 +
 .../infiniband/hw/vmw_pvrdma/pvrdma_verbs.h        |   2 +-
 include/standard-headers/linux/ethtool.h           |   2 +
 include/standard-headers/linux/fuse.h              |  50 +-
 include/standard-headers/linux/input-event-codes.h |   4 +
 include/standard-headers/linux/pci_regs.h          |   6 +-
 include/standard-headers/linux/virtio_fs.h         |   3 +
 include/standard-headers/linux/virtio_gpu.h        |  19 +
 include/standard-headers/linux/virtio_mmio.h       |  11 +
 include/standard-headers/linux/virtio_pci.h        |  11 +-
 linux-headers/asm-arm64/kvm.h                      |  25 +
 linux-headers/asm-arm64/mman.h                     |   1 +
 linux-headers/asm-generic/hugetlb_encode.h         |   1 +
 linux-headers/asm-generic/unistd.h                 |  18 +-
 linux-headers/asm-mips/unistd_n32.h                |   1 +
 linux-headers/asm-mips/unistd_n64.h                |   1 +
 linux-headers/asm-mips/unistd_o32.h                |   1 +
 linux-headers/asm-powerpc/unistd_32.h              |   1 +
 linux-headers/asm-powerpc/unistd_64.h              |   1 +
 linux-headers/asm-s390/unistd_32.h                 |   1 +
 linux-headers/asm-s390/unistd_64.h                 |   1 +
 linux-headers/asm-x86/kvm.h                        |  20 +
 linux-headers/asm-x86/unistd_32.h                  |   1 +
 linux-headers/asm-x86/unistd_64.h                  |   1 +
 linux-headers/asm-x86/unistd_x32.h                 |   1 +
 linux-headers/linux/kvm.h                          |  19 +
 linux-headers/linux/mman.h                         |   1 +
 linux-headers/linux/vfio.h                         |  29 +-
 linux-headers/linux/vfio_zdev.h                    |  78 ++
 migration/migration.c                              |  17 +
 monitor/hmp-cmds.c                                 |   6 +
 qapi/migration.json                                |  17 +
 scripts/update-linux-headers.sh                    |   2 +-
 softmmu/memory.c                                   |   2 +-
 52 files changed, 2467 insertions(+), 217 deletions(-)
 create mode 100644 hw/s390x/s390-pci-vfio.c
 create mode 100644 hw/vfio/migration.c
 rename {hw => include/hw}/s390x/s390-pci-bus.h (94%)
 rename hw/s390x/s390-pci-inst.h => include/hw/s390x/s390-pci-clp.h (59%)
 create mode 100644 include/hw/s390x/s390-pci-inst.h
 create mode 100644 include/hw/s390x/s390-pci-vfio.h
 create mode 100644 linux-headers/linux/vfio_zdev.h


Re: [PULL v2 00/32] VFIO updates 2020-10-28 (for QEMU 5.2 soft-freeze)
Posted by Peter Maydell 3 years, 6 months ago
On Wed, 28 Oct 2020 at 16:42, Alex Williamson
<alex.williamson@redhat.com> wrote:
>
> The following changes since commit 33dc9914eac581dea9bdea35dcda4d542531d66a:
>
>   Revert series: virtiofsd: Announce submounts to the guest (2020-10-28 13:17:32 +0000)
>
> are available in the Git repository at:
>
>   git://github.com/awilliam/qemu-vfio.git tags/vfio-update-20201028.0
>
> for you to fetch changes up to 83d64f2efe383f1f70e180cf1579d3bbe2fbcdf5:
>
>   vfio: fix incorrect print type (2020-10-28 10:30:37 -0600)
>
> ----------------------------------------------------------------
> VFIO update 2020-10-28
>
>  * Migration support (Kirti Wankhede)
>  * s390 DMA limiting (Matthew Rosato)
>  * zPCI hardware info (Matthew Rosato)
>  * Lock guard (Amey Narkhede)
>  * Print fixes (Zhengui li)
>  * Warning/build fixes
>

Hi; this fails to build on OSX and the BSDs:

../../hw/s390x/s390-pci-vfio.c:13:10: fatal error: 'linux/vfio.h' file not found
#include <linux/vfio.h>
         ^~~~~~~~~~~~~~

fails differently but on the same file on windows builds:

../../hw/s390x/s390-pci-vfio.c:12:23: fatal error: sys/ioctl.h: No
such file or directory

and has this error on 32-bit hosts:

../../hw/vfio/common.c: In function 'vfio_dma_unmap_bitmap':
../../hw/vfio/common.c:414:48: error: passing argument 1 of
'cpu_physical_memory_set_dirty_lebitmap' from incompatible pointer
type [-Werror=incompatible-pointer-types]
         cpu_physical_memory_set_dirty_lebitmap((uint64_t *)bitmap->data,
                                                ^
In file included from ../../hw/vfio/common.c:32:0:
/home/peter.maydell/qemu/include/exec/ram_addr.h:337:20: note:
expected 'long unsigned int *' but argument is of type 'uint64_t *
{aka long long unsigned int *}'
 static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned
long *bitmap,
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../hw/vfio/common.c: In function 'vfio_get_dirty_bitmap':
../../hw/vfio/common.c:1008:44: error: passing argument 1 of
'cpu_physical_memory_set_dirty_lebitmap' from incompatible pointer
type [-Werror=incompatible-pointer-types]
     cpu_physical_memory_set_dirty_lebitmap((uint64_t *)range->bitmap.data,
                                            ^
In file included from ../../hw/vfio/common.c:32:0:
/home/peter.maydell/qemu/include/exec/ram_addr.h:337:20: note:
expected 'long unsigned int *' but argument is of type 'uint64_t *
{aka long long unsigned int *}'
 static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned
long *bitmap,
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

thanks
-- PMM

Re: [PULL v2 00/32] VFIO updates 2020-10-28 (for QEMU 5.2 soft-freeze)
Posted by Alex Williamson 3 years, 6 months ago
On Sat, 31 Oct 2020 14:54:54 +0000
Peter Maydell <peter.maydell@linaro.org> wrote:

> On Wed, 28 Oct 2020 at 16:42, Alex Williamson
> <alex.williamson@redhat.com> wrote:
> >
> > The following changes since commit 33dc9914eac581dea9bdea35dcda4d542531d66a:
> >
> >   Revert series: virtiofsd: Announce submounts to the guest (2020-10-28 13:17:32 +0000)
> >
> > are available in the Git repository at:
> >
> >   git://github.com/awilliam/qemu-vfio.git tags/vfio-update-20201028.0
> >
> > for you to fetch changes up to 83d64f2efe383f1f70e180cf1579d3bbe2fbcdf5:
> >
> >   vfio: fix incorrect print type (2020-10-28 10:30:37 -0600)
> >
> > ----------------------------------------------------------------
> > VFIO update 2020-10-28
> >
> >  * Migration support (Kirti Wankhede)
> >  * s390 DMA limiting (Matthew Rosato)
> >  * zPCI hardware info (Matthew Rosato)
> >  * Lock guard (Amey Narkhede)
> >  * Print fixes (Zhengui li)
> >  * Warning/build fixes
> >  
> 
> Hi; this fails to build on OSX and the BSDs:
> 
> ../../hw/s390x/s390-pci-vfio.c:13:10: fatal error: 'linux/vfio.h' file not found
> #include <linux/vfio.h>
>          ^~~~~~~~~~~~~~
> 
> fails differently but on the same file on windows builds:
> 
> ../../hw/s390x/s390-pci-vfio.c:12:23: fatal error: sys/ioctl.h: No
> such file or directory

I think this can be solved by making s390-pci-vfio.c only compiled
under CONFIG_LINUX and stubbing the functions with static inlines in
the header.  It seems to resolve the windows warning in a mingw build.

> and has this error on 32-bit hosts:
> 
> ../../hw/vfio/common.c: In function 'vfio_dma_unmap_bitmap':
> ../../hw/vfio/common.c:414:48: error: passing argument 1 of
> 'cpu_physical_memory_set_dirty_lebitmap' from incompatible pointer
> type [-Werror=incompatible-pointer-types]
>          cpu_physical_memory_set_dirty_lebitmap((uint64_t *)bitmap->data,
>                                                 ^
> In file included from ../../hw/vfio/common.c:32:0:
> /home/peter.maydell/qemu/include/exec/ram_addr.h:337:20: note:
> expected 'long unsigned int *' but argument is of type 'uint64_t *
> {aka long long unsigned int *}'
>  static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned
> long *bitmap,
>                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> ../../hw/vfio/common.c: In function 'vfio_get_dirty_bitmap':
> ../../hw/vfio/common.c:1008:44: error: passing argument 1 of
> 'cpu_physical_memory_set_dirty_lebitmap' from incompatible pointer
> type [-Werror=incompatible-pointer-types]
>      cpu_physical_memory_set_dirty_lebitmap((uint64_t *)range->bitmap.data,
>                                             ^
> In file included from ../../hw/vfio/common.c:32:0:
> /home/peter.maydell/qemu/include/exec/ram_addr.h:337:20: note:
> expected 'long unsigned int *' but argument is of type 'uint64_t *
> {aka long long unsigned int *}'
>  static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned
> long *bitmap,
>                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Seems that our bitmap in just being incorrectly cast as a uint64_t*
rather than an unsigned long*.

Both fixes being rolled into the next pull request, which I've build 32
and 64 bit and with mingw.  Thanks,

Alex