[PATCH v14 00/21] Initial support for multi-process Qemu

elena.ufimtseva@oracle.com posted 21 patches 3 years, 4 months ago
Test checkpatch passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/cover.1608263017.git.elena.ufimtseva@oracle.com
Maintainers: Igor Mammedov <imammedo@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>, John G Johnson <john.g.johnson@oracle.com>, Jagannathan Raman <jag.raman@oracle.com>, Gerd Hoffmann <kraxel@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, Elena Ufimtseva <elena.ufimtseva@oracle.com>, "Daniel P. Berrangé" <berrange@redhat.com>
There is a newer version of this series
docs/devel/index.rst                      |   1 +
docs/devel/multi-process.rst              | 966 ++++++++++++++++++++++
docs/multi-process.rst                    |  64 ++
configure                                 |  10 +
meson.build                               |   3 +
hw/remote/trace.h                         |   1 +
include/exec/memory.h                     |   2 +
include/exec/ram_addr.h                   |   2 +-
include/hw/pci-host/remote.h              |  31 +
include/hw/pci/pci_ids.h                  |   3 +
include/hw/remote/iohub.h                 |  42 +
include/hw/remote/machine.h               |  40 +
include/hw/remote/memory.h                |  19 +
include/hw/remote/mpqemu-link.h           |  99 +++
include/hw/remote/proxy-memory-listener.h |  28 +
include/hw/remote/proxy.h                 |  52 ++
include/io/channel.h                      |  50 ++
include/qemu/mmap-alloc.h                 |   4 +-
include/qemu/sockets.h                    |   1 +
include/sysemu/iothread.h                 |   6 +
backends/hostmem-memfd.c                  |   2 +-
hw/misc/ivshmem.c                         |   3 +-
hw/pci-host/remote.c                      |  75 ++
hw/remote/iohub.c                         | 119 +++
hw/remote/machine.c                       |  80 ++
hw/remote/memory.c                        |  65 ++
hw/remote/message.c                       | 230 ++++++
hw/remote/mpqemu-link.c                   | 267 ++++++
hw/remote/proxy-memory-listener.c         | 227 +++++
hw/remote/proxy.c                         | 371 +++++++++
hw/remote/remote-obj.c                    | 194 +++++
io/channel.c                              |  88 +-
iothread.c                                |   6 +
softmmu/memory.c                          |   3 +-
softmmu/physmem.c                         |  11 +-
stubs/monitor.c                           |   2 +-
tests/test-util-sockets.c                 |   2 +-
util/mmap-alloc.c                         |   7 +-
util/oslib-posix.c                        |   2 +-
util/qemu-sockets.c                       |  18 +-
Kconfig.host                              |   4 +
MAINTAINERS                               |  24 +
hw/Kconfig                                |   1 +
hw/meson.build                            |   1 +
hw/pci-host/Kconfig                       |   3 +
hw/pci-host/meson.build                   |   1 +
hw/remote/Kconfig                         |   4 +
hw/remote/meson.build                     |  13 +
hw/remote/trace-events                    |   4 +
49 files changed, 3202 insertions(+), 49 deletions(-)
create mode 100644 docs/devel/multi-process.rst
create mode 100644 docs/multi-process.rst
create mode 100644 hw/remote/trace.h
create mode 100644 include/hw/pci-host/remote.h
create mode 100644 include/hw/remote/iohub.h
create mode 100644 include/hw/remote/machine.h
create mode 100644 include/hw/remote/memory.h
create mode 100644 include/hw/remote/mpqemu-link.h
create mode 100644 include/hw/remote/proxy-memory-listener.h
create mode 100644 include/hw/remote/proxy.h
create mode 100644 hw/pci-host/remote.c
create mode 100644 hw/remote/iohub.c
create mode 100644 hw/remote/machine.c
create mode 100644 hw/remote/memory.c
create mode 100644 hw/remote/message.c
create mode 100644 hw/remote/mpqemu-link.c
create mode 100644 hw/remote/proxy-memory-listener.c
create mode 100644 hw/remote/proxy.c
create mode 100644 hw/remote/remote-obj.c
create mode 100644 hw/remote/Kconfig
create mode 100644 hw/remote/meson.build
create mode 100644 hw/remote/trace-events
[PATCH v14 00/21] Initial support for multi-process Qemu
Posted by elena.ufimtseva@oracle.com 3 years, 4 months ago
From: Elena Ufimtseva <elena.ufimtseva@oracle.com>

Hi

This is the v14 of the patchset. Thank you very much for reviewing v13 and
sharing your feedback.

We have addressed all the comments from the v13 review with changelog below:

- [PATCH v14 05/21] multi-process: Add config option for multi-process QEMU
  add config option to disable the feature (--disable-multiprocess).

- [PATCH v14 08/21] multi-process: add qio channel write function
- [PATCH v14 09/21] multi-process: add qio channel read function
  refactor code in qio channel and split in two for read and write.

- [PATCH v14 04/21] socket: export socket_get_fd() function
  export socket_get_fd() and use in remote process with some
  modifications.

In addition, we will be sending a follow-up patch with the acceptance
test that can be used to verify that this feature is working correctly.

From the v13 review few items are TODOs:
- Prefix log messages with PID in the logging subsystem.
- Refactor the MemoryListener code in vfio-user and multi-process code
  to avoid logic duplication.

To touch upon the history of this project, we posted the Proof Of Concept
patches before the BoF session in 2018. Subsequently, we have posted 13 
versions on the qemu-devel mailing list. You can find them by following the
links below ([1] - [13]). Following people contributed to the design and
implementation of this project:
Jagannathan Raman <jag.raman@oracle.com>
Elena Ufimtseva <elena.ufimtseva@oracle.com>
John G Johnson <john.g.johnson@oracle.com>
Stefan Hajnoczi <stefanha@redhat.com>
Konrad Wilk <konrad.wilk@oracle.com>
Kanth Ghatraju <kanth.ghatraju@oracle.com>

We would like to thank the QEMU community for your feedback in the
design and implementation of this project. Qemu wiki page:
https://wiki.qemu.org/Features/MultiProcessQEMU

For the full concept writeup about QEMU multi-process, please
refer to docs/devel/qemu-multiprocess.rst. Also, see
docs/qemu-multiprocess.txt for usage information.

Thank you for reviewing these series.

[POC]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg566538.html
[1]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg602285.html
[2]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg624877.html
[3]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg642000.html
[4]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg655118.html
[5]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg682429.html
[6]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg697484.html
[7]: https://patchew.org/QEMU/cover.1593273671.git.elena.ufimtseva@oracle.com/
[8]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg727007.html
[9]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg734275.html
[10]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg747638.html
[11]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg750972.html
[12]: https://patchew.org/QEMU/cover.1606853298.git.jag.raman@oracle.com/
[13]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg766825.html

Elena Ufimtseva (8):
  multi-process: add configure and usage information
  multi-process: add qio channel write function
  multi-process: add qio channel read function
  multi-process: define MPQemuMsg format and transmission functions
  multi-process: introduce proxy object
  multi-process: add proxy communication functions
  multi-process: Forward PCI config space acceses to the remote process
  multi-process: perform device reset in the remote process

Jagannathan Raman (12):
  memory: alloc RAM from file at offset
  socket: export socket_get_fd() function
  multi-process: Add config option for multi-process QEMU
  multi-process: setup PCI host bridge for remote device
  multi-process: setup a machine object for remote device process
  multi-process: Initialize message handler in remote device
  multi-process: Associate fd of a PCIDevice with its object
  multi-process: setup memory manager for remote device
  multi-process: PCI BAR read/write handling for proxy & remote
    endpoints
  multi-process: Synchronize remote memory
  multi-process: create IOHUB object to handle irq
  multi-process: Retrieve PCI info from remote process

John G Johnson (1):
  multi-process: add the concept description to
    docs/devel/qemu-multiprocess

 docs/devel/index.rst                      |   1 +
 docs/devel/multi-process.rst              | 966 ++++++++++++++++++++++
 docs/multi-process.rst                    |  64 ++
 configure                                 |  10 +
 meson.build                               |   3 +
 hw/remote/trace.h                         |   1 +
 include/exec/memory.h                     |   2 +
 include/exec/ram_addr.h                   |   2 +-
 include/hw/pci-host/remote.h              |  31 +
 include/hw/pci/pci_ids.h                  |   3 +
 include/hw/remote/iohub.h                 |  42 +
 include/hw/remote/machine.h               |  40 +
 include/hw/remote/memory.h                |  19 +
 include/hw/remote/mpqemu-link.h           |  99 +++
 include/hw/remote/proxy-memory-listener.h |  28 +
 include/hw/remote/proxy.h                 |  52 ++
 include/io/channel.h                      |  50 ++
 include/qemu/mmap-alloc.h                 |   4 +-
 include/qemu/sockets.h                    |   1 +
 include/sysemu/iothread.h                 |   6 +
 backends/hostmem-memfd.c                  |   2 +-
 hw/misc/ivshmem.c                         |   3 +-
 hw/pci-host/remote.c                      |  75 ++
 hw/remote/iohub.c                         | 119 +++
 hw/remote/machine.c                       |  80 ++
 hw/remote/memory.c                        |  65 ++
 hw/remote/message.c                       | 230 ++++++
 hw/remote/mpqemu-link.c                   | 267 ++++++
 hw/remote/proxy-memory-listener.c         | 227 +++++
 hw/remote/proxy.c                         | 371 +++++++++
 hw/remote/remote-obj.c                    | 194 +++++
 io/channel.c                              |  88 +-
 iothread.c                                |   6 +
 softmmu/memory.c                          |   3 +-
 softmmu/physmem.c                         |  11 +-
 stubs/monitor.c                           |   2 +-
 tests/test-util-sockets.c                 |   2 +-
 util/mmap-alloc.c                         |   7 +-
 util/oslib-posix.c                        |   2 +-
 util/qemu-sockets.c                       |  18 +-
 Kconfig.host                              |   4 +
 MAINTAINERS                               |  24 +
 hw/Kconfig                                |   1 +
 hw/meson.build                            |   1 +
 hw/pci-host/Kconfig                       |   3 +
 hw/pci-host/meson.build                   |   1 +
 hw/remote/Kconfig                         |   4 +
 hw/remote/meson.build                     |  13 +
 hw/remote/trace-events                    |   4 +
 49 files changed, 3202 insertions(+), 49 deletions(-)
 create mode 100644 docs/devel/multi-process.rst
 create mode 100644 docs/multi-process.rst
 create mode 100644 hw/remote/trace.h
 create mode 100644 include/hw/pci-host/remote.h
 create mode 100644 include/hw/remote/iohub.h
 create mode 100644 include/hw/remote/machine.h
 create mode 100644 include/hw/remote/memory.h
 create mode 100644 include/hw/remote/mpqemu-link.h
 create mode 100644 include/hw/remote/proxy-memory-listener.h
 create mode 100644 include/hw/remote/proxy.h
 create mode 100644 hw/pci-host/remote.c
 create mode 100644 hw/remote/iohub.c
 create mode 100644 hw/remote/machine.c
 create mode 100644 hw/remote/memory.c
 create mode 100644 hw/remote/message.c
 create mode 100644 hw/remote/mpqemu-link.c
 create mode 100644 hw/remote/proxy-memory-listener.c
 create mode 100644 hw/remote/proxy.c
 create mode 100644 hw/remote/remote-obj.c
 create mode 100644 hw/remote/Kconfig
 create mode 100644 hw/remote/meson.build
 create mode 100644 hw/remote/trace-events

-- 
2.25.GIT


Re: [PATCH v14 00/21] Initial support for multi-process Qemu
Posted by Elena Ufimtseva 3 years, 4 months ago
On Thu, Dec 17, 2020 at 07:57:47PM -0800, elena.ufimtseva@oracle.com wrote:
> From: Elena Ufimtseva <elena.ufimtseva@oracle.com>
> 
> Hi
>

For some reason the patchset was not sent fully and only few patches
made trough.
I am resending the series with prefix in the subject "[PATCH v14 RESEND... "

I am sorry for this inconvenience.

Elena 
> This is the v14 of the patchset. Thank you very much for reviewing v13 and
> sharing your feedback.
> 
> We have addressed all the comments from the v13 review with changelog below:
> 
> - [PATCH v14 05/21] multi-process: Add config option for multi-process QEMU
>   add config option to disable the feature (--disable-multiprocess).
> 
> - [PATCH v14 08/21] multi-process: add qio channel write function
> - [PATCH v14 09/21] multi-process: add qio channel read function
>   refactor code in qio channel and split in two for read and write.
> 
> - [PATCH v14 04/21] socket: export socket_get_fd() function
>   export socket_get_fd() and use in remote process with some
>   modifications.
> 
> In addition, we will be sending a follow-up patch with the acceptance
> test that can be used to verify that this feature is working correctly.
> 
> From the v13 review few items are TODOs:
> - Prefix log messages with PID in the logging subsystem.
> - Refactor the MemoryListener code in vfio-user and multi-process code
>   to avoid logic duplication.
> 
> To touch upon the history of this project, we posted the Proof Of Concept
> patches before the BoF session in 2018. Subsequently, we have posted 13 
> versions on the qemu-devel mailing list. You can find them by following the
> links below ([1] - [13]). Following people contributed to the design and
> implementation of this project:
> Jagannathan Raman <jag.raman@oracle.com>
> Elena Ufimtseva <elena.ufimtseva@oracle.com>
> John G Johnson <john.g.johnson@oracle.com>
> Stefan Hajnoczi <stefanha@redhat.com>
> Konrad Wilk <konrad.wilk@oracle.com>
> Kanth Ghatraju <kanth.ghatraju@oracle.com>
> 
> We would like to thank the QEMU community for your feedback in the
> design and implementation of this project. Qemu wiki page:
> https://wiki.qemu.org/Features/MultiProcessQEMU
> 
> For the full concept writeup about QEMU multi-process, please
> refer to docs/devel/qemu-multiprocess.rst. Also, see
> docs/qemu-multiprocess.txt for usage information.
> 
> Thank you for reviewing these series.
> 
> [POC]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg566538.html
> [1]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg602285.html
> [2]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg624877.html
> [3]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg642000.html
> [4]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg655118.html
> [5]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg682429.html
> [6]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg697484.html
> [7]: https://patchew.org/QEMU/cover.1593273671.git.elena.ufimtseva@oracle.com/
> [8]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg727007.html
> [9]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg734275.html
> [10]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg747638.html
> [11]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg750972.html
> [12]: https://patchew.org/QEMU/cover.1606853298.git.jag.raman@oracle.com/
> [13]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg766825.html
> 
> Elena Ufimtseva (8):
>   multi-process: add configure and usage information
>   multi-process: add qio channel write function
>   multi-process: add qio channel read function
>   multi-process: define MPQemuMsg format and transmission functions
>   multi-process: introduce proxy object
>   multi-process: add proxy communication functions
>   multi-process: Forward PCI config space acceses to the remote process
>   multi-process: perform device reset in the remote process
> 
> Jagannathan Raman (12):
>   memory: alloc RAM from file at offset
>   socket: export socket_get_fd() function
>   multi-process: Add config option for multi-process QEMU
>   multi-process: setup PCI host bridge for remote device
>   multi-process: setup a machine object for remote device process
>   multi-process: Initialize message handler in remote device
>   multi-process: Associate fd of a PCIDevice with its object
>   multi-process: setup memory manager for remote device
>   multi-process: PCI BAR read/write handling for proxy & remote
>     endpoints
>   multi-process: Synchronize remote memory
>   multi-process: create IOHUB object to handle irq
>   multi-process: Retrieve PCI info from remote process
> 
> John G Johnson (1):
>   multi-process: add the concept description to
>     docs/devel/qemu-multiprocess
> 
>  docs/devel/index.rst                      |   1 +
>  docs/devel/multi-process.rst              | 966 ++++++++++++++++++++++
>  docs/multi-process.rst                    |  64 ++
>  configure                                 |  10 +
>  meson.build                               |   3 +
>  hw/remote/trace.h                         |   1 +
>  include/exec/memory.h                     |   2 +
>  include/exec/ram_addr.h                   |   2 +-
>  include/hw/pci-host/remote.h              |  31 +
>  include/hw/pci/pci_ids.h                  |   3 +
>  include/hw/remote/iohub.h                 |  42 +
>  include/hw/remote/machine.h               |  40 +
>  include/hw/remote/memory.h                |  19 +
>  include/hw/remote/mpqemu-link.h           |  99 +++
>  include/hw/remote/proxy-memory-listener.h |  28 +
>  include/hw/remote/proxy.h                 |  52 ++
>  include/io/channel.h                      |  50 ++
>  include/qemu/mmap-alloc.h                 |   4 +-
>  include/qemu/sockets.h                    |   1 +
>  include/sysemu/iothread.h                 |   6 +
>  backends/hostmem-memfd.c                  |   2 +-
>  hw/misc/ivshmem.c                         |   3 +-
>  hw/pci-host/remote.c                      |  75 ++
>  hw/remote/iohub.c                         | 119 +++
>  hw/remote/machine.c                       |  80 ++
>  hw/remote/memory.c                        |  65 ++
>  hw/remote/message.c                       | 230 ++++++
>  hw/remote/mpqemu-link.c                   | 267 ++++++
>  hw/remote/proxy-memory-listener.c         | 227 +++++
>  hw/remote/proxy.c                         | 371 +++++++++
>  hw/remote/remote-obj.c                    | 194 +++++
>  io/channel.c                              |  88 +-
>  iothread.c                                |   6 +
>  softmmu/memory.c                          |   3 +-
>  softmmu/physmem.c                         |  11 +-
>  stubs/monitor.c                           |   2 +-
>  tests/test-util-sockets.c                 |   2 +-
>  util/mmap-alloc.c                         |   7 +-
>  util/oslib-posix.c                        |   2 +-
>  util/qemu-sockets.c                       |  18 +-
>  Kconfig.host                              |   4 +
>  MAINTAINERS                               |  24 +
>  hw/Kconfig                                |   1 +
>  hw/meson.build                            |   1 +
>  hw/pci-host/Kconfig                       |   3 +
>  hw/pci-host/meson.build                   |   1 +
>  hw/remote/Kconfig                         |   4 +
>  hw/remote/meson.build                     |  13 +
>  hw/remote/trace-events                    |   4 +
>  49 files changed, 3202 insertions(+), 49 deletions(-)
>  create mode 100644 docs/devel/multi-process.rst
>  create mode 100644 docs/multi-process.rst
>  create mode 100644 hw/remote/trace.h
>  create mode 100644 include/hw/pci-host/remote.h
>  create mode 100644 include/hw/remote/iohub.h
>  create mode 100644 include/hw/remote/machine.h
>  create mode 100644 include/hw/remote/memory.h
>  create mode 100644 include/hw/remote/mpqemu-link.h
>  create mode 100644 include/hw/remote/proxy-memory-listener.h
>  create mode 100644 include/hw/remote/proxy.h
>  create mode 100644 hw/pci-host/remote.c
>  create mode 100644 hw/remote/iohub.c
>  create mode 100644 hw/remote/machine.c
>  create mode 100644 hw/remote/memory.c
>  create mode 100644 hw/remote/message.c
>  create mode 100644 hw/remote/mpqemu-link.c
>  create mode 100644 hw/remote/proxy-memory-listener.c
>  create mode 100644 hw/remote/proxy.c
>  create mode 100644 hw/remote/remote-obj.c
>  create mode 100644 hw/remote/Kconfig
>  create mode 100644 hw/remote/meson.build
>  create mode 100644 hw/remote/trace-events
> 
> -- 
> 2.25.GIT
> 

Re: [PATCH v14 00/21] Initial support for multi-process Qemu
Posted by Stefan Hajnoczi 3 years, 4 months ago
On Thu, Dec 17, 2020 at 07:57:47PM -0800, elena.ufimtseva@oracle.com wrote:
> From: Elena Ufimtseva <elena.ufimtseva@oracle.com>
> 
> Hi
> 
> This is the v14 of the patchset. Thank you very much for reviewing v13 and
> sharing your feedback.

Please see Marc-André's comment and my comment about a possible
fd/memory leak.

Stefan