[v21] Initial support for multi-process Qemu

[PATCH v21 00/20] Initial support for multi-process Qemu

Posted by Jagannathan Raman 3 years, 3 months ago

Hi,

This is the v21 of the patchset. This version has the following changes:

[PATCH v18 08/20] io: add qio_channel_readv_full_all_eof &
               qio_channel_readv_full_all helpers
  - Separates out the error handling for (len = 0) & (len < 0)
  - (len = 0) does not necessarily mean EOF. It could also be
    a case where only FDs are read, without any data with more data
    possibly on the way. Addresses this scenario as well, as was
    pointed out in the feedback from last time.

To touch upon the history of this project, we posted the Proof Of Concept
patches before the BoF session in 2018. Subsequently, we have posted 20
versions on the qemu-devel mailing list. You can find them by following
the links below ([1] - [20]). Following people contributed to the design and
implementation of this project:
Jagannathan Raman <jag.raman@oracle.com>
Elena Ufimtseva <elena.ufimtseva@oracle.com>
John G Johnson <john.g.johnson@oracle.com>
Stefan Hajnoczi <stefanha@redhat.com>
Konrad Wilk <konrad.wilk@oracle.com>
Kanth Ghatraju <kanth.ghatraju@oracle.com>

We would like to thank you for your feedback in the
design and implementation of this project. Qemu wiki page:
https://wiki.qemu.org/Features/MultiProcessQEMU

For the full concept writeup about QEMU multi-process, please
refer to docs/devel/qemu-multiprocess.rst. Also, see
docs/qemu-multiprocess.txt for usage information.

Thank you for reviewing this series!

[POC]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg566538.html
[1]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg602285.html
[2]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg624877.html
[3]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg642000.html
[4]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg655118.html
[5]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg682429.html
[6]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg697484.html
[7]: https://patchew.org/QEMU/cover.1593273671.git.elena.ufimtseva@oracle.com/
[8]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg727007.html
[9]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg734275.html
[10]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg747638.html
[11]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg750972.html
[12]: https://patchew.org/QEMU/cover.1606853298.git.jag.raman@oracle.com/
[13]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg766825.html
[14]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg768376.html
[15]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg769178.html
[16]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg771455.html
[17]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg772512.html
[18]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg772602.html
[19]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg772980.html
[20]: https://patchew.org/QEMU/cover.1611081587.git.jag.raman@oracle.com/

Elena Ufimtseva (8):
  multi-process: add configure and usage information
  io: add qio_channel_writev_full_all helper
  io: add qio_channel_readv_full_all_eof & qio_channel_readv_full_all
    helpers
  multi-process: define MPQemuMsg format and transmission functions
  multi-process: introduce proxy object
  multi-process: add proxy communication functions
  multi-process: Forward PCI config space acceses to the remote process
  multi-process: perform device reset in the remote process

Jagannathan Raman (11):
  memory: alloc RAM from file at offset
  multi-process: Add config option for multi-process QEMU
  multi-process: setup PCI host bridge for remote device
  multi-process: setup a machine object for remote device process
  multi-process: Initialize message handler in remote device
  multi-process: Associate fd of a PCIDevice with its object
  multi-process: setup memory manager for remote device
  multi-process: PCI BAR read/write handling for proxy & remote
    endpoints
  multi-process: Synchronize remote memory
  multi-process: create IOHUB object to handle irq
  multi-process: Retrieve PCI info from remote process

John G Johnson (1):
  multi-process: add the concept description to
    docs/devel/qemu-multiprocess

 docs/devel/index.rst                      |   1 +
 docs/devel/multi-process.rst              | 966 ++++++++++++++++++++++++++++++
 docs/multi-process.rst                    |  64 ++
 configure                                 |  10 +
 meson.build                               |   5 +-
 hw/remote/trace.h                         |   1 +
 include/exec/memory.h                     |   2 +
 include/exec/ram_addr.h                   |   2 +-
 include/hw/pci-host/remote.h              |  30 +
 include/hw/pci/pci_ids.h                  |   3 +
 include/hw/remote/iohub.h                 |  42 ++
 include/hw/remote/machine.h               |  38 ++
 include/hw/remote/memory.h                |  19 +
 include/hw/remote/mpqemu-link.h           |  99 +++
 include/hw/remote/proxy-memory-listener.h |  28 +
 include/hw/remote/proxy.h                 |  48 ++
 include/io/channel.h                      |  78 +++
 include/qemu/mmap-alloc.h                 |   4 +-
 include/sysemu/iothread.h                 |   6 +
 backends/hostmem-memfd.c                  |   2 +-
 hw/misc/ivshmem.c                         |   3 +-
 hw/pci-host/remote.c                      |  75 +++
 hw/remote/iohub.c                         | 119 ++++
 hw/remote/machine.c                       |  80 +++
 hw/remote/memory.c                        |  65 ++
 hw/remote/message.c                       | 230 +++++++
 hw/remote/mpqemu-link.c                   | 267 +++++++++
 hw/remote/proxy-memory-listener.c         | 227 +++++++
 hw/remote/proxy.c                         | 379 ++++++++++++
 hw/remote/remote-obj.c                    | 203 +++++++
 io/channel.c                              | 116 +++-
 iothread.c                                |   6 +
 softmmu/memory.c                          |   3 +-
 softmmu/physmem.c                         |  11 +-
 util/mmap-alloc.c                         |   7 +-
 util/oslib-posix.c                        |   2 +-
 Kconfig.host                              |   4 +
 MAINTAINERS                               |  24 +
 hw/Kconfig                                |   1 +
 hw/meson.build                            |   1 +
 hw/pci-host/Kconfig                       |   3 +
 hw/pci-host/meson.build                   |   1 +
 hw/remote/Kconfig                         |   4 +
 hw/remote/meson.build                     |  13 +
 hw/remote/trace-events                    |   4 +
 45 files changed, 3261 insertions(+), 35 deletions(-)
 create mode 100644 docs/devel/multi-process.rst
 create mode 100644 docs/multi-process.rst
 create mode 100644 hw/remote/trace.h
 create mode 100644 include/hw/pci-host/remote.h
 create mode 100644 include/hw/remote/iohub.h
 create mode 100644 include/hw/remote/machine.h
 create mode 100644 include/hw/remote/memory.h
 create mode 100644 include/hw/remote/mpqemu-link.h
 create mode 100644 include/hw/remote/proxy-memory-listener.h
 create mode 100644 include/hw/remote/proxy.h
 create mode 100644 hw/pci-host/remote.c
 create mode 100644 hw/remote/iohub.c
 create mode 100644 hw/remote/machine.c
 create mode 100644 hw/remote/memory.c
 create mode 100644 hw/remote/message.c
 create mode 100644 hw/remote/mpqemu-link.c
 create mode 100644 hw/remote/proxy-memory-listener.c
 create mode 100644 hw/remote/proxy.c
 create mode 100644 hw/remote/remote-obj.c
 create mode 100644 hw/remote/Kconfig
 create mode 100644 hw/remote/meson.build
 create mode 100644 hw/remote/trace-events

-- 
1.8.3.1

Re: [PATCH v21 00/20] Initial support for multi-process Qemu

Posted by Stefan Hajnoczi 3 years, 2 months ago

On Fri, Jan 29, 2021 at 11:46:01AM -0500, Jagannathan Raman wrote:
> This is the v21 of the patchset. This version has the following changes:

The docs/multi-process.rst file caused Sphinx to fail with "document
isn't included in any toctree". I moved it into the system emulator
documentation to resolve this. The new path is
docs/system/multi-process.rst.

Thanks, applied to my block tree:
https://gitlab.com/stefanha/qemu/commits/block

Stefan

Re: [PATCH v21 00/20] Initial support for multi-process Qemu

Posted by Elena Ufimtseva 3 years, 2 months ago

On Wed, Feb 03, 2021 at 04:11:50PM +0000, Stefan Hajnoczi wrote:
> On Fri, Jan 29, 2021 at 11:46:01AM -0500, Jagannathan Raman wrote:
> > This is the v21 of the patchset. This version has the following changes:
> 
> The docs/multi-process.rst file caused Sphinx to fail with "document
> isn't included in any toctree". I moved it into the system emulator
> documentation to resolve this. The new path is
> docs/system/multi-process.rst.
>

Hi Stefan

Ah, we did not --enable-docs and .travis.yml also disables them.
Will include into the tests we do for submission.
 
> Thanks, applied to my block tree:
> https://gitlab.com/stefanha/qemu/commits/block
>

Thank you Stefan!

Elena 
> Stefan

Re: [PATCH v21 00/20] Initial support for multi-process Qemu

Posted by Daniel P. Berrangé 3 years, 2 months ago

On Wed, Feb 03, 2021 at 12:02:05PM -0800, Elena Ufimtseva wrote:
> On Wed, Feb 03, 2021 at 04:11:50PM +0000, Stefan Hajnoczi wrote:
> > On Fri, Jan 29, 2021 at 11:46:01AM -0500, Jagannathan Raman wrote:
> > > This is the v21 of the patchset. This version has the following changes:
> > 
> > The docs/multi-process.rst file caused Sphinx to fail with "document
> > isn't included in any toctree". I moved it into the system emulator
> > documentation to resolve this. The new path is
> > docs/system/multi-process.rst.
> >
> 
> Hi Stefan
> 
> Ah, we did not --enable-docs and .travis.yml also disables them.
> Will include into the tests we do for submission.

FWIW if you're relying on travis for testing before submission that's
something you'll want to change real soon. Travis has discontinued its
free service and turned it into a one time free trial. So all QEMU
primary CI is moving to GitLab CI now and we'll be deleting remaining
travis CI jobs any day now.  All you need todo is have a fork of the
main qemu repo on gitlab, and when you push to your forkk the CI
pipeline will run in GitLab.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v21 00/20] Initial support for multi-process Qemu

Posted by elena 3 years, 2 months ago

On Thu, Feb 04, 2021 at 09:51:45AM +0000, Daniel P. Berrangé wrote:
> On Wed, Feb 03, 2021 at 12:02:05PM -0800, Elena Ufimtseva wrote:
> > On Wed, Feb 03, 2021 at 04:11:50PM +0000, Stefan Hajnoczi wrote:
> > > On Fri, Jan 29, 2021 at 11:46:01AM -0500, Jagannathan Raman wrote:
> > > > This is the v21 of the patchset. This version has the following changes:
> > > 
> > > The docs/multi-process.rst file caused Sphinx to fail with "document
> > > isn't included in any toctree". I moved it into the system emulator
> > > documentation to resolve this. The new path is
> > > docs/system/multi-process.rst.
> > >
> > 
> > Hi Stefan
> > 
> > Ah, we did not --enable-docs and .travis.yml also disables them.
> > Will include into the tests we do for submission.
> 
> FWIW if you're relying on travis for testing before submission that's
> something you'll want to change real soon. Travis has discontinued its
> free service and turned it into a one time free trial. So all QEMU
> primary CI is moving to GitLab CI now and we'll be deleting remaining
> travis CI jobs any day now.  All you need todo is have a fork of the
> main qemu repo on gitlab, and when you push to your forkk the CI
> pipeline will run in GitLab.
>

Daniel, thank you for reminding about Travis and GitLab.

Elena
> 
> Regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
>

Re: [PATCH v21 00/20] Initial support for multi-process Qemu

Posted by Thomas Huth 3 years, 1 month ago

On 29/01/2021 17.46, Jagannathan Raman wrote:
> Hi,
> 
> This is the v21 of the patchset. This version has the following changes:
> 
> [PATCH v18 08/20] io: add qio_channel_readv_full_all_eof &
>                 qio_channel_readv_full_all helpers
>    - Separates out the error handling for (len = 0) & (len < 0)
>    - (len = 0) does not necessarily mean EOF. It could also be
>      a case where only FDs are read, without any data with more data
>      possibly on the way. Addresses this scenario as well, as was
>      pointed out in the feedback from last time.
> 
> To touch upon the history of this project, we posted the Proof Of Concept
> patches before the BoF session in 2018. Subsequently, we have posted 20
> versions on the qemu-devel mailing list. You can find them by following
> the links below ([1] - [20]). Following people contributed to the design and
> implementation of this project:
> Jagannathan Raman <jag.raman@oracle.com>
> Elena Ufimtseva <elena.ufimtseva@oracle.com>
> John G Johnson <john.g.johnson@oracle.com>
> Stefan Hajnoczi <stefanha@redhat.com>
> Konrad Wilk <konrad.wilk@oracle.com>
> Kanth Ghatraju <kanth.ghatraju@oracle.com>
> 
> We would like to thank you for your feedback in the
> design and implementation of this project. Qemu wiki page:
> https://wiki.qemu.org/Features/MultiProcessQEMU
> 
> For the full concept writeup about QEMU multi-process, please
> refer to docs/devel/qemu-multiprocess.rst. Also, see
> docs/qemu-multiprocess.txt for usage information.

  Hi!

FWIW, the x-remote machine can be used to crash QEMU:

$ ./qemu-system-i386 -machine x-remote -device piix3-ide
Segmentation fault (core dumped)

... discovered with the scripts/device-crash-test script.

  Thomas

Re: [PATCH v21 00/20] Initial support for multi-process Qemu

Posted by elena 3 years, 1 month ago

On Fri, Mar 12, 2021 at 01:30:24PM +0100, Thomas Huth wrote:
> On 29/01/2021 17.46, Jagannathan Raman wrote:
> > Hi,
> > 
> > This is the v21 of the patchset. This version has the following changes:
> > 
> > [PATCH v18 08/20] io: add qio_channel_readv_full_all_eof &
> >                 qio_channel_readv_full_all helpers
> >    - Separates out the error handling for (len = 0) & (len < 0)
> >    - (len = 0) does not necessarily mean EOF. It could also be
> >      a case where only FDs are read, without any data with more data
> >      possibly on the way. Addresses this scenario as well, as was
> >      pointed out in the feedback from last time.
> > 
> > To touch upon the history of this project, we posted the Proof Of Concept
> > patches before the BoF session in 2018. Subsequently, we have posted 20
> > versions on the qemu-devel mailing list. You can find them by following
> > the links below ([1] - [20]). Following people contributed to the design and
> > implementation of this project:
> > Jagannathan Raman <jag.raman@oracle.com>
> > Elena Ufimtseva <elena.ufimtseva@oracle.com>
> > John G Johnson <john.g.johnson@oracle.com>
> > Stefan Hajnoczi <stefanha@redhat.com>
> > Konrad Wilk <konrad.wilk@oracle.com>
> > Kanth Ghatraju <kanth.ghatraju@oracle.com>
> > 
> > We would like to thank you for your feedback in the
> > design and implementation of this project. Qemu wiki page:
> > https://wiki.qemu.org/Features/MultiProcessQEMU
> > 
> > For the full concept writeup about QEMU multi-process, please
> > refer to docs/devel/qemu-multiprocess.rst. Also, see
> > docs/qemu-multiprocess.txt for usage information.
> 
>  Hi!
> 
> FWIW, the x-remote machine can be used to crash QEMU:
> 
> $ ./qemu-system-i386 -machine x-remote -device piix3-ide
> Segmentation fault (core dumped)
> 
> ... discovered with the scripts/device-crash-test script.
>

Thank you Thomas,

We will have the patch shortly.


Elena
>  Thomas
>