[Qemu-devel] [PATCH v4 00/29] postcopy+vhost-user/shared ram

Dr. David Alan Gilbert (git) posted 29 patches 6 years, 1 month ago
Failed in applying to current master (apply log)
Test checkpatch passed
Test docker-build@min-glib passed
Test docker-mingw@fedora passed
Test docker-quick@centos6 passed
Test ppcbe passed
Test ppcle passed
Test s390x passed
There is a newer version of this series
contrib/libvhost-user/libvhost-user.c | 303 ++++++++++++++++++++++++-
contrib/libvhost-user/libvhost-user.h |  11 +
docs/devel/migration.rst              |  41 ++++
docs/interop/vhost-user.txt           |  51 +++++
exec.c                                |  86 +++++--
hw/virtio/trace-events                |  16 +-
hw/virtio/vhost-user.c                | 411 +++++++++++++++++++++++++++++++++-
hw/virtio/vhost.c                     |  66 +++++-
include/exec/cpu-common.h             |   4 +
migration/migration.c                 |   6 +
migration/migration.h                 |   4 +
migration/postcopy-ram.c              | 354 +++++++++++++++++++++++------
migration/postcopy-ram.h              |  69 ++++++
migration/ram.c                       |   5 +
migration/ram.h                       |   1 +
migration/savevm.c                    |  13 ++
migration/trace-events                |   6 +
trace-events                          |   3 +-
vl.c                                  |   2 +
19 files changed, 1354 insertions(+), 98 deletions(-)
[Qemu-devel] [PATCH v4 00/29] postcopy+vhost-user/shared ram
Posted by Dr. David Alan Gilbert (git) 6 years, 1 month ago
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

  This set enables postcopy migration with shared memory to a vhost user process.
It's based off current head.

  Testing is mostly performed with dpdk, with corresponding modifications by
Maxime.

  v4 is mostly just fixes from comments received during review of v3,
and the normal updating of all the virtio enum's to catch up with other
things getting in before us.

Dave


Dr. David Alan Gilbert (29):
  migrate: Update ram_block_discard_range for shared
  qemu_ram_block_host_offset
  postcopy: use UFFDIO_ZEROPAGE only when available
  postcopy: Add notifier chain
  postcopy: Add vhost-user flag for postcopy and check it
  vhost-user: Add 'VHOST_USER_POSTCOPY_ADVISE' message
  libvhost-user: Support sending fds back to qemu
  libvhost-user: Open userfaultfd
  postcopy: Allow registering of fd handler
  vhost+postcopy: Register shared ufd with postcopy
  vhost+postcopy: Transmit 'listen' to client
  postcopy+vhost-user: Split set_mem_table for postcopy
  migration/ram: ramblock_recv_bitmap_test_byte_offset
  libvhost-user+postcopy: Register new regions with the ufd
  vhost+postcopy: Send address back to qemu
  vhost+postcopy: Stash RAMBlock and offset
  vhost+postcopy: Send requests to source for shared pages
  vhost+postcopy: Resolve client address
  postcopy: helper for waking shared
  postcopy: postcopy_notify_shared_wake
  vhost+postcopy: Add vhost waker
  vhost+postcopy: Call wakeups
  libvhost-user: mprotect & madvises for postcopy
  vhost-user: Add VHOST_USER_POSTCOPY_END message
  vhost+postcopy: Wire up POSTCOPY_END notify
  vhost: Huge page align and merge
  postcopy: Allow shared memory
  libvhost-user: Claim support for postcopy
  postcopy shared docs

 contrib/libvhost-user/libvhost-user.c | 303 ++++++++++++++++++++++++-
 contrib/libvhost-user/libvhost-user.h |  11 +
 docs/devel/migration.rst              |  41 ++++
 docs/interop/vhost-user.txt           |  51 +++++
 exec.c                                |  86 +++++--
 hw/virtio/trace-events                |  16 +-
 hw/virtio/vhost-user.c                | 411 +++++++++++++++++++++++++++++++++-
 hw/virtio/vhost.c                     |  66 +++++-
 include/exec/cpu-common.h             |   4 +
 migration/migration.c                 |   6 +
 migration/migration.h                 |   4 +
 migration/postcopy-ram.c              | 354 +++++++++++++++++++++++------
 migration/postcopy-ram.h              |  69 ++++++
 migration/ram.c                       |   5 +
 migration/ram.h                       |   1 +
 migration/savevm.c                    |  13 ++
 migration/trace-events                |   6 +
 trace-events                          |   3 +-
 vl.c                                  |   2 +
 19 files changed, 1354 insertions(+), 98 deletions(-)

-- 
2.14.3


Re: [Qemu-devel] [PATCH v4 00/29] postcopy+vhost-user/shared ram
Posted by Dr. David Alan Gilbert 6 years, 1 month ago
* Dr. David Alan Gilbert (git) (dgilbert@redhat.com) wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> 
>   This set enables postcopy migration with shared memory to a vhost user process.
> It's based off current head.
> 
>   Testing is mostly performed with dpdk, with corresponding modifications by
> Maxime.

Maxime's dpdk world is available here:
  https://gitlab.com/mcoquelin/dpdk-next-virtio.git
on the
   postcopy_proto_v4

branch.


Dave

>   v4 is mostly just fixes from comments received during review of v3,
> and the normal updating of all the virtio enum's to catch up with other
> things getting in before us.
> 
> Dave
> 
> 
> Dr. David Alan Gilbert (29):
>   migrate: Update ram_block_discard_range for shared
>   qemu_ram_block_host_offset
>   postcopy: use UFFDIO_ZEROPAGE only when available
>   postcopy: Add notifier chain
>   postcopy: Add vhost-user flag for postcopy and check it
>   vhost-user: Add 'VHOST_USER_POSTCOPY_ADVISE' message
>   libvhost-user: Support sending fds back to qemu
>   libvhost-user: Open userfaultfd
>   postcopy: Allow registering of fd handler
>   vhost+postcopy: Register shared ufd with postcopy
>   vhost+postcopy: Transmit 'listen' to client
>   postcopy+vhost-user: Split set_mem_table for postcopy
>   migration/ram: ramblock_recv_bitmap_test_byte_offset
>   libvhost-user+postcopy: Register new regions with the ufd
>   vhost+postcopy: Send address back to qemu
>   vhost+postcopy: Stash RAMBlock and offset
>   vhost+postcopy: Send requests to source for shared pages
>   vhost+postcopy: Resolve client address
>   postcopy: helper for waking shared
>   postcopy: postcopy_notify_shared_wake
>   vhost+postcopy: Add vhost waker
>   vhost+postcopy: Call wakeups
>   libvhost-user: mprotect & madvises for postcopy
>   vhost-user: Add VHOST_USER_POSTCOPY_END message
>   vhost+postcopy: Wire up POSTCOPY_END notify
>   vhost: Huge page align and merge
>   postcopy: Allow shared memory
>   libvhost-user: Claim support for postcopy
>   postcopy shared docs
> 
>  contrib/libvhost-user/libvhost-user.c | 303 ++++++++++++++++++++++++-
>  contrib/libvhost-user/libvhost-user.h |  11 +
>  docs/devel/migration.rst              |  41 ++++
>  docs/interop/vhost-user.txt           |  51 +++++
>  exec.c                                |  86 +++++--
>  hw/virtio/trace-events                |  16 +-
>  hw/virtio/vhost-user.c                | 411 +++++++++++++++++++++++++++++++++-
>  hw/virtio/vhost.c                     |  66 +++++-
>  include/exec/cpu-common.h             |   4 +
>  migration/migration.c                 |   6 +
>  migration/migration.h                 |   4 +
>  migration/postcopy-ram.c              | 354 +++++++++++++++++++++++------
>  migration/postcopy-ram.h              |  69 ++++++
>  migration/ram.c                       |   5 +
>  migration/ram.h                       |   1 +
>  migration/savevm.c                    |  13 ++
>  migration/trace-events                |   6 +
>  trace-events                          |   3 +-
>  vl.c                                  |   2 +
>  19 files changed, 1354 insertions(+), 98 deletions(-)
> 
> -- 
> 2.14.3
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH v4 00/29] postcopy+vhost-user/shared ram
Posted by Michael S. Tsirkin 5 years, 7 months ago
On Thu, Mar 08, 2018 at 07:57:42PM +0000, Dr. David Alan Gilbert (git) wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> 
>   This set enables postcopy migration with shared memory to a vhost user process.
> It's based off current head.
> 
>   Testing is mostly performed with dpdk, with corresponding modifications by
> Maxime.
> 
>   v4 is mostly just fixes from comments received during review of v3,
> and the normal updating of all the virtio enum's to catch up with other
> things getting in before us.
> 
> Dave

For vhost bits:

Acked-by: Michael S. Tsirkin <mst@redhat.com>

So how is all this going to be merged?
If it's through my tree pls let me know.
I'm fine with another tree though.

> 
> Dr. David Alan Gilbert (29):
>   migrate: Update ram_block_discard_range for shared
>   qemu_ram_block_host_offset
>   postcopy: use UFFDIO_ZEROPAGE only when available
>   postcopy: Add notifier chain
>   postcopy: Add vhost-user flag for postcopy and check it
>   vhost-user: Add 'VHOST_USER_POSTCOPY_ADVISE' message
>   libvhost-user: Support sending fds back to qemu
>   libvhost-user: Open userfaultfd
>   postcopy: Allow registering of fd handler
>   vhost+postcopy: Register shared ufd with postcopy
>   vhost+postcopy: Transmit 'listen' to client
>   postcopy+vhost-user: Split set_mem_table for postcopy
>   migration/ram: ramblock_recv_bitmap_test_byte_offset
>   libvhost-user+postcopy: Register new regions with the ufd
>   vhost+postcopy: Send address back to qemu
>   vhost+postcopy: Stash RAMBlock and offset
>   vhost+postcopy: Send requests to source for shared pages
>   vhost+postcopy: Resolve client address
>   postcopy: helper for waking shared
>   postcopy: postcopy_notify_shared_wake
>   vhost+postcopy: Add vhost waker
>   vhost+postcopy: Call wakeups
>   libvhost-user: mprotect & madvises for postcopy
>   vhost-user: Add VHOST_USER_POSTCOPY_END message
>   vhost+postcopy: Wire up POSTCOPY_END notify
>   vhost: Huge page align and merge
>   postcopy: Allow shared memory
>   libvhost-user: Claim support for postcopy
>   postcopy shared docs
> 
>  contrib/libvhost-user/libvhost-user.c | 303 ++++++++++++++++++++++++-
>  contrib/libvhost-user/libvhost-user.h |  11 +
>  docs/devel/migration.rst              |  41 ++++
>  docs/interop/vhost-user.txt           |  51 +++++
>  exec.c                                |  86 +++++--
>  hw/virtio/trace-events                |  16 +-
>  hw/virtio/vhost-user.c                | 411 +++++++++++++++++++++++++++++++++-
>  hw/virtio/vhost.c                     |  66 +++++-
>  include/exec/cpu-common.h             |   4 +
>  migration/migration.c                 |   6 +
>  migration/migration.h                 |   4 +
>  migration/postcopy-ram.c              | 354 +++++++++++++++++++++++------
>  migration/postcopy-ram.h              |  69 ++++++
>  migration/ram.c                       |   5 +
>  migration/ram.h                       |   1 +
>  migration/savevm.c                    |  13 ++
>  migration/trace-events                |   6 +
>  trace-events                          |   3 +-
>  vl.c                                  |   2 +
>  19 files changed, 1354 insertions(+), 98 deletions(-)
> 
> -- 
> 2.14.3

Re: [Qemu-devel] [PATCH v4 00/29] postcopy+vhost-user/shared ram
Posted by Dr. David Alan Gilbert 5 years, 7 months ago
* Michael S. Tsirkin (mst@redhat.com) wrote:
> On Thu, Mar 08, 2018 at 07:57:42PM +0000, Dr. David Alan Gilbert (git) wrote:
> > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > 
> >   This set enables postcopy migration with shared memory to a vhost user process.
> > It's based off current head.
> > 
> >   Testing is mostly performed with dpdk, with corresponding modifications by
> > Maxime.
> > 
> >   v4 is mostly just fixes from comments received during review of v3,
> > and the normal updating of all the virtio enum's to catch up with other
> > things getting in before us.
> > 
> > Dave
> 
> For vhost bits:
> 
> Acked-by: Michael S. Tsirkin <mst@redhat.com>
> 
> So how is all this going to be merged?
> If it's through my tree pls let me know.
> I'm fine with another tree though.

You already merged it!  Back in March in ed627b2ad37469eeba9e
Maxime has a dpdk set to be merged, but we're just working a bug
at the moment that we've not figured out yet (could be either side).

Dave

> 
> > 
> > Dr. David Alan Gilbert (29):
> >   migrate: Update ram_block_discard_range for shared
> >   qemu_ram_block_host_offset
> >   postcopy: use UFFDIO_ZEROPAGE only when available
> >   postcopy: Add notifier chain
> >   postcopy: Add vhost-user flag for postcopy and check it
> >   vhost-user: Add 'VHOST_USER_POSTCOPY_ADVISE' message
> >   libvhost-user: Support sending fds back to qemu
> >   libvhost-user: Open userfaultfd
> >   postcopy: Allow registering of fd handler
> >   vhost+postcopy: Register shared ufd with postcopy
> >   vhost+postcopy: Transmit 'listen' to client
> >   postcopy+vhost-user: Split set_mem_table for postcopy
> >   migration/ram: ramblock_recv_bitmap_test_byte_offset
> >   libvhost-user+postcopy: Register new regions with the ufd
> >   vhost+postcopy: Send address back to qemu
> >   vhost+postcopy: Stash RAMBlock and offset
> >   vhost+postcopy: Send requests to source for shared pages
> >   vhost+postcopy: Resolve client address
> >   postcopy: helper for waking shared
> >   postcopy: postcopy_notify_shared_wake
> >   vhost+postcopy: Add vhost waker
> >   vhost+postcopy: Call wakeups
> >   libvhost-user: mprotect & madvises for postcopy
> >   vhost-user: Add VHOST_USER_POSTCOPY_END message
> >   vhost+postcopy: Wire up POSTCOPY_END notify
> >   vhost: Huge page align and merge
> >   postcopy: Allow shared memory
> >   libvhost-user: Claim support for postcopy
> >   postcopy shared docs
> > 
> >  contrib/libvhost-user/libvhost-user.c | 303 ++++++++++++++++++++++++-
> >  contrib/libvhost-user/libvhost-user.h |  11 +
> >  docs/devel/migration.rst              |  41 ++++
> >  docs/interop/vhost-user.txt           |  51 +++++
> >  exec.c                                |  86 +++++--
> >  hw/virtio/trace-events                |  16 +-
> >  hw/virtio/vhost-user.c                | 411 +++++++++++++++++++++++++++++++++-
> >  hw/virtio/vhost.c                     |  66 +++++-
> >  include/exec/cpu-common.h             |   4 +
> >  migration/migration.c                 |   6 +
> >  migration/migration.h                 |   4 +
> >  migration/postcopy-ram.c              | 354 +++++++++++++++++++++++------
> >  migration/postcopy-ram.h              |  69 ++++++
> >  migration/ram.c                       |   5 +
> >  migration/ram.h                       |   1 +
> >  migration/savevm.c                    |  13 ++
> >  migration/trace-events                |   6 +
> >  trace-events                          |   3 +-
> >  vl.c                                  |   2 +
> >  19 files changed, 1354 insertions(+), 98 deletions(-)
> > 
> > -- 
> > 2.14.3
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK