[Qemu-devel] [PATCH v3 0/8] WIP: Multifd compression support

Juan Quintela posted 8 patches 4 years, 11 months ago
Test docker-mingw@fedora passed
Test docker-clang@ubuntu failed
Test checkpatch passed
Test asan passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20190515121544.4597-1-quintela@redhat.com
Maintainers: Thomas Huth <thuth@redhat.com>, Juan Quintela <quintela@redhat.com>, Eric Blake <eblake@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, "Dr. David Alan Gilbert" <dgilbert@redhat.com>, Laurent Vivier <lvivier@redhat.com>, Markus Armbruster <armbru@redhat.com>
hmp.c                        |  17 +++
hw/core/qdev-properties.c    |  13 +++
include/hw/qdev-properties.h |   1 +
migration/migration.c        |  25 +++++
migration/migration.h        |   1 +
migration/ram.c              | 203 ++++++++++++++++++++++++++++++++++-
migration/trace-events       |   2 +-
qapi/migration.json          |  30 +++++-
tests/migration-test.c       | 147 ++++++++++++++++++++-----
9 files changed, 406 insertions(+), 33 deletions(-)
[Qemu-devel] [PATCH v3 0/8] WIP: Multifd compression support
Posted by Juan Quintela 4 years, 11 months ago
v3:
- improve the code
- address David and Markus comments
- make compression code into methods
  so we can add any other method ading just three functions

Please review, as far as I know everything is ok now.

Todo: Add zstd support

v2:
- improve the code left and right
- Split better the zlib code
- rename everything to v4.1
- Add tests for multifd-compress zlib
- Parameter is now an enum (soon will see sztd)

ToDo:
- Make operations for diferent methods:
  * multifd_prepare_send_none/zlib
  * multifd_send_none/zlib
  * multifd_recv_none/zlib
- Use the MULTIFD_FLAG_ZLIB (it is unused so far).

Please review and comment.

v1:

This series create compression code on top of multifd.  It is still
WIP, but it is already:
- faster that current compression code
- it does the minimum amount of copies possible
- we allow support for other compression codes
- it pass the multifd test sent in my previous series

Test for existing code didn't work because code is too slow, I need to
make downtime 10 times bigger to make it to converge on my test
machine.  This code works with same limits that multifd no-

ToDo:
- move printf's  to traces
- move code to a struct instead of if (zlib) inside the main threads.
- improve error handling.

Please, review and coment.

Juan Quintela (8):
  migration: fix multifd_recv event typo
  migration-test: rename parameter to parameter_int
  tests: Add migration multifd test
  migration-test: introduce functions to handle string parameters
  migration: Add multifd-compress parameter
  migration: Make none operations into its own structure
  multifd: Add zlib compression support
  multifd: rest of zlib compression

 hmp.c                        |  17 +++
 hw/core/qdev-properties.c    |  13 +++
 include/hw/qdev-properties.h |   1 +
 migration/migration.c        |  25 +++++
 migration/migration.h        |   1 +
 migration/ram.c              | 203 ++++++++++++++++++++++++++++++++++-
 migration/trace-events       |   2 +-
 qapi/migration.json          |  30 +++++-
 tests/migration-test.c       | 147 ++++++++++++++++++++-----
 9 files changed, 406 insertions(+), 33 deletions(-)

-- 
2.21.0


Re: [Qemu-devel] [PATCH v3 0/8] WIP: Multifd compression support
Posted by Wei Yang 4 years, 11 months ago
On Wed, May 15, 2019 at 02:15:36PM +0200, Juan Quintela wrote:
>v3:
>- improve the code
>- address David and Markus comments
>- make compression code into methods
>  so we can add any other method ading just three functions
>
>Please review, as far as I know everything is ok now.
>
>Todo: Add zstd support

Confusion here. It is zstd or sztd?

BTW, I am not sure what it is :-)

>
>v2:
>- improve the code left and right
>- Split better the zlib code
>- rename everything to v4.1
>- Add tests for multifd-compress zlib
>- Parameter is now an enum (soon will see sztd)
                                           ^^^
>

-- 
Wei Yang
Help you, Help me

Re: [Qemu-devel] [PATCH v3 0/8] WIP: Multifd compression support
Posted by Eric Blake 4 years, 10 months ago
On 5/20/19 1:35 AM, Wei Yang wrote:
> On Wed, May 15, 2019 at 02:15:36PM +0200, Juan Quintela wrote:
>> v3:
>> - improve the code
>> - address David and Markus comments
>> - make compression code into methods
>>  so we can add any other method ading just three functions
>>
>> Please review, as far as I know everything is ok now.
>>
>> Todo: Add zstd support
> 
> Confusion here. It is zstd or sztd?
> 
> BTW, I am not sure what it is :-)
> 
>>
>> v2:
>> - improve the code left and right
>> - Split better the zlib code
>> - rename everything to v4.1
>> - Add tests for multifd-compress zlib
>> - Parameter is now an enum (soon will see sztd)
>                                            ^^^

zstd is the name of the new compression algorithm.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org