[Patch v2 0/6] migration/postcopy: enable compress during postcopy

Wei Yang posted 6 patches 29 weeks ago
Test asan passed
Test checkpatch passed
Test FreeBSD passed
Test docker-mingw@fedora passed
Test docker-clang@ubuntu passed
Test docker-quick@centos7 passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20191107123907.29791-1-richardw.yang@linux.intel.com
Maintainers: "Dr. David Alan Gilbert" <dgilbert@redhat.com>, Juan Quintela <quintela@redhat.com>
migration/migration.c | 11 -------
migration/ram.c       | 67 +++++++++++++++++++++++++++++++++----------
2 files changed, 52 insertions(+), 26 deletions(-)

[Patch v2 0/6] migration/postcopy: enable compress during postcopy

Posted by Wei Yang 29 weeks ago
This patch set tries enable compress during postcopy.

postcopy requires to place a whole host page, while migration thread migrate
memory in target page size. This makes postcopy need to collect all target
pages in one host page before placing via userfaultfd.

To enable compress during postcopy, there are two problems to solve:

    1. Random order for target page arrival
    2. Target pages in one host page arrives without interrupt by target
       page from other host page

The first one is handled by counting the number of target pages arrived
instead of the last target page arrived.

The second one is handled by:

    1. Flush compress thread for each host page
    2. Wait for decompress thread for before placing host page

With the combination of these two changes, compress is enabled during
postcopy.

---
v2:
     * use uintptr_t to calculate place_dest
     * check target pages belongs to the same host page

Wei Yang (6):
  migration/postcopy: reduce memset when it is zero page and
    matches_target_page_size
  migration/postcopy: wait for decompress thread in precopy
  migration/postcopy: count target page number to decide the
    place_needed
  migration/postcopy: set all_zero to true on the first target page
  migration/postcopy: enable random order target page arrival
  migration/postcopy: enable compress during postcopy

 migration/migration.c | 11 -------
 migration/ram.c       | 67 +++++++++++++++++++++++++++++++++----------
 2 files changed, 52 insertions(+), 26 deletions(-)

-- 
2.17.1


Re: [Patch v2 0/6] migration/postcopy: enable compress during postcopy

Posted by Wei Yang 23 weeks ago
Would this one be picked up in this version?

On Thu, Nov 07, 2019 at 08:39:01PM +0800, Wei Yang wrote:
>This patch set tries enable compress during postcopy.
>
>postcopy requires to place a whole host page, while migration thread migrate
>memory in target page size. This makes postcopy need to collect all target
>pages in one host page before placing via userfaultfd.
>
>To enable compress during postcopy, there are two problems to solve:
>
>    1. Random order for target page arrival
>    2. Target pages in one host page arrives without interrupt by target
>       page from other host page
>
>The first one is handled by counting the number of target pages arrived
>instead of the last target page arrived.
>
>The second one is handled by:
>
>    1. Flush compress thread for each host page
>    2. Wait for decompress thread for before placing host page
>
>With the combination of these two changes, compress is enabled during
>postcopy.
>
>---
>v2:
>     * use uintptr_t to calculate place_dest
>     * check target pages belongs to the same host page
>
>Wei Yang (6):
>  migration/postcopy: reduce memset when it is zero page and
>    matches_target_page_size
>  migration/postcopy: wait for decompress thread in precopy
>  migration/postcopy: count target page number to decide the
>    place_needed
>  migration/postcopy: set all_zero to true on the first target page
>  migration/postcopy: enable random order target page arrival
>  migration/postcopy: enable compress during postcopy
>
> migration/migration.c | 11 -------
> migration/ram.c       | 67 +++++++++++++++++++++++++++++++++----------
> 2 files changed, 52 insertions(+), 26 deletions(-)
>
>-- 
>2.17.1

-- 
Wei Yang
Help you, Help me

Re: [Patch v2 0/6] migration/postcopy: enable compress during postcopy

Posted by Dr. David Alan Gilbert 23 weeks ago
* Wei Yang (richardw.yang@linux.intel.com) wrote:
> Would this one be picked up in this version?

I think that one is on Juan's list for the pull he's going to do soon.

Dave


> On Thu, Nov 07, 2019 at 08:39:01PM +0800, Wei Yang wrote:
> >This patch set tries enable compress during postcopy.
> >
> >postcopy requires to place a whole host page, while migration thread migrate
> >memory in target page size. This makes postcopy need to collect all target
> >pages in one host page before placing via userfaultfd.
> >
> >To enable compress during postcopy, there are two problems to solve:
> >
> >    1. Random order for target page arrival
> >    2. Target pages in one host page arrives without interrupt by target
> >       page from other host page
> >
> >The first one is handled by counting the number of target pages arrived
> >instead of the last target page arrived.
> >
> >The second one is handled by:
> >
> >    1. Flush compress thread for each host page
> >    2. Wait for decompress thread for before placing host page
> >
> >With the combination of these two changes, compress is enabled during
> >postcopy.
> >
> >---
> >v2:
> >     * use uintptr_t to calculate place_dest
> >     * check target pages belongs to the same host page
> >
> >Wei Yang (6):
> >  migration/postcopy: reduce memset when it is zero page and
> >    matches_target_page_size
> >  migration/postcopy: wait for decompress thread in precopy
> >  migration/postcopy: count target page number to decide the
> >    place_needed
> >  migration/postcopy: set all_zero to true on the first target page
> >  migration/postcopy: enable random order target page arrival
> >  migration/postcopy: enable compress during postcopy
> >
> > migration/migration.c | 11 -------
> > migration/ram.c       | 67 +++++++++++++++++++++++++++++++++----------
> > 2 files changed, 52 insertions(+), 26 deletions(-)
> >
> >-- 
> >2.17.1
> 
> -- 
> Wei Yang
> Help you, Help me
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK


Re: [Patch v2 0/6] migration/postcopy: enable compress during postcopy

Posted by Wei Yang 20 weeks ago
On Wed, Dec 18, 2019 at 07:55:38PM +0000, Dr. David Alan Gilbert wrote:
>* Wei Yang (richardw.yang@linux.intel.com) wrote:
>> Would this one be picked up in this version?
>
>I think that one is on Juan's list for the pull he's going to do soon.
>
>Dave
>

Happy New Year to all~

May I ask the plan for this patch set?


-- 
Wei Yang
Help you, Help me