[Qemu-devel] [PATCH V4 00/16] COLO: integrate colo frame with block replication and COLO proxy

Zhang Chen posted 16 patches 6 years, 2 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/1516369485-5374-1-git-send-email-zhangckid@gmail.com
Test checkpatch passed
Test docker-build@min-glib passed
Test docker-mingw@fedora passed
Test docker-quick@centos6 passed
Test ppc passed
Test s390x passed
There is a newer version of this series
include/exec/ram_addr.h  |   1 +
include/migration/colo.h |  11 ++-
include/net/filter.h     |   5 ++
migration/Makefile.objs  |   2 +-
migration/colo-comm.c    |  76 -------------------
migration/colo.c         | 188 ++++++++++++++++++++++++++++++++++++++++++++---
migration/migration.c    |  38 +++++++++-
migration/ram.c          | 181 ++++++++++++++++++++++++++++++++++++++++++++-
migration/ram.h          |   4 +
migration/savevm.c       |  54 ++++++++++++--
migration/savevm.h       |   5 ++
migration/trace-events   |   3 +
net/colo-compare.c       | 108 +++++++++++++++++++++++++--
net/colo-compare.h       |  24 ++++++
net/colo.h               |   4 +
net/filter-rewriter.c    | 109 +++++++++++++++++++++++++--
net/filter.c             |  17 +++++
net/net.c                |  28 +++++++
qapi/migration.json      |  35 +++++++++
vl.c                     |   2 -
20 files changed, 777 insertions(+), 118 deletions(-)
delete mode 100644 migration/colo-comm.c
create mode 100644 net/colo-compare.h
[Qemu-devel] [PATCH V4 00/16] COLO: integrate colo frame with block replication and COLO proxy
Posted by Zhang Chen 6 years, 2 months ago
From: Zhang Chen <chen.zhang@intel.com>

Hi~

COLO Frame, block replication and COLO proxy(colo-compare,filter-mirror,
filter-redirector,filter-rewriter) have been exist in qemu
for long time, it's time to integrate these three parts to make COLO really works.

In this series, we have some optimizations for COLO frame, including separating the
process of saving ram and device state, using an COLO_EXIT event to notify users that
VM exits COLO, for these parts, most of them have been reviewed long time ago in old version,
but since this series have just rebased on upstream which had merged a new series of migration,
parts of pathes in this series deserve review again.

We use notifier/callback method for COLO compare to notify COLO frame about
net packets inconsistent event, and add a handle_event method for NetFilterClass to
help COLO frame to notify filters and colo-compare about checkpoint/failover event, 
it is flexible.

For the neweset version, please refer to:
https://github.com/zhangckid/qemu/tree/qemu-colo-18jan19

Please review, thanks.

V4:
 - Address Eric's comments in patch 10/16.
 - Rebase on upstream codes.
 - Fix mingw compile error in patch 02/16.
 - Fix some comments.
 - Fix conflect with the patch "migration: remove "enable_colo" var" in patch 06/16.

V3:
 - Address community comments from V2.
 - Rebase on upstream codes.
 - Fix several bugs.
 - Splite shared disk part to indepentent patch set.
 - Optimize codes.


Zhang Chen (8):
  filter-rewriter: fix memory leak for connection in
    connection_track_table
  colo-compare: implement the process of checkpoint
  colo-compare: use notifier to notify packets comparing result
  COLO: integrate colo compare with colo frame
  COLO: Add block replication into colo process
  ram/COLO: Record the dirty pages that SVM received
  filter: Add handle_event method for NetFilterClass
  filter-rewriter: handle checkpoint and failover event

zhanghailiang (8):
  COLO: Remove colo_state migration struct
  COLO: Load dirty pages into SVM's RAM cache firstly
  COLO: Flush memory data from ram cache
  qmp event: Add COLO_EXIT event to notify users while exited COLO
  savevm: split the process of different stages for loadvm/savevm
  COLO: flush host dirty ram from cache
  COLO: notify net filters about checkpoint/failover event
  COLO: quick failover process by kick COLO thread

 include/exec/ram_addr.h  |   1 +
 include/migration/colo.h |  11 ++-
 include/net/filter.h     |   5 ++
 migration/Makefile.objs  |   2 +-
 migration/colo-comm.c    |  76 -------------------
 migration/colo.c         | 188 ++++++++++++++++++++++++++++++++++++++++++++---
 migration/migration.c    |  38 +++++++++-
 migration/ram.c          | 181 ++++++++++++++++++++++++++++++++++++++++++++-
 migration/ram.h          |   4 +
 migration/savevm.c       |  54 ++++++++++++--
 migration/savevm.h       |   5 ++
 migration/trace-events   |   3 +
 net/colo-compare.c       | 108 +++++++++++++++++++++++++--
 net/colo-compare.h       |  24 ++++++
 net/colo.h               |   4 +
 net/filter-rewriter.c    | 109 +++++++++++++++++++++++++--
 net/filter.c             |  17 +++++
 net/net.c                |  28 +++++++
 qapi/migration.json      |  35 +++++++++
 vl.c                     |   2 -
 20 files changed, 777 insertions(+), 118 deletions(-)
 delete mode 100644 migration/colo-comm.c
 create mode 100644 net/colo-compare.h

-- 
2.7.4


Re: [Qemu-devel] [PATCH V4 00/16] COLO: integrate colo frame with block replication and COLO proxy
Posted by Zhang Chen 6 years, 1 month ago
Hi~ All~

No news for a long time, this is the last part to make COLO running in
upstream.
Any comments for this series?

Thanks
Zhang Chen


On Fri, Jan 19, 2018 at 1:44 PM, Zhang Chen <zhangckid@gmail.com> wrote:

> From: Zhang Chen <chen.zhang@intel.com>
>
> Hi~
>
> COLO Frame, block replication and COLO proxy(colo-compare,filter-mirror,
> filter-redirector,filter-rewriter) have been exist in qemu
> for long time, it's time to integrate these three parts to make COLO
> really works.
>
> In this series, we have some optimizations for COLO frame, including
> separating the
> process of saving ram and device state, using an COLO_EXIT event to notify
> users that
> VM exits COLO, for these parts, most of them have been reviewed long time
> ago in old version,
> but since this series have just rebased on upstream which had merged a new
> series of migration,
> parts of pathes in this series deserve review again.
>
> We use notifier/callback method for COLO compare to notify COLO frame about
> net packets inconsistent event, and add a handle_event method for
> NetFilterClass to
> help COLO frame to notify filters and colo-compare about
> checkpoint/failover event,
> it is flexible.
>
> For the neweset version, please refer to:
> https://github.com/zhangckid/qemu/tree/qemu-colo-18jan19
>
> Please review, thanks.
>
> V4:
>  - Address Eric's comments in patch 10/16.
>  - Rebase on upstream codes.
>  - Fix mingw compile error in patch 02/16.
>  - Fix some comments.
>  - Fix conflect with the patch "migration: remove "enable_colo" var" in
> patch 06/16.
>
> V3:
>  - Address community comments from V2.
>  - Rebase on upstream codes.
>  - Fix several bugs.
>  - Splite shared disk part to indepentent patch set.
>  - Optimize codes.
>
>
> Zhang Chen (8):
>   filter-rewriter: fix memory leak for connection in
>     connection_track_table
>   colo-compare: implement the process of checkpoint
>   colo-compare: use notifier to notify packets comparing result
>   COLO: integrate colo compare with colo frame
>   COLO: Add block replication into colo process
>   ram/COLO: Record the dirty pages that SVM received
>   filter: Add handle_event method for NetFilterClass
>   filter-rewriter: handle checkpoint and failover event
>
> zhanghailiang (8):
>   COLO: Remove colo_state migration struct
>   COLO: Load dirty pages into SVM's RAM cache firstly
>   COLO: Flush memory data from ram cache
>   qmp event: Add COLO_EXIT event to notify users while exited COLO
>   savevm: split the process of different stages for loadvm/savevm
>   COLO: flush host dirty ram from cache
>   COLO: notify net filters about checkpoint/failover event
>   COLO: quick failover process by kick COLO thread
>
>  include/exec/ram_addr.h  |   1 +
>  include/migration/colo.h |  11 ++-
>  include/net/filter.h     |   5 ++
>  migration/Makefile.objs  |   2 +-
>  migration/colo-comm.c    |  76 -------------------
>  migration/colo.c         | 188 ++++++++++++++++++++++++++++++
> ++++++++++++++---
>  migration/migration.c    |  38 +++++++++-
>  migration/ram.c          | 181 ++++++++++++++++++++++++++++++
> ++++++++++++++-
>  migration/ram.h          |   4 +
>  migration/savevm.c       |  54 ++++++++++++--
>  migration/savevm.h       |   5 ++
>  migration/trace-events   |   3 +
>  net/colo-compare.c       | 108 +++++++++++++++++++++++++--
>  net/colo-compare.h       |  24 ++++++
>  net/colo.h               |   4 +
>  net/filter-rewriter.c    | 109 +++++++++++++++++++++++++--
>  net/filter.c             |  17 +++++
>  net/net.c                |  28 +++++++
>  qapi/migration.json      |  35 +++++++++
>  vl.c                     |   2 -
>  20 files changed, 777 insertions(+), 118 deletions(-)
>  delete mode 100644 migration/colo-comm.c
>  create mode 100644 net/colo-compare.h
>
> --
> 2.7.4
>
>