[PATCH 0/7] UFFD write-tracking migration/snapshots

Andrey Gruzdev via posted 7 patches 3 years, 5 months ago
Test checkpatch failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20201118112233.264530-1-andrey.gruzdev@virtuozzo.com
Maintainers: Eric Blake <eblake@redhat.com>, "Dr. David Alan Gilbert" <dgilbert@redhat.com>, Markus Armbruster <armbru@redhat.com>, Juan Quintela <quintela@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>
There is a newer version of this series
include/exec/memory.h |   7 +
migration/migration.c | 344 ++++++++++++++++++++++++++++++++-
migration/migration.h |   4 +
migration/ram.c       | 439 +++++++++++++++++++++++++++++++++++++++++-
migration/ram.h       |   4 +
migration/savevm.c    |   1 -
migration/savevm.h    |   2 +
qapi/migration.json   |   7 +-
8 files changed, 796 insertions(+), 12 deletions(-)
[PATCH 0/7] UFFD write-tracking migration/snapshots
Posted by Andrey Gruzdev via 3 years, 5 months ago
Currently the only way to make (external) live VM snapshot is using existing
dirty page logging migration mechanism. The main problem is that it tends to
produce a lot of page duplicates while running VM goes on updating already
saved pages. That leads to the fact that vmstate image size is commonly several
times bigger then non-zero part of virtual machine's RSS. Time required to
converge RAM migration and the size of snapshot image severely depend on the
guest memory write rate, sometimes resulting in unacceptably long snapshot
creation time and huge image size 

This series propose a way to solve the aforementioned problems. This is done
by using different RAM migration mechanism based on UFFD write protection
management introduced in v5.7 kernel. The migration strategy is to 'freeze'
guest RAM content using write-protection and iteratively release protection
for memory ranges that have already been saved to the migration stream.
At the same time we read in pending UFFD write fault events and save those
pages out-of-order with higher priority.

How to use:
1. Enable write-tracking migration capability
   virsh qemu-monitor-command <domain> --hmp migrate_set_capability track-writes-ram on

2. Start the external migration to a file
   virsh qemu-monitor-command <domain> --hmp migrate exec:'cat > ./vm_state'

3. Wait for the migration finish and check that the migration has completed state.

Andrey Gruzdev (7):
  Introduce 'track-writes-ram' migration capability.
  Introduced UFFD-WP low-level interface helpers. Implemented support
    for the whole RAM block memory protection/un-protection. Higher
    level ram_write_tracking_start() and ram_write_tracking_stop() to
    start/stop tracking memory writes on the whole VM memory.
  Support UFFD write fault processing in ram_save_iterate().
  Implementation of write-tracking migration thread.
  Implementation of vm_start() BH.
  The rest of write tracking migration code.
  Introduced simple linear scan rate limiting mechanism for write
    tracking migration.

 include/exec/memory.h |   7 +
 migration/migration.c | 344 ++++++++++++++++++++++++++++++++-
 migration/migration.h |   4 +
 migration/ram.c       | 439 +++++++++++++++++++++++++++++++++++++++++-
 migration/ram.h       |   4 +
 migration/savevm.c    |   1 -
 migration/savevm.h    |   2 +
 qapi/migration.json   |   7 +-
 8 files changed, 796 insertions(+), 12 deletions(-)

-- 
2.25.1


Re: [PATCH 0/7] UFFD write-tracking migration/snapshots
Posted by no-reply@patchew.org 3 years, 5 months ago
Patchew URL: https://patchew.org/QEMU/20201118112233.264530-1-andrey.gruzdev@virtuozzo.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20201118112233.264530-1-andrey.gruzdev@virtuozzo.com
Subject: [PATCH 0/7] UFFD write-tracking migration/snapshots

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]         patchew/20201118112233.264530-1-andrey.gruzdev@virtuozzo.com -> patchew/20201118112233.264530-1-andrey.gruzdev@virtuozzo.com
Switched to a new branch 'test'
5589aed Introduced simple linear scan rate limiting mechanism for write tracking migration.
3c614ac The rest of write tracking migration code.
1aa4d6b Implementation of vm_start() BH.
39d7ce1 Implementation of write-tracking migration thread.
b33c61f Support UFFD write fault processing in ram_save_iterate().
eb9f1b9 Introduced UFFD-WP low-level interface helpers. Implemented support for the whole RAM block memory protection/un-protection. Higher level ram_write_tracking_start() and ram_write_tracking_stop() to start/stop tracking memory writes on the whole VM memory.
a32445e Introduce 'track-writes-ram' migration capability.

=== OUTPUT BEGIN ===
1/7 Checking commit a32445ea3f4d (Introduce 'track-writes-ram' migration capability.)
WARNING: line over 80 characters
#134: FILE: migration/migration.c:3881:
+    DEFINE_PROP_MIG_CAP("x-track-writes-ram", MIGRATION_CAPABILITY_TRACK_WRITES_RAM),

total: 0 errors, 1 warnings, 146 lines checked

Patch 1/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
2/7 Checking commit eb9f1b9226fd (Introduced UFFD-WP low-level interface helpers. Implemented support for the whole RAM block memory protection/un-protection. Higher level ram_write_tracking_start() and ram_write_tracking_stop() to start/stop tracking memory writes on the whole VM memory.)
3/7 Checking commit b33c61fdea27 (Support UFFD write fault processing in ram_save_iterate().)
WARNING: line over 80 characters
#70: FILE: migration/ram.c:1952:
+                    ((page_address - (hwaddr) pss->block->host) >> TARGET_PAGE_BITS);

WARNING: line over 80 characters
#120: FILE: migration/ram.c:2002:
+        /* In case we couldn't find respective block, just unprotect faulting page */

WARNING: line over 80 characters
#121: FILE: migration/ram.c:2003:
+        uffd_protect_memory(rs->uffdio_fd, page_address, TARGET_PAGE_SIZE, false);

WARNING: Block comments use a leading /* on a separate line
#156: FILE: migration/ram.c:2050:
+        /* In case of 'write-tracking' migration we first try

WARNING: Block comments use a trailing */ on a separate line
#157: FILE: migration/ram.c:2051:
+         * to poll UFFD and get write page fault event */

WARNING: line over 80 characters
#177: FILE: migration/ram.c:2071:
+                hwaddr run_length = (hwaddr) (page_to - page + 1) << TARGET_PAGE_BITS;

WARNING: line over 80 characters
#183: FILE: migration/ram.c:2077:
+                res = uffd_protect_memory(rs->uffdio_fd, page_address, run_length, false);

total: 0 errors, 7 warnings, 167 lines checked

Patch 3/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
4/7 Checking commit 39d7ce1f4949 (Implementation of write-tracking migration thread.)
ERROR: suspicious ; after while (0)
#142: FILE: migration/migration.c:3811:
+    } while (false);

total: 1 errors, 0 warnings, 229 lines checked

Patch 4/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

5/7 Checking commit 1aa4d6b82d4b (Implementation of vm_start() BH.)
6/7 Checking commit 3c614ac0babe (The rest of write tracking migration code.)
ERROR: line over 90 characters
#19: FILE: migration/migration.c:3216:
+ * wt_migration_completion: Used by wt_migration_thread when after all the RAM has been saved.

WARNING: line over 80 characters
#28: FILE: migration/migration.c:3225:
+    /* Stop tracking RAM writes - un-protect memory, un-register UFFD memory ranges,

WARNING: Block comments use a leading /* on a separate line
#28: FILE: migration/migration.c:3225:
+    /* Stop tracking RAM writes - un-protect memory, un-register UFFD memory ranges,

WARNING: line over 80 characters
#29: FILE: migration/migration.c:3226:
+     * flush kernel wait queues and wake up threads waiting for write fault to be

WARNING: line over 80 characters
#30: FILE: migration/migration.c:3227:
+     * resolved. All of this is essentially done by closing UFFD file descriptor */

WARNING: Block comments use a trailing */ on a separate line
#30: FILE: migration/migration.c:3227:
+     * resolved. All of this is essentially done by closing UFFD file descriptor */

ERROR: switch and case should be at the same indent
#69: FILE: migration/migration.c:3600:
+    switch (s->state) {
+        case MIGRATION_STATUS_COMPLETED:
[...]
+        case MIGRATION_STATUS_ACTIVE:
+        case MIGRATION_STATUS_FAILED:
+        case MIGRATION_STATUS_CANCELLED:
+        case MIGRATION_STATUS_CANCELLING:
[...]
+        default:

total: 2 errors, 5 warnings, 90 lines checked

Patch 6/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

7/7 Checking commit 5589aed36ed6 (Introduced simple linear scan rate limiting mechanism for write tracking migration.)
WARNING: Block comments use a leading /* on a separate line
#78: FILE: migration/ram.c:2040:
+    /* In case last fault time was available and we have

WARNING: Block comments use a trailing */ on a separate line
#79: FILE: migration/ram.c:2041:
+     * latency value, check if it's not too high */

WARNING: Block comments use a leading /* on a separate line
#84: FILE: migration/ram.c:2046:
+    /* Delay thread execution till next write fault occures or timeout expires.

WARNING: line over 80 characters
#85: FILE: migration/ram.c:2047:
+     * Next SLOW_FAULT_SKIP_PAGES can be write fault pages only, not from pages going from

WARNING: line over 80 characters
#86: FILE: migration/ram.c:2048:
+     * linear scan logic. Thus we moderate migration stream rate to reduce latencies */

WARNING: Block comments use a trailing */ on a separate line
#86: FILE: migration/ram.c:2048:
+     * linear scan logic. Thus we moderate migration stream rate to reduce latencies */

WARNING: line over 80 characters
#118: FILE: migration/ram.c:2248:
+#define WT_MAX_WAIT 1000 /* 1000 ms, need bigger limit for 'write-tracking' migration */

total: 0 errors, 7 warnings, 103 lines checked

Patch 7/7 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20201118112233.264530-1-andrey.gruzdev@virtuozzo.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com