migration/Makefile.objs | 1 + migration/dirtyrate.c | 448 ++++++++++++++++++++++++++++++++++++++++++++++++ migration/dirtyrate.h | 86 ++++++++++ migration/ram.c | 11 +- migration/ram.h | 10 ++ qapi/migration.json | 42 +++++ 6 files changed, 588 insertions(+), 10 deletions(-) create mode 100644 migration/dirtyrate.c create mode 100644 migration/dirtyrate.h
v1 -> v2: use g_rand_new() to generate rand_buf move RAMBLOCK_FOREACH_MIGRATABLE into migration/ram.h add skip_sample_ramblock to filter sampled ramblock fix multi-numa vm coredump when query dirtyrate rename qapi interface and rename some structures and functions succeed to compile by appling each patch add test for migrating vm Sometimes it is neccessary to evaluate dirty page rate before migration. Users could decide whether to proceed migration based on the evaluation in case of vm performance loss due to heavy workload. Unlikey simulating dirtylog sync which could do harm on runnning vm, we provide a sample-hash method to compare hash results for samping page. In this way, it would have hardly no impact on vm performance. Evaluate the dirtypage rate both on running and migration vm. The VM specifications for migration are as follows: - VM use 4-K page; - the number of VCPU is 32; - the total memory is 32Gigabit; - use 'mempress' tool to pressurize VM(mempress 4096 1024); - migration bandwidth is 1GB/s ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | | running | migrating | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | no mempress | 4MB/s | 8MB/s (migrated success) | ------------------------------------------------------------------------------------ | mempress 4096 1024 | 1188MB/s | 536MB/s ~ 1044MB/s (cpu throttle triggered) | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | mempress 4096 4096 | 4152MB/s | 608MB/s ~ 4125MB/s (migation failed) | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Test dirtyrate by qmp command like this: 1. virsh qemu-monitor-command [vmname] '{"execute":"calc-dirty-rate", "arguments": {"calc-time": [sleep-time]}}'; 2. sleep specific time which is a bit larger than sleep-time 3. virsh qemu-monitor-command [vmname] '{"execute":"query-dirty-rate"}' Further test dirtyrate by libvirt api like this: virsh getdirtyrate [vmname] [sleep-time] Zheng Chuan (10): migration/dirtyrate: Add get_dirtyrate_thread() function migration/dirtyrate: Add RamlockDirtyInfo to store sampled page info migration/dirtyrate: Add dirtyrate statistics series functions migration/dirtyrate: move RAMBLOCK_FOREACH_MIGRATABLE into ram.h migration/dirtyrate: Record hash results for each sampled page migration/dirtyrate: Compare page hash results for recorded sampled page migration/dirtyrate: skip sampling ramblock with size below MIN_RAMBLOCK_SIZE migration/dirtyrate: Implement get_sample_page_period() and block_sample_page_period() migration/dirtyrate: Implement calculate_dirtyrate() function migration/dirtyrate: Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function migration/Makefile.objs | 1 + migration/dirtyrate.c | 448 ++++++++++++++++++++++++++++++++++++++++++++++++ migration/dirtyrate.h | 86 ++++++++++ migration/ram.c | 11 +- migration/ram.h | 10 ++ qapi/migration.json | 42 +++++ 6 files changed, 588 insertions(+), 10 deletions(-) create mode 100644 migration/dirtyrate.c create mode 100644 migration/dirtyrate.h -- 1.8.3.1
Patchew URL: https://patchew.org/QEMU/1597458180-16945-1-git-send-email-zhengchuan@huawei.com/ Hi, This series seems to have some coding style problems. See output below for more information: Type: series Message-id: 1597458180-16945-1-git-send-email-zhengchuan@huawei.com Subject: [PATCH v2 00/10] *** A Method for evaluating dirty page rate *** === TEST SCRIPT BEGIN === #!/bin/bash git rev-parse base > /dev/null || exit 0 git config --local diff.renamelimit 0 git config --local diff.renames True git config --local diff.algorithm histogram ./scripts/checkpatch.pl --mailback base.. === TEST SCRIPT END === Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 Switched to a new branch 'test' 9895a95 migration/dirtyrate: Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function b1e62b0 migration/dirtyrate: Implement calculate_dirtyrate() function 422368f migration/dirtyrate: Implement get_sample_page_period() and block_sample_page_period() b83696a migration/dirtyrate: skip sampling ramblock with size below MIN_RAMBLOCK_SIZE 1e807c0 migration/dirtyrate: Compare page hash results for recorded sampled page 383d30a migration/dirtyrate: Record hash results for each sampled page 95a361f migration/dirtyrate: move RAMBLOCK_FOREACH_MIGRATABLE into ram.h 07d224f migration/dirtyrate: Add dirtyrate statistics series functions dfd9476 migration/dirtyrate: Add RamlockDirtyInfo to store sampled page info 1b3ba07 migration/dirtyrate: Add get_dirtyrate_thread() function === OUTPUT BEGIN === 1/10 Checking commit 1b3ba075587a (migration/dirtyrate: Add get_dirtyrate_thread() function) WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #26: new file mode 100644 total: 0 errors, 1 warnings, 115 lines checked Patch 1/10 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. 2/10 Checking commit dfd947626dc5 (migration/dirtyrate: Add RamlockDirtyInfo to store sampled page info) 3/10 Checking commit 07d224f2772b (migration/dirtyrate: Add dirtyrate statistics series functions) 4/10 Checking commit 95a361f00817 (migration/dirtyrate: move RAMBLOCK_FOREACH_MIGRATABLE into ram.h) ERROR: Macros with multiple statements should be enclosed in a do - while loop #62: FILE: migration/ram.h:42: +#define RAMBLOCK_FOREACH_NOT_IGNORED(block) \ + INTERNAL_RAMBLOCK_FOREACH(block) \ + if (ramblock_is_ignored(block)) {} else ERROR: trailing statements should be on next line #64: FILE: migration/ram.h:44: + if (ramblock_is_ignored(block)) {} else ERROR: Macros with multiple statements should be enclosed in a do - while loop #66: FILE: migration/ram.h:46: +#define RAMBLOCK_FOREACH_MIGRATABLE(block) \ + INTERNAL_RAMBLOCK_FOREACH(block) \ + if (!qemu_ram_is_migratable(block)) {} else ERROR: trailing statements should be on next line #68: FILE: migration/ram.h:48: + if (!qemu_ram_is_migratable(block)) {} else ERROR: braces {} are necessary for all arms of this statement #68: FILE: migration/ram.h:48: + if (!qemu_ram_is_migratable(block)) {} else [...] + if (!qemu_ram_is_migratable(block)) {} else [...] total: 5 errors, 0 warnings, 45 lines checked Patch 4/10 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. 5/10 Checking commit 383d30ad41de (migration/dirtyrate: Record hash results for each sampled page) WARNING: line over 80 characters #33: FILE: migration/dirtyrate.c:79: +static int get_ramblock_vfn_hash(struct RamblockDirtyInfo *info, unsigned long vfn, ERROR: "foo* bar" should be "foo *bar" #59: FILE: migration/dirtyrate.c:105: + GRand* rand = g_rand_new(); ERROR: line over 90 characters #69: FILE: migration/dirtyrate.c:115: + info->hash_result = g_try_malloc0_n(sample_pages_count, sizeof(uint8_t) * qcrypto_hash_len); WARNING: line over 80 characters #75: FILE: migration/dirtyrate.c:121: + info->sample_page_vfn = g_try_malloc0_n(sample_pages_count, sizeof(unsigned long)); WARNING: line over 80 characters #84: FILE: migration/dirtyrate.c:130: + info->sample_page_vfn[i] = g_rand_int_range(rand, 0, info->ramblock_pages - 1); WARNING: line over 80 characters #97: FILE: migration/dirtyrate.c:143: +static void get_ramblock_dirty_info(RAMBlock *block, struct RamblockDirtyInfo *info, ERROR: line over 90 characters #103: FILE: migration/dirtyrate.c:149: + info->sample_pages_count = (qemu_ram_get_used_length(block) * sample_pages_per_gigabytes) >> 30; ERROR: braces {} are necessary for all arms of this statement #126: FILE: migration/dirtyrate.c:172: + if (!block_dinfo) [...] ERROR: line over 90 characters #137: FILE: migration/dirtyrate.c:183: + struct RamblockDirtyInfo **block_dinfo, int *block_index) ERROR: braces {} are necessary for all arms of this statement #146: FILE: migration/dirtyrate.c:192: + if (dinfo == NULL) [...] total: 6 errors, 4 warnings, 164 lines checked Patch 5/10 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. 6/10 Checking commit 1e807c015c51 (migration/dirtyrate: Compare page hash results for recorded sampled page) ERROR: braces {} are necessary for all arms of this statement #28: FILE: migration/dirtyrate.c:216: + if (!md) [...] WARNING: line over 80 characters #37: FILE: migration/dirtyrate.c:225: + if (memcmp(md, info->hash_result + i * qcrypto_hash_len, qcrypto_hash_len) != 0) { WARNING: line over 80 characters #72: FILE: migration/dirtyrate.c:260: +static int compare_page_hash_info(struct RamblockDirtyInfo *info, int block_index) total: 1 errors, 2 warnings, 79 lines checked Patch 6/10 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. 7/10 Checking commit b83696ab4e41 (migration/dirtyrate: skip sampling ramblock with size below MIN_RAMBLOCK_SIZE) 8/10 Checking commit 422368fb328b (migration/dirtyrate: Implement get_sample_page_period() and block_sample_page_period()) WARNING: line over 80 characters #38: FILE: migration/dirtyrate.c:47: + if (sec <= MIN_FETCH_DIRTYRATE_TIME_SEC || sec > MAX_FETCH_DIRTYRATE_TIME_SEC) { total: 0 errors, 1 warnings, 37 lines checked Patch 8/10 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. 9/10 Checking commit b1e62b03b437 (migration/dirtyrate: Implement calculate_dirtyrate() function) 10/10 Checking commit 9895a955bb69 (migration/dirtyrate: Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function) ERROR: space required before the open parenthesis '(' #25: FILE: migration/dirtyrate.c:74: + switch(CalculatingState) { total: 1 errors, 0 warnings, 110 lines checked Patch 10/10 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. === OUTPUT END === Test command exited with code: 1 The full log is available at http://patchew.org/logs/1597458180-16945-1-git-send-email-zhengchuan@huawei.com/testing.checkpatch/?type=message. --- Email generated automatically by Patchew [https://patchew.org/]. Please send your feedback to patchew-devel@redhat.com
Patchew URL: https://patchew.org/QEMU/1597458180-16945-1-git-send-email-zhengchuan@huawei.com/ Hi, This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN === #!/bin/bash make docker-image-centos7 V=1 NETWORK=1 time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1 === TEST SCRIPT END === TEST check-unit: tests/test-char Unexpected error in object_property_try_add() at /tmp/qemu-test/src/qom/object.c:1181: attempt to add duplicate property 'serial-id' to object (type 'container') ERROR test-char - too few tests run (expected 38, got 9) make: *** [check-unit] Error 1 make: *** Waiting for unfinished jobs.... TEST check-qtest-x86_64: tests/qtest/hd-geo-test qemu-system-aarch64: -accel kvm: invalid accelerator kvm --- raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=d30e4f11da184f32a6a18e98d408d1d3', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-27geyh53/src/docker-src.2020-08-15-14.38.06.8943:/var/tmp/qemu:z,ro', 'qemu/centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=d30e4f11da184f32a6a18e98d408d1d3 make[1]: *** [docker-run] Error 1 make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-27geyh53/src' make: *** [docker-run-test-quick@centos7] Error 2 real 12m9.637s user 0m9.264s The full log is available at http://patchew.org/logs/1597458180-16945-1-git-send-email-zhengchuan@huawei.com/testing.docker-quick@centos7/?type=message. --- Email generated automatically by Patchew [https://patchew.org/]. Please send your feedback to patchew-devel@redhat.com
Patchew URL: https://patchew.org/QEMU/1597458180-16945-1-git-send-email-zhengchuan@huawei.com/ Hi, This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN === #! /bin/bash export ARCH=x86_64 make docker-image-fedora V=1 NETWORK=1 time make docker-test-mingw@fedora J=14 NETWORK=1 === TEST SCRIPT END === CC qapi/qapi-commands-trace.o CC qapi/qapi-commands-ui.o /tmp/qemu-test/src/migration/dirtyrate.c: In function 'get_ramblock_vfn_hash': /tmp/qemu-test/src/migration/dirtyrate.c:148:33: error: passing argument 5 of 'qcrypto_hash_bytesv' from incompatible pointer type [-Werror=incompatible-pointer-types] 148 | md, &qcrypto_hash_len, NULL) < 0) { | ^~~~~~~~~~~~~~~~~ | | --- 72 | size_t *resultlen, | ~~~~~~~~^~~~~~~~~ cc1: all warnings being treated as errors make: *** [/tmp/qemu-test/src/rules.mak:69: migration/dirtyrate.o] Error 1 make: *** Waiting for unfinished jobs.... Traceback (most recent call last): File "./tests/docker/docker.py", line 709, in <module> --- raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=8329e3f2f3674af4b07ce49b1f62baa7', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-5wne06va/src/docker-src.2020-08-15-14.33.44.2906:/var/tmp/qemu:z,ro', 'qemu/fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=8329e3f2f3674af4b07ce49b1f62baa7 make[1]: *** [docker-run] Error 1 make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-5wne06va/src' make: *** [docker-run-test-mingw@fedora] Error 2 real 2m56.036s user 0m9.525s The full log is available at http://patchew.org/logs/1597458180-16945-1-git-send-email-zhengchuan@huawei.com/testing.docker-mingw@fedora/?type=message. --- Email generated automatically by Patchew [https://patchew.org/]. Please send your feedback to patchew-devel@redhat.com
© 2016 - 2024 Red Hat, Inc.