[RFC 00/24] backup performance: block_status + async

Vladimir Sementsov-Ogievskiy posted 24 patches 4 years, 5 months ago
Test asan failed
Test checkpatch failed
Test FreeBSD passed
Test docker-mingw@fedora failed
Test docker-clang@ubuntu failed
Test docker-quick@centos7 failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20191115141444.24155-1-vsementsov@virtuozzo.com
Maintainers: Eric Blake <eblake@redhat.com>, John Snow <jsnow@redhat.com>, Max Reitz <mreitz@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Xie Changlong <xiechanglong.d@gmail.com>, Cleber Rosa <crosa@redhat.com>, Wen Congyang <wencongyang2@huawei.com>, Markus Armbruster <armbru@redhat.com>
There is a newer version of this series
qapi/block-core.json           |   9 +-
include/block/block-copy.h     |  90 ++---
include/block/block_int.h      |   7 +
include/block/blockjob_int.h   |   2 +
block/backup-top.c             |   6 +-
block/backup.c                 | 184 ++++++----
block/block-copy.c             | 608 ++++++++++++++++++++++++++++-----
block/replication.c            |   1 +
blockdev.c                     |   5 +
blockjob.c                     |   6 +
job.c                          |   1 +
block/trace-events             |   1 +
python/bench-example.py        |  93 +++++
python/qemu/bench_block_job.py | 114 +++++++
python/simplebench.py          | 122 +++++++
tests/qemu-iotests/129         |   3 +-
tests/qemu-iotests/185         |   3 +-
tests/qemu-iotests/219         |   1 +
tests/qemu-iotests/257         |   1 +
tests/qemu-iotests/257.out     | 306 ++++++++---------
20 files changed, 1184 insertions(+), 379 deletions(-)
create mode 100755 python/bench-example.py
create mode 100755 python/qemu/bench_block_job.py
create mode 100644 python/simplebench.py
[RFC 00/24] backup performance: block_status + async
Posted by Vladimir Sementsov-Ogievskiy 4 years, 5 months ago
Hi all!

These series does the following things:

1. bring block_status to block-copy, for efficient chunk sizes and
handling ZERO clusters. (mirror does it)
2. bring aio-task-pool to block-copy, for parallel copying loop
iteration. (mirror does it its own way)
4. add speed limit and cancelling possibility to block-copy (for 5.)
5. use block-copy in backup directly, to bring block_status and async.
tasks into backup
6. add some python scripts to benchmark the results

The main theme is async handling of copying loop iterations, which
already works and bring performance in mirror (in its own way) and in
qcow2 (using aio-task-pool).

Here are the results:

----------  -------------  -----------------  -------------  -----------------  -------------
            backup-old     backup-old(no CR)  backup-new     backup-new(no CR)  mirror
ssd -> ssd  9.88 +- 0.85   8.85 +- 0.48       5.39 +- 0.04   4.06 +- 0.01       4.15 +- 0.03
ssd -> hdd  10.90 +- 0.30  10.39 +- 0.41      9.36 +- 0.06   9.24 +- 0.06       9.00 +- 0.06
hdd -> hdd  20.09 +- 0.23  20.15 +- 0.07      48.65 +- 1.86  20.62 +- 0.08      19.82 +- 0.37
----------  -------------  -----------------  -------------  -----------------  -------------

----------  -------------  -------------  -------------
            backup-old     backup-new     mirror
nbd -> ssd  30.69 +- 0.23  9.02 +- 0.00   9.06 +- 0.03
ssd -> nbd  36.94 +- 0.01  11.50 +- 0.08  10.12 +- 0.05
----------  -------------  -------------  -------------

Here:
"old" means "before series"
"new" means "after series"
"no CR" means "copy_range disabled"

nbd is nbd server on another node, running like
"qemu-nbd --persistent --nocache -p 10810 ones1000M-source"

RFC.1
What I noticed, is that copy_range makes things worse.. Is there any
case or benchmarking which shows that copy_range increases performance?
Possibly we should disable it by default..

RFC.2
Last patch isn't for commit, possibly I should make some generic
example, ideas are welcome

RFC.3
The series are big and splittable. The reason to send it alltogether is
that I wanted to get the whole picture to benchmark.

Also, I keep in mind benchmarking backup to qcow2 with different levels
of defragmentation and with/without compression.

Future plan is obvious: move block-commit and block-stream to use
block-copy, which will unify code path and bring performance to commit
and stream.

Vladimir Sementsov-Ogievskiy (24):
  block/block-copy: specialcase first copy_range request
  block/block-copy: use block_status
  block/block-copy: factor out block_copy_find_inflight_req
  block/block-copy: refactor interfaces to use bytes instead of end
  block/block-copy: rename start to offset in interfaces
  block/block-copy: reduce intersecting request lock
  block/block-copy: hide structure definitions
  block/block-copy: rename in-flight requests to tasks
  block/block-copy: alloc task on each iteration
  block/block-copy: add state pointer to BlockCopyTask
  block/block-copy: move task size initial calculation to _task_create
  block/block-copy: move block_copy_task_create down
  block/block-copy: use aio-task-pool API
  block/block-copy: More explicit call_state
  block/block-copy: implement block_copy_async
  block/block-copy: add max_chunk and max_workers paramters
  block/block-copy: add ratelimit to block-copy
  block/block-copy: add block_copy_cancel
  blockjob: add set_speed to BlockJobDriver
  job: call job_enter from job_user_pause
  backup: move to block-copy
  python: add simplebench.py
  python: add qemu/bench_block_job.py
  python: benchmark new backup architecture

 qapi/block-core.json           |   9 +-
 include/block/block-copy.h     |  90 ++---
 include/block/block_int.h      |   7 +
 include/block/blockjob_int.h   |   2 +
 block/backup-top.c             |   6 +-
 block/backup.c                 | 184 ++++++----
 block/block-copy.c             | 608 ++++++++++++++++++++++++++++-----
 block/replication.c            |   1 +
 blockdev.c                     |   5 +
 blockjob.c                     |   6 +
 job.c                          |   1 +
 block/trace-events             |   1 +
 python/bench-example.py        |  93 +++++
 python/qemu/bench_block_job.py | 114 +++++++
 python/simplebench.py          | 122 +++++++
 tests/qemu-iotests/129         |   3 +-
 tests/qemu-iotests/185         |   3 +-
 tests/qemu-iotests/219         |   1 +
 tests/qemu-iotests/257         |   1 +
 tests/qemu-iotests/257.out     | 306 ++++++++---------
 20 files changed, 1184 insertions(+), 379 deletions(-)
 create mode 100755 python/bench-example.py
 create mode 100755 python/qemu/bench_block_job.py
 create mode 100644 python/simplebench.py

-- 
2.21.0


Re: [RFC 00/24] backup performance: block_status + async
Posted by no-reply@patchew.org 4 years, 5 months ago
Patchew URL: https://patchew.org/QEMU/20191115141444.24155-1-vsementsov@virtuozzo.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [RFC 00/24] backup performance: block_status + async
Type: series
Message-id: 20191115141444.24155-1-vsementsov@virtuozzo.com

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
b3ea574 python: benchmark new backup architecture
899a413 python: add qemu/bench_block_job.py
46552d2 python: add simplebench.py
e20f1fb backup: move to block-copy
9b0e5f0 job: call job_enter from job_user_pause
fcc0a72 blockjob: add set_speed to BlockJobDriver
62d85a7 block/block-copy: add block_copy_cancel
19066cf block/block-copy: add ratelimit to block-copy
db53990 block/block-copy: add max_chunk and max_workers paramters
97fe936 block/block-copy: implement block_copy_async
feb18da block/block-copy: More explicit call_state
5f9fd94 block/block-copy: use aio-task-pool API
94a4f4f block/block-copy: move block_copy_task_create down
74bc1b1 block/block-copy: move task size initial calculation to _task_create
aaf398c block/block-copy: add state pointer to BlockCopyTask
5c4dbd2 block/block-copy: alloc task on each iteration
44e698f block/block-copy: rename in-flight requests to tasks
6d78584 block/block-copy: hide structure definitions
4b541b5 block/block-copy: reduce intersecting request lock
33d4eb1 block/block-copy: rename start to offset in interfaces
d74cc59 block/block-copy: refactor interfaces to use bytes instead of end
bc4e836 block/block-copy: factor out block_copy_find_inflight_req
f22200c block/block-copy: use block_status
5c23778 block/block-copy: specialcase first copy_range request

=== OUTPUT BEGIN ===
1/24 Checking commit 5c23778bb3f4 (block/block-copy: specialcase first copy_range request)
2/24 Checking commit f22200c5b50a (block/block-copy: use block_status)
3/24 Checking commit bc4e83640682 (block/block-copy: factor out block_copy_find_inflight_req)
4/24 Checking commit d74cc59b9921 (block/block-copy: refactor interfaces to use bytes instead of end)
5/24 Checking commit 33d4eb118fd6 (block/block-copy: rename start to offset in interfaces)
6/24 Checking commit 4b541b586cbf (block/block-copy: reduce intersecting request lock)
7/24 Checking commit 6d785845c236 (block/block-copy: hide structure definitions)
8/24 Checking commit 44e698fcabd9 (block/block-copy: rename in-flight requests to tasks)
9/24 Checking commit 5c4dbd281da7 (block/block-copy: alloc task on each iteration)
10/24 Checking commit aaf398c95343 (block/block-copy: add state pointer to BlockCopyTask)
11/24 Checking commit 74bc1b1a2719 (block/block-copy: move task size initial calculation to _task_create)
12/24 Checking commit 94a4f4fbaf78 (block/block-copy: move block_copy_task_create down)
13/24 Checking commit 5f9fd943c782 (block/block-copy: use aio-task-pool API)
14/24 Checking commit feb18da8d233 (block/block-copy: More explicit call_state)
15/24 Checking commit 97fe9366ea2b (block/block-copy: implement block_copy_async)
16/24 Checking commit db5399013d42 (block/block-copy: add max_chunk and max_workers paramters)
17/24 Checking commit 19066cfd3aeb (block/block-copy: add ratelimit to block-copy)
18/24 Checking commit 62d85a7434d8 (block/block-copy: add block_copy_cancel)
19/24 Checking commit fcc0a7228683 (blockjob: add set_speed to BlockJobDriver)
20/24 Checking commit 9b0e5f043396 (job: call job_enter from job_user_pause)
21/24 Checking commit e20f1fb84dd4 (backup: move to block-copy)
22/24 Checking commit 46552d28b459 (python: add simplebench.py)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#13: 
new file mode 100644

total: 0 errors, 1 warnings, 122 lines checked

Patch 22/24 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
23/24 Checking commit 899a41312a82 (python: add qemu/bench_block_job.py)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#13: 
new file mode 100755

total: 0 errors, 1 warnings, 114 lines checked

Patch 23/24 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
24/24 Checking commit b3ea574749ec (python: benchmark new backup architecture)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#28: 
new file mode 100755

ERROR: line over 90 characters
#66: FILE: python/bench-example.py:34:
+        'qemu_binary': '/work/src/qemu/up-block-copy-block-status--before/x86_64-softmmu/qemu-system-x86_64'

ERROR: line over 90 characters
#71: FILE: python/bench-example.py:39:
+        'qemu_binary': '/work/src/qemu/up-block-copy-block-status--before--no-copy-range/x86_64-softmmu/qemu-system-x86_64'

ERROR: line over 90 characters
#76: FILE: python/bench-example.py:44:
+        'qemu_binary': '/work/src/qemu/up-block-copy-block-status/x86_64-softmmu/qemu-system-x86_64'

ERROR: line over 90 characters
#81: FILE: python/bench-example.py:49:
+        'qemu_binary': '/work/src/qemu/up-block-copy-block-status--no-copy_range/x86_64-softmmu/qemu-system-x86_64'

ERROR: line over 90 characters
#86: FILE: python/bench-example.py:54:
+        'qemu_binary': '/work/src/qemu/up-block-copy-block-status/x86_64-softmmu/qemu-system-x86_64'

ERROR: line over 90 characters
#110: FILE: python/bench-example.py:78:
+        'qemu_binary': '/work/src/qemu/up-block-copy-block-status--before/x86_64-softmmu/qemu-system-x86_64'

ERROR: line over 90 characters
#115: FILE: python/bench-example.py:83:
+        'qemu_binary': '/work/src/qemu/up-block-copy-block-status/x86_64-softmmu/qemu-system-x86_64'

ERROR: line over 90 characters
#120: FILE: python/bench-example.py:88:
+        'qemu_binary': '/work/src/qemu/up-block-copy-block-status/x86_64-softmmu/qemu-system-x86_64'

total: 8 errors, 1 warnings, 93 lines checked

Patch 24/24 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20191115141444.24155-1-vsementsov@virtuozzo.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [RFC 00/24] backup performance: block_status + async
Posted by Vladimir Sementsov-Ogievskiy 4 years, 5 months ago
15.11.2019 20:30, no-reply@patchew.org wrote:
> Patchew URL: https://patchew.org/QEMU/20191115141444.24155-1-vsementsov@virtuozzo.com/
> 
> 
> 
> Hi,
> 
> This series seems to have some coding style problems. See output below for
> more information:
> 
> Subject: [RFC 00/24] backup performance: block_status + async
> Type: series
> Message-id: 20191115141444.24155-1-vsementsov@virtuozzo.com
> 
> === TEST SCRIPT BEGIN ===
> #!/bin/bash
> git rev-parse base > /dev/null || exit 0
> git config --local diff.renamelimit 0
> git config --local diff.renames True
> git config --local diff.algorithm histogram
> ./scripts/checkpatch.pl --mailback base..
> === TEST SCRIPT END ===
> 
> Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
> Switched to a new branch 'test'
> b3ea574 python: benchmark new backup architecture
> 899a413 python: add qemu/bench_block_job.py
> 46552d2 python: add simplebench.py
> e20f1fb backup: move to block-copy
> 9b0e5f0 job: call job_enter from job_user_pause
> fcc0a72 blockjob: add set_speed to BlockJobDriver
> 62d85a7 block/block-copy: add block_copy_cancel
> 19066cf block/block-copy: add ratelimit to block-copy
> db53990 block/block-copy: add max_chunk and max_workers paramters
> 97fe936 block/block-copy: implement block_copy_async
> feb18da block/block-copy: More explicit call_state
> 5f9fd94 block/block-copy: use aio-task-pool API
> 94a4f4f block/block-copy: move block_copy_task_create down
> 74bc1b1 block/block-copy: move task size initial calculation to _task_create
> aaf398c block/block-copy: add state pointer to BlockCopyTask
> 5c4dbd2 block/block-copy: alloc task on each iteration
> 44e698f block/block-copy: rename in-flight requests to tasks
> 6d78584 block/block-copy: hide structure definitions
> 4b541b5 block/block-copy: reduce intersecting request lock
> 33d4eb1 block/block-copy: rename start to offset in interfaces
> d74cc59 block/block-copy: refactor interfaces to use bytes instead of end
> bc4e836 block/block-copy: factor out block_copy_find_inflight_req
> f22200c block/block-copy: use block_status
> 5c23778 block/block-copy: specialcase first copy_range request
> 
> === OUTPUT BEGIN ===
> 1/24 Checking commit 5c23778bb3f4 (block/block-copy: specialcase first copy_range request)
> 2/24 Checking commit f22200c5b50a (block/block-copy: use block_status)
> 3/24 Checking commit bc4e83640682 (block/block-copy: factor out block_copy_find_inflight_req)
> 4/24 Checking commit d74cc59b9921 (block/block-copy: refactor interfaces to use bytes instead of end)
> 5/24 Checking commit 33d4eb118fd6 (block/block-copy: rename start to offset in interfaces)
> 6/24 Checking commit 4b541b586cbf (block/block-copy: reduce intersecting request lock)
> 7/24 Checking commit 6d785845c236 (block/block-copy: hide structure definitions)
> 8/24 Checking commit 44e698fcabd9 (block/block-copy: rename in-flight requests to tasks)
> 9/24 Checking commit 5c4dbd281da7 (block/block-copy: alloc task on each iteration)
> 10/24 Checking commit aaf398c95343 (block/block-copy: add state pointer to BlockCopyTask)
> 11/24 Checking commit 74bc1b1a2719 (block/block-copy: move task size initial calculation to _task_create)
> 12/24 Checking commit 94a4f4fbaf78 (block/block-copy: move block_copy_task_create down)
> 13/24 Checking commit 5f9fd943c782 (block/block-copy: use aio-task-pool API)
> 14/24 Checking commit feb18da8d233 (block/block-copy: More explicit call_state)
> 15/24 Checking commit 97fe9366ea2b (block/block-copy: implement block_copy_async)
> 16/24 Checking commit db5399013d42 (block/block-copy: add max_chunk and max_workers paramters)
> 17/24 Checking commit 19066cfd3aeb (block/block-copy: add ratelimit to block-copy)
> 18/24 Checking commit 62d85a7434d8 (block/block-copy: add block_copy_cancel)
> 19/24 Checking commit fcc0a7228683 (blockjob: add set_speed to BlockJobDriver)
> 20/24 Checking commit 9b0e5f043396 (job: call job_enter from job_user_pause)
> 21/24 Checking commit e20f1fb84dd4 (backup: move to block-copy)
> 22/24 Checking commit 46552d28b459 (python: add simplebench.py)
> WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
> #13:
> new file mode 100644
> 
> total: 0 errors, 1 warnings, 122 lines checked
> 
> Patch 22/24 has style problems, please review.  If any of these errors
> are false positives report them to the maintainer, see
> CHECKPATCH in MAINTAINERS.
> 23/24 Checking commit 899a41312a82 (python: add qemu/bench_block_job.py)
> WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
> #13:
> new file mode 100755
> 
> total: 0 errors, 1 warnings, 114 lines checked
> 
> Patch 23/24 has style problems, please review.  If any of these errors
> are false positives report them to the maintainer, see
> CHECKPATCH in MAINTAINERS.
> 24/24 Checking commit b3ea574749ec (python: benchmark new backup architecture)
> WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
> #28:
> new file mode 100755
> 
> ERROR: line over 90 characters
> #66: FILE: python/bench-example.py:34:
> +        'qemu_binary': '/work/src/qemu/up-block-copy-block-status--before/x86_64-softmmu/qemu-system-x86_64'
> 
> ERROR: line over 90 characters
> #71: FILE: python/bench-example.py:39:
> +        'qemu_binary': '/work/src/qemu/up-block-copy-block-status--before--no-copy-range/x86_64-softmmu/qemu-system-x86_64'
> 
> ERROR: line over 90 characters
> #76: FILE: python/bench-example.py:44:
> +        'qemu_binary': '/work/src/qemu/up-block-copy-block-status/x86_64-softmmu/qemu-system-x86_64'
> 
> ERROR: line over 90 characters
> #81: FILE: python/bench-example.py:49:
> +        'qemu_binary': '/work/src/qemu/up-block-copy-block-status--no-copy_range/x86_64-softmmu/qemu-system-x86_64'
> 
> ERROR: line over 90 characters
> #86: FILE: python/bench-example.py:54:
> +        'qemu_binary': '/work/src/qemu/up-block-copy-block-status/x86_64-softmmu/qemu-system-x86_64'
> 
> ERROR: line over 90 characters
> #110: FILE: python/bench-example.py:78:
> +        'qemu_binary': '/work/src/qemu/up-block-copy-block-status--before/x86_64-softmmu/qemu-system-x86_64'
> 
> ERROR: line over 90 characters
> #115: FILE: python/bench-example.py:83:
> +        'qemu_binary': '/work/src/qemu/up-block-copy-block-status/x86_64-softmmu/qemu-system-x86_64'
> 
> ERROR: line over 90 characters
> #120: FILE: python/bench-example.py:88:
> +        'qemu_binary': '/work/src/qemu/up-block-copy-block-status/x86_64-softmmu/qemu-system-x86_64'
> 

I know. still I decided to keep it as is, this patch is not for commit anyway..
My fs paths make sense only for me I think.

> total: 8 errors, 1 warnings, 93 lines checked
> 
> Patch 24/24 has style problems, please review.  If any of these errors
> are false positives report them to the maintainer, see
> CHECKPATCH in MAINTAINERS.
> 
> === OUTPUT END ===
> 
> Test command exited with code: 1
> 
> 
> The full log is available at
> http://patchew.org/logs/20191115141444.24155-1-vsementsov@virtuozzo.com/testing.checkpatch/?type=message.
> ---
> Email generated automatically by Patchew [https://patchew.org/].
> Please send your feedback to patchew-devel@redhat.com
> 


-- 
Best regards,
Vladimir