[Qemu-devel] [PATCH v2 00/16] block: Make blockdev-create a job and stable API

Kevin Wolf posted 16 patches 7 years, 5 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20180529203910.7615-1-kwolf@redhat.com
Test checkpatch failed
Test docker-mingw@fedora passed
Test docker-quick@centos7 passed
Test s390x passed
qapi/block-core.json          |  18 +-
qapi/job.json                 |   4 +-
include/qemu/job.h            |   7 +-
block/backup.c                |   2 +-
block/commit.c                |   2 +-
block/create.c                |  67 +++--
block/mirror.c                |   2 +-
block/stream.c                |   2 +-
block/vdi.c                   |   1 +
block/vhdx.c                  |   2 +-
job-qmp.c                     |   9 +-
job.c                         |  16 +-
tests/test-bdrv-drain.c       |   2 +-
tests/test-blockjob-txn.c     |   2 +-
tests/test-blockjob.c         |   2 +-
tests/qemu-iotests/206        | 680 ++++++++++++++++--------------------------
tests/qemu-iotests/206.out    | 253 +++++++++-------
tests/qemu-iotests/207        | 440 ++++++++++++---------------
tests/qemu-iotests/207.out    | 107 +++----
tests/qemu-iotests/210        | 393 ++++++++++--------------
tests/qemu-iotests/210.out    | 197 ++++++++----
tests/qemu-iotests/211        | 381 ++++++++++-------------
tests/qemu-iotests/211.out    | 133 +++++----
tests/qemu-iotests/212        | 483 +++++++++++-------------------
tests/qemu-iotests/212.out    | 191 +++++++-----
tests/qemu-iotests/213        | 520 ++++++++++++--------------------
tests/qemu-iotests/213.out    | 208 ++++++++-----
tests/qemu-iotests/iotests.py |  78 +++++
28 files changed, 1979 insertions(+), 2223 deletions(-)
[Qemu-devel] [PATCH v2 00/16] block: Make blockdev-create a job and stable API
Posted by Kevin Wolf 7 years, 5 months ago
This changes the x-blockdev-create QMP command so that it doesn't block
the monitor and the main loop any more, but starts a background job that
performs the image creation.

The basic job as implemented here is all that is necessary to make image
creation asynchronous and to provide a QMP interface that can be marked
stable, but it still lacks a few features that jobs usually provide: The
job will ignore pause commands and it doesn't publish progress yet (so
both current-progress and total-progress stay at 0). These features can
be added later without breaking compatibility.

At the end of the series, the interface is declared stable and the x-
prefix is removed.

v2:
- Made job->error == NULL iff job->ret == 0 [Max]
- Fixed a comment in qmp_blockdev_create() and added a TODO comment for
  AioContext locking at least [Max]
- Many small changes in the tests [Max, Jeff]

Kevin Wolf (16):
  vdi: Fix vdi_co_do_create() return value
  vhdx: Fix vhdx_co_create() return value
  job: Add error message for failing jobs
  block/create: Make x-blockdev-create a job
  qemu-iotests: Add VM.get_qmp_events_filtered()
  qemu-iotests: Add VM.qmp_log()
  qemu-iotests: Add iotests.img_info_log()
  qemu-iotests: Add VM.run_job()
  qemu-iotests: iotests.py helper for non-file protocols
  qemu-iotests: Rewrite 206 for blockdev-create job
  qemu-iotests: Rewrite 207 for blockdev-create job
  qemu-iotests: Rewrite 210 for blockdev-create job
  qemu-iotests: Rewrite 211 for blockdev-create job
  qemu-iotests: Rewrite 212 for blockdev-create job
  qemu-iotests: Rewrite 213 for blockdev-create job
  block/create: Mark blockdev-create stable

 qapi/block-core.json          |  18 +-
 qapi/job.json                 |   4 +-
 include/qemu/job.h            |   7 +-
 block/backup.c                |   2 +-
 block/commit.c                |   2 +-
 block/create.c                |  67 +++--
 block/mirror.c                |   2 +-
 block/stream.c                |   2 +-
 block/vdi.c                   |   1 +
 block/vhdx.c                  |   2 +-
 job-qmp.c                     |   9 +-
 job.c                         |  16 +-
 tests/test-bdrv-drain.c       |   2 +-
 tests/test-blockjob-txn.c     |   2 +-
 tests/test-blockjob.c         |   2 +-
 tests/qemu-iotests/206        | 680 ++++++++++++++++--------------------------
 tests/qemu-iotests/206.out    | 253 +++++++++-------
 tests/qemu-iotests/207        | 440 ++++++++++++---------------
 tests/qemu-iotests/207.out    | 107 +++----
 tests/qemu-iotests/210        | 393 ++++++++++--------------
 tests/qemu-iotests/210.out    | 197 ++++++++----
 tests/qemu-iotests/211        | 381 ++++++++++-------------
 tests/qemu-iotests/211.out    | 133 +++++----
 tests/qemu-iotests/212        | 483 +++++++++++-------------------
 tests/qemu-iotests/212.out    | 191 +++++++-----
 tests/qemu-iotests/213        | 520 ++++++++++++--------------------
 tests/qemu-iotests/213.out    | 208 ++++++++-----
 tests/qemu-iotests/iotests.py |  78 +++++
 28 files changed, 1979 insertions(+), 2223 deletions(-)

-- 
2.13.6


Re: [Qemu-devel] [PATCH v2 00/16] block: Make blockdev-create a job and stable API
Posted by no-reply@patchew.org 7 years, 5 months ago
Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20180529203910.7615-1-kwolf@redhat.com
Subject: [Qemu-devel] [PATCH v2 00/16] block: Make blockdev-create a job and stable API

=== TEST SCRIPT BEGIN ===
#!/bin/bash

BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0

git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram

commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
    echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
    if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
        failed=1
        echo
    fi
    n=$((n+1))
done

exit $failed
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 t [tag update]            patchew/1527612092-65980-1-git-send-email-anton.nefedov@virtuozzo.com -> patchew/1527612092-65980-1-git-send-email-anton.nefedov@virtuozzo.com
 * [new tag]               patchew/20180529203910.7615-1-kwolf@redhat.com -> patchew/20180529203910.7615-1-kwolf@redhat.com
Switched to a new branch 'test'
122de4416e block/create: Mark blockdev-create stable
a083e7beeb qemu-iotests: Rewrite 213 for blockdev-create job
c947dfa69d qemu-iotests: Rewrite 212 for blockdev-create job
df37241f2a qemu-iotests: Rewrite 211 for blockdev-create job
b2775b9daf qemu-iotests: Rewrite 210 for blockdev-create job
f60be88204 qemu-iotests: Rewrite 207 for blockdev-create job
bafc347497 qemu-iotests: Rewrite 206 for blockdev-create job
868a906170 qemu-iotests: iotests.py helper for non-file protocols
30b2fe3962 qemu-iotests: Add VM.run_job()
eee2106258 qemu-iotests: Add iotests.img_info_log()
bdd0743777 qemu-iotests: Add VM.qmp_log()
f1cc62a518 qemu-iotests: Add VM.get_qmp_events_filtered()
f0c0ceee97 block/create: Make x-blockdev-create a job
d4d0e41747 job: Add error message for failing jobs
763841f94b vhdx: Fix vhdx_co_create() return value
bd60e71ce1 vdi: Fix vdi_co_do_create() return value

=== OUTPUT BEGIN ===
Checking PATCH 1/16: vdi: Fix vdi_co_do_create() return value...
Checking PATCH 2/16: vhdx: Fix vhdx_co_create() return value...
Checking PATCH 3/16: job: Add error message for failing jobs...
Checking PATCH 4/16: block/create: Make x-blockdev-create a job...
Checking PATCH 5/16: qemu-iotests: Add VM.get_qmp_events_filtered()...
Checking PATCH 6/16: qemu-iotests: Add VM.qmp_log()...
Checking PATCH 7/16: qemu-iotests: Add iotests.img_info_log()...
ERROR: line over 90 characters
#42: FILE: tests/qemu-iotests/iotests.py:227:
+        line = re.sub('uuid: [-a-f0-9]+', 'uuid: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX', line)

total: 1 errors, 0 warnings, 30 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

Checking PATCH 8/16: qemu-iotests: Add VM.run_job()...
Checking PATCH 9/16: qemu-iotests: iotests.py helper for non-file protocols...
Checking PATCH 10/16: qemu-iotests: Rewrite 206 for blockdev-create job...
Checking PATCH 11/16: qemu-iotests: Rewrite 207 for blockdev-create job...
Checking PATCH 12/16: qemu-iotests: Rewrite 210 for blockdev-create job...
Checking PATCH 13/16: qemu-iotests: Rewrite 211 for blockdev-create job...
Checking PATCH 14/16: qemu-iotests: Rewrite 212 for blockdev-create job...
Checking PATCH 15/16: qemu-iotests: Rewrite 213 for blockdev-create job...
Checking PATCH 16/16: block/create: Mark blockdev-create stable...
=== OUTPUT END ===

Test command exited with code: 1


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [Qemu-devel] [PATCH v2 00/16] block: Make blockdev-create a job and stable API
Posted by Kevin Wolf 7 years, 5 months ago
Am 29.05.2018 um 22:38 hat Kevin Wolf geschrieben:
> This changes the x-blockdev-create QMP command so that it doesn't block
> the monitor and the main loop any more, but starts a background job that
> performs the image creation.
> 
> The basic job as implemented here is all that is necessary to make image
> creation asynchronous and to provide a QMP interface that can be marked
> stable, but it still lacks a few features that jobs usually provide: The
> job will ignore pause commands and it doesn't publish progress yet (so
> both current-progress and total-progress stay at 0). These features can
> be added later without breaking compatibility.
> 
> At the end of the series, the interface is declared stable and the x-
> prefix is removed.

Applied to the block branch.

Kevin

Re: [Qemu-devel] [PATCH v2 00/16] block: Make blockdev-create a job and stable API
Posted by Jeff Cody 7 years, 5 months ago
On Tue, May 29, 2018 at 10:38:54PM +0200, Kevin Wolf wrote:
> This changes the x-blockdev-create QMP command so that it doesn't block
> the monitor and the main loop any more, but starts a background job that
> performs the image creation.
> 

I'm sure modifying the drivers is fodder for a future series, since this
series lays the groundwork.  But from testing, some image formats (e.g.
qcow2) still block the main loop and monitor when doing blockdev-create
(preallocation:full, and a larger image size is particularly noticeable).

> The basic job as implemented here is all that is necessary to make image
> creation asynchronous and to provide a QMP interface that can be marked
> stable, but it still lacks a few features that jobs usually provide: The
> job will ignore pause commands and it doesn't publish progress yet (so
> both current-progress and total-progress stay at 0). These features can
> be added later without breaking compatibility.
> 
> At the end of the series, the interface is declared stable and the x-
> prefix is removed.
> 
> v2:
> - Made job->error == NULL iff job->ret == 0 [Max]
> - Fixed a comment in qmp_blockdev_create() and added a TODO comment for
>   AioContext locking at least [Max]
> - Many small changes in the tests [Max, Jeff]
> 
> Kevin Wolf (16):
>   vdi: Fix vdi_co_do_create() return value
>   vhdx: Fix vhdx_co_create() return value
>   job: Add error message for failing jobs
>   block/create: Make x-blockdev-create a job
>   qemu-iotests: Add VM.get_qmp_events_filtered()
>   qemu-iotests: Add VM.qmp_log()
>   qemu-iotests: Add iotests.img_info_log()
>   qemu-iotests: Add VM.run_job()
>   qemu-iotests: iotests.py helper for non-file protocols
>   qemu-iotests: Rewrite 206 for blockdev-create job
>   qemu-iotests: Rewrite 207 for blockdev-create job
>   qemu-iotests: Rewrite 210 for blockdev-create job
>   qemu-iotests: Rewrite 211 for blockdev-create job
>   qemu-iotests: Rewrite 212 for blockdev-create job
>   qemu-iotests: Rewrite 213 for blockdev-create job
>   block/create: Mark blockdev-create stable
> 
>  qapi/block-core.json          |  18 +-
>  qapi/job.json                 |   4 +-
>  include/qemu/job.h            |   7 +-
>  block/backup.c                |   2 +-
>  block/commit.c                |   2 +-
>  block/create.c                |  67 +++--
>  block/mirror.c                |   2 +-
>  block/stream.c                |   2 +-
>  block/vdi.c                   |   1 +
>  block/vhdx.c                  |   2 +-
>  job-qmp.c                     |   9 +-
>  job.c                         |  16 +-
>  tests/test-bdrv-drain.c       |   2 +-
>  tests/test-blockjob-txn.c     |   2 +-
>  tests/test-blockjob.c         |   2 +-
>  tests/qemu-iotests/206        | 680 ++++++++++++++++--------------------------
>  tests/qemu-iotests/206.out    | 253 +++++++++-------
>  tests/qemu-iotests/207        | 440 ++++++++++++---------------
>  tests/qemu-iotests/207.out    | 107 +++----
>  tests/qemu-iotests/210        | 393 ++++++++++--------------
>  tests/qemu-iotests/210.out    | 197 ++++++++----
>  tests/qemu-iotests/211        | 381 ++++++++++-------------
>  tests/qemu-iotests/211.out    | 133 +++++----
>  tests/qemu-iotests/212        | 483 +++++++++++-------------------
>  tests/qemu-iotests/212.out    | 191 +++++++-----
>  tests/qemu-iotests/213        | 520 ++++++++++++--------------------
>  tests/qemu-iotests/213.out    | 208 ++++++++-----
>  tests/qemu-iotests/iotests.py |  78 +++++
>  28 files changed, 1979 insertions(+), 2223 deletions(-)
> 
> -- 
> 2.13.6
> 

Re: [Qemu-devel] [PATCH v2 00/16] block: Make blockdev-create a job and stable API
Posted by Jeff Cody 7 years, 5 months ago
On Tue, May 29, 2018 at 09:43:16PM -0400, Jeff Cody wrote:
> On Tue, May 29, 2018 at 10:38:54PM +0200, Kevin Wolf wrote:
> > This changes the x-blockdev-create QMP command so that it doesn't block
> > the monitor and the main loop any more, but starts a background job that
> > performs the image creation.
> > 
> 
> I'm sure modifying the drivers is fodder for a future series, since this
> series lays the groundwork.  But from testing, some image formats (e.g.
> qcow2) still block the main loop and monitor when doing blockdev-create
> (preallocation:full, and a larger image size is particularly noticeable).
> 

If it helps to reproduce what I'm seeing, here is the QAPI command I am
using when testing:

    {
    "execute": "blockdev-create",
        "arguments": {
            "job-id": "jobtest1",
            "options": {
                "driver": "qcow2",
                "file": {
                    "driver": "file",
                    "filename": "/home/jcody/deleteme.qcow2"
                },
                "preallocation": "full",
                "size": 5120000000
            }
        }
    }


> > The basic job as implemented here is all that is necessary to make image
> > creation asynchronous and to provide a QMP interface that can be marked
> > stable, but it still lacks a few features that jobs usually provide: The
> > job will ignore pause commands and it doesn't publish progress yet (so
> > both current-progress and total-progress stay at 0). These features can
> > be added later without breaking compatibility.
> > 
> > At the end of the series, the interface is declared stable and the x-
> > prefix is removed.
> > 
> > v2:
> > - Made job->error == NULL iff job->ret == 0 [Max]
> > - Fixed a comment in qmp_blockdev_create() and added a TODO comment for
> >   AioContext locking at least [Max]
> > - Many small changes in the tests [Max, Jeff]
> > 
> > Kevin Wolf (16):
> >   vdi: Fix vdi_co_do_create() return value
> >   vhdx: Fix vhdx_co_create() return value
> >   job: Add error message for failing jobs
> >   block/create: Make x-blockdev-create a job
> >   qemu-iotests: Add VM.get_qmp_events_filtered()
> >   qemu-iotests: Add VM.qmp_log()
> >   qemu-iotests: Add iotests.img_info_log()
> >   qemu-iotests: Add VM.run_job()
> >   qemu-iotests: iotests.py helper for non-file protocols
> >   qemu-iotests: Rewrite 206 for blockdev-create job
> >   qemu-iotests: Rewrite 207 for blockdev-create job
> >   qemu-iotests: Rewrite 210 for blockdev-create job
> >   qemu-iotests: Rewrite 211 for blockdev-create job
> >   qemu-iotests: Rewrite 212 for blockdev-create job
> >   qemu-iotests: Rewrite 213 for blockdev-create job
> >   block/create: Mark blockdev-create stable
> > 
> >  qapi/block-core.json          |  18 +-
> >  qapi/job.json                 |   4 +-
> >  include/qemu/job.h            |   7 +-
> >  block/backup.c                |   2 +-
> >  block/commit.c                |   2 +-
> >  block/create.c                |  67 +++--
> >  block/mirror.c                |   2 +-
> >  block/stream.c                |   2 +-
> >  block/vdi.c                   |   1 +
> >  block/vhdx.c                  |   2 +-
> >  job-qmp.c                     |   9 +-
> >  job.c                         |  16 +-
> >  tests/test-bdrv-drain.c       |   2 +-
> >  tests/test-blockjob-txn.c     |   2 +-
> >  tests/test-blockjob.c         |   2 +-
> >  tests/qemu-iotests/206        | 680 ++++++++++++++++--------------------------
> >  tests/qemu-iotests/206.out    | 253 +++++++++-------
> >  tests/qemu-iotests/207        | 440 ++++++++++++---------------
> >  tests/qemu-iotests/207.out    | 107 +++----
> >  tests/qemu-iotests/210        | 393 ++++++++++--------------
> >  tests/qemu-iotests/210.out    | 197 ++++++++----
> >  tests/qemu-iotests/211        | 381 ++++++++++-------------
> >  tests/qemu-iotests/211.out    | 133 +++++----
> >  tests/qemu-iotests/212        | 483 +++++++++++-------------------
> >  tests/qemu-iotests/212.out    | 191 +++++++-----
> >  tests/qemu-iotests/213        | 520 ++++++++++++--------------------
> >  tests/qemu-iotests/213.out    | 208 ++++++++-----
> >  tests/qemu-iotests/iotests.py |  78 +++++
> >  28 files changed, 1979 insertions(+), 2223 deletions(-)
> > 
> > -- 
> > 2.13.6
> >