[PULL 0/7] Block layer patches
[Qemu-devel] [PULL v2 00/40] Block layer patches
1
The following changes since commit 7bc8f9734213b76e76631a483be13d6737c2adbc:
1
The following changes since commit 77d472291812cf04f97974dadbda767e59e31fde:
2
2
3
Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20191025' into staging (2019-10-25 13:12:16 +0100)
3
Merge remote-tracking branch 'remotes/sstabellini/tags/xen-20170707-tag' into staging (2017-07-10 10:29:11 +0100)
4
4
5
are available in the Git repository at:
5
are available in the git repository at:
6
6
7
7
git://repo.or.cz/qemu/kevin.git tags/for-upstream
8
git://repo.or.cz/qemu/kevin.git tags/for-upstream
8
9
9
for you to fetch changes up to 5e9785505210e2477e590e61b1ab100d0ec22b01:
10
for you to fetch changes up to 51b0a488882328f8f02519bb47ca7e0e7fbe12ff:
10
11
11
qcow2: Fix corruption bug in qcow2_detect_metadata_preallocation() (2019-10-25 15:18:55 +0200)
12
block: Make bdrv_is_allocated_above() byte-based (2017-07-10 13:18:07 +0200)
12
13
13
----------------------------------------------------------------
14
----------------------------------------------------------------
14
Block layer patches:
15
Block layer patches
15
16
- qcow2: Fix data corruption bug that is triggered in partial cluster
17
allocation with default options
18
- qapi: add support for blkreplay driver
19
- doc: Describe missing generic -blockdev options
20
- iotests: Fix 118 when run as root
21
- Minor code cleanups
22
16
23
----------------------------------------------------------------
17
----------------------------------------------------------------
24
Kevin Wolf (5):
18
Daniel P. Berrange (1):
25
iotests: Skip read-only cases in 118 when run as root
19
qemu-img: drop -e and -6 options from the 'create' & 'convert' commands
26
blockdev: Use error_report() in hmp_commit()
27
doc: Describe missing generic -blockdev options
28
coroutine: Add qemu_co_mutex_assert_locked()
29
qcow2: Fix corruption bug in qcow2_detect_metadata_preallocation()
30
20
31
Pavel Dovgaluk (1):
21
Eric Blake (25):
32
qapi: add support for blkreplay driver
22
qemu-io: Don't die on second open
23
block: Guarantee that *file is set on bdrv_get_block_status()
24
block: Simplify use of BDRV_BLOCK_RAW
25
blkdebug: Support .bdrv_co_get_block_status
26
blockjob: Track job ratelimits via bytes, not sectors
27
trace: Show blockjob actions via bytes, not sectors
28
stream: Switch stream_populate() to byte-based
29
stream: Drop reached_end for stream_complete()
30
stream: Switch stream_run() to byte-based
31
commit: Switch commit_populate() to byte-based
32
commit: Switch commit_run() to byte-based
33
mirror: Switch MirrorBlockJob to byte-based
34
mirror: Switch mirror_do_zero_or_discard() to byte-based
35
mirror: Update signature of mirror_clip_sectors()
36
mirror: Switch mirror_cow_align() to byte-based
37
mirror: Switch mirror_do_read() to byte-based
38
mirror: Switch mirror_iteration() to byte-based
39
block: Drop unused bdrv_round_sectors_to_clusters()
40
backup: Switch BackupBlockJob to byte-based
41
backup: Switch block_backup.h to byte-based
42
backup: Switch backup_do_cow() to byte-based
43
backup: Switch backup_run() to byte-based
44
block: Make bdrv_is_allocated() byte-based
45
block: Minimize raw use of bds->total_sectors
46
block: Make bdrv_is_allocated_above() byte-based
33
47
34
Vladimir Sementsov-Ogievskiy (1):
48
Hervé Poussineau (13):
35
block/backup: drop dead code from backup_job_create
49
vvfat: fix qemu-img map and qemu-img convert
50
vvfat: replace tabs by 8 spaces
51
vvfat: fix typos
52
vvfat: rename useless enumeration values
53
vvfat: introduce offset_to_bootsector, offset_to_fat and offset_to_root_dir
54
vvfat: fix field names in FAT12/FAT16 and FAT32 boot sectors
55
vvfat: always create . and .. entries at first and in that order
56
vvfat: correctly create long names for non-ASCII filenames
57
vvfat: correctly create base short names for non-ASCII filenames
58
vvfat: correctly generate numeric-tail of short file names
59
vvfat: limit number of entries in root directory in FAT12/FAT16
60
vvfat: handle KANJI lead byte 0xe5
61
vvfat: change OEM name to 'MSWIN4.1'
36
62
37
qapi/block-core.json | 18 ++++++++++++++++--
63
Thomas Huth (1):
38
include/qemu/coroutine.h | 15 +++++++++++++++
64
blockdev: Print a warning for legacy drive options that belong to -device
39
block/backup.c | 5 +----
40
block/qcow2-refcount.c | 2 ++
41
block/qcow2.c | 3 ++-
42
blockdev.c | 7 +++----
43
qemu-options.hx | 22 +++++++++++++++++++++-
44
tests/qemu-iotests/118 | 3 +++
45
tests/qemu-iotests/iotests.py | 10 ++++++++++
46
9 files changed, 73 insertions(+), 12 deletions(-)
47
65
66
block/backup.c | 128 +--
67
block/blkdebug.c | 11 +
68
block/commit.c | 56 +-
69
block/io.c | 102 +-
70
block/mirror.c | 310 +++---
71
block/raw-format.c | 2 +-
72
block/replication.c | 29 +-
73
block/stream.c | 37 +-
74
block/trace-events | 14 +-
75
block/vpc.c | 2 +-
76
block/vvfat.c | 2336 ++++++++++++++++++++++--------------------
77
blockdev.c | 14 +
78
include/block/block.h | 16 +-
79
include/block/block_backup.h | 11 +-
80
include/qemu/ratelimit.h | 3 +-
81
migration/block.c | 16 +-
82
qemu-img.c | 41 +-
83
qemu-io-cmds.c | 70 +-
84
qemu-io.c | 7 +-
85
qemu-options.hx | 9 +-
86
tests/qemu-iotests/060.out | 1 +
87
tests/qemu-iotests/114.out | 5 +-
88
tests/qemu-iotests/153.out | 6 +
89
tests/qemu-iotests/177 | 3 +
90
tests/qemu-iotests/177.out | 5 +
91
25 files changed, 1675 insertions(+), 1559 deletions(-)
48
92
diff view generated by jsdifflib
[PULL 1/7] qapi: add support for blkreplay driver
Deleted patch
1
From: Pavel Dovgalyuk <pavel.dovgaluk@gmail.com>
2
1
3
This patch adds support for blkreplay driver to the blockdev options.
4
Now blkreplay can be used with -blockdev command line option
5
in the following format:
6
-blockdev driver=blkreplay,image=file-node-name,node-name=replay-node-name
7
8
This option makes possible implementation of the better command
9
line support for record/replay invocations.
10
11
Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
12
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
13
---
14
qapi/block-core.json | 18 ++++++++++++++++--
15
1 file changed, 16 insertions(+), 2 deletions(-)
16
17
diff --git a/qapi/block-core.json b/qapi/block-core.json
18
index XXXXXXX..XXXXXXX 100644
19
--- a/qapi/block-core.json
20
+++ b/qapi/block-core.json
21
@@ -XXX,XX +XXX,XX @@
22
# @nvme: Since 2.12
23
# @copy-on-read: Since 3.0
24
# @blklogwrites: Since 3.0
25
+# @blkreplay: Since 4.2
26
#
27
# Since: 2.9
28
##
29
{ 'enum': 'BlockdevDriver',
30
- 'data': [ 'blkdebug', 'blklogwrites', 'blkverify', 'bochs', 'cloop',
31
- 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps', 'gluster',
32
+ 'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs',
33
+ 'cloop', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps', 'gluster',
34
'host_cdrom', 'host_device', 'http', 'https', 'iscsi', 'luks',
35
'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels', 'qcow',
36
'qcow2', 'qed', 'quorum', 'raw', 'rbd',
37
@@ -XXX,XX +XXX,XX @@
38
'data': { 'test': 'BlockdevRef',
39
'raw': 'BlockdevRef' } }
40
41
+##
42
+# @BlockdevOptionsBlkreplay:
43
+#
44
+# Driver specific block device options for blkreplay.
45
+#
46
+# @image: disk image which should be controlled with blkreplay
47
+#
48
+# Since: 4.2
49
+##
50
+{ 'struct': 'BlockdevOptionsBlkreplay',
51
+ 'data': { 'image': 'BlockdevRef' } }
52
+
53
##
54
# @QuorumReadPattern:
55
#
56
@@ -XXX,XX +XXX,XX @@
57
'blkdebug': 'BlockdevOptionsBlkdebug',
58
'blklogwrites':'BlockdevOptionsBlklogwrites',
59
'blkverify': 'BlockdevOptionsBlkverify',
60
+ 'blkreplay': 'BlockdevOptionsBlkreplay',
61
'bochs': 'BlockdevOptionsGenericFormat',
62
'cloop': 'BlockdevOptionsGenericFormat',
63
'copy-on-read':'BlockdevOptionsGenericFormat',
64
--
65
2.20.1
66
67
diff view generated by jsdifflib
[PULL 2/7] iotests: Skip read-only cases in 118 when run as root
Deleted patch
1
Some tests in 118 use chmod to remove write permissions from the file
2
and assume that the image can indeed not be opened read-write
3
afterwards. This doesn't work when the test is run as root, because root
4
can still open the file as writable even when the permission bit isn't
5
set.
6
1
7
Introduce a @skip_if_root decorator and use it in 118 to skip the tests
8
in question when the script is run as root.
9
10
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
11
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
12
---
13
tests/qemu-iotests/118 | 3 +++
14
tests/qemu-iotests/iotests.py | 10 ++++++++++
15
2 files changed, 13 insertions(+)
16
17
diff --git a/tests/qemu-iotests/118 b/tests/qemu-iotests/118
18
index XXXXXXX..XXXXXXX 100755
19
--- a/tests/qemu-iotests/118
20
+++ b/tests/qemu-iotests/118
21
@@ -XXX,XX +XXX,XX @@ class TestChangeReadOnly(ChangeBaseClass):
22
self.assert_qmp(result, 'return[0]/inserted/ro', True)
23
self.assert_qmp(result, 'return[0]/inserted/image/filename', new_img)
24
25
+ @iotests.skip_if_user_is_root
26
def test_rw_ro_retain(self):
27
os.chmod(new_img, 0o444)
28
self.vm.add_drive(old_img, 'media=disk', 'none')
29
@@ -XXX,XX +XXX,XX @@ class TestChangeReadOnly(ChangeBaseClass):
30
self.assert_qmp(result, 'return[0]/inserted/ro', True)
31
self.assert_qmp(result, 'return[0]/inserted/image/filename', new_img)
32
33
+ @iotests.skip_if_user_is_root
34
def test_make_ro_rw(self):
35
os.chmod(new_img, 0o444)
36
self.vm.add_drive(old_img, 'media=disk', 'none')
37
@@ -XXX,XX +XXX,XX @@ class TestChangeReadOnly(ChangeBaseClass):
38
self.assert_qmp(result, 'return[0]/inserted/ro', True)
39
self.assert_qmp(result, 'return[0]/inserted/image/filename', new_img)
40
41
+ @iotests.skip_if_user_is_root
42
def test_make_ro_rw_by_retain(self):
43
os.chmod(new_img, 0o444)
44
self.vm.add_drive(old_img, 'media=disk', 'none')
45
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
46
index XXXXXXX..XXXXXXX 100644
47
--- a/tests/qemu-iotests/iotests.py
48
+++ b/tests/qemu-iotests/iotests.py
49
@@ -XXX,XX +XXX,XX @@ def skip_if_unsupported(required_formats=[], read_only=False):
50
return func_wrapper
51
return skip_test_decorator
52
53
+def skip_if_user_is_root(func):
54
+ '''Skip Test Decorator
55
+ Runs the test only without root permissions'''
56
+ def func_wrapper(*args, **kwargs):
57
+ if os.getuid() == 0:
58
+ case_notrun('{}: cannot be run as root'.format(args[0]))
59
+ else:
60
+ return func(*args, **kwargs)
61
+ return func_wrapper
62
+
63
def execute_unittest(output, verbosity, debug):
64
runner = unittest.TextTestRunner(stream=output, descriptions=True,
65
verbosity=verbosity)
66
--
67
2.20.1
68
69
diff view generated by jsdifflib
[PULL 3/7] blockdev: Use error_report() in hmp_commit()
Deleted patch
1
Instead of using monitor_printf() to report errors, hmp_commit() should
2
use error_report() like other places do.
3
1
4
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
5
Reviewed-by: Eric Blake <eblake@redhat.com>
6
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
7
---
8
blockdev.c | 7 +++----
9
1 file changed, 3 insertions(+), 4 deletions(-)
10
11
diff --git a/blockdev.c b/blockdev.c
12
index XXXXXXX..XXXXXXX 100644
13
--- a/blockdev.c
14
+++ b/blockdev.c
15
@@ -XXX,XX +XXX,XX @@ void hmp_commit(Monitor *mon, const QDict *qdict)
16
17
blk = blk_by_name(device);
18
if (!blk) {
19
- monitor_printf(mon, "Device '%s' not found\n", device);
20
+ error_report("Device '%s' not found", device);
21
return;
22
}
23
if (!blk_is_available(blk)) {
24
- monitor_printf(mon, "Device '%s' has no medium\n", device);
25
+ error_report("Device '%s' has no medium", device);
26
return;
27
}
28
29
@@ -XXX,XX +XXX,XX @@ void hmp_commit(Monitor *mon, const QDict *qdict)
30
aio_context_release(aio_context);
31
}
32
if (ret < 0) {
33
- monitor_printf(mon, "'commit' error for '%s': %s\n", device,
34
- strerror(-ret));
35
+ error_report("'commit' error for '%s': %s", device, strerror(-ret));
36
}
37
}
38
39
--
40
2.20.1
41
42
diff view generated by jsdifflib
[PULL 4/7] block/backup: drop dead code from backup_job_create
Deleted patch
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
1
3
After commit 00e30f05de1d195, there is no more "goto error" points
4
after job creation, so after "error:" @job is always NULL and we don't
5
need roll-back job creation.
6
7
Reported-by: Coverity (CID 1406402)
8
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
9
Acked-by: Stefano Garzarella <sgarzare@redhat.com>
10
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
11
---
12
block/backup.c | 5 +----
13
1 file changed, 1 insertion(+), 4 deletions(-)
14
15
diff --git a/block/backup.c b/block/backup.c
16
index XXXXXXX..XXXXXXX 100644
17
--- a/block/backup.c
18
+++ b/block/backup.c
19
@@ -XXX,XX +XXX,XX @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
20
if (sync_bitmap) {
21
bdrv_reclaim_dirty_bitmap(sync_bitmap, NULL);
22
}
23
- if (job) {
24
- backup_clean(&job->common.job);
25
- job_early_fail(&job->common.job);
26
- } else if (backup_top) {
27
+ if (backup_top) {
28
bdrv_backup_top_drop(backup_top);
29
}
30
31
--
32
2.20.1
33
34
diff view generated by jsdifflib
[PULL 5/7] doc: Describe missing generic -blockdev options
Deleted patch
1
We added more generic options after introducing -blockdev and forgot to
2
update the documentation (man page and --help output) accordingly. Do
3
that now.
4
1
5
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
6
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
7
---
8
qemu-options.hx | 22 +++++++++++++++++++++-
9
1 file changed, 21 insertions(+), 1 deletion(-)
10
11
diff --git a/qemu-options.hx b/qemu-options.hx
12
index XXXXXXX..XXXXXXX 100644
13
--- a/qemu-options.hx
14
+++ b/qemu-options.hx
15
@@ -XXX,XX +XXX,XX @@ ETEXI
16
DEF("blockdev", HAS_ARG, QEMU_OPTION_blockdev,
17
"-blockdev [driver=]driver[,node-name=N][,discard=ignore|unmap]\n"
18
" [,cache.direct=on|off][,cache.no-flush=on|off]\n"
19
- " [,read-only=on|off][,detect-zeroes=on|off|unmap]\n"
20
+ " [,read-only=on|off][,auto-read-only=on|off]\n"
21
+ " [,force-share=on|off][,detect-zeroes=on|off|unmap]\n"
22
" [,driver specific parameters...]\n"
23
" configure a block backend\n", QEMU_ARCH_ALL)
24
STEXI
25
@@ -XXX,XX +XXX,XX @@ name is not intended to be predictable and changes between QEMU invocations.
26
For the top level, an explicit node name must be specified.
27
@item read-only
28
Open the node read-only. Guest write attempts will fail.
29
+
30
+Note that some block drivers support only read-only access, either generally or
31
+in certain configurations. In this case, the default value
32
+@option{read-only=off} does not work and the option must be specified
33
+explicitly.
34
+@item auto-read-only
35
+If @option{auto-read-only=on} is set, QEMU may fall back to read-only usage
36
+even when @option{read-only=off} is requested, or even switch between modes as
37
+needed, e.g. depending on whether the image file is writable or whether a
38
+writing user is attached to the node.
39
+@item force-share
40
+Override the image locking system of QEMU by forcing the node to utilize
41
+weaker shared access for permissions where it would normally request exclusive
42
+access. When there is the potential for multiple instances to have the same
43
+file open (whether this invocation of QEMU is the first or the second
44
+instance), both instances must permit shared access for the second instance to
45
+succeed at opening the file.
46
+
47
+Enabling @option{force-share=on} requires @option{read-only=on}.
48
@item cache.direct
49
The host page cache can be avoided with @option{cache.direct=on}. This will
50
attempt to do disk IO directly to the guest's memory. QEMU may still perform an
51
--
52
2.20.1
53
54
diff view generated by jsdifflib
[PULL 6/7] coroutine: Add qemu_co_mutex_assert_locked()
Deleted patch
1
Some functions require that the caller holds a certain CoMutex for them
2
to operate correctly. Add a function so that they can assert the lock is
3
really held.
4
1
5
Cc: qemu-stable@nongnu.org
6
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
7
Tested-by: Michael Weiser <michael.weiser@gmx.de>
8
Reviewed-by: Michael Weiser <michael.weiser@gmx.de>
9
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
10
Reviewed-by: Denis V. Lunev <den@openvz.org>
11
Reviewed-by: Max Reitz <mreitz@redhat.com>
12
---
13
include/qemu/coroutine.h | 15 +++++++++++++++
14
1 file changed, 15 insertions(+)
15
16
diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h
17
index XXXXXXX..XXXXXXX 100644
18
--- a/include/qemu/coroutine.h
19
+++ b/include/qemu/coroutine.h
20
@@ -XXX,XX +XXX,XX @@ void coroutine_fn qemu_co_mutex_lock(CoMutex *mutex);
21
*/
22
void coroutine_fn qemu_co_mutex_unlock(CoMutex *mutex);
23
24
+/**
25
+ * Assert that the current coroutine holds @mutex.
26
+ */
27
+static inline coroutine_fn void qemu_co_mutex_assert_locked(CoMutex *mutex)
28
+{
29
+ /*
30
+ * mutex->holder doesn't need any synchronisation if the assertion holds
31
+ * true because the mutex protects it. If it doesn't hold true, we still
32
+ * don't mind if another thread takes or releases mutex behind our back,
33
+ * because the condition will be false no matter whether we read NULL or
34
+ * the pointer for any other coroutine.
35
+ */
36
+ assert(atomic_read(&mutex->locked) &&
37
+ mutex->holder == qemu_coroutine_self());
38
+}
39
40
/**
41
* CoQueues are a mechanism to queue coroutines in order to continue executing
42
--
43
2.20.1
44
45
diff view generated by jsdifflib
[PULL 7/7] qcow2: Fix corruption bug in qcow2_detect_metadata_preallocation()
Deleted patch
1
qcow2_detect_metadata_preallocation() calls qcow2_get_refcount() which
2
requires s->lock to be taken to protect its accesses to the refcount
3
table and refcount blocks. However, nothing in this code path actually
4
took the lock. This could cause the same cache entry to be used by two
5
requests at the same time, for different tables at different offsets,
6
resulting in image corruption.
7
1
8
As it would be preferable to base the detection on consistent data (even
9
though it's just heuristics), let's take the lock not only around the
10
qcow2_get_refcount() calls, but around the whole function.
11
12
This patch takes the lock in qcow2_co_block_status() earlier and asserts
13
in qcow2_detect_metadata_preallocation() that we hold the lock.
14
15
Fixes: 69f47505ee66afaa513305de0c1895a224e52c45
16
Cc: qemu-stable@nongnu.org
17
Reported-by: Michael Weiser <michael.weiser@gmx.de>
18
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
19
Tested-by: Michael Weiser <michael.weiser@gmx.de>
20
Reviewed-by: Michael Weiser <michael.weiser@gmx.de>
21
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
22
Reviewed-by: Max Reitz <mreitz@redhat.com>
23
---
24
block/qcow2-refcount.c | 2 ++
25
block/qcow2.c | 3 ++-
26
2 files changed, 4 insertions(+), 1 deletion(-)
27
28
diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
29
index XXXXXXX..XXXXXXX 100644
30
--- a/block/qcow2-refcount.c
31
+++ b/block/qcow2-refcount.c
32
@@ -XXX,XX +XXX,XX @@ int qcow2_detect_metadata_preallocation(BlockDriverState *bs)
33
int64_t i, end_cluster, cluster_count = 0, threshold;
34
int64_t file_length, real_allocation, real_clusters;
35
36
+ qemu_co_mutex_assert_locked(&s->lock);
37
+
38
file_length = bdrv_getlength(bs->file->bs);
39
if (file_length < 0) {
40
return file_length;
41
diff --git a/block/qcow2.c b/block/qcow2.c
42
index XXXXXXX..XXXXXXX 100644
43
--- a/block/qcow2.c
44
+++ b/block/qcow2.c
45
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,
46
unsigned int bytes;
47
int status = 0;
48
49
+ qemu_co_mutex_lock(&s->lock);
50
+
51
if (!s->metadata_preallocation_checked) {
52
ret = qcow2_detect_metadata_preallocation(bs);
53
s->metadata_preallocation = (ret == 1);
54
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,
55
}
56
57
bytes = MIN(INT_MAX, count);
58
- qemu_co_mutex_lock(&s->lock);
59
ret = qcow2_get_cluster_offset(bs, offset, &bytes, &cluster_offset);
60
qemu_co_mutex_unlock(&s->lock);
61
if (ret < 0) {
62
--
63
2.20.1
64
65
diff view generated by jsdifflib

Patchew 2.1-devel-b67e4c2

© 2016 - 2025 Red Hat, Inc.