[PULL 00/34] Block patches
[PULL 00/23] Block patches
1
The following changes since commit 30aa19446d82358a30eac3b556b4d6641e00b7c1:
1
The following changes since commit 9d662a6b22a0838a85c5432385f35db2488a33a5:
2
2
3
Merge remote-tracking branch 'remotes/cschoenebeck/tags/pull-9p-20200812' into staging (2020-08-24 16:39:53 +0100)
3
Merge remote-tracking branch 'remotes/legoater/tags/pull-ppc-20220305' into staging (2022-03-05 18:03:15 +0000)
4
4
5
are available in the Git repository at:
5
are available in the Git repository at:
6
6
7
https://github.com/XanClic/qemu.git tags/pull-block-2020-08-25
7
https://gitlab.com/hreitz/qemu.git tags/pull-block-2022-03-07
8
8
9
for you to fetch changes up to c576fd97d4ca77b5a1a27728df11a61083dbfa98:
9
for you to fetch changes up to 743da0b401cdc3ee94bc519975e339a3cdbe0ad1:
10
10
11
iotests: Add tests for qcow2 images with extended L2 entries (2020-08-25 10:20:18 +0200)
11
iotests/image-fleecing: test push backup with fleecing (2022-03-07 09:33:31 +0100)
12
12
13
----------------------------------------------------------------
13
----------------------------------------------------------------
14
Block patches:
14
Block patches for 7.0-rc0:
15
- qcow2 subclusters (extended L2 entries)
15
- New fleecing backup scheme
16
- iotest fixes
17
- Fixes for the curl block driver
18
- Fix for the preallocate block driver
19
- IDE fix for zero-length TRIM requests
16
20
17
----------------------------------------------------------------
21
----------------------------------------------------------------
18
Alberto Garcia (34):
22
Hanna Reitz (2):
19
qcow2: Make Qcow2AioTask store the full host offset
23
ide: Increment BB in-flight counter for TRIM BH
20
qcow2: Convert qcow2_get_cluster_offset() into qcow2_get_host_offset()
24
iotests: Write test output to TEST_DIR
21
qcow2: Add calculate_l2_meta()
22
qcow2: Split cluster_needs_cow() out of count_cow_clusters()
23
qcow2: Process QCOW2_CLUSTER_ZERO_ALLOC clusters in handle_copied()
24
qcow2: Add get_l2_entry() and set_l2_entry()
25
qcow2: Document the Extended L2 Entries feature
26
qcow2: Add dummy has_subclusters() function
27
qcow2: Add subcluster-related fields to BDRVQcow2State
28
qcow2: Add offset_to_sc_index()
29
qcow2: Add offset_into_subcluster() and size_to_subclusters()
30
qcow2: Add l2_entry_size()
31
qcow2: Update get/set_l2_entry() and add get/set_l2_bitmap()
32
qcow2: Add QCow2SubclusterType and qcow2_get_subcluster_type()
33
qcow2: Add qcow2_get_subcluster_range_type()
34
qcow2: Add qcow2_cluster_is_allocated()
35
qcow2: Add cluster type parameter to qcow2_get_host_offset()
36
qcow2: Replace QCOW2_CLUSTER_* with QCOW2_SUBCLUSTER_*
37
qcow2: Handle QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC
38
qcow2: Add subcluster support to calculate_l2_meta()
39
qcow2: Add subcluster support to qcow2_get_host_offset()
40
qcow2: Add subcluster support to zero_in_l2_slice()
41
qcow2: Add subcluster support to discard_in_l2_slice()
42
qcow2: Add subcluster support to check_refcounts_l2()
43
qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2()
44
qcow2: Clear the L2 bitmap when allocating a compressed cluster
45
qcow2: Add subcluster support to handle_alloc_space()
46
qcow2: Add subcluster support to qcow2_co_pwrite_zeroes()
47
qcow2: Add subcluster support to qcow2_measure()
48
qcow2: Add prealloc field to QCowL2Meta
49
qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit
50
qcow2: Allow preallocation and backing files if extended_l2 is set
51
qcow2: Assert that expand_zero_clusters_in_l1() does not support
52
subclusters
53
iotests: Add tests for qcow2 images with extended L2 entries
54
25
55
docs/interop/qcow2.txt | 68 ++-
26
Peter Maydell (2):
56
docs/qcow2-cache.txt | 19 +-
27
block/curl.c: Set error message string if curl_init_state() fails
57
qapi/block-core.json | 7 +
28
block/curl.c: Check error return from curl_easy_setopt()
58
block/qcow2.h | 211 ++++++-
29
59
include/block/block_int.h | 1 +
30
Thomas Huth (2):
60
block/qcow2-cluster.c | 906 +++++++++++++++++++++----------
31
tests/qemu-iotests/040: Skip TestCommitWithFilters without 'throttle'
61
block/qcow2-refcount.c | 47 +-
32
tests/qemu-iotests/testrunner: Quote "case not run" lines in TAP mode
62
block/qcow2.c | 302 +++++++----
33
63
block/trace-events | 2 +-
34
Vladimir Sementsov-Ogievskiy (17):
64
tests/qemu-iotests/031.out | 8 +-
35
block: fix preallocate filter: don't do unaligned preallocate requests
65
tests/qemu-iotests/036.out | 4 +-
36
block/block-copy: move copy_bitmap initialization to
66
tests/qemu-iotests/049.out | 102 ++--
37
block_copy_state_new()
67
tests/qemu-iotests/060.out | 3 +-
38
block/dirty-bitmap: bdrv_merge_dirty_bitmap(): add return value
68
tests/qemu-iotests/061 | 6 +
39
block/block-copy: block_copy_state_new(): add bitmap parameter
69
tests/qemu-iotests/061.out | 25 +-
40
block/copy-before-write: add bitmap open parameter
70
tests/qemu-iotests/065 | 12 +-
41
block/block-copy: add block_copy_reset()
71
tests/qemu-iotests/082.out | 39 +-
42
block: intoduce reqlist
72
tests/qemu-iotests/085.out | 38 +-
43
block/reqlist: reqlist_find_conflict(): use ranges_overlap()
73
tests/qemu-iotests/144.out | 4 +-
44
block/dirty-bitmap: introduce bdrv_dirty_bitmap_status()
74
tests/qemu-iotests/182.out | 2 +-
45
block/reqlist: add reqlist_wait_all()
75
tests/qemu-iotests/185.out | 8 +-
46
block/io: introduce block driver snapshot-access API
76
tests/qemu-iotests/198 | 2 +
47
block: introduce snapshot-access block driver
77
tests/qemu-iotests/206.out | 6 +-
48
block: copy-before-write: realize snapshot-access API
78
tests/qemu-iotests/242.out | 5 +
49
iotests/image-fleecing: add test-case for fleecing format node
79
tests/qemu-iotests/255.out | 8 +-
50
iotests.py: add qemu_io_pipe_and_status()
80
tests/qemu-iotests/271 | 901 ++++++++++++++++++++++++++++++
51
iotests/image-fleecing: add test case with bitmap
81
tests/qemu-iotests/271.out | 726 +++++++++++++++++++++++++
52
iotests/image-fleecing: test push backup with fleecing
82
tests/qemu-iotests/274.out | 49 +-
53
83
tests/qemu-iotests/280.out | 2 +-
54
qapi/block-core.json | 14 +-
84
tests/qemu-iotests/291.out | 2 +
55
include/block/block-common.h | 3 +-
85
tests/qemu-iotests/302.out | 1 +
56
include/block/block-copy.h | 2 +
86
tests/qemu-iotests/303.out | 4 +-
57
include/block/block_int-common.h | 24 ++
87
tests/qemu-iotests/common.filter | 1 +
58
include/block/block_int-io.h | 9 +
88
tests/qemu-iotests/group | 1 +
59
include/block/dirty-bitmap.h | 4 +-
89
34 files changed, 2952 insertions(+), 570 deletions(-)
60
include/block/reqlist.h | 75 ++++++
90
create mode 100755 tests/qemu-iotests/271
61
include/qemu/hbitmap.h | 12 +
91
create mode 100644 tests/qemu-iotests/271.out
62
block/block-copy.c | 150 +++++------
63
block/copy-before-write.c | 265 +++++++++++++++++++-
64
block/curl.c | 92 ++++---
65
block/dirty-bitmap.c | 15 +-
66
block/io.c | 76 ++++++
67
block/monitor/bitmap-qmp-cmds.c | 5 +-
68
block/preallocate.c | 15 +-
69
block/reqlist.c | 85 +++++++
70
block/snapshot-access.c | 132 ++++++++++
71
hw/ide/core.c | 7 +
72
util/hbitmap.c | 33 +++
73
MAINTAINERS | 5 +-
74
block/meson.build | 2 +
75
tests/qemu-iotests/040 | 1 +
76
tests/qemu-iotests/257.out | 224 +++++++++++++++++
77
tests/qemu-iotests/common.rc | 6 +-
78
tests/qemu-iotests/iotests.py | 8 +-
79
tests/qemu-iotests/testenv.py | 5 +-
80
tests/qemu-iotests/testrunner.py | 19 +-
81
tests/qemu-iotests/tests/image-fleecing | 185 +++++++++++---
82
tests/qemu-iotests/tests/image-fleecing.out | 221 +++++++++++++++-
83
29 files changed, 1499 insertions(+), 195 deletions(-)
84
create mode 100644 include/block/reqlist.h
85
create mode 100644 block/reqlist.c
86
create mode 100644 block/snapshot-access.c
92
87
93
--
88
--
94
2.26.2
89
2.34.1
95
96
diff view generated by jsdifflib
[PULL 21/34] qcow2: Add subcluster support to qcow2_get_host_offset()
[PULL 01/23] ide: Increment BB in-flight counter for TRIM BH
1
From: Alberto Garcia <berto@igalia.com>
1
When we still have an AIOCB registered for DMA operations, we try to
2
settle the respective operation by draining the BlockBackend associated
3
with the IDE device.
2
4
3
The logic of this function remains pretty much the same, except that
5
However, this assumes that every DMA operation is associated with an
4
it uses count_contiguous_subclusters(), which combines the logic of
6
increment of the BlockBackend’s in-flight counter (e.g. through some
5
count_contiguous_clusters() / count_contiguous_clusters_unallocated()
7
ongoing I/O operation), so that draining the BB until its in-flight
6
and checks individual subclusters.
8
counter reaches 0 will settle all DMA operations. That is not the case:
9
For TRIM, the guest can issue a zero-length operation that will not
10
result in any I/O operation forwarded to the BlockBackend, and also not
11
increment the in-flight counter in any other way. In such a case,
12
blk_drain() will be a no-op if no other operations are in flight.
7
13
8
qcow2_cluster_to_subcluster_type() is not necessary as a separate
14
It is clear that if blk_drain() is a no-op, the value of
9
function anymore so it's inlined into its caller.
15
s->bus->dma->aiocb will not change between checking it in the `if`
16
condition and asserting that it is NULL after blk_drain().
10
17
11
Signed-off-by: Alberto Garcia <berto@igalia.com>
18
The particular problem is that ide_issue_trim() creates a BH
12
Reviewed-by: Max Reitz <mreitz@redhat.com>
19
(ide_trim_bh_cb()) to settle the TRIM request: iocb->common.cb() is
13
Message-Id: <d2193fd48653a350d80f0eca1c67b1d9053fb2f3.1594396418.git.berto@igalia.com>
20
ide_dma_cb(), which will either create a new request, or find the
14
[mreitz: Initialize expected_type to anything]
21
transfer to be done and call ide_set_inactive(), which clears
15
Signed-off-by: Max Reitz <mreitz@redhat.com>
22
s->bus->dma->aiocb. Therefore, the blk_drain() must wait for
23
ide_trim_bh_cb() to run, which currently it will not always do.
24
25
To fix this issue, we increment the BlockBackend's in-flight counter
26
when the TRIM operation begins (in ide_issue_trim(), when the
27
ide_trim_bh_cb() BH is created) and decrement it when ide_trim_bh_cb()
28
is done.
29
30
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2029980
31
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
32
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
33
Message-Id: <20220120142259.120189-1-hreitz@redhat.com>
34
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
35
Reviewed-by: John Snow <jsnow@redhat.com>
36
Tested-by: John Snow <jsnow@redhat.com>
16
---
37
---
17
block/qcow2.h | 38 ++++-------
38
hw/ide/core.c | 7 +++++++
18
block/qcow2-cluster.c | 150 ++++++++++++++++++++++--------------------
39
1 file changed, 7 insertions(+)
19
2 files changed, 92 insertions(+), 96 deletions(-)
20
40
21
diff --git a/block/qcow2.h b/block/qcow2.h
41
diff --git a/hw/ide/core.c b/hw/ide/core.c
22
index XXXXXXX..XXXXXXX 100644
42
index XXXXXXX..XXXXXXX 100644
23
--- a/block/qcow2.h
43
--- a/hw/ide/core.c
24
+++ b/block/qcow2.h
44
+++ b/hw/ide/core.c
25
@@ -XXX,XX +XXX,XX @@ static inline QCow2ClusterType qcow2_get_cluster_type(BlockDriverState *bs,
45
@@ -XXX,XX +XXX,XX @@ static const AIOCBInfo trim_aiocb_info = {
26
}
46
static void ide_trim_bh_cb(void *opaque)
47
{
48
TrimAIOCB *iocb = opaque;
49
+ BlockBackend *blk = iocb->s->blk;
50
51
iocb->common.cb(iocb->common.opaque, iocb->ret);
52
53
qemu_bh_delete(iocb->bh);
54
iocb->bh = NULL;
55
qemu_aio_unref(iocb);
56
+
57
+ /* Paired with an increment in ide_issue_trim() */
58
+ blk_dec_in_flight(blk);
27
}
59
}
28
60
29
-/*
61
static void ide_issue_trim_cb(void *opaque, int ret)
30
- * For an image without extended L2 entries, return the
62
@@ -XXX,XX +XXX,XX @@ BlockAIOCB *ide_issue_trim(
31
- * QCow2SubclusterType equivalent of a given QCow2ClusterType.
63
IDEState *s = opaque;
32
- */
64
TrimAIOCB *iocb;
33
-static inline
65
34
-QCow2SubclusterType qcow2_cluster_to_subcluster_type(QCow2ClusterType type)
66
+ /* Paired with a decrement in ide_trim_bh_cb() */
35
-{
67
+ blk_inc_in_flight(s->blk);
36
- switch (type) {
68
+
37
- case QCOW2_CLUSTER_COMPRESSED:
69
iocb = blk_aio_get(&trim_aiocb_info, s->blk, cb, cb_opaque);
38
- return QCOW2_SUBCLUSTER_COMPRESSED;
70
iocb->s = s;
39
- case QCOW2_CLUSTER_ZERO_PLAIN:
71
iocb->bh = qemu_bh_new(ide_trim_bh_cb, iocb);
40
- return QCOW2_SUBCLUSTER_ZERO_PLAIN;
41
- case QCOW2_CLUSTER_ZERO_ALLOC:
42
- return QCOW2_SUBCLUSTER_ZERO_ALLOC;
43
- case QCOW2_CLUSTER_NORMAL:
44
- return QCOW2_SUBCLUSTER_NORMAL;
45
- case QCOW2_CLUSTER_UNALLOCATED:
46
- return QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN;
47
- default:
48
- g_assert_not_reached();
49
- }
50
-}
51
-
52
/*
53
* In an image without subsclusters @l2_bitmap is ignored and
54
* @sc_index must be 0.
55
@@ -XXX,XX +XXX,XX @@ QCow2SubclusterType qcow2_get_subcluster_type(BlockDriverState *bs,
56
g_assert_not_reached();
57
}
58
} else {
59
- return qcow2_cluster_to_subcluster_type(type);
60
+ switch (type) {
61
+ case QCOW2_CLUSTER_COMPRESSED:
62
+ return QCOW2_SUBCLUSTER_COMPRESSED;
63
+ case QCOW2_CLUSTER_ZERO_PLAIN:
64
+ return QCOW2_SUBCLUSTER_ZERO_PLAIN;
65
+ case QCOW2_CLUSTER_ZERO_ALLOC:
66
+ return QCOW2_SUBCLUSTER_ZERO_ALLOC;
67
+ case QCOW2_CLUSTER_NORMAL:
68
+ return QCOW2_SUBCLUSTER_NORMAL;
69
+ case QCOW2_CLUSTER_UNALLOCATED:
70
+ return QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN;
71
+ default:
72
+ g_assert_not_reached();
73
+ }
74
}
75
}
76
77
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
78
index XXXXXXX..XXXXXXX 100644
79
--- a/block/qcow2-cluster.c
80
+++ b/block/qcow2-cluster.c
81
@@ -XXX,XX +XXX,XX @@ static int qcow2_get_subcluster_range_type(BlockDriverState *bs,
82
}
83
84
/*
85
- * Checks how many clusters in a given L2 slice are contiguous in the image
86
- * file. As soon as one of the flags in the bitmask stop_flags changes compared
87
- * to the first cluster, the search is stopped and the cluster is not counted
88
- * as contiguous. (This allows it, for example, to stop at the first compressed
89
- * cluster which may require a different handling)
90
+ * Return the number of contiguous subclusters of the exact same type
91
+ * in a given L2 slice, starting from cluster @l2_index, subcluster
92
+ * @sc_index. Allocated subclusters are required to be contiguous in
93
+ * the image file.
94
+ * At most @nb_clusters are checked (note that this means clusters,
95
+ * not subclusters).
96
+ * Compressed clusters are always processed one by one but for the
97
+ * purpose of this count they are treated as if they were divided into
98
+ * subclusters of size s->subcluster_size.
99
+ * On failure return -errno and update @l2_index to point to the
100
+ * invalid entry.
101
*/
102
-static int count_contiguous_clusters(BlockDriverState *bs, int nb_clusters,
103
- int cluster_size, uint64_t *l2_slice, int l2_index, uint64_t stop_flags)
104
+static int count_contiguous_subclusters(BlockDriverState *bs, int nb_clusters,
105
+ unsigned sc_index, uint64_t *l2_slice,
106
+ unsigned *l2_index)
107
{
108
BDRVQcow2State *s = bs->opaque;
109
- int i;
110
- QCow2ClusterType first_cluster_type;
111
- uint64_t mask = stop_flags | L2E_OFFSET_MASK | QCOW_OFLAG_COMPRESSED;
112
- uint64_t first_entry = get_l2_entry(s, l2_slice, l2_index);
113
- uint64_t offset = first_entry & mask;
114
+ int i, count = 0;
115
+ bool check_offset = false;
116
+ uint64_t expected_offset = 0;
117
+ QCow2SubclusterType expected_type = QCOW2_SUBCLUSTER_NORMAL, type;
118
119
- first_cluster_type = qcow2_get_cluster_type(bs, first_entry);
120
- if (first_cluster_type == QCOW2_CLUSTER_UNALLOCATED) {
121
- return 0;
122
- }
123
-
124
- /* must be allocated */
125
- assert(first_cluster_type == QCOW2_CLUSTER_NORMAL ||
126
- first_cluster_type == QCOW2_CLUSTER_ZERO_ALLOC);
127
+ assert(*l2_index + nb_clusters <= s->l2_slice_size);
128
129
for (i = 0; i < nb_clusters; i++) {
130
- uint64_t l2_entry = get_l2_entry(s, l2_slice, l2_index + i) & mask;
131
- if (offset + (uint64_t) i * cluster_size != l2_entry) {
132
+ unsigned first_sc = (i == 0) ? sc_index : 0;
133
+ uint64_t l2_entry = get_l2_entry(s, l2_slice, *l2_index + i);
134
+ uint64_t l2_bitmap = get_l2_bitmap(s, l2_slice, *l2_index + i);
135
+ int ret = qcow2_get_subcluster_range_type(bs, l2_entry, l2_bitmap,
136
+ first_sc, &type);
137
+ if (ret < 0) {
138
+ *l2_index += i; /* Point to the invalid entry */
139
+ return -EIO;
140
+ }
141
+ if (i == 0) {
142
+ if (type == QCOW2_SUBCLUSTER_COMPRESSED) {
143
+ /* Compressed clusters are always processed one by one */
144
+ return ret;
145
+ }
146
+ expected_type = type;
147
+ expected_offset = l2_entry & L2E_OFFSET_MASK;
148
+ check_offset = (type == QCOW2_SUBCLUSTER_NORMAL ||
149
+ type == QCOW2_SUBCLUSTER_ZERO_ALLOC ||
150
+ type == QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC);
151
+ } else if (type != expected_type) {
152
break;
153
+ } else if (check_offset) {
154
+ expected_offset += s->cluster_size;
155
+ if (expected_offset != (l2_entry & L2E_OFFSET_MASK)) {
156
+ break;
157
+ }
158
}
159
- }
160
-
161
- return i;
162
-}
163
-
164
-/*
165
- * Checks how many consecutive unallocated clusters in a given L2
166
- * slice have the same cluster type.
167
- */
168
-static int count_contiguous_clusters_unallocated(BlockDriverState *bs,
169
- int nb_clusters,
170
- uint64_t *l2_slice,
171
- int l2_index,
172
- QCow2ClusterType wanted_type)
173
-{
174
- BDRVQcow2State *s = bs->opaque;
175
- int i;
176
-
177
- assert(wanted_type == QCOW2_CLUSTER_ZERO_PLAIN ||
178
- wanted_type == QCOW2_CLUSTER_UNALLOCATED);
179
- for (i = 0; i < nb_clusters; i++) {
180
- uint64_t entry = get_l2_entry(s, l2_slice, l2_index + i);
181
- QCow2ClusterType type = qcow2_get_cluster_type(bs, entry);
182
-
183
- if (type != wanted_type) {
184
+ count += ret;
185
+ /* Stop if there are type changes before the end of the cluster */
186
+ if (first_sc + ret < s->subclusters_per_cluster) {
187
break;
188
}
189
}
190
191
- return i;
192
+ return count;
193
}
194
195
static int coroutine_fn do_perform_cow_read(BlockDriverState *bs,
196
@@ -XXX,XX +XXX,XX @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
197
QCow2SubclusterType *subcluster_type)
198
{
199
BDRVQcow2State *s = bs->opaque;
200
- unsigned int l2_index;
201
- uint64_t l1_index, l2_offset, *l2_slice, l2_entry;
202
- int c;
203
+ unsigned int l2_index, sc_index;
204
+ uint64_t l1_index, l2_offset, *l2_slice, l2_entry, l2_bitmap;
205
+ int sc;
206
unsigned int offset_in_cluster;
207
uint64_t bytes_available, bytes_needed, nb_clusters;
208
- QCow2ClusterType type;
209
+ QCow2SubclusterType type;
210
int ret;
211
212
offset_in_cluster = offset_into_cluster(s, offset);
213
@@ -XXX,XX +XXX,XX @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
214
215
l1_index = offset_to_l1_index(s, offset);
216
if (l1_index >= s->l1_size) {
217
- type = QCOW2_CLUSTER_UNALLOCATED;
218
+ type = QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN;
219
goto out;
220
}
221
222
l2_offset = s->l1_table[l1_index] & L1E_OFFSET_MASK;
223
if (!l2_offset) {
224
- type = QCOW2_CLUSTER_UNALLOCATED;
225
+ type = QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN;
226
goto out;
227
}
228
229
@@ -XXX,XX +XXX,XX @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
230
/* find the cluster offset for the given disk offset */
231
232
l2_index = offset_to_l2_slice_index(s, offset);
233
+ sc_index = offset_to_sc_index(s, offset);
234
l2_entry = get_l2_entry(s, l2_slice, l2_index);
235
+ l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index);
236
237
nb_clusters = size_to_clusters(s, bytes_needed);
238
/* bytes_needed <= *bytes + offset_in_cluster, both of which are unsigned
239
@@ -XXX,XX +XXX,XX @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
240
* true */
241
assert(nb_clusters <= INT_MAX);
242
243
- type = qcow2_get_cluster_type(bs, l2_entry);
244
- if (s->qcow_version < 3 && (type == QCOW2_CLUSTER_ZERO_PLAIN ||
245
- type == QCOW2_CLUSTER_ZERO_ALLOC)) {
246
+ type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, sc_index);
247
+ if (s->qcow_version < 3 && (type == QCOW2_SUBCLUSTER_ZERO_PLAIN ||
248
+ type == QCOW2_SUBCLUSTER_ZERO_ALLOC)) {
249
qcow2_signal_corruption(bs, true, -1, -1, "Zero cluster entry found"
250
" in pre-v3 image (L2 offset: %#" PRIx64
251
", L2 index: %#x)", l2_offset, l2_index);
252
@@ -XXX,XX +XXX,XX @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
253
goto fail;
254
}
255
switch (type) {
256
- case QCOW2_CLUSTER_COMPRESSED:
257
+ case QCOW2_SUBCLUSTER_INVALID:
258
+ break; /* This is handled by count_contiguous_subclusters() below */
259
+ case QCOW2_SUBCLUSTER_COMPRESSED:
260
if (has_data_file(bs)) {
261
qcow2_signal_corruption(bs, true, -1, -1, "Compressed cluster "
262
"entry found in image with external data "
263
@@ -XXX,XX +XXX,XX @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
264
ret = -EIO;
265
goto fail;
266
}
267
- /* Compressed clusters can only be processed one by one */
268
- c = 1;
269
*host_offset = l2_entry & L2E_COMPRESSED_OFFSET_SIZE_MASK;
270
break;
271
- case QCOW2_CLUSTER_ZERO_PLAIN:
272
- case QCOW2_CLUSTER_UNALLOCATED:
273
- /* how many empty clusters ? */
274
- c = count_contiguous_clusters_unallocated(bs, nb_clusters,
275
- l2_slice, l2_index, type);
276
+ case QCOW2_SUBCLUSTER_ZERO_PLAIN:
277
+ case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN:
278
break;
279
- case QCOW2_CLUSTER_ZERO_ALLOC:
280
- case QCOW2_CLUSTER_NORMAL: {
281
+ case QCOW2_SUBCLUSTER_ZERO_ALLOC:
282
+ case QCOW2_SUBCLUSTER_NORMAL:
283
+ case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: {
284
uint64_t host_cluster_offset = l2_entry & L2E_OFFSET_MASK;
285
*host_offset = host_cluster_offset + offset_in_cluster;
286
- /* how many allocated clusters ? */
287
- c = count_contiguous_clusters(bs, nb_clusters, s->cluster_size,
288
- l2_slice, l2_index, QCOW_OFLAG_ZERO);
289
if (offset_into_cluster(s, host_cluster_offset)) {
290
qcow2_signal_corruption(bs, true, -1, -1,
291
"Cluster allocation offset %#"
292
@@ -XXX,XX +XXX,XX @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
293
abort();
294
}
295
296
+ sc = count_contiguous_subclusters(bs, nb_clusters, sc_index,
297
+ l2_slice, &l2_index);
298
+ if (sc < 0) {
299
+ qcow2_signal_corruption(bs, true, -1, -1, "Invalid cluster entry found "
300
+ " (L2 offset: %#" PRIx64 ", L2 index: %#x)",
301
+ l2_offset, l2_index);
302
+ ret = -EIO;
303
+ goto fail;
304
+ }
305
qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice);
306
307
- bytes_available = (int64_t)c * s->cluster_size;
308
+ bytes_available = ((int64_t)sc + sc_index) << s->subcluster_bits;
309
310
out:
311
if (bytes_available > bytes_needed) {
312
@@ -XXX,XX +XXX,XX @@ out:
313
assert(bytes_available - offset_in_cluster <= UINT_MAX);
314
*bytes = bytes_available - offset_in_cluster;
315
316
- *subcluster_type = qcow2_cluster_to_subcluster_type(type);
317
+ *subcluster_type = type;
318
319
return 0;
320
321
--
72
--
322
2.26.2
73
2.34.1
323
74
324
75
diff view generated by jsdifflib
[PULL 32/34] qcow2: Allow preallocation and backing files if extended_l2 is set
[PULL 02/23] block/curl.c: Set error message string if curl_init_state() fails
1
From: Alberto Garcia <berto@igalia.com>
1
From: Peter Maydell <peter.maydell@linaro.org>
2
2
3
Traditional qcow2 images don't allow preallocation if a backing file
3
In curl_open(), the 'out' label assumes that the state->errmsg string
4
is set. This is because once a cluster is allocated there is no way to
4
has been set (either by curl_easy_perform() or by manually copying a
5
tell that its data should be read from the backing file.
5
string into it); however if curl_init_state() fails we will jump to
6
that label without setting the string. Add the missing error string
7
setup.
6
8
7
Extended L2 entries have individual allocation bits for each
9
(We can't be specific about the cause of failure: the documentation
8
subcluster, and therefore it is perfectly possible to have an
10
of curl_easy_init() just says "If this function returns NULL,
9
allocated cluster with all its subclusters unallocated.
11
something went wrong".)
10
12
11
Signed-off-by: Alberto Garcia <berto@igalia.com>
13
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
12
Reviewed-by: Eric Blake <eblake@redhat.com>
14
Message-Id: <20220222152341.850419-2-peter.maydell@linaro.org>
13
Reviewed-by: Max Reitz <mreitz@redhat.com>
15
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
14
Message-Id: <6d5b0f38e7dc5f2f31d8cab1cb92044e9909aece.1594396418.git.berto@igalia.com>
16
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
15
Signed-off-by: Max Reitz <mreitz@redhat.com>
17
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
16
---
18
---
17
block/qcow2.c | 7 ++++---
19
block/curl.c | 2 ++
18
tests/qemu-iotests/206.out | 2 +-
20
1 file changed, 2 insertions(+)
19
2 files changed, 5 insertions(+), 4 deletions(-)
20
21
21
diff --git a/block/qcow2.c b/block/qcow2.c
22
diff --git a/block/curl.c b/block/curl.c
22
index XXXXXXX..XXXXXXX 100644
23
index XXXXXXX..XXXXXXX 100644
23
--- a/block/qcow2.c
24
--- a/block/curl.c
24
+++ b/block/qcow2.c
25
+++ b/block/curl.c
25
@@ -XXX,XX +XXX,XX @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp)
26
@@ -XXX,XX +XXX,XX @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
26
qcow2_opts->preallocation = PREALLOC_MODE_OFF;
27
// Get file size
27
}
28
28
if (qcow2_opts->has_backing_file &&
29
if (curl_init_state(s, state) < 0) {
29
- qcow2_opts->preallocation != PREALLOC_MODE_OFF)
30
+ pstrcpy(state->errmsg, CURL_ERROR_SIZE,
30
+ qcow2_opts->preallocation != PREALLOC_MODE_OFF &&
31
+ "curl library initialization failed.");
31
+ !qcow2_opts->extended_l2)
32
{
33
- error_setg(errp, "Backing file and preallocation cannot be used at "
34
- "the same time");
35
+ error_setg(errp, "Backing file and preallocation can only be used at "
36
+ "the same time if extended_l2 is on");
37
ret = -EINVAL;
38
goto out;
32
goto out;
39
}
33
}
40
diff --git a/tests/qemu-iotests/206.out b/tests/qemu-iotests/206.out
41
index XXXXXXX..XXXXXXX 100644
42
--- a/tests/qemu-iotests/206.out
43
+++ b/tests/qemu-iotests/206.out
44
@@ -XXX,XX +XXX,XX @@ Job failed: Different refcount widths than 16 bits require compatibility level 1
45
=== Invalid backing file options ===
46
{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": {"backing-file": "/dev/null", "driver": "qcow2", "file": "node0", "preallocation": "full", "size": 67108864}}}
47
{"return": {}}
48
-Job failed: Backing file and preallocation cannot be used at the same time
49
+Job failed: Backing file and preallocation can only be used at the same time if extended_l2 is on
50
{"execute": "job-dismiss", "arguments": {"id": "job0"}}
51
{"return": {}}
52
34
53
--
35
--
54
2.26.2
36
2.34.1
55
37
56
38
diff view generated by jsdifflib
[PULL 20/34] qcow2: Add subcluster support to calculate_l2_meta()
[PULL 03/23] block/curl.c: Check error return from curl_easy_setopt()
1
From: Alberto Garcia <berto@igalia.com>
1
From: Peter Maydell <peter.maydell@linaro.org>
2
2
3
If an image has subclusters then there are more copy-on-write
3
Coverity points out that we aren't checking the return value
4
scenarios that we need to consider. Let's say we have a write request
4
from curl_easy_setopt() for any of the calls to it we make
5
from the middle of subcluster #3 until the end of the cluster:
5
in block/curl.c.
6
6
7
1) If we are writing to a newly allocated cluster then we need
7
Some of these options are documented as always succeeding (e.g.
8
copy-on-write. The previous contents of subclusters #0 to #3 must
8
CURLOPT_VERBOSE) but others have documented failure cases (e.g.
9
be copied to the new cluster. We can optimize this process by
9
CURLOPT_URL). For consistency we check every call, even the ones
10
skipping all leading unallocated or zero subclusters (the status of
10
that theoretically cannot fail.
11
those skipped subclusters will be reflected in the new L2 bitmap).
12
11
13
2) If we are overwriting an existing cluster:
12
Fixes: Coverity CID 1459336, 1459482, 1460331
13
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
14
Message-Id: <20220222152341.850419-3-peter.maydell@linaro.org>
15
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
16
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
17
---
18
block/curl.c | 90 +++++++++++++++++++++++++++++++++-------------------
19
1 file changed, 57 insertions(+), 33 deletions(-)
14
20
15
2.1) If subcluster #3 is unallocated or has the all-zeroes bit set
21
diff --git a/block/curl.c b/block/curl.c
16
then we need copy-on-write (on subcluster #3 only).
17
18
2.2) If subcluster #3 was already allocated then there is no need
19
for any copy-on-write. However we still need to update the L2
20
bitmap to reflect possible changes in the allocation status of
21
subclusters #4 to #31. Because of this, this function checks
22
if all the overwritten subclusters are already allocated and
23
in this case it returns without creating a new QCowL2Meta
24
structure.
25
26
After all these changes l2meta_cow_start() and l2meta_cow_end()
27
are not necessarily cluster-aligned anymore. We need to update the
28
calculation of old_start and old_end in handle_dependencies() to
29
guarantee that no two requests try to write on the same cluster.
30
31
Signed-off-by: Alberto Garcia <berto@igalia.com>
32
Reviewed-by: Eric Blake <eblake@redhat.com>
33
Reviewed-by: Max Reitz <mreitz@redhat.com>
34
Message-Id: <4292dd56e4446d386a2fe307311737a711c00708.1594396418.git.berto@igalia.com>
35
Signed-off-by: Max Reitz <mreitz@redhat.com>
36
---
37
block/qcow2-cluster.c | 167 +++++++++++++++++++++++++++++++++---------
38
1 file changed, 133 insertions(+), 34 deletions(-)
39
40
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
41
index XXXXXXX..XXXXXXX 100644
22
index XXXXXXX..XXXXXXX 100644
42
--- a/block/qcow2-cluster.c
23
--- a/block/curl.c
43
+++ b/block/qcow2-cluster.c
24
+++ b/block/curl.c
44
@@ -XXX,XX +XXX,XX @@ fail:
25
@@ -XXX,XX +XXX,XX @@ static int curl_init_state(BDRVCURLState *s, CURLState *state)
45
* If the L2 entry is invalid return -errno and set @type to
26
if (!state->curl) {
46
* QCOW2_SUBCLUSTER_INVALID.
27
return -EIO;
47
*/
48
-G_GNUC_UNUSED
49
static int qcow2_get_subcluster_range_type(BlockDriverState *bs,
50
uint64_t l2_entry,
51
uint64_t l2_bitmap,
52
@@ -XXX,XX +XXX,XX @@ void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m)
53
* If @keep_old is true it means that the clusters were already
54
* allocated and will be overwritten. If false then the clusters are
55
* new and we have to decrease the reference count of the old ones.
56
+ *
57
+ * Returns 0 on success, -errno on failure.
58
*/
59
-static void calculate_l2_meta(BlockDriverState *bs,
60
- uint64_t host_cluster_offset,
61
- uint64_t guest_offset, unsigned bytes,
62
- uint64_t *l2_slice, QCowL2Meta **m, bool keep_old)
63
+static int calculate_l2_meta(BlockDriverState *bs, uint64_t host_cluster_offset,
64
+ uint64_t guest_offset, unsigned bytes,
65
+ uint64_t *l2_slice, QCowL2Meta **m, bool keep_old)
66
{
67
BDRVQcow2State *s = bs->opaque;
68
- int l2_index = offset_to_l2_slice_index(s, guest_offset);
69
- uint64_t l2_entry;
70
+ int sc_index, l2_index = offset_to_l2_slice_index(s, guest_offset);
71
+ uint64_t l2_entry, l2_bitmap;
72
unsigned cow_start_from, cow_end_to;
73
unsigned cow_start_to = offset_into_cluster(s, guest_offset);
74
unsigned cow_end_from = cow_start_to + bytes;
75
unsigned nb_clusters = size_to_clusters(s, cow_end_from);
76
QCowL2Meta *old_m = *m;
77
- QCow2ClusterType type;
78
+ QCow2SubclusterType type;
79
+ int i;
80
+ bool skip_cow = keep_old;
81
82
assert(nb_clusters <= s->l2_slice_size - l2_index);
83
84
- /* Return if there's no COW (all clusters are normal and we keep them) */
85
- if (keep_old) {
86
- int i;
87
- for (i = 0; i < nb_clusters; i++) {
88
- l2_entry = get_l2_entry(s, l2_slice, l2_index + i);
89
- if (qcow2_get_cluster_type(bs, l2_entry) != QCOW2_CLUSTER_NORMAL) {
90
- break;
91
+ /* Check the type of all affected subclusters */
92
+ for (i = 0; i < nb_clusters; i++) {
93
+ l2_entry = get_l2_entry(s, l2_slice, l2_index + i);
94
+ l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index + i);
95
+ if (skip_cow) {
96
+ unsigned write_from = MAX(cow_start_to, i << s->cluster_bits);
97
+ unsigned write_to = MIN(cow_end_from, (i + 1) << s->cluster_bits);
98
+ int first_sc = offset_to_sc_index(s, write_from);
99
+ int last_sc = offset_to_sc_index(s, write_to - 1);
100
+ int cnt = qcow2_get_subcluster_range_type(bs, l2_entry, l2_bitmap,
101
+ first_sc, &type);
102
+ /* Is any of the subclusters of type != QCOW2_SUBCLUSTER_NORMAL ? */
103
+ if (type != QCOW2_SUBCLUSTER_NORMAL || first_sc + cnt <= last_sc) {
104
+ skip_cow = false;
105
}
106
+ } else {
107
+ /* If we can't skip the cow we can still look for invalid entries */
108
+ type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, 0);
109
}
28
}
110
- if (i == nb_clusters) {
29
- curl_easy_setopt(state->curl, CURLOPT_URL, s->url);
111
- return;
30
- curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYPEER,
112
+ if (type == QCOW2_SUBCLUSTER_INVALID) {
31
- (long) s->sslverify);
113
+ int l1_index = offset_to_l1_index(s, guest_offset);
32
- curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYHOST,
114
+ uint64_t l2_offset = s->l1_table[l1_index] & L1E_OFFSET_MASK;
33
- s->sslverify ? 2L : 0L);
115
+ qcow2_signal_corruption(bs, true, -1, -1, "Invalid cluster "
34
+ if (curl_easy_setopt(state->curl, CURLOPT_URL, s->url) ||
116
+ "entry found (L2 offset: %#" PRIx64
35
+ curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYPEER,
117
+ ", L2 index: %#x)",
36
+ (long) s->sslverify) ||
118
+ l2_offset, l2_index + i);
37
+ curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYHOST,
119
+ return -EIO;
38
+ s->sslverify ? 2L : 0L)) {
39
+ goto err;
40
+ }
41
if (s->cookie) {
42
- curl_easy_setopt(state->curl, CURLOPT_COOKIE, s->cookie);
43
+ if (curl_easy_setopt(state->curl, CURLOPT_COOKIE, s->cookie)) {
44
+ goto err;
45
+ }
46
+ }
47
+ if (curl_easy_setopt(state->curl, CURLOPT_TIMEOUT, (long)s->timeout) ||
48
+ curl_easy_setopt(state->curl, CURLOPT_WRITEFUNCTION,
49
+ (void *)curl_read_cb) ||
50
+ curl_easy_setopt(state->curl, CURLOPT_WRITEDATA, (void *)state) ||
51
+ curl_easy_setopt(state->curl, CURLOPT_PRIVATE, (void *)state) ||
52
+ curl_easy_setopt(state->curl, CURLOPT_AUTOREFERER, 1) ||
53
+ curl_easy_setopt(state->curl, CURLOPT_FOLLOWLOCATION, 1) ||
54
+ curl_easy_setopt(state->curl, CURLOPT_NOSIGNAL, 1) ||
55
+ curl_easy_setopt(state->curl, CURLOPT_ERRORBUFFER, state->errmsg) ||
56
+ curl_easy_setopt(state->curl, CURLOPT_FAILONERROR, 1)) {
57
+ goto err;
120
}
58
}
59
- curl_easy_setopt(state->curl, CURLOPT_TIMEOUT, (long)s->timeout);
60
- curl_easy_setopt(state->curl, CURLOPT_WRITEFUNCTION,
61
- (void *)curl_read_cb);
62
- curl_easy_setopt(state->curl, CURLOPT_WRITEDATA, (void *)state);
63
- curl_easy_setopt(state->curl, CURLOPT_PRIVATE, (void *)state);
64
- curl_easy_setopt(state->curl, CURLOPT_AUTOREFERER, 1);
65
- curl_easy_setopt(state->curl, CURLOPT_FOLLOWLOCATION, 1);
66
- curl_easy_setopt(state->curl, CURLOPT_NOSIGNAL, 1);
67
- curl_easy_setopt(state->curl, CURLOPT_ERRORBUFFER, state->errmsg);
68
- curl_easy_setopt(state->curl, CURLOPT_FAILONERROR, 1);
69
-
70
if (s->username) {
71
- curl_easy_setopt(state->curl, CURLOPT_USERNAME, s->username);
72
+ if (curl_easy_setopt(state->curl, CURLOPT_USERNAME, s->username)) {
73
+ goto err;
74
+ }
75
}
76
if (s->password) {
77
- curl_easy_setopt(state->curl, CURLOPT_PASSWORD, s->password);
78
+ if (curl_easy_setopt(state->curl, CURLOPT_PASSWORD, s->password)) {
79
+ goto err;
80
+ }
81
}
82
if (s->proxyusername) {
83
- curl_easy_setopt(state->curl,
84
- CURLOPT_PROXYUSERNAME, s->proxyusername);
85
+ if (curl_easy_setopt(state->curl,
86
+ CURLOPT_PROXYUSERNAME, s->proxyusername)) {
87
+ goto err;
88
+ }
89
}
90
if (s->proxypassword) {
91
- curl_easy_setopt(state->curl,
92
- CURLOPT_PROXYPASSWORD, s->proxypassword);
93
+ if (curl_easy_setopt(state->curl,
94
+ CURLOPT_PROXYPASSWORD, s->proxypassword)) {
95
+ goto err;
96
+ }
97
}
98
99
/* Restrict supported protocols to avoid security issues in the more
100
@@ -XXX,XX +XXX,XX @@ static int curl_init_state(BDRVCURLState *s, CURLState *state)
101
* Restricting protocols is only supported from 7.19.4 upwards.
102
*/
103
#if LIBCURL_VERSION_NUM >= 0x071304
104
- curl_easy_setopt(state->curl, CURLOPT_PROTOCOLS, PROTOCOLS);
105
- curl_easy_setopt(state->curl, CURLOPT_REDIR_PROTOCOLS, PROTOCOLS);
106
+ if (curl_easy_setopt(state->curl, CURLOPT_PROTOCOLS, PROTOCOLS) ||
107
+ curl_easy_setopt(state->curl, CURLOPT_REDIR_PROTOCOLS, PROTOCOLS)) {
108
+ goto err;
109
+ }
110
#endif
111
112
#ifdef DEBUG_VERBOSE
113
- curl_easy_setopt(state->curl, CURLOPT_VERBOSE, 1);
114
+ if (curl_easy_setopt(state->curl, CURLOPT_VERBOSE, 1)) {
115
+ goto err;
116
+ }
117
#endif
121
}
118
}
122
119
123
+ if (skip_cow) {
120
state->s = s;
124
+ return 0;
121
125
+ }
122
return 0;
126
+
123
+
127
/* Get the L2 entry of the first cluster */
124
+err:
128
l2_entry = get_l2_entry(s, l2_slice, l2_index);
125
+ curl_easy_cleanup(state->curl);
129
- type = qcow2_get_cluster_type(bs, l2_entry);
126
+ state->curl = NULL;
130
-
127
+ return -EIO;
131
- if (type == QCOW2_CLUSTER_NORMAL && keep_old) {
128
}
132
- cow_start_from = cow_start_to;
129
133
+ l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index);
130
/* Called with s->mutex held. */
134
+ sc_index = offset_to_sc_index(s, guest_offset);
131
@@ -XXX,XX +XXX,XX @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
135
+ type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, sc_index);
136
+
137
+ if (!keep_old) {
138
+ switch (type) {
139
+ case QCOW2_SUBCLUSTER_COMPRESSED:
140
+ cow_start_from = 0;
141
+ break;
142
+ case QCOW2_SUBCLUSTER_NORMAL:
143
+ case QCOW2_SUBCLUSTER_ZERO_ALLOC:
144
+ case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC:
145
+ if (has_subclusters(s)) {
146
+ /* Skip all leading zero and unallocated subclusters */
147
+ uint32_t alloc_bitmap = l2_bitmap & QCOW_L2_BITMAP_ALL_ALLOC;
148
+ cow_start_from =
149
+ MIN(sc_index, ctz32(alloc_bitmap)) << s->subcluster_bits;
150
+ } else {
151
+ cow_start_from = 0;
152
+ }
153
+ break;
154
+ case QCOW2_SUBCLUSTER_ZERO_PLAIN:
155
+ case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN:
156
+ cow_start_from = sc_index << s->subcluster_bits;
157
+ break;
158
+ default:
159
+ g_assert_not_reached();
160
+ }
161
} else {
162
- cow_start_from = 0;
163
+ switch (type) {
164
+ case QCOW2_SUBCLUSTER_NORMAL:
165
+ cow_start_from = cow_start_to;
166
+ break;
167
+ case QCOW2_SUBCLUSTER_ZERO_ALLOC:
168
+ case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC:
169
+ cow_start_from = sc_index << s->subcluster_bits;
170
+ break;
171
+ default:
172
+ g_assert_not_reached();
173
+ }
174
}
132
}
175
133
176
/* Get the L2 entry of the last cluster */
134
s->accept_range = false;
177
- l2_entry = get_l2_entry(s, l2_slice, l2_index + nb_clusters - 1);
135
- curl_easy_setopt(state->curl, CURLOPT_NOBODY, 1);
178
- type = qcow2_get_cluster_type(bs, l2_entry);
136
- curl_easy_setopt(state->curl, CURLOPT_HEADERFUNCTION,
179
-
137
- curl_header_cb);
180
- if (type == QCOW2_CLUSTER_NORMAL && keep_old) {
138
- curl_easy_setopt(state->curl, CURLOPT_HEADERDATA, s);
181
- cow_end_to = cow_end_from;
139
+ if (curl_easy_setopt(state->curl, CURLOPT_NOBODY, 1) ||
182
+ l2_index += nb_clusters - 1;
140
+ curl_easy_setopt(state->curl, CURLOPT_HEADERFUNCTION, curl_header_cb) ||
183
+ l2_entry = get_l2_entry(s, l2_slice, l2_index);
141
+ curl_easy_setopt(state->curl, CURLOPT_HEADERDATA, s)) {
184
+ l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index);
142
+ pstrcpy(state->errmsg, CURL_ERROR_SIZE,
185
+ sc_index = offset_to_sc_index(s, guest_offset + bytes - 1);
143
+ "curl library initialization failed.");
186
+ type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, sc_index);
187
+
188
+ if (!keep_old) {
189
+ switch (type) {
190
+ case QCOW2_SUBCLUSTER_COMPRESSED:
191
+ cow_end_to = ROUND_UP(cow_end_from, s->cluster_size);
192
+ break;
193
+ case QCOW2_SUBCLUSTER_NORMAL:
194
+ case QCOW2_SUBCLUSTER_ZERO_ALLOC:
195
+ case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC:
196
+ cow_end_to = ROUND_UP(cow_end_from, s->cluster_size);
197
+ if (has_subclusters(s)) {
198
+ /* Skip all trailing zero and unallocated subclusters */
199
+ uint32_t alloc_bitmap = l2_bitmap & QCOW_L2_BITMAP_ALL_ALLOC;
200
+ cow_end_to -=
201
+ MIN(s->subclusters_per_cluster - sc_index - 1,
202
+ clz32(alloc_bitmap)) << s->subcluster_bits;
203
+ }
204
+ break;
205
+ case QCOW2_SUBCLUSTER_ZERO_PLAIN:
206
+ case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN:
207
+ cow_end_to = ROUND_UP(cow_end_from, s->subcluster_size);
208
+ break;
209
+ default:
210
+ g_assert_not_reached();
211
+ }
212
} else {
213
- cow_end_to = ROUND_UP(cow_end_from, s->cluster_size);
214
+ switch (type) {
215
+ case QCOW2_SUBCLUSTER_NORMAL:
216
+ cow_end_to = cow_end_from;
217
+ break;
218
+ case QCOW2_SUBCLUSTER_ZERO_ALLOC:
219
+ case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC:
220
+ cow_end_to = ROUND_UP(cow_end_from, s->subcluster_size);
221
+ break;
222
+ default:
223
+ g_assert_not_reached();
224
+ }
225
}
226
227
*m = g_malloc0(sizeof(**m));
228
@@ -XXX,XX +XXX,XX @@ static void calculate_l2_meta(BlockDriverState *bs,
229
230
qemu_co_queue_init(&(*m)->dependent_requests);
231
QLIST_INSERT_HEAD(&s->cluster_allocs, *m, next_in_flight);
232
+
233
+ return 0;
234
}
235
236
/*
237
@@ -XXX,XX +XXX,XX @@ static int handle_dependencies(BlockDriverState *bs, uint64_t guest_offset,
238
239
uint64_t start = guest_offset;
240
uint64_t end = start + bytes;
241
- uint64_t old_start = l2meta_cow_start(old_alloc);
242
- uint64_t old_end = l2meta_cow_end(old_alloc);
243
+ uint64_t old_start = start_of_cluster(s, l2meta_cow_start(old_alloc));
244
+ uint64_t old_end = ROUND_UP(l2meta_cow_end(old_alloc), s->cluster_size);
245
246
if (end <= old_start || start >= old_end) {
247
/* No intersection */
248
@@ -XXX,XX +XXX,XX @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset,
249
- offset_into_cluster(s, guest_offset));
250
assert(*bytes != 0);
251
252
- calculate_l2_meta(bs, cluster_offset, guest_offset,
253
- *bytes, l2_slice, m, true);
254
+ ret = calculate_l2_meta(bs, cluster_offset, guest_offset,
255
+ *bytes, l2_slice, m, true);
256
+ if (ret < 0) {
257
+ goto out;
258
+ }
259
260
ret = 1;
261
} else {
262
@@ -XXX,XX +XXX,XX @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,
263
*bytes = MIN(*bytes, nb_bytes - offset_into_cluster(s, guest_offset));
264
assert(*bytes != 0);
265
266
- calculate_l2_meta(bs, alloc_cluster_offset, guest_offset, *bytes, l2_slice,
267
- m, false);
268
+ ret = calculate_l2_meta(bs, alloc_cluster_offset, guest_offset, *bytes,
269
+ l2_slice, m, false);
270
+ if (ret < 0) {
271
+ goto out;
144
+ goto out;
272
+ }
145
+ }
273
146
if (curl_easy_perform(state->curl))
274
ret = 1;
147
goto out;
148
if (curl_easy_getinfo(state->curl, CURLINFO_CONTENT_LENGTH_DOWNLOAD, &d)) {
149
@@ -XXX,XX +XXX,XX @@ static void curl_setup_preadv(BlockDriverState *bs, CURLAIOCB *acb)
150
151
snprintf(state->range, 127, "%" PRIu64 "-%" PRIu64, start, end);
152
trace_curl_setup_preadv(acb->bytes, start, state->range);
153
- curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range);
154
-
155
- if (curl_multi_add_handle(s->multi, state->curl) != CURLM_OK) {
156
+ if (curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range) ||
157
+ curl_multi_add_handle(s->multi, state->curl) != CURLM_OK) {
158
state->acb[0] = NULL;
159
acb->ret = -EIO;
275
160
276
--
161
--
277
2.26.2
162
2.34.1
278
279
diff view generated by jsdifflib
[PULL 34/34] iotests: Add tests for qcow2 images with extended L2 entries
[PULL 04/23] block: fix preallocate filter: don't do unaligned preallocate requests
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
Signed-off-by: Alberto Garcia <berto@igalia.com>
3
There is a bug in handling BDRV_REQ_NO_WAIT flag: we still may wait in
4
Reviewed-by: Max Reitz <mreitz@redhat.com>
4
wait_serialising_requests() if request is unaligned. And this is
5
Message-Id: <e6dd0429cafe84ca603179c298a8703bddca2904.1594396418.git.berto@igalia.com>
5
possible for the only user of this flag (preallocate filter) if
6
Signed-off-by: Max Reitz <mreitz@redhat.com>
6
underlying file is unaligned to its request_alignment on start.
7
8
So, we have to fix preallocate filter to do only aligned preallocate
9
requests.
10
11
Next, we should fix generic block/io.c somehow. Keeping in mind that
12
preallocate is the only user of BDRV_REQ_NO_WAIT and that we have to
13
fix its behavior now, it seems more safe to just assert that we never
14
use BDRV_REQ_NO_WAIT with unaligned requests and add corresponding
15
comment. Let's do so.
16
17
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
18
Reviewed-by: Denis V. Lunev <den@openvz.org>
19
Message-Id: <20220215121609.38570-1-vsementsov@virtuozzo.com>
20
[hreitz: Rebased on block GS/IO split]
21
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
7
---
22
---
8
tests/qemu-iotests/271 | 901 +++++++++++++++++++++++++++++++++++++
23
include/block/block-common.h | 3 ++-
9
tests/qemu-iotests/271.out | 726 ++++++++++++++++++++++++++++++
24
block/io.c | 4 ++++
10
tests/qemu-iotests/group | 1 +
25
block/preallocate.c | 15 ++++++++++++---
11
3 files changed, 1628 insertions(+)
26
3 files changed, 18 insertions(+), 4 deletions(-)
12
create mode 100755 tests/qemu-iotests/271
13
create mode 100644 tests/qemu-iotests/271.out
14
27
15
diff --git a/tests/qemu-iotests/271 b/tests/qemu-iotests/271
28
diff --git a/include/block/block-common.h b/include/block/block-common.h
16
new file mode 100755
29
index XXXXXXX..XXXXXXX 100644
17
index XXXXXXX..XXXXXXX
30
--- a/include/block/block-common.h
18
--- /dev/null
31
+++ b/include/block/block-common.h
19
+++ b/tests/qemu-iotests/271
32
@@ -XXX,XX +XXX,XX @@ typedef enum {
20
@@ -XXX,XX +XXX,XX @@
33
21
+#!/bin/bash
34
/*
22
+#
35
* If we need to wait for other requests, just fail immediately. Used
23
+# Test qcow2 images with extended L2 entries
36
- * only together with BDRV_REQ_SERIALISING.
24
+#
37
+ * only together with BDRV_REQ_SERIALISING. Used only with requests aligned
25
+# Copyright (C) 2019-2020 Igalia, S.L.
38
+ * to request_alignment (corresponding assertions are in block/io.c).
26
+# Author: Alberto Garcia <berto@igalia.com>
39
*/
27
+#
40
BDRV_REQ_NO_WAIT = 0x400,
28
+# This program is free software; you can redistribute it and/or modify
41
29
+# it under the terms of the GNU General Public License as published by
42
diff --git a/block/io.c b/block/io.c
30
+# the Free Software Foundation; either version 2 of the License, or
43
index XXXXXXX..XXXXXXX 100644
31
+# (at your option) any later version.
44
--- a/block/io.c
32
+#
45
+++ b/block/io.c
33
+# This program is distributed in the hope that it will be useful,
46
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_co_do_zero_pwritev(BdrvChild *child,
34
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
47
35
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
48
padding = bdrv_init_padding(bs, offset, bytes, &pad);
36
+# GNU General Public License for more details.
49
if (padding) {
37
+#
50
+ assert(!(flags & BDRV_REQ_NO_WAIT));
38
+# You should have received a copy of the GNU General Public License
51
bdrv_make_request_serialising(req, align);
39
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
52
40
+#
53
bdrv_padding_rmw_read(child, req, &pad, true);
54
@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_pwritev_part(BdrvChild *child,
55
* serialize the request to prevent interactions of the
56
* widened region with other transactions.
57
*/
58
+ assert(!(flags & BDRV_REQ_NO_WAIT));
59
bdrv_make_request_serialising(&req, align);
60
bdrv_padding_rmw_read(child, &req, &pad, false);
61
}
62
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_co_copy_range_internal(
63
/* TODO We can support BDRV_REQ_NO_FALLBACK here */
64
assert(!(read_flags & BDRV_REQ_NO_FALLBACK));
65
assert(!(write_flags & BDRV_REQ_NO_FALLBACK));
66
+ assert(!(read_flags & BDRV_REQ_NO_WAIT));
67
+ assert(!(write_flags & BDRV_REQ_NO_WAIT));
68
69
if (!dst || !dst->bs || !bdrv_is_inserted(dst->bs)) {
70
return -ENOMEDIUM;
71
diff --git a/block/preallocate.c b/block/preallocate.c
72
index XXXXXXX..XXXXXXX 100644
73
--- a/block/preallocate.c
74
+++ b/block/preallocate.c
75
@@ -XXX,XX +XXX,XX @@ static bool coroutine_fn handle_write(BlockDriverState *bs, int64_t offset,
76
int64_t end = offset + bytes;
77
int64_t prealloc_start, prealloc_end;
78
int ret;
79
+ uint32_t file_align = bs->file->bs->bl.request_alignment;
80
+ uint32_t prealloc_align = MAX(s->opts.prealloc_align, file_align);
41
+
81
+
42
+# creator
82
+ assert(QEMU_IS_ALIGNED(prealloc_align, file_align));
43
+owner=berto@igalia.com
83
84
if (!has_prealloc_perms(bs)) {
85
/* We don't have state neither should try to recover it */
86
@@ -XXX,XX +XXX,XX @@ static bool coroutine_fn handle_write(BlockDriverState *bs, int64_t offset,
87
88
/* Now we want new preallocation, as request writes beyond s->file_end. */
89
90
- prealloc_start = want_merge_zero ? MIN(offset, s->file_end) : s->file_end;
91
- prealloc_end = QEMU_ALIGN_UP(end + s->opts.prealloc_size,
92
- s->opts.prealloc_align);
93
+ prealloc_start = QEMU_ALIGN_UP(
94
+ want_merge_zero ? MIN(offset, s->file_end) : s->file_end,
95
+ file_align);
96
+ prealloc_end = QEMU_ALIGN_UP(
97
+ MAX(prealloc_start, end) + s->opts.prealloc_size,
98
+ prealloc_align);
44
+
99
+
45
+seq="$(basename $0)"
100
+ want_merge_zero = want_merge_zero && (prealloc_start <= offset);
46
+echo "QA output created by $seq"
101
47
+
102
ret = bdrv_co_pwrite_zeroes(
48
+here="$PWD"
103
bs->file, prealloc_start, prealloc_end - prealloc_start,
49
+status=1    # failure is the default!
50
+
51
+_cleanup()
52
+{
53
+ _cleanup_test_img
54
+ rm -f "$TEST_IMG.raw"
55
+}
56
+trap "_cleanup; exit \$status" 0 1 2 3 15
57
+
58
+# get standard environment, filters and checks
59
+. ./common.rc
60
+. ./common.filter
61
+
62
+_supported_fmt qcow2
63
+_supported_proto file nfs
64
+_supported_os Linux
65
+_unsupported_imgopts extended_l2 compat=0.10 cluster_size data_file refcount_bits=1[^0-9]
66
+
67
+l2_offset=$((0x40000))
68
+
69
+_verify_img()
70
+{
71
+ $QEMU_IMG compare "$TEST_IMG" "$TEST_IMG.raw" | grep -v 'Images are identical'
72
+ $QEMU_IMG check "$TEST_IMG" | _filter_qemu_img_check | \
73
+ grep -v 'No errors were found on the image'
74
+}
75
+
76
+# Compare the bitmap of an extended L2 entry against an expected value
77
+_verify_l2_bitmap()
78
+{
79
+ entry_no="$1" # L2 entry number, starting from 0
80
+ expected_alloc="$alloc" # Space-separated list of allocated subcluster indexes
81
+ expected_zero="$zero" # Space-separated list of zero subcluster indexes
82
+
83
+ offset=$(($l2_offset + $entry_no * 16))
84
+ entry=$(peek_file_be "$TEST_IMG" $offset 8)
85
+ offset=$(($offset + 8))
86
+ bitmap=$(peek_file_be "$TEST_IMG" $offset 8)
87
+
88
+ expected_bitmap=0
89
+ for bit in $expected_alloc; do
90
+ expected_bitmap=$(($expected_bitmap | (1 << $bit)))
91
+ done
92
+ for bit in $expected_zero; do
93
+ expected_bitmap=$(($expected_bitmap | (1 << (32 + $bit))))
94
+ done
95
+ printf -v expected_bitmap "%u" $expected_bitmap # Convert to unsigned
96
+
97
+ printf "L2 entry #%d: 0x%016x %016x\n" "$entry_no" "$entry" "$bitmap"
98
+ if [ "$bitmap" != "$expected_bitmap" ]; then
99
+ printf "ERROR: expecting bitmap 0x%016x\n" "$expected_bitmap"
100
+ fi
101
+}
102
+
103
+# This should be called as _run_test c=XXX sc=XXX off=XXX len=XXX cmd=XXX
104
+# c: cluster number (0 if unset)
105
+# sc: subcluster number inside cluster @c (0 if unset)
106
+# off: offset inside subcluster @sc, in kilobytes (0 if unset)
107
+# len: request length, passed directly to qemu-io (e.g: 256, 4k, 1M, ...)
108
+# cmd: the command to pass to qemu-io, must be one of
109
+# write -> write
110
+# zero -> write -z
111
+# unmap -> write -z -u
112
+# compress -> write -c
113
+# discard -> discard
114
+_run_test()
115
+{
116
+ unset c sc off len cmd
117
+ for var in "$@"; do eval "$var"; done
118
+ case "${cmd:-write}" in
119
+ zero)
120
+ cmd="write -q -z";;
121
+ unmap)
122
+ cmd="write -q -z -u";;
123
+ compress)
124
+ pat=$((${pat:-0} + 1))
125
+ cmd="write -q -c -P ${pat}";;
126
+ write)
127
+ pat=$((${pat:-0} + 1))
128
+ cmd="write -q -P ${pat}";;
129
+ discard)
130
+ cmd="discard -q";;
131
+ *)
132
+ echo "Unknown option $cmd"
133
+ exit 1;;
134
+ esac
135
+ c="${c:-0}"
136
+ sc="${sc:-0}"
137
+ off="${off:-0}"
138
+ offset="$(($c * 64 + $sc * 2 + $off))"
139
+ [ "$offset" != 0 ] && offset="${offset}k"
140
+ cmd="$cmd ${offset} ${len}"
141
+ raw_cmd=$(echo $cmd | sed s/-c//) # Raw images don't support -c
142
+ echo $cmd | sed 's/-P [0-9][0-9]\?/-P PATTERN/'
143
+ $QEMU_IO -c "$cmd" "$TEST_IMG" | _filter_qemu_io
144
+ $QEMU_IO -c "$raw_cmd" -f raw "$TEST_IMG.raw" | _filter_qemu_io
145
+ _verify_img
146
+ _verify_l2_bitmap "$c"
147
+}
148
+
149
+_reset_img()
150
+{
151
+ size="$1"
152
+ $QEMU_IMG create -f raw "$TEST_IMG.raw" "$size" | _filter_img_create
153
+ if [ "$use_backing_file" = "yes" ]; then
154
+ $QEMU_IMG create -f raw "$TEST_IMG.base" "$size" | _filter_img_create
155
+ $QEMU_IO -c "write -q -P 0xFF 0 $size" -f raw "$TEST_IMG.base" | _filter_qemu_io
156
+ $QEMU_IO -c "write -q -P 0xFF 0 $size" -f raw "$TEST_IMG.raw" | _filter_qemu_io
157
+ _make_test_img -o extended_l2=on -F raw -b "$TEST_IMG.base" "$size"
158
+ else
159
+ _make_test_img -o extended_l2=on "$size"
160
+ fi
161
+}
162
+
163
+############################################################
164
+############################################################
165
+############################################################
166
+
167
+# Test that writing to an image with subclusters produces the expected
168
+# results, in images with and without backing files
169
+for use_backing_file in yes no; do
170
+ echo
171
+ echo "### Standard write tests (backing file: $use_backing_file) ###"
172
+ echo
173
+ _reset_img 1M
174
+ ### Write subcluster #0 (beginning of subcluster) ###
175
+ alloc="0"; zero=""
176
+ _run_test sc=0 len=1k
177
+
178
+ ### Write subcluster #1 (middle of subcluster) ###
179
+ alloc="0 1"; zero=""
180
+ _run_test sc=1 off=1 len=512
181
+
182
+ ### Write subcluster #2 (end of subcluster) ###
183
+ alloc="0 1 2"; zero=""
184
+ _run_test sc=2 off=1 len=1k
185
+
186
+ ### Write subcluster #3 (full subcluster) ###
187
+ alloc="0 1 2 3"; zero=""
188
+ _run_test sc=3 len=2k
189
+
190
+ ### Write subclusters #4-6 (full subclusters) ###
191
+ alloc="$(seq 0 6)"; zero=""
192
+ _run_test sc=4 len=6k
193
+
194
+ ### Write subclusters #7-9 (partial subclusters) ###
195
+ alloc="$(seq 0 9)"; zero=""
196
+ _run_test sc=7 off=1 len=4k
197
+
198
+ ### Write subcluster #16 (partial subcluster) ###
199
+ alloc="$(seq 0 9) 16"; zero=""
200
+ _run_test sc=16 len=1k
201
+
202
+ ### Write subcluster #31-#33 (cluster overlap) ###
203
+ alloc="$(seq 0 9) 16 31"; zero=""
204
+ _run_test sc=31 off=1 len=4k
205
+ alloc="0 1" ; zero=""
206
+ _verify_l2_bitmap 1
207
+
208
+ ### Zero subcluster #1
209
+ alloc="0 $(seq 2 9) 16 31"; zero="1"
210
+ _run_test sc=1 len=2k cmd=zero
211
+
212
+ ### Zero cluster #0
213
+ alloc=""; zero="$(seq 0 31)"
214
+ _run_test sc=0 len=64k cmd=zero
215
+
216
+ ### Fill cluster #0 with data
217
+ alloc="$(seq 0 31)"; zero=""
218
+ _run_test sc=0 len=64k
219
+
220
+ ### Zero and unmap half of cluster #0 (this won't unmap it)
221
+ alloc="$(seq 16 31)"; zero="$(seq 0 15)"
222
+ _run_test sc=0 len=32k cmd=unmap
223
+
224
+ ### Zero and unmap cluster #0
225
+ alloc=""; zero="$(seq 0 31)"
226
+ _run_test sc=0 len=64k cmd=unmap
227
+
228
+ ### Write subcluster #1 (middle of subcluster)
229
+ alloc="1"; zero="0 $(seq 2 31)"
230
+ _run_test sc=1 off=1 len=512
231
+
232
+ ### Fill cluster #0 with data
233
+ alloc="$(seq 0 31)"; zero=""
234
+ _run_test sc=0 len=64k
235
+
236
+ ### Discard cluster #0
237
+ alloc=""; zero="$(seq 0 31)"
238
+ _run_test sc=0 len=64k cmd=discard
239
+
240
+ ### Write compressed data to cluster #0
241
+ alloc=""; zero=""
242
+ _run_test sc=0 len=64k cmd=compress
243
+
244
+ ### Write subcluster #1 (middle of subcluster)
245
+ alloc="$(seq 0 31)"; zero=""
246
+ _run_test sc=1 off=1 len=512
247
+done
248
+
249
+############################################################
250
+############################################################
251
+############################################################
252
+
253
+# calculate_l2_meta() checks if none of the clusters affected by a
254
+# write operation need COW or changes to their L2 metadata and simply
255
+# returns when they don't. This is a test for that optimization.
256
+# Here clusters #0-#3 are overwritten but only #1 and #2 need changes.
257
+echo
258
+echo '### Overwriting several clusters without COW ###'
259
+echo
260
+use_backing_file="no" _reset_img 1M
261
+# Write cluster #0, subclusters #12-#31
262
+alloc="$(seq 12 31)"; zero=""
263
+_run_test sc=12 len=40k
264
+
265
+# Write cluster #1, subcluster #13
266
+alloc="13"; zero=""
267
+_run_test c=1 sc=13 len=2k
268
+
269
+# Zeroize cluster #2, subcluster #14
270
+alloc="14"; zero=""
271
+_run_test c=2 sc=14 len=2k
272
+alloc=""; zero="14"
273
+_run_test c=2 sc=14 len=2k cmd=zero
274
+
275
+# Write cluster #3, subclusters #0-#16
276
+alloc="$(seq 0 16)"; zero=""
277
+_run_test c=3 sc=0 len=34k
278
+
279
+# Write from cluster #0, subcluster #12 to cluster #3, subcluster #11
280
+alloc="$(seq 12 31)"; zero=""
281
+_run_test sc=12 len=192k
282
+alloc="$(seq 0 31)"; zero=""
283
+_verify_l2_bitmap 1
284
+_verify_l2_bitmap 2
285
+
286
+alloc="$(seq 0 16)"; zero=""
287
+_verify_l2_bitmap 3
288
+
289
+############################################################
290
+############################################################
291
+############################################################
292
+
293
+# Test different patterns of writing zeroes
294
+for use_backing_file in yes no; do
295
+ echo
296
+ echo "### Writing zeroes 1: unallocated clusters (backing file: $use_backing_file) ###"
297
+ echo
298
+ # Note that the image size is not a multiple of the cluster size
299
+ _reset_img 2083k
300
+
301
+ # Cluster-aligned request from clusters #0 to #2
302
+ alloc=""; zero="$(seq 0 31)"
303
+ _run_test c=0 sc=0 len=192k cmd=zero
304
+ _verify_l2_bitmap 1
305
+ _verify_l2_bitmap 2
306
+
307
+ # Subcluster-aligned request from clusters #3 to #5
308
+ alloc=""; zero="$(seq 16 31)"
309
+ _run_test c=3 sc=16 len=128k cmd=zero
310
+ alloc=""; zero="$(seq 0 31)"
311
+ _verify_l2_bitmap 4
312
+ alloc=""; zero="$(seq 0 15)"
313
+ _verify_l2_bitmap 5
314
+
315
+ # Unaligned request from clusters #6 to #8
316
+ if [ "$use_backing_file" = "yes" ]; then
317
+ alloc="15"; zero="$(seq 16 31)" # copy-on-write happening here
318
+ else
319
+ alloc=""; zero="$(seq 15 31)"
320
+ fi
321
+ _run_test c=6 sc=15 off=1 len=128k cmd=zero
322
+ alloc=""; zero="$(seq 0 31)"
323
+ _verify_l2_bitmap 7
324
+ if [ "$use_backing_file" = "yes" ]; then
325
+ alloc="15"; zero="$(seq 0 14)" # copy-on-write happening here
326
+ else
327
+ alloc=""; zero="$(seq 0 15)"
328
+ fi
329
+ _verify_l2_bitmap 8
330
+
331
+ echo
332
+ echo "### Writing zeroes 2: allocated clusters (backing file: $use_backing_file) ###"
333
+ echo
334
+ alloc="$(seq 0 31)"; zero=""
335
+ _run_test c=9 sc=0 len=576k
336
+ _verify_l2_bitmap 10
337
+ _verify_l2_bitmap 11
338
+ _verify_l2_bitmap 12
339
+ _verify_l2_bitmap 13
340
+ _verify_l2_bitmap 14
341
+ _verify_l2_bitmap 15
342
+ _verify_l2_bitmap 16
343
+ _verify_l2_bitmap 17
344
+
345
+ # Cluster-aligned request from clusters #9 to #11
346
+ alloc=""; zero="$(seq 0 31)"
347
+ _run_test c=9 sc=0 len=192k cmd=zero
348
+ _verify_l2_bitmap 10
349
+ _verify_l2_bitmap 11
350
+
351
+ # Subcluster-aligned request from clusters #12 to #14
352
+ alloc="$(seq 0 15)"; zero="$(seq 16 31)"
353
+ _run_test c=12 sc=16 len=128k cmd=zero
354
+ alloc=""; zero="$(seq 0 31)"
355
+ _verify_l2_bitmap 13
356
+ alloc="$(seq 16 31)"; zero="$(seq 0 15)"
357
+ _verify_l2_bitmap 14
358
+
359
+ # Unaligned request from clusters #15 to #17
360
+ alloc="$(seq 0 15)"; zero="$(seq 16 31)"
361
+ _run_test c=15 sc=15 off=1 len=128k cmd=zero
362
+ alloc=""; zero="$(seq 0 31)"
363
+ _verify_l2_bitmap 16
364
+ alloc="$(seq 15 31)"; zero="$(seq 0 14)"
365
+ _verify_l2_bitmap 17
366
+
367
+ echo
368
+ echo "### Writing zeroes 3: compressed clusters (backing file: $use_backing_file) ###"
369
+ echo
370
+ alloc=""; zero=""
371
+ for c in $(seq 18 28); do
372
+ _run_test c=$c sc=0 len=64k cmd=compress
373
+ done
374
+
375
+ # Cluster-aligned request from clusters #18 to #20
376
+ alloc=""; zero="$(seq 0 31)"
377
+ _run_test c=18 sc=0 len=192k cmd=zero
378
+ _verify_l2_bitmap 19
379
+ _verify_l2_bitmap 20
380
+
381
+ # Subcluster-aligned request from clusters #21 to #23.
382
+ # We cannot partially zero a compressed cluster so the code
383
+ # returns -ENOTSUP, which means copy-on-write of the compressed
384
+ # data and fill the rest with actual zeroes on disk.
385
+ # TODO: cluster #22 should use the 'all zeroes' bits.
386
+ alloc="$(seq 0 31)"; zero=""
387
+ _run_test c=21 sc=16 len=128k cmd=zero
388
+ _verify_l2_bitmap 22
389
+ _verify_l2_bitmap 23
390
+
391
+ # Unaligned request from clusters #24 to #26
392
+ # In this case QEMU internally sends a 1k request followed by a
393
+ # subcluster-aligned 128k request. The first request decompresses
394
+ # cluster #24, but that's not enough to perform the second request
395
+ # efficiently because it partially writes to cluster #26 (which is
396
+ # compressed) so we hit the same problem as before.
397
+ alloc="$(seq 0 31)"; zero=""
398
+ _run_test c=24 sc=15 off=1 len=129k cmd=zero
399
+ _verify_l2_bitmap 25
400
+ _verify_l2_bitmap 26
401
+
402
+ # Unaligned request from clusters #27 to #29
403
+ # Similar to the previous case, but this time the tail of the
404
+ # request does not correspond to a compressed cluster, so it can
405
+ # be zeroed efficiently.
406
+ # Note that the very last subcluster is partially written, so if
407
+ # there's a backing file we need to perform cow.
408
+ alloc="$(seq 0 15)"; zero="$(seq 16 31)"
409
+ _run_test c=27 sc=15 off=1 len=128k cmd=zero
410
+ alloc=""; zero="$(seq 0 31)"
411
+ _verify_l2_bitmap 28
412
+ if [ "$use_backing_file" = "yes" ]; then
413
+ alloc="15"; zero="$(seq 0 14)" # copy-on-write happening here
414
+ else
415
+ alloc=""; zero="$(seq 0 15)"
416
+ fi
417
+ _verify_l2_bitmap 29
418
+
419
+ echo
420
+ echo "### Writing zeroes 4: other tests (backing file: $use_backing_file) ###"
421
+ echo
422
+ # Unaligned request in the middle of cluster #30.
423
+ # If there's a backing file we need to allocate and do
424
+ # copy-on-write on the partially zeroed subclusters.
425
+ # If not we can set the 'all zeroes' bit on them.
426
+ if [ "$use_backing_file" = "yes" ]; then
427
+ alloc="15 19"; zero="$(seq 16 18)" # copy-on-write happening here
428
+ else
429
+ alloc=""; zero="$(seq 15 19)"
430
+ fi
431
+ _run_test c=30 sc=15 off=1 len=8k cmd=zero
432
+
433
+ # Fill the last cluster with zeroes, up to the end of the image
434
+ # (the image size is not a multiple of the cluster or subcluster size).
435
+ alloc=""; zero="$(seq 0 17)"
436
+ _run_test c=32 sc=0 len=35k cmd=zero
437
+done
438
+
439
+############################################################
440
+############################################################
441
+############################################################
442
+
443
+# Zero + unmap
444
+for use_backing_file in yes no; do
445
+ echo
446
+ echo "### Zero + unmap 1: allocated clusters (backing file: $use_backing_file) ###"
447
+ echo
448
+ # Note that the image size is not a multiple of the cluster size
449
+ _reset_img 2083k
450
+ alloc="$(seq 0 31)"; zero=""
451
+ _run_test c=9 sc=0 len=576k
452
+ _verify_l2_bitmap 10
453
+ _verify_l2_bitmap 11
454
+ _verify_l2_bitmap 12
455
+ _verify_l2_bitmap 13
456
+ _verify_l2_bitmap 14
457
+ _verify_l2_bitmap 15
458
+ _verify_l2_bitmap 16
459
+ _verify_l2_bitmap 17
460
+
461
+ # Cluster-aligned request from clusters #9 to #11
462
+ alloc=""; zero="$(seq 0 31)"
463
+ _run_test c=9 sc=0 len=192k cmd=unmap
464
+ _verify_l2_bitmap 10
465
+ _verify_l2_bitmap 11
466
+
467
+ # Subcluster-aligned request from clusters #12 to #14
468
+ alloc="$(seq 0 15)"; zero="$(seq 16 31)"
469
+ _run_test c=12 sc=16 len=128k cmd=unmap
470
+ alloc=""; zero="$(seq 0 31)"
471
+ _verify_l2_bitmap 13
472
+ alloc="$(seq 16 31)"; zero="$(seq 0 15)"
473
+ _verify_l2_bitmap 14
474
+
475
+ # Unaligned request from clusters #15 to #17
476
+ alloc="$(seq 0 15)"; zero="$(seq 16 31)"
477
+ _run_test c=15 sc=15 off=1 len=128k cmd=unmap
478
+ alloc=""; zero="$(seq 0 31)"
479
+ _verify_l2_bitmap 16
480
+ alloc="$(seq 15 31)"; zero="$(seq 0 14)"
481
+ _verify_l2_bitmap 17
482
+
483
+ echo
484
+ echo "### Zero + unmap 2: compressed clusters (backing file: $use_backing_file) ###"
485
+ echo
486
+ alloc=""; zero=""
487
+ for c in $(seq 18 28); do
488
+ _run_test c=$c sc=0 len=64k cmd=compress
489
+ done
490
+
491
+ # Cluster-aligned request from clusters #18 to #20
492
+ alloc=""; zero="$(seq 0 31)"
493
+ _run_test c=18 sc=0 len=192k cmd=unmap
494
+ _verify_l2_bitmap 19
495
+ _verify_l2_bitmap 20
496
+
497
+ # Subcluster-aligned request from clusters #21 to #23.
498
+ # We cannot partially zero a compressed cluster so the code
499
+ # returns -ENOTSUP, which means copy-on-write of the compressed
500
+ # data and fill the rest with actual zeroes on disk.
501
+ # TODO: cluster #22 should use the 'all zeroes' bits.
502
+ alloc="$(seq 0 31)"; zero=""
503
+ _run_test c=21 sc=16 len=128k cmd=unmap
504
+ _verify_l2_bitmap 22
505
+ _verify_l2_bitmap 23
506
+
507
+ # Unaligned request from clusters #24 to #26
508
+ # In this case QEMU internally sends a 1k request followed by a
509
+ # subcluster-aligned 128k request. The first request decompresses
510
+ # cluster #24, but that's not enough to perform the second request
511
+ # efficiently because it partially writes to cluster #26 (which is
512
+ # compressed) so we hit the same problem as before.
513
+ alloc="$(seq 0 31)"; zero=""
514
+ _run_test c=24 sc=15 off=1 len=129k cmd=unmap
515
+ _verify_l2_bitmap 25
516
+ _verify_l2_bitmap 26
517
+
518
+ # Unaligned request from clusters #27 to #29
519
+ # Similar to the previous case, but this time the tail of the
520
+ # request does not correspond to a compressed cluster, so it can
521
+ # be zeroed efficiently.
522
+ # Note that the very last subcluster is partially written, so if
523
+ # there's a backing file we need to perform cow.
524
+ alloc="$(seq 0 15)"; zero="$(seq 16 31)"
525
+ _run_test c=27 sc=15 off=1 len=128k cmd=unmap
526
+ alloc=""; zero="$(seq 0 31)"
527
+ _verify_l2_bitmap 28
528
+ if [ "$use_backing_file" = "yes" ]; then
529
+ alloc="15"; zero="$(seq 0 14)" # copy-on-write happening here
530
+ else
531
+ alloc=""; zero="$(seq 0 15)"
532
+ fi
533
+ _verify_l2_bitmap 29
534
+done
535
+
536
+############################################################
537
+############################################################
538
+############################################################
539
+
540
+# Test qcow2_cluster_discard() with full and normal discards
541
+for use_backing_file in yes no; do
542
+ echo
543
+ echo "### Discarding clusters with non-zero bitmaps (backing file: $use_backing_file) ###"
544
+ echo
545
+ if [ "$use_backing_file" = "yes" ]; then
546
+ _make_test_img -o extended_l2=on -F raw -b "$TEST_IMG.base" 1M
547
+ else
548
+ _make_test_img -o extended_l2=on 1M
549
+ fi
550
+ # Write clusters #0-#2 and then discard them
551
+ $QEMU_IO -c 'write -q 0 128k' "$TEST_IMG"
552
+ $QEMU_IO -c 'discard -q 0 128k' "$TEST_IMG"
553
+ # 'qemu-io discard' doesn't do a full discard, it zeroizes the
554
+ # cluster, so both clusters have all zero bits set now
555
+ alloc=""; zero="$(seq 0 31)"
556
+ _verify_l2_bitmap 0
557
+ _verify_l2_bitmap 1
558
+ # Now mark the 2nd half of the subclusters from cluster #0 as unallocated
559
+ poke_file "$TEST_IMG" $(($l2_offset+8)) "\x00\x00"
560
+ # Discard cluster #0 again to see how the zero bits have changed
561
+ $QEMU_IO -c 'discard -q 0 64k' "$TEST_IMG"
562
+ # And do a full discard of cluster #1 by shrinking and growing the image
563
+ $QEMU_IMG resize --shrink "$TEST_IMG" 64k
564
+ $QEMU_IMG resize "$TEST_IMG" 1M
565
+ # A normal discard sets all 'zero' bits only if the image has a
566
+ # backing file, otherwise it won't touch them.
567
+ if [ "$use_backing_file" = "yes" ]; then
568
+ alloc=""; zero="$(seq 0 31)"
569
+ else
570
+ alloc=""; zero="$(seq 0 15)"
571
+ fi
572
+ _verify_l2_bitmap 0
573
+ # A full discard should clear the L2 entry completely. However
574
+ # when growing an image with a backing file the new clusters are
575
+ # zeroized to hide the stale data from the backing file
576
+ if [ "$use_backing_file" = "yes" ]; then
577
+ alloc=""; zero="$(seq 0 31)"
578
+ else
579
+ alloc=""; zero=""
580
+ fi
581
+ _verify_l2_bitmap 1
582
+done
583
+
584
+############################################################
585
+############################################################
586
+############################################################
587
+
588
+# Test that corrupted L2 entries are detected in both read and write
589
+# operations
590
+for corruption_test_cmd in read write; do
591
+ echo
592
+ echo "### Corrupted L2 entries - $corruption_test_cmd test (allocated) ###"
593
+ echo
594
+ echo "# 'cluster is zero' bit set on the standard cluster descriptor"
595
+ echo
596
+ # We actually don't consider this a corrupted image.
597
+ # The 'cluster is zero' bit is unused in extended L2 entries so
598
+ # QEMU ignores it.
599
+ # TODO: maybe treat the image as corrupted and make qemu-img check fix it?
600
+ _make_test_img -o extended_l2=on 1M
601
+ $QEMU_IO -c 'write -q -P 0x11 0 2k' "$TEST_IMG"
602
+ poke_file "$TEST_IMG" $(($l2_offset+7)) "\x01"
603
+ alloc="0"; zero=""
604
+ _verify_l2_bitmap 0
605
+ $QEMU_IO -c "$corruption_test_cmd -q -P 0x11 0 1k" "$TEST_IMG"
606
+ if [ "$corruption_test_cmd" = "write" ]; then
607
+ alloc="0"; zero=""
608
+ fi
609
+ _verify_l2_bitmap 0
610
+
611
+ echo
612
+ echo "# Both 'subcluster is zero' and 'subcluster is allocated' bits set"
613
+ echo
614
+ _make_test_img -o extended_l2=on 1M
615
+ # Write from the middle of cluster #0 to the middle of cluster #2
616
+ $QEMU_IO -c 'write -q 32k 128k' "$TEST_IMG"
617
+ # Corrupt the L2 entry from cluster #1
618
+ poke_file_be "$TEST_IMG" $(($l2_offset+24)) 4 1
619
+ alloc="$(seq 0 31)"; zero="0"
620
+ _verify_l2_bitmap 1
621
+ $QEMU_IO -c "$corruption_test_cmd 0 192k" "$TEST_IMG"
622
+
623
+ echo
624
+ echo "### Corrupted L2 entries - $corruption_test_cmd test (unallocated) ###"
625
+ echo
626
+ echo "# 'cluster is zero' bit set on the standard cluster descriptor"
627
+ echo
628
+ # We actually don't consider this a corrupted image.
629
+ # The 'cluster is zero' bit is unused in extended L2 entries so
630
+ # QEMU ignores it.
631
+ # TODO: maybe treat the image as corrupted and make qemu-img check fix it?
632
+ _make_test_img -o extended_l2=on 1M
633
+ # We want to modify the (empty) L2 entry from cluster #0,
634
+ # but we write to #4 in order to initialize the L2 table first
635
+ $QEMU_IO -c 'write -q 256k 1k' "$TEST_IMG"
636
+ poke_file "$TEST_IMG" $(($l2_offset+7)) "\x01"
637
+ alloc=""; zero=""
638
+ _verify_l2_bitmap 0
639
+ $QEMU_IO -c "$corruption_test_cmd -q 0 1k" "$TEST_IMG"
640
+ if [ "$corruption_test_cmd" = "write" ]; then
641
+ alloc="0"; zero=""
642
+ fi
643
+ _verify_l2_bitmap 0
644
+
645
+ echo
646
+ echo "# 'subcluster is allocated' bit set"
647
+ echo
648
+ _make_test_img -o extended_l2=on 1M
649
+ # We want to corrupt the (empty) L2 entry from cluster #0,
650
+ # but we write to #4 in order to initialize the L2 table first
651
+ $QEMU_IO -c 'write -q 256k 1k' "$TEST_IMG"
652
+ poke_file "$TEST_IMG" $(($l2_offset+15)) "\x01"
653
+ alloc="0"; zero=""
654
+ _verify_l2_bitmap 0
655
+ $QEMU_IO -c "$corruption_test_cmd 0 1k" "$TEST_IMG"
656
+
657
+ echo
658
+ echo "# Both 'subcluster is zero' and 'subcluster is allocated' bits set"
659
+ echo
660
+ _make_test_img -o extended_l2=on 1M
661
+ # We want to corrupt the (empty) L2 entry from cluster #1,
662
+ # but we write to #4 in order to initialize the L2 table first
663
+ $QEMU_IO -c 'write -q 256k 1k' "$TEST_IMG"
664
+ # Corrupt the L2 entry from cluster #1
665
+ poke_file_be "$TEST_IMG" $(($l2_offset+24)) 8 $(((1 << 32) | 1))
666
+ alloc="0"; zero="0"
667
+ _verify_l2_bitmap 1
668
+ $QEMU_IO -c "$corruption_test_cmd 0 192k" "$TEST_IMG"
669
+
670
+ echo
671
+ echo "### Compressed cluster with subcluster bitmap != 0 - $corruption_test_cmd test ###"
672
+ echo
673
+ # We actually don't consider this a corrupted image.
674
+ # The bitmap in compressed clusters is unused so QEMU should just ignore it.
675
+ _make_test_img -o extended_l2=on 1M
676
+ $QEMU_IO -c 'write -q -P 11 -c 0 64k' "$TEST_IMG"
677
+ # Change the L2 bitmap to allocate subcluster #31 and zeroize subcluster #0
678
+ poke_file "$TEST_IMG" $(($l2_offset+11)) "\x01\x80"
679
+ alloc="31"; zero="0"
680
+ _verify_l2_bitmap 0
681
+ $QEMU_IO -c "$corruption_test_cmd -P 11 0 64k" "$TEST_IMG" | _filter_qemu_io
682
+ # Writing allocates a new uncompressed cluster so we get a new bitmap
683
+ if [ "$corruption_test_cmd" = "write" ]; then
684
+ alloc="$(seq 0 31)"; zero=""
685
+ fi
686
+ _verify_l2_bitmap 0
687
+done
688
+
689
+############################################################
690
+############################################################
691
+############################################################
692
+
693
+echo
694
+echo "### Detect and repair unaligned clusters ###"
695
+echo
696
+# Create a backing file and fill it with data
697
+$QEMU_IMG create -f raw "$TEST_IMG.base" 128k | _filter_img_create
698
+$QEMU_IO -c "write -q -P 0xff 0 128k" -f raw "$TEST_IMG.base" | _filter_qemu_io
699
+
700
+echo "# Corrupted L2 entry, allocated subcluster #"
701
+# Create a new image, allocate a cluster and write some data to it
702
+_make_test_img -o extended_l2=on -F raw -b "$TEST_IMG.base"
703
+$QEMU_IO -c 'write -q -P 1 4k 2k' "$TEST_IMG"
704
+# Corrupt the L2 entry by making the offset unaligned
705
+poke_file "$TEST_IMG" "$(($l2_offset+6))" "\x02"
706
+# This cannot be repaired, qemu-img check will fail to fix it
707
+_check_test_img -r all
708
+# Attempting to read the image will still show that it's corrupted
709
+$QEMU_IO -c 'read -q 0 2k' "$TEST_IMG"
710
+
711
+echo "# Corrupted L2 entry, no allocated subclusters #"
712
+# Create a new image, allocate a cluster and zeroize subcluster #2
713
+_make_test_img -o extended_l2=on -F raw -b "$TEST_IMG.base"
714
+$QEMU_IO -c 'write -q -P 1 4k 2k' "$TEST_IMG"
715
+$QEMU_IO -c 'write -q -z 4k 2k' "$TEST_IMG"
716
+# Corrupt the L2 entry by making the offset unaligned
717
+poke_file "$TEST_IMG" "$(($l2_offset+6))" "\x02"
718
+# This time none of the subclusters are allocated so we can repair the image
719
+_check_test_img -r all
720
+# And the data can be read normally
721
+$QEMU_IO -c 'read -q -P 0xff 0 4k' "$TEST_IMG"
722
+$QEMU_IO -c 'read -q -P 0x00 4k 2k' "$TEST_IMG"
723
+$QEMU_IO -c 'read -q -P 0xff 6k 122k' "$TEST_IMG"
724
+
725
+############################################################
726
+############################################################
727
+############################################################
728
+
729
+echo
730
+echo "### Image creation options ###"
731
+echo
732
+echo "# cluster_size < 16k"
733
+_make_test_img -o extended_l2=on,cluster_size=8k 1M
734
+
735
+echo "# backing file and preallocation=metadata"
736
+# For preallocation with backing files, create a backing file first
737
+$QEMU_IMG create -f raw "$TEST_IMG.base" 1M | _filter_img_create
738
+$QEMU_IO -c "write -q -P 0xff 0 1M" -f raw "$TEST_IMG.base" | _filter_qemu_io
739
+
740
+_make_test_img -o extended_l2=on,preallocation=metadata -F raw -b "$TEST_IMG.base" 512k
741
+$QEMU_IMG resize "$TEST_IMG" 1M
742
+$QEMU_IO -c 'read -P 0xff 0 512k' "$TEST_IMG" | _filter_qemu_io
743
+$QEMU_IO -c 'read -P 0x00 512k 512k' "$TEST_IMG" | _filter_qemu_io
744
+$QEMU_IMG map "$TEST_IMG" | _filter_testdir
745
+
746
+echo "# backing file and preallocation=falloc"
747
+_make_test_img -o extended_l2=on,preallocation=falloc -F raw -b "$TEST_IMG.base" 512k
748
+$QEMU_IMG resize "$TEST_IMG" 1M
749
+$QEMU_IO -c 'read -P 0xff 0 512k' "$TEST_IMG" | _filter_qemu_io
750
+$QEMU_IO -c 'read -P 0x00 512k 512k' "$TEST_IMG" | _filter_qemu_io
751
+$QEMU_IMG map "$TEST_IMG" | _filter_testdir
752
+
753
+echo "# backing file and preallocation=full"
754
+_make_test_img -o extended_l2=on,preallocation=full -F raw -b "$TEST_IMG.base" 512k
755
+$QEMU_IMG resize "$TEST_IMG" 1M
756
+$QEMU_IO -c 'read -P 0xff 0 512k' "$TEST_IMG" | _filter_qemu_io
757
+$QEMU_IO -c 'read -P 0x00 512k 512k' "$TEST_IMG" | _filter_qemu_io
758
+$QEMU_IMG map "$TEST_IMG" | _filter_testdir
759
+
760
+echo
761
+echo "### Image resizing with preallocation and backing files ###"
762
+echo
763
+# In this case the new subclusters must have the 'all zeroes' bit set
764
+echo "# resize --preallocation=metadata"
765
+_make_test_img -o extended_l2=on -F raw -b "$TEST_IMG.base" 503k
766
+$QEMU_IMG resize --preallocation=metadata "$TEST_IMG" 1013k
767
+$QEMU_IO -c 'read -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io
768
+$QEMU_IO -c 'read -P 0x00 503k 510k' "$TEST_IMG" | _filter_qemu_io
769
+
770
+# In this case and the next one the new subclusters must be allocated
771
+echo "# resize --preallocation=falloc"
772
+_make_test_img -o extended_l2=on -F raw -b "$TEST_IMG.base" 503k
773
+$QEMU_IMG resize --preallocation=falloc "$TEST_IMG" 1013k
774
+$QEMU_IO -c 'read -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io
775
+$QEMU_IO -c 'read -P 0x00 503k 510k' "$TEST_IMG" | _filter_qemu_io
776
+
777
+echo "# resize --preallocation=full"
778
+_make_test_img -o extended_l2=on -F raw -b "$TEST_IMG.base" 503k
779
+$QEMU_IMG resize --preallocation=full "$TEST_IMG" 1013k
780
+$QEMU_IO -c 'read -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io
781
+$QEMU_IO -c 'read -P 0x00 503k 510k' "$TEST_IMG" | _filter_qemu_io
782
+
783
+echo
784
+echo "### Image resizing with preallocation without backing files ###"
785
+echo
786
+# In this case the new subclusters must have the 'all zeroes' bit set
787
+echo "# resize --preallocation=metadata"
788
+_make_test_img -o extended_l2=on 503k
789
+$QEMU_IO -c 'write -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io
790
+$QEMU_IMG resize --preallocation=metadata "$TEST_IMG" 1013k
791
+$QEMU_IO -c 'read -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io
792
+$QEMU_IO -c 'read -P 0x00 503k 510k' "$TEST_IMG" | _filter_qemu_io
793
+
794
+# In this case and the next one the new subclusters must be allocated
795
+echo "# resize --preallocation=falloc"
796
+_make_test_img -o extended_l2=on 503k
797
+$QEMU_IO -c 'write -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io
798
+$QEMU_IMG resize --preallocation=falloc "$TEST_IMG" 1013k
799
+$QEMU_IO -c 'read -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io
800
+$QEMU_IO -c 'read -P 0x00 503k 510k' "$TEST_IMG" | _filter_qemu_io
801
+
802
+echo "# resize --preallocation=full"
803
+_make_test_img -o extended_l2=on 503k
804
+$QEMU_IO -c 'write -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io
805
+$QEMU_IMG resize --preallocation=full "$TEST_IMG" 1013k
806
+$QEMU_IO -c 'read -P 0xff 0 503k' "$TEST_IMG" | _filter_qemu_io
807
+$QEMU_IO -c 'read -P 0x00 503k 510k' "$TEST_IMG" | _filter_qemu_io
808
+
809
+echo
810
+echo "### qemu-img measure ###"
811
+echo
812
+echo "# 512MB, extended_l2=off" # This needs one L2 table
813
+$QEMU_IMG measure --size 512M -O qcow2 -o extended_l2=off
814
+echo "# 512MB, extended_l2=on" # This needs two L2 tables
815
+$QEMU_IMG measure --size 512M -O qcow2 -o extended_l2=on
816
+
817
+echo "# 16K clusters, 64GB, extended_l2=off" # This needs one full L1 table cluster
818
+$QEMU_IMG measure --size 64G -O qcow2 -o cluster_size=16k,extended_l2=off
819
+echo "# 16K clusters, 64GB, extended_l2=on" # This needs two full L2 table clusters
820
+$QEMU_IMG measure --size 64G -O qcow2 -o cluster_size=16k,extended_l2=on
821
+
822
+echo "# 8k clusters" # This should fail
823
+$QEMU_IMG measure --size 1M -O qcow2 -o cluster_size=8k,extended_l2=on
824
+
825
+echo "# 1024 TB" # Maximum allowed size with extended_l2=on and 64K clusters
826
+$QEMU_IMG measure --size 1024T -O qcow2 -o extended_l2=on
827
+echo "# 1025 TB" # This should fail
828
+$QEMU_IMG measure --size 1025T -O qcow2 -o extended_l2=on
829
+
830
+echo
831
+echo "### qemu-img amend ###"
832
+echo
833
+_make_test_img -o extended_l2=on 1M
834
+$QEMU_IMG amend -o extended_l2=off "$TEST_IMG" && echo "Unexpected pass"
835
+
836
+_make_test_img -o extended_l2=off 1M
837
+$QEMU_IMG amend -o extended_l2=on "$TEST_IMG" && echo "Unexpected pass"
838
+
839
+echo
840
+echo "### Test copy-on-write on an image with snapshots ###"
841
+echo
842
+_make_test_img -o extended_l2=on 1M
843
+
844
+# For each cluster from #0 to #9 this loop zeroes subcluster #7
845
+# and allocates subclusters #13 and #18.
846
+alloc="13 18"; zero="7"
847
+for c in $(seq 0 9); do
848
+ $QEMU_IO -c "write -q -z $((64*$c+14))k 2k" \
849
+ -c "write -q -P $((0xd0+$c)) $((64*$c+26))k 2k" \
850
+ -c "write -q -P $((0xe0+$c)) $((64*$c+36))k 2k" "$TEST_IMG"
851
+ _verify_l2_bitmap "$c"
852
+done
853
+
854
+# Create a snapshot and set l2_offset to the new L2 table
855
+$QEMU_IMG snapshot -c snap1 "$TEST_IMG"
856
+l2_offset=$((0x110000))
857
+
858
+# Write different patterns to each one of the clusters
859
+# in order to see how copy-on-write behaves in each case.
860
+$QEMU_IO -c "write -q -P 0xf0 $((64*0+30))k 1k" \
861
+ -c "write -q -P 0xf1 $((64*1+20))k 1k" \
862
+ -c "write -q -P 0xf2 $((64*2+40))k 1k" \
863
+ -c "write -q -P 0xf3 $((64*3+26))k 1k" \
864
+ -c "write -q -P 0xf4 $((64*4+14))k 1k" \
865
+ -c "write -q -P 0xf5 $((64*5+1))k 1k" \
866
+ -c "write -q -z $((64*6+30))k 3k" \
867
+ -c "write -q -z $((64*7+26))k 2k" \
868
+ -c "write -q -z $((64*8+26))k 1k" \
869
+ -c "write -q -z $((64*9+12))k 1k" \
870
+ "$TEST_IMG"
871
+alloc="$(seq 13 18)"; zero="7" _verify_l2_bitmap 0
872
+alloc="$(seq 10 18)"; zero="7" _verify_l2_bitmap 1
873
+alloc="$(seq 13 20)"; zero="7" _verify_l2_bitmap 2
874
+alloc="$(seq 13 18)"; zero="7" _verify_l2_bitmap 3
875
+alloc="$(seq 7 18)"; zero="" _verify_l2_bitmap 4
876
+alloc="$(seq 0 18)"; zero="" _verify_l2_bitmap 5
877
+alloc="13 18"; zero="7 15 16" _verify_l2_bitmap 6
878
+alloc="18"; zero="7 13" _verify_l2_bitmap 7
879
+alloc="$(seq 13 18)"; zero="7" _verify_l2_bitmap 8
880
+alloc="13 18"; zero="6 7" _verify_l2_bitmap 9
881
+
882
+echo
883
+echo "### Test concurrent requests ###"
884
+echo
885
+
886
+_concurrent_io()
887
+{
888
+# Allocate three subclusters in the same cluster.
889
+# This works because handle_dependencies() checks whether the requests
890
+# allocate the same cluster, even if the COW regions don't overlap (in
891
+# this case they don't).
892
+cat <<EOF
893
+open -o driver=$IMGFMT blkdebug::$TEST_IMG
894
+break write_aio A
895
+aio_write -P 10 30k 2k
896
+wait_break A
897
+aio_write -P 11 20k 2k
898
+aio_write -P 12 40k 2k
899
+resume A
900
+aio_flush
901
+EOF
902
+}
903
+
904
+_concurrent_verify()
905
+{
906
+cat <<EOF
907
+open -o driver=$IMGFMT $TEST_IMG
908
+read -q -P 10 30k 2k
909
+read -q -P 11 20k 2k
910
+read -q -P 12 40k 2k
911
+EOF
912
+}
913
+
914
+_make_test_img -o extended_l2=on 1M
915
+_concurrent_io | $QEMU_IO | _filter_qemu_io
916
+_concurrent_verify | $QEMU_IO | _filter_qemu_io
917
+
918
+# success, all done
919
+echo "*** done"
920
+rm -f $seq.full
921
+status=0
922
diff --git a/tests/qemu-iotests/271.out b/tests/qemu-iotests/271.out
923
new file mode 100644
924
index XXXXXXX..XXXXXXX
925
--- /dev/null
926
+++ b/tests/qemu-iotests/271.out
927
@@ -XXX,XX +XXX,XX @@
928
+QA output created by 271
929
+
930
+### Standard write tests (backing file: yes) ###
931
+
932
+Formatting 'TEST_DIR/t.IMGFMT.raw', fmt=raw size=1048576
933
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=raw size=1048576
934
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw
935
+write -q -P PATTERN 0 1k
936
+L2 entry #0: 0x8000000000050000 0000000000000001
937
+write -q -P PATTERN 3k 512
938
+L2 entry #0: 0x8000000000050000 0000000000000003
939
+write -q -P PATTERN 5k 1k
940
+L2 entry #0: 0x8000000000050000 0000000000000007
941
+write -q -P PATTERN 6k 2k
942
+L2 entry #0: 0x8000000000050000 000000000000000f
943
+write -q -P PATTERN 8k 6k
944
+L2 entry #0: 0x8000000000050000 000000000000007f
945
+write -q -P PATTERN 15k 4k
946
+L2 entry #0: 0x8000000000050000 00000000000003ff
947
+write -q -P PATTERN 32k 1k
948
+L2 entry #0: 0x8000000000050000 00000000000103ff
949
+write -q -P PATTERN 63k 4k
950
+L2 entry #0: 0x8000000000050000 00000000800103ff
951
+L2 entry #1: 0x8000000000060000 0000000000000003
952
+write -q -z 2k 2k
953
+L2 entry #0: 0x8000000000050000 00000002800103fd
954
+write -q -z 0 64k
955
+L2 entry #0: 0x8000000000050000 ffffffff00000000
956
+write -q -P PATTERN 0 64k
957
+L2 entry #0: 0x8000000000050000 00000000ffffffff
958
+write -q -z -u 0 32k
959
+L2 entry #0: 0x8000000000050000 0000ffffffff0000
960
+write -q -z -u 0 64k
961
+L2 entry #0: 0x0000000000000000 ffffffff00000000
962
+write -q -P PATTERN 3k 512
963
+L2 entry #0: 0x8000000000050000 fffffffd00000002
964
+write -q -P PATTERN 0 64k
965
+L2 entry #0: 0x8000000000050000 00000000ffffffff
966
+discard -q 0 64k
967
+L2 entry #0: 0x0000000000000000 ffffffff00000000
968
+write -q -c -P PATTERN 0 64k
969
+L2 entry #0: 0x4000000000050000 0000000000000000
970
+write -q -P PATTERN 3k 512
971
+L2 entry #0: 0x8000000000070000 00000000ffffffff
972
+
973
+### Standard write tests (backing file: no) ###
974
+
975
+Formatting 'TEST_DIR/t.IMGFMT.raw', fmt=raw size=1048576
976
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
977
+write -q -P PATTERN 0 1k
978
+L2 entry #0: 0x8000000000050000 0000000000000001
979
+write -q -P PATTERN 3k 512
980
+L2 entry #0: 0x8000000000050000 0000000000000003
981
+write -q -P PATTERN 5k 1k
982
+L2 entry #0: 0x8000000000050000 0000000000000007
983
+write -q -P PATTERN 6k 2k
984
+L2 entry #0: 0x8000000000050000 000000000000000f
985
+write -q -P PATTERN 8k 6k
986
+L2 entry #0: 0x8000000000050000 000000000000007f
987
+write -q -P PATTERN 15k 4k
988
+L2 entry #0: 0x8000000000050000 00000000000003ff
989
+write -q -P PATTERN 32k 1k
990
+L2 entry #0: 0x8000000000050000 00000000000103ff
991
+write -q -P PATTERN 63k 4k
992
+L2 entry #0: 0x8000000000050000 00000000800103ff
993
+L2 entry #1: 0x8000000000060000 0000000000000003
994
+write -q -z 2k 2k
995
+L2 entry #0: 0x8000000000050000 00000002800103fd
996
+write -q -z 0 64k
997
+L2 entry #0: 0x8000000000050000 ffffffff00000000
998
+write -q -P PATTERN 0 64k
999
+L2 entry #0: 0x8000000000050000 00000000ffffffff
1000
+write -q -z -u 0 32k
1001
+L2 entry #0: 0x8000000000050000 0000ffffffff0000
1002
+write -q -z -u 0 64k
1003
+L2 entry #0: 0x0000000000000000 ffffffff00000000
1004
+write -q -P PATTERN 3k 512
1005
+L2 entry #0: 0x8000000000050000 fffffffd00000002
1006
+write -q -P PATTERN 0 64k
1007
+L2 entry #0: 0x8000000000050000 00000000ffffffff
1008
+discard -q 0 64k
1009
+L2 entry #0: 0x0000000000000000 ffffffff00000000
1010
+write -q -c -P PATTERN 0 64k
1011
+L2 entry #0: 0x4000000000050000 0000000000000000
1012
+write -q -P PATTERN 3k 512
1013
+L2 entry #0: 0x8000000000070000 00000000ffffffff
1014
+
1015
+### Overwriting several clusters without COW ###
1016
+
1017
+Formatting 'TEST_DIR/t.IMGFMT.raw', fmt=raw size=1048576
1018
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1019
+write -q -P PATTERN 24k 40k
1020
+L2 entry #0: 0x8000000000050000 00000000fffff000
1021
+write -q -P PATTERN 90k 2k
1022
+L2 entry #1: 0x8000000000060000 0000000000002000
1023
+write -q -P PATTERN 156k 2k
1024
+L2 entry #2: 0x8000000000070000 0000000000004000
1025
+write -q -z 156k 2k
1026
+L2 entry #2: 0x8000000000070000 0000400000000000
1027
+write -q -P PATTERN 192k 34k
1028
+L2 entry #3: 0x8000000000080000 000000000001ffff
1029
+write -q -P PATTERN 24k 192k
1030
+L2 entry #0: 0x8000000000050000 00000000fffff000
1031
+L2 entry #1: 0x8000000000060000 00000000ffffffff
1032
+L2 entry #2: 0x8000000000070000 00000000ffffffff
1033
+L2 entry #3: 0x8000000000080000 000000000001ffff
1034
+
1035
+### Writing zeroes 1: unallocated clusters (backing file: yes) ###
1036
+
1037
+Formatting 'TEST_DIR/t.IMGFMT.raw', fmt=raw size=2132992
1038
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=raw size=2132992
1039
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2132992 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw
1040
+write -q -z 0 192k
1041
+L2 entry #0: 0x0000000000000000 ffffffff00000000
1042
+L2 entry #1: 0x0000000000000000 ffffffff00000000
1043
+L2 entry #2: 0x0000000000000000 ffffffff00000000
1044
+write -q -z 224k 128k
1045
+L2 entry #3: 0x0000000000000000 ffff000000000000
1046
+L2 entry #4: 0x0000000000000000 ffffffff00000000
1047
+L2 entry #5: 0x0000000000000000 0000ffff00000000
1048
+write -q -z 415k 128k
1049
+L2 entry #6: 0x8000000000050000 ffff000000008000
1050
+L2 entry #7: 0x0000000000000000 ffffffff00000000
1051
+L2 entry #8: 0x8000000000060000 00007fff00008000
1052
+
1053
+### Writing zeroes 2: allocated clusters (backing file: yes) ###
1054
+
1055
+write -q -P PATTERN 576k 576k
1056
+L2 entry #9: 0x8000000000070000 00000000ffffffff
1057
+L2 entry #10: 0x8000000000080000 00000000ffffffff
1058
+L2 entry #11: 0x8000000000090000 00000000ffffffff
1059
+L2 entry #12: 0x80000000000a0000 00000000ffffffff
1060
+L2 entry #13: 0x80000000000b0000 00000000ffffffff
1061
+L2 entry #14: 0x80000000000c0000 00000000ffffffff
1062
+L2 entry #15: 0x80000000000d0000 00000000ffffffff
1063
+L2 entry #16: 0x80000000000e0000 00000000ffffffff
1064
+L2 entry #17: 0x80000000000f0000 00000000ffffffff
1065
+write -q -z 576k 192k
1066
+L2 entry #9: 0x8000000000070000 ffffffff00000000
1067
+L2 entry #10: 0x8000000000080000 ffffffff00000000
1068
+L2 entry #11: 0x8000000000090000 ffffffff00000000
1069
+write -q -z 800k 128k
1070
+L2 entry #12: 0x80000000000a0000 ffff00000000ffff
1071
+L2 entry #13: 0x80000000000b0000 ffffffff00000000
1072
+L2 entry #14: 0x80000000000c0000 0000ffffffff0000
1073
+write -q -z 991k 128k
1074
+L2 entry #15: 0x80000000000d0000 ffff00000000ffff
1075
+L2 entry #16: 0x80000000000e0000 ffffffff00000000
1076
+L2 entry #17: 0x80000000000f0000 00007fffffff8000
1077
+
1078
+### Writing zeroes 3: compressed clusters (backing file: yes) ###
1079
+
1080
+write -q -c -P PATTERN 1152k 64k
1081
+L2 entry #18: 0x4000000000100000 0000000000000000
1082
+write -q -c -P PATTERN 1216k 64k
1083
+L2 entry #19: 0x4000000000110000 0000000000000000
1084
+write -q -c -P PATTERN 1280k 64k
1085
+L2 entry #20: 0x4000000000120000 0000000000000000
1086
+write -q -c -P PATTERN 1344k 64k
1087
+L2 entry #21: 0x4000000000130000 0000000000000000
1088
+write -q -c -P PATTERN 1408k 64k
1089
+L2 entry #22: 0x4000000000140000 0000000000000000
1090
+write -q -c -P PATTERN 1472k 64k
1091
+L2 entry #23: 0x4000000000150000 0000000000000000
1092
+write -q -c -P PATTERN 1536k 64k
1093
+L2 entry #24: 0x4000000000160000 0000000000000000
1094
+write -q -c -P PATTERN 1600k 64k
1095
+L2 entry #25: 0x4000000000170000 0000000000000000
1096
+write -q -c -P PATTERN 1664k 64k
1097
+L2 entry #26: 0x4000000000180000 0000000000000000
1098
+write -q -c -P PATTERN 1728k 64k
1099
+L2 entry #27: 0x4000000000190000 0000000000000000
1100
+write -q -c -P PATTERN 1792k 64k
1101
+L2 entry #28: 0x40000000001a0000 0000000000000000
1102
+write -q -z 1152k 192k
1103
+L2 entry #18: 0x0000000000000000 ffffffff00000000
1104
+L2 entry #19: 0x0000000000000000 ffffffff00000000
1105
+L2 entry #20: 0x0000000000000000 ffffffff00000000
1106
+write -q -z 1376k 128k
1107
+L2 entry #21: 0x8000000000100000 00000000ffffffff
1108
+L2 entry #22: 0x8000000000110000 00000000ffffffff
1109
+L2 entry #23: 0x8000000000120000 00000000ffffffff
1110
+write -q -z 1567k 129k
1111
+L2 entry #24: 0x8000000000130000 00000000ffffffff
1112
+L2 entry #25: 0x8000000000140000 00000000ffffffff
1113
+L2 entry #26: 0x8000000000150000 00000000ffffffff
1114
+write -q -z 1759k 128k
1115
+L2 entry #27: 0x8000000000160000 ffff00000000ffff
1116
+L2 entry #28: 0x0000000000000000 ffffffff00000000
1117
+L2 entry #29: 0x8000000000170000 00007fff00008000
1118
+
1119
+### Writing zeroes 4: other tests (backing file: yes) ###
1120
+
1121
+write -q -z 1951k 8k
1122
+L2 entry #30: 0x8000000000180000 0007000000088000
1123
+write -q -z 2048k 35k
1124
+L2 entry #32: 0x0000000000000000 0003ffff00000000
1125
+
1126
+### Writing zeroes 1: unallocated clusters (backing file: no) ###
1127
+
1128
+Formatting 'TEST_DIR/t.IMGFMT.raw', fmt=raw size=2132992
1129
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2132992
1130
+write -q -z 0 192k
1131
+L2 entry #0: 0x0000000000000000 ffffffff00000000
1132
+L2 entry #1: 0x0000000000000000 ffffffff00000000
1133
+L2 entry #2: 0x0000000000000000 ffffffff00000000
1134
+write -q -z 224k 128k
1135
+L2 entry #3: 0x0000000000000000 ffff000000000000
1136
+L2 entry #4: 0x0000000000000000 ffffffff00000000
1137
+L2 entry #5: 0x0000000000000000 0000ffff00000000
1138
+write -q -z 415k 128k
1139
+L2 entry #6: 0x0000000000000000 ffff800000000000
1140
+L2 entry #7: 0x0000000000000000 ffffffff00000000
1141
+L2 entry #8: 0x0000000000000000 0000ffff00000000
1142
+
1143
+### Writing zeroes 2: allocated clusters (backing file: no) ###
1144
+
1145
+write -q -P PATTERN 576k 576k
1146
+L2 entry #9: 0x8000000000050000 00000000ffffffff
1147
+L2 entry #10: 0x8000000000060000 00000000ffffffff
1148
+L2 entry #11: 0x8000000000070000 00000000ffffffff
1149
+L2 entry #12: 0x8000000000080000 00000000ffffffff
1150
+L2 entry #13: 0x8000000000090000 00000000ffffffff
1151
+L2 entry #14: 0x80000000000a0000 00000000ffffffff
1152
+L2 entry #15: 0x80000000000b0000 00000000ffffffff
1153
+L2 entry #16: 0x80000000000c0000 00000000ffffffff
1154
+L2 entry #17: 0x80000000000d0000 00000000ffffffff
1155
+write -q -z 576k 192k
1156
+L2 entry #9: 0x8000000000050000 ffffffff00000000
1157
+L2 entry #10: 0x8000000000060000 ffffffff00000000
1158
+L2 entry #11: 0x8000000000070000 ffffffff00000000
1159
+write -q -z 800k 128k
1160
+L2 entry #12: 0x8000000000080000 ffff00000000ffff
1161
+L2 entry #13: 0x8000000000090000 ffffffff00000000
1162
+L2 entry #14: 0x80000000000a0000 0000ffffffff0000
1163
+write -q -z 991k 128k
1164
+L2 entry #15: 0x80000000000b0000 ffff00000000ffff
1165
+L2 entry #16: 0x80000000000c0000 ffffffff00000000
1166
+L2 entry #17: 0x80000000000d0000 00007fffffff8000
1167
+
1168
+### Writing zeroes 3: compressed clusters (backing file: no) ###
1169
+
1170
+write -q -c -P PATTERN 1152k 64k
1171
+L2 entry #18: 0x40000000000e0000 0000000000000000
1172
+write -q -c -P PATTERN 1216k 64k
1173
+L2 entry #19: 0x40000000000f0000 0000000000000000
1174
+write -q -c -P PATTERN 1280k 64k
1175
+L2 entry #20: 0x4000000000100000 0000000000000000
1176
+write -q -c -P PATTERN 1344k 64k
1177
+L2 entry #21: 0x4000000000110000 0000000000000000
1178
+write -q -c -P PATTERN 1408k 64k
1179
+L2 entry #22: 0x4000000000120000 0000000000000000
1180
+write -q -c -P PATTERN 1472k 64k
1181
+L2 entry #23: 0x4000000000130000 0000000000000000
1182
+write -q -c -P PATTERN 1536k 64k
1183
+L2 entry #24: 0x4000000000140000 0000000000000000
1184
+write -q -c -P PATTERN 1600k 64k
1185
+L2 entry #25: 0x4000000000150000 0000000000000000
1186
+write -q -c -P PATTERN 1664k 64k
1187
+L2 entry #26: 0x4000000000160000 0000000000000000
1188
+write -q -c -P PATTERN 1728k 64k
1189
+L2 entry #27: 0x4000000000170000 0000000000000000
1190
+write -q -c -P PATTERN 1792k 64k
1191
+L2 entry #28: 0x4000000000180000 0000000000000000
1192
+write -q -z 1152k 192k
1193
+L2 entry #18: 0x0000000000000000 ffffffff00000000
1194
+L2 entry #19: 0x0000000000000000 ffffffff00000000
1195
+L2 entry #20: 0x0000000000000000 ffffffff00000000
1196
+write -q -z 1376k 128k
1197
+L2 entry #21: 0x80000000000e0000 00000000ffffffff
1198
+L2 entry #22: 0x80000000000f0000 00000000ffffffff
1199
+L2 entry #23: 0x8000000000100000 00000000ffffffff
1200
+write -q -z 1567k 129k
1201
+L2 entry #24: 0x8000000000110000 00000000ffffffff
1202
+L2 entry #25: 0x8000000000120000 00000000ffffffff
1203
+L2 entry #26: 0x8000000000130000 00000000ffffffff
1204
+write -q -z 1759k 128k
1205
+L2 entry #27: 0x8000000000140000 ffff00000000ffff
1206
+L2 entry #28: 0x0000000000000000 ffffffff00000000
1207
+L2 entry #29: 0x0000000000000000 0000ffff00000000
1208
+
1209
+### Writing zeroes 4: other tests (backing file: no) ###
1210
+
1211
+write -q -z 1951k 8k
1212
+L2 entry #30: 0x0000000000000000 000f800000000000
1213
+write -q -z 2048k 35k
1214
+L2 entry #32: 0x0000000000000000 0003ffff00000000
1215
+
1216
+### Zero + unmap 1: allocated clusters (backing file: yes) ###
1217
+
1218
+Formatting 'TEST_DIR/t.IMGFMT.raw', fmt=raw size=2132992
1219
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=raw size=2132992
1220
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2132992 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw
1221
+write -q -P PATTERN 576k 576k
1222
+L2 entry #9: 0x8000000000050000 00000000ffffffff
1223
+L2 entry #10: 0x8000000000060000 00000000ffffffff
1224
+L2 entry #11: 0x8000000000070000 00000000ffffffff
1225
+L2 entry #12: 0x8000000000080000 00000000ffffffff
1226
+L2 entry #13: 0x8000000000090000 00000000ffffffff
1227
+L2 entry #14: 0x80000000000a0000 00000000ffffffff
1228
+L2 entry #15: 0x80000000000b0000 00000000ffffffff
1229
+L2 entry #16: 0x80000000000c0000 00000000ffffffff
1230
+L2 entry #17: 0x80000000000d0000 00000000ffffffff
1231
+write -q -z -u 576k 192k
1232
+L2 entry #9: 0x0000000000000000 ffffffff00000000
1233
+L2 entry #10: 0x0000000000000000 ffffffff00000000
1234
+L2 entry #11: 0x0000000000000000 ffffffff00000000
1235
+write -q -z -u 800k 128k
1236
+L2 entry #12: 0x8000000000080000 ffff00000000ffff
1237
+L2 entry #13: 0x0000000000000000 ffffffff00000000
1238
+L2 entry #14: 0x80000000000a0000 0000ffffffff0000
1239
+write -q -z -u 991k 128k
1240
+L2 entry #15: 0x80000000000b0000 ffff00000000ffff
1241
+L2 entry #16: 0x0000000000000000 ffffffff00000000
1242
+L2 entry #17: 0x80000000000d0000 00007fffffff8000
1243
+
1244
+### Zero + unmap 2: compressed clusters (backing file: yes) ###
1245
+
1246
+write -q -c -P PATTERN 1152k 64k
1247
+L2 entry #18: 0x4000000000050000 0000000000000000
1248
+write -q -c -P PATTERN 1216k 64k
1249
+L2 entry #19: 0x4000000000060000 0000000000000000
1250
+write -q -c -P PATTERN 1280k 64k
1251
+L2 entry #20: 0x4000000000070000 0000000000000000
1252
+write -q -c -P PATTERN 1344k 64k
1253
+L2 entry #21: 0x4000000000090000 0000000000000000
1254
+write -q -c -P PATTERN 1408k 64k
1255
+L2 entry #22: 0x40000000000c0000 0000000000000000
1256
+write -q -c -P PATTERN 1472k 64k
1257
+L2 entry #23: 0x40000000000e0000 0000000000000000
1258
+write -q -c -P PATTERN 1536k 64k
1259
+L2 entry #24: 0x40000000000f0000 0000000000000000
1260
+write -q -c -P PATTERN 1600k 64k
1261
+L2 entry #25: 0x4000000000100000 0000000000000000
1262
+write -q -c -P PATTERN 1664k 64k
1263
+L2 entry #26: 0x4000000000110000 0000000000000000
1264
+write -q -c -P PATTERN 1728k 64k
1265
+L2 entry #27: 0x4000000000120000 0000000000000000
1266
+write -q -c -P PATTERN 1792k 64k
1267
+L2 entry #28: 0x4000000000130000 0000000000000000
1268
+write -q -z -u 1152k 192k
1269
+L2 entry #18: 0x0000000000000000 ffffffff00000000
1270
+L2 entry #19: 0x0000000000000000 ffffffff00000000
1271
+L2 entry #20: 0x0000000000000000 ffffffff00000000
1272
+write -q -z -u 1376k 128k
1273
+L2 entry #21: 0x8000000000050000 00000000ffffffff
1274
+L2 entry #22: 0x8000000000060000 00000000ffffffff
1275
+L2 entry #23: 0x8000000000070000 00000000ffffffff
1276
+write -q -z -u 1567k 129k
1277
+L2 entry #24: 0x8000000000090000 00000000ffffffff
1278
+L2 entry #25: 0x80000000000e0000 00000000ffffffff
1279
+L2 entry #26: 0x80000000000f0000 00000000ffffffff
1280
+write -q -z -u 1759k 128k
1281
+L2 entry #27: 0x80000000000c0000 ffff00000000ffff
1282
+L2 entry #28: 0x0000000000000000 ffffffff00000000
1283
+L2 entry #29: 0x8000000000100000 00007fff00008000
1284
+
1285
+### Zero + unmap 1: allocated clusters (backing file: no) ###
1286
+
1287
+Formatting 'TEST_DIR/t.IMGFMT.raw', fmt=raw size=2132992
1288
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2132992
1289
+write -q -P PATTERN 576k 576k
1290
+L2 entry #9: 0x8000000000050000 00000000ffffffff
1291
+L2 entry #10: 0x8000000000060000 00000000ffffffff
1292
+L2 entry #11: 0x8000000000070000 00000000ffffffff
1293
+L2 entry #12: 0x8000000000080000 00000000ffffffff
1294
+L2 entry #13: 0x8000000000090000 00000000ffffffff
1295
+L2 entry #14: 0x80000000000a0000 00000000ffffffff
1296
+L2 entry #15: 0x80000000000b0000 00000000ffffffff
1297
+L2 entry #16: 0x80000000000c0000 00000000ffffffff
1298
+L2 entry #17: 0x80000000000d0000 00000000ffffffff
1299
+write -q -z -u 576k 192k
1300
+L2 entry #9: 0x0000000000000000 ffffffff00000000
1301
+L2 entry #10: 0x0000000000000000 ffffffff00000000
1302
+L2 entry #11: 0x0000000000000000 ffffffff00000000
1303
+write -q -z -u 800k 128k
1304
+L2 entry #12: 0x8000000000080000 ffff00000000ffff
1305
+L2 entry #13: 0x0000000000000000 ffffffff00000000
1306
+L2 entry #14: 0x80000000000a0000 0000ffffffff0000
1307
+write -q -z -u 991k 128k
1308
+L2 entry #15: 0x80000000000b0000 ffff00000000ffff
1309
+L2 entry #16: 0x0000000000000000 ffffffff00000000
1310
+L2 entry #17: 0x80000000000d0000 00007fffffff8000
1311
+
1312
+### Zero + unmap 2: compressed clusters (backing file: no) ###
1313
+
1314
+write -q -c -P PATTERN 1152k 64k
1315
+L2 entry #18: 0x4000000000050000 0000000000000000
1316
+write -q -c -P PATTERN 1216k 64k
1317
+L2 entry #19: 0x4000000000060000 0000000000000000
1318
+write -q -c -P PATTERN 1280k 64k
1319
+L2 entry #20: 0x4000000000070000 0000000000000000
1320
+write -q -c -P PATTERN 1344k 64k
1321
+L2 entry #21: 0x4000000000090000 0000000000000000
1322
+write -q -c -P PATTERN 1408k 64k
1323
+L2 entry #22: 0x40000000000c0000 0000000000000000
1324
+write -q -c -P PATTERN 1472k 64k
1325
+L2 entry #23: 0x40000000000e0000 0000000000000000
1326
+write -q -c -P PATTERN 1536k 64k
1327
+L2 entry #24: 0x40000000000f0000 0000000000000000
1328
+write -q -c -P PATTERN 1600k 64k
1329
+L2 entry #25: 0x4000000000100000 0000000000000000
1330
+write -q -c -P PATTERN 1664k 64k
1331
+L2 entry #26: 0x4000000000110000 0000000000000000
1332
+write -q -c -P PATTERN 1728k 64k
1333
+L2 entry #27: 0x4000000000120000 0000000000000000
1334
+write -q -c -P PATTERN 1792k 64k
1335
+L2 entry #28: 0x4000000000130000 0000000000000000
1336
+write -q -z -u 1152k 192k
1337
+L2 entry #18: 0x0000000000000000 ffffffff00000000
1338
+L2 entry #19: 0x0000000000000000 ffffffff00000000
1339
+L2 entry #20: 0x0000000000000000 ffffffff00000000
1340
+write -q -z -u 1376k 128k
1341
+L2 entry #21: 0x8000000000050000 00000000ffffffff
1342
+L2 entry #22: 0x8000000000060000 00000000ffffffff
1343
+L2 entry #23: 0x8000000000070000 00000000ffffffff
1344
+write -q -z -u 1567k 129k
1345
+L2 entry #24: 0x8000000000090000 00000000ffffffff
1346
+L2 entry #25: 0x80000000000e0000 00000000ffffffff
1347
+L2 entry #26: 0x80000000000f0000 00000000ffffffff
1348
+write -q -z -u 1759k 128k
1349
+L2 entry #27: 0x80000000000c0000 ffff00000000ffff
1350
+L2 entry #28: 0x0000000000000000 ffffffff00000000
1351
+L2 entry #29: 0x0000000000000000 0000ffff00000000
1352
+
1353
+### Discarding clusters with non-zero bitmaps (backing file: yes) ###
1354
+
1355
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw
1356
+L2 entry #0: 0x0000000000000000 ffffffff00000000
1357
+L2 entry #1: 0x0000000000000000 ffffffff00000000
1358
+Image resized.
1359
+Image resized.
1360
+L2 entry #0: 0x0000000000000000 ffffffff00000000
1361
+L2 entry #1: 0x0000000000000000 ffffffff00000000
1362
+
1363
+### Discarding clusters with non-zero bitmaps (backing file: no) ###
1364
+
1365
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1366
+L2 entry #0: 0x0000000000000000 ffffffff00000000
1367
+L2 entry #1: 0x0000000000000000 ffffffff00000000
1368
+Image resized.
1369
+Image resized.
1370
+L2 entry #0: 0x0000000000000000 0000ffff00000000
1371
+L2 entry #1: 0x0000000000000000 0000000000000000
1372
+
1373
+### Corrupted L2 entries - read test (allocated) ###
1374
+
1375
+# 'cluster is zero' bit set on the standard cluster descriptor
1376
+
1377
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1378
+L2 entry #0: 0x8000000000050001 0000000000000001
1379
+L2 entry #0: 0x8000000000050001 0000000000000001
1380
+
1381
+# Both 'subcluster is zero' and 'subcluster is allocated' bits set
1382
+
1383
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1384
+L2 entry #1: 0x8000000000060000 00000001ffffffff
1385
+qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0x1); further corruption events will be suppressed
1386
+read failed: Input/output error
1387
+
1388
+### Corrupted L2 entries - read test (unallocated) ###
1389
+
1390
+# 'cluster is zero' bit set on the standard cluster descriptor
1391
+
1392
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1393
+L2 entry #0: 0x0000000000000001 0000000000000000
1394
+L2 entry #0: 0x0000000000000001 0000000000000000
1395
+
1396
+# 'subcluster is allocated' bit set
1397
+
1398
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1399
+L2 entry #0: 0x0000000000000000 0000000000000001
1400
+qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0); further corruption events will be suppressed
1401
+read failed: Input/output error
1402
+
1403
+# Both 'subcluster is zero' and 'subcluster is allocated' bits set
1404
+
1405
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1406
+L2 entry #1: 0x0000000000000000 0000000100000001
1407
+qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0x1); further corruption events will be suppressed
1408
+read failed: Input/output error
1409
+
1410
+### Compressed cluster with subcluster bitmap != 0 - read test ###
1411
+
1412
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1413
+L2 entry #0: 0x4000000000050000 0000000180000000
1414
+read 65536/65536 bytes at offset 0
1415
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1416
+L2 entry #0: 0x4000000000050000 0000000180000000
1417
+
1418
+### Corrupted L2 entries - write test (allocated) ###
1419
+
1420
+# 'cluster is zero' bit set on the standard cluster descriptor
1421
+
1422
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1423
+L2 entry #0: 0x8000000000050001 0000000000000001
1424
+L2 entry #0: 0x8000000000050001 0000000000000001
1425
+
1426
+# Both 'subcluster is zero' and 'subcluster is allocated' bits set
1427
+
1428
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1429
+L2 entry #1: 0x8000000000060000 00000001ffffffff
1430
+qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0x1); further corruption events will be suppressed
1431
+write failed: Input/output error
1432
+
1433
+### Corrupted L2 entries - write test (unallocated) ###
1434
+
1435
+# 'cluster is zero' bit set on the standard cluster descriptor
1436
+
1437
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1438
+L2 entry #0: 0x0000000000000001 0000000000000000
1439
+L2 entry #0: 0x8000000000060000 0000000000000001
1440
+
1441
+# 'subcluster is allocated' bit set
1442
+
1443
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1444
+L2 entry #0: 0x0000000000000000 0000000000000001
1445
+qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0); further corruption events will be suppressed
1446
+write failed: Input/output error
1447
+
1448
+# Both 'subcluster is zero' and 'subcluster is allocated' bits set
1449
+
1450
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1451
+L2 entry #1: 0x0000000000000000 0000000100000001
1452
+qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0x1); further corruption events will be suppressed
1453
+write failed: Input/output error
1454
+
1455
+### Compressed cluster with subcluster bitmap != 0 - write test ###
1456
+
1457
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1458
+L2 entry #0: 0x4000000000050000 0000000180000000
1459
+wrote 65536/65536 bytes at offset 0
1460
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1461
+L2 entry #0: 0x8000000000060000 00000000ffffffff
1462
+
1463
+### Detect and repair unaligned clusters ###
1464
+
1465
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=raw size=131072
1466
+# Corrupted L2 entry, allocated subcluster #
1467
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=131072 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw
1468
+ERROR offset=50200: Data cluster is not properly aligned; L2 entry corrupted.
1469
+ERROR cluster 6 refcount=0 reference=1
1470
+Rebuilding refcount structure
1471
+ERROR offset=50200: Data cluster is not properly aligned; L2 entry corrupted.
1472
+Repairing cluster 1 refcount=1 reference=0
1473
+Repairing cluster 2 refcount=1 reference=0
1474
+ERROR offset=50200: Data cluster is not properly aligned; L2 entry corrupted.
1475
+The following inconsistencies were found and repaired:
1476
+
1477
+ 0 leaked clusters
1478
+ 1 corruptions
1479
+
1480
+Double checking the fixed image now...
1481
+
1482
+1 errors were found on the image.
1483
+Data may be corrupted, or further writes to the image may corrupt it.
1484
+qcow2: Marking image as corrupt: Cluster allocation offset 0x50200 unaligned (L2 offset: 0x40000, L2 index: 0); further corruption events will be suppressed
1485
+read failed: Input/output error
1486
+# Corrupted L2 entry, no allocated subclusters #
1487
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=131072 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw
1488
+Repairing offset=50200: Preallocated cluster is not properly aligned; L2 entry corrupted.
1489
+Leaked cluster 5 refcount=1 reference=0
1490
+Repairing cluster 5 refcount=1 reference=0
1491
+The following inconsistencies were found and repaired:
1492
+
1493
+ 1 leaked clusters
1494
+ 1 corruptions
1495
+
1496
+Double checking the fixed image now...
1497
+No errors were found on the image.
1498
+
1499
+### Image creation options ###
1500
+
1501
+# cluster_size < 16k
1502
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1503
+qemu-img: TEST_DIR/t.IMGFMT: Extended L2 entries are only supported with cluster sizes of at least 16384 bytes
1504
+# backing file and preallocation=metadata
1505
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=raw size=1048576
1506
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=524288 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw preallocation=metadata
1507
+Image resized.
1508
+read 524288/524288 bytes at offset 0
1509
+512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1510
+read 524288/524288 bytes at offset 524288
1511
+512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1512
+Offset Length Mapped to File
1513
+0 0x80000 0 TEST_DIR/t.qcow2.base
1514
+# backing file and preallocation=falloc
1515
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=524288 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw preallocation=falloc
1516
+Image resized.
1517
+read 524288/524288 bytes at offset 0
1518
+512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1519
+read 524288/524288 bytes at offset 524288
1520
+512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1521
+Offset Length Mapped to File
1522
+0 0x80000 0 TEST_DIR/t.qcow2.base
1523
+# backing file and preallocation=full
1524
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=524288 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw preallocation=full
1525
+Image resized.
1526
+read 524288/524288 bytes at offset 0
1527
+512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1528
+read 524288/524288 bytes at offset 524288
1529
+512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1530
+Offset Length Mapped to File
1531
+0 0x80000 0 TEST_DIR/t.qcow2.base
1532
+
1533
+### Image resizing with preallocation and backing files ###
1534
+
1535
+# resize --preallocation=metadata
1536
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=515072 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw
1537
+Image resized.
1538
+read 515072/515072 bytes at offset 0
1539
+503 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1540
+read 522240/522240 bytes at offset 515072
1541
+510 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1542
+# resize --preallocation=falloc
1543
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=515072 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw
1544
+Image resized.
1545
+read 515072/515072 bytes at offset 0
1546
+503 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1547
+read 522240/522240 bytes at offset 515072
1548
+510 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1549
+# resize --preallocation=full
1550
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=515072 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw
1551
+Image resized.
1552
+read 515072/515072 bytes at offset 0
1553
+503 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1554
+read 522240/522240 bytes at offset 515072
1555
+510 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1556
+
1557
+### Image resizing with preallocation without backing files ###
1558
+
1559
+# resize --preallocation=metadata
1560
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=515072
1561
+wrote 515072/515072 bytes at offset 0
1562
+503 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1563
+Image resized.
1564
+read 515072/515072 bytes at offset 0
1565
+503 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1566
+read 522240/522240 bytes at offset 515072
1567
+510 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1568
+# resize --preallocation=falloc
1569
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=515072
1570
+wrote 515072/515072 bytes at offset 0
1571
+503 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1572
+Image resized.
1573
+read 515072/515072 bytes at offset 0
1574
+503 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1575
+read 522240/522240 bytes at offset 515072
1576
+510 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1577
+# resize --preallocation=full
1578
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=515072
1579
+wrote 515072/515072 bytes at offset 0
1580
+503 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1581
+Image resized.
1582
+read 515072/515072 bytes at offset 0
1583
+503 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1584
+read 522240/522240 bytes at offset 515072
1585
+510 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1586
+
1587
+### qemu-img measure ###
1588
+
1589
+# 512MB, extended_l2=off
1590
+required size: 327680
1591
+fully allocated size: 537198592
1592
+# 512MB, extended_l2=on
1593
+required size: 393216
1594
+fully allocated size: 537264128
1595
+# 16K clusters, 64GB, extended_l2=off
1596
+required size: 42008576
1597
+fully allocated size: 68761485312
1598
+# 16K clusters, 64GB, extended_l2=on
1599
+required size: 75579392
1600
+fully allocated size: 68795056128
1601
+# 8k clusters
1602
+qemu-img: Extended L2 entries are only supported with cluster sizes of at least 16384 bytes
1603
+# 1024 TB
1604
+required size: 309285027840
1605
+fully allocated size: 1126209191870464
1606
+# 1025 TB
1607
+qemu-img: The image size is too large (try using a larger cluster size)
1608
+
1609
+### qemu-img amend ###
1610
+
1611
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1612
+qemu-img: Invalid parameter 'extended_l2'
1613
+This option is only supported for image creation
1614
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1615
+qemu-img: Invalid parameter 'extended_l2'
1616
+This option is only supported for image creation
1617
+
1618
+### Test copy-on-write on an image with snapshots ###
1619
+
1620
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1621
+L2 entry #0: 0x8000000000050000 0000008000042000
1622
+L2 entry #1: 0x8000000000060000 0000008000042000
1623
+L2 entry #2: 0x8000000000070000 0000008000042000
1624
+L2 entry #3: 0x8000000000080000 0000008000042000
1625
+L2 entry #4: 0x8000000000090000 0000008000042000
1626
+L2 entry #5: 0x80000000000a0000 0000008000042000
1627
+L2 entry #6: 0x80000000000b0000 0000008000042000
1628
+L2 entry #7: 0x80000000000c0000 0000008000042000
1629
+L2 entry #8: 0x80000000000d0000 0000008000042000
1630
+L2 entry #9: 0x80000000000e0000 0000008000042000
1631
+L2 entry #0: 0x8000000000120000 000000800007e000
1632
+L2 entry #1: 0x8000000000130000 000000800007fc00
1633
+L2 entry #2: 0x8000000000140000 00000080001fe000
1634
+L2 entry #3: 0x8000000000150000 000000800007e000
1635
+L2 entry #4: 0x8000000000160000 000000000007ff80
1636
+L2 entry #5: 0x8000000000170000 000000000007ffff
1637
+L2 entry #6: 0x00000000000b0000 0001808000042000
1638
+L2 entry #7: 0x00000000000c0000 0000208000040000
1639
+L2 entry #8: 0x8000000000180000 000000800007e000
1640
+L2 entry #9: 0x00000000000e0000 000000c000042000
1641
+
1642
+### Test concurrent requests ###
1643
+
1644
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
1645
+blkdebug: Suspended request 'A'
1646
+blkdebug: Resuming request 'A'
1647
+wrote 2048/2048 bytes at offset 30720
1648
+2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1649
+wrote 2048/2048 bytes at offset 20480
1650
+2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1651
+wrote 2048/2048 bytes at offset 40960
1652
+2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1653
+*** done
1654
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
1655
index XXXXXXX..XXXXXXX 100644
1656
--- a/tests/qemu-iotests/group
1657
+++ b/tests/qemu-iotests/group
1658
@@ -XXX,XX +XXX,XX @@
1659
267 rw auto quick snapshot
1660
268 rw auto quick
1661
270 rw backing quick
1662
+271 rw auto
1663
272 rw
1664
273 backing quick
1665
274 rw backing
1666
--
104
--
1667
2.26.2
105
2.34.1
1668
1669
diff view generated by jsdifflib
[PULL 33/34] qcow2: Assert that expand_zero_clusters_in_l1() does not support subclusters
[PULL 05/23] tests/qemu-iotests/040: Skip TestCommitWithFilters without 'throttle'
1
From: Alberto Garcia <berto@igalia.com>
1
From: Thomas Huth <thuth@redhat.com>
2
2
3
This function is only used by qcow2_expand_zero_clusters() to
3
iotest 040 already has some checks for the availability of the 'throttle'
4
downgrade a qcow2 image to a previous version. This would require
4
driver, but some new code has been added in the course of time that
5
transforming all extended L2 entries into normal L2 entries but this
5
depends on 'throttle' but does not check for its availability. Add
6
is not a simple task and there are no plans to implement this at the
6
a check to the TestCommitWithFilters class so that this iotest now
7
moment.
7
also passes again if 'throttle' has not been enabled in the QEMU
8
binaries.
8
9
9
Signed-off-by: Alberto Garcia <berto@igalia.com>
10
Signed-off-by: Thomas Huth <thuth@redhat.com>
10
Reviewed-by: Eric Blake <eblake@redhat.com>
11
Message-Id: <20220223123127.3206042-1-thuth@redhat.com>
11
Reviewed-by: Max Reitz <mreitz@redhat.com>
12
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
12
Message-Id: <15e65112b4144381b4d8c0bdf8fb76b0d813e3d1.1594396418.git.berto@igalia.com>
13
[mreitz: Fixed comment style]
14
Signed-off-by: Max Reitz <mreitz@redhat.com>
15
---
13
---
16
block/qcow2-cluster.c | 14 ++++++++++++--
14
tests/qemu-iotests/040 | 1 +
17
tests/qemu-iotests/061 | 6 ++++++
15
1 file changed, 1 insertion(+)
18
tests/qemu-iotests/061.out | 5 +++++
19
3 files changed, 23 insertions(+), 2 deletions(-)
20
16
21
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
17
diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040
22
index XXXXXXX..XXXXXXX 100644
23
--- a/block/qcow2-cluster.c
24
+++ b/block/qcow2-cluster.c
25
@@ -XXX,XX +XXX,XX @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table,
26
int ret;
27
int i, j;
28
29
+ /* qcow2_downgrade() is not allowed in images with subclusters */
30
+ assert(!has_subclusters(s));
31
+
32
slice_size2 = s->l2_slice_size * l2_entry_size(s);
33
n_slices = s->cluster_size / slice_size2;
34
35
@@ -XXX,XX +XXX,XX @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table,
36
37
if (cluster_type == QCOW2_CLUSTER_ZERO_PLAIN) {
38
if (!bs->backing) {
39
- /* not backed; therefore we can simply deallocate the
40
- * cluster */
41
+ /*
42
+ * not backed; therefore we can simply deallocate the
43
+ * cluster. No need to call set_l2_bitmap(), this
44
+ * function doesn't support images with subclusters.
45
+ */
46
set_l2_entry(s, l2_slice, j, 0);
47
l2_dirty = true;
48
continue;
49
@@ -XXX,XX +XXX,XX @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table,
50
} else {
51
set_l2_entry(s, l2_slice, j, offset);
52
}
53
+ /*
54
+ * No need to call set_l2_bitmap() after set_l2_entry() because
55
+ * this function doesn't support images with subclusters.
56
+ */
57
l2_dirty = true;
58
}
59
60
diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061
61
index XXXXXXX..XXXXXXX 100755
18
index XXXXXXX..XXXXXXX 100755
62
--- a/tests/qemu-iotests/061
19
--- a/tests/qemu-iotests/040
63
+++ b/tests/qemu-iotests/061
20
+++ b/tests/qemu-iotests/040
64
@@ -XXX,XX +XXX,XX @@ $QEMU_IMG amend -o "compat=0.10" "$TEST_IMG"
21
@@ -XXX,XX +XXX,XX @@ class TestCommitWithFilters(iotests.QMPTestCase):
65
_img_info --format-specific
22
pattern_file)
66
_check_test_img
23
self.assertFalse('Pattern verification failed' in result)
67
24
68
+echo
25
+ @iotests.skip_if_unsupported(['throttle'])
69
+echo "=== Testing version downgrade with extended L2 entries ==="
26
def setUp(self):
70
+echo
27
qemu_img('create', '-f', iotests.imgfmt, self.img0, '64M')
71
+_make_test_img -o "compat=1.1,extended_l2=on" 64M
28
qemu_img('create', '-f', iotests.imgfmt, self.img1, '64M')
72
+$QEMU_IMG amend -o "compat=0.10" "$TEST_IMG"
73
+
74
echo
75
echo "=== Try changing the external data file ==="
76
echo
77
diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out
78
index XXXXXXX..XXXXXXX 100644
79
--- a/tests/qemu-iotests/061.out
80
+++ b/tests/qemu-iotests/061.out
81
@@ -XXX,XX +XXX,XX @@ Format specific information:
82
extended l2: false
83
No errors were found on the image.
84
85
+=== Testing version downgrade with extended L2 entries ===
86
+
87
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
88
+qemu-img: Cannot downgrade an image with incompatible features 0x10 set
89
+
90
=== Try changing the external data file ===
91
92
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
93
--
29
--
94
2.26.2
30
2.34.1
95
96
diff view generated by jsdifflib
[PULL 30/34] qcow2: Add prealloc field to QCowL2Meta
[PULL 06/23] tests/qemu-iotests/testrunner: Quote "case not run" lines in TAP mode
1
From: Alberto Garcia <berto@igalia.com>
1
From: Thomas Huth <thuth@redhat.com>
2
2
3
This field allows us to indicate that the L2 metadata update does not
3
In TAP mode, the stdout is reserved for the TAP protocol, so we
4
come from a write request with actual data but from a preallocation
4
have to make sure to mark other lines with a comment '#' character
5
request.
5
at the beginning to avoid that the TAP parser at the other end
6
gets confused.
6
7
7
For traditional images this does not make any difference, but for
8
To test this condition, run "configure" for example with:
8
images with extended L2 entries this means that the clusters are
9
allocated normally in the L2 table but individual subclusters are
10
marked as unallocated.
11
9
12
This will allow preallocating images that have a backing file.
10
--block-drv-rw-whitelist=copy-before-write,qcow2,raw,file,host_device,blkdebug,null-co,copy-on-read
13
11
14
There is one special case: when we resize an existing image we can
12
so that iotest 041 will report that some tests are not run due to
15
also request that the new clusters are preallocated. If the image
13
the missing "quorum" driver. Without this change, "make check-block"
16
already had a backing file then we have to hide any possible stale
14
fails since the meson tap parser gets confused by these messages.
17
data and zero out the new clusters (see commit 955c7d6687 for more
18
details).
19
15
20
In this case the subclusters cannot be left as unallocated so the L2
16
Signed-off-by: Thomas Huth <thuth@redhat.com>
21
bitmap must be updated.
17
Message-Id: <20220223124353.3273898-1-thuth@redhat.com>
18
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
19
---
20
tests/qemu-iotests/testrunner.py | 5 ++++-
21
1 file changed, 4 insertions(+), 1 deletion(-)
22
22
23
Signed-off-by: Alberto Garcia <berto@igalia.com>
23
diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
24
Reviewed-by: Eric Blake <eblake@redhat.com>
25
Reviewed-by: Max Reitz <mreitz@redhat.com>
26
Message-Id: <960d4c444a4f5a870e2b47e5da322a73cd9a2f5a.1594396418.git.berto@igalia.com>
27
Signed-off-by: Max Reitz <mreitz@redhat.com>
28
---
29
block/qcow2.h | 8 ++++++++
30
block/qcow2-cluster.c | 2 +-
31
block/qcow2.c | 6 ++++++
32
3 files changed, 15 insertions(+), 1 deletion(-)
33
34
diff --git a/block/qcow2.h b/block/qcow2.h
35
index XXXXXXX..XXXXXXX 100644
24
index XXXXXXX..XXXXXXX 100644
36
--- a/block/qcow2.h
25
--- a/tests/qemu-iotests/testrunner.py
37
+++ b/block/qcow2.h
26
+++ b/tests/qemu-iotests/testrunner.py
38
@@ -XXX,XX +XXX,XX @@ typedef struct QCowL2Meta
27
@@ -XXX,XX +XXX,XX @@ def run_test(self, test: str,
39
*/
28
description=res.description)
40
bool skip_cow;
29
41
30
if res.casenotrun:
42
+ /**
31
- print(res.casenotrun)
43
+ * Indicates that this is not a normal write request but a preallocation.
32
+ if self.tap:
44
+ * If the image has extended L2 entries this means that no new individual
33
+ print('#' + res.casenotrun.replace('\n', '\n#'))
45
+ * subclusters will be marked as allocated in the L2 bitmap (but any
34
+ else:
46
+ * existing contents of that bitmap will be kept).
35
+ print(res.casenotrun)
47
+ */
36
48
+ bool prealloc;
37
return res
49
+
50
/**
51
* The I/O vector with the data from the actual guest write request.
52
* If non-NULL, this is meant to be merged together with the data
53
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
54
index XXXXXXX..XXXXXXX 100644
55
--- a/block/qcow2-cluster.c
56
+++ b/block/qcow2-cluster.c
57
@@ -XXX,XX +XXX,XX @@ int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m)
58
set_l2_entry(s, l2_slice, l2_index + i, offset | QCOW_OFLAG_COPIED);
59
60
/* Update bitmap with the subclusters that were just written */
61
- if (has_subclusters(s)) {
62
+ if (has_subclusters(s) && !m->prealloc) {
63
uint64_t l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index + i);
64
unsigned written_from = m->cow_start.offset;
65
unsigned written_to = m->cow_end.offset + m->cow_end.nb_bytes ?:
66
diff --git a/block/qcow2.c b/block/qcow2.c
67
index XXXXXXX..XXXXXXX 100644
68
--- a/block/qcow2.c
69
+++ b/block/qcow2.c
70
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_handle_l2meta(BlockDriverState *bs,
71
QCowL2Meta *next;
72
73
if (link_l2) {
74
+ assert(!l2meta->prealloc);
75
ret = qcow2_alloc_cluster_link_l2(bs, l2meta);
76
if (ret) {
77
goto out;
78
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn preallocate_co(BlockDriverState *bs, uint64_t offset,
79
80
while (meta) {
81
QCowL2Meta *next = meta->next;
82
+ meta->prealloc = true;
83
84
ret = qcow2_alloc_cluster_link_l2(bs, meta);
85
if (ret < 0) {
86
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset,
87
int64_t clusters_allocated;
88
int64_t old_file_size, last_cluster, new_file_size;
89
uint64_t nb_new_data_clusters, nb_new_l2_tables;
90
+ bool subclusters_need_allocation = false;
91
92
/* With a data file, preallocation means just allocating the metadata
93
* and forwarding the truncate request to the data file */
94
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset,
95
BDRV_REQ_ZERO_WRITE, NULL);
96
if (ret >= 0) {
97
flags &= ~BDRV_REQ_ZERO_WRITE;
98
+ /* Ensure that we read zeroes and not backing file data */
99
+ subclusters_need_allocation = true;
100
}
101
} else {
102
ret = -1;
103
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset,
104
.offset = nb_clusters << s->cluster_bits,
105
.nb_bytes = 0,
106
},
107
+ .prealloc = !subclusters_need_allocation,
108
};
109
qemu_co_queue_init(&allocation.dependent_requests);
110
38
111
--
39
--
112
2.26.2
40
2.34.1
113
114
diff view generated by jsdifflib
[PULL 16/34] qcow2: Add qcow2_cluster_is_allocated()
[PULL 07/23] iotests: Write test output to TEST_DIR
1
From: Alberto Garcia <berto@igalia.com>
1
Drop the use of OUTPUT_DIR (test/qemu-iotests under the build
2
directory), and instead write test output files (.out.bad, .notrun, and
3
.casenotrun) to TEST_DIR.
2
4
3
This helper function tells us if a cluster is allocated (that is,
5
With this, the same test can be run concurrently without the separate
4
there is an associated host offset for it).
6
instances interfering, because they will need separate TEST_DIRs anyway.
7
Running the same test separately is useful when running the iotests with
8
various format/protocol combinations in parallel, or when you just want
9
to aggressively exercise a single test (e.g. when it fails only
10
sporadically).
5
11
6
Signed-off-by: Alberto Garcia <berto@igalia.com>
12
Putting this output into TEST_DIR means that it will stick around for
7
Reviewed-by: Eric Blake <eblake@redhat.com>
13
inspection after the test run is done (though running the same test in
8
Reviewed-by: Max Reitz <mreitz@redhat.com>
14
the same TEST_DIR will overwrite it, just as it used to be); but given
9
Message-Id: <6d8771c5c79cbdc6c519875a5078e1cc85856d63.1594396418.git.berto@igalia.com>
15
that TEST_DIR is a scratch directory, it should be clear that users can
10
Signed-off-by: Max Reitz <mreitz@redhat.com>
16
delete all of its content at any point. (And if TEST_DIR is on tmpfs,
17
it will just disappear on shutdown.) Contrarily, alternative approaches
18
that would put these output files into OUTPUT_DIR with some prefix to
19
differentiate between separate test runs might easily lead to cluttering
20
OUTPUT_DIR.
21
22
(This change means OUTPUT_DIR is no longer written to by the iotests, so
23
we can drop its usage altogether.)
24
25
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
26
Message-Id: <20220221172909.762858-1-hreitz@redhat.com>
27
[hreitz: Simplified `Path(os.path.join(x, y))` to `Path(x, y)`, as
28
suggested by Vladimir; and rebased on 9086c7639822b6
29
("tests/qemu-iotests: Rework the checks and spots using GNU
30
sed")]
31
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
11
---
32
---
12
block/qcow2.h | 6 ++++++
33
tests/qemu-iotests/common.rc | 6 +++---
13
1 file changed, 6 insertions(+)
34
tests/qemu-iotests/iotests.py | 5 ++---
35
tests/qemu-iotests/testenv.py | 5 +----
36
tests/qemu-iotests/testrunner.py | 14 ++++++++------
37
4 files changed, 14 insertions(+), 16 deletions(-)
14
38
15
diff --git a/block/qcow2.h b/block/qcow2.h
39
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
16
index XXXXXXX..XXXXXXX 100644
40
index XXXXXXX..XXXXXXX 100644
17
--- a/block/qcow2.h
41
--- a/tests/qemu-iotests/common.rc
18
+++ b/block/qcow2.h
42
+++ b/tests/qemu-iotests/common.rc
19
@@ -XXX,XX +XXX,XX @@ QCow2SubclusterType qcow2_get_subcluster_type(BlockDriverState *bs,
43
@@ -XXX,XX +XXX,XX @@
20
}
44
# bail out, setting up .notrun file
45
_notrun()
46
{
47
- echo "$*" >"$OUTPUT_DIR/$seq.notrun"
48
+ echo "$*" >"$TEST_DIR/$seq.notrun"
49
echo "$seq not run: $*"
50
status=0
51
exit
52
@@ -XXX,XX +XXX,XX @@ _img_info()
53
#
54
_casenotrun()
55
{
56
- echo " [case not run] $*" >>"$OUTPUT_DIR/$seq.casenotrun"
57
+ echo " [case not run] $*" >>"$TEST_DIR/$seq.casenotrun"
21
}
58
}
22
59
23
+static inline bool qcow2_cluster_is_allocated(QCow2ClusterType type)
60
# just plain bail out
24
+{
61
#
25
+ return (type == QCOW2_CLUSTER_COMPRESSED || type == QCOW2_CLUSTER_NORMAL ||
62
_fail()
26
+ type == QCOW2_CLUSTER_ZERO_ALLOC);
63
{
27
+}
64
- echo "$*" | tee -a "$OUTPUT_DIR/$seq.full"
65
+ echo "$*" | tee -a "$TEST_DIR/$seq.full"
66
echo "(see $seq.full for details)"
67
status=1
68
exit 1
69
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
70
index XXXXXXX..XXXXXXX 100644
71
--- a/tests/qemu-iotests/iotests.py
72
+++ b/tests/qemu-iotests/iotests.py
73
@@ -XXX,XX +XXX,XX @@
74
75
imgfmt = os.environ.get('IMGFMT', 'raw')
76
imgproto = os.environ.get('IMGPROTO', 'file')
77
-output_dir = os.environ.get('OUTPUT_DIR', '.')
78
79
try:
80
test_dir = os.environ['TEST_DIR']
81
@@ -XXX,XX +XXX,XX @@ def notrun(reason):
82
# Each test in qemu-iotests has a number ("seq")
83
seq = os.path.basename(sys.argv[0])
84
85
- with open('%s/%s.notrun' % (output_dir, seq), 'w', encoding='utf-8') \
86
+ with open('%s/%s.notrun' % (test_dir, seq), 'w', encoding='utf-8') \
87
as outfile:
88
outfile.write(reason + '\n')
89
logger.warning("%s not run: %s", seq, reason)
90
@@ -XXX,XX +XXX,XX @@ def case_notrun(reason):
91
# Each test in qemu-iotests has a number ("seq")
92
seq = os.path.basename(sys.argv[0])
93
94
- with open('%s/%s.casenotrun' % (output_dir, seq), 'a', encoding='utf-8') \
95
+ with open('%s/%s.casenotrun' % (test_dir, seq), 'a', encoding='utf-8') \
96
as outfile:
97
outfile.write(' [case not run] ' + reason + '\n')
98
99
diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py
100
index XXXXXXX..XXXXXXX 100644
101
--- a/tests/qemu-iotests/testenv.py
102
+++ b/tests/qemu-iotests/testenv.py
103
@@ -XXX,XX +XXX,XX @@ class TestEnv(ContextManager['TestEnv']):
104
# pylint: disable=too-many-instance-attributes
105
106
env_variables = ['PYTHONPATH', 'TEST_DIR', 'SOCK_DIR', 'SAMPLE_IMG_DIR',
107
- 'OUTPUT_DIR', 'PYTHON', 'QEMU_PROG', 'QEMU_IMG_PROG',
108
+ 'PYTHON', 'QEMU_PROG', 'QEMU_IMG_PROG',
109
'QEMU_IO_PROG', 'QEMU_NBD_PROG', 'QSD_PROG',
110
'QEMU_OPTIONS', 'QEMU_IMG_OPTIONS',
111
'QEMU_IO_OPTIONS', 'QEMU_IO_OPTIONS_NO_FMT',
112
@@ -XXX,XX +XXX,XX @@ def init_directories(self) -> None:
113
TEST_DIR
114
SOCK_DIR
115
SAMPLE_IMG_DIR
116
- OUTPUT_DIR
117
"""
118
119
# Path where qemu goodies live in this source tree.
120
@@ -XXX,XX +XXX,XX @@ def init_directories(self) -> None:
121
os.path.join(self.source_iotests,
122
'sample_images'))
123
124
- self.output_dir = os.getcwd() # OUTPUT_DIR
125
-
126
def init_binaries(self) -> None:
127
"""Init binary path variables:
128
PYTHON (for bash tests)
129
diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
130
index XXXXXXX..XXXXXXX 100644
131
--- a/tests/qemu-iotests/testrunner.py
132
+++ b/tests/qemu-iotests/testrunner.py
133
@@ -XXX,XX +XXX,XX @@ def do_run_test(self, test: str, mp: bool) -> TestResult:
134
"""
135
136
f_test = Path(test)
137
- f_bad = Path(f_test.name + '.out.bad')
138
- f_notrun = Path(f_test.name + '.notrun')
139
- f_casenotrun = Path(f_test.name + '.casenotrun')
140
f_reference = Path(self.find_reference(test))
141
142
if not f_test.exists():
143
@@ -XXX,XX +XXX,XX @@ def do_run_test(self, test: str, mp: bool) -> TestResult:
144
description='No qualified output '
145
f'(expected {f_reference})')
146
147
- for p in (f_bad, f_notrun, f_casenotrun):
148
- silent_unlink(p)
149
-
150
args = [str(f_test.resolve())]
151
env = self.env.prepare_subprocess(args)
152
if mp:
153
@@ -XXX,XX +XXX,XX @@ def do_run_test(self, test: str, mp: bool) -> TestResult:
154
env[d] = os.path.join(env[d], f_test.name)
155
Path(env[d]).mkdir(parents=True, exist_ok=True)
156
157
+ test_dir = env['TEST_DIR']
158
+ f_bad = Path(test_dir, f_test.name + '.out.bad')
159
+ f_notrun = Path(test_dir, f_test.name + '.notrun')
160
+ f_casenotrun = Path(test_dir, f_test.name + '.casenotrun')
28
+
161
+
29
/* Check whether refcounts are eager or lazy */
162
+ for p in (f_notrun, f_casenotrun):
30
static inline bool qcow2_need_accurate_refcounts(BDRVQcow2State *s)
163
+ silent_unlink(p)
31
{
164
+
165
t0 = time.time()
166
with f_bad.open('w', encoding="utf-8") as f:
167
with subprocess.Popen(args, cwd=str(f_test.parent), env=env,
32
--
168
--
33
2.26.2
169
2.34.1
34
35
diff view generated by jsdifflib
[PULL 01/34] qcow2: Make Qcow2AioTask store the full host offset
[PULL 08/23] block/block-copy: move copy_bitmap initialization to block_copy_state_new()
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
The file_cluster_offset field of Qcow2AioTask stores a cluster-aligned
3
We are going to complicate bitmap initialization in the further
4
host offset. In practice this is not very useful because all users(*)
4
commit. And in future, backup job will be able to work without filter
5
of this structure need the final host offset into the cluster, which
5
(when source is immutable), so we'll need same bitmap initialization in
6
they calculate using
6
copy-before-write filter and in backup job. So, it's reasonable to do
7
it in block-copy.
7
8
8
host_offset = file_cluster_offset + offset_into_cluster(s, offset)
9
Note that for now cbw_open() is the only caller of
10
block_copy_state_new().
9
11
10
There is no reason why Qcow2AioTask cannot store host_offset directly
12
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
11
and that is what this patch does.
13
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
14
Message-Id: <20220303194349.2304213-2-vsementsov@virtuozzo.com>
15
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
16
---
17
block/block-copy.c | 1 +
18
block/copy-before-write.c | 4 ----
19
2 files changed, 1 insertion(+), 4 deletions(-)
12
20
13
(*) compressed clusters are the exception: in this case what
21
diff --git a/block/block-copy.c b/block/block-copy.c
14
file_cluster_offset was storing was the full compressed cluster
15
descriptor (offset + size). This does not change with this patch
16
but it is documented now.
17
18
Signed-off-by: Alberto Garcia <berto@igalia.com>
19
Reviewed-by: Eric Blake <eblake@redhat.com>
20
Reviewed-by: Max Reitz <mreitz@redhat.com>
21
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
22
Message-Id: <07c4b15c644dcf06c9459f98846ac1c4ea96e26f.1594396418.git.berto@igalia.com>
23
Signed-off-by: Max Reitz <mreitz@redhat.com>
24
---
25
block/qcow2.c | 69 ++++++++++++++++++++++------------------------
26
block/trace-events | 2 +-
27
2 files changed, 34 insertions(+), 37 deletions(-)
28
29
diff --git a/block/qcow2.c b/block/qcow2.c
30
index XXXXXXX..XXXXXXX 100644
22
index XXXXXXX..XXXXXXX 100644
31
--- a/block/qcow2.c
23
--- a/block/block-copy.c
32
+++ b/block/qcow2.c
24
+++ b/block/block-copy.c
33
@@ -XXX,XX +XXX,XX @@ typedef struct {
25
@@ -XXX,XX +XXX,XX @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
34
26
return NULL;
35
static int coroutine_fn
36
qcow2_co_preadv_compressed(BlockDriverState *bs,
37
- uint64_t file_cluster_offset,
38
+ uint64_t cluster_descriptor,
39
uint64_t offset,
40
uint64_t bytes,
41
QEMUIOVector *qiov,
42
@@ -XXX,XX +XXX,XX @@ out:
43
44
static coroutine_fn int
45
qcow2_co_preadv_encrypted(BlockDriverState *bs,
46
- uint64_t file_cluster_offset,
47
+ uint64_t host_offset,
48
uint64_t offset,
49
uint64_t bytes,
50
QEMUIOVector *qiov,
51
@@ -XXX,XX +XXX,XX @@ qcow2_co_preadv_encrypted(BlockDriverState *bs,
52
}
27
}
53
28
bdrv_disable_dirty_bitmap(copy_bitmap);
54
BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO);
29
+ bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap));
55
- ret = bdrv_co_pread(s->data_file,
30
56
- file_cluster_offset + offset_into_cluster(s, offset),
31
/*
57
- bytes, buf, 0);
32
* If source is in backing chain of target assume that target is going to be
58
+ ret = bdrv_co_pread(s->data_file, host_offset, bytes, buf, 0);
33
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
59
if (ret < 0) {
34
index XXXXXXX..XXXXXXX 100644
60
goto fail;
35
--- a/block/copy-before-write.c
36
+++ b/block/copy-before-write.c
37
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
38
Error **errp)
39
{
40
BDRVCopyBeforeWriteState *s = bs->opaque;
41
- BdrvDirtyBitmap *copy_bitmap;
42
43
bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
44
BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
45
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
46
return -EINVAL;
61
}
47
}
62
48
63
- if (qcow2_co_decrypt(bs,
49
- copy_bitmap = block_copy_dirty_bitmap(s->bcs);
64
- file_cluster_offset + offset_into_cluster(s, offset),
50
- bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap));
65
- offset, buf, bytes) < 0)
51
-
66
+ if (qcow2_co_decrypt(bs, host_offset, offset, buf, bytes) < 0)
52
return 0;
67
{
68
ret = -EIO;
69
goto fail;
70
@@ -XXX,XX +XXX,XX @@ typedef struct Qcow2AioTask {
71
72
BlockDriverState *bs;
73
QCow2ClusterType cluster_type; /* only for read */
74
- uint64_t file_cluster_offset;
75
+ uint64_t host_offset; /* or full descriptor in compressed clusters */
76
uint64_t offset;
77
uint64_t bytes;
78
QEMUIOVector *qiov;
79
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs,
80
AioTaskPool *pool,
81
AioTaskFunc func,
82
QCow2ClusterType cluster_type,
83
- uint64_t file_cluster_offset,
84
+ uint64_t host_offset,
85
uint64_t offset,
86
uint64_t bytes,
87
QEMUIOVector *qiov,
88
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs,
89
.bs = bs,
90
.cluster_type = cluster_type,
91
.qiov = qiov,
92
- .file_cluster_offset = file_cluster_offset,
93
+ .host_offset = host_offset,
94
.offset = offset,
95
.bytes = bytes,
96
.qiov_offset = qiov_offset,
97
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs,
98
99
trace_qcow2_add_task(qemu_coroutine_self(), bs, pool,
100
func == qcow2_co_preadv_task_entry ? "read" : "write",
101
- cluster_type, file_cluster_offset, offset, bytes,
102
+ cluster_type, host_offset, offset, bytes,
103
qiov, qiov_offset);
104
105
if (!pool) {
106
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs,
107
108
static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs,
109
QCow2ClusterType cluster_type,
110
- uint64_t file_cluster_offset,
111
+ uint64_t host_offset,
112
uint64_t offset, uint64_t bytes,
113
QEMUIOVector *qiov,
114
size_t qiov_offset)
115
{
116
BDRVQcow2State *s = bs->opaque;
117
- int offset_in_cluster = offset_into_cluster(s, offset);
118
119
switch (cluster_type) {
120
case QCOW2_CLUSTER_ZERO_PLAIN:
121
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs,
122
qiov, qiov_offset, 0);
123
124
case QCOW2_CLUSTER_COMPRESSED:
125
- return qcow2_co_preadv_compressed(bs, file_cluster_offset,
126
+ return qcow2_co_preadv_compressed(bs, host_offset,
127
offset, bytes, qiov, qiov_offset);
128
129
case QCOW2_CLUSTER_NORMAL:
130
- assert(offset_into_cluster(s, file_cluster_offset) == 0);
131
if (bs->encrypted) {
132
- return qcow2_co_preadv_encrypted(bs, file_cluster_offset,
133
+ return qcow2_co_preadv_encrypted(bs, host_offset,
134
offset, bytes, qiov, qiov_offset);
135
}
136
137
BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO);
138
- return bdrv_co_preadv_part(s->data_file,
139
- file_cluster_offset + offset_in_cluster,
140
+ return bdrv_co_preadv_part(s->data_file, host_offset,
141
bytes, qiov, qiov_offset, 0);
142
143
default:
144
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_task_entry(AioTask *task)
145
146
assert(!t->l2meta);
147
148
- return qcow2_co_preadv_task(t->bs, t->cluster_type, t->file_cluster_offset,
149
+ return qcow2_co_preadv_task(t->bs, t->cluster_type, t->host_offset,
150
t->offset, t->bytes, t->qiov, t->qiov_offset);
151
}
53
}
152
54
153
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
154
{
155
qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes);
156
} else {
157
+ /*
158
+ * For compressed clusters the variable cluster_offset
159
+ * does not actually store the offset but the full
160
+ * descriptor. We need to leave it unchanged because
161
+ * that's what qcow2_co_preadv_compressed() expects.
162
+ */
163
+ uint64_t host_offset = (ret == QCOW2_CLUSTER_COMPRESSED) ?
164
+ cluster_offset :
165
+ cluster_offset + offset_into_cluster(s, offset);
166
if (!aio && cur_bytes != bytes) {
167
aio = aio_task_pool_new(QCOW2_MAX_WORKERS);
168
}
169
ret = qcow2_add_task(bs, aio, qcow2_co_preadv_task_entry, ret,
170
- cluster_offset, offset, cur_bytes,
171
+ host_offset, offset, cur_bytes,
172
qiov, qiov_offset, NULL);
173
if (ret < 0) {
174
goto out;
175
@@ -XXX,XX +XXX,XX @@ static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta)
176
* not use it somehow after qcow2_co_pwritev_task() call
177
*/
178
static coroutine_fn int qcow2_co_pwritev_task(BlockDriverState *bs,
179
- uint64_t file_cluster_offset,
180
+ uint64_t host_offset,
181
uint64_t offset, uint64_t bytes,
182
QEMUIOVector *qiov,
183
uint64_t qiov_offset,
184
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev_task(BlockDriverState *bs,
185
int ret;
186
BDRVQcow2State *s = bs->opaque;
187
void *crypt_buf = NULL;
188
- int offset_in_cluster = offset_into_cluster(s, offset);
189
QEMUIOVector encrypted_qiov;
190
191
if (bs->encrypted) {
192
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev_task(BlockDriverState *bs,
193
}
194
qemu_iovec_to_buf(qiov, qiov_offset, crypt_buf, bytes);
195
196
- if (qcow2_co_encrypt(bs, file_cluster_offset + offset_in_cluster,
197
- offset, crypt_buf, bytes) < 0)
198
- {
199
+ if (qcow2_co_encrypt(bs, host_offset, offset, crypt_buf, bytes) < 0) {
200
ret = -EIO;
201
goto out_unlocked;
202
}
203
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev_task(BlockDriverState *bs,
204
*/
205
if (!merge_cow(offset, bytes, qiov, qiov_offset, l2meta)) {
206
BLKDBG_EVENT(bs->file, BLKDBG_WRITE_AIO);
207
- trace_qcow2_writev_data(qemu_coroutine_self(),
208
- file_cluster_offset + offset_in_cluster);
209
- ret = bdrv_co_pwritev_part(s->data_file,
210
- file_cluster_offset + offset_in_cluster,
211
+ trace_qcow2_writev_data(qemu_coroutine_self(), host_offset);
212
+ ret = bdrv_co_pwritev_part(s->data_file, host_offset,
213
bytes, qiov, qiov_offset, 0);
214
if (ret < 0) {
215
goto out_unlocked;
216
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev_task_entry(AioTask *task)
217
218
assert(!t->cluster_type);
219
220
- return qcow2_co_pwritev_task(t->bs, t->file_cluster_offset,
221
+ return qcow2_co_pwritev_task(t->bs, t->host_offset,
222
t->offset, t->bytes, t->qiov, t->qiov_offset,
223
t->l2meta);
224
}
225
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev_part(
226
aio = aio_task_pool_new(QCOW2_MAX_WORKERS);
227
}
228
ret = qcow2_add_task(bs, aio, qcow2_co_pwritev_task_entry, 0,
229
- cluster_offset, offset, cur_bytes,
230
- qiov, qiov_offset, l2meta);
231
+ cluster_offset + offset_in_cluster, offset,
232
+ cur_bytes, qiov, qiov_offset, l2meta);
233
l2meta = NULL; /* l2meta is consumed by qcow2_co_pwritev_task() */
234
if (ret < 0) {
235
goto fail_nometa;
236
@@ -XXX,XX +XXX,XX @@ qcow2_co_pwritev_compressed_part(BlockDriverState *bs,
237
238
static int coroutine_fn
239
qcow2_co_preadv_compressed(BlockDriverState *bs,
240
- uint64_t file_cluster_offset,
241
+ uint64_t cluster_descriptor,
242
uint64_t offset,
243
uint64_t bytes,
244
QEMUIOVector *qiov,
245
@@ -XXX,XX +XXX,XX @@ qcow2_co_preadv_compressed(BlockDriverState *bs,
246
uint8_t *buf, *out_buf;
247
int offset_in_cluster = offset_into_cluster(s, offset);
248
249
- coffset = file_cluster_offset & s->cluster_offset_mask;
250
- nb_csectors = ((file_cluster_offset >> s->csize_shift) & s->csize_mask) + 1;
251
+ coffset = cluster_descriptor & s->cluster_offset_mask;
252
+ nb_csectors = ((cluster_descriptor >> s->csize_shift) & s->csize_mask) + 1;
253
csize = nb_csectors * QCOW2_COMPRESSED_SECTOR_SIZE -
254
(coffset & ~QCOW2_COMPRESSED_SECTOR_MASK);
255
256
diff --git a/block/trace-events b/block/trace-events
257
index XXXXXXX..XXXXXXX 100644
258
--- a/block/trace-events
259
+++ b/block/trace-events
260
@@ -XXX,XX +XXX,XX @@ luring_io_uring_submit(void *s, int ret) "LuringState %p ret %d"
261
luring_resubmit_short_read(void *s, void *luringcb, int nread) "LuringState %p luringcb %p nread %d"
262
263
# qcow2.c
264
-qcow2_add_task(void *co, void *bs, void *pool, const char *action, int cluster_type, uint64_t file_cluster_offset, uint64_t offset, uint64_t bytes, void *qiov, size_t qiov_offset) "co %p bs %p pool %p: %s: cluster_type %d file_cluster_offset %" PRIu64 " offset %" PRIu64 " bytes %" PRIu64 " qiov %p qiov_offset %zu"
265
+qcow2_add_task(void *co, void *bs, void *pool, const char *action, int cluster_type, uint64_t host_offset, uint64_t offset, uint64_t bytes, void *qiov, size_t qiov_offset) "co %p bs %p pool %p: %s: cluster_type %d file_cluster_offset %" PRIu64 " offset %" PRIu64 " bytes %" PRIu64 " qiov %p qiov_offset %zu"
266
qcow2_writev_start_req(void *co, int64_t offset, int bytes) "co %p offset 0x%" PRIx64 " bytes %d"
267
qcow2_writev_done_req(void *co, int ret) "co %p ret %d"
268
qcow2_writev_start_part(void *co) "co %p"
269
--
55
--
270
2.26.2
56
2.34.1
271
272
diff view generated by jsdifflib
[PULL 02/34] qcow2: Convert qcow2_get_cluster_offset() into qcow2_get_host_offset()
Deleted patch
1
From: Alberto Garcia <berto@igalia.com>
2
1
3
qcow2_get_cluster_offset() takes an (unaligned) guest offset and
4
returns the (aligned) offset of the corresponding cluster in the qcow2
5
image.
6
7
In practice none of the callers need to know where the cluster starts
8
so this patch makes the function calculate and return the final host
9
offset directly. The function is also renamed accordingly.
10
11
There is a pre-existing exception with compressed clusters: in this
12
case the function returns the complete cluster descriptor (containing
13
the offset and size of the compressed data). This does not change with
14
this patch but it is now documented.
15
16
Signed-off-by: Alberto Garcia <berto@igalia.com>
17
Reviewed-by: Max Reitz <mreitz@redhat.com>
18
Message-Id: <ffae6cdc5ca8950e8280ac0f696dcc376cb07095.1594396418.git.berto@igalia.com>
19
Signed-off-by: Max Reitz <mreitz@redhat.com>
20
---
21
block/qcow2.h | 4 ++--
22
block/qcow2-cluster.c | 41 +++++++++++++++++++++++------------------
23
block/qcow2.c | 24 +++++++-----------------
24
3 files changed, 32 insertions(+), 37 deletions(-)
25
26
diff --git a/block/qcow2.h b/block/qcow2.h
27
index XXXXXXX..XXXXXXX 100644
28
--- a/block/qcow2.h
29
+++ b/block/qcow2.h
30
@@ -XXX,XX +XXX,XX @@ int qcow2_write_l1_entry(BlockDriverState *bs, int l1_index);
31
int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num,
32
uint8_t *buf, int nb_sectors, bool enc, Error **errp);
33
34
-int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset,
35
- unsigned int *bytes, uint64_t *cluster_offset);
36
+int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
37
+ unsigned int *bytes, uint64_t *host_offset);
38
int qcow2_alloc_cluster_offset(BlockDriverState *bs, uint64_t offset,
39
unsigned int *bytes, uint64_t *host_offset,
40
QCowL2Meta **m);
41
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
42
index XXXXXXX..XXXXXXX 100644
43
--- a/block/qcow2-cluster.c
44
+++ b/block/qcow2-cluster.c
45
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn do_perform_cow_write(BlockDriverState *bs,
46
47
48
/*
49
- * get_cluster_offset
50
+ * get_host_offset
51
*
52
- * For a given offset of the virtual disk, find the cluster type and offset in
53
- * the qcow2 file. The offset is stored in *cluster_offset.
54
+ * For a given offset of the virtual disk find the equivalent host
55
+ * offset in the qcow2 file and store it in *host_offset. Neither
56
+ * offset needs to be aligned to a cluster boundary.
57
+ *
58
+ * If the cluster is unallocated then *host_offset will be 0.
59
+ * If the cluster is compressed then *host_offset will contain the
60
+ * complete compressed cluster descriptor.
61
*
62
* On entry, *bytes is the maximum number of contiguous bytes starting at
63
* offset that we are interested in.
64
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn do_perform_cow_write(BlockDriverState *bs,
65
* Returns the cluster type (QCOW2_CLUSTER_*) on success, -errno in error
66
* cases.
67
*/
68
-int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset,
69
- unsigned int *bytes, uint64_t *cluster_offset)
70
+int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
71
+ unsigned int *bytes, uint64_t *host_offset)
72
{
73
BDRVQcow2State *s = bs->opaque;
74
unsigned int l2_index;
75
- uint64_t l1_index, l2_offset, *l2_slice;
76
+ uint64_t l1_index, l2_offset, *l2_slice, l2_entry;
77
int c;
78
unsigned int offset_in_cluster;
79
uint64_t bytes_available, bytes_needed, nb_clusters;
80
@@ -XXX,XX +XXX,XX @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset,
81
bytes_needed = bytes_available;
82
}
83
84
- *cluster_offset = 0;
85
+ *host_offset = 0;
86
87
/* seek to the l2 offset in the l1 table */
88
89
@@ -XXX,XX +XXX,XX @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset,
90
/* find the cluster offset for the given disk offset */
91
92
l2_index = offset_to_l2_slice_index(s, offset);
93
- *cluster_offset = be64_to_cpu(l2_slice[l2_index]);
94
+ l2_entry = be64_to_cpu(l2_slice[l2_index]);
95
96
nb_clusters = size_to_clusters(s, bytes_needed);
97
/* bytes_needed <= *bytes + offset_in_cluster, both of which are unsigned
98
@@ -XXX,XX +XXX,XX @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset,
99
* true */
100
assert(nb_clusters <= INT_MAX);
101
102
- type = qcow2_get_cluster_type(bs, *cluster_offset);
103
+ type = qcow2_get_cluster_type(bs, l2_entry);
104
if (s->qcow_version < 3 && (type == QCOW2_CLUSTER_ZERO_PLAIN ||
105
type == QCOW2_CLUSTER_ZERO_ALLOC)) {
106
qcow2_signal_corruption(bs, true, -1, -1, "Zero cluster entry found"
107
@@ -XXX,XX +XXX,XX @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset,
108
}
109
/* Compressed clusters can only be processed one by one */
110
c = 1;
111
- *cluster_offset &= L2E_COMPRESSED_OFFSET_SIZE_MASK;
112
+ *host_offset = l2_entry & L2E_COMPRESSED_OFFSET_SIZE_MASK;
113
break;
114
case QCOW2_CLUSTER_ZERO_PLAIN:
115
case QCOW2_CLUSTER_UNALLOCATED:
116
/* how many empty clusters ? */
117
c = count_contiguous_clusters_unallocated(bs, nb_clusters,
118
&l2_slice[l2_index], type);
119
- *cluster_offset = 0;
120
break;
121
case QCOW2_CLUSTER_ZERO_ALLOC:
122
- case QCOW2_CLUSTER_NORMAL:
123
+ case QCOW2_CLUSTER_NORMAL: {
124
+ uint64_t host_cluster_offset = l2_entry & L2E_OFFSET_MASK;
125
+ *host_offset = host_cluster_offset + offset_in_cluster;
126
/* how many allocated clusters ? */
127
c = count_contiguous_clusters(bs, nb_clusters, s->cluster_size,
128
&l2_slice[l2_index], QCOW_OFLAG_ZERO);
129
- *cluster_offset &= L2E_OFFSET_MASK;
130
- if (offset_into_cluster(s, *cluster_offset)) {
131
+ if (offset_into_cluster(s, host_cluster_offset)) {
132
qcow2_signal_corruption(bs, true, -1, -1,
133
"Cluster allocation offset %#"
134
PRIx64 " unaligned (L2 offset: %#" PRIx64
135
- ", L2 index: %#x)", *cluster_offset,
136
+ ", L2 index: %#x)", host_cluster_offset,
137
l2_offset, l2_index);
138
ret = -EIO;
139
goto fail;
140
}
141
- if (has_data_file(bs) && *cluster_offset != offset - offset_in_cluster)
142
- {
143
+ if (has_data_file(bs) && *host_offset != offset) {
144
qcow2_signal_corruption(bs, true, -1, -1,
145
"External data file host cluster offset %#"
146
PRIx64 " does not match guest cluster "
147
"offset: %#" PRIx64
148
- ", L2 index: %#x)", *cluster_offset,
149
+ ", L2 index: %#x)", host_cluster_offset,
150
offset - offset_in_cluster, l2_index);
151
ret = -EIO;
152
goto fail;
153
}
154
break;
155
+ }
156
default:
157
abort();
158
}
159
diff --git a/block/qcow2.c b/block/qcow2.c
160
index XXXXXXX..XXXXXXX 100644
161
--- a/block/qcow2.c
162
+++ b/block/qcow2.c
163
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,
164
BlockDriverState **file)
165
{
166
BDRVQcow2State *s = bs->opaque;
167
- uint64_t cluster_offset;
168
+ uint64_t host_offset;
169
unsigned int bytes;
170
int ret, status = 0;
171
172
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,
173
}
174
175
bytes = MIN(INT_MAX, count);
176
- ret = qcow2_get_cluster_offset(bs, offset, &bytes, &cluster_offset);
177
+ ret = qcow2_get_host_offset(bs, offset, &bytes, &host_offset);
178
qemu_co_mutex_unlock(&s->lock);
179
if (ret < 0) {
180
return ret;
181
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,
182
183
if ((ret == QCOW2_CLUSTER_NORMAL || ret == QCOW2_CLUSTER_ZERO_ALLOC) &&
184
!s->crypto) {
185
- *map = cluster_offset | offset_into_cluster(s, offset);
186
+ *map = host_offset;
187
*file = s->data_file->bs;
188
status |= BDRV_BLOCK_OFFSET_VALID;
189
}
190
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
191
BDRVQcow2State *s = bs->opaque;
192
int ret = 0;
193
unsigned int cur_bytes; /* number of bytes in current iteration */
194
- uint64_t cluster_offset = 0;
195
+ uint64_t host_offset = 0;
196
AioTaskPool *aio = NULL;
197
198
while (bytes != 0 && aio_task_pool_status(aio) == 0) {
199
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
200
}
201
202
qemu_co_mutex_lock(&s->lock);
203
- ret = qcow2_get_cluster_offset(bs, offset, &cur_bytes, &cluster_offset);
204
+ ret = qcow2_get_host_offset(bs, offset, &cur_bytes, &host_offset);
205
qemu_co_mutex_unlock(&s->lock);
206
if (ret < 0) {
207
goto out;
208
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
209
{
210
qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes);
211
} else {
212
- /*
213
- * For compressed clusters the variable cluster_offset
214
- * does not actually store the offset but the full
215
- * descriptor. We need to leave it unchanged because
216
- * that's what qcow2_co_preadv_compressed() expects.
217
- */
218
- uint64_t host_offset = (ret == QCOW2_CLUSTER_COMPRESSED) ?
219
- cluster_offset :
220
- cluster_offset + offset_into_cluster(s, offset);
221
if (!aio && cur_bytes != bytes) {
222
aio = aio_task_pool_new(QCOW2_MAX_WORKERS);
223
}
224
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs,
225
offset = QEMU_ALIGN_DOWN(offset, s->cluster_size);
226
bytes = s->cluster_size;
227
nr = s->cluster_size;
228
- ret = qcow2_get_cluster_offset(bs, offset, &nr, &off);
229
+ ret = qcow2_get_host_offset(bs, offset, &nr, &off);
230
if (ret != QCOW2_CLUSTER_UNALLOCATED &&
231
ret != QCOW2_CLUSTER_ZERO_PLAIN &&
232
ret != QCOW2_CLUSTER_ZERO_ALLOC) {
233
@@ -XXX,XX +XXX,XX @@ qcow2_co_copy_range_from(BlockDriverState *bs,
234
cur_bytes = MIN(bytes, INT_MAX);
235
cur_write_flags = write_flags;
236
237
- ret = qcow2_get_cluster_offset(bs, src_offset, &cur_bytes, &copy_offset);
238
+ ret = qcow2_get_host_offset(bs, src_offset, &cur_bytes, &copy_offset);
239
if (ret < 0) {
240
goto out;
241
}
242
@@ -XXX,XX +XXX,XX @@ qcow2_co_copy_range_from(BlockDriverState *bs,
243
244
case QCOW2_CLUSTER_NORMAL:
245
child = s->data_file;
246
- copy_offset += offset_into_cluster(s, src_offset);
247
break;
248
249
default:
250
--
251
2.26.2
252
253
diff view generated by jsdifflib
[PULL 03/34] qcow2: Add calculate_l2_meta()
Deleted patch
1
From: Alberto Garcia <berto@igalia.com>
2
1
3
handle_alloc() creates a QCowL2Meta structure in order to update the
4
image metadata and perform the necessary copy-on-write operations.
5
6
This patch moves that code to a separate function so it can be used
7
from other places.
8
9
Signed-off-by: Alberto Garcia <berto@igalia.com>
10
Reviewed-by: Max Reitz <mreitz@redhat.com>
11
Message-Id: <e5bc4a648dac31972bfa7a0e554be8064be78799.1594396418.git.berto@igalia.com>
12
Signed-off-by: Max Reitz <mreitz@redhat.com>
13
---
14
block/qcow2-cluster.c | 77 +++++++++++++++++++++++++++++--------------
15
1 file changed, 53 insertions(+), 24 deletions(-)
16
17
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
18
index XXXXXXX..XXXXXXX 100644
19
--- a/block/qcow2-cluster.c
20
+++ b/block/qcow2-cluster.c
21
@@ -XXX,XX +XXX,XX @@ void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m)
22
}
23
}
24
25
+/*
26
+ * For a given write request, create a new QCowL2Meta structure, add
27
+ * it to @m and the BDRVQcow2State.cluster_allocs list.
28
+ *
29
+ * @host_cluster_offset points to the beginning of the first cluster.
30
+ *
31
+ * @guest_offset and @bytes indicate the offset and length of the
32
+ * request.
33
+ *
34
+ * If @keep_old is true it means that the clusters were already
35
+ * allocated and will be overwritten. If false then the clusters are
36
+ * new and we have to decrease the reference count of the old ones.
37
+ */
38
+static void calculate_l2_meta(BlockDriverState *bs,
39
+ uint64_t host_cluster_offset,
40
+ uint64_t guest_offset, unsigned bytes,
41
+ QCowL2Meta **m, bool keep_old)
42
+{
43
+ BDRVQcow2State *s = bs->opaque;
44
+ unsigned cow_start_from = 0;
45
+ unsigned cow_start_to = offset_into_cluster(s, guest_offset);
46
+ unsigned cow_end_from = cow_start_to + bytes;
47
+ unsigned cow_end_to = ROUND_UP(cow_end_from, s->cluster_size);
48
+ unsigned nb_clusters = size_to_clusters(s, cow_end_from);
49
+ QCowL2Meta *old_m = *m;
50
+
51
+ *m = g_malloc0(sizeof(**m));
52
+ **m = (QCowL2Meta) {
53
+ .next = old_m,
54
+
55
+ .alloc_offset = host_cluster_offset,
56
+ .offset = start_of_cluster(s, guest_offset),
57
+ .nb_clusters = nb_clusters,
58
+
59
+ .keep_old_clusters = keep_old,
60
+
61
+ .cow_start = {
62
+ .offset = cow_start_from,
63
+ .nb_bytes = cow_start_to - cow_start_from,
64
+ },
65
+ .cow_end = {
66
+ .offset = cow_end_from,
67
+ .nb_bytes = cow_end_to - cow_end_from,
68
+ },
69
+ };
70
+
71
+ qemu_co_queue_init(&(*m)->dependent_requests);
72
+ QLIST_INSERT_HEAD(&s->cluster_allocs, *m, next_in_flight);
73
+}
74
+
75
/*
76
* Returns the number of contiguous clusters that can be used for an allocating
77
* write, but require COW to be performed (this includes yet unallocated space,
78
@@ -XXX,XX +XXX,XX @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,
79
uint64_t requested_bytes = *bytes + offset_into_cluster(s, guest_offset);
80
int avail_bytes = nb_clusters << s->cluster_bits;
81
int nb_bytes = MIN(requested_bytes, avail_bytes);
82
- QCowL2Meta *old_m = *m;
83
-
84
- *m = g_malloc0(sizeof(**m));
85
-
86
- **m = (QCowL2Meta) {
87
- .next = old_m,
88
-
89
- .alloc_offset = alloc_cluster_offset,
90
- .offset = start_of_cluster(s, guest_offset),
91
- .nb_clusters = nb_clusters,
92
-
93
- .keep_old_clusters = keep_old_clusters,
94
-
95
- .cow_start = {
96
- .offset = 0,
97
- .nb_bytes = offset_into_cluster(s, guest_offset),
98
- },
99
- .cow_end = {
100
- .offset = nb_bytes,
101
- .nb_bytes = avail_bytes - nb_bytes,
102
- },
103
- };
104
- qemu_co_queue_init(&(*m)->dependent_requests);
105
- QLIST_INSERT_HEAD(&s->cluster_allocs, *m, next_in_flight);
106
107
*host_offset = alloc_cluster_offset + offset_into_cluster(s, guest_offset);
108
*bytes = MIN(*bytes, nb_bytes - offset_into_cluster(s, guest_offset));
109
assert(*bytes != 0);
110
111
+ calculate_l2_meta(bs, alloc_cluster_offset, guest_offset, *bytes,
112
+ m, keep_old_clusters);
113
+
114
return 1;
115
116
fail:
117
--
118
2.26.2
119
120
diff view generated by jsdifflib
[PULL 17/34] qcow2: Add cluster type parameter to qcow2_get_host_offset()
[PULL 09/23] block/dirty-bitmap: bdrv_merge_dirty_bitmap(): add return value
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
This function returns an integer that can be either an error code or a
3
That simplifies handling failure in existing code and in further new
4
cluster type (a value from the QCow2ClusterType enum).
4
usage of bdrv_merge_dirty_bitmap().
5
5
6
We are going to start using subcluster types instead of cluster types
6
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
7
in some functions so it's better to use the exact data types instead
7
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
8
of integers for clarity and in order to detect errors more easily.
8
Message-Id: <20220303194349.2304213-3-vsementsov@virtuozzo.com>
9
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
10
---
11
include/block/dirty-bitmap.h | 2 +-
12
block/dirty-bitmap.c | 9 +++++++--
13
block/monitor/bitmap-qmp-cmds.c | 5 +----
14
3 files changed, 9 insertions(+), 7 deletions(-)
9
15
10
This patch makes qcow2_get_host_offset() return 0 on success and
16
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
11
puts the returned cluster type in a separate parameter. There are no
12
semantic changes.
13
14
Signed-off-by: Alberto Garcia <berto@igalia.com>
15
Reviewed-by: Max Reitz <mreitz@redhat.com>
16
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
17
Message-Id: <396b6eab1859a271551dcd7dcba77f8934aa3c3f.1594396418.git.berto@igalia.com>
18
Signed-off-by: Max Reitz <mreitz@redhat.com>
19
---
20
block/qcow2.h | 3 ++-
21
block/qcow2-cluster.c | 11 +++++++----
22
block/qcow2.c | 37 ++++++++++++++++++++++---------------
23
3 files changed, 31 insertions(+), 20 deletions(-)
24
25
diff --git a/block/qcow2.h b/block/qcow2.h
26
index XXXXXXX..XXXXXXX 100644
17
index XXXXXXX..XXXXXXX 100644
27
--- a/block/qcow2.h
18
--- a/include/block/dirty-bitmap.h
28
+++ b/block/qcow2.h
19
+++ b/include/block/dirty-bitmap.h
29
@@ -XXX,XX +XXX,XX @@ int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num,
20
@@ -XXX,XX +XXX,XX @@ void bdrv_dirty_bitmap_set_persistence(BdrvDirtyBitmap *bitmap,
30
uint8_t *buf, int nb_sectors, bool enc, Error **errp);
21
bool persistent);
31
22
void bdrv_dirty_bitmap_set_inconsistent(BdrvDirtyBitmap *bitmap);
32
int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
23
void bdrv_dirty_bitmap_set_busy(BdrvDirtyBitmap *bitmap, bool busy);
33
- unsigned int *bytes, uint64_t *host_offset);
24
-void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src,
34
+ unsigned int *bytes, uint64_t *host_offset,
25
+bool bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src,
35
+ QCow2ClusterType *cluster_type);
26
HBitmap **backup, Error **errp);
36
int qcow2_alloc_cluster_offset(BlockDriverState *bs, uint64_t offset,
27
void bdrv_dirty_bitmap_skip_store(BdrvDirtyBitmap *bitmap, bool skip);
37
unsigned int *bytes, uint64_t *host_offset,
28
bool bdrv_dirty_bitmap_get(BdrvDirtyBitmap *bitmap, int64_t offset);
38
QCowL2Meta **m);
29
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
39
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
40
index XXXXXXX..XXXXXXX 100644
30
index XXXXXXX..XXXXXXX 100644
41
--- a/block/qcow2-cluster.c
31
--- a/block/dirty-bitmap.c
42
+++ b/block/qcow2-cluster.c
32
+++ b/block/dirty-bitmap.c
43
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn do_perform_cow_write(BlockDriverState *bs,
33
@@ -XXX,XX +XXX,XX @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
34
* Ensures permissions on bitmaps are reasonable; use for public API.
44
*
35
*
45
* On exit, *bytes is the number of bytes starting at offset that have the same
36
* @backup: If provided, make a copy of dest here prior to merge.
46
* cluster type and (if applicable) are stored contiguously in the image file.
37
+ *
47
+ * The cluster type is stored in *cluster_type.
38
+ * Returns true on success, false on failure. In case of failure bitmaps are
48
* Compressed clusters are always returned one by one.
39
+ * untouched.
49
*
50
- * Returns the cluster type (QCOW2_CLUSTER_*) on success, -errno in error
51
- * cases.
52
+ * Returns 0 on success, -errno in error cases.
53
*/
40
*/
54
int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
41
-void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src,
55
- unsigned int *bytes, uint64_t *host_offset)
42
+bool bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src,
56
+ unsigned int *bytes, uint64_t *host_offset,
43
HBitmap **backup, Error **errp)
57
+ QCow2ClusterType *cluster_type)
58
{
44
{
59
BDRVQcow2State *s = bs->opaque;
45
- bool ret;
60
unsigned int l2_index;
46
+ bool ret = false;
47
48
bdrv_dirty_bitmaps_lock(dest->bs);
49
if (src->bs != dest->bs) {
61
@@ -XXX,XX +XXX,XX @@ out:
50
@@ -XXX,XX +XXX,XX @@ out:
62
assert(bytes_available - offset_in_cluster <= UINT_MAX);
51
if (src->bs != dest->bs) {
63
*bytes = bytes_available - offset_in_cluster;
52
bdrv_dirty_bitmaps_unlock(src->bs);
64
53
}
65
- return type;
66
+ *cluster_type = type;
67
+
54
+
68
+ return 0;
55
+ return ret;
69
56
}
70
fail:
57
71
qcow2_cache_put(s->l2_table_cache, (void **)&l2_slice);
58
/**
72
diff --git a/block/qcow2.c b/block/qcow2.c
59
diff --git a/block/monitor/bitmap-qmp-cmds.c b/block/monitor/bitmap-qmp-cmds.c
73
index XXXXXXX..XXXXXXX 100644
60
index XXXXXXX..XXXXXXX 100644
74
--- a/block/qcow2.c
61
--- a/block/monitor/bitmap-qmp-cmds.c
75
+++ b/block/qcow2.c
62
+++ b/block/monitor/bitmap-qmp-cmds.c
76
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,
63
@@ -XXX,XX +XXX,XX @@ BdrvDirtyBitmap *block_dirty_bitmap_merge(const char *node, const char *target,
77
BDRVQcow2State *s = bs->opaque;
64
BlockDriverState *bs;
78
uint64_t host_offset;
65
BdrvDirtyBitmap *dst, *src, *anon;
79
unsigned int bytes;
66
BlockDirtyBitmapMergeSourceList *lst;
80
+ QCow2ClusterType type;
67
- Error *local_err = NULL;
81
int ret, status = 0;
68
82
69
GLOBAL_STATE_CODE();
83
qemu_co_mutex_lock(&s->lock);
70
84
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,
71
@@ -XXX,XX +XXX,XX @@ BdrvDirtyBitmap *block_dirty_bitmap_merge(const char *node, const char *target,
85
}
72
abort();
86
87
bytes = MIN(INT_MAX, count);
88
- ret = qcow2_get_host_offset(bs, offset, &bytes, &host_offset);
89
+ ret = qcow2_get_host_offset(bs, offset, &bytes, &host_offset, &type);
90
qemu_co_mutex_unlock(&s->lock);
91
if (ret < 0) {
92
return ret;
93
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,
94
95
*pnum = bytes;
96
97
- if ((ret == QCOW2_CLUSTER_NORMAL || ret == QCOW2_CLUSTER_ZERO_ALLOC) &&
98
+ if ((type == QCOW2_CLUSTER_NORMAL || type == QCOW2_CLUSTER_ZERO_ALLOC) &&
99
!s->crypto) {
100
*map = host_offset;
101
*file = s->data_file->bs;
102
status |= BDRV_BLOCK_OFFSET_VALID;
103
}
104
- if (ret == QCOW2_CLUSTER_ZERO_PLAIN || ret == QCOW2_CLUSTER_ZERO_ALLOC) {
105
+ if (type == QCOW2_CLUSTER_ZERO_PLAIN || type == QCOW2_CLUSTER_ZERO_ALLOC) {
106
status |= BDRV_BLOCK_ZERO;
107
- } else if (ret != QCOW2_CLUSTER_UNALLOCATED) {
108
+ } else if (type != QCOW2_CLUSTER_UNALLOCATED) {
109
status |= BDRV_BLOCK_DATA;
110
}
111
if (s->metadata_preallocation && (status & BDRV_BLOCK_DATA) &&
112
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
113
int ret = 0;
114
unsigned int cur_bytes; /* number of bytes in current iteration */
115
uint64_t host_offset = 0;
116
+ QCow2ClusterType type;
117
AioTaskPool *aio = NULL;
118
119
while (bytes != 0 && aio_task_pool_status(aio) == 0) {
120
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
121
}
73
}
122
74
123
qemu_co_mutex_lock(&s->lock);
75
- bdrv_merge_dirty_bitmap(anon, src, NULL, &local_err);
124
- ret = qcow2_get_host_offset(bs, offset, &cur_bytes, &host_offset);
76
- if (local_err) {
125
+ ret = qcow2_get_host_offset(bs, offset, &cur_bytes,
77
- error_propagate(errp, local_err);
126
+ &host_offset, &type);
78
+ if (!bdrv_merge_dirty_bitmap(anon, src, NULL, errp)) {
127
qemu_co_mutex_unlock(&s->lock);
79
dst = NULL;
128
if (ret < 0) {
129
goto out;
80
goto out;
130
}
81
}
131
132
- if (ret == QCOW2_CLUSTER_ZERO_PLAIN ||
133
- ret == QCOW2_CLUSTER_ZERO_ALLOC ||
134
- (ret == QCOW2_CLUSTER_UNALLOCATED && !bs->backing))
135
+ if (type == QCOW2_CLUSTER_ZERO_PLAIN ||
136
+ type == QCOW2_CLUSTER_ZERO_ALLOC ||
137
+ (type == QCOW2_CLUSTER_UNALLOCATED && !bs->backing))
138
{
139
qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes);
140
} else {
141
if (!aio && cur_bytes != bytes) {
142
aio = aio_task_pool_new(QCOW2_MAX_WORKERS);
143
}
144
- ret = qcow2_add_task(bs, aio, qcow2_co_preadv_task_entry, ret,
145
+ ret = qcow2_add_task(bs, aio, qcow2_co_preadv_task_entry, type,
146
host_offset, offset, cur_bytes,
147
qiov, qiov_offset, NULL);
148
if (ret < 0) {
149
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs,
150
if (head || tail) {
151
uint64_t off;
152
unsigned int nr;
153
+ QCow2ClusterType type;
154
155
assert(head + bytes <= s->cluster_size);
156
157
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs,
158
offset = QEMU_ALIGN_DOWN(offset, s->cluster_size);
159
bytes = s->cluster_size;
160
nr = s->cluster_size;
161
- ret = qcow2_get_host_offset(bs, offset, &nr, &off);
162
- if (ret != QCOW2_CLUSTER_UNALLOCATED &&
163
- ret != QCOW2_CLUSTER_ZERO_PLAIN &&
164
- ret != QCOW2_CLUSTER_ZERO_ALLOC) {
165
+ ret = qcow2_get_host_offset(bs, offset, &nr, &off, &type);
166
+ if (ret < 0 ||
167
+ (type != QCOW2_CLUSTER_UNALLOCATED &&
168
+ type != QCOW2_CLUSTER_ZERO_PLAIN &&
169
+ type != QCOW2_CLUSTER_ZERO_ALLOC)) {
170
qemu_co_mutex_unlock(&s->lock);
171
return -ENOTSUP;
172
}
173
@@ -XXX,XX +XXX,XX @@ qcow2_co_copy_range_from(BlockDriverState *bs,
174
175
while (bytes != 0) {
176
uint64_t copy_offset = 0;
177
+ QCow2ClusterType type;
178
/* prepare next request */
179
cur_bytes = MIN(bytes, INT_MAX);
180
cur_write_flags = write_flags;
181
182
- ret = qcow2_get_host_offset(bs, src_offset, &cur_bytes, &copy_offset);
183
+ ret = qcow2_get_host_offset(bs, src_offset, &cur_bytes,
184
+ &copy_offset, &type);
185
if (ret < 0) {
186
goto out;
187
}
188
189
- switch (ret) {
190
+ switch (type) {
191
case QCOW2_CLUSTER_UNALLOCATED:
192
if (bs->backing && bs->backing->bs) {
193
int64_t backing_length = bdrv_getlength(bs->backing->bs);
194
--
82
--
195
2.26.2
83
2.34.1
196
197
diff view generated by jsdifflib
[PULL 29/34] qcow2: Add subcluster support to qcow2_measure()
[PULL 10/23] block/block-copy: block_copy_state_new(): add bitmap parameter
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
Extended L2 entries are bigger than normal L2 entries so this has an
3
This will be used in the following commit to bring "incremental" mode
4
impact on the amount of metadata needed for a qcow2 file.
4
to copy-before-write filter.
5
5
6
Signed-off-by: Alberto Garcia <berto@igalia.com>
6
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
7
Reviewed-by: Max Reitz <mreitz@redhat.com>
7
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
8
Message-Id: <7efae2efd5e36b42d2570743a12576d68ce53685.1594396418.git.berto@igalia.com>
8
Message-Id: <20220303194349.2304213-4-vsementsov@virtuozzo.com>
9
Signed-off-by: Max Reitz <mreitz@redhat.com>
9
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
10
---
10
---
11
block/qcow2.c | 20 +++++++++++++-------
11
include/block/block-copy.h | 1 +
12
1 file changed, 13 insertions(+), 7 deletions(-)
12
block/block-copy.c | 14 +++++++++++++-
13
block/copy-before-write.c | 2 +-
14
3 files changed, 15 insertions(+), 2 deletions(-)
13
15
14
diff --git a/block/qcow2.c b/block/qcow2.c
16
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
15
index XXXXXXX..XXXXXXX 100644
17
index XXXXXXX..XXXXXXX 100644
16
--- a/block/qcow2.c
18
--- a/include/block/block-copy.h
17
+++ b/block/qcow2.c
19
+++ b/include/block/block-copy.h
18
@@ -XXX,XX +XXX,XX @@ int64_t qcow2_refcount_metadata_size(int64_t clusters, size_t cluster_size,
20
@@ -XXX,XX +XXX,XX @@ typedef struct BlockCopyState BlockCopyState;
19
* @total_size: virtual disk size in bytes
21
typedef struct BlockCopyCallState BlockCopyCallState;
20
* @cluster_size: cluster size in bytes
22
21
* @refcount_order: refcount bits power-of-2 exponent
23
BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
22
+ * @extended_l2: true if the image has extended L2 entries
24
+ const BdrvDirtyBitmap *bitmap,
23
*
25
Error **errp);
24
* Returns: Total number of bytes required for the fully allocated image
26
25
* (including metadata).
27
/* Function should be called prior any actual copy request */
26
*/
28
diff --git a/block/block-copy.c b/block/block-copy.c
27
static int64_t qcow2_calc_prealloc_size(int64_t total_size,
29
index XXXXXXX..XXXXXXX 100644
28
size_t cluster_size,
30
--- a/block/block-copy.c
29
- int refcount_order)
31
+++ b/block/block-copy.c
30
+ int refcount_order,
32
@@ -XXX,XX +XXX,XX @@ static int64_t block_copy_calculate_cluster_size(BlockDriverState *target,
31
+ bool extended_l2)
33
}
34
35
BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
36
+ const BdrvDirtyBitmap *bitmap,
37
Error **errp)
32
{
38
{
33
int64_t meta_size = 0;
39
+ ERRP_GUARD();
34
uint64_t nl1e, nl2e;
40
BlockCopyState *s;
35
int64_t aligned_total_size = ROUND_UP(total_size, cluster_size);
41
int64_t cluster_size;
36
+ size_t l2e_size = extended_l2 ? L2E_SIZE_EXTENDED : L2E_SIZE_NORMAL;
42
BdrvDirtyBitmap *copy_bitmap;
37
43
@@ -XXX,XX +XXX,XX @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
38
/* header: 1 cluster */
44
return NULL;
39
meta_size += cluster_size;
40
41
/* total size of L2 tables */
42
nl2e = aligned_total_size / cluster_size;
43
- nl2e = ROUND_UP(nl2e, cluster_size / sizeof(uint64_t));
44
- meta_size += nl2e * sizeof(uint64_t);
45
+ nl2e = ROUND_UP(nl2e, cluster_size / l2e_size);
46
+ meta_size += nl2e * l2e_size;
47
48
/* total size of L1 tables */
49
- nl1e = nl2e * sizeof(uint64_t) / cluster_size;
50
+ nl1e = nl2e * l2e_size / cluster_size;
51
nl1e = ROUND_UP(nl1e, cluster_size / sizeof(uint64_t));
52
meta_size += nl1e * sizeof(uint64_t);
53
54
@@ -XXX,XX +XXX,XX @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs,
55
PreallocMode prealloc;
56
bool has_backing_file;
57
bool has_luks;
58
+ bool extended_l2 = false; /* Set to false until the option is added */
59
+ size_t l2e_size;
60
61
/* Parse image creation options */
62
cluster_size = qcow2_opt_get_cluster_size_del(opts, &local_err);
63
@@ -XXX,XX +XXX,XX @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs,
64
virtual_size = ROUND_UP(virtual_size, cluster_size);
65
66
/* Check that virtual disk size is valid */
67
+ l2e_size = extended_l2 ? L2E_SIZE_EXTENDED : L2E_SIZE_NORMAL;
68
l2_tables = DIV_ROUND_UP(virtual_size / cluster_size,
69
- cluster_size / sizeof(uint64_t));
70
+ cluster_size / l2e_size);
71
if (l2_tables * sizeof(uint64_t) > QCOW_MAX_L1_SIZE) {
72
error_setg(&local_err, "The image size is too large "
73
"(try using a larger cluster size)");
74
@@ -XXX,XX +XXX,XX @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs,
75
}
45
}
76
46
bdrv_disable_dirty_bitmap(copy_bitmap);
77
info = g_new0(BlockMeasureInfo, 1);
47
- bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap));
78
- info->fully_allocated =
48
+ if (bitmap) {
79
+ info->fully_allocated = luks_payload_size +
49
+ if (!bdrv_merge_dirty_bitmap(copy_bitmap, bitmap, NULL, errp)) {
80
qcow2_calc_prealloc_size(virtual_size, cluster_size,
50
+ error_prepend(errp, "Failed to merge bitmap '%s' to internal "
81
- ctz32(refcount_bits)) + luks_payload_size;
51
+ "copy-bitmap: ", bdrv_dirty_bitmap_name(bitmap));
82
+ ctz32(refcount_bits), extended_l2);
52
+ bdrv_release_dirty_bitmap(copy_bitmap);
53
+ return NULL;
54
+ }
55
+ } else {
56
+ bdrv_set_dirty_bitmap(copy_bitmap, 0,
57
+ bdrv_dirty_bitmap_size(copy_bitmap));
58
+ }
83
59
84
/*
60
/*
85
* Remove data clusters that are not required. This overestimates the
61
* If source is in backing chain of target assume that target is going to be
62
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
63
index XXXXXXX..XXXXXXX 100644
64
--- a/block/copy-before-write.c
65
+++ b/block/copy-before-write.c
66
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
67
((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
68
bs->file->bs->supported_zero_flags);
69
70
- s->bcs = block_copy_state_new(bs->file, s->target, errp);
71
+ s->bcs = block_copy_state_new(bs->file, s->target, NULL, errp);
72
if (!s->bcs) {
73
error_prepend(errp, "Cannot create block-copy-state: ");
74
return -EINVAL;
86
--
75
--
87
2.26.2
76
2.34.1
88
89
diff view generated by jsdifflib
[PULL 31/34] qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit
[PULL 11/23] block/copy-before-write: add bitmap open parameter
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
Now that the implementation of subclusters is complete we can finally
3
This brings "incremental" mode to copy-before-write filter: user can
4
add the necessary options to create and read images with this feature,
4
specify bitmap so that filter will copy only "dirty" areas.
5
which we call "extended L2 entries".
6
5
7
Signed-off-by: Alberto Garcia <berto@igalia.com>
6
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
8
Reviewed-by: Eric Blake <eblake@redhat.com>
7
Message-Id: <20220303194349.2304213-5-vsementsov@virtuozzo.com>
9
Reviewed-by: Max Reitz <mreitz@redhat.com>
8
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
10
Message-Id: <6476caaa73216bd05b7bb2d504a20415e1665176.1594396418.git.berto@igalia.com>
11
[mreitz: %s/5\.1/5.2/; fixed 302's and 303's reference output]
12
Signed-off-by: Max Reitz <mreitz@redhat.com>
13
---
9
---
14
qapi/block-core.json | 7 +++
10
qapi/block-core.json | 10 +++++++-
15
block/qcow2.h | 8 ++-
11
block/copy-before-write.c | 51 ++++++++++++++++++++++++++++++++++++++-
16
include/block/block_int.h | 1 +
12
2 files changed, 59 insertions(+), 2 deletions(-)
17
block/qcow2.c | 66 ++++++++++++++++++--
18
tests/qemu-iotests/031.out | 8 +--
19
tests/qemu-iotests/036.out | 4 +-
20
tests/qemu-iotests/049.out | 102 +++++++++++++++----------------
21
tests/qemu-iotests/060.out | 1 +
22
tests/qemu-iotests/061.out | 20 +++---
23
tests/qemu-iotests/065 | 12 ++--
24
tests/qemu-iotests/082.out | 39 +++++++++---
25
tests/qemu-iotests/085.out | 38 ++++++------
26
tests/qemu-iotests/144.out | 4 +-
27
tests/qemu-iotests/182.out | 2 +-
28
tests/qemu-iotests/185.out | 8 +--
29
tests/qemu-iotests/198 | 2 +
30
tests/qemu-iotests/206.out | 4 ++
31
tests/qemu-iotests/242.out | 5 ++
32
tests/qemu-iotests/255.out | 8 +--
33
tests/qemu-iotests/274.out | 49 ++++++++-------
34
tests/qemu-iotests/280.out | 2 +-
35
tests/qemu-iotests/291.out | 2 +
36
tests/qemu-iotests/302.out | 1 +
37
tests/qemu-iotests/303.out | 4 +-
38
tests/qemu-iotests/common.filter | 1 +
39
25 files changed, 256 insertions(+), 142 deletions(-)
40
13
41
diff --git a/qapi/block-core.json b/qapi/block-core.json
14
diff --git a/qapi/block-core.json b/qapi/block-core.json
42
index XXXXXXX..XXXXXXX 100644
15
index XXXXXXX..XXXXXXX 100644
43
--- a/qapi/block-core.json
16
--- a/qapi/block-core.json
44
+++ b/qapi/block-core.json
17
+++ b/qapi/block-core.json
45
@@ -XXX,XX +XXX,XX @@
18
@@ -XXX,XX +XXX,XX @@
46
# standalone (read-only) raw image without looking at qcow2
47
# metadata (since: 4.0)
48
#
19
#
49
+# @extended-l2: true if the image has extended L2 entries; only valid for
20
# @target: The target for copy-before-write operations.
50
+# compat >= 1.1 (since 5.2)
21
#
22
+# @bitmap: If specified, copy-before-write filter will do
23
+# copy-before-write operations only for dirty regions of the
24
+# bitmap. Bitmap size must be equal to length of file and
25
+# target child of the filter. Note also, that bitmap is used
26
+# only to initialize internal bitmap of the process, so further
27
+# modifications (or removing) of specified bitmap doesn't
28
+# influence the filter. (Since 7.0)
51
+#
29
+#
52
# @lazy-refcounts: on or off; only valid for compat >= 1.1
30
# Since: 6.2
53
#
31
##
54
# @corrupt: true if the image has been marked corrupt; only valid for
32
{ 'struct': 'BlockdevOptionsCbw',
33
'base': 'BlockdevOptionsGenericFormat',
34
- 'data': { 'target': 'BlockdevRef' } }
35
+ 'data': { 'target': 'BlockdevRef', '*bitmap': 'BlockDirtyBitmap' } }
36
37
##
38
# @BlockdevOptions:
39
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
40
index XXXXXXX..XXXXXXX 100644
41
--- a/block/copy-before-write.c
42
+++ b/block/copy-before-write.c
55
@@ -XXX,XX +XXX,XX @@
43
@@ -XXX,XX +XXX,XX @@
56
'compat': 'str',
44
57
'*data-file': 'str',
45
#include "block/copy-before-write.h"
58
'*data-file-raw': 'bool',
46
59
+ '*extended-l2': 'bool',
47
+#include "qapi/qapi-visit-block-core.h"
60
'*lazy-refcounts': 'bool',
48
+
61
'*corrupt': 'bool',
49
typedef struct BDRVCopyBeforeWriteState {
62
'refcount-bits': 'int',
50
BlockCopyState *bcs;
63
@@ -XXX,XX +XXX,XX @@
51
BdrvChild *target;
64
# @data-file-raw: True if the external data file must stay valid as a
52
@@ -XXX,XX +XXX,XX @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
65
# standalone (read-only) raw image without looking at qcow2
53
}
66
# metadata (default: false; since: 4.0)
67
+# @extended-l2 True to make the image have extended L2 entries
68
+# (default: false; since 5.2)
69
# @size: Size of the virtual disk in bytes
70
# @version: Compatibility level (default: v3)
71
# @backing-file: File name of the backing file if a backing file
72
@@ -XXX,XX +XXX,XX @@
73
'data': { 'file': 'BlockdevRef',
74
'*data-file': 'BlockdevRef',
75
'*data-file-raw': 'bool',
76
+ '*extended-l2': 'bool',
77
'size': 'size',
78
'*version': 'BlockdevQcow2Version',
79
'*backing-file': 'str',
80
diff --git a/block/qcow2.h b/block/qcow2.h
81
index XXXXXXX..XXXXXXX 100644
82
--- a/block/qcow2.h
83
+++ b/block/qcow2.h
84
@@ -XXX,XX +XXX,XX @@ enum {
85
QCOW2_INCOMPAT_CORRUPT_BITNR = 1,
86
QCOW2_INCOMPAT_DATA_FILE_BITNR = 2,
87
QCOW2_INCOMPAT_COMPRESSION_BITNR = 3,
88
+ QCOW2_INCOMPAT_EXTL2_BITNR = 4,
89
QCOW2_INCOMPAT_DIRTY = 1 << QCOW2_INCOMPAT_DIRTY_BITNR,
90
QCOW2_INCOMPAT_CORRUPT = 1 << QCOW2_INCOMPAT_CORRUPT_BITNR,
91
QCOW2_INCOMPAT_DATA_FILE = 1 << QCOW2_INCOMPAT_DATA_FILE_BITNR,
92
QCOW2_INCOMPAT_COMPRESSION = 1 << QCOW2_INCOMPAT_COMPRESSION_BITNR,
93
+ QCOW2_INCOMPAT_EXTL2 = 1 << QCOW2_INCOMPAT_EXTL2_BITNR,
94
95
QCOW2_INCOMPAT_MASK = QCOW2_INCOMPAT_DIRTY
96
| QCOW2_INCOMPAT_CORRUPT
97
| QCOW2_INCOMPAT_DATA_FILE
98
- | QCOW2_INCOMPAT_COMPRESSION,
99
+ | QCOW2_INCOMPAT_COMPRESSION
100
+ | QCOW2_INCOMPAT_EXTL2,
101
};
102
103
/* Compatible feature bits */
104
@@ -XXX,XX +XXX,XX @@ typedef enum QCow2MetadataOverlap {
105
106
static inline bool has_subclusters(BDRVQcow2State *s)
107
{
108
- /* FIXME: Return false until this feature is complete */
109
- return false;
110
+ return s->incompatible_features & QCOW2_INCOMPAT_EXTL2;
111
}
54
}
112
55
113
static inline size_t l2_entry_size(BDRVQcow2State *s)
56
+static bool cbw_parse_bitmap_option(QDict *options, BdrvDirtyBitmap **bitmap,
114
diff --git a/include/block/block_int.h b/include/block/block_int.h
57
+ Error **errp)
115
index XXXXXXX..XXXXXXX 100644
58
+{
116
--- a/include/block/block_int.h
59
+ QDict *bitmap_qdict = NULL;
117
+++ b/include/block/block_int.h
60
+ BlockDirtyBitmap *bmp_param = NULL;
118
@@ -XXX,XX +XXX,XX @@
61
+ Visitor *v = NULL;
119
#define BLOCK_OPT_DATA_FILE "data_file"
62
+ bool ret = false;
120
#define BLOCK_OPT_DATA_FILE_RAW "data_file_raw"
63
+
121
#define BLOCK_OPT_COMPRESSION_TYPE "compression_type"
64
+ *bitmap = NULL;
122
+#define BLOCK_OPT_EXTL2 "extended_l2"
65
+
123
66
+ qdict_extract_subqdict(options, &bitmap_qdict, "bitmap.");
124
#define BLOCK_PROBE_BUF_SIZE 512
67
+ if (!qdict_size(bitmap_qdict)) {
125
68
+ ret = true;
126
diff --git a/block/qcow2.c b/block/qcow2.c
69
+ goto out;
127
index XXXXXXX..XXXXXXX 100644
128
--- a/block/qcow2.c
129
+++ b/block/qcow2.c
130
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options,
131
s->subcluster_size = s->cluster_size / s->subclusters_per_cluster;
132
s->subcluster_bits = ctz32(s->subcluster_size);
133
134
+ if (s->subcluster_size < (1 << MIN_CLUSTER_BITS)) {
135
+ error_setg(errp, "Unsupported subcluster size: %d", s->subcluster_size);
136
+ ret = -EINVAL;
137
+ goto fail;
138
+ }
70
+ }
139
+
71
+
140
/* Check support for various header values */
72
+ v = qobject_input_visitor_new_flat_confused(bitmap_qdict, errp);
141
if (header.refcount_order > 6) {
73
+ if (!v) {
142
error_setg(errp, "Reference count entry width too large; may not "
74
+ goto out;
143
@@ -XXX,XX +XXX,XX @@ int qcow2_update_header(BlockDriverState *bs)
144
.bit = QCOW2_INCOMPAT_COMPRESSION_BITNR,
145
.name = "compression type",
146
},
147
+ {
148
+ .type = QCOW2_FEAT_TYPE_INCOMPATIBLE,
149
+ .bit = QCOW2_INCOMPAT_EXTL2_BITNR,
150
+ .name = "extended L2 entries",
151
+ },
152
{
153
.type = QCOW2_FEAT_TYPE_COMPATIBLE,
154
.bit = QCOW2_COMPAT_LAZY_REFCOUNTS_BITNR,
155
@@ -XXX,XX +XXX,XX @@ static int64_t qcow2_calc_prealloc_size(int64_t total_size,
156
return meta_size + aligned_total_size;
157
}
158
159
-static bool validate_cluster_size(size_t cluster_size, Error **errp)
160
+static bool validate_cluster_size(size_t cluster_size, bool extended_l2,
161
+ Error **errp)
162
{
163
int cluster_bits = ctz32(cluster_size);
164
if (cluster_bits < MIN_CLUSTER_BITS || cluster_bits > MAX_CLUSTER_BITS ||
165
@@ -XXX,XX +XXX,XX @@ static bool validate_cluster_size(size_t cluster_size, Error **errp)
166
"%dk", 1 << MIN_CLUSTER_BITS, 1 << (MAX_CLUSTER_BITS - 10));
167
return false;
168
}
169
+
170
+ if (extended_l2) {
171
+ unsigned min_cluster_size =
172
+ (1 << MIN_CLUSTER_BITS) * QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER;
173
+ if (cluster_size < min_cluster_size) {
174
+ error_setg(errp, "Extended L2 entries are only supported with "
175
+ "cluster sizes of at least %u bytes", min_cluster_size);
176
+ return false;
177
+ }
178
+ }
75
+ }
179
+
76
+
180
return true;
77
+ visit_type_BlockDirtyBitmap(v, NULL, &bmp_param, errp);
181
}
78
+ if (!bmp_param) {
182
79
+ goto out;
183
-static size_t qcow2_opt_get_cluster_size_del(QemuOpts *opts, Error **errp)
184
+static size_t qcow2_opt_get_cluster_size_del(QemuOpts *opts, bool extended_l2,
185
+ Error **errp)
186
{
187
size_t cluster_size;
188
189
cluster_size = qemu_opt_get_size_del(opts, BLOCK_OPT_CLUSTER_SIZE,
190
DEFAULT_CLUSTER_SIZE);
191
- if (!validate_cluster_size(cluster_size, errp)) {
192
+ if (!validate_cluster_size(cluster_size, extended_l2, errp)) {
193
return 0;
194
}
195
return cluster_size;
196
@@ -XXX,XX +XXX,XX @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp)
197
cluster_size = DEFAULT_CLUSTER_SIZE;
198
}
199
200
- if (!validate_cluster_size(cluster_size, errp)) {
201
+ if (!qcow2_opts->has_extended_l2) {
202
+ qcow2_opts->extended_l2 = false;
203
+ }
204
+ if (qcow2_opts->extended_l2) {
205
+ if (version < 3) {
206
+ error_setg(errp, "Extended L2 entries are only supported with "
207
+ "compatibility level 1.1 and above (use version=v3 or "
208
+ "greater)");
209
+ ret = -EINVAL;
210
+ goto out;
211
+ }
212
+ }
80
+ }
213
+
81
+
214
+ if (!validate_cluster_size(cluster_size, qcow2_opts->extended_l2, errp)) {
82
+ *bitmap = block_dirty_bitmap_lookup(bmp_param->node, bmp_param->name, NULL,
215
ret = -EINVAL;
83
+ errp);
216
goto out;
84
+ if (!*bitmap) {
217
}
85
+ goto out;
218
@@ -XXX,XX +XXX,XX @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp)
219
cpu_to_be64(QCOW2_INCOMPAT_COMPRESSION);
220
}
221
222
+ if (qcow2_opts->extended_l2) {
223
+ header->incompatible_features |=
224
+ cpu_to_be64(QCOW2_INCOMPAT_EXTL2);
225
+ }
86
+ }
226
+
87
+
227
ret = blk_pwrite(blk, 0, header, cluster_size, 0);
88
+ ret = true;
228
g_free(header);
229
if (ret < 0) {
230
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_create_opts(BlockDriver *drv,
231
{ BLOCK_OPT_BACKING_FMT, "backing-fmt" },
232
{ BLOCK_OPT_CLUSTER_SIZE, "cluster-size" },
233
{ BLOCK_OPT_LAZY_REFCOUNTS, "lazy-refcounts" },
234
+ { BLOCK_OPT_EXTL2, "extended-l2" },
235
{ BLOCK_OPT_REFCOUNT_BITS, "refcount-bits" },
236
{ BLOCK_OPT_ENCRYPT, BLOCK_OPT_ENCRYPT_FORMAT },
237
{ BLOCK_OPT_COMPAT_LEVEL, "version" },
238
@@ -XXX,XX +XXX,XX @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs,
239
PreallocMode prealloc;
240
bool has_backing_file;
241
bool has_luks;
242
- bool extended_l2 = false; /* Set to false until the option is added */
243
+ bool extended_l2;
244
size_t l2e_size;
245
246
/* Parse image creation options */
247
- cluster_size = qcow2_opt_get_cluster_size_del(opts, &local_err);
248
+ extended_l2 = qemu_opt_get_bool_del(opts, BLOCK_OPT_EXTL2, false);
249
+
89
+
250
+ cluster_size = qcow2_opt_get_cluster_size_del(opts, extended_l2,
90
+out:
251
+ &local_err);
91
+ qapi_free_BlockDirtyBitmap(bmp_param);
252
if (local_err) {
92
+ visit_free(v);
253
goto err;
93
+ qobject_unref(bitmap_qdict);
94
+
95
+ return ret;
96
+}
97
+
98
static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
99
Error **errp)
100
{
101
BDRVCopyBeforeWriteState *s = bs->opaque;
102
+ BdrvDirtyBitmap *bitmap = NULL;
103
104
bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
105
BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
106
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
107
return -EINVAL;
254
}
108
}
255
@@ -XXX,XX +XXX,XX @@ static ImageInfoSpecific *qcow2_get_specific_info(BlockDriverState *bs,
109
256
.corrupt = s->incompatible_features &
110
+ if (!cbw_parse_bitmap_option(options, &bitmap, errp)) {
257
QCOW2_INCOMPAT_CORRUPT,
111
+ return -EINVAL;
258
.has_corrupt = true,
112
+ }
259
+ .has_extended_l2 = true,
113
+
260
+ .extended_l2 = has_subclusters(s),
114
bs->total_sectors = bs->file->bs->total_sectors;
261
.refcount_bits = s->refcount_bits,
115
bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
262
.has_bitmaps = !!bitmaps,
116
(BDRV_REQ_FUA & bs->file->bs->supported_write_flags);
263
.bitmaps = bitmaps,
117
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
264
@@ -XXX,XX +XXX,XX @@ static QemuOptsList qcow2_create_opts = {
118
((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
265
.help = "qcow2 cluster size", \
119
bs->file->bs->supported_zero_flags);
266
.def_value_str = stringify(DEFAULT_CLUSTER_SIZE) \
120
267
}, \
121
- s->bcs = block_copy_state_new(bs->file, s->target, NULL, errp);
268
+ { \
122
+ s->bcs = block_copy_state_new(bs->file, s->target, bitmap, errp);
269
+ .name = BLOCK_OPT_EXTL2, \
123
if (!s->bcs) {
270
+ .type = QEMU_OPT_BOOL, \
124
error_prepend(errp, "Cannot create block-copy-state: ");
271
+ .help = "Extended L2 tables", \
125
return -EINVAL;
272
+ .def_value_str = "off" \
273
+ }, \
274
{ \
275
.name = BLOCK_OPT_PREALLOC, \
276
.type = QEMU_OPT_STRING, \
277
diff --git a/tests/qemu-iotests/031.out b/tests/qemu-iotests/031.out
278
index XXXXXXX..XXXXXXX 100644
279
--- a/tests/qemu-iotests/031.out
280
+++ b/tests/qemu-iotests/031.out
281
@@ -XXX,XX +XXX,XX @@ header_length 112
282
283
Header extension:
284
magic 0x6803f857 (Feature table)
285
-length 336
286
+length 384
287
data <binary>
288
289
Header extension:
290
@@ -XXX,XX +XXX,XX @@ header_length 112
291
292
Header extension:
293
magic 0x6803f857 (Feature table)
294
-length 336
295
+length 384
296
data <binary>
297
298
Header extension:
299
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
300
301
magic 0x514649fb
302
version 3
303
-backing_file_offset 0x210
304
+backing_file_offset 0x240
305
backing_file_size 0x17
306
cluster_bits 16
307
size 67108864
308
@@ -XXX,XX +XXX,XX @@ data 'host_device'
309
310
Header extension:
311
magic 0x6803f857 (Feature table)
312
-length 336
313
+length 384
314
data <binary>
315
316
Header extension:
317
diff --git a/tests/qemu-iotests/036.out b/tests/qemu-iotests/036.out
318
index XXXXXXX..XXXXXXX 100644
319
--- a/tests/qemu-iotests/036.out
320
+++ b/tests/qemu-iotests/036.out
321
@@ -XXX,XX +XXX,XX @@ compatible_features []
322
autoclear_features [63]
323
Header extension:
324
magic 0x6803f857 (Feature table)
325
-length 336
326
+length 384
327
data <binary>
328
329
330
@@ -XXX,XX +XXX,XX @@ compatible_features []
331
autoclear_features []
332
Header extension:
333
magic 0x6803f857 (Feature table)
334
-length 336
335
+length 384
336
data <binary>
337
338
*** done
339
diff --git a/tests/qemu-iotests/049.out b/tests/qemu-iotests/049.out
340
index XXXXXXX..XXXXXXX 100644
341
--- a/tests/qemu-iotests/049.out
342
+++ b/tests/qemu-iotests/049.out
343
@@ -XXX,XX +XXX,XX @@ QA output created by 049
344
== 1. Traditional size parameter ==
345
346
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024
347
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
348
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
349
350
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024b
351
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
352
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
353
354
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1k
355
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
356
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
357
358
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1K
359
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
360
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
361
362
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1M
363
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16
364
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16
365
366
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1G
367
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 lazy_refcounts=off refcount_bits=16
368
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1073741824 lazy_refcounts=off refcount_bits=16
369
370
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1T
371
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1099511627776 lazy_refcounts=off refcount_bits=16
372
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1099511627776 lazy_refcounts=off refcount_bits=16
373
374
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024.0
375
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
376
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
377
378
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024.0b
379
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
380
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
381
382
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5k
383
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
384
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
385
386
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5K
387
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
388
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
389
390
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5M
391
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1572864 lazy_refcounts=off refcount_bits=16
392
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1572864 lazy_refcounts=off refcount_bits=16
393
394
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5G
395
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1610612736 lazy_refcounts=off refcount_bits=16
396
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1610612736 lazy_refcounts=off refcount_bits=16
397
398
qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5T
399
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1649267441664 lazy_refcounts=off refcount_bits=16
400
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1649267441664 lazy_refcounts=off refcount_bits=16
401
402
== 2. Specifying size via -o ==
403
404
qemu-img create -f qcow2 -o size=1024 TEST_DIR/t.qcow2
405
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
406
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
407
408
qemu-img create -f qcow2 -o size=1024b TEST_DIR/t.qcow2
409
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
410
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
411
412
qemu-img create -f qcow2 -o size=1k TEST_DIR/t.qcow2
413
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
414
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
415
416
qemu-img create -f qcow2 -o size=1K TEST_DIR/t.qcow2
417
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
418
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
419
420
qemu-img create -f qcow2 -o size=1M TEST_DIR/t.qcow2
421
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16
422
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16
423
424
qemu-img create -f qcow2 -o size=1G TEST_DIR/t.qcow2
425
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 lazy_refcounts=off refcount_bits=16
426
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1073741824 lazy_refcounts=off refcount_bits=16
427
428
qemu-img create -f qcow2 -o size=1T TEST_DIR/t.qcow2
429
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1099511627776 lazy_refcounts=off refcount_bits=16
430
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1099511627776 lazy_refcounts=off refcount_bits=16
431
432
qemu-img create -f qcow2 -o size=1024.0 TEST_DIR/t.qcow2
433
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
434
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
435
436
qemu-img create -f qcow2 -o size=1024.0b TEST_DIR/t.qcow2
437
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
438
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
439
440
qemu-img create -f qcow2 -o size=1.5k TEST_DIR/t.qcow2
441
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
442
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
443
444
qemu-img create -f qcow2 -o size=1.5K TEST_DIR/t.qcow2
445
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
446
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
447
448
qemu-img create -f qcow2 -o size=1.5M TEST_DIR/t.qcow2
449
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1572864 lazy_refcounts=off refcount_bits=16
450
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1572864 lazy_refcounts=off refcount_bits=16
451
452
qemu-img create -f qcow2 -o size=1.5G TEST_DIR/t.qcow2
453
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1610612736 lazy_refcounts=off refcount_bits=16
454
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1610612736 lazy_refcounts=off refcount_bits=16
455
456
qemu-img create -f qcow2 -o size=1.5T TEST_DIR/t.qcow2
457
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1649267441664 lazy_refcounts=off refcount_bits=16
458
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1649267441664 lazy_refcounts=off refcount_bits=16
459
460
== 3. Invalid sizes ==
461
462
@@ -XXX,XX +XXX,XX @@ qemu-img: TEST_DIR/t.qcow2: The image size must be specified only once
463
== Check correct interpretation of suffixes for cluster size ==
464
465
qemu-img create -f qcow2 -o cluster_size=1024 TEST_DIR/t.qcow2 64M
466
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
467
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
468
469
qemu-img create -f qcow2 -o cluster_size=1024b TEST_DIR/t.qcow2 64M
470
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
471
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
472
473
qemu-img create -f qcow2 -o cluster_size=1k TEST_DIR/t.qcow2 64M
474
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
475
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
476
477
qemu-img create -f qcow2 -o cluster_size=1K TEST_DIR/t.qcow2 64M
478
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
479
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
480
481
qemu-img create -f qcow2 -o cluster_size=1M TEST_DIR/t.qcow2 64M
482
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1048576 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
483
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1048576 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
484
485
qemu-img create -f qcow2 -o cluster_size=1024.0 TEST_DIR/t.qcow2 64M
486
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
487
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
488
489
qemu-img create -f qcow2 -o cluster_size=1024.0b TEST_DIR/t.qcow2 64M
490
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
491
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
492
493
qemu-img create -f qcow2 -o cluster_size=0.5k TEST_DIR/t.qcow2 64M
494
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=512 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
495
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=512 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
496
497
qemu-img create -f qcow2 -o cluster_size=0.5K TEST_DIR/t.qcow2 64M
498
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=512 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
499
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=512 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
500
501
qemu-img create -f qcow2 -o cluster_size=0.5M TEST_DIR/t.qcow2 64M
502
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=524288 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
503
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=524288 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
504
505
== Check compat level option ==
506
507
qemu-img create -f qcow2 -o compat=0.10 TEST_DIR/t.qcow2 64M
508
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=off refcount_bits=16
509
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=off refcount_bits=16
510
511
qemu-img create -f qcow2 -o compat=1.1 TEST_DIR/t.qcow2 64M
512
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=off refcount_bits=16
513
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=off refcount_bits=16
514
515
qemu-img create -f qcow2 -o compat=0.42 TEST_DIR/t.qcow2 64M
516
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.42 lazy_refcounts=off refcount_bits=16
517
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=0.42 lazy_refcounts=off refcount_bits=16
518
qemu-img: TEST_DIR/t.qcow2: Invalid parameter '0.42'
519
520
qemu-img create -f qcow2 -o compat=foobar TEST_DIR/t.qcow2 64M
521
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=foobar lazy_refcounts=off refcount_bits=16
522
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=foobar lazy_refcounts=off refcount_bits=16
523
qemu-img: TEST_DIR/t.qcow2: Invalid parameter 'foobar'
524
525
== Check preallocation option ==
526
527
qemu-img create -f qcow2 -o preallocation=off TEST_DIR/t.qcow2 64M
528
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 preallocation=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
529
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
530
531
qemu-img create -f qcow2 -o preallocation=metadata TEST_DIR/t.qcow2 64M
532
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 preallocation=metadata compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
533
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
534
535
qemu-img create -f qcow2 -o preallocation=1234 TEST_DIR/t.qcow2 64M
536
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 preallocation=1234 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
537
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=1234 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
538
qemu-img: TEST_DIR/t.qcow2: Invalid parameter '1234'
539
540
== Check encryption option ==
541
542
qemu-img create -f qcow2 -o encryption=off TEST_DIR/t.qcow2 64M
543
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 encryption=off cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
544
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 encryption=off cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
545
546
qemu-img create -f qcow2 --object secret,id=sec0,data=123456 -o encryption=on,encrypt.key-secret=sec0 TEST_DIR/t.qcow2 64M
547
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 encryption=on encrypt.key-secret=sec0 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
548
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 encryption=on encrypt.key-secret=sec0 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
549
550
== Check lazy_refcounts option (only with v3) ==
551
552
qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=off TEST_DIR/t.qcow2 64M
553
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=off refcount_bits=16
554
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=off refcount_bits=16
555
556
qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=on TEST_DIR/t.qcow2 64M
557
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=on refcount_bits=16
558
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=on refcount_bits=16
559
560
qemu-img create -f qcow2 -o compat=0.10,lazy_refcounts=off TEST_DIR/t.qcow2 64M
561
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=off refcount_bits=16
562
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=off refcount_bits=16
563
564
qemu-img create -f qcow2 -o compat=0.10,lazy_refcounts=on TEST_DIR/t.qcow2 64M
565
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=on refcount_bits=16
566
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=on refcount_bits=16
567
qemu-img: TEST_DIR/t.qcow2: Lazy refcounts only supported with compatibility level 1.1 and above (use version=v3 or greater)
568
569
*** done
570
diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out
571
index XXXXXXX..XXXXXXX 100644
572
--- a/tests/qemu-iotests/060.out
573
+++ b/tests/qemu-iotests/060.out
574
@@ -XXX,XX +XXX,XX @@ Format specific information:
575
lazy refcounts: false
576
refcount bits: 16
577
corrupt: true
578
+ extended l2: false
579
qemu-io: can't open device TEST_DIR/t.IMGFMT: IMGFMT: Image is corrupt; cannot be opened read/write
580
no file open, try 'help open'
581
read 512/512 bytes at offset 0
582
diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out
583
index XXXXXXX..XXXXXXX 100644
584
--- a/tests/qemu-iotests/061.out
585
+++ b/tests/qemu-iotests/061.out
586
@@ -XXX,XX +XXX,XX @@ header_length 112
587
588
Header extension:
589
magic 0x6803f857 (Feature table)
590
-length 336
591
+length 384
592
data <binary>
593
594
magic 0x514649fb
595
@@ -XXX,XX +XXX,XX @@ header_length 112
596
597
Header extension:
598
magic 0x6803f857 (Feature table)
599
-length 336
600
+length 384
601
data <binary>
602
603
magic 0x514649fb
604
@@ -XXX,XX +XXX,XX @@ header_length 112
605
606
Header extension:
607
magic 0x6803f857 (Feature table)
608
-length 336
609
+length 384
610
data <binary>
611
612
ERROR cluster 5 refcount=0 reference=1
613
@@ -XXX,XX +XXX,XX @@ header_length 112
614
615
Header extension:
616
magic 0x6803f857 (Feature table)
617
-length 336
618
+length 384
619
data <binary>
620
621
magic 0x514649fb
622
@@ -XXX,XX +XXX,XX @@ header_length 112
623
624
Header extension:
625
magic 0x6803f857 (Feature table)
626
-length 336
627
+length 384
628
data <binary>
629
630
read 65536/65536 bytes at offset 44040192
631
@@ -XXX,XX +XXX,XX @@ header_length 112
632
633
Header extension:
634
magic 0x6803f857 (Feature table)
635
-length 336
636
+length 384
637
data <binary>
638
639
ERROR cluster 5 refcount=0 reference=1
640
@@ -XXX,XX +XXX,XX @@ header_length 112
641
642
Header extension:
643
magic 0x6803f857 (Feature table)
644
-length 336
645
+length 384
646
data <binary>
647
648
read 131072/131072 bytes at offset 0
649
@@ -XXX,XX +XXX,XX @@ Format specific information:
650
data file: TEST_DIR/t.IMGFMT.data
651
data file raw: false
652
corrupt: false
653
+ extended l2: false
654
No errors were found on the image.
655
656
=== Try changing the external data file ===
657
@@ -XXX,XX +XXX,XX @@ Format specific information:
658
data file: foo
659
data file raw: false
660
corrupt: false
661
+ extended l2: false
662
663
qemu-img: Could not open 'TEST_DIR/t.IMGFMT': 'data-file' is required for this image
664
image: TEST_DIR/t.IMGFMT
665
@@ -XXX,XX +XXX,XX @@ Format specific information:
666
refcount bits: 16
667
data file raw: false
668
corrupt: false
669
+ extended l2: false
670
671
=== Clearing and setting data-file-raw ===
672
673
@@ -XXX,XX +XXX,XX @@ Format specific information:
674
data file: TEST_DIR/t.IMGFMT.data
675
data file raw: true
676
corrupt: false
677
+ extended l2: false
678
No errors were found on the image.
679
image: TEST_DIR/t.IMGFMT
680
file format: IMGFMT
681
@@ -XXX,XX +XXX,XX @@ Format specific information:
682
data file: TEST_DIR/t.IMGFMT.data
683
data file raw: false
684
corrupt: false
685
+ extended l2: false
686
No errors were found on the image.
687
qemu-img: data-file-raw cannot be set on existing images
688
image: TEST_DIR/t.IMGFMT
689
@@ -XXX,XX +XXX,XX @@ Format specific information:
690
data file: TEST_DIR/t.IMGFMT.data
691
data file raw: false
692
corrupt: false
693
+ extended l2: false
694
No errors were found on the image.
695
*** done
696
diff --git a/tests/qemu-iotests/065 b/tests/qemu-iotests/065
697
index XXXXXXX..XXXXXXX 100755
698
--- a/tests/qemu-iotests/065
699
+++ b/tests/qemu-iotests/065
700
@@ -XXX,XX +XXX,XX @@ class TestQCow3NotLazy(TestQemuImgInfo):
701
img_options = 'compat=1.1,lazy_refcounts=off'
702
json_compare = { 'compat': '1.1', 'lazy-refcounts': False,
703
'refcount-bits': 16, 'corrupt': False,
704
- 'compression-type': 'zlib' }
705
+ 'compression-type': 'zlib', 'extended-l2': False }
706
human_compare = [ 'compat: 1.1', 'compression type: zlib',
707
'lazy refcounts: false', 'refcount bits: 16',
708
- 'corrupt: false' ]
709
+ 'corrupt: false', 'extended l2: false' ]
710
711
class TestQCow3Lazy(TestQemuImgInfo):
712
'''Testing a qcow2 version 3 image with lazy refcounts enabled'''
713
img_options = 'compat=1.1,lazy_refcounts=on'
714
json_compare = { 'compat': '1.1', 'lazy-refcounts': True,
715
'refcount-bits': 16, 'corrupt': False,
716
- 'compression-type': 'zlib' }
717
+ 'compression-type': 'zlib', 'extended-l2': False }
718
human_compare = [ 'compat: 1.1', 'compression type: zlib',
719
'lazy refcounts: true', 'refcount bits: 16',
720
- 'corrupt: false' ]
721
+ 'corrupt: false', 'extended l2: false' ]
722
723
class TestQCow3NotLazyQMP(TestQMP):
724
'''Testing a qcow2 version 3 image with lazy refcounts disabled, opening
725
@@ -XXX,XX +XXX,XX @@ class TestQCow3NotLazyQMP(TestQMP):
726
qemu_options = 'lazy-refcounts=on'
727
compare = { 'compat': '1.1', 'lazy-refcounts': False,
728
'refcount-bits': 16, 'corrupt': False,
729
- 'compression-type': 'zlib' }
730
+ 'compression-type': 'zlib', 'extended-l2': False }
731
732
733
class TestQCow3LazyQMP(TestQMP):
734
@@ -XXX,XX +XXX,XX @@ class TestQCow3LazyQMP(TestQMP):
735
qemu_options = 'lazy-refcounts=off'
736
compare = { 'compat': '1.1', 'lazy-refcounts': True,
737
'refcount-bits': 16, 'corrupt': False,
738
- 'compression-type': 'zlib' }
739
+ 'compression-type': 'zlib', 'extended-l2': False }
740
741
TestImageInfoSpecific = None
742
TestQemuImgInfo = None
743
diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out
744
index XXXXXXX..XXXXXXX 100644
745
--- a/tests/qemu-iotests/082.out
746
+++ b/tests/qemu-iotests/082.out
747
@@ -XXX,XX +XXX,XX @@ QA output created by 082
748
=== create: Options specified more than once ===
749
750
Testing: create -f foo -f qcow2 TEST_DIR/t.qcow2 128M
751
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
752
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
753
image: TEST_DIR/t.IMGFMT
754
file format: IMGFMT
755
virtual size: 128 MiB (134217728 bytes)
756
cluster_size: 65536
757
758
Testing: create -f qcow2 -o cluster_size=4k -o lazy_refcounts=on TEST_DIR/t.qcow2 128M
759
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=4096 compression_type=zlib size=134217728 lazy_refcounts=on refcount_bits=16
760
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=4096 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=on refcount_bits=16
761
image: TEST_DIR/t.IMGFMT
762
file format: IMGFMT
763
virtual size: 128 MiB (134217728 bytes)
764
@@ -XXX,XX +XXX,XX @@ Format specific information:
765
lazy refcounts: true
766
refcount bits: 16
767
corrupt: false
768
+ extended l2: false
769
770
Testing: create -f qcow2 -o cluster_size=4k -o lazy_refcounts=on -o cluster_size=8k TEST_DIR/t.qcow2 128M
771
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=8192 compression_type=zlib size=134217728 lazy_refcounts=on refcount_bits=16
772
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=8192 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=on refcount_bits=16
773
image: TEST_DIR/t.IMGFMT
774
file format: IMGFMT
775
virtual size: 128 MiB (134217728 bytes)
776
@@ -XXX,XX +XXX,XX @@ Format specific information:
777
lazy refcounts: true
778
refcount bits: 16
779
corrupt: false
780
+ extended l2: false
781
782
Testing: create -f qcow2 -o cluster_size=4k,cluster_size=8k TEST_DIR/t.qcow2 128M
783
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=8192 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
784
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=8192 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
785
image: TEST_DIR/t.IMGFMT
786
file format: IMGFMT
787
virtual size: 128 MiB (134217728 bytes)
788
@@ -XXX,XX +XXX,XX @@ Supported options:
789
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
790
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
791
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
792
+ extended_l2=<bool (on/off)> - Extended L2 tables
793
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
794
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
795
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
796
@@ -XXX,XX +XXX,XX @@ Supported options:
797
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
798
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
799
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
800
+ extended_l2=<bool (on/off)> - Extended L2 tables
801
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
802
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
803
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
804
@@ -XXX,XX +XXX,XX @@ Supported options:
805
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
806
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
807
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
808
+ extended_l2=<bool (on/off)> - Extended L2 tables
809
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
810
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
811
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
812
@@ -XXX,XX +XXX,XX @@ Supported options:
813
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
814
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
815
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
816
+ extended_l2=<bool (on/off)> - Extended L2 tables
817
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
818
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
819
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
820
@@ -XXX,XX +XXX,XX @@ Supported options:
821
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
822
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
823
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
824
+ extended_l2=<bool (on/off)> - Extended L2 tables
825
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
826
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
827
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
828
@@ -XXX,XX +XXX,XX @@ Supported options:
829
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
830
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
831
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
832
+ extended_l2=<bool (on/off)> - Extended L2 tables
833
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
834
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
835
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
836
@@ -XXX,XX +XXX,XX @@ Supported options:
837
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
838
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
839
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
840
+ extended_l2=<bool (on/off)> - Extended L2 tables
841
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
842
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
843
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
844
@@ -XXX,XX +XXX,XX @@ Supported options:
845
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
846
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
847
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
848
+ extended_l2=<bool (on/off)> - Extended L2 tables
849
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
850
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
851
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
852
@@ -XXX,XX +XXX,XX @@ Supported options:
853
size=<size> - Virtual disk size
854
855
Testing: create -f qcow2 -u -o backing_file=TEST_DIR/t.qcow2,,help -F qcow2 TEST_DIR/t.qcow2 128M
856
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2,,help backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
857
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2,,help backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
858
859
Testing: create -f qcow2 -u -o backing_file=TEST_DIR/t.qcow2,,? -F qcow2 TEST_DIR/t.qcow2 128M
860
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2,,? backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
861
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2,,? backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
862
863
Testing: create -f qcow2 -o backing_file=TEST_DIR/t.qcow2, -o help TEST_DIR/t.qcow2 128M
864
qemu-img: Invalid option list: backing_file=TEST_DIR/t.qcow2,
865
@@ -XXX,XX +XXX,XX @@ Supported qcow2 options:
866
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
867
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
868
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
869
+ extended_l2=<bool (on/off)> - Extended L2 tables
870
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
871
preallocation=<str> - Preallocation mode (allowed values: off, metadata, falloc, full)
872
refcount_bits=<num> - Width of a reference count entry in bits
873
@@ -XXX,XX +XXX,XX @@ qemu-img: Format driver 'bochs' does not support image creation
874
=== convert: Options specified more than once ===
875
876
Testing: create -f qcow2 TEST_DIR/t.qcow2 128M
877
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
878
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
879
880
Testing: convert -f foo -f qcow2 TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.base
881
image: TEST_DIR/t.IMGFMT.base
882
@@ -XXX,XX +XXX,XX @@ Format specific information:
883
lazy refcounts: true
884
refcount bits: 16
885
corrupt: false
886
+ extended l2: false
887
888
Testing: convert -O qcow2 -o cluster_size=4k -o lazy_refcounts=on -o cluster_size=8k TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.base
889
image: TEST_DIR/t.IMGFMT.base
890
@@ -XXX,XX +XXX,XX @@ Format specific information:
891
lazy refcounts: true
892
refcount bits: 16
893
corrupt: false
894
+ extended l2: false
895
896
Testing: convert -O qcow2 -o cluster_size=4k,cluster_size=8k TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.base
897
image: TEST_DIR/t.IMGFMT.base
898
@@ -XXX,XX +XXX,XX @@ Supported options:
899
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
900
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
901
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
902
+ extended_l2=<bool (on/off)> - Extended L2 tables
903
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
904
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
905
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
906
@@ -XXX,XX +XXX,XX @@ Supported options:
907
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
908
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
909
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
910
+ extended_l2=<bool (on/off)> - Extended L2 tables
911
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
912
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
913
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
914
@@ -XXX,XX +XXX,XX @@ Supported options:
915
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
916
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
917
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
918
+ extended_l2=<bool (on/off)> - Extended L2 tables
919
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
920
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
921
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
922
@@ -XXX,XX +XXX,XX @@ Supported options:
923
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
924
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
925
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
926
+ extended_l2=<bool (on/off)> - Extended L2 tables
927
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
928
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
929
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
930
@@ -XXX,XX +XXX,XX @@ Supported options:
931
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
932
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
933
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
934
+ extended_l2=<bool (on/off)> - Extended L2 tables
935
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
936
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
937
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
938
@@ -XXX,XX +XXX,XX @@ Supported options:
939
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
940
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
941
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
942
+ extended_l2=<bool (on/off)> - Extended L2 tables
943
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
944
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
945
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
946
@@ -XXX,XX +XXX,XX @@ Supported options:
947
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
948
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
949
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
950
+ extended_l2=<bool (on/off)> - Extended L2 tables
951
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
952
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
953
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
954
@@ -XXX,XX +XXX,XX @@ Supported options:
955
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
956
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
957
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
958
+ extended_l2=<bool (on/off)> - Extended L2 tables
959
extent_size_hint=<size> - Extent size hint for the image file, 0 to disable
960
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
961
nocow=<bool (on/off)> - Turn off copy-on-write (valid only on btrfs)
962
@@ -XXX,XX +XXX,XX @@ Supported qcow2 options:
963
encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
964
encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
965
encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
966
+ extended_l2=<bool (on/off)> - Extended L2 tables
967
lazy_refcounts=<bool (on/off)> - Postpone refcount updates
968
preallocation=<str> - Preallocation mode (allowed values: off, metadata, falloc, full)
969
refcount_bits=<num> - Width of a reference count entry in bits
970
@@ -XXX,XX +XXX,XX @@ Format specific information:
971
lazy refcounts: true
972
refcount bits: 16
973
corrupt: false
974
+ extended l2: false
975
976
Testing: amend -f qcow2 -o size=130M -o lazy_refcounts=off TEST_DIR/t.qcow2
977
image: TEST_DIR/t.IMGFMT
978
@@ -XXX,XX +XXX,XX @@ Format specific information:
979
lazy refcounts: false
980
refcount bits: 16
981
corrupt: false
982
+ extended l2: false
983
984
Testing: amend -f qcow2 -o size=8M -o lazy_refcounts=on -o size=132M TEST_DIR/t.qcow2
985
image: TEST_DIR/t.IMGFMT
986
@@ -XXX,XX +XXX,XX @@ Format specific information:
987
lazy refcounts: true
988
refcount bits: 16
989
corrupt: false
990
+ extended l2: false
991
992
Testing: amend -f qcow2 -o size=4M,size=148M TEST_DIR/t.qcow2
993
image: TEST_DIR/t.IMGFMT
994
diff --git a/tests/qemu-iotests/085.out b/tests/qemu-iotests/085.out
995
index XXXXXXX..XXXXXXX 100644
996
--- a/tests/qemu-iotests/085.out
997
+++ b/tests/qemu-iotests/085.out
998
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=134217728
999
=== Create a single snapshot on virtio0 ===
1000
1001
{ 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/1-snapshot-v0.IMGFMT', 'format': 'IMGFMT' } }
1002
-Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1003
+Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1004
{"return": {}}
1005
1006
=== Invalid command - missing device and nodename ===
1007
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compress
1008
=== Create several transactional group snapshots ===
1009
1010
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/2-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/2-snapshot-v1.IMGFMT' } } ] } }
1011
-Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1012
-Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1013
+Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1014
+Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1015
{"return": {}}
1016
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/3-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/3-snapshot-v1.IMGFMT' } } ] } }
1017
-Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1018
-Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1019
+Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1020
+Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1021
{"return": {}}
1022
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/4-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/4-snapshot-v1.IMGFMT' } } ] } }
1023
-Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1024
-Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1025
+Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1026
+Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1027
{"return": {}}
1028
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/5-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/5-snapshot-v1.IMGFMT' } } ] } }
1029
-Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1030
-Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1031
+Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1032
+Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1033
{"return": {}}
1034
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/6-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/6-snapshot-v1.IMGFMT' } } ] } }
1035
-Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1036
-Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1037
+Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1038
+Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1039
{"return": {}}
1040
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/7-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/7-snapshot-v1.IMGFMT' } } ] } }
1041
-Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1042
-Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1043
+Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1044
+Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1045
{"return": {}}
1046
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/8-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/8-snapshot-v1.IMGFMT' } } ] } }
1047
-Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1048
-Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1049
+Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1050
+Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1051
{"return": {}}
1052
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/9-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/9-snapshot-v1.IMGFMT' } } ] } }
1053
-Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1054
-Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1055
+Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1056
+Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1057
{"return": {}}
1058
{ 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/10-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/10-snapshot-v1.IMGFMT' } } ] } }
1059
-Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1060
-Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1061
+Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1062
+Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1063
{"return": {}}
1064
1065
=== Create a couple of snapshots using blockdev-snapshot ===
1066
diff --git a/tests/qemu-iotests/144.out b/tests/qemu-iotests/144.out
1067
index XXXXXXX..XXXXXXX 100644
1068
--- a/tests/qemu-iotests/144.out
1069
+++ b/tests/qemu-iotests/144.out
1070
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=536870912
1071
{ 'execute': 'qmp_capabilities' }
1072
{"return": {}}
1073
{ 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/tmp.IMGFMT', 'format': 'IMGFMT' } }
1074
-Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1075
+Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1076
{"return": {}}
1077
1078
=== Performing block-commit on active layer ===
1079
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 cluster_size=65536 compression_type=z
1080
=== Performing Live Snapshot 2 ===
1081
1082
{ 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/tmp2.IMGFMT', 'format': 'IMGFMT' } }
1083
-Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1084
+Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1085
{"return": {}}
1086
*** done
1087
diff --git a/tests/qemu-iotests/182.out b/tests/qemu-iotests/182.out
1088
index XXXXXXX..XXXXXXX 100644
1089
--- a/tests/qemu-iotests/182.out
1090
+++ b/tests/qemu-iotests/182.out
1091
@@ -XXX,XX +XXX,XX @@ Is another process using the image [TEST_DIR/t.qcow2]?
1092
{'execute': 'blockdev-add', 'arguments': { 'node-name': 'node0', 'driver': 'file', 'filename': 'TEST_DIR/t.IMGFMT', 'locking': 'on' } }
1093
{"return": {}}
1094
{'execute': 'blockdev-snapshot-sync', 'arguments': { 'node-name': 'node0', 'snapshot-file': 'TEST_DIR/t.IMGFMT.overlay', 'snapshot-node-name': 'node1' } }
1095
-Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 cluster_size=65536 compression_type=zlib size=197120 backing_file=TEST_DIR/t.qcow2 backing_fmt=file lazy_refcounts=off refcount_bits=16
1096
+Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=197120 backing_file=TEST_DIR/t.qcow2 backing_fmt=file lazy_refcounts=off refcount_bits=16
1097
{"return": {}}
1098
{'execute': 'blockdev-add', 'arguments': { 'node-name': 'node1', 'driver': 'file', 'filename': 'TEST_DIR/t.IMGFMT', 'locking': 'on' } }
1099
{"return": {}}
1100
diff --git a/tests/qemu-iotests/185.out b/tests/qemu-iotests/185.out
1101
index XXXXXXX..XXXXXXX 100644
1102
--- a/tests/qemu-iotests/185.out
1103
+++ b/tests/qemu-iotests/185.out
1104
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
1105
=== Creating backing chain ===
1106
1107
{ 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'disk', 'snapshot-file': 'TEST_DIR/t.IMGFMT.mid', 'format': 'IMGFMT', 'mode': 'absolute-paths' } }
1108
-Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1109
+Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1110
{"return": {}}
1111
{ 'execute': 'human-monitor-command', 'arguments': { 'command-line': 'qemu-io disk "write 0 4M"' } }
1112
wrote 4194304/4194304 bytes at offset 0
1113
4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1114
{"return": ""}
1115
{ 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'disk', 'snapshot-file': 'TEST_DIR/t.IMGFMT', 'format': 'IMGFMT', 'mode': 'absolute-paths' } }
1116
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1117
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1118
{"return": {}}
1119
1120
=== Start commit job and exit qemu ===
1121
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zli
1122
{ 'execute': 'qmp_capabilities' }
1123
{"return": {}}
1124
{ 'execute': 'drive-mirror', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } }
1125
-Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
1126
+Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
1127
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}}
1128
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}}
1129
{"return": {}}
1130
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 compression_typ
1131
{ 'execute': 'qmp_capabilities' }
1132
{"return": {}}
1133
{ 'execute': 'drive-backup', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } }
1134
-Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
1135
+Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
1136
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}}
1137
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}}
1138
{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}}
1139
diff --git a/tests/qemu-iotests/198 b/tests/qemu-iotests/198
1140
index XXXXXXX..XXXXXXX 100755
1141
--- a/tests/qemu-iotests/198
1142
+++ b/tests/qemu-iotests/198
1143
@@ -XXX,XX +XXX,XX @@ echo "== checking image base =="
1144
$QEMU_IMG info --image-opts $IMGSPECBASE | _filter_img_info --format-specific \
1145
| sed -e "/^disk size:/ D" -e '/refcount bits:/ D' -e '/compat:/ D' \
1146
-e '/lazy refcounts:/ D' -e '/corrupt:/ D' -e '/^\s*data file/ D' \
1147
+ -e '/extended l2:/ D' \
1148
| _filter_json_filename
1149
1150
echo
1151
@@ -XXX,XX +XXX,XX @@ echo "== checking image layer =="
1152
$QEMU_IMG info --image-opts $IMGSPECLAYER | _filter_img_info --format-specific \
1153
| sed -e "/^disk size:/ D" -e '/refcount bits:/ D' -e '/compat:/ D' \
1154
-e '/lazy refcounts:/ D' -e '/corrupt:/ D' -e '/^\s*data file/ D' \
1155
+ -e '/extended l2:/ D' \
1156
| _filter_json_filename
1157
1158
1159
diff --git a/tests/qemu-iotests/206.out b/tests/qemu-iotests/206.out
1160
index XXXXXXX..XXXXXXX 100644
1161
--- a/tests/qemu-iotests/206.out
1162
+++ b/tests/qemu-iotests/206.out
1163
@@ -XXX,XX +XXX,XX @@ Format specific information:
1164
lazy refcounts: false
1165
refcount bits: 16
1166
corrupt: false
1167
+ extended l2: false
1168
1169
=== Successful image creation (inline blockdev-add, explicit defaults) ===
1170
1171
@@ -XXX,XX +XXX,XX @@ Format specific information:
1172
lazy refcounts: false
1173
refcount bits: 16
1174
corrupt: false
1175
+ extended l2: false
1176
1177
=== Successful image creation (v3 non-default options) ===
1178
1179
@@ -XXX,XX +XXX,XX @@ Format specific information:
1180
lazy refcounts: true
1181
refcount bits: 1
1182
corrupt: false
1183
+ extended l2: false
1184
1185
=== Successful image creation (v2 non-default options) ===
1186
1187
@@ -XXX,XX +XXX,XX @@ Format specific information:
1188
payload offset: 528384
1189
master key iters: XXX
1190
corrupt: false
1191
+ extended l2: false
1192
1193
=== Invalid BlockdevRef ===
1194
1195
diff --git a/tests/qemu-iotests/242.out b/tests/qemu-iotests/242.out
1196
index XXXXXXX..XXXXXXX 100644
1197
--- a/tests/qemu-iotests/242.out
1198
+++ b/tests/qemu-iotests/242.out
1199
@@ -XXX,XX +XXX,XX @@ Format specific information:
1200
lazy refcounts: false
1201
refcount bits: 16
1202
corrupt: false
1203
+ extended l2: false
1204
1205
No bitmap in JSON format output
1206
1207
@@ -XXX,XX +XXX,XX @@ Format specific information:
1208
granularity: 32768
1209
refcount bits: 16
1210
corrupt: false
1211
+ extended l2: false
1212
1213
The same bitmaps in JSON format:
1214
[
1215
@@ -XXX,XX +XXX,XX @@ Format specific information:
1216
granularity: 65536
1217
refcount bits: 16
1218
corrupt: false
1219
+ extended l2: false
1220
1221
The same bitmaps in JSON format:
1222
[
1223
@@ -XXX,XX +XXX,XX @@ Format specific information:
1224
granularity: 65536
1225
refcount bits: 16
1226
corrupt: false
1227
+ extended l2: false
1228
1229
The same bitmaps in JSON format:
1230
[
1231
@@ -XXX,XX +XXX,XX @@ Format specific information:
1232
granularity: 16384
1233
refcount bits: 16
1234
corrupt: false
1235
+ extended l2: false
1236
1237
Test complete
1238
diff --git a/tests/qemu-iotests/255.out b/tests/qemu-iotests/255.out
1239
index XXXXXXX..XXXXXXX 100644
1240
--- a/tests/qemu-iotests/255.out
1241
+++ b/tests/qemu-iotests/255.out
1242
@@ -XXX,XX +XXX,XX @@ Finishing a commit job with background reads
1243
1244
=== Create backing chain and start VM ===
1245
1246
-Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
1247
+Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
1248
1249
-Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
1250
+Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
1251
1252
=== Start background read requests ===
1253
1254
@@ -XXX,XX +XXX,XX @@ Closing the VM while a job is being cancelled
1255
1256
=== Create images and start VM ===
1257
1258
-Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
1259
+Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
1260
1261
-Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
1262
+Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
1263
1264
wrote 1048576/1048576 bytes at offset 0
1265
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1266
diff --git a/tests/qemu-iotests/274.out b/tests/qemu-iotests/274.out
1267
index XXXXXXX..XXXXXXX 100644
1268
--- a/tests/qemu-iotests/274.out
1269
+++ b/tests/qemu-iotests/274.out
1270
@@ -XXX,XX +XXX,XX @@
1271
== Commit tests ==
1272
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
1273
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
1274
1275
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1276
+Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1277
1278
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1279
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1280
1281
wrote 2097152/2097152 bytes at offset 0
1282
2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1283
@@ -XXX,XX +XXX,XX @@ Format specific information:
1284
lazy refcounts: false
1285
refcount bits: 16
1286
corrupt: false
1287
+ extended l2: false
1288
1289
read 1048576/1048576 bytes at offset 0
1290
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1291
@@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
1292
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1293
1294
=== Testing HMP commit (top -> mid) ===
1295
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
1296
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
1297
1298
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1299
+Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1300
1301
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1302
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1303
1304
wrote 2097152/2097152 bytes at offset 0
1305
2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1306
@@ -XXX,XX +XXX,XX @@ Format specific information:
1307
lazy refcounts: false
1308
refcount bits: 16
1309
corrupt: false
1310
+ extended l2: false
1311
1312
read 1048576/1048576 bytes at offset 0
1313
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1314
@@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
1315
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1316
1317
=== Testing QMP active commit (top -> mid) ===
1318
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
1319
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
1320
1321
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1322
+Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1323
1324
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1325
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1326
1327
wrote 2097152/2097152 bytes at offset 0
1328
2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1329
@@ -XXX,XX +XXX,XX @@ Format specific information:
1330
lazy refcounts: false
1331
refcount bits: 16
1332
corrupt: false
1333
+ extended l2: false
1334
1335
read 1048576/1048576 bytes at offset 0
1336
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1337
@@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
1338
1339
== Resize tests ==
1340
=== preallocation=off ===
1341
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=6442450944 lazy_refcounts=off refcount_bits=16
1342
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=6442450944 lazy_refcounts=off refcount_bits=16
1343
1344
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1345
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1073741824 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1346
1347
wrote 65536/65536 bytes at offset 5368709120
1348
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1349
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 5368709120
1350
{ "start": 1073741824, "length": 7516192768, "depth": 0, "zero": true, "data": false}]
1351
1352
=== preallocation=metadata ===
1353
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=34359738368 lazy_refcounts=off refcount_bits=16
1354
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=34359738368 lazy_refcounts=off refcount_bits=16
1355
1356
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=32212254720 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1357
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=32212254720 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1358
1359
wrote 65536/65536 bytes at offset 33285996544
1360
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1361
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 33285996544
1362
{ "start": 34896609280, "length": 536870912, "depth": 0, "zero": true, "data": false, "offset": 2685075456}]
1363
1364
=== preallocation=falloc ===
1365
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=10485760 lazy_refcounts=off refcount_bits=16
1366
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=10485760 lazy_refcounts=off refcount_bits=16
1367
1368
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=5242880 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1369
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=5242880 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1370
1371
wrote 65536/65536 bytes at offset 9437184
1372
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1373
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 9437184
1374
{ "start": 5242880, "length": 10485760, "depth": 0, "zero": false, "data": true, "offset": 327680}]
1375
1376
=== preallocation=full ===
1377
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=16777216 lazy_refcounts=off refcount_bits=16
1378
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=16777216 lazy_refcounts=off refcount_bits=16
1379
1380
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=8388608 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1381
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=8388608 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1382
1383
wrote 65536/65536 bytes at offset 11534336
1384
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1385
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 11534336
1386
{ "start": 8388608, "length": 4194304, "depth": 0, "zero": false, "data": true, "offset": 327680}]
1387
1388
=== preallocation=off ===
1389
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=393216 lazy_refcounts=off refcount_bits=16
1390
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=393216 lazy_refcounts=off refcount_bits=16
1391
1392
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=259072 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1393
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=259072 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1394
1395
wrote 65536/65536 bytes at offset 259072
1396
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1397
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 259072
1398
{ "start": 262144, "length": 262144, "depth": 0, "zero": true, "data": false}]
1399
1400
=== preallocation=off ===
1401
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=409600 lazy_refcounts=off refcount_bits=16
1402
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=409600 lazy_refcounts=off refcount_bits=16
1403
1404
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1405
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1406
1407
wrote 65536/65536 bytes at offset 344064
1408
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1409
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 344064
1410
{ "start": 262144, "length": 262144, "depth": 0, "zero": true, "data": false}]
1411
1412
=== preallocation=off ===
1413
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=524288 lazy_refcounts=off refcount_bits=16
1414
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=524288 lazy_refcounts=off refcount_bits=16
1415
1416
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1417
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
1418
1419
wrote 65536/65536 bytes at offset 446464
1420
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
1421
diff --git a/tests/qemu-iotests/280.out b/tests/qemu-iotests/280.out
1422
index XXXXXXX..XXXXXXX 100644
1423
--- a/tests/qemu-iotests/280.out
1424
+++ b/tests/qemu-iotests/280.out
1425
@@ -XXX,XX +XXX,XX @@
1426
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
1427
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
1428
1429
=== Launch VM ===
1430
Enabling migration QMP events on VM...
1431
diff --git a/tests/qemu-iotests/291.out b/tests/qemu-iotests/291.out
1432
index XXXXXXX..XXXXXXX 100644
1433
--- a/tests/qemu-iotests/291.out
1434
+++ b/tests/qemu-iotests/291.out
1435
@@ -XXX,XX +XXX,XX @@ Format specific information:
1436
granularity: 65536
1437
refcount bits: 16
1438
corrupt: false
1439
+ extended l2: false
1440
image: TEST_DIR/t.IMGFMT
1441
file format: IMGFMT
1442
virtual size: 10 MiB (10485760 bytes)
1443
@@ -XXX,XX +XXX,XX @@ Format specific information:
1444
granularity: 65536
1445
refcount bits: 16
1446
corrupt: false
1447
+ extended l2: false
1448
1449
=== Check bitmap contents ===
1450
1451
diff --git a/tests/qemu-iotests/302.out b/tests/qemu-iotests/302.out
1452
index XXXXXXX..XXXXXXX 100644
1453
--- a/tests/qemu-iotests/302.out
1454
+++ b/tests/qemu-iotests/302.out
1455
@@ -XXX,XX +XXX,XX @@ Format specific information:
1456
lazy refcounts: false
1457
refcount bits: 16
1458
corrupt: false
1459
+ extended l2: false
1460
1461
=== Converted image check ===
1462
No errors were found on the image.
1463
diff --git a/tests/qemu-iotests/303.out b/tests/qemu-iotests/303.out
1464
index XXXXXXX..XXXXXXX 100644
1465
--- a/tests/qemu-iotests/303.out
1466
+++ b/tests/qemu-iotests/303.out
1467
@@ -XXX,XX +XXX,XX @@ header_length 112
1468
1469
Header extension:
1470
magic 0x6803f857 (Feature table)
1471
-length 336
1472
+length 384
1473
data <binary>
1474
1475
Header extension:
1476
@@ -XXX,XX +XXX,XX @@ Bitmap table type size offset
1477
{
1478
"name": "Feature table",
1479
"magic": 1745090647,
1480
- "length": 336,
1481
+ "length": 384,
1482
"data_str": "<binary>"
1483
},
1484
{
1485
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
1486
index XXXXXXX..XXXXXXX 100644
1487
--- a/tests/qemu-iotests/common.filter
1488
+++ b/tests/qemu-iotests/common.filter
1489
@@ -XXX,XX +XXX,XX @@ _filter_img_info()
1490
-e "/adapter_type: '[^']*'/d" \
1491
-e "/hwversion: '[^']*'/d" \
1492
-e "/lazy_refcounts: \\(on\\|off\\)/d" \
1493
+ -e "/extended_l2=\\(on\\|off\\)/d" \
1494
-e "/block_size: [0-9]\\+/d" \
1495
-e "/block_state_zero: \\(on\\|off\\)/d" \
1496
-e "/log_size: [0-9]\\+/d" \
1497
--
126
--
1498
2.26.2
127
2.34.1
1499
1500
diff view generated by jsdifflib
[PULL 04/34] qcow2: Split cluster_needs_cow() out of count_cow_clusters()
[PULL 12/23] block/block-copy: add block_copy_reset()
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
We are going to need it in other places.
3
Split block_copy_reset() out of block_copy_reset_unallocated() to be
4
used separately later.
4
5
5
Signed-off-by: Alberto Garcia <berto@igalia.com>
6
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
6
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
7
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
7
Reviewed-by: Max Reitz <mreitz@redhat.com>
8
Message-Id: <20220303194349.2304213-6-vsementsov@virtuozzo.com>
8
Message-Id: <65e5d9627ca2ebe7e62deaeddf60949c33067d9d.1594396418.git.berto@igalia.com>
9
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
9
Signed-off-by: Max Reitz <mreitz@redhat.com>
10
---
10
---
11
block/qcow2-cluster.c | 34 +++++++++++++++++++---------------
11
include/block/block-copy.h | 1 +
12
1 file changed, 19 insertions(+), 15 deletions(-)
12
block/block-copy.c | 21 +++++++++++++--------
13
2 files changed, 14 insertions(+), 8 deletions(-)
13
14
14
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
15
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
15
index XXXXXXX..XXXXXXX 100644
16
index XXXXXXX..XXXXXXX 100644
16
--- a/block/qcow2-cluster.c
17
--- a/include/block/block-copy.h
17
+++ b/block/qcow2-cluster.c
18
+++ b/include/block/block-copy.h
18
@@ -XXX,XX +XXX,XX @@ static void calculate_l2_meta(BlockDriverState *bs,
19
@@ -XXX,XX +XXX,XX @@ void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm);
19
QLIST_INSERT_HEAD(&s->cluster_allocs, *m, next_in_flight);
20
21
void block_copy_state_free(BlockCopyState *s);
22
23
+void block_copy_reset(BlockCopyState *s, int64_t offset, int64_t bytes);
24
int64_t block_copy_reset_unallocated(BlockCopyState *s,
25
int64_t offset, int64_t *count);
26
27
diff --git a/block/block-copy.c b/block/block-copy.c
28
index XXXXXXX..XXXXXXX 100644
29
--- a/block/block-copy.c
30
+++ b/block/block-copy.c
31
@@ -XXX,XX +XXX,XX @@ static int block_copy_is_cluster_allocated(BlockCopyState *s, int64_t offset,
32
}
20
}
33
}
21
34
22
+/* Returns true if writing to a cluster requires COW */
35
+void block_copy_reset(BlockCopyState *s, int64_t offset, int64_t bytes)
23
+static bool cluster_needs_cow(BlockDriverState *bs, uint64_t l2_entry)
24
+{
36
+{
25
+ switch (qcow2_get_cluster_type(bs, l2_entry)) {
37
+ QEMU_LOCK_GUARD(&s->lock);
26
+ case QCOW2_CLUSTER_NORMAL:
38
+
27
+ if (l2_entry & QCOW_OFLAG_COPIED) {
39
+ bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes);
28
+ return false;
40
+ if (s->progress) {
29
+ }
41
+ progress_set_remaining(s->progress,
30
+ case QCOW2_CLUSTER_UNALLOCATED:
42
+ bdrv_get_dirty_count(s->copy_bitmap) +
31
+ case QCOW2_CLUSTER_COMPRESSED:
43
+ s->in_flight_bytes);
32
+ case QCOW2_CLUSTER_ZERO_PLAIN:
33
+ case QCOW2_CLUSTER_ZERO_ALLOC:
34
+ return true;
35
+ default:
36
+ abort();
37
+ }
44
+ }
38
+}
45
+}
39
+
46
+
40
/*
47
/*
41
* Returns the number of contiguous clusters that can be used for an allocating
48
* Reset bits in copy_bitmap starting at offset if they represent unallocated
42
* write, but require COW to be performed (this includes yet unallocated space,
49
* data in the image. May reset subsequent contiguous bits.
43
@@ -XXX,XX +XXX,XX @@ static int count_cow_clusters(BlockDriverState *bs, int nb_clusters,
50
@@ -XXX,XX +XXX,XX @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,
44
51
bytes = clusters * s->cluster_size;
45
for (i = 0; i < nb_clusters; i++) {
52
46
uint64_t l2_entry = be64_to_cpu(l2_slice[l2_index + i]);
53
if (!ret) {
47
- QCow2ClusterType cluster_type = qcow2_get_cluster_type(bs, l2_entry);
54
- qemu_co_mutex_lock(&s->lock);
48
-
55
- bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes);
49
- switch(cluster_type) {
56
- if (s->progress) {
50
- case QCOW2_CLUSTER_NORMAL:
57
- progress_set_remaining(s->progress,
51
- if (l2_entry & QCOW_OFLAG_COPIED) {
58
- bdrv_get_dirty_count(s->copy_bitmap) +
52
- goto out;
59
- s->in_flight_bytes);
53
- }
60
- }
54
+ if (!cluster_needs_cow(bs, l2_entry)) {
61
- qemu_co_mutex_unlock(&s->lock);
55
break;
62
+ block_copy_reset(s, offset, bytes);
56
- case QCOW2_CLUSTER_UNALLOCATED:
57
- case QCOW2_CLUSTER_COMPRESSED:
58
- case QCOW2_CLUSTER_ZERO_PLAIN:
59
- case QCOW2_CLUSTER_ZERO_ALLOC:
60
- break;
61
- default:
62
- abort();
63
}
64
}
63
}
65
64
66
-out:
65
*count = bytes;
67
assert(i <= nb_clusters);
68
return i;
69
}
70
--
66
--
71
2.26.2
67
2.34.1
72
73
diff view generated by jsdifflib
[PULL 13/34] qcow2: Update get/set_l2_entry() and add get/set_l2_bitmap()
[PULL 13/23] block: intoduce reqlist
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
Extended L2 entries are 128-bit wide: 64 bits for the entry itself and
3
Split intersecting-requests functionality out of block-copy to be
4
64 bits for the subcluster allocation bitmap.
4
reused in copy-before-write filter.
5
5
6
In order to support them correctly get/set_l2_entry() need to be
6
Note: while being here, fix tiny typo in MAINTAINERS.
7
updated so they take the entry width into account in order to
8
calculate the correct offset.
9
7
10
This patch also adds the get/set_l2_bitmap() functions that are
8
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
11
used to access the bitmaps. For convenience we allow calling
9
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
12
get_l2_bitmap() on images without subclusters. In this case the
10
Message-Id: <20220303194349.2304213-7-vsementsov@virtuozzo.com>
13
returned value is always 0 and has no meaning.
11
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
12
---
13
include/block/reqlist.h | 67 +++++++++++++++++++++++
14
block/block-copy.c | 116 +++++++++++++---------------------------
15
block/reqlist.c | 76 ++++++++++++++++++++++++++
16
MAINTAINERS | 4 +-
17
block/meson.build | 1 +
18
5 files changed, 184 insertions(+), 80 deletions(-)
19
create mode 100644 include/block/reqlist.h
20
create mode 100644 block/reqlist.c
14
21
15
Signed-off-by: Alberto Garcia <berto@igalia.com>
22
diff --git a/include/block/reqlist.h b/include/block/reqlist.h
16
Reviewed-by: Eric Blake <eblake@redhat.com>
23
new file mode 100644
17
Reviewed-by: Max Reitz <mreitz@redhat.com>
24
index XXXXXXX..XXXXXXX
18
Message-Id: <6ee0f81ae3329c991de125618b3675e1e46acdbb.1594396418.git.berto@igalia.com>
25
--- /dev/null
19
Signed-off-by: Max Reitz <mreitz@redhat.com>
26
+++ b/include/block/reqlist.h
20
---
27
@@ -XXX,XX +XXX,XX @@
21
block/qcow2.h | 21 +++++++++++++++++++++
28
+/*
22
1 file changed, 21 insertions(+)
29
+ * reqlist API
23
30
+ *
24
diff --git a/block/qcow2.h b/block/qcow2.h
31
+ * Copyright (C) 2013 Proxmox Server Solutions
32
+ * Copyright (c) 2021 Virtuozzo International GmbH.
33
+ *
34
+ * Authors:
35
+ * Dietmar Maurer (dietmar@proxmox.com)
36
+ * Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
37
+ *
38
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
39
+ * See the COPYING file in the top-level directory.
40
+ */
41
+
42
+#ifndef REQLIST_H
43
+#define REQLIST_H
44
+
45
+#include "qemu/coroutine.h"
46
+
47
+/*
48
+ * The API is not thread-safe and shouldn't be. The struct is public to be part
49
+ * of other structures and protected by third-party locks, see
50
+ * block/block-copy.c for example.
51
+ */
52
+
53
+typedef struct BlockReq {
54
+ int64_t offset;
55
+ int64_t bytes;
56
+
57
+ CoQueue wait_queue; /* coroutines blocked on this req */
58
+ QLIST_ENTRY(BlockReq) list;
59
+} BlockReq;
60
+
61
+typedef QLIST_HEAD(, BlockReq) BlockReqList;
62
+
63
+/*
64
+ * Initialize new request and add it to the list. Caller must be sure that
65
+ * there are no conflicting requests in the list.
66
+ */
67
+void reqlist_init_req(BlockReqList *reqs, BlockReq *req, int64_t offset,
68
+ int64_t bytes);
69
+/* Search for request in the list intersecting with @offset/@bytes area. */
70
+BlockReq *reqlist_find_conflict(BlockReqList *reqs, int64_t offset,
71
+ int64_t bytes);
72
+
73
+/*
74
+ * If there are no intersecting requests return false. Otherwise, wait for the
75
+ * first found intersecting request to finish and return true.
76
+ *
77
+ * @lock is passed to qemu_co_queue_wait()
78
+ * False return value proves that lock was released at no point.
79
+ */
80
+bool coroutine_fn reqlist_wait_one(BlockReqList *reqs, int64_t offset,
81
+ int64_t bytes, CoMutex *lock);
82
+
83
+/*
84
+ * Shrink request and wake all waiting coroutines (maybe some of them are not
85
+ * intersecting with shrunk request).
86
+ */
87
+void coroutine_fn reqlist_shrink_req(BlockReq *req, int64_t new_bytes);
88
+
89
+/*
90
+ * Remove request and wake all waiting coroutines. Do not release any memory.
91
+ */
92
+void coroutine_fn reqlist_remove_req(BlockReq *req);
93
+
94
+#endif /* REQLIST_H */
95
diff --git a/block/block-copy.c b/block/block-copy.c
25
index XXXXXXX..XXXXXXX 100644
96
index XXXXXXX..XXXXXXX 100644
26
--- a/block/qcow2.h
97
--- a/block/block-copy.c
27
+++ b/block/qcow2.h
98
+++ b/block/block-copy.c
28
@@ -XXX,XX +XXX,XX @@ static inline size_t l2_entry_size(BDRVQcow2State *s)
99
@@ -XXX,XX +XXX,XX @@
29
static inline uint64_t get_l2_entry(BDRVQcow2State *s, uint64_t *l2_slice,
100
#include "trace.h"
30
int idx)
101
#include "qapi/error.h"
102
#include "block/block-copy.h"
103
+#include "block/reqlist.h"
104
#include "sysemu/block-backend.h"
105
#include "qemu/units.h"
106
#include "qemu/coroutine.h"
107
@@ -XXX,XX +XXX,XX @@ typedef struct BlockCopyTask {
108
*/
109
BlockCopyState *s;
110
BlockCopyCallState *call_state;
111
- int64_t offset;
112
/*
113
* @method can also be set again in the while loop of
114
* block_copy_dirty_clusters(), but it is never accessed concurrently
115
@@ -XXX,XX +XXX,XX @@ typedef struct BlockCopyTask {
116
BlockCopyMethod method;
117
118
/*
119
- * Fields whose state changes throughout the execution
120
- * Protected by lock in BlockCopyState.
121
+ * Generally, req is protected by lock in BlockCopyState, Still req.offset
122
+ * is only set on task creation, so may be read concurrently after creation.
123
+ * req.bytes is changed at most once, and need only protecting the case of
124
+ * parallel read while updating @bytes value in block_copy_task_shrink().
125
*/
126
- CoQueue wait_queue; /* coroutines blocked on this task */
127
- /*
128
- * Only protect the case of parallel read while updating @bytes
129
- * value in block_copy_task_shrink().
130
- */
131
- int64_t bytes;
132
- QLIST_ENTRY(BlockCopyTask) list;
133
+ BlockReq req;
134
} BlockCopyTask;
135
136
static int64_t task_end(BlockCopyTask *task)
31
{
137
{
32
+ idx *= l2_entry_size(s) / sizeof(uint64_t);
138
- return task->offset + task->bytes;
33
return be64_to_cpu(l2_slice[idx]);
139
+ return task->req.offset + task->req.bytes;
34
}
140
}
35
141
36
+static inline uint64_t get_l2_bitmap(BDRVQcow2State *s, uint64_t *l2_slice,
142
typedef struct BlockCopyState {
37
+ int idx)
143
@@ -XXX,XX +XXX,XX @@ typedef struct BlockCopyState {
144
CoMutex lock;
145
int64_t in_flight_bytes;
146
BlockCopyMethod method;
147
- QLIST_HEAD(, BlockCopyTask) tasks; /* All tasks from all block-copy calls */
148
+ BlockReqList reqs;
149
QLIST_HEAD(, BlockCopyCallState) calls;
150
/*
151
* skip_unallocated:
152
@@ -XXX,XX +XXX,XX @@ typedef struct BlockCopyState {
153
RateLimit rate_limit;
154
} BlockCopyState;
155
156
-/* Called with lock held */
157
-static BlockCopyTask *find_conflicting_task(BlockCopyState *s,
158
- int64_t offset, int64_t bytes)
159
-{
160
- BlockCopyTask *t;
161
-
162
- QLIST_FOREACH(t, &s->tasks, list) {
163
- if (offset + bytes > t->offset && offset < t->offset + t->bytes) {
164
- return t;
165
- }
166
- }
167
-
168
- return NULL;
169
-}
170
-
171
-/*
172
- * If there are no intersecting tasks return false. Otherwise, wait for the
173
- * first found intersecting tasks to finish and return true.
174
- *
175
- * Called with lock held. May temporary release the lock.
176
- * Return value of 0 proves that lock was NOT released.
177
- */
178
-static bool coroutine_fn block_copy_wait_one(BlockCopyState *s, int64_t offset,
179
- int64_t bytes)
180
-{
181
- BlockCopyTask *task = find_conflicting_task(s, offset, bytes);
182
-
183
- if (!task) {
184
- return false;
185
- }
186
-
187
- qemu_co_queue_wait(&task->wait_queue, &s->lock);
188
-
189
- return true;
190
-}
191
-
192
/* Called with lock held */
193
static int64_t block_copy_chunk_size(BlockCopyState *s)
194
{
195
@@ -XXX,XX +XXX,XX @@ block_copy_task_create(BlockCopyState *s, BlockCopyCallState *call_state,
196
bytes = QEMU_ALIGN_UP(bytes, s->cluster_size);
197
198
/* region is dirty, so no existent tasks possible in it */
199
- assert(!find_conflicting_task(s, offset, bytes));
200
+ assert(!reqlist_find_conflict(&s->reqs, offset, bytes));
201
202
bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes);
203
s->in_flight_bytes += bytes;
204
@@ -XXX,XX +XXX,XX @@ block_copy_task_create(BlockCopyState *s, BlockCopyCallState *call_state,
205
.task.func = block_copy_task_entry,
206
.s = s,
207
.call_state = call_state,
208
- .offset = offset,
209
- .bytes = bytes,
210
.method = s->method,
211
};
212
- qemu_co_queue_init(&task->wait_queue);
213
- QLIST_INSERT_HEAD(&s->tasks, task, list);
214
+ reqlist_init_req(&s->reqs, &task->req, offset, bytes);
215
216
return task;
217
}
218
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn block_copy_task_shrink(BlockCopyTask *task,
219
int64_t new_bytes)
220
{
221
QEMU_LOCK_GUARD(&task->s->lock);
222
- if (new_bytes == task->bytes) {
223
+ if (new_bytes == task->req.bytes) {
224
return;
225
}
226
227
- assert(new_bytes > 0 && new_bytes < task->bytes);
228
+ assert(new_bytes > 0 && new_bytes < task->req.bytes);
229
230
- task->s->in_flight_bytes -= task->bytes - new_bytes;
231
+ task->s->in_flight_bytes -= task->req.bytes - new_bytes;
232
bdrv_set_dirty_bitmap(task->s->copy_bitmap,
233
- task->offset + new_bytes, task->bytes - new_bytes);
234
+ task->req.offset + new_bytes,
235
+ task->req.bytes - new_bytes);
236
237
- task->bytes = new_bytes;
238
- qemu_co_queue_restart_all(&task->wait_queue);
239
+ reqlist_shrink_req(&task->req, new_bytes);
240
}
241
242
static void coroutine_fn block_copy_task_end(BlockCopyTask *task, int ret)
243
{
244
QEMU_LOCK_GUARD(&task->s->lock);
245
- task->s->in_flight_bytes -= task->bytes;
246
+ task->s->in_flight_bytes -= task->req.bytes;
247
if (ret < 0) {
248
- bdrv_set_dirty_bitmap(task->s->copy_bitmap, task->offset, task->bytes);
249
+ bdrv_set_dirty_bitmap(task->s->copy_bitmap, task->req.offset,
250
+ task->req.bytes);
251
}
252
- QLIST_REMOVE(task, list);
253
if (task->s->progress) {
254
progress_set_remaining(task->s->progress,
255
bdrv_get_dirty_count(task->s->copy_bitmap) +
256
task->s->in_flight_bytes);
257
}
258
- qemu_co_queue_restart_all(&task->wait_queue);
259
+ reqlist_remove_req(&task->req);
260
}
261
262
void block_copy_state_free(BlockCopyState *s)
263
@@ -XXX,XX +XXX,XX @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
264
265
ratelimit_init(&s->rate_limit);
266
qemu_co_mutex_init(&s->lock);
267
- QLIST_INIT(&s->tasks);
268
+ QLIST_INIT(&s->reqs);
269
QLIST_INIT(&s->calls);
270
271
return s;
272
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int block_copy_task_run(AioTaskPool *pool,
273
274
aio_task_pool_wait_slot(pool);
275
if (aio_task_pool_status(pool) < 0) {
276
- co_put_to_shres(task->s->mem, task->bytes);
277
+ co_put_to_shres(task->s->mem, task->req.bytes);
278
block_copy_task_end(task, -ECANCELED);
279
g_free(task);
280
return -ECANCELED;
281
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int block_copy_task_entry(AioTask *task)
282
BlockCopyMethod method = t->method;
283
int ret;
284
285
- ret = block_copy_do_copy(s, t->offset, t->bytes, &method, &error_is_read);
286
+ ret = block_copy_do_copy(s, t->req.offset, t->req.bytes, &method,
287
+ &error_is_read);
288
289
WITH_QEMU_LOCK_GUARD(&s->lock) {
290
if (s->method == t->method) {
291
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int block_copy_task_entry(AioTask *task)
292
t->call_state->error_is_read = error_is_read;
293
}
294
} else if (s->progress) {
295
- progress_work_done(s->progress, t->bytes);
296
+ progress_work_done(s->progress, t->req.bytes);
297
}
298
}
299
- co_put_to_shres(s->mem, t->bytes);
300
+ co_put_to_shres(s->mem, t->req.bytes);
301
block_copy_task_end(t, ret);
302
303
return ret;
304
@@ -XXX,XX +XXX,XX @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
305
trace_block_copy_skip_range(s, offset, bytes);
306
break;
307
}
308
- if (task->offset > offset) {
309
- trace_block_copy_skip_range(s, offset, task->offset - offset);
310
+ if (task->req.offset > offset) {
311
+ trace_block_copy_skip_range(s, offset, task->req.offset - offset);
312
}
313
314
found_dirty = true;
315
316
- ret = block_copy_block_status(s, task->offset, task->bytes,
317
+ ret = block_copy_block_status(s, task->req.offset, task->req.bytes,
318
&status_bytes);
319
assert(ret >= 0); /* never fail */
320
- if (status_bytes < task->bytes) {
321
+ if (status_bytes < task->req.bytes) {
322
block_copy_task_shrink(task, status_bytes);
323
}
324
if (qatomic_read(&s->skip_unallocated) &&
325
!(ret & BDRV_BLOCK_ALLOCATED)) {
326
block_copy_task_end(task, 0);
327
- trace_block_copy_skip_range(s, task->offset, task->bytes);
328
+ trace_block_copy_skip_range(s, task->req.offset, task->req.bytes);
329
offset = task_end(task);
330
bytes = end - offset;
331
g_free(task);
332
@@ -XXX,XX +XXX,XX @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
333
}
334
}
335
336
- ratelimit_calculate_delay(&s->rate_limit, task->bytes);
337
+ ratelimit_calculate_delay(&s->rate_limit, task->req.bytes);
338
339
- trace_block_copy_process(s, task->offset);
340
+ trace_block_copy_process(s, task->req.offset);
341
342
- co_get_from_shres(s->mem, task->bytes);
343
+ co_get_from_shres(s->mem, task->req.bytes);
344
345
offset = task_end(task);
346
bytes = end - offset;
347
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
348
* Check that there is no task we still need to
349
* wait to complete
350
*/
351
- ret = block_copy_wait_one(s, call_state->offset,
352
- call_state->bytes);
353
+ ret = reqlist_wait_one(&s->reqs, call_state->offset,
354
+ call_state->bytes, &s->lock);
355
if (ret == 0) {
356
/*
357
* No pending tasks, but check again the bitmap in this
358
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
359
* between this and the critical section in
360
* block_copy_dirty_clusters().
361
*
362
- * block_copy_wait_one return value 0 also means that it
363
+ * reqlist_wait_one return value 0 also means that it
364
* didn't release the lock. So, we are still in the same
365
* critical section, not interrupted by any concurrent
366
* access to state.
367
diff --git a/block/reqlist.c b/block/reqlist.c
368
new file mode 100644
369
index XXXXXXX..XXXXXXX
370
--- /dev/null
371
+++ b/block/reqlist.c
372
@@ -XXX,XX +XXX,XX @@
373
+/*
374
+ * reqlist API
375
+ *
376
+ * Copyright (C) 2013 Proxmox Server Solutions
377
+ * Copyright (c) 2021 Virtuozzo International GmbH.
378
+ *
379
+ * Authors:
380
+ * Dietmar Maurer (dietmar@proxmox.com)
381
+ * Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
382
+ *
383
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
384
+ * See the COPYING file in the top-level directory.
385
+ */
386
+
387
+#include "qemu/osdep.h"
388
+
389
+#include "block/reqlist.h"
390
+
391
+void reqlist_init_req(BlockReqList *reqs, BlockReq *req, int64_t offset,
392
+ int64_t bytes)
38
+{
393
+{
39
+ if (has_subclusters(s)) {
394
+ assert(!reqlist_find_conflict(reqs, offset, bytes));
40
+ idx *= l2_entry_size(s) / sizeof(uint64_t);
395
+
41
+ return be64_to_cpu(l2_slice[idx + 1]);
396
+ *req = (BlockReq) {
42
+ } else {
397
+ .offset = offset,
43
+ return 0; /* For convenience only; this value has no meaning. */
398
+ .bytes = bytes,
399
+ };
400
+ qemu_co_queue_init(&req->wait_queue);
401
+ QLIST_INSERT_HEAD(reqs, req, list);
402
+}
403
+
404
+BlockReq *reqlist_find_conflict(BlockReqList *reqs, int64_t offset,
405
+ int64_t bytes)
406
+{
407
+ BlockReq *r;
408
+
409
+ QLIST_FOREACH(r, reqs, list) {
410
+ if (offset + bytes > r->offset && offset < r->offset + r->bytes) {
411
+ return r;
412
+ }
44
+ }
413
+ }
414
+
415
+ return NULL;
45
+}
416
+}
46
+
417
+
47
static inline void set_l2_entry(BDRVQcow2State *s, uint64_t *l2_slice,
418
+bool coroutine_fn reqlist_wait_one(BlockReqList *reqs, int64_t offset,
48
int idx, uint64_t entry)
419
+ int64_t bytes, CoMutex *lock)
49
{
50
+ idx *= l2_entry_size(s) / sizeof(uint64_t);
51
l2_slice[idx] = cpu_to_be64(entry);
52
}
53
54
+static inline void set_l2_bitmap(BDRVQcow2State *s, uint64_t *l2_slice,
55
+ int idx, uint64_t bitmap)
56
+{
420
+{
57
+ assert(has_subclusters(s));
421
+ BlockReq *r = reqlist_find_conflict(reqs, offset, bytes);
58
+ idx *= l2_entry_size(s) / sizeof(uint64_t);
422
+
59
+ l2_slice[idx + 1] = cpu_to_be64(bitmap);
423
+ if (!r) {
424
+ return false;
425
+ }
426
+
427
+ qemu_co_queue_wait(&r->wait_queue, lock);
428
+
429
+ return true;
60
+}
430
+}
61
+
431
+
62
static inline bool has_data_file(BlockDriverState *bs)
432
+void coroutine_fn reqlist_shrink_req(BlockReq *req, int64_t new_bytes)
63
{
433
+{
64
BDRVQcow2State *s = bs->opaque;
434
+ if (new_bytes == req->bytes) {
435
+ return;
436
+ }
437
+
438
+ assert(new_bytes > 0 && new_bytes < req->bytes);
439
+
440
+ req->bytes = new_bytes;
441
+ qemu_co_queue_restart_all(&req->wait_queue);
442
+}
443
+
444
+void coroutine_fn reqlist_remove_req(BlockReq *req)
445
+{
446
+ QLIST_REMOVE(req, list);
447
+ qemu_co_queue_restart_all(&req->wait_queue);
448
+}
449
diff --git a/MAINTAINERS b/MAINTAINERS
450
index XXXXXXX..XXXXXXX 100644
451
--- a/MAINTAINERS
452
+++ b/MAINTAINERS
453
@@ -XXX,XX +XXX,XX @@ F: block/stream.c
454
F: block/mirror.c
455
F: qapi/job.json
456
F: block/block-copy.c
457
-F: include/block/block-copy.c
458
+F: include/block/block-copy.h
459
+F: block/reqlist.c
460
+F: include/block/reqlist.h
461
F: block/copy-before-write.h
462
F: block/copy-before-write.c
463
F: include/block/aio_task.h
464
diff --git a/block/meson.build b/block/meson.build
465
index XXXXXXX..XXXXXXX 100644
466
--- a/block/meson.build
467
+++ b/block/meson.build
468
@@ -XXX,XX +XXX,XX @@ block_ss.add(files(
469
'qcow2.c',
470
'quorum.c',
471
'raw-format.c',
472
+ 'reqlist.c',
473
'snapshot.c',
474
'throttle-groups.c',
475
'throttle.c',
65
--
476
--
66
2.26.2
477
2.34.1
67
68
diff view generated by jsdifflib
[PULL 23/34] qcow2: Add subcluster support to discard_in_l2_slice()
[PULL 14/23] block/reqlist: reqlist_find_conflict(): use ranges_overlap()
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
Two things need to be taken into account here:
3
Let's reuse convenient helper.
4
4
5
1) With full_discard == true the L2 entry must be cleared completely.
5
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
6
This also includes the L2 bitmap if the image has extended L2
6
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
7
entries.
7
Message-Id: <20220303194349.2304213-8-vsementsov@virtuozzo.com>
8
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
9
---
10
block/reqlist.c | 3 ++-
11
1 file changed, 2 insertions(+), 1 deletion(-)
8
12
9
2) With full_discard == false we have to make the discarded cluster
13
diff --git a/block/reqlist.c b/block/reqlist.c
10
read back as zeroes. With normal L2 entries this is done with the
11
QCOW_OFLAG_ZERO bit, whereas with extended L2 entries this is done
12
with the individual 'all zeroes' bits for each subcluster.
13
14
Note however that QCOW_OFLAG_ZERO is not supported in v2 qcow2
15
images so, if there is a backing file, discard cannot guarantee
16
that the image will read back as zeroes. If this is important for
17
the caller it should forbid it as qcow2_co_pdiscard() does (see
18
80f5c01183 for more details).
19
20
Signed-off-by: Alberto Garcia <berto@igalia.com>
21
Reviewed-by: Eric Blake <eblake@redhat.com>
22
Reviewed-by: Max Reitz <mreitz@redhat.com>
23
Message-Id: <5ef8274e628aa3ab559bfac467abf488534f2b76.1594396418.git.berto@igalia.com>
24
Signed-off-by: Max Reitz <mreitz@redhat.com>
25
---
26
block/qcow2-cluster.c | 52 +++++++++++++++++++------------------------
27
1 file changed, 23 insertions(+), 29 deletions(-)
28
29
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
30
index XXXXXXX..XXXXXXX 100644
14
index XXXXXXX..XXXXXXX 100644
31
--- a/block/qcow2-cluster.c
15
--- a/block/reqlist.c
32
+++ b/block/qcow2-cluster.c
16
+++ b/block/reqlist.c
33
@@ -XXX,XX +XXX,XX @@ static int discard_in_l2_slice(BlockDriverState *bs, uint64_t offset,
17
@@ -XXX,XX +XXX,XX @@
34
assert(nb_clusters <= INT_MAX);
18
*/
35
19
36
for (i = 0; i < nb_clusters; i++) {
20
#include "qemu/osdep.h"
37
- uint64_t old_l2_entry;
21
+#include "qemu/range.h"
38
-
22
39
- old_l2_entry = get_l2_entry(s, l2_slice, l2_index + i);
23
#include "block/reqlist.h"
40
+ uint64_t old_l2_entry = get_l2_entry(s, l2_slice, l2_index + i);
24
41
+ uint64_t old_l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index + i);
25
@@ -XXX,XX +XXX,XX @@ BlockReq *reqlist_find_conflict(BlockReqList *reqs, int64_t offset,
42
+ uint64_t new_l2_entry = old_l2_entry;
26
BlockReq *r;
43
+ uint64_t new_l2_bitmap = old_l2_bitmap;
27
44
+ QCow2ClusterType cluster_type =
28
QLIST_FOREACH(r, reqs, list) {
45
+ qcow2_get_cluster_type(bs, old_l2_entry);
29
- if (offset + bytes > r->offset && offset < r->offset + r->bytes) {
46
30
+ if (ranges_overlap(offset, bytes, r->offset, r->bytes)) {
47
/*
31
return r;
48
+ * If full_discard is true, the cluster should not read back as zeroes,
49
+ * but rather fall through to the backing file.
50
+ *
51
* If full_discard is false, make sure that a discarded area reads back
52
* as zeroes for v3 images (we cannot do it for v2 without actually
53
* writing a zero-filled buffer). We can skip the operation if the
54
@@ -XXX,XX +XXX,XX @@ static int discard_in_l2_slice(BlockDriverState *bs, uint64_t offset,
55
*
56
* TODO We might want to use bdrv_block_status(bs) here, but we're
57
* holding s->lock, so that doesn't work today.
58
- *
59
- * If full_discard is true, the sector should not read back as zeroes,
60
- * but rather fall through to the backing file.
61
*/
62
- switch (qcow2_get_cluster_type(bs, old_l2_entry)) {
63
- case QCOW2_CLUSTER_UNALLOCATED:
64
- if (full_discard || !bs->backing) {
65
- continue;
66
- }
67
- break;
68
-
69
- case QCOW2_CLUSTER_ZERO_PLAIN:
70
- if (!full_discard) {
71
- continue;
72
+ if (full_discard) {
73
+ new_l2_entry = new_l2_bitmap = 0;
74
+ } else if (bs->backing || qcow2_cluster_is_allocated(cluster_type)) {
75
+ if (has_subclusters(s)) {
76
+ new_l2_entry = 0;
77
+ new_l2_bitmap = QCOW_L2_BITMAP_ALL_ZEROES;
78
+ } else {
79
+ new_l2_entry = s->qcow_version >= 3 ? QCOW_OFLAG_ZERO : 0;
80
}
81
- break;
82
-
83
- case QCOW2_CLUSTER_ZERO_ALLOC:
84
- case QCOW2_CLUSTER_NORMAL:
85
- case QCOW2_CLUSTER_COMPRESSED:
86
- break;
87
+ }
88
89
- default:
90
- abort();
91
+ if (old_l2_entry == new_l2_entry && old_l2_bitmap == new_l2_bitmap) {
92
+ continue;
93
}
32
}
94
95
/* First remove L2 entries */
96
qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice);
97
- if (!full_discard && s->qcow_version >= 3) {
98
- set_l2_entry(s, l2_slice, l2_index + i, QCOW_OFLAG_ZERO);
99
- } else {
100
- set_l2_entry(s, l2_slice, l2_index + i, 0);
101
+ set_l2_entry(s, l2_slice, l2_index + i, new_l2_entry);
102
+ if (has_subclusters(s)) {
103
+ set_l2_bitmap(s, l2_slice, l2_index + i, new_l2_bitmap);
104
}
105
-
106
/* Then decrease the refcount */
107
qcow2_free_any_clusters(bs, old_l2_entry, 1, type);
108
}
33
}
109
--
34
--
110
2.26.2
35
2.34.1
111
112
diff view generated by jsdifflib
[PULL 11/34] qcow2: Add offset_into_subcluster() and size_to_subclusters()
[PULL 15/23] block/dirty-bitmap: introduce bdrv_dirty_bitmap_status()
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
Like offset_into_cluster() and size_to_clusters(), but for
3
Add a convenient function similar with bdrv_block_status() to get
4
subclusters.
4
status of dirty bitmap.
5
5
6
Signed-off-by: Alberto Garcia <berto@igalia.com>
6
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
7
Reviewed-by: Eric Blake <eblake@redhat.com>
7
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
8
Reviewed-by: Max Reitz <mreitz@redhat.com>
8
Message-Id: <20220303194349.2304213-9-vsementsov@virtuozzo.com>
9
Message-Id: <3cc2390dcdef3d234d47c741b708bd8734490862.1594396418.git.berto@igalia.com>
9
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
10
Signed-off-by: Max Reitz <mreitz@redhat.com>
11
---
10
---
12
block/qcow2.h | 10 ++++++++++
11
include/block/dirty-bitmap.h | 2 ++
13
1 file changed, 10 insertions(+)
12
include/qemu/hbitmap.h | 12 ++++++++++++
13
block/dirty-bitmap.c | 6 ++++++
14
util/hbitmap.c | 33 +++++++++++++++++++++++++++++++++
15
4 files changed, 53 insertions(+)
14
16
15
diff --git a/block/qcow2.h b/block/qcow2.h
17
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
16
index XXXXXXX..XXXXXXX 100644
18
index XXXXXXX..XXXXXXX 100644
17
--- a/block/qcow2.h
19
--- a/include/block/dirty-bitmap.h
18
+++ b/block/qcow2.h
20
+++ b/include/block/dirty-bitmap.h
19
@@ -XXX,XX +XXX,XX @@ static inline int64_t offset_into_cluster(BDRVQcow2State *s, int64_t offset)
21
@@ -XXX,XX +XXX,XX @@ int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, int64_t offset,
20
return offset & (s->cluster_size - 1);
22
bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
23
int64_t start, int64_t end, int64_t max_dirty_count,
24
int64_t *dirty_start, int64_t *dirty_count);
25
+bool bdrv_dirty_bitmap_status(BdrvDirtyBitmap *bitmap, int64_t offset,
26
+ int64_t bytes, int64_t *count);
27
BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
28
Error **errp);
29
30
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
31
index XXXXXXX..XXXXXXX 100644
32
--- a/include/qemu/hbitmap.h
33
+++ b/include/qemu/hbitmap.h
34
@@ -XXX,XX +XXX,XX @@ bool hbitmap_next_dirty_area(const HBitmap *hb, int64_t start, int64_t end,
35
int64_t max_dirty_count,
36
int64_t *dirty_start, int64_t *dirty_count);
37
38
+/*
39
+ * bdrv_dirty_bitmap_status:
40
+ * @hb: The HBitmap to operate on
41
+ * @start: The bit to start from
42
+ * @count: Number of bits to proceed
43
+ * @pnum: Out-parameter. How many bits has same value starting from @start
44
+ *
45
+ * Returns true if bitmap is dirty at @start, false otherwise.
46
+ */
47
+bool hbitmap_status(const HBitmap *hb, int64_t start, int64_t count,
48
+ int64_t *pnum);
49
+
50
/**
51
* hbitmap_iter_next:
52
* @hbi: HBitmapIter to operate on.
53
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
54
index XXXXXXX..XXXXXXX 100644
55
--- a/block/dirty-bitmap.c
56
+++ b/block/dirty-bitmap.c
57
@@ -XXX,XX +XXX,XX @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
58
dirty_start, dirty_count);
21
}
59
}
22
60
23
+static inline int64_t offset_into_subcluster(BDRVQcow2State *s, int64_t offset)
61
+bool bdrv_dirty_bitmap_status(BdrvDirtyBitmap *bitmap, int64_t offset,
62
+ int64_t bytes, int64_t *count)
24
+{
63
+{
25
+ return offset & (s->subcluster_size - 1);
64
+ return hbitmap_status(bitmap->bitmap, offset, bytes, count);
26
+}
65
+}
27
+
66
+
28
static inline uint64_t size_to_clusters(BDRVQcow2State *s, uint64_t size)
67
/**
29
{
68
* bdrv_merge_dirty_bitmap: merge src into dest.
30
return (size + (s->cluster_size - 1)) >> s->cluster_bits;
69
* Ensures permissions on bitmaps are reasonable; use for public API.
70
diff --git a/util/hbitmap.c b/util/hbitmap.c
71
index XXXXXXX..XXXXXXX 100644
72
--- a/util/hbitmap.c
73
+++ b/util/hbitmap.c
74
@@ -XXX,XX +XXX,XX @@ bool hbitmap_next_dirty_area(const HBitmap *hb, int64_t start, int64_t end,
75
return true;
31
}
76
}
32
77
33
+static inline uint64_t size_to_subclusters(BDRVQcow2State *s, uint64_t size)
78
+bool hbitmap_status(const HBitmap *hb, int64_t start, int64_t count,
79
+ int64_t *pnum)
34
+{
80
+{
35
+ return (size + (s->subcluster_size - 1)) >> s->subcluster_bits;
81
+ int64_t next_dirty, next_zero;
82
+
83
+ assert(start >= 0);
84
+ assert(count > 0);
85
+ assert(start + count <= hb->orig_size);
86
+
87
+ next_dirty = hbitmap_next_dirty(hb, start, count);
88
+ if (next_dirty == -1) {
89
+ *pnum = count;
90
+ return false;
91
+ }
92
+
93
+ if (next_dirty > start) {
94
+ *pnum = next_dirty - start;
95
+ return false;
96
+ }
97
+
98
+ assert(next_dirty == start);
99
+
100
+ next_zero = hbitmap_next_zero(hb, start, count);
101
+ if (next_zero == -1) {
102
+ *pnum = count;
103
+ return true;
104
+ }
105
+
106
+ assert(next_zero > start);
107
+ *pnum = next_zero - start;
108
+ return false;
36
+}
109
+}
37
+
110
+
38
static inline int64_t size_to_l1(BDRVQcow2State *s, int64_t size)
111
bool hbitmap_empty(const HBitmap *hb)
39
{
112
{
40
int shift = s->cluster_bits + s->l2_bits;
113
return hb->count == 0;
41
--
114
--
42
2.26.2
115
2.34.1
43
44
diff view generated by jsdifflib
[PULL 15/34] qcow2: Add qcow2_get_subcluster_range_type()
[PULL 16/23] block/reqlist: add reqlist_wait_all()
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
There are situations in which we want to know how many contiguous
3
Add function to wait for all intersecting requests.
4
subclusters of the same type there are in a given cluster. This can be
4
To be used in the further commit.
5
done by simply iterating over the subclusters and repeatedly calling
6
qcow2_get_subcluster_type() for each one of them.
7
5
8
However once we determined the type of a subcluster we can check the
6
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
9
rest efficiently by counting the number of adjacent ones (or zeroes)
7
Reviewed-by: Nikita Lapshin <nikita.lapshin@virtuozzo.com>
10
in the bitmap. This is what this function does.
8
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
9
Message-Id: <20220303194349.2304213-10-vsementsov@virtuozzo.com>
10
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
11
---
12
include/block/reqlist.h | 8 ++++++++
13
block/reqlist.c | 8 ++++++++
14
2 files changed, 16 insertions(+)
11
15
12
Signed-off-by: Alberto Garcia <berto@igalia.com>
16
diff --git a/include/block/reqlist.h b/include/block/reqlist.h
13
Reviewed-by: Eric Blake <eblake@redhat.com>
14
Reviewed-by: Max Reitz <mreitz@redhat.com>
15
Message-Id: <db917263d568ec6ffb4a41cac3c9100f96bf6c18.1594396418.git.berto@igalia.com>
16
Signed-off-by: Max Reitz <mreitz@redhat.com>
17
---
18
block/qcow2-cluster.c | 51 +++++++++++++++++++++++++++++++++++++++++++
19
1 file changed, 51 insertions(+)
20
21
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
22
index XXXXXXX..XXXXXXX 100644
17
index XXXXXXX..XXXXXXX 100644
23
--- a/block/qcow2-cluster.c
18
--- a/include/block/reqlist.h
24
+++ b/block/qcow2-cluster.c
19
+++ b/include/block/reqlist.h
25
@@ -XXX,XX +XXX,XX @@ fail:
20
@@ -XXX,XX +XXX,XX @@ BlockReq *reqlist_find_conflict(BlockReqList *reqs, int64_t offset,
26
return ret;
21
bool coroutine_fn reqlist_wait_one(BlockReqList *reqs, int64_t offset,
22
int64_t bytes, CoMutex *lock);
23
24
+/*
25
+ * Wait for all intersecting requests. It just calls reqlist_wait_one() in a
26
+ * loop, caller is responsible to stop producing new requests in this region
27
+ * in parallel, otherwise reqlist_wait_all() may never return.
28
+ */
29
+void coroutine_fn reqlist_wait_all(BlockReqList *reqs, int64_t offset,
30
+ int64_t bytes, CoMutex *lock);
31
+
32
/*
33
* Shrink request and wake all waiting coroutines (maybe some of them are not
34
* intersecting with shrunk request).
35
diff --git a/block/reqlist.c b/block/reqlist.c
36
index XXXXXXX..XXXXXXX 100644
37
--- a/block/reqlist.c
38
+++ b/block/reqlist.c
39
@@ -XXX,XX +XXX,XX @@ bool coroutine_fn reqlist_wait_one(BlockReqList *reqs, int64_t offset,
40
return true;
27
}
41
}
28
42
29
+/*
43
+void coroutine_fn reqlist_wait_all(BlockReqList *reqs, int64_t offset,
30
+ * For a given L2 entry, count the number of contiguous subclusters of
44
+ int64_t bytes, CoMutex *lock)
31
+ * the same type starting from @sc_from. Compressed clusters are
32
+ * treated as if they were divided into subclusters of size
33
+ * s->subcluster_size.
34
+ *
35
+ * Return the number of contiguous subclusters and set @type to the
36
+ * subcluster type.
37
+ *
38
+ * If the L2 entry is invalid return -errno and set @type to
39
+ * QCOW2_SUBCLUSTER_INVALID.
40
+ */
41
+G_GNUC_UNUSED
42
+static int qcow2_get_subcluster_range_type(BlockDriverState *bs,
43
+ uint64_t l2_entry,
44
+ uint64_t l2_bitmap,
45
+ unsigned sc_from,
46
+ QCow2SubclusterType *type)
47
+{
45
+{
48
+ BDRVQcow2State *s = bs->opaque;
46
+ while (reqlist_wait_one(reqs, offset, bytes, lock)) {
49
+ uint32_t val;
47
+ /* continue */
50
+
51
+ *type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, sc_from);
52
+
53
+ if (*type == QCOW2_SUBCLUSTER_INVALID) {
54
+ return -EINVAL;
55
+ } else if (!has_subclusters(s) || *type == QCOW2_SUBCLUSTER_COMPRESSED) {
56
+ return s->subclusters_per_cluster - sc_from;
57
+ }
58
+
59
+ switch (*type) {
60
+ case QCOW2_SUBCLUSTER_NORMAL:
61
+ val = l2_bitmap | QCOW_OFLAG_SUB_ALLOC_RANGE(0, sc_from);
62
+ return cto32(val) - sc_from;
63
+
64
+ case QCOW2_SUBCLUSTER_ZERO_PLAIN:
65
+ case QCOW2_SUBCLUSTER_ZERO_ALLOC:
66
+ val = (l2_bitmap | QCOW_OFLAG_SUB_ZERO_RANGE(0, sc_from)) >> 32;
67
+ return cto32(val) - sc_from;
68
+
69
+ case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN:
70
+ case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC:
71
+ val = ((l2_bitmap >> 32) | l2_bitmap)
72
+ & ~QCOW_OFLAG_SUB_ALLOC_RANGE(0, sc_from);
73
+ return ctz32(val) - sc_from;
74
+
75
+ default:
76
+ g_assert_not_reached();
77
+ }
48
+ }
78
+}
49
+}
79
+
50
+
80
/*
51
void coroutine_fn reqlist_shrink_req(BlockReq *req, int64_t new_bytes)
81
* Checks how many clusters in a given L2 slice are contiguous in the image
52
{
82
* file. As soon as one of the flags in the bitmask stop_flags changes compared
53
if (new_bytes == req->bytes) {
83
--
54
--
84
2.26.2
55
2.34.1
85
86
diff view generated by jsdifflib
[PULL 14/34] qcow2: Add QCow2SubclusterType and qcow2_get_subcluster_type()
[PULL 17/23] block/io: introduce block driver snapshot-access API
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
This patch adds QCow2SubclusterType, which is the subcluster-level
3
Add new block driver handlers and corresponding generic wrappers.
4
version of QCow2ClusterType. All QCOW2_SUBCLUSTER_* values have the
4
It will be used to allow copy-before-write filter to provide
5
the same meaning as their QCOW2_CLUSTER_* equivalents (when they
5
reach fleecing interface in further commit.
6
exist). See below for details and caveats.
7
6
8
In images without extended L2 entries clusters are treated as having
7
In future this approach may be used to allow reading qcow2 internal
9
exactly one subcluster so it is possible to replace one data type with
8
snapshots, for example to export them through NBD.
10
the other while keeping the exact same semantics.
11
9
12
With extended L2 entries there are new possible values, and every
10
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
13
subcluster in the same cluster can obviously have a different
11
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
14
QCow2SubclusterType so functions need to be adapted to work on the
12
Message-Id: <20220303194349.2304213-11-vsementsov@virtuozzo.com>
15
subcluster level.
13
[hreitz: Rebased on block GS/IO split]
14
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
15
---
16
include/block/block_int-common.h | 18 ++++++++
17
include/block/block_int-io.h | 9 ++++
18
block/io.c | 72 ++++++++++++++++++++++++++++++++
19
3 files changed, 99 insertions(+)
16
20
17
There are several things that have to be taken into account:
21
diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
18
19
a) QCOW2_SUBCLUSTER_COMPRESSED means that the whole cluster is
20
compressed. We do not support compression at the subcluster
21
level.
22
23
b) There are two different values for unallocated subclusters:
24
QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN which means that the whole
25
cluster is unallocated, and QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC
26
which means that the cluster is allocated but the subcluster is
27
not. The latter can only happen in images with extended L2
28
entries.
29
30
c) QCOW2_SUBCLUSTER_INVALID is used to detect the cases where an L2
31
entry has a value that violates the specification. The caller is
32
responsible for handling these situations.
33
34
To prevent compatibility problems with images that have invalid
35
values but are currently being read by QEMU without causing side
36
effects, QCOW2_SUBCLUSTER_INVALID is only returned for images
37
with extended L2 entries.
38
39
qcow2_cluster_to_subcluster_type() is added as a separate function
40
from qcow2_get_subcluster_type(), but this is only temporary and both
41
will be merged in a subsequent patch.
42
43
Signed-off-by: Alberto Garcia <berto@igalia.com>
44
Reviewed-by: Eric Blake <eblake@redhat.com>
45
Reviewed-by: Max Reitz <mreitz@redhat.com>
46
Message-Id: <26ef38e270f25851c98b51278852b4c4a7f97e69.1594396418.git.berto@igalia.com>
47
Signed-off-by: Max Reitz <mreitz@redhat.com>
48
---
49
block/qcow2.h | 126 +++++++++++++++++++++++++++++++++++++++++++++++++-
50
1 file changed, 125 insertions(+), 1 deletion(-)
51
52
diff --git a/block/qcow2.h b/block/qcow2.h
53
index XXXXXXX..XXXXXXX 100644
22
index XXXXXXX..XXXXXXX 100644
54
--- a/block/qcow2.h
23
--- a/include/block/block_int-common.h
55
+++ b/block/qcow2.h
24
+++ b/include/block/block_int-common.h
25
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
26
bool want_zero, int64_t offset, int64_t bytes, int64_t *pnum,
27
int64_t *map, BlockDriverState **file);
28
29
+ /*
30
+ * Snapshot-access API.
31
+ *
32
+ * Block-driver may provide snapshot-access API: special functions to access
33
+ * some internal "snapshot". The functions are similar with normal
34
+ * read/block_status/discard handler, but don't have any specific handling
35
+ * in generic block-layer: no serializing, no alignment, no tracked
36
+ * requests. So, block-driver that realizes these APIs is fully responsible
37
+ * for synchronization between snapshot-access API and normal IO requests.
38
+ */
39
+ int coroutine_fn (*bdrv_co_preadv_snapshot)(BlockDriverState *bs,
40
+ int64_t offset, int64_t bytes, QEMUIOVector *qiov, size_t qiov_offset);
41
+ int coroutine_fn (*bdrv_co_snapshot_block_status)(BlockDriverState *bs,
42
+ bool want_zero, int64_t offset, int64_t bytes, int64_t *pnum,
43
+ int64_t *map, BlockDriverState **file);
44
+ int coroutine_fn (*bdrv_co_pdiscard_snapshot)(BlockDriverState *bs,
45
+ int64_t offset, int64_t bytes);
46
+
47
/*
48
* Invalidate any cached meta-data.
49
*/
50
diff --git a/include/block/block_int-io.h b/include/block/block_int-io.h
51
index XXXXXXX..XXXXXXX 100644
52
--- a/include/block/block_int-io.h
53
+++ b/include/block/block_int-io.h
56
@@ -XXX,XX +XXX,XX @@
54
@@ -XXX,XX +XXX,XX @@
57
55
* the I/O API.
58
#define QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER 32
56
*/
59
57
60
+/* The subcluster X [0..31] is allocated */
58
+int coroutine_fn bdrv_co_preadv_snapshot(BdrvChild *child,
61
+#define QCOW_OFLAG_SUB_ALLOC(X) (1ULL << (X))
59
+ int64_t offset, int64_t bytes, QEMUIOVector *qiov, size_t qiov_offset);
62
+/* The subcluster X [0..31] reads as zeroes */
60
+int coroutine_fn bdrv_co_snapshot_block_status(BlockDriverState *bs,
63
+#define QCOW_OFLAG_SUB_ZERO(X) (QCOW_OFLAG_SUB_ALLOC(X) << 32)
61
+ bool want_zero, int64_t offset, int64_t bytes, int64_t *pnum,
64
+/* Subclusters [X, Y) (0 <= X <= Y <= 32) are allocated */
62
+ int64_t *map, BlockDriverState **file);
65
+#define QCOW_OFLAG_SUB_ALLOC_RANGE(X, Y) \
63
+int coroutine_fn bdrv_co_pdiscard_snapshot(BlockDriverState *bs,
66
+ (QCOW_OFLAG_SUB_ALLOC(Y) - QCOW_OFLAG_SUB_ALLOC(X))
64
+ int64_t offset, int64_t bytes);
67
+/* Subclusters [X, Y) (0 <= X <= Y <= 32) read as zeroes */
68
+#define QCOW_OFLAG_SUB_ZERO_RANGE(X, Y) \
69
+ (QCOW_OFLAG_SUB_ALLOC_RANGE(X, Y) << 32)
70
+/* L2 entry bitmap with all allocation bits set */
71
+#define QCOW_L2_BITMAP_ALL_ALLOC (QCOW_OFLAG_SUB_ALLOC_RANGE(0, 32))
72
+/* L2 entry bitmap with all "read as zeroes" bits set */
73
+#define QCOW_L2_BITMAP_ALL_ZEROES (QCOW_OFLAG_SUB_ZERO_RANGE(0, 32))
74
+
65
+
75
/* Size of normal and extended L2 entries */
76
#define L2E_SIZE_NORMAL (sizeof(uint64_t))
77
#define L2E_SIZE_EXTENDED (sizeof(uint64_t) * 2)
78
@@ -XXX,XX +XXX,XX @@ typedef struct QCowL2Meta
79
QLIST_ENTRY(QCowL2Meta) next_in_flight;
80
} QCowL2Meta;
81
82
+/*
83
+ * In images with standard L2 entries all clusters are treated as if
84
+ * they had one subcluster so QCow2ClusterType and QCow2SubclusterType
85
+ * can be mapped to each other and have the exact same meaning
86
+ * (QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC cannot happen in these images).
87
+ *
88
+ * In images with extended L2 entries QCow2ClusterType refers to the
89
+ * complete cluster and QCow2SubclusterType to each of the individual
90
+ * subclusters, so there are several possible combinations:
91
+ *
92
+ * |--------------+---------------------------|
93
+ * | Cluster type | Possible subcluster types |
94
+ * |--------------+---------------------------|
95
+ * | UNALLOCATED | UNALLOCATED_PLAIN |
96
+ * | | ZERO_PLAIN |
97
+ * |--------------+---------------------------|
98
+ * | NORMAL | UNALLOCATED_ALLOC |
99
+ * | | ZERO_ALLOC |
100
+ * | | NORMAL |
101
+ * |--------------+---------------------------|
102
+ * | COMPRESSED | COMPRESSED |
103
+ * |--------------+---------------------------|
104
+ *
105
+ * QCOW2_SUBCLUSTER_INVALID means that the L2 entry is incorrect and
106
+ * the image should be marked corrupt.
107
+ */
108
+
66
+
109
typedef enum QCow2ClusterType {
67
int coroutine_fn bdrv_co_preadv(BdrvChild *child,
110
QCOW2_CLUSTER_UNALLOCATED,
68
int64_t offset, int64_t bytes, QEMUIOVector *qiov,
111
QCOW2_CLUSTER_ZERO_PLAIN,
69
BdrvRequestFlags flags);
112
@@ -XXX,XX +XXX,XX @@ typedef enum QCow2ClusterType {
70
diff --git a/block/io.c b/block/io.c
113
QCOW2_CLUSTER_COMPRESSED,
71
index XXXXXXX..XXXXXXX 100644
114
} QCow2ClusterType;
72
--- a/block/io.c
115
73
+++ b/block/io.c
116
+typedef enum QCow2SubclusterType {
74
@@ -XXX,XX +XXX,XX @@ void bdrv_cancel_in_flight(BlockDriverState *bs)
117
+ QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN,
75
bs->drv->bdrv_cancel_in_flight(bs);
118
+ QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC,
119
+ QCOW2_SUBCLUSTER_ZERO_PLAIN,
120
+ QCOW2_SUBCLUSTER_ZERO_ALLOC,
121
+ QCOW2_SUBCLUSTER_NORMAL,
122
+ QCOW2_SUBCLUSTER_COMPRESSED,
123
+ QCOW2_SUBCLUSTER_INVALID,
124
+} QCow2SubclusterType;
125
+
126
typedef enum QCow2MetadataOverlap {
127
QCOW2_OL_MAIN_HEADER_BITNR = 0,
128
QCOW2_OL_ACTIVE_L1_BITNR = 1,
129
@@ -XXX,XX +XXX,XX @@ static inline int64_t qcow2_vm_state_offset(BDRVQcow2State *s)
130
static inline QCow2ClusterType qcow2_get_cluster_type(BlockDriverState *bs,
131
uint64_t l2_entry)
132
{
133
+ BDRVQcow2State *s = bs->opaque;
134
+
135
if (l2_entry & QCOW_OFLAG_COMPRESSED) {
136
return QCOW2_CLUSTER_COMPRESSED;
137
- } else if (l2_entry & QCOW_OFLAG_ZERO) {
138
+ } else if ((l2_entry & QCOW_OFLAG_ZERO) && !has_subclusters(s)) {
139
if (l2_entry & L2E_OFFSET_MASK) {
140
return QCOW2_CLUSTER_ZERO_ALLOC;
141
}
142
@@ -XXX,XX +XXX,XX @@ static inline QCow2ClusterType qcow2_get_cluster_type(BlockDriverState *bs,
143
}
76
}
144
}
77
}
145
78
+
146
+/*
79
+int coroutine_fn
147
+ * For an image without extended L2 entries, return the
80
+bdrv_co_preadv_snapshot(BdrvChild *child, int64_t offset, int64_t bytes,
148
+ * QCow2SubclusterType equivalent of a given QCow2ClusterType.
81
+ QEMUIOVector *qiov, size_t qiov_offset)
149
+ */
150
+static inline
151
+QCow2SubclusterType qcow2_cluster_to_subcluster_type(QCow2ClusterType type)
152
+{
82
+{
153
+ switch (type) {
83
+ BlockDriverState *bs = child->bs;
154
+ case QCOW2_CLUSTER_COMPRESSED:
84
+ BlockDriver *drv = bs->drv;
155
+ return QCOW2_SUBCLUSTER_COMPRESSED;
85
+ int ret;
156
+ case QCOW2_CLUSTER_ZERO_PLAIN:
86
+ IO_CODE();
157
+ return QCOW2_SUBCLUSTER_ZERO_PLAIN;
87
+
158
+ case QCOW2_CLUSTER_ZERO_ALLOC:
88
+ if (!drv) {
159
+ return QCOW2_SUBCLUSTER_ZERO_ALLOC;
89
+ return -ENOMEDIUM;
160
+ case QCOW2_CLUSTER_NORMAL:
161
+ return QCOW2_SUBCLUSTER_NORMAL;
162
+ case QCOW2_CLUSTER_UNALLOCATED:
163
+ return QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN;
164
+ default:
165
+ g_assert_not_reached();
166
+ }
90
+ }
91
+
92
+ if (!drv->bdrv_co_preadv_snapshot) {
93
+ return -ENOTSUP;
94
+ }
95
+
96
+ bdrv_inc_in_flight(bs);
97
+ ret = drv->bdrv_co_preadv_snapshot(bs, offset, bytes, qiov, qiov_offset);
98
+ bdrv_dec_in_flight(bs);
99
+
100
+ return ret;
167
+}
101
+}
168
+
102
+
169
+/*
103
+int coroutine_fn
170
+ * In an image without subsclusters @l2_bitmap is ignored and
104
+bdrv_co_snapshot_block_status(BlockDriverState *bs,
171
+ * @sc_index must be 0.
105
+ bool want_zero, int64_t offset, int64_t bytes,
172
+ * Return QCOW2_SUBCLUSTER_INVALID if an invalid l2 entry is detected
106
+ int64_t *pnum, int64_t *map,
173
+ * (this checks the whole entry and bitmap, not only the bits related
107
+ BlockDriverState **file)
174
+ * to subcluster @sc_index).
175
+ */
176
+static inline
177
+QCow2SubclusterType qcow2_get_subcluster_type(BlockDriverState *bs,
178
+ uint64_t l2_entry,
179
+ uint64_t l2_bitmap,
180
+ unsigned sc_index)
181
+{
108
+{
182
+ BDRVQcow2State *s = bs->opaque;
109
+ BlockDriver *drv = bs->drv;
183
+ QCow2ClusterType type = qcow2_get_cluster_type(bs, l2_entry);
110
+ int ret;
184
+ assert(sc_index < s->subclusters_per_cluster);
111
+ IO_CODE();
185
+
112
+
186
+ if (has_subclusters(s)) {
113
+ if (!drv) {
187
+ switch (type) {
114
+ return -ENOMEDIUM;
188
+ case QCOW2_CLUSTER_COMPRESSED:
189
+ return QCOW2_SUBCLUSTER_COMPRESSED;
190
+ case QCOW2_CLUSTER_NORMAL:
191
+ if ((l2_bitmap >> 32) & l2_bitmap) {
192
+ return QCOW2_SUBCLUSTER_INVALID;
193
+ } else if (l2_bitmap & QCOW_OFLAG_SUB_ZERO(sc_index)) {
194
+ return QCOW2_SUBCLUSTER_ZERO_ALLOC;
195
+ } else if (l2_bitmap & QCOW_OFLAG_SUB_ALLOC(sc_index)) {
196
+ return QCOW2_SUBCLUSTER_NORMAL;
197
+ } else {
198
+ return QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC;
199
+ }
200
+ case QCOW2_CLUSTER_UNALLOCATED:
201
+ if (l2_bitmap & QCOW_L2_BITMAP_ALL_ALLOC) {
202
+ return QCOW2_SUBCLUSTER_INVALID;
203
+ } else if (l2_bitmap & QCOW_OFLAG_SUB_ZERO(sc_index)) {
204
+ return QCOW2_SUBCLUSTER_ZERO_PLAIN;
205
+ } else {
206
+ return QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN;
207
+ }
208
+ default:
209
+ g_assert_not_reached();
210
+ }
211
+ } else {
212
+ return qcow2_cluster_to_subcluster_type(type);
213
+ }
115
+ }
116
+
117
+ if (!drv->bdrv_co_snapshot_block_status) {
118
+ return -ENOTSUP;
119
+ }
120
+
121
+ bdrv_inc_in_flight(bs);
122
+ ret = drv->bdrv_co_snapshot_block_status(bs, want_zero, offset, bytes,
123
+ pnum, map, file);
124
+ bdrv_dec_in_flight(bs);
125
+
126
+ return ret;
214
+}
127
+}
215
+
128
+
216
/* Check whether refcounts are eager or lazy */
129
+int coroutine_fn
217
static inline bool qcow2_need_accurate_refcounts(BDRVQcow2State *s)
130
+bdrv_co_pdiscard_snapshot(BlockDriverState *bs, int64_t offset, int64_t bytes)
218
{
131
+{
132
+ BlockDriver *drv = bs->drv;
133
+ int ret;
134
+ IO_CODE();
135
+
136
+ if (!drv) {
137
+ return -ENOMEDIUM;
138
+ }
139
+
140
+ if (!drv->bdrv_co_pdiscard_snapshot) {
141
+ return -ENOTSUP;
142
+ }
143
+
144
+ bdrv_inc_in_flight(bs);
145
+ ret = drv->bdrv_co_pdiscard_snapshot(bs, offset, bytes);
146
+ bdrv_dec_in_flight(bs);
147
+
148
+ return ret;
149
+}
219
--
150
--
220
2.26.2
151
2.34.1
221
222
diff view generated by jsdifflib
[PULL 05/34] qcow2: Process QCOW2_CLUSTER_ZERO_ALLOC clusters in handle_copied()
[PULL 18/23] block: introduce snapshot-access block driver
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
When writing to a qcow2 file there are two functions that take a
3
The new block driver simply utilizes snapshot-access API of underlying
4
virtual offset and return a host offset, possibly allocating new
4
block node.
5
clusters if necessary:
5
6
6
In further patches we want to use it like this:
7
- handle_copied() looks for normal data clusters that are already
7
8
allocated and have a reference count of 1. In those clusters we
8
[guest] [NBD export]
9
can simply write the data and there is no need to perform any
9
| |
10
copy-on-write.
10
| root | root
11
11
v file v
12
- handle_alloc() looks for clusters that do need copy-on-write,
12
[copy-before-write]<------[snapshot-access]
13
either because they haven't been allocated yet, because their
13
| |
14
reference count is != 1 or because they are ZERO_ALLOC clusters.
14
| file | target
15
15
v v
16
The ZERO_ALLOC case is a bit special because those are clusters that
16
[active-disk] [temp.img]
17
are already allocated and they could perfectly be dealt with in
17
18
handle_copied() (as long as copy-on-write is performed when required).
18
This way, NBD client will be able to read snapshotted state of active
19
19
disk, when active disk is continued to be written by guest. This is
20
In fact, there is extra code specifically for them in handle_alloc()
20
known as "fleecing", and currently uses another scheme based on qcow2
21
that tries to reuse the existing allocation if possible and frees them
21
temporary image which backing file is active-disk. New scheme comes
22
otherwise.
22
with benefits - see next commit.
23
23
24
This patch changes the handling of ZERO_ALLOC clusters so the
24
The other possible application is exporting internal snapshots of
25
semantics of these two functions are now like this:
25
qcow2, like this:
26
26
27
- handle_copied() looks for clusters that are already allocated and
27
[guest] [NBD export]
28
which we can overwrite (NORMAL and ZERO_ALLOC clusters with a
28
| |
29
reference count of 1).
29
| root | root
30
30
v file v
31
- handle_alloc() looks for clusters for which we need a new
31
[qcow2]<---------[snapshot-access]
32
allocation (all other cases).
32
33
33
For this, we'll need to implement snapshot-access API handlers in
34
One important difference after this change is that clusters found
34
qcow2 driver, and improve snapshot-access block driver (and API) to
35
in handle_copied() may now require copy-on-write, but this will be
35
make it possible to select snapshot by name. Another thing to improve
36
necessary anyway once we add support for subclusters.
36
is size of snapshot. Now for simplicity we just use size of bs->file,
37
37
which is OK for backup, but for qcow2 snapshots export we'll need to
38
Signed-off-by: Alberto Garcia <berto@igalia.com>
38
imporve snapshot-access API to get size of snapshot.
39
Reviewed-by: Eric Blake <eblake@redhat.com>
39
40
Reviewed-by: Max Reitz <mreitz@redhat.com>
40
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
41
Message-Id: <eb17fc938f6be7be2e8d8ff42763d2c19241f866.1594396418.git.berto@igalia.com>
41
Message-Id: <20220303194349.2304213-12-vsementsov@virtuozzo.com>
42
Signed-off-by: Max Reitz <mreitz@redhat.com>
42
[hreitz: Rebased on block GS/IO split]
43
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
43
---
44
---
44
block/qcow2-cluster.c | 252 +++++++++++++++++++++++-------------------
45
qapi/block-core.json | 4 +-
45
1 file changed, 139 insertions(+), 113 deletions(-)
46
include/block/block_int-common.h | 6 ++
46
47
block/snapshot-access.c | 132 +++++++++++++++++++++++++++++++
47
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
48
MAINTAINERS | 1 +
48
index XXXXXXX..XXXXXXX 100644
49
block/meson.build | 1 +
49
--- a/block/qcow2-cluster.c
50
5 files changed, 143 insertions(+), 1 deletion(-)
50
+++ b/block/qcow2-cluster.c
51
create mode 100644 block/snapshot-access.c
51
@@ -XXX,XX +XXX,XX @@ void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m)
52
52
53
diff --git a/qapi/block-core.json b/qapi/block-core.json
53
/*
54
index XXXXXXX..XXXXXXX 100644
54
* For a given write request, create a new QCowL2Meta structure, add
55
--- a/qapi/block-core.json
55
- * it to @m and the BDRVQcow2State.cluster_allocs list.
56
+++ b/qapi/block-core.json
56
+ * it to @m and the BDRVQcow2State.cluster_allocs list. If the write
57
@@ -XXX,XX +XXX,XX @@
57
+ * request does not need copy-on-write or changes to the L2 metadata
58
# @blkreplay: Since 4.2
58
+ * then this function does nothing.
59
# @compress: Since 5.0
59
*
60
# @copy-before-write: Since 6.2
60
* @host_cluster_offset points to the beginning of the first cluster.
61
+# @snapshot-access: Since 7.0
61
*
62
#
62
* @guest_offset and @bytes indicate the offset and length of the
63
# Since: 2.9
63
* request.
64
##
64
*
65
{ 'enum': 'BlockdevDriver',
65
+ * @l2_slice contains the L2 entries of all clusters involved in this
66
'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs',
66
+ * write request.
67
'cloop', 'compress', 'copy-before-write', 'copy-on-read', 'dmg',
67
+ *
68
- 'file', 'ftp', 'ftps', 'gluster',
68
* If @keep_old is true it means that the clusters were already
69
+ 'file', 'snapshot-access', 'ftp', 'ftps', 'gluster',
69
* allocated and will be overwritten. If false then the clusters are
70
{'name': 'host_cdrom', 'if': 'HAVE_HOST_BLOCK_DEVICE' },
70
* new and we have to decrease the reference count of the old ones.
71
{'name': 'host_device', 'if': 'HAVE_HOST_BLOCK_DEVICE' },
71
@@ -XXX,XX +XXX,XX @@ void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m)
72
'http', 'https', 'iscsi',
72
static void calculate_l2_meta(BlockDriverState *bs,
73
@@ -XXX,XX +XXX,XX @@
73
uint64_t host_cluster_offset,
74
'rbd': 'BlockdevOptionsRbd',
74
uint64_t guest_offset, unsigned bytes,
75
'replication': { 'type': 'BlockdevOptionsReplication',
75
- QCowL2Meta **m, bool keep_old)
76
'if': 'CONFIG_REPLICATION' },
76
+ uint64_t *l2_slice, QCowL2Meta **m, bool keep_old)
77
+ 'snapshot-access': 'BlockdevOptionsGenericFormat',
77
{
78
'ssh': 'BlockdevOptionsSsh',
78
BDRVQcow2State *s = bs->opaque;
79
'throttle': 'BlockdevOptionsThrottle',
79
- unsigned cow_start_from = 0;
80
'vdi': 'BlockdevOptionsGenericFormat',
80
+ int l2_index = offset_to_l2_slice_index(s, guest_offset);
81
diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
81
+ uint64_t l2_entry;
82
index XXXXXXX..XXXXXXX 100644
82
+ unsigned cow_start_from, cow_end_to;
83
--- a/include/block/block_int-common.h
83
unsigned cow_start_to = offset_into_cluster(s, guest_offset);
84
+++ b/include/block/block_int-common.h
84
unsigned cow_end_from = cow_start_to + bytes;
85
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
85
- unsigned cow_end_to = ROUND_UP(cow_end_from, s->cluster_size);
86
* in generic block-layer: no serializing, no alignment, no tracked
86
unsigned nb_clusters = size_to_clusters(s, cow_end_from);
87
* requests. So, block-driver that realizes these APIs is fully responsible
87
QCowL2Meta *old_m = *m;
88
* for synchronization between snapshot-access API and normal IO requests.
88
+ QCow2ClusterType type;
89
+ *
89
+
90
+ * TODO: To be able to support qcow2's internal snapshots, this API will
90
+ assert(nb_clusters <= s->l2_slice_size - l2_index);
91
+ * need to be extended to:
91
+
92
+ * - be able to select a specific snapshot
92
+ /* Return if there's no COW (all clusters are normal and we keep them) */
93
+ * - receive the snapshot's actual length (which may differ from bs's
93
+ if (keep_old) {
94
+ * length)
94
+ int i;
95
*/
95
+ for (i = 0; i < nb_clusters; i++) {
96
int coroutine_fn (*bdrv_co_preadv_snapshot)(BlockDriverState *bs,
96
+ l2_entry = be64_to_cpu(l2_slice[l2_index + i]);
97
int64_t offset, int64_t bytes, QEMUIOVector *qiov, size_t qiov_offset);
97
+ if (qcow2_get_cluster_type(bs, l2_entry) != QCOW2_CLUSTER_NORMAL) {
98
diff --git a/block/snapshot-access.c b/block/snapshot-access.c
98
+ break;
99
new file mode 100644
99
+ }
100
index XXXXXXX..XXXXXXX
100
+ }
101
--- /dev/null
101
+ if (i == nb_clusters) {
102
+++ b/block/snapshot-access.c
102
+ return;
103
@@ -XXX,XX +XXX,XX @@
103
+ }
104
+/*
105
+ * snapshot_access block driver
106
+ *
107
+ * Copyright (c) 2022 Virtuozzo International GmbH.
108
+ *
109
+ * Author:
110
+ * Sementsov-Ogievskiy Vladimir <vsementsov@virtuozzo.com>
111
+ *
112
+ * This program is free software; you can redistribute it and/or modify
113
+ * it under the terms of the GNU General Public License as published by
114
+ * the Free Software Foundation; either version 2 of the License, or
115
+ * (at your option) any later version.
116
+ *
117
+ * This program is distributed in the hope that it will be useful,
118
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
119
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
120
+ * GNU General Public License for more details.
121
+ *
122
+ * You should have received a copy of the GNU General Public License
123
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
124
+ */
125
+
126
+#include "qemu/osdep.h"
127
+
128
+#include "sysemu/block-backend.h"
129
+#include "qemu/cutils.h"
130
+#include "block/block_int.h"
131
+
132
+static coroutine_fn int
133
+snapshot_access_co_preadv_part(BlockDriverState *bs,
134
+ int64_t offset, int64_t bytes,
135
+ QEMUIOVector *qiov, size_t qiov_offset,
136
+ BdrvRequestFlags flags)
137
+{
138
+ if (flags) {
139
+ return -ENOTSUP;
104
+ }
140
+ }
105
+
141
+
106
+ /* Get the L2 entry of the first cluster */
142
+ return bdrv_co_preadv_snapshot(bs->file, offset, bytes, qiov, qiov_offset);
107
+ l2_entry = be64_to_cpu(l2_slice[l2_index]);
143
+}
108
+ type = qcow2_get_cluster_type(bs, l2_entry);
144
+
109
+
145
+static int coroutine_fn
110
+ if (type == QCOW2_CLUSTER_NORMAL && keep_old) {
146
+snapshot_access_co_block_status(BlockDriverState *bs,
111
+ cow_start_from = cow_start_to;
147
+ bool want_zero, int64_t offset,
112
+ } else {
148
+ int64_t bytes, int64_t *pnum,
113
+ cow_start_from = 0;
149
+ int64_t *map, BlockDriverState **file)
150
+{
151
+ return bdrv_co_snapshot_block_status(bs->file->bs, want_zero, offset,
152
+ bytes, pnum, map, file);
153
+}
154
+
155
+static int coroutine_fn snapshot_access_co_pdiscard(BlockDriverState *bs,
156
+ int64_t offset, int64_t bytes)
157
+{
158
+ return bdrv_co_pdiscard_snapshot(bs->file->bs, offset, bytes);
159
+}
160
+
161
+static int coroutine_fn
162
+snapshot_access_co_pwrite_zeroes(BlockDriverState *bs,
163
+ int64_t offset, int64_t bytes,
164
+ BdrvRequestFlags flags)
165
+{
166
+ return -ENOTSUP;
167
+}
168
+
169
+static coroutine_fn int
170
+snapshot_access_co_pwritev_part(BlockDriverState *bs,
171
+ int64_t offset, int64_t bytes,
172
+ QEMUIOVector *qiov, size_t qiov_offset,
173
+ BdrvRequestFlags flags)
174
+{
175
+ return -ENOTSUP;
176
+}
177
+
178
+
179
+static void snapshot_access_refresh_filename(BlockDriverState *bs)
180
+{
181
+ pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
182
+ bs->file->bs->filename);
183
+}
184
+
185
+static int snapshot_access_open(BlockDriverState *bs, QDict *options, int flags,
186
+ Error **errp)
187
+{
188
+ bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
189
+ BDRV_CHILD_DATA | BDRV_CHILD_PRIMARY,
190
+ false, errp);
191
+ if (!bs->file) {
192
+ return -EINVAL;
114
+ }
193
+ }
115
+
194
+
116
+ /* Get the L2 entry of the last cluster */
195
+ bs->total_sectors = bs->file->bs->total_sectors;
117
+ l2_entry = be64_to_cpu(l2_slice[l2_index + nb_clusters - 1]);
196
+
118
+ type = qcow2_get_cluster_type(bs, l2_entry);
197
+ return 0;
119
+
198
+}
120
+ if (type == QCOW2_CLUSTER_NORMAL && keep_old) {
199
+
121
+ cow_end_to = cow_end_from;
200
+static void snapshot_access_child_perm(BlockDriverState *bs, BdrvChild *c,
122
+ } else {
201
+ BdrvChildRole role,
123
+ cow_end_to = ROUND_UP(cow_end_from, s->cluster_size);
202
+ BlockReopenQueue *reopen_queue,
124
+ }
203
+ uint64_t perm, uint64_t shared,
125
204
+ uint64_t *nperm, uint64_t *nshared)
126
*m = g_malloc0(sizeof(**m));
205
+{
127
**m = (QCowL2Meta) {
206
+ /*
128
@@ -XXX,XX +XXX,XX @@ static void calculate_l2_meta(BlockDriverState *bs,
207
+ * Currently, we don't need any permissions. If bs->file provides
129
QLIST_INSERT_HEAD(&s->cluster_allocs, *m, next_in_flight);
208
+ * snapshot-access API, we can use it.
130
}
209
+ */
131
210
+ *nperm = 0;
132
-/* Returns true if writing to a cluster requires COW */
211
+ *nshared = BLK_PERM_ALL;
133
-static bool cluster_needs_cow(BlockDriverState *bs, uint64_t l2_entry)
212
+}
134
+/*
213
+
135
+ * Returns true if writing to the cluster pointed to by @l2_entry
214
+BlockDriver bdrv_snapshot_access_drv = {
136
+ * requires a new allocation (that is, if the cluster is unallocated
215
+ .format_name = "snapshot-access",
137
+ * or has refcount > 1 and therefore cannot be written in-place).
216
+
138
+ */
217
+ .bdrv_open = snapshot_access_open,
139
+static bool cluster_needs_new_alloc(BlockDriverState *bs, uint64_t l2_entry)
218
+
140
{
219
+ .bdrv_co_preadv_part = snapshot_access_co_preadv_part,
141
switch (qcow2_get_cluster_type(bs, l2_entry)) {
220
+ .bdrv_co_pwritev_part = snapshot_access_co_pwritev_part,
142
case QCOW2_CLUSTER_NORMAL:
221
+ .bdrv_co_pwrite_zeroes = snapshot_access_co_pwrite_zeroes,
143
+ case QCOW2_CLUSTER_ZERO_ALLOC:
222
+ .bdrv_co_pdiscard = snapshot_access_co_pdiscard,
144
if (l2_entry & QCOW_OFLAG_COPIED) {
223
+ .bdrv_co_block_status = snapshot_access_co_block_status,
145
return false;
224
+
146
}
225
+ .bdrv_refresh_filename = snapshot_access_refresh_filename,
147
case QCOW2_CLUSTER_UNALLOCATED:
226
+
148
case QCOW2_CLUSTER_COMPRESSED:
227
+ .bdrv_child_perm = snapshot_access_child_perm,
149
case QCOW2_CLUSTER_ZERO_PLAIN:
228
+};
150
- case QCOW2_CLUSTER_ZERO_ALLOC:
229
+
151
return true;
230
+static void snapshot_access_init(void)
152
default:
231
+{
153
abort();
232
+ bdrv_register(&bdrv_snapshot_access_drv);
154
@@ -XXX,XX +XXX,XX @@ static bool cluster_needs_cow(BlockDriverState *bs, uint64_t l2_entry)
233
+}
155
}
234
+
156
235
+block_init(snapshot_access_init);
157
/*
236
diff --git a/MAINTAINERS b/MAINTAINERS
158
- * Returns the number of contiguous clusters that can be used for an allocating
237
index XXXXXXX..XXXXXXX 100644
159
- * write, but require COW to be performed (this includes yet unallocated space,
238
--- a/MAINTAINERS
160
- * which must copy from the backing file)
239
+++ b/MAINTAINERS
161
+ * Returns the number of contiguous clusters that can be written to
240
@@ -XXX,XX +XXX,XX @@ F: block/reqlist.c
162
+ * using one single write request, starting from @l2_index.
241
F: include/block/reqlist.h
163
+ * At most @nb_clusters are checked.
242
F: block/copy-before-write.h
164
+ *
243
F: block/copy-before-write.c
165
+ * If @new_alloc is true this counts clusters that are either
244
+F: block/snapshot-access.c
166
+ * unallocated, or allocated but with refcount > 1 (so they need to be
245
F: include/block/aio_task.h
167
+ * newly allocated and COWed).
246
F: block/aio_task.c
168
+ *
247
F: util/qemu-co-shared-resource.c
169
+ * If @new_alloc is false this counts clusters that are already
248
diff --git a/block/meson.build b/block/meson.build
170
+ * allocated and can be overwritten in-place (this includes clusters
249
index XXXXXXX..XXXXXXX 100644
171
+ * of type QCOW2_CLUSTER_ZERO_ALLOC).
250
--- a/block/meson.build
172
*/
251
+++ b/block/meson.build
173
-static int count_cow_clusters(BlockDriverState *bs, int nb_clusters,
252
@@ -XXX,XX +XXX,XX @@ block_ss.add(files(
174
- uint64_t *l2_slice, int l2_index)
253
'raw-format.c',
175
+static int count_single_write_clusters(BlockDriverState *bs, int nb_clusters,
254
'reqlist.c',
176
+ uint64_t *l2_slice, int l2_index,
255
'snapshot.c',
177
+ bool new_alloc)
256
+ 'snapshot-access.c',
178
{
257
'throttle-groups.c',
179
+ BDRVQcow2State *s = bs->opaque;
258
'throttle.c',
180
+ uint64_t l2_entry = be64_to_cpu(l2_slice[l2_index]);
259
'vhdx-endian.c',
181
+ uint64_t expected_offset = l2_entry & L2E_OFFSET_MASK;
182
int i;
183
184
for (i = 0; i < nb_clusters; i++) {
185
- uint64_t l2_entry = be64_to_cpu(l2_slice[l2_index + i]);
186
- if (!cluster_needs_cow(bs, l2_entry)) {
187
+ l2_entry = be64_to_cpu(l2_slice[l2_index + i]);
188
+ if (cluster_needs_new_alloc(bs, l2_entry) != new_alloc) {
189
break;
190
}
191
+ if (!new_alloc) {
192
+ if (expected_offset != (l2_entry & L2E_OFFSET_MASK)) {
193
+ break;
194
+ }
195
+ expected_offset += s->cluster_size;
196
+ }
197
}
198
199
assert(i <= nb_clusters);
200
@@ -XXX,XX +XXX,XX @@ static int handle_dependencies(BlockDriverState *bs, uint64_t guest_offset,
201
}
202
203
/*
204
- * Checks how many already allocated clusters that don't require a copy on
205
- * write there are at the given guest_offset (up to *bytes). If *host_offset is
206
- * not INV_OFFSET, only physically contiguous clusters beginning at this host
207
- * offset are counted.
208
+ * Checks how many already allocated clusters that don't require a new
209
+ * allocation there are at the given guest_offset (up to *bytes).
210
+ * If *host_offset is not INV_OFFSET, only physically contiguous clusters
211
+ * beginning at this host offset are counted.
212
*
213
* Note that guest_offset may not be cluster aligned. In this case, the
214
* returned *host_offset points to exact byte referenced by guest_offset and
215
@@ -XXX,XX +XXX,XX @@ static int handle_dependencies(BlockDriverState *bs, uint64_t guest_offset,
216
* Returns:
217
* 0: if no allocated clusters are available at the given offset.
218
* *bytes is normally unchanged. It is set to 0 if the cluster
219
- * is allocated and doesn't need COW, but doesn't have the right
220
- * physical offset.
221
+ * is allocated and can be overwritten in-place but doesn't have
222
+ * the right physical offset.
223
*
224
- * 1: if allocated clusters that don't require a COW are available at
225
- * the requested offset. *bytes may have decreased and describes
226
- * the length of the area that can be written to.
227
+ * 1: if allocated clusters that can be overwritten in place are
228
+ * available at the requested offset. *bytes may have decreased
229
+ * and describes the length of the area that can be written to.
230
*
231
* -errno: in error cases
232
*/
233
@@ -XXX,XX +XXX,XX @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset,
234
{
235
BDRVQcow2State *s = bs->opaque;
236
int l2_index;
237
- uint64_t cluster_offset;
238
+ uint64_t l2_entry, cluster_offset;
239
uint64_t *l2_slice;
240
uint64_t nb_clusters;
241
unsigned int keep_clusters;
242
@@ -XXX,XX +XXX,XX @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset,
243
244
l2_index = offset_to_l2_slice_index(s, guest_offset);
245
nb_clusters = MIN(nb_clusters, s->l2_slice_size - l2_index);
246
- assert(nb_clusters <= INT_MAX);
247
+ /* Limit total byte count to BDRV_REQUEST_MAX_BYTES */
248
+ nb_clusters = MIN(nb_clusters, BDRV_REQUEST_MAX_BYTES >> s->cluster_bits);
249
250
/* Find L2 entry for the first involved cluster */
251
ret = get_cluster_table(bs, guest_offset, &l2_slice, &l2_index);
252
@@ -XXX,XX +XXX,XX @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset,
253
return ret;
254
}
255
256
- cluster_offset = be64_to_cpu(l2_slice[l2_index]);
257
-
258
- /* Check how many clusters are already allocated and don't need COW */
259
- if (qcow2_get_cluster_type(bs, cluster_offset) == QCOW2_CLUSTER_NORMAL
260
- && (cluster_offset & QCOW_OFLAG_COPIED))
261
- {
262
- /* If a specific host_offset is required, check it */
263
- bool offset_matches =
264
- (cluster_offset & L2E_OFFSET_MASK) == *host_offset;
265
-
266
- if (offset_into_cluster(s, cluster_offset & L2E_OFFSET_MASK)) {
267
- qcow2_signal_corruption(bs, true, -1, -1, "Data cluster offset "
268
- "%#llx unaligned (guest offset: %#" PRIx64
269
- ")", cluster_offset & L2E_OFFSET_MASK,
270
- guest_offset);
271
+ l2_entry = be64_to_cpu(l2_slice[l2_index]);
272
+ cluster_offset = l2_entry & L2E_OFFSET_MASK;
273
+
274
+ if (!cluster_needs_new_alloc(bs, l2_entry)) {
275
+ if (offset_into_cluster(s, cluster_offset)) {
276
+ qcow2_signal_corruption(bs, true, -1, -1, "%s cluster offset "
277
+ "%#" PRIx64 " unaligned (guest offset: %#"
278
+ PRIx64 ")", l2_entry & QCOW_OFLAG_ZERO ?
279
+ "Preallocated zero" : "Data",
280
+ cluster_offset, guest_offset);
281
ret = -EIO;
282
goto out;
283
}
284
285
- if (*host_offset != INV_OFFSET && !offset_matches) {
286
+ /* If a specific host_offset is required, check it */
287
+ if (*host_offset != INV_OFFSET && cluster_offset != *host_offset) {
288
*bytes = 0;
289
ret = 0;
290
goto out;
291
}
292
293
/* We keep all QCOW_OFLAG_COPIED clusters */
294
- keep_clusters =
295
- count_contiguous_clusters(bs, nb_clusters, s->cluster_size,
296
- &l2_slice[l2_index],
297
- QCOW_OFLAG_COPIED | QCOW_OFLAG_ZERO);
298
+ keep_clusters = count_single_write_clusters(bs, nb_clusters, l2_slice,
299
+ l2_index, false);
300
assert(keep_clusters <= nb_clusters);
301
302
*bytes = MIN(*bytes,
303
keep_clusters * s->cluster_size
304
- offset_into_cluster(s, guest_offset));
305
+ assert(*bytes != 0);
306
+
307
+ calculate_l2_meta(bs, cluster_offset, guest_offset,
308
+ *bytes, l2_slice, m, true);
309
310
ret = 1;
311
} else {
312
@@ -XXX,XX +XXX,XX @@ out:
313
/* Only return a host offset if we actually made progress. Otherwise we
314
* would make requirements for handle_alloc() that it can't fulfill */
315
if (ret > 0) {
316
- *host_offset = (cluster_offset & L2E_OFFSET_MASK)
317
- + offset_into_cluster(s, guest_offset);
318
+ *host_offset = cluster_offset + offset_into_cluster(s, guest_offset);
319
}
320
321
return ret;
322
@@ -XXX,XX +XXX,XX @@ static int do_alloc_cluster_offset(BlockDriverState *bs, uint64_t guest_offset,
323
}
324
325
/*
326
- * Allocates new clusters for an area that either is yet unallocated or needs a
327
- * copy on write. If *host_offset is not INV_OFFSET, clusters are only
328
- * allocated if the new allocation can match the specified host offset.
329
+ * Allocates new clusters for an area that is either still unallocated or
330
+ * cannot be overwritten in-place. If *host_offset is not INV_OFFSET,
331
+ * clusters are only allocated if the new allocation can match the specified
332
+ * host offset.
333
*
334
* Note that guest_offset may not be cluster aligned. In this case, the
335
* returned *host_offset points to exact byte referenced by guest_offset and
336
@@ -XXX,XX +XXX,XX @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,
337
BDRVQcow2State *s = bs->opaque;
338
int l2_index;
339
uint64_t *l2_slice;
340
- uint64_t entry;
341
uint64_t nb_clusters;
342
int ret;
343
- bool keep_old_clusters = false;
344
345
- uint64_t alloc_cluster_offset = INV_OFFSET;
346
+ uint64_t alloc_cluster_offset;
347
348
trace_qcow2_handle_alloc(qemu_coroutine_self(), guest_offset, *host_offset,
349
*bytes);
350
@@ -XXX,XX +XXX,XX @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,
351
352
l2_index = offset_to_l2_slice_index(s, guest_offset);
353
nb_clusters = MIN(nb_clusters, s->l2_slice_size - l2_index);
354
- assert(nb_clusters <= INT_MAX);
355
-
356
- /* Limit total allocation byte count to INT_MAX */
357
- nb_clusters = MIN(nb_clusters, INT_MAX >> s->cluster_bits);
358
+ /* Limit total allocation byte count to BDRV_REQUEST_MAX_BYTES */
359
+ nb_clusters = MIN(nb_clusters, BDRV_REQUEST_MAX_BYTES >> s->cluster_bits);
360
361
/* Find L2 entry for the first involved cluster */
362
ret = get_cluster_table(bs, guest_offset, &l2_slice, &l2_index);
363
@@ -XXX,XX +XXX,XX @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,
364
return ret;
365
}
366
367
- entry = be64_to_cpu(l2_slice[l2_index]);
368
- nb_clusters = count_cow_clusters(bs, nb_clusters, l2_slice, l2_index);
369
+ nb_clusters = count_single_write_clusters(bs, nb_clusters,
370
+ l2_slice, l2_index, true);
371
372
/* This function is only called when there were no non-COW clusters, so if
373
* we can't find any unallocated or COW clusters either, something is
374
* wrong with our code. */
375
assert(nb_clusters > 0);
376
377
- if (qcow2_get_cluster_type(bs, entry) == QCOW2_CLUSTER_ZERO_ALLOC &&
378
- (entry & QCOW_OFLAG_COPIED) &&
379
- (*host_offset == INV_OFFSET ||
380
- start_of_cluster(s, *host_offset) == (entry & L2E_OFFSET_MASK)))
381
- {
382
- int preallocated_nb_clusters;
383
-
384
- if (offset_into_cluster(s, entry & L2E_OFFSET_MASK)) {
385
- qcow2_signal_corruption(bs, true, -1, -1, "Preallocated zero "
386
- "cluster offset %#llx unaligned (guest "
387
- "offset: %#" PRIx64 ")",
388
- entry & L2E_OFFSET_MASK, guest_offset);
389
- ret = -EIO;
390
- goto fail;
391
- }
392
-
393
- /* Try to reuse preallocated zero clusters; contiguous normal clusters
394
- * would be fine, too, but count_cow_clusters() above has limited
395
- * nb_clusters already to a range of COW clusters */
396
- preallocated_nb_clusters =
397
- count_contiguous_clusters(bs, nb_clusters, s->cluster_size,
398
- &l2_slice[l2_index], QCOW_OFLAG_COPIED);
399
- assert(preallocated_nb_clusters > 0);
400
-
401
- nb_clusters = preallocated_nb_clusters;
402
- alloc_cluster_offset = entry & L2E_OFFSET_MASK;
403
-
404
- /* We want to reuse these clusters, so qcow2_alloc_cluster_link_l2()
405
- * should not free them. */
406
- keep_old_clusters = true;
407
+ /* Allocate at a given offset in the image file */
408
+ alloc_cluster_offset = *host_offset == INV_OFFSET ? INV_OFFSET :
409
+ start_of_cluster(s, *host_offset);
410
+ ret = do_alloc_cluster_offset(bs, guest_offset, &alloc_cluster_offset,
411
+ &nb_clusters);
412
+ if (ret < 0) {
413
+ goto out;
414
}
415
416
- qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice);
417
-
418
- if (alloc_cluster_offset == INV_OFFSET) {
419
- /* Allocate, if necessary at a given offset in the image file */
420
- alloc_cluster_offset = *host_offset == INV_OFFSET ? INV_OFFSET :
421
- start_of_cluster(s, *host_offset);
422
- ret = do_alloc_cluster_offset(bs, guest_offset, &alloc_cluster_offset,
423
- &nb_clusters);
424
- if (ret < 0) {
425
- goto fail;
426
- }
427
-
428
- /* Can't extend contiguous allocation */
429
- if (nb_clusters == 0) {
430
- *bytes = 0;
431
- return 0;
432
- }
433
-
434
- assert(alloc_cluster_offset != INV_OFFSET);
435
+ /* Can't extend contiguous allocation */
436
+ if (nb_clusters == 0) {
437
+ *bytes = 0;
438
+ ret = 0;
439
+ goto out;
440
}
441
442
+ assert(alloc_cluster_offset != INV_OFFSET);
443
+
444
/*
445
* Save info needed for meta data update.
446
*
447
@@ -XXX,XX +XXX,XX @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,
448
*bytes = MIN(*bytes, nb_bytes - offset_into_cluster(s, guest_offset));
449
assert(*bytes != 0);
450
451
- calculate_l2_meta(bs, alloc_cluster_offset, guest_offset, *bytes,
452
- m, keep_old_clusters);
453
+ calculate_l2_meta(bs, alloc_cluster_offset, guest_offset, *bytes, l2_slice,
454
+ m, false);
455
456
- return 1;
457
+ ret = 1;
458
459
-fail:
460
- if (*m && (*m)->nb_clusters > 0) {
461
+out:
462
+ qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice);
463
+ if (ret < 0 && *m && (*m)->nb_clusters > 0) {
464
QLIST_REMOVE(*m, next_in_flight);
465
}
466
return ret;
467
--
260
--
468
2.26.2
261
2.34.1
469
470
diff view generated by jsdifflib
[PULL 06/34] qcow2: Add get_l2_entry() and set_l2_entry()
Deleted patch
1
From: Alberto Garcia <berto@igalia.com>
2
1
3
The size of an L2 entry is 64 bits, but if we want to have subclusters
4
we need extended L2 entries. This means that we have to access L2
5
tables and slices differently depending on whether an image has
6
extended L2 entries or not.
7
8
This patch replaces all l2_slice[] accesses with calls to
9
get_l2_entry() and set_l2_entry().
10
11
Signed-off-by: Alberto Garcia <berto@igalia.com>
12
Reviewed-by: Eric Blake <eblake@redhat.com>
13
Reviewed-by: Max Reitz <mreitz@redhat.com>
14
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
15
Message-Id: <9586363531fec125ba1386e561762d3e4224e9fc.1594396418.git.berto@igalia.com>
16
Signed-off-by: Max Reitz <mreitz@redhat.com>
17
---
18
block/qcow2.h | 12 ++++++++
19
block/qcow2-cluster.c | 63 ++++++++++++++++++++++--------------------
20
block/qcow2-refcount.c | 17 ++++++------
21
3 files changed, 54 insertions(+), 38 deletions(-)
22
23
diff --git a/block/qcow2.h b/block/qcow2.h
24
index XXXXXXX..XXXXXXX 100644
25
--- a/block/qcow2.h
26
+++ b/block/qcow2.h
27
@@ -XXX,XX +XXX,XX @@ typedef enum QCow2MetadataOverlap {
28
29
#define INV_OFFSET (-1ULL)
30
31
+static inline uint64_t get_l2_entry(BDRVQcow2State *s, uint64_t *l2_slice,
32
+ int idx)
33
+{
34
+ return be64_to_cpu(l2_slice[idx]);
35
+}
36
+
37
+static inline void set_l2_entry(BDRVQcow2State *s, uint64_t *l2_slice,
38
+ int idx, uint64_t entry)
39
+{
40
+ l2_slice[idx] = cpu_to_be64(entry);
41
+}
42
+
43
static inline bool has_data_file(BlockDriverState *bs)
44
{
45
BDRVQcow2State *s = bs->opaque;
46
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
47
index XXXXXXX..XXXXXXX 100644
48
--- a/block/qcow2-cluster.c
49
+++ b/block/qcow2-cluster.c
50
@@ -XXX,XX +XXX,XX @@ fail:
51
* cluster which may require a different handling)
52
*/
53
static int count_contiguous_clusters(BlockDriverState *bs, int nb_clusters,
54
- int cluster_size, uint64_t *l2_slice, uint64_t stop_flags)
55
+ int cluster_size, uint64_t *l2_slice, int l2_index, uint64_t stop_flags)
56
{
57
+ BDRVQcow2State *s = bs->opaque;
58
int i;
59
QCow2ClusterType first_cluster_type;
60
uint64_t mask = stop_flags | L2E_OFFSET_MASK | QCOW_OFLAG_COMPRESSED;
61
- uint64_t first_entry = be64_to_cpu(l2_slice[0]);
62
+ uint64_t first_entry = get_l2_entry(s, l2_slice, l2_index);
63
uint64_t offset = first_entry & mask;
64
65
first_cluster_type = qcow2_get_cluster_type(bs, first_entry);
66
@@ -XXX,XX +XXX,XX @@ static int count_contiguous_clusters(BlockDriverState *bs, int nb_clusters,
67
first_cluster_type == QCOW2_CLUSTER_ZERO_ALLOC);
68
69
for (i = 0; i < nb_clusters; i++) {
70
- uint64_t l2_entry = be64_to_cpu(l2_slice[i]) & mask;
71
+ uint64_t l2_entry = get_l2_entry(s, l2_slice, l2_index + i) & mask;
72
if (offset + (uint64_t) i * cluster_size != l2_entry) {
73
break;
74
}
75
@@ -XXX,XX +XXX,XX @@ static int count_contiguous_clusters(BlockDriverState *bs, int nb_clusters,
76
static int count_contiguous_clusters_unallocated(BlockDriverState *bs,
77
int nb_clusters,
78
uint64_t *l2_slice,
79
+ int l2_index,
80
QCow2ClusterType wanted_type)
81
{
82
+ BDRVQcow2State *s = bs->opaque;
83
int i;
84
85
assert(wanted_type == QCOW2_CLUSTER_ZERO_PLAIN ||
86
wanted_type == QCOW2_CLUSTER_UNALLOCATED);
87
for (i = 0; i < nb_clusters; i++) {
88
- uint64_t entry = be64_to_cpu(l2_slice[i]);
89
+ uint64_t entry = get_l2_entry(s, l2_slice, l2_index + i);
90
QCow2ClusterType type = qcow2_get_cluster_type(bs, entry);
91
92
if (type != wanted_type) {
93
@@ -XXX,XX +XXX,XX @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
94
/* find the cluster offset for the given disk offset */
95
96
l2_index = offset_to_l2_slice_index(s, offset);
97
- l2_entry = be64_to_cpu(l2_slice[l2_index]);
98
+ l2_entry = get_l2_entry(s, l2_slice, l2_index);
99
100
nb_clusters = size_to_clusters(s, bytes_needed);
101
/* bytes_needed <= *bytes + offset_in_cluster, both of which are unsigned
102
@@ -XXX,XX +XXX,XX @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
103
case QCOW2_CLUSTER_UNALLOCATED:
104
/* how many empty clusters ? */
105
c = count_contiguous_clusters_unallocated(bs, nb_clusters,
106
- &l2_slice[l2_index], type);
107
+ l2_slice, l2_index, type);
108
break;
109
case QCOW2_CLUSTER_ZERO_ALLOC:
110
case QCOW2_CLUSTER_NORMAL: {
111
@@ -XXX,XX +XXX,XX @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
112
*host_offset = host_cluster_offset + offset_in_cluster;
113
/* how many allocated clusters ? */
114
c = count_contiguous_clusters(bs, nb_clusters, s->cluster_size,
115
- &l2_slice[l2_index], QCOW_OFLAG_ZERO);
116
+ l2_slice, l2_index, QCOW_OFLAG_ZERO);
117
if (offset_into_cluster(s, host_cluster_offset)) {
118
qcow2_signal_corruption(bs, true, -1, -1,
119
"Cluster allocation offset %#"
120
@@ -XXX,XX +XXX,XX @@ int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs,
121
122
/* Compression can't overwrite anything. Fail if the cluster was already
123
* allocated. */
124
- cluster_offset = be64_to_cpu(l2_slice[l2_index]);
125
+ cluster_offset = get_l2_entry(s, l2_slice, l2_index);
126
if (cluster_offset & L2E_OFFSET_MASK) {
127
qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice);
128
return -EIO;
129
@@ -XXX,XX +XXX,XX @@ int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs,
130
131
BLKDBG_EVENT(bs->file, BLKDBG_L2_UPDATE_COMPRESSED);
132
qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice);
133
- l2_slice[l2_index] = cpu_to_be64(cluster_offset);
134
+ set_l2_entry(s, l2_slice, l2_index, cluster_offset);
135
qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice);
136
137
*host_offset = cluster_offset & s->cluster_offset_mask;
138
@@ -XXX,XX +XXX,XX @@ int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m)
139
* cluster the second one has to do RMW (which is done above by
140
* perform_cow()), update l2 table with its cluster pointer and free
141
* old cluster. This is what this loop does */
142
- if (l2_slice[l2_index + i] != 0) {
143
- old_cluster[j++] = l2_slice[l2_index + i];
144
+ if (get_l2_entry(s, l2_slice, l2_index + i) != 0) {
145
+ old_cluster[j++] = get_l2_entry(s, l2_slice, l2_index + i);
146
}
147
148
/* The offset must fit in the offset field of the L2 table entry */
149
assert((offset & L2E_OFFSET_MASK) == offset);
150
151
- l2_slice[l2_index + i] = cpu_to_be64(offset | QCOW_OFLAG_COPIED);
152
+ set_l2_entry(s, l2_slice, l2_index + i, offset | QCOW_OFLAG_COPIED);
153
}
154
155
156
@@ -XXX,XX +XXX,XX @@ int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m)
157
*/
158
if (!m->keep_old_clusters && j != 0) {
159
for (i = 0; i < j; i++) {
160
- qcow2_free_any_clusters(bs, be64_to_cpu(old_cluster[i]), 1,
161
- QCOW2_DISCARD_NEVER);
162
+ qcow2_free_any_clusters(bs, old_cluster[i], 1, QCOW2_DISCARD_NEVER);
163
}
164
}
165
166
@@ -XXX,XX +XXX,XX @@ static void calculate_l2_meta(BlockDriverState *bs,
167
if (keep_old) {
168
int i;
169
for (i = 0; i < nb_clusters; i++) {
170
- l2_entry = be64_to_cpu(l2_slice[l2_index + i]);
171
+ l2_entry = get_l2_entry(s, l2_slice, l2_index + i);
172
if (qcow2_get_cluster_type(bs, l2_entry) != QCOW2_CLUSTER_NORMAL) {
173
break;
174
}
175
@@ -XXX,XX +XXX,XX @@ static void calculate_l2_meta(BlockDriverState *bs,
176
}
177
178
/* Get the L2 entry of the first cluster */
179
- l2_entry = be64_to_cpu(l2_slice[l2_index]);
180
+ l2_entry = get_l2_entry(s, l2_slice, l2_index);
181
type = qcow2_get_cluster_type(bs, l2_entry);
182
183
if (type == QCOW2_CLUSTER_NORMAL && keep_old) {
184
@@ -XXX,XX +XXX,XX @@ static void calculate_l2_meta(BlockDriverState *bs,
185
}
186
187
/* Get the L2 entry of the last cluster */
188
- l2_entry = be64_to_cpu(l2_slice[l2_index + nb_clusters - 1]);
189
+ l2_entry = get_l2_entry(s, l2_slice, l2_index + nb_clusters - 1);
190
type = qcow2_get_cluster_type(bs, l2_entry);
191
192
if (type == QCOW2_CLUSTER_NORMAL && keep_old) {
193
@@ -XXX,XX +XXX,XX @@ static int count_single_write_clusters(BlockDriverState *bs, int nb_clusters,
194
bool new_alloc)
195
{
196
BDRVQcow2State *s = bs->opaque;
197
- uint64_t l2_entry = be64_to_cpu(l2_slice[l2_index]);
198
+ uint64_t l2_entry = get_l2_entry(s, l2_slice, l2_index);
199
uint64_t expected_offset = l2_entry & L2E_OFFSET_MASK;
200
int i;
201
202
for (i = 0; i < nb_clusters; i++) {
203
- l2_entry = be64_to_cpu(l2_slice[l2_index + i]);
204
+ l2_entry = get_l2_entry(s, l2_slice, l2_index + i);
205
if (cluster_needs_new_alloc(bs, l2_entry) != new_alloc) {
206
break;
207
}
208
@@ -XXX,XX +XXX,XX @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset,
209
return ret;
210
}
211
212
- l2_entry = be64_to_cpu(l2_slice[l2_index]);
213
+ l2_entry = get_l2_entry(s, l2_slice, l2_index);
214
cluster_offset = l2_entry & L2E_OFFSET_MASK;
215
216
if (!cluster_needs_new_alloc(bs, l2_entry)) {
217
@@ -XXX,XX +XXX,XX @@ static int discard_in_l2_slice(BlockDriverState *bs, uint64_t offset,
218
for (i = 0; i < nb_clusters; i++) {
219
uint64_t old_l2_entry;
220
221
- old_l2_entry = be64_to_cpu(l2_slice[l2_index + i]);
222
+ old_l2_entry = get_l2_entry(s, l2_slice, l2_index + i);
223
224
/*
225
* If full_discard is false, make sure that a discarded area reads back
226
@@ -XXX,XX +XXX,XX @@ static int discard_in_l2_slice(BlockDriverState *bs, uint64_t offset,
227
/* First remove L2 entries */
228
qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice);
229
if (!full_discard && s->qcow_version >= 3) {
230
- l2_slice[l2_index + i] = cpu_to_be64(QCOW_OFLAG_ZERO);
231
+ set_l2_entry(s, l2_slice, l2_index + i, QCOW_OFLAG_ZERO);
232
} else {
233
- l2_slice[l2_index + i] = cpu_to_be64(0);
234
+ set_l2_entry(s, l2_slice, l2_index + i, 0);
235
}
236
237
/* Then decrease the refcount */
238
@@ -XXX,XX +XXX,XX @@ static int zero_in_l2_slice(BlockDriverState *bs, uint64_t offset,
239
uint64_t old_offset;
240
QCow2ClusterType cluster_type;
241
242
- old_offset = be64_to_cpu(l2_slice[l2_index + i]);
243
+ old_offset = get_l2_entry(s, l2_slice, l2_index + i);
244
245
/*
246
* Minimize L2 changes if the cluster already reads back as
247
@@ -XXX,XX +XXX,XX @@ static int zero_in_l2_slice(BlockDriverState *bs, uint64_t offset,
248
249
qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice);
250
if (cluster_type == QCOW2_CLUSTER_COMPRESSED || unmap) {
251
- l2_slice[l2_index + i] = cpu_to_be64(QCOW_OFLAG_ZERO);
252
+ set_l2_entry(s, l2_slice, l2_index + i, QCOW_OFLAG_ZERO);
253
qcow2_free_any_clusters(bs, old_offset, 1, QCOW2_DISCARD_REQUEST);
254
} else {
255
- l2_slice[l2_index + i] |= cpu_to_be64(QCOW_OFLAG_ZERO);
256
+ uint64_t entry = get_l2_entry(s, l2_slice, l2_index + i);
257
+ set_l2_entry(s, l2_slice, l2_index + i, entry | QCOW_OFLAG_ZERO);
258
}
259
}
260
261
@@ -XXX,XX +XXX,XX @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table,
262
}
263
264
for (j = 0; j < s->l2_slice_size; j++) {
265
- uint64_t l2_entry = be64_to_cpu(l2_slice[j]);
266
+ uint64_t l2_entry = get_l2_entry(s, l2_slice, j);
267
int64_t offset = l2_entry & L2E_OFFSET_MASK;
268
QCow2ClusterType cluster_type =
269
qcow2_get_cluster_type(bs, l2_entry);
270
@@ -XXX,XX +XXX,XX @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table,
271
if (!bs->backing) {
272
/* not backed; therefore we can simply deallocate the
273
* cluster */
274
- l2_slice[j] = 0;
275
+ set_l2_entry(s, l2_slice, j, 0);
276
l2_dirty = true;
277
continue;
278
}
279
@@ -XXX,XX +XXX,XX @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table,
280
}
281
282
if (l2_refcount == 1) {
283
- l2_slice[j] = cpu_to_be64(offset | QCOW_OFLAG_COPIED);
284
+ set_l2_entry(s, l2_slice, j, offset | QCOW_OFLAG_COPIED);
285
} else {
286
- l2_slice[j] = cpu_to_be64(offset);
287
+ set_l2_entry(s, l2_slice, j, offset);
288
}
289
l2_dirty = true;
290
}
291
diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
292
index XXXXXXX..XXXXXXX 100644
293
--- a/block/qcow2-refcount.c
294
+++ b/block/qcow2-refcount.c
295
@@ -XXX,XX +XXX,XX @@ int qcow2_update_snapshot_refcount(BlockDriverState *bs,
296
uint64_t cluster_index;
297
uint64_t offset;
298
299
- entry = be64_to_cpu(l2_slice[j]);
300
+ entry = get_l2_entry(s, l2_slice, j);
301
old_entry = entry;
302
entry &= ~QCOW_OFLAG_COPIED;
303
offset = entry & L2E_OFFSET_MASK;
304
@@ -XXX,XX +XXX,XX @@ int qcow2_update_snapshot_refcount(BlockDriverState *bs,
305
qcow2_cache_set_dependency(bs, s->l2_table_cache,
306
s->refcount_block_cache);
307
}
308
- l2_slice[j] = cpu_to_be64(entry);
309
+ set_l2_entry(s, l2_slice, j, entry);
310
qcow2_cache_entry_mark_dirty(s->l2_table_cache,
311
l2_slice);
312
}
313
@@ -XXX,XX +XXX,XX @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res,
314
315
/* Do the actual checks */
316
for(i = 0; i < s->l2_size; i++) {
317
- l2_entry = be64_to_cpu(l2_table[i]);
318
+ l2_entry = get_l2_entry(s, l2_table, i);
319
320
switch (qcow2_get_cluster_type(bs, l2_entry)) {
321
case QCOW2_CLUSTER_COMPRESSED:
322
@@ -XXX,XX +XXX,XX @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res,
323
QCOW2_OL_INACTIVE_L2;
324
325
l2_entry = QCOW_OFLAG_ZERO;
326
- l2_table[i] = cpu_to_be64(l2_entry);
327
+ set_l2_entry(s, l2_table, i, l2_entry);
328
ret = qcow2_pre_write_overlap_check(bs, ign,
329
l2e_offset, sizeof(uint64_t), false);
330
if (ret < 0) {
331
@@ -XXX,XX +XXX,XX @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res,
332
}
333
334
for (j = 0; j < s->l2_size; j++) {
335
- uint64_t l2_entry = be64_to_cpu(l2_table[j]);
336
+ uint64_t l2_entry = get_l2_entry(s, l2_table, j);
337
uint64_t data_offset = l2_entry & L2E_OFFSET_MASK;
338
QCow2ClusterType cluster_type = qcow2_get_cluster_type(bs, l2_entry);
339
340
@@ -XXX,XX +XXX,XX @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res,
341
"l2_entry=%" PRIx64 " refcount=%" PRIu64 "\n",
342
repair ? "Repairing" : "ERROR", l2_entry, refcount);
343
if (repair) {
344
- l2_table[j] = cpu_to_be64(refcount == 1
345
- ? l2_entry | QCOW_OFLAG_COPIED
346
- : l2_entry & ~QCOW_OFLAG_COPIED);
347
+ set_l2_entry(s, l2_table, j,
348
+ refcount == 1 ?
349
+ l2_entry | QCOW_OFLAG_COPIED :
350
+ l2_entry & ~QCOW_OFLAG_COPIED);
351
l2_dirty++;
352
}
353
}
354
--
355
2.26.2
356
357
diff view generated by jsdifflib
[PULL 07/34] qcow2: Document the Extended L2 Entries feature
Deleted patch
1
From: Alberto Garcia <berto@igalia.com>
2
1
3
Subcluster allocation in qcow2 is implemented by extending the
4
existing L2 table entries and adding additional information to
5
indicate the allocation status of each subcluster.
6
7
This patch documents the changes to the qcow2 format and how they
8
affect the calculation of the L2 cache size.
9
10
Signed-off-by: Alberto Garcia <berto@igalia.com>
11
Reviewed-by: Max Reitz <mreitz@redhat.com>
12
Reviewed-by: Eric Blake <eblake@redhat.com>
13
Message-Id: <5199f2e1c717bcaa58b48142c9062b803145ff7f.1594396418.git.berto@igalia.com>
14
Signed-off-by: Max Reitz <mreitz@redhat.com>
15
---
16
docs/interop/qcow2.txt | 68 ++++++++++++++++++++++++++++++++++++++++--
17
docs/qcow2-cache.txt | 19 +++++++++++-
18
2 files changed, 83 insertions(+), 4 deletions(-)
19
20
diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt
21
index XXXXXXX..XXXXXXX 100644
22
--- a/docs/interop/qcow2.txt
23
+++ b/docs/interop/qcow2.txt
24
@@ -XXX,XX +XXX,XX @@ The first cluster of a qcow2 image contains the file header:
25
as the maximum cluster size and won't be able to open images
26
with larger cluster sizes.
27
28
+ Note: if the image has Extended L2 Entries then cluster_bits
29
+ must be at least 14 (i.e. 16384 byte clusters).
30
+
31
24 - 31: size
32
Virtual disk size in bytes.
33
34
@@ -XXX,XX +XXX,XX @@ the next fields through header_length.
35
clusters. The compression_type field must be
36
present and not zero.
37
38
- Bits 4-63: Reserved (set to 0)
39
+ Bit 4: Extended L2 Entries. If this bit is set then
40
+ L2 table entries use an extended format that
41
+ allows subcluster-based allocation. See the
42
+ Extended L2 Entries section for more details.
43
+
44
+ Bits 5-63: Reserved (set to 0)
45
46
80 - 87: compatible_features
47
Bitmask of compatible features. An implementation can
48
@@ -XXX,XX +XXX,XX @@ cannot be relaxed without an incompatible layout change).
49
Given an offset into the virtual disk, the offset into the image file can be
50
obtained as follows:
51
52
- l2_entries = (cluster_size / sizeof(uint64_t))
53
+ l2_entries = (cluster_size / sizeof(uint64_t)) [*]
54
55
l2_index = (offset / cluster_size) % l2_entries
56
l1_index = (offset / cluster_size) / l2_entries
57
@@ -XXX,XX +XXX,XX @@ obtained as follows:
58
59
return cluster_offset + (offset % cluster_size)
60
61
+ [*] this changes if Extended L2 Entries are enabled, see next section
62
+
63
L1 table entry:
64
65
Bit 0 - 8: Reserved (set to 0)
66
@@ -XXX,XX +XXX,XX @@ Standard Cluster Descriptor:
67
nor is data read from the backing file if the cluster is
68
unallocated.
69
70
- With version 2, this is always 0.
71
+ With version 2 or with extended L2 entries (see the next
72
+ section), this is always 0.
73
74
1 - 8: Reserved (set to 0)
75
76
@@ -XXX,XX +XXX,XX @@ file (except if bit 0 in the Standard Cluster Descriptor is set). If there is
77
no backing file or the backing file is smaller than the image, they shall read
78
zeros for all parts that are not covered by the backing file.
79
80
+== Extended L2 Entries ==
81
+
82
+An image uses Extended L2 Entries if bit 4 is set on the incompatible_features
83
+field of the header.
84
+
85
+In these images standard data clusters are divided into 32 subclusters of the
86
+same size. They are contiguous and start from the beginning of the cluster.
87
+Subclusters can be allocated independently and the L2 entry contains information
88
+indicating the status of each one of them. Compressed data clusters don't have
89
+subclusters so they are treated the same as in images without this feature.
90
+
91
+The size of an extended L2 entry is 128 bits so the number of entries per table
92
+is calculated using this formula:
93
+
94
+ l2_entries = (cluster_size / (2 * sizeof(uint64_t)))
95
+
96
+The first 64 bits have the same format as the standard L2 table entry described
97
+in the previous section, with the exception of bit 0 of the standard cluster
98
+descriptor.
99
+
100
+The last 64 bits contain a subcluster allocation bitmap with this format:
101
+
102
+Subcluster Allocation Bitmap (for standard clusters):
103
+
104
+ Bit 0 - 31: Allocation status (one bit per subcluster)
105
+
106
+ 1: the subcluster is allocated. In this case the
107
+ host cluster offset field must contain a valid
108
+ offset.
109
+ 0: the subcluster is not allocated. In this case
110
+ read requests shall go to the backing file or
111
+ return zeros if there is no backing file data.
112
+
113
+ Bits are assigned starting from the least significant
114
+ one (i.e. bit x is used for subcluster x).
115
+
116
+ 32 - 63 Subcluster reads as zeros (one bit per subcluster)
117
+
118
+ 1: the subcluster reads as zeros. In this case the
119
+ allocation status bit must be unset. The host
120
+ cluster offset field may or may not be set.
121
+ 0: no effect.
122
+
123
+ Bits are assigned starting from the least significant
124
+ one (i.e. bit x is used for subcluster x - 32).
125
+
126
+Subcluster Allocation Bitmap (for compressed clusters):
127
+
128
+ Bit 0 - 63: Reserved (set to 0)
129
+ Compressed clusters don't have subclusters,
130
+ so this field is not used.
131
132
== Snapshots ==
133
134
diff --git a/docs/qcow2-cache.txt b/docs/qcow2-cache.txt
135
index XXXXXXX..XXXXXXX 100644
136
--- a/docs/qcow2-cache.txt
137
+++ b/docs/qcow2-cache.txt
138
@@ -XXX,XX +XXX,XX @@
139
qcow2 L2/refcount cache configuration
140
=====================================
141
-Copyright (C) 2015, 2018 Igalia, S.L.
142
+Copyright (C) 2015, 2018-2020 Igalia, S.L.
143
Author: Alberto Garcia <berto@igalia.com>
144
145
This work is licensed under the terms of the GNU GPL, version 2 or
146
@@ -XXX,XX +XXX,XX @@ support this functionality, and is 0 (disabled) on other platforms.
147
This functionality currently relies on the MADV_DONTNEED argument for
148
madvise() to actually free the memory. This is a Linux-specific feature,
149
so cache-clean-interval is not supported on other systems.
150
+
151
+
152
+Extended L2 Entries
153
+-------------------
154
+All numbers shown in this document are valid for qcow2 images with normal
155
+64-bit L2 entries.
156
+
157
+Images with extended L2 entries need twice as much L2 metadata, so the L2
158
+cache size must be twice as large for the same disk space.
159
+
160
+ disk_size = l2_cache_size * cluster_size / 16
161
+
162
+i.e.
163
+
164
+ l2_cache_size = disk_size * 16 / cluster_size
165
+
166
+Refcount blocks are not affected by this.
167
--
168
2.26.2
169
170
diff view generated by jsdifflib
[PULL 08/34] qcow2: Add dummy has_subclusters() function
Deleted patch
1
From: Alberto Garcia <berto@igalia.com>
2
1
3
This function will be used by the qcow2 code to check if an image has
4
subclusters or not.
5
6
At the moment this simply returns false. Once all patches needed for
7
subcluster support are ready then QEMU will be able to create and
8
read images with subclusters and this function will return the actual
9
value.
10
11
Signed-off-by: Alberto Garcia <berto@igalia.com>
12
Reviewed-by: Eric Blake <eblake@redhat.com>
13
Reviewed-by: Max Reitz <mreitz@redhat.com>
14
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
15
Message-Id: <905526221083581a1b7057bca1585487661c5c13.1594396418.git.berto@igalia.com>
16
Signed-off-by: Max Reitz <mreitz@redhat.com>
17
---
18
block/qcow2.h | 6 ++++++
19
1 file changed, 6 insertions(+)
20
21
diff --git a/block/qcow2.h b/block/qcow2.h
22
index XXXXXXX..XXXXXXX 100644
23
--- a/block/qcow2.h
24
+++ b/block/qcow2.h
25
@@ -XXX,XX +XXX,XX @@ typedef enum QCow2MetadataOverlap {
26
27
#define INV_OFFSET (-1ULL)
28
29
+static inline bool has_subclusters(BDRVQcow2State *s)
30
+{
31
+ /* FIXME: Return false until this feature is complete */
32
+ return false;
33
+}
34
+
35
static inline uint64_t get_l2_entry(BDRVQcow2State *s, uint64_t *l2_slice,
36
int idx)
37
{
38
--
39
2.26.2
40
41
diff view generated by jsdifflib
[PULL 09/34] qcow2: Add subcluster-related fields to BDRVQcow2State
Deleted patch
1
From: Alberto Garcia <berto@igalia.com>
2
1
3
This patch adds the following new fields to BDRVQcow2State:
4
5
- subclusters_per_cluster: Number of subclusters in a cluster
6
- subcluster_size: The size of each subcluster, in bytes
7
- subcluster_bits: No. of bits so 1 << subcluster_bits = subcluster_size
8
9
Images without subclusters are treated as if they had exactly one
10
subcluster per cluster (i.e. subcluster_size = cluster_size).
11
12
Signed-off-by: Alberto Garcia <berto@igalia.com>
13
Reviewed-by: Max Reitz <mreitz@redhat.com>
14
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
15
Message-Id: <55bfeac86b092fa2c9d182a95cbeb479ff7eca4f.1594396418.git.berto@igalia.com>
16
Signed-off-by: Max Reitz <mreitz@redhat.com>
17
---
18
block/qcow2.h | 5 +++++
19
block/qcow2.c | 5 +++++
20
2 files changed, 10 insertions(+)
21
22
diff --git a/block/qcow2.h b/block/qcow2.h
23
index XXXXXXX..XXXXXXX 100644
24
--- a/block/qcow2.h
25
+++ b/block/qcow2.h
26
@@ -XXX,XX +XXX,XX @@
27
/* The cluster reads as all zeros */
28
#define QCOW_OFLAG_ZERO (1ULL << 0)
29
30
+#define QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER 32
31
+
32
#define MIN_CLUSTER_BITS 9
33
#define MAX_CLUSTER_BITS 21
34
35
@@ -XXX,XX +XXX,XX @@ typedef struct BDRVQcow2State {
36
int cluster_bits;
37
int cluster_size;
38
int l2_slice_size;
39
+ int subcluster_bits;
40
+ int subcluster_size;
41
+ int subclusters_per_cluster;
42
int l2_bits;
43
int l2_size;
44
int l1_size;
45
diff --git a/block/qcow2.c b/block/qcow2.c
46
index XXXXXXX..XXXXXXX 100644
47
--- a/block/qcow2.c
48
+++ b/block/qcow2.c
49
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options,
50
}
51
}
52
53
+ s->subclusters_per_cluster =
54
+ has_subclusters(s) ? QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER : 1;
55
+ s->subcluster_size = s->cluster_size / s->subclusters_per_cluster;
56
+ s->subcluster_bits = ctz32(s->subcluster_size);
57
+
58
/* Check support for various header values */
59
if (header.refcount_order > 6) {
60
error_setg(errp, "Reference count entry width too large; may not "
61
--
62
2.26.2
63
64
diff view generated by jsdifflib
[PULL 10/34] qcow2: Add offset_to_sc_index()
Deleted patch
1
From: Alberto Garcia <berto@igalia.com>
2
1
3
For a given offset, return the subcluster number within its cluster
4
(i.e. with 32 subclusters per cluster it returns a number between 0
5
and 31).
6
7
Signed-off-by: Alberto Garcia <berto@igalia.com>
8
Reviewed-by: Max Reitz <mreitz@redhat.com>
9
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
10
Message-Id: <56e3e4ac0d827c6a2f5f259106c5ddb7c4ca2653.1594396418.git.berto@igalia.com>
11
Signed-off-by: Max Reitz <mreitz@redhat.com>
12
---
13
block/qcow2.h | 5 +++++
14
1 file changed, 5 insertions(+)
15
16
diff --git a/block/qcow2.h b/block/qcow2.h
17
index XXXXXXX..XXXXXXX 100644
18
--- a/block/qcow2.h
19
+++ b/block/qcow2.h
20
@@ -XXX,XX +XXX,XX @@ static inline int offset_to_l2_slice_index(BDRVQcow2State *s, int64_t offset)
21
return (offset >> s->cluster_bits) & (s->l2_slice_size - 1);
22
}
23
24
+static inline int offset_to_sc_index(BDRVQcow2State *s, int64_t offset)
25
+{
26
+ return (offset >> s->subcluster_bits) & (s->subclusters_per_cluster - 1);
27
+}
28
+
29
static inline int64_t qcow2_vm_state_offset(BDRVQcow2State *s)
30
{
31
return (int64_t)s->l1_vm_state_index << (s->cluster_bits + s->l2_bits);
32
--
33
2.26.2
34
35
diff view generated by jsdifflib
[PULL 12/34] qcow2: Add l2_entry_size()
Deleted patch
1
From: Alberto Garcia <berto@igalia.com>
2
1
3
qcow2 images with subclusters have 128-bit L2 entries. The first 64
4
bits contain the same information as traditional images and the last
5
64 bits form a bitmap with the status of each individual subcluster.
6
7
Because of that we cannot assume that L2 entries are sizeof(uint64_t)
8
anymore. This function returns the proper value for the image.
9
10
Signed-off-by: Alberto Garcia <berto@igalia.com>
11
Reviewed-by: Max Reitz <mreitz@redhat.com>
12
Message-Id: <d34d578bd0380e739e2dde3e8dd6187d3d249fa9.1594396418.git.berto@igalia.com>
13
Signed-off-by: Max Reitz <mreitz@redhat.com>
14
---
15
block/qcow2.h | 9 +++++++++
16
block/qcow2-cluster.c | 12 ++++++------
17
block/qcow2-refcount.c | 14 ++++++++------
18
block/qcow2.c | 8 ++++----
19
4 files changed, 27 insertions(+), 16 deletions(-)
20
21
diff --git a/block/qcow2.h b/block/qcow2.h
22
index XXXXXXX..XXXXXXX 100644
23
--- a/block/qcow2.h
24
+++ b/block/qcow2.h
25
@@ -XXX,XX +XXX,XX @@
26
27
#define QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER 32
28
29
+/* Size of normal and extended L2 entries */
30
+#define L2E_SIZE_NORMAL (sizeof(uint64_t))
31
+#define L2E_SIZE_EXTENDED (sizeof(uint64_t) * 2)
32
+
33
#define MIN_CLUSTER_BITS 9
34
#define MAX_CLUSTER_BITS 21
35
36
@@ -XXX,XX +XXX,XX @@ static inline bool has_subclusters(BDRVQcow2State *s)
37
return false;
38
}
39
40
+static inline size_t l2_entry_size(BDRVQcow2State *s)
41
+{
42
+ return has_subclusters(s) ? L2E_SIZE_EXTENDED : L2E_SIZE_NORMAL;
43
+}
44
+
45
static inline uint64_t get_l2_entry(BDRVQcow2State *s, uint64_t *l2_slice,
46
int idx)
47
{
48
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
49
index XXXXXXX..XXXXXXX 100644
50
--- a/block/qcow2-cluster.c
51
+++ b/block/qcow2-cluster.c
52
@@ -XXX,XX +XXX,XX @@ static int l2_load(BlockDriverState *bs, uint64_t offset,
53
uint64_t l2_offset, uint64_t **l2_slice)
54
{
55
BDRVQcow2State *s = bs->opaque;
56
- int start_of_slice = sizeof(uint64_t) *
57
+ int start_of_slice = l2_entry_size(s) *
58
(offset_to_l2_index(s, offset) - offset_to_l2_slice_index(s, offset));
59
60
return qcow2_cache_get(bs, s->l2_table_cache, l2_offset + start_of_slice,
61
@@ -XXX,XX +XXX,XX @@ static int l2_allocate(BlockDriverState *bs, int l1_index)
62
63
/* allocate a new l2 entry */
64
65
- l2_offset = qcow2_alloc_clusters(bs, s->l2_size * sizeof(uint64_t));
66
+ l2_offset = qcow2_alloc_clusters(bs, s->l2_size * l2_entry_size(s));
67
if (l2_offset < 0) {
68
ret = l2_offset;
69
goto fail;
70
@@ -XXX,XX +XXX,XX @@ static int l2_allocate(BlockDriverState *bs, int l1_index)
71
72
/* allocate a new entry in the l2 cache */
73
74
- slice_size2 = s->l2_slice_size * sizeof(uint64_t);
75
+ slice_size2 = s->l2_slice_size * l2_entry_size(s);
76
n_slices = s->cluster_size / slice_size2;
77
78
trace_qcow2_l2_allocate_get_empty(bs, l1_index);
79
@@ -XXX,XX +XXX,XX @@ fail:
80
}
81
s->l1_table[l1_index] = old_l2_offset;
82
if (l2_offset > 0) {
83
- qcow2_free_clusters(bs, l2_offset, s->l2_size * sizeof(uint64_t),
84
+ qcow2_free_clusters(bs, l2_offset, s->l2_size * l2_entry_size(s),
85
QCOW2_DISCARD_ALWAYS);
86
}
87
return ret;
88
@@ -XXX,XX +XXX,XX @@ static int get_cluster_table(BlockDriverState *bs, uint64_t offset,
89
90
/* Then decrease the refcount of the old table */
91
if (l2_offset) {
92
- qcow2_free_clusters(bs, l2_offset, s->l2_size * sizeof(uint64_t),
93
+ qcow2_free_clusters(bs, l2_offset, s->l2_size * l2_entry_size(s),
94
QCOW2_DISCARD_OTHER);
95
}
96
97
@@ -XXX,XX +XXX,XX @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table,
98
int ret;
99
int i, j;
100
101
- slice_size2 = s->l2_slice_size * sizeof(uint64_t);
102
+ slice_size2 = s->l2_slice_size * l2_entry_size(s);
103
n_slices = s->cluster_size / slice_size2;
104
105
if (!is_active_l1) {
106
diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
107
index XXXXXXX..XXXXXXX 100644
108
--- a/block/qcow2-refcount.c
109
+++ b/block/qcow2-refcount.c
110
@@ -XXX,XX +XXX,XX @@ int qcow2_update_snapshot_refcount(BlockDriverState *bs,
111
l2_slice = NULL;
112
l1_table = NULL;
113
l1_size2 = l1_size * sizeof(uint64_t);
114
- slice_size2 = s->l2_slice_size * sizeof(uint64_t);
115
+ slice_size2 = s->l2_slice_size * l2_entry_size(s);
116
n_slices = s->cluster_size / slice_size2;
117
118
s->cache_discards = true;
119
@@ -XXX,XX +XXX,XX @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res,
120
int i, l2_size, nb_csectors, ret;
121
122
/* Read L2 table from disk */
123
- l2_size = s->l2_size * sizeof(uint64_t);
124
+ l2_size = s->l2_size * l2_entry_size(s);
125
l2_table = g_malloc(l2_size);
126
127
ret = bdrv_pread(bs->file, l2_offset, l2_table, l2_size);
128
@@ -XXX,XX +XXX,XX @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res,
129
fix & BDRV_FIX_ERRORS ? "Repairing" : "ERROR",
130
offset);
131
if (fix & BDRV_FIX_ERRORS) {
132
+ int idx = i * (l2_entry_size(s) / sizeof(uint64_t));
133
uint64_t l2e_offset =
134
- l2_offset + (uint64_t)i * sizeof(uint64_t);
135
+ l2_offset + (uint64_t)i * l2_entry_size(s);
136
int ign = active ? QCOW2_OL_ACTIVE_L2 :
137
QCOW2_OL_INACTIVE_L2;
138
139
l2_entry = QCOW_OFLAG_ZERO;
140
set_l2_entry(s, l2_table, i, l2_entry);
141
ret = qcow2_pre_write_overlap_check(bs, ign,
142
- l2e_offset, sizeof(uint64_t), false);
143
+ l2e_offset, l2_entry_size(s), false);
144
if (ret < 0) {
145
fprintf(stderr, "ERROR: Overlap check failed\n");
146
res->check_errors++;
147
@@ -XXX,XX +XXX,XX @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res,
148
}
149
150
ret = bdrv_pwrite_sync(bs->file, l2e_offset,
151
- &l2_table[i], sizeof(uint64_t));
152
+ &l2_table[idx],
153
+ l2_entry_size(s));
154
if (ret < 0) {
155
fprintf(stderr, "ERROR: Failed to overwrite L2 "
156
"table entry: %s\n", strerror(-ret));
157
@@ -XXX,XX +XXX,XX @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res,
158
}
159
160
ret = bdrv_pread(bs->file, l2_offset, l2_table,
161
- s->l2_size * sizeof(uint64_t));
162
+ s->l2_size * l2_entry_size(s));
163
if (ret < 0) {
164
fprintf(stderr, "ERROR: Could not read L2 table: %s\n",
165
strerror(-ret));
166
diff --git a/block/qcow2.c b/block/qcow2.c
167
index XXXXXXX..XXXXXXX 100644
168
--- a/block/qcow2.c
169
+++ b/block/qcow2.c
170
@@ -XXX,XX +XXX,XX @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts,
171
uint64_t max_l2_entries = DIV_ROUND_UP(virtual_disk_size, s->cluster_size);
172
/* An L2 table is always one cluster in size so the max cache size
173
* should be a multiple of the cluster size. */
174
- uint64_t max_l2_cache = ROUND_UP(max_l2_entries * sizeof(uint64_t),
175
+ uint64_t max_l2_cache = ROUND_UP(max_l2_entries * l2_entry_size(s),
176
s->cluster_size);
177
178
combined_cache_size_set = qemu_opt_get(opts, QCOW2_OPT_CACHE_SIZE);
179
@@ -XXX,XX +XXX,XX @@ static int qcow2_update_options_prepare(BlockDriverState *bs,
180
}
181
}
182
183
- r->l2_slice_size = l2_cache_entry_size / sizeof(uint64_t);
184
+ r->l2_slice_size = l2_cache_entry_size / l2_entry_size(s);
185
r->l2_table_cache = qcow2_cache_create(bs, l2_cache_size,
186
l2_cache_entry_size);
187
r->refcount_block_cache = qcow2_cache_create(bs, refcount_cache_size,
188
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options,
189
bs->encrypted = true;
190
}
191
192
- s->l2_bits = s->cluster_bits - 3; /* L2 is always one cluster */
193
+ s->l2_bits = s->cluster_bits - ctz32(l2_entry_size(s));
194
s->l2_size = 1 << s->l2_bits;
195
/* 2^(s->refcount_order - 3) is the refcount width in bytes */
196
s->refcount_block_bits = s->cluster_bits - (s->refcount_order - 3);
197
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset,
198
* preallocation. All that matters is that we will not have to allocate
199
* new refcount structures for them.) */
200
nb_new_l2_tables = DIV_ROUND_UP(nb_new_data_clusters,
201
- s->cluster_size / sizeof(uint64_t));
202
+ s->cluster_size / l2_entry_size(s));
203
/* The cluster range may not be aligned to L2 boundaries, so add one L2
204
* table for a potential head/tail */
205
nb_new_l2_tables++;
206
--
207
2.26.2
208
209
diff view generated by jsdifflib
[PULL 18/34] qcow2: Replace QCOW2_CLUSTER_* with QCOW2_SUBCLUSTER_*
Deleted patch
1
From: Alberto Garcia <berto@igalia.com>
2
1
3
In order to support extended L2 entries some functions of the qcow2
4
driver need to start dealing with subclusters instead of clusters.
5
6
qcow2_get_host_offset() is modified to return the subcluster type
7
instead of the cluster type, and all callers are updated to replace
8
all values of QCow2ClusterType with their QCow2SubclusterType
9
equivalents.
10
11
This patch only changes the data types, there are no semantic changes.
12
13
Signed-off-by: Alberto Garcia <berto@igalia.com>
14
Reviewed-by: Max Reitz <mreitz@redhat.com>
15
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
16
Message-Id: <f6c29737c295f32cbee74c903c30b01820363b34.1594396418.git.berto@igalia.com>
17
Signed-off-by: Max Reitz <mreitz@redhat.com>
18
---
19
block/qcow2.h | 2 +-
20
block/qcow2-cluster.c | 10 +++----
21
block/qcow2.c | 70 ++++++++++++++++++++++---------------------
22
3 files changed, 42 insertions(+), 40 deletions(-)
23
24
diff --git a/block/qcow2.h b/block/qcow2.h
25
index XXXXXXX..XXXXXXX 100644
26
--- a/block/qcow2.h
27
+++ b/block/qcow2.h
28
@@ -XXX,XX +XXX,XX @@ int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num,
29
30
int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
31
unsigned int *bytes, uint64_t *host_offset,
32
- QCow2ClusterType *cluster_type);
33
+ QCow2SubclusterType *subcluster_type);
34
int qcow2_alloc_cluster_offset(BlockDriverState *bs, uint64_t offset,
35
unsigned int *bytes, uint64_t *host_offset,
36
QCowL2Meta **m);
37
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
38
index XXXXXXX..XXXXXXX 100644
39
--- a/block/qcow2-cluster.c
40
+++ b/block/qcow2-cluster.c
41
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn do_perform_cow_write(BlockDriverState *bs,
42
* offset that we are interested in.
43
*
44
* On exit, *bytes is the number of bytes starting at offset that have the same
45
- * cluster type and (if applicable) are stored contiguously in the image file.
46
- * The cluster type is stored in *cluster_type.
47
- * Compressed clusters are always returned one by one.
48
+ * subcluster type and (if applicable) are stored contiguously in the image
49
+ * file. The subcluster type is stored in *subcluster_type.
50
+ * Compressed clusters are always processed one by one.
51
*
52
* Returns 0 on success, -errno in error cases.
53
*/
54
int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset,
55
unsigned int *bytes, uint64_t *host_offset,
56
- QCow2ClusterType *cluster_type)
57
+ QCow2SubclusterType *subcluster_type)
58
{
59
BDRVQcow2State *s = bs->opaque;
60
unsigned int l2_index;
61
@@ -XXX,XX +XXX,XX @@ out:
62
assert(bytes_available - offset_in_cluster <= UINT_MAX);
63
*bytes = bytes_available - offset_in_cluster;
64
65
- *cluster_type = type;
66
+ *subcluster_type = qcow2_cluster_to_subcluster_type(type);
67
68
return 0;
69
70
diff --git a/block/qcow2.c b/block/qcow2.c
71
index XXXXXXX..XXXXXXX 100644
72
--- a/block/qcow2.c
73
+++ b/block/qcow2.c
74
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,
75
BDRVQcow2State *s = bs->opaque;
76
uint64_t host_offset;
77
unsigned int bytes;
78
- QCow2ClusterType type;
79
+ QCow2SubclusterType type;
80
int ret, status = 0;
81
82
qemu_co_mutex_lock(&s->lock);
83
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,
84
85
*pnum = bytes;
86
87
- if ((type == QCOW2_CLUSTER_NORMAL || type == QCOW2_CLUSTER_ZERO_ALLOC) &&
88
- !s->crypto) {
89
+ if ((type == QCOW2_SUBCLUSTER_NORMAL ||
90
+ type == QCOW2_SUBCLUSTER_ZERO_ALLOC) && !s->crypto) {
91
*map = host_offset;
92
*file = s->data_file->bs;
93
status |= BDRV_BLOCK_OFFSET_VALID;
94
}
95
- if (type == QCOW2_CLUSTER_ZERO_PLAIN || type == QCOW2_CLUSTER_ZERO_ALLOC) {
96
+ if (type == QCOW2_SUBCLUSTER_ZERO_PLAIN ||
97
+ type == QCOW2_SUBCLUSTER_ZERO_ALLOC) {
98
status |= BDRV_BLOCK_ZERO;
99
- } else if (type != QCOW2_CLUSTER_UNALLOCATED) {
100
+ } else if (type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN) {
101
status |= BDRV_BLOCK_DATA;
102
}
103
if (s->metadata_preallocation && (status & BDRV_BLOCK_DATA) &&
104
@@ -XXX,XX +XXX,XX @@ typedef struct Qcow2AioTask {
105
AioTask task;
106
107
BlockDriverState *bs;
108
- QCow2ClusterType cluster_type; /* only for read */
109
+ QCow2SubclusterType subcluster_type; /* only for read */
110
uint64_t host_offset; /* or full descriptor in compressed clusters */
111
uint64_t offset;
112
uint64_t bytes;
113
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_task_entry(AioTask *task);
114
static coroutine_fn int qcow2_add_task(BlockDriverState *bs,
115
AioTaskPool *pool,
116
AioTaskFunc func,
117
- QCow2ClusterType cluster_type,
118
+ QCow2SubclusterType subcluster_type,
119
uint64_t host_offset,
120
uint64_t offset,
121
uint64_t bytes,
122
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs,
123
*task = (Qcow2AioTask) {
124
.task.func = func,
125
.bs = bs,
126
- .cluster_type = cluster_type,
127
+ .subcluster_type = subcluster_type,
128
.qiov = qiov,
129
.host_offset = host_offset,
130
.offset = offset,
131
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs,
132
133
trace_qcow2_add_task(qemu_coroutine_self(), bs, pool,
134
func == qcow2_co_preadv_task_entry ? "read" : "write",
135
- cluster_type, host_offset, offset, bytes,
136
+ subcluster_type, host_offset, offset, bytes,
137
qiov, qiov_offset);
138
139
if (!pool) {
140
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs,
141
}
142
143
static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs,
144
- QCow2ClusterType cluster_type,
145
+ QCow2SubclusterType subc_type,
146
uint64_t host_offset,
147
uint64_t offset, uint64_t bytes,
148
QEMUIOVector *qiov,
149
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs,
150
{
151
BDRVQcow2State *s = bs->opaque;
152
153
- switch (cluster_type) {
154
- case QCOW2_CLUSTER_ZERO_PLAIN:
155
- case QCOW2_CLUSTER_ZERO_ALLOC:
156
+ switch (subc_type) {
157
+ case QCOW2_SUBCLUSTER_ZERO_PLAIN:
158
+ case QCOW2_SUBCLUSTER_ZERO_ALLOC:
159
/* Both zero types are handled in qcow2_co_preadv_part */
160
g_assert_not_reached();
161
162
- case QCOW2_CLUSTER_UNALLOCATED:
163
+ case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN:
164
assert(bs->backing); /* otherwise handled in qcow2_co_preadv_part */
165
166
BLKDBG_EVENT(bs->file, BLKDBG_READ_BACKING_AIO);
167
return bdrv_co_preadv_part(bs->backing, offset, bytes,
168
qiov, qiov_offset, 0);
169
170
- case QCOW2_CLUSTER_COMPRESSED:
171
+ case QCOW2_SUBCLUSTER_COMPRESSED:
172
return qcow2_co_preadv_compressed(bs, host_offset,
173
offset, bytes, qiov, qiov_offset);
174
175
- case QCOW2_CLUSTER_NORMAL:
176
+ case QCOW2_SUBCLUSTER_NORMAL:
177
if (bs->encrypted) {
178
return qcow2_co_preadv_encrypted(bs, host_offset,
179
offset, bytes, qiov, qiov_offset);
180
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_task_entry(AioTask *task)
181
182
assert(!t->l2meta);
183
184
- return qcow2_co_preadv_task(t->bs, t->cluster_type, t->host_offset,
185
- t->offset, t->bytes, t->qiov, t->qiov_offset);
186
+ return qcow2_co_preadv_task(t->bs, t->subcluster_type,
187
+ t->host_offset, t->offset, t->bytes,
188
+ t->qiov, t->qiov_offset);
189
}
190
191
static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
192
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
193
int ret = 0;
194
unsigned int cur_bytes; /* number of bytes in current iteration */
195
uint64_t host_offset = 0;
196
- QCow2ClusterType type;
197
+ QCow2SubclusterType type;
198
AioTaskPool *aio = NULL;
199
200
while (bytes != 0 && aio_task_pool_status(aio) == 0) {
201
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
202
goto out;
203
}
204
205
- if (type == QCOW2_CLUSTER_ZERO_PLAIN ||
206
- type == QCOW2_CLUSTER_ZERO_ALLOC ||
207
- (type == QCOW2_CLUSTER_UNALLOCATED && !bs->backing))
208
+ if (type == QCOW2_SUBCLUSTER_ZERO_PLAIN ||
209
+ type == QCOW2_SUBCLUSTER_ZERO_ALLOC ||
210
+ (type == QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN && !bs->backing))
211
{
212
qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes);
213
} else {
214
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev_task_entry(AioTask *task)
215
{
216
Qcow2AioTask *t = container_of(task, Qcow2AioTask, task);
217
218
- assert(!t->cluster_type);
219
+ assert(!t->subcluster_type);
220
221
return qcow2_co_pwritev_task(t->bs, t->host_offset,
222
t->offset, t->bytes, t->qiov, t->qiov_offset,
223
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs,
224
if (head || tail) {
225
uint64_t off;
226
unsigned int nr;
227
- QCow2ClusterType type;
228
+ QCow2SubclusterType type;
229
230
assert(head + bytes <= s->cluster_size);
231
232
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs,
233
nr = s->cluster_size;
234
ret = qcow2_get_host_offset(bs, offset, &nr, &off, &type);
235
if (ret < 0 ||
236
- (type != QCOW2_CLUSTER_UNALLOCATED &&
237
- type != QCOW2_CLUSTER_ZERO_PLAIN &&
238
- type != QCOW2_CLUSTER_ZERO_ALLOC)) {
239
+ (type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN &&
240
+ type != QCOW2_SUBCLUSTER_ZERO_PLAIN &&
241
+ type != QCOW2_SUBCLUSTER_ZERO_ALLOC)) {
242
qemu_co_mutex_unlock(&s->lock);
243
return -ENOTSUP;
244
}
245
@@ -XXX,XX +XXX,XX @@ qcow2_co_copy_range_from(BlockDriverState *bs,
246
247
while (bytes != 0) {
248
uint64_t copy_offset = 0;
249
- QCow2ClusterType type;
250
+ QCow2SubclusterType type;
251
/* prepare next request */
252
cur_bytes = MIN(bytes, INT_MAX);
253
cur_write_flags = write_flags;
254
@@ -XXX,XX +XXX,XX @@ qcow2_co_copy_range_from(BlockDriverState *bs,
255
}
256
257
switch (type) {
258
- case QCOW2_CLUSTER_UNALLOCATED:
259
+ case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN:
260
if (bs->backing && bs->backing->bs) {
261
int64_t backing_length = bdrv_getlength(bs->backing->bs);
262
if (src_offset >= backing_length) {
263
@@ -XXX,XX +XXX,XX @@ qcow2_co_copy_range_from(BlockDriverState *bs,
264
}
265
break;
266
267
- case QCOW2_CLUSTER_ZERO_PLAIN:
268
- case QCOW2_CLUSTER_ZERO_ALLOC:
269
+ case QCOW2_SUBCLUSTER_ZERO_PLAIN:
270
+ case QCOW2_SUBCLUSTER_ZERO_ALLOC:
271
cur_write_flags |= BDRV_REQ_ZERO_WRITE;
272
break;
273
274
- case QCOW2_CLUSTER_COMPRESSED:
275
+ case QCOW2_SUBCLUSTER_COMPRESSED:
276
ret = -ENOTSUP;
277
goto out;
278
279
- case QCOW2_CLUSTER_NORMAL:
280
+ case QCOW2_SUBCLUSTER_NORMAL:
281
child = s->data_file;
282
break;
283
284
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev_compressed_task_entry(AioTask *task)
285
{
286
Qcow2AioTask *t = container_of(task, Qcow2AioTask, task);
287
288
- assert(!t->cluster_type && !t->l2meta);
289
+ assert(!t->subcluster_type && !t->l2meta);
290
291
return qcow2_co_pwritev_compressed_task(t->bs, t->offset, t->bytes, t->qiov,
292
t->qiov_offset);
293
--
294
2.26.2
295
296
diff view generated by jsdifflib
[PULL 19/34] qcow2: Handle QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC
Deleted patch
1
From: Alberto Garcia <berto@igalia.com>
2
1
3
When dealing with subcluster types there is a new value called
4
QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC that has no equivalent in
5
QCow2ClusterType.
6
7
This patch handles that value in all places where subcluster types
8
are processed.
9
10
Signed-off-by: Alberto Garcia <berto@igalia.com>
11
Reviewed-by: Max Reitz <mreitz@redhat.com>
12
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
13
Message-Id: <bf09e2e2439a468a901bb96ace411eed9ee50295.1594396418.git.berto@igalia.com>
14
Signed-off-by: Max Reitz <mreitz@redhat.com>
15
---
16
block/qcow2.c | 12 +++++++++---
17
1 file changed, 9 insertions(+), 3 deletions(-)
18
19
diff --git a/block/qcow2.c b/block/qcow2.c
20
index XXXXXXX..XXXXXXX 100644
21
--- a/block/qcow2.c
22
+++ b/block/qcow2.c
23
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,
24
*pnum = bytes;
25
26
if ((type == QCOW2_SUBCLUSTER_NORMAL ||
27
- type == QCOW2_SUBCLUSTER_ZERO_ALLOC) && !s->crypto) {
28
+ type == QCOW2_SUBCLUSTER_ZERO_ALLOC ||
29
+ type == QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC) && !s->crypto) {
30
*map = host_offset;
31
*file = s->data_file->bs;
32
status |= BDRV_BLOCK_OFFSET_VALID;
33
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,
34
if (type == QCOW2_SUBCLUSTER_ZERO_PLAIN ||
35
type == QCOW2_SUBCLUSTER_ZERO_ALLOC) {
36
status |= BDRV_BLOCK_ZERO;
37
- } else if (type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN) {
38
+ } else if (type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN &&
39
+ type != QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC) {
40
status |= BDRV_BLOCK_DATA;
41
}
42
if (s->metadata_preallocation && (status & BDRV_BLOCK_DATA) &&
43
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs,
44
g_assert_not_reached();
45
46
case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN:
47
+ case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC:
48
assert(bs->backing); /* otherwise handled in qcow2_co_preadv_part */
49
50
BLKDBG_EVENT(bs->file, BLKDBG_READ_BACKING_AIO);
51
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
52
53
if (type == QCOW2_SUBCLUSTER_ZERO_PLAIN ||
54
type == QCOW2_SUBCLUSTER_ZERO_ALLOC ||
55
- (type == QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN && !bs->backing))
56
+ (type == QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN && !bs->backing) ||
57
+ (type == QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC && !bs->backing))
58
{
59
qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes);
60
} else {
61
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs,
62
ret = qcow2_get_host_offset(bs, offset, &nr, &off, &type);
63
if (ret < 0 ||
64
(type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN &&
65
+ type != QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC &&
66
type != QCOW2_SUBCLUSTER_ZERO_PLAIN &&
67
type != QCOW2_SUBCLUSTER_ZERO_ALLOC)) {
68
qemu_co_mutex_unlock(&s->lock);
69
@@ -XXX,XX +XXX,XX @@ qcow2_co_copy_range_from(BlockDriverState *bs,
70
71
switch (type) {
72
case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN:
73
+ case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC:
74
if (bs->backing && bs->backing->bs) {
75
int64_t backing_length = bdrv_getlength(bs->backing->bs);
76
if (src_offset >= backing_length) {
77
--
78
2.26.2
79
80
diff view generated by jsdifflib
[PULL 28/34] qcow2: Add subcluster support to qcow2_co_pwrite_zeroes()
[PULL 19/23] block: copy-before-write: realize snapshot-access API
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
This works now at the subcluster level and pwrite_zeroes_alignment is
3
Current scheme of image fleecing looks like this:
4
updated accordingly.
4
5
5
[guest] [NBD export]
6
qcow2_cluster_zeroize() is turned into qcow2_subcluster_zeroize() with
6
| |
7
the following changes:
7
|root | root
8
8
v v
9
- The request can now be subcluster-aligned.
9
[copy-before-write] -----> [temp.qcow2]
10
10
| target |
11
- The cluster-aligned body of the request is still zeroized using
11
|file |backing
12
zero_in_l2_slice() as before.
12
v |
13
13
[active disk] <-------------+
14
- The subcluster-aligned head and tail of the request are zeroized
14
15
with the new zero_l2_subclusters() function.
15
- On guest writes copy-before-write filter copies old data from active
16
16
disk to temp.qcow2. So fleecing client (NBD export) when reads
17
There is just one thing to take into account for a possible future
17
changed regions from temp.qcow2 image and unchanged from active disk
18
improvement: compressed clusters cannot be partially zeroized so
18
through backing link.
19
zero_l2_subclusters() on the head or the tail can return -ENOTSUP.
19
20
This makes the caller repeat the *complete* request and write actual
20
This patch makes possible new image fleecing scheme:
21
zeroes to disk. This is sub-optimal because
21
22
22
[guest] [NBD export]
23
1) if the head area was compressed we would still be able to use
23
| |
24
the fast path for the body and possibly the tail.
24
| root | root
25
25
v file v
26
2) if the tail area was compressed we are writing zeroes to the
26
[copy-before-write]<------[snapshot-access]
27
head and the body areas, which are already zeroized.
27
| |
28
28
| file | target
29
Signed-off-by: Alberto Garcia <berto@igalia.com>
29
v v
30
Reviewed-by: Max Reitz <mreitz@redhat.com>
30
[active-disk] [temp.img]
31
Message-Id: <17e05e2ee7e12f10dcf012da81e83ebe27eb3bef.1594396418.git.berto@igalia.com>
31
32
Signed-off-by: Max Reitz <mreitz@redhat.com>
32
- copy-before-write does CBW operations and also provides
33
snapshot-access API. The API may be accessed through
34
snapshot-access driver.
35
36
Benefits of new scheme:
37
38
1. Access control: if remote client try to read data that not covered
39
by original dirty bitmap used on copy-before-write open, client gets
40
-EACCES.
41
42
2. Discard support: if remote client do DISCARD, this additionally to
43
discarding data in temp.img informs block-copy process to not copy
44
these clusters. Next read from discarded area will return -EACCES.
45
This is significant thing: when fleecing user reads data that was
46
not yet copied to temp.img, we can avoid copying it on further guest
47
write.
48
49
3. Synchronisation between client reads and block-copy write is more
50
efficient. In old scheme we just rely on BDRV_REQ_SERIALISING flag
51
used for writes to temp.qcow2. New scheme is less blocking:
52
- fleecing reads are never blocked: if data region is untouched or
53
in-flight, we just read from active-disk, otherwise we read from
54
temp.img
55
- writes to temp.img are not blocked by fleecing reads
56
- still, guest writes of-course are blocked by in-flight fleecing
57
reads, that currently read from active-disk - it's the minimum
58
necessary blocking
59
60
4. Temporary image may be of any format, as we don't rely on backing
61
feature.
62
63
5. Permission relation are simplified. With old scheme we have to share
64
write permission on target child of copy-before-write, otherwise
65
backing link conflicts with copy-before-write file child write
66
permissions. With new scheme we don't have backing link, and
67
copy-before-write node may have unshared access to temporary node.
68
(Not realized in this commit, will be in future).
69
70
6. Having control on fleecing reads we'll be able to implement
71
alternative behavior on failed copy-before-write operations.
72
Currently we just break guest request (that's a historical behavior
73
of backup). But in some scenarios it's a bad behavior: better
74
is to drop the backup as failed but don't break guest request.
75
With new scheme we can simply unset some bits in a bitmap on CBW
76
failure and further fleecing reads will -EACCES, or something like
77
this. (Not implemented in this commit, will be in future)
78
Additional application for this is implementing timeout for CBW
79
operations.
80
81
Iotest 257 output is updated, as two more bitmaps now live in
82
copy-before-write filter.
83
84
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
85
Message-Id: <20220303194349.2304213-13-vsementsov@virtuozzo.com>
86
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
33
---
87
---
34
block/qcow2.h | 4 +--
88
block/copy-before-write.c | 212 ++++++++++++++++++++++++++++++++++-
35
block/qcow2-cluster.c | 81 +++++++++++++++++++++++++++++++++++++++----
89
tests/qemu-iotests/257.out | 224 +++++++++++++++++++++++++++++++++++++
36
block/qcow2.c | 33 +++++++++---------
90
2 files changed, 435 insertions(+), 1 deletion(-)
37
3 files changed, 94 insertions(+), 24 deletions(-)
91
38
92
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
39
diff --git a/block/qcow2.h b/block/qcow2.h
40
index XXXXXXX..XXXXXXX 100644
93
index XXXXXXX..XXXXXXX 100644
41
--- a/block/qcow2.h
94
--- a/block/copy-before-write.c
42
+++ b/block/qcow2.h
95
+++ b/block/copy-before-write.c
43
@@ -XXX,XX +XXX,XX @@ void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m);
96
@@ -XXX,XX +XXX,XX @@
44
int qcow2_cluster_discard(BlockDriverState *bs, uint64_t offset,
97
#include "block/block-copy.h"
45
uint64_t bytes, enum qcow2_discard_type type,
98
46
bool full_discard);
99
#include "block/copy-before-write.h"
47
-int qcow2_cluster_zeroize(BlockDriverState *bs, uint64_t offset,
100
+#include "block/reqlist.h"
48
- uint64_t bytes, int flags);
101
49
+int qcow2_subcluster_zeroize(BlockDriverState *bs, uint64_t offset,
102
#include "qapi/qapi-visit-block-core.h"
50
+ uint64_t bytes, int flags);
103
51
104
typedef struct BDRVCopyBeforeWriteState {
52
int qcow2_expand_zero_clusters(BlockDriverState *bs,
105
BlockCopyState *bcs;
53
BlockDriverAmendStatusCB *status_cb,
106
BdrvChild *target;
54
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
107
+
55
index XXXXXXX..XXXXXXX 100644
108
+ /*
56
--- a/block/qcow2-cluster.c
109
+ * @lock: protects access to @access_bitmap, @done_bitmap and
57
+++ b/block/qcow2-cluster.c
110
+ * @frozen_read_reqs
58
@@ -XXX,XX +XXX,XX @@ static int zero_in_l2_slice(BlockDriverState *bs, uint64_t offset,
111
+ */
59
return nb_clusters;
112
+ CoMutex lock;
113
+
114
+ /*
115
+ * @access_bitmap: represents areas allowed for reading by fleecing user.
116
+ * Reading from non-dirty areas leads to -EACCES.
117
+ */
118
+ BdrvDirtyBitmap *access_bitmap;
119
+
120
+ /*
121
+ * @done_bitmap: represents areas that was successfully copied to @target by
122
+ * copy-before-write operations.
123
+ */
124
+ BdrvDirtyBitmap *done_bitmap;
125
+
126
+ /*
127
+ * @frozen_read_reqs: current read requests for fleecing user in bs->file
128
+ * node. These areas must not be rewritten by guest.
129
+ */
130
+ BlockReqList frozen_read_reqs;
131
} BDRVCopyBeforeWriteState;
132
133
static coroutine_fn int cbw_co_preadv(
134
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int cbw_co_preadv(
135
return bdrv_co_preadv(bs->file, offset, bytes, qiov, flags);
60
}
136
}
61
137
62
-int qcow2_cluster_zeroize(BlockDriverState *bs, uint64_t offset,
138
+/*
63
- uint64_t bytes, int flags)
139
+ * Do copy-before-write operation.
64
+static int zero_l2_subclusters(BlockDriverState *bs, uint64_t offset,
140
+ *
65
+ unsigned nb_subclusters)
141
+ * On failure guest request must be failed too.
66
+{
142
+ *
67
+ BDRVQcow2State *s = bs->opaque;
143
+ * On success, we also wait for all in-flight fleecing read requests in source
68
+ uint64_t *l2_slice;
144
+ * node, and it's guaranteed that after cbw_do_copy_before_write() successful
69
+ uint64_t old_l2_bitmap, l2_bitmap;
145
+ * return there are no such requests and they will never appear.
70
+ int l2_index, ret, sc = offset_to_sc_index(s, offset);
146
+ */
71
+
147
static coroutine_fn int cbw_do_copy_before_write(BlockDriverState *bs,
72
+ /* For full clusters use zero_in_l2_slice() instead */
148
uint64_t offset, uint64_t bytes, BdrvRequestFlags flags)
73
+ assert(nb_subclusters > 0 && nb_subclusters < s->subclusters_per_cluster);
149
{
74
+ assert(sc + nb_subclusters <= s->subclusters_per_cluster);
150
BDRVCopyBeforeWriteState *s = bs->opaque;
75
+ assert(offset_into_subcluster(s, offset) == 0);
151
+ int ret;
76
+
152
uint64_t off, end;
77
+ ret = get_cluster_table(bs, offset, &l2_slice, &l2_index);
153
int64_t cluster_size = block_copy_cluster_size(s->bcs);
154
155
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int cbw_do_copy_before_write(BlockDriverState *bs,
156
off = QEMU_ALIGN_DOWN(offset, cluster_size);
157
end = QEMU_ALIGN_UP(offset + bytes, cluster_size);
158
159
- return block_copy(s->bcs, off, end - off, true);
160
+ ret = block_copy(s->bcs, off, end - off, true);
78
+ if (ret < 0) {
161
+ if (ret < 0) {
79
+ return ret;
162
+ return ret;
80
+ }
163
+ }
81
+
164
+
82
+ switch (qcow2_get_cluster_type(bs, get_l2_entry(s, l2_slice, l2_index))) {
165
+ WITH_QEMU_LOCK_GUARD(&s->lock) {
83
+ case QCOW2_CLUSTER_COMPRESSED:
166
+ bdrv_set_dirty_bitmap(s->done_bitmap, off, end - off);
84
+ ret = -ENOTSUP; /* We cannot partially zeroize compressed clusters */
167
+ reqlist_wait_all(&s->frozen_read_reqs, off, end - off, &s->lock);
85
+ goto out;
168
+ }
86
+ case QCOW2_CLUSTER_NORMAL:
169
+
87
+ case QCOW2_CLUSTER_UNALLOCATED:
170
+ return 0;
88
+ break;
171
}
89
+ default:
172
90
+ g_assert_not_reached();
173
static int coroutine_fn cbw_co_pdiscard(BlockDriverState *bs,
91
+ }
174
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn cbw_co_flush(BlockDriverState *bs)
92
+
175
return bdrv_co_flush(bs->file->bs);
93
+ old_l2_bitmap = l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index);
176
}
94
+
177
95
+ l2_bitmap |= QCOW_OFLAG_SUB_ZERO_RANGE(sc, sc + nb_subclusters);
178
+/*
96
+ l2_bitmap &= ~QCOW_OFLAG_SUB_ALLOC_RANGE(sc, sc + nb_subclusters);
179
+ * If @offset not accessible - return NULL.
97
+
180
+ *
98
+ if (old_l2_bitmap != l2_bitmap) {
181
+ * Otherwise, set @pnum to some bytes that accessible from @file (@file is set
99
+ set_l2_bitmap(s, l2_slice, l2_index, l2_bitmap);
182
+ * to bs->file or to s->target). Return newly allocated BlockReq object that
100
+ qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice);
183
+ * should be than passed to cbw_snapshot_read_unlock().
101
+ }
184
+ *
102
+
185
+ * It's guaranteed that guest writes will not interact in the region until
103
+ ret = 0;
186
+ * cbw_snapshot_read_unlock() called.
104
+out:
187
+ */
105
+ qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice);
188
+static BlockReq *cbw_snapshot_read_lock(BlockDriverState *bs,
189
+ int64_t offset, int64_t bytes,
190
+ int64_t *pnum, BdrvChild **file)
191
+{
192
+ BDRVCopyBeforeWriteState *s = bs->opaque;
193
+ BlockReq *req = g_new(BlockReq, 1);
194
+ bool done;
195
+
196
+ QEMU_LOCK_GUARD(&s->lock);
197
+
198
+ if (bdrv_dirty_bitmap_next_zero(s->access_bitmap, offset, bytes) != -1) {
199
+ g_free(req);
200
+ return NULL;
201
+ }
202
+
203
+ done = bdrv_dirty_bitmap_status(s->done_bitmap, offset, bytes, pnum);
204
+ if (done) {
205
+ /*
206
+ * Special invalid BlockReq, that is handled in
207
+ * cbw_snapshot_read_unlock(). We don't need to lock something to read
208
+ * from s->target.
209
+ */
210
+ *req = (BlockReq) {.offset = -1, .bytes = -1};
211
+ *file = s->target;
212
+ } else {
213
+ reqlist_init_req(&s->frozen_read_reqs, req, offset, bytes);
214
+ *file = bs->file;
215
+ }
216
+
217
+ return req;
218
+}
219
+
220
+static void cbw_snapshot_read_unlock(BlockDriverState *bs, BlockReq *req)
221
+{
222
+ BDRVCopyBeforeWriteState *s = bs->opaque;
223
+
224
+ if (req->offset == -1 && req->bytes == -1) {
225
+ g_free(req);
226
+ return;
227
+ }
228
+
229
+ QEMU_LOCK_GUARD(&s->lock);
230
+
231
+ reqlist_remove_req(req);
232
+ g_free(req);
233
+}
234
+
235
+static coroutine_fn int
236
+cbw_co_preadv_snapshot(BlockDriverState *bs, int64_t offset, int64_t bytes,
237
+ QEMUIOVector *qiov, size_t qiov_offset)
238
+{
239
+ BlockReq *req;
240
+ BdrvChild *file;
241
+ int ret;
242
+
243
+ /* TODO: upgrade to async loop using AioTask */
244
+ while (bytes) {
245
+ int64_t cur_bytes;
246
+
247
+ req = cbw_snapshot_read_lock(bs, offset, bytes, &cur_bytes, &file);
248
+ if (!req) {
249
+ return -EACCES;
250
+ }
251
+
252
+ ret = bdrv_co_preadv_part(file, offset, cur_bytes,
253
+ qiov, qiov_offset, 0);
254
+ cbw_snapshot_read_unlock(bs, req);
255
+ if (ret < 0) {
256
+ return ret;
257
+ }
258
+
259
+ bytes -= cur_bytes;
260
+ offset += cur_bytes;
261
+ qiov_offset += cur_bytes;
262
+ }
263
+
264
+ return 0;
265
+}
266
+
267
+static int coroutine_fn
268
+cbw_co_snapshot_block_status(BlockDriverState *bs,
269
+ bool want_zero, int64_t offset, int64_t bytes,
270
+ int64_t *pnum, int64_t *map,
271
+ BlockDriverState **file)
272
+{
273
+ BDRVCopyBeforeWriteState *s = bs->opaque;
274
+ BlockReq *req;
275
+ int ret;
276
+ int64_t cur_bytes;
277
+ BdrvChild *child;
278
+
279
+ req = cbw_snapshot_read_lock(bs, offset, bytes, &cur_bytes, &child);
280
+ if (!req) {
281
+ return -EACCES;
282
+ }
283
+
284
+ ret = bdrv_block_status(child->bs, offset, cur_bytes, pnum, map, file);
285
+ if (child == s->target) {
286
+ /*
287
+ * We refer to s->target only for areas that we've written to it.
288
+ * And we can not report unallocated blocks in s->target: this will
289
+ * break generic block-status-above logic, that will go to
290
+ * copy-before-write filtered child in this case.
291
+ */
292
+ assert(ret & BDRV_BLOCK_ALLOCATED);
293
+ }
294
+
295
+ cbw_snapshot_read_unlock(bs, req);
106
+
296
+
107
+ return ret;
297
+ return ret;
108
+}
298
+}
109
+
299
+
110
+int qcow2_subcluster_zeroize(BlockDriverState *bs, uint64_t offset,
300
+static int coroutine_fn cbw_co_pdiscard_snapshot(BlockDriverState *bs,
111
+ uint64_t bytes, int flags)
301
+ int64_t offset, int64_t bytes)
112
{
302
+{
113
BDRVQcow2State *s = bs->opaque;
303
+ BDRVCopyBeforeWriteState *s = bs->opaque;
114
uint64_t end_offset = offset + bytes;
304
+
115
uint64_t nb_clusters;
305
+ WITH_QEMU_LOCK_GUARD(&s->lock) {
116
+ unsigned head, tail;
306
+ bdrv_reset_dirty_bitmap(s->access_bitmap, offset, bytes);
117
int64_t cleared;
307
+ }
118
int ret;
308
+
119
309
+ block_copy_reset(s->bcs, offset, bytes);
120
@@ -XXX,XX +XXX,XX @@ int qcow2_cluster_zeroize(BlockDriverState *bs, uint64_t offset,
310
+
311
+ return bdrv_co_pdiscard(s->target, offset, bytes);
312
+}
313
+
314
static void cbw_refresh_filename(BlockDriverState *bs)
315
{
316
pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
317
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
318
{
319
BDRVCopyBeforeWriteState *s = bs->opaque;
320
BdrvDirtyBitmap *bitmap = NULL;
321
+ int64_t cluster_size;
322
323
bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
324
BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
325
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
326
return -EINVAL;
121
}
327
}
122
328
123
/* Caller must pass aligned values, except at image end */
329
+ cluster_size = block_copy_cluster_size(s->bcs);
124
- assert(QEMU_IS_ALIGNED(offset, s->cluster_size));
330
+
125
- assert(QEMU_IS_ALIGNED(end_offset, s->cluster_size) ||
331
+ s->done_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
126
+ assert(offset_into_subcluster(s, offset) == 0);
332
+ if (!s->done_bitmap) {
127
+ assert(offset_into_subcluster(s, end_offset) == 0 ||
333
+ return -EINVAL;
128
end_offset >= bs->total_sectors << BDRV_SECTOR_BITS);
334
+ }
129
335
+ bdrv_disable_dirty_bitmap(s->done_bitmap);
130
/*
336
+
131
@@ -XXX,XX +XXX,XX @@ int qcow2_cluster_zeroize(BlockDriverState *bs, uint64_t offset,
337
+ /* s->access_bitmap starts equal to bcs bitmap */
132
return -ENOTSUP;
338
+ s->access_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
133
}
339
+ if (!s->access_bitmap) {
134
340
+ return -EINVAL;
135
- /* Each L2 slice is handled by its own loop iteration */
341
+ }
136
- nb_clusters = size_to_clusters(s, bytes);
342
+ bdrv_disable_dirty_bitmap(s->access_bitmap);
137
+ head = MIN(end_offset, ROUND_UP(offset, s->cluster_size)) - offset;
343
+ bdrv_dirty_bitmap_merge_internal(s->access_bitmap,
138
+ offset += head;
344
+ block_copy_dirty_bitmap(s->bcs), NULL,
139
+
345
+ true);
140
+ tail = (end_offset >= bs->total_sectors << BDRV_SECTOR_BITS) ? 0 :
346
+
141
+ end_offset - MAX(offset, start_of_cluster(s, end_offset));
347
+ qemu_co_mutex_init(&s->lock);
142
+ end_offset -= tail;
348
+ QLIST_INIT(&s->frozen_read_reqs);
143
349
+
144
s->cache_discards = true;
350
return 0;
145
351
}
146
+ if (head) {
352
147
+ ret = zero_l2_subclusters(bs, offset - head,
353
@@ -XXX,XX +XXX,XX @@ static void cbw_close(BlockDriverState *bs)
148
+ size_to_subclusters(s, head));
354
{
149
+ if (ret < 0) {
355
BDRVCopyBeforeWriteState *s = bs->opaque;
150
+ goto fail;
356
151
+ }
357
+ bdrv_release_dirty_bitmap(s->access_bitmap);
152
+ }
358
+ bdrv_release_dirty_bitmap(s->done_bitmap);
153
+
359
+
154
+ /* Each L2 slice is handled by its own loop iteration */
360
block_copy_state_free(s->bcs);
155
+ nb_clusters = size_to_clusters(s, end_offset - offset);
361
s->bcs = NULL;
156
+
362
}
157
while (nb_clusters > 0) {
363
@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_cbw_filter = {
158
cleared = zero_in_l2_slice(bs, offset, nb_clusters, flags);
364
.bdrv_co_pdiscard = cbw_co_pdiscard,
159
if (cleared < 0) {
365
.bdrv_co_flush = cbw_co_flush,
160
@@ -XXX,XX +XXX,XX @@ int qcow2_cluster_zeroize(BlockDriverState *bs, uint64_t offset,
366
161
offset += (cleared * s->cluster_size);
367
+ .bdrv_co_preadv_snapshot = cbw_co_preadv_snapshot,
162
}
368
+ .bdrv_co_pdiscard_snapshot = cbw_co_pdiscard_snapshot,
163
369
+ .bdrv_co_snapshot_block_status = cbw_co_snapshot_block_status,
164
+ if (tail) {
370
+
165
+ ret = zero_l2_subclusters(bs, end_offset, size_to_subclusters(s, tail));
371
.bdrv_refresh_filename = cbw_refresh_filename,
166
+ if (ret < 0) {
372
167
+ goto fail;
373
.bdrv_child_perm = cbw_child_perm,
168
+ }
374
diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out
169
+ }
170
+
171
ret = 0;
172
fail:
173
s->cache_discards = false;
174
diff --git a/block/qcow2.c b/block/qcow2.c
175
index XXXXXXX..XXXXXXX 100644
375
index XXXXXXX..XXXXXXX 100644
176
--- a/block/qcow2.c
376
--- a/tests/qemu-iotests/257.out
177
+++ b/block/qcow2.c
377
+++ b/tests/qemu-iotests/257.out
178
@@ -XXX,XX +XXX,XX @@ static void qcow2_refresh_limits(BlockDriverState *bs, Error **errp)
378
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
179
/* Encryption works on a sector granularity */
379
{"return": ""}
180
bs->bl.request_alignment = qcrypto_block_get_sector_size(s->crypto);
380
{
181
}
381
"bitmaps": {
182
- bs->bl.pwrite_zeroes_alignment = s->cluster_size;
382
+ "backup-top": [
183
+ bs->bl.pwrite_zeroes_alignment = s->subcluster_size;
383
+ {
184
bs->bl.pdiscard_alignment = s->cluster_size;
384
+ "busy": false,
185
}
385
+ "count": 67108864,
186
386
+ "granularity": 65536,
187
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs,
387
+ "persistent": false,
188
int ret;
388
+ "recording": false
189
BDRVQcow2State *s = bs->opaque;
389
+ },
190
390
+ {
191
- uint32_t head = offset % s->cluster_size;
391
+ "busy": false,
192
- uint32_t tail = (offset + bytes) % s->cluster_size;
392
+ "count": 458752,
193
+ uint32_t head = offset_into_subcluster(s, offset);
393
+ "granularity": 65536,
194
+ uint32_t tail = ROUND_UP(offset + bytes, s->subcluster_size) -
394
+ "persistent": false,
195
+ (offset + bytes);
395
+ "recording": false
196
396
+ }
197
trace_qcow2_pwrite_zeroes_start_req(qemu_coroutine_self(), offset, bytes);
397
+ ],
198
if (offset + bytes == bs->total_sectors * BDRV_SECTOR_SIZE) {
398
"drive0": [
199
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs,
399
{
200
unsigned int nr;
400
"busy": false,
201
QCow2SubclusterType type;
401
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
202
402
{"return": ""}
203
- assert(head + bytes <= s->cluster_size);
403
{
204
+ assert(head + bytes + tail <= s->subcluster_size);
404
"bitmaps": {
205
405
+ "backup-top": [
206
/* check whether remainder of cluster already reads as zero */
406
+ {
207
if (!(is_zero(bs, offset - head, head) &&
407
+ "busy": false,
208
- is_zero(bs, offset + bytes,
408
+ "count": 67108864,
209
- tail ? s->cluster_size - tail : 0))) {
409
+ "granularity": 65536,
210
+ is_zero(bs, offset + bytes, tail))) {
410
+ "persistent": false,
211
return -ENOTSUP;
411
+ "recording": false
212
}
412
+ },
213
413
+ {
214
qemu_co_mutex_lock(&s->lock);
414
+ "busy": false,
215
/* We can have new write after previous check */
415
+ "count": 458752,
216
- offset = QEMU_ALIGN_DOWN(offset, s->cluster_size);
416
+ "granularity": 65536,
217
- bytes = s->cluster_size;
417
+ "persistent": false,
218
- nr = s->cluster_size;
418
+ "recording": false
219
+ offset -= head;
419
+ }
220
+ bytes = s->subcluster_size;
420
+ ],
221
+ nr = s->subcluster_size;
421
"drive0": [
222
ret = qcow2_get_host_offset(bs, offset, &nr, &off, &type);
422
{
223
if (ret < 0 ||
423
"busy": false,
224
(type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN &&
424
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
225
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs,
425
{"return": ""}
226
426
{
227
trace_qcow2_pwrite_zeroes(qemu_coroutine_self(), offset, bytes);
427
"bitmaps": {
228
428
+ "backup-top": [
229
- /* Whatever is left can use real zero clusters */
429
+ {
230
- ret = qcow2_cluster_zeroize(bs, offset, bytes, flags);
430
+ "busy": false,
231
+ /* Whatever is left can use real zero subclusters */
431
+ "count": 67108864,
232
+ ret = qcow2_subcluster_zeroize(bs, offset, bytes, flags);
432
+ "granularity": 65536,
233
qemu_co_mutex_unlock(&s->lock);
433
+ "persistent": false,
234
434
+ "recording": false
235
return ret;
435
+ },
236
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset,
436
+ {
237
}
437
+ "busy": false,
238
438
+ "count": 458752,
239
if ((flags & BDRV_REQ_ZERO_WRITE) && offset > old_length) {
439
+ "granularity": 65536,
240
- uint64_t zero_start = QEMU_ALIGN_UP(old_length, s->cluster_size);
440
+ "persistent": false,
241
+ uint64_t zero_start = QEMU_ALIGN_UP(old_length, s->subcluster_size);
441
+ "recording": false
242
442
+ }
243
/*
443
+ ],
244
- * Use zero clusters as much as we can. qcow2_cluster_zeroize()
444
"drive0": [
245
- * requires a cluster-aligned start. The end may be unaligned if it is
445
{
246
- * at the end of the image (which it is here).
446
"busy": false,
247
+ * Use zero clusters as much as we can. qcow2_subcluster_zeroize()
447
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
248
+ * requires a subcluster-aligned start. The end may be unaligned if
448
{"return": ""}
249
+ * it is at the end of the image (which it is here).
449
{
250
*/
450
"bitmaps": {
251
if (offset > zero_start) {
451
+ "backup-top": [
252
- ret = qcow2_cluster_zeroize(bs, zero_start, offset - zero_start, 0);
452
+ {
253
+ ret = qcow2_subcluster_zeroize(bs, zero_start, offset - zero_start,
453
+ "busy": false,
254
+ 0);
454
+ "count": 67108864,
255
if (ret < 0) {
455
+ "granularity": 65536,
256
error_setg_errno(errp, -ret, "Failed to zero out new clusters");
456
+ "persistent": false,
257
goto fail;
457
+ "recording": false
458
+ },
459
+ {
460
+ "busy": false,
461
+ "count": 458752,
462
+ "granularity": 65536,
463
+ "persistent": false,
464
+ "recording": false
465
+ }
466
+ ],
467
"drive0": [
468
{
469
"busy": false,
470
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
471
{"return": ""}
472
{
473
"bitmaps": {
474
+ "backup-top": [
475
+ {
476
+ "busy": false,
477
+ "count": 67108864,
478
+ "granularity": 65536,
479
+ "persistent": false,
480
+ "recording": false
481
+ },
482
+ {
483
+ "busy": false,
484
+ "count": 458752,
485
+ "granularity": 65536,
486
+ "persistent": false,
487
+ "recording": false
488
+ }
489
+ ],
490
"drive0": [
491
{
492
"busy": false,
493
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
494
{"return": ""}
495
{
496
"bitmaps": {
497
+ "backup-top": [
498
+ {
499
+ "busy": false,
500
+ "count": 67108864,
501
+ "granularity": 65536,
502
+ "persistent": false,
503
+ "recording": false
504
+ },
505
+ {
506
+ "busy": false,
507
+ "count": 458752,
508
+ "granularity": 65536,
509
+ "persistent": false,
510
+ "recording": false
511
+ }
512
+ ],
513
"drive0": [
514
{
515
"busy": false,
516
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
517
{"return": ""}
518
{
519
"bitmaps": {
520
+ "backup-top": [
521
+ {
522
+ "busy": false,
523
+ "count": 67108864,
524
+ "granularity": 65536,
525
+ "persistent": false,
526
+ "recording": false
527
+ },
528
+ {
529
+ "busy": false,
530
+ "count": 458752,
531
+ "granularity": 65536,
532
+ "persistent": false,
533
+ "recording": false
534
+ }
535
+ ],
536
"drive0": [
537
{
538
"busy": false,
539
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
540
{"return": ""}
541
{
542
"bitmaps": {
543
+ "backup-top": [
544
+ {
545
+ "busy": false,
546
+ "count": 67108864,
547
+ "granularity": 65536,
548
+ "persistent": false,
549
+ "recording": false
550
+ },
551
+ {
552
+ "busy": false,
553
+ "count": 458752,
554
+ "granularity": 65536,
555
+ "persistent": false,
556
+ "recording": false
557
+ }
558
+ ],
559
"drive0": [
560
{
561
"busy": false,
562
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
563
{"return": ""}
564
{
565
"bitmaps": {
566
+ "backup-top": [
567
+ {
568
+ "busy": false,
569
+ "count": 67108864,
570
+ "granularity": 65536,
571
+ "persistent": false,
572
+ "recording": false
573
+ },
574
+ {
575
+ "busy": false,
576
+ "count": 458752,
577
+ "granularity": 65536,
578
+ "persistent": false,
579
+ "recording": false
580
+ }
581
+ ],
582
"drive0": [
583
{
584
"busy": false,
585
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
586
{"return": ""}
587
{
588
"bitmaps": {
589
+ "backup-top": [
590
+ {
591
+ "busy": false,
592
+ "count": 67108864,
593
+ "granularity": 65536,
594
+ "persistent": false,
595
+ "recording": false
596
+ },
597
+ {
598
+ "busy": false,
599
+ "count": 458752,
600
+ "granularity": 65536,
601
+ "persistent": false,
602
+ "recording": false
603
+ }
604
+ ],
605
"drive0": [
606
{
607
"busy": false,
608
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
609
{"return": ""}
610
{
611
"bitmaps": {
612
+ "backup-top": [
613
+ {
614
+ "busy": false,
615
+ "count": 67108864,
616
+ "granularity": 65536,
617
+ "persistent": false,
618
+ "recording": false
619
+ },
620
+ {
621
+ "busy": false,
622
+ "count": 458752,
623
+ "granularity": 65536,
624
+ "persistent": false,
625
+ "recording": false
626
+ }
627
+ ],
628
"drive0": [
629
{
630
"busy": false,
631
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
632
{"return": ""}
633
{
634
"bitmaps": {
635
+ "backup-top": [
636
+ {
637
+ "busy": false,
638
+ "count": 67108864,
639
+ "granularity": 65536,
640
+ "persistent": false,
641
+ "recording": false
642
+ },
643
+ {
644
+ "busy": false,
645
+ "count": 458752,
646
+ "granularity": 65536,
647
+ "persistent": false,
648
+ "recording": false
649
+ }
650
+ ],
651
"drive0": [
652
{
653
"busy": false,
654
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
655
{"return": ""}
656
{
657
"bitmaps": {
658
+ "backup-top": [
659
+ {
660
+ "busy": false,
661
+ "count": 67108864,
662
+ "granularity": 65536,
663
+ "persistent": false,
664
+ "recording": false
665
+ },
666
+ {
667
+ "busy": false,
668
+ "count": 458752,
669
+ "granularity": 65536,
670
+ "persistent": false,
671
+ "recording": false
672
+ }
673
+ ],
674
"drive0": [
675
{
676
"busy": false,
677
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
678
{"return": ""}
679
{
680
"bitmaps": {
681
+ "backup-top": [
682
+ {
683
+ "busy": false,
684
+ "count": 67108864,
685
+ "granularity": 65536,
686
+ "persistent": false,
687
+ "recording": false
688
+ },
689
+ {
690
+ "busy": false,
691
+ "count": 458752,
692
+ "granularity": 65536,
693
+ "persistent": false,
694
+ "recording": false
695
+ }
696
+ ],
697
"drive0": [
698
{
699
"busy": false,
258
--
700
--
259
2.26.2
701
2.34.1
260
261
diff view generated by jsdifflib
[PULL 25/34] qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2()
[PULL 20/23] iotests/image-fleecing: add test-case for fleecing format node
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
The L2 bitmap needs to be updated after each write to indicate what
3
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
4
new subclusters are now allocated. This needs to happen even if the
4
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
5
cluster was already allocated and the L2 entry was otherwise valid.
5
Message-Id: <20220303194349.2304213-14-vsementsov@virtuozzo.com>
6
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
7
---
8
tests/qemu-iotests/tests/image-fleecing | 64 +++++++++++++-----
9
tests/qemu-iotests/tests/image-fleecing.out | 74 ++++++++++++++++++++-
10
2 files changed, 119 insertions(+), 19 deletions(-)
6
11
7
In some cases however a write operation doesn't need change the L2
12
diff --git a/tests/qemu-iotests/tests/image-fleecing b/tests/qemu-iotests/tests/image-fleecing
8
bitmap (because all affected subclusters were already allocated). This
13
index XXXXXXX..XXXXXXX 100755
9
is detected in calculate_l2_meta(), and qcow2_alloc_cluster_link_l2()
14
--- a/tests/qemu-iotests/tests/image-fleecing
10
is never called in those cases.
15
+++ b/tests/qemu-iotests/tests/image-fleecing
11
16
@@ -XXX,XX +XXX,XX @@ remainder = [('0xd5', '0x108000', '32k'), # Right-end of partial-left [1]
12
Signed-off-by: Alberto Garcia <berto@igalia.com>
17
('0xdc', '32M', '32k'), # Left-end of partial-right [2]
13
Reviewed-by: Eric Blake <eblake@redhat.com>
18
('0xcd', '0x3ff0000', '64k')] # patterns[3]
14
Reviewed-by: Max Reitz <mreitz@redhat.com>
19
15
Message-Id: <0875620d49f44320334b6a91c73b3f301f975f38.1594396418.git.berto@igalia.com>
20
-def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
16
Signed-off-by: Max Reitz <mreitz@redhat.com>
21
+def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
17
---
22
+ fleece_img_path, nbd_sock_path, vm):
18
block/qcow2-cluster.c | 18 ++++++++++++++++++
23
log('--- Setting up images ---')
19
1 file changed, 18 insertions(+)
24
log('')
20
25
21
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
26
assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0
27
- assert qemu_img('create', '-f', 'qcow2', fleece_img_path, '64M') == 0
28
+ if use_snapshot_access_filter:
29
+ assert use_cbw
30
+ assert qemu_img('create', '-f', 'raw', fleece_img_path, '64M') == 0
31
+ else:
32
+ assert qemu_img('create', '-f', 'qcow2', fleece_img_path, '64M') == 0
33
34
for p in patterns:
35
qemu_io('-f', iotests.imgfmt,
36
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
37
log('')
38
39
40
- # create tmp_node backed by src_node
41
- log(vm.qmp('blockdev-add', {
42
- 'driver': 'qcow2',
43
- 'node-name': tmp_node,
44
- 'file': {
45
+ if use_snapshot_access_filter:
46
+ log(vm.qmp('blockdev-add', {
47
+ 'node-name': tmp_node,
48
'driver': 'file',
49
'filename': fleece_img_path,
50
- },
51
- 'backing': src_node,
52
- }))
53
+ }))
54
+ else:
55
+ # create tmp_node backed by src_node
56
+ log(vm.qmp('blockdev-add', {
57
+ 'driver': 'qcow2',
58
+ 'node-name': tmp_node,
59
+ 'file': {
60
+ 'driver': 'file',
61
+ 'filename': fleece_img_path,
62
+ },
63
+ 'backing': src_node,
64
+ }))
65
66
# Establish CBW from source to fleecing node
67
if use_cbw:
68
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
69
}))
70
71
log(vm.qmp('qom-set', path=qom_path, property='drive', value='fl-cbw'))
72
+
73
+ if use_snapshot_access_filter:
74
+ log(vm.qmp('blockdev-add', {
75
+ 'driver': 'snapshot-access',
76
+ 'node-name': 'fl-access',
77
+ 'file': 'fl-cbw',
78
+ }))
79
else:
80
log(vm.qmp('blockdev-backup',
81
job_id='fleecing',
82
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
83
target=tmp_node,
84
sync='none'))
85
86
+ export_node = 'fl-access' if use_snapshot_access_filter else tmp_node
87
+
88
log('')
89
log('--- Setting up NBD Export ---')
90
log('')
91
92
- nbd_uri = 'nbd+unix:///%s?socket=%s' % (tmp_node, nbd_sock_path)
93
+ nbd_uri = 'nbd+unix:///%s?socket=%s' % (export_node, nbd_sock_path)
94
log(vm.qmp('nbd-server-start',
95
{'addr': {'type': 'unix',
96
'data': {'path': nbd_sock_path}}}))
97
98
- log(vm.qmp('nbd-server-add', device=tmp_node))
99
+ log(vm.qmp('nbd-server-add', device=export_node))
100
101
log('')
102
log('--- Sanity Check ---')
103
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
104
log('--- Cleanup ---')
105
log('')
106
107
+ log(vm.qmp('nbd-server-stop'))
108
+
109
if use_cbw:
110
+ if use_snapshot_access_filter:
111
+ log(vm.qmp('blockdev-del', node_name='fl-access'))
112
log(vm.qmp('qom-set', path=qom_path, property='drive', value=src_node))
113
log(vm.qmp('blockdev-del', node_name='fl-cbw'))
114
else:
115
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
116
assert e is not None
117
log(e, filters=[iotests.filter_qmp_event])
118
119
- log(vm.qmp('nbd-server-stop'))
120
log(vm.qmp('blockdev-del', node_name=tmp_node))
121
vm.shutdown()
122
123
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
124
log('Done')
125
126
127
-def test(use_cbw):
128
+def test(use_cbw, use_snapshot_access_filter):
129
with iotests.FilePath('base.img') as base_img_path, \
130
iotests.FilePath('fleece.img') as fleece_img_path, \
131
iotests.FilePath('nbd.sock',
132
base_dir=iotests.sock_dir) as nbd_sock_path, \
133
iotests.VM() as vm:
134
- do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm)
135
+ do_test(use_cbw, use_snapshot_access_filter, base_img_path,
136
+ fleece_img_path, nbd_sock_path, vm)
137
138
139
log('=== Test backup(sync=none) based fleecing ===\n')
140
-test(False)
141
+test(False, False)
142
+
143
+log('=== Test cbw-filter based fleecing ===\n')
144
+test(True, False)
145
146
-log('=== Test filter based fleecing ===\n')
147
-test(True)
148
+log('=== Test fleecing-format based fleecing ===\n')
149
+test(True, True)
150
diff --git a/tests/qemu-iotests/tests/image-fleecing.out b/tests/qemu-iotests/tests/image-fleecing.out
22
index XXXXXXX..XXXXXXX 100644
151
index XXXXXXX..XXXXXXX 100644
23
--- a/block/qcow2-cluster.c
152
--- a/tests/qemu-iotests/tests/image-fleecing.out
24
+++ b/block/qcow2-cluster.c
153
+++ b/tests/qemu-iotests/tests/image-fleecing.out
25
@@ -XXX,XX +XXX,XX @@ int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m)
154
@@ -XXX,XX +XXX,XX @@ read -P0 0x3fe0000 64k
26
assert((offset & L2E_OFFSET_MASK) == offset);
155
27
156
--- Cleanup ---
28
set_l2_entry(s, l2_slice, l2_index + i, offset | QCOW_OFLAG_COPIED);
157
29
+
158
+{"return": {}}
30
+ /* Update bitmap with the subclusters that were just written */
159
{"return": {}}
31
+ if (has_subclusters(s)) {
160
{"data": {"device": "fleecing", "len": 67108864, "offset": 393216, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
32
+ uint64_t l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index + i);
161
{"return": {}}
33
+ unsigned written_from = m->cow_start.offset;
162
+
34
+ unsigned written_to = m->cow_end.offset + m->cow_end.nb_bytes ?:
163
+--- Confirming writes ---
35
+ m->nb_clusters << s->cluster_bits;
164
+
36
+ int first_sc, last_sc;
165
+read -P0xab 0 64k
37
+ /* Narrow written_from and written_to down to the current cluster */
166
+read -P0xad 0x00f8000 64k
38
+ written_from = MAX(written_from, i << s->cluster_bits);
167
+read -P0x1d 0x2008000 64k
39
+ written_to = MIN(written_to, (i + 1) << s->cluster_bits);
168
+read -P0xea 0x3fe0000 64k
40
+ assert(written_from < written_to);
169
+read -P0xd5 0x108000 32k
41
+ first_sc = offset_to_sc_index(s, written_from);
170
+read -P0xdc 32M 32k
42
+ last_sc = offset_to_sc_index(s, written_to - 1);
171
+read -P0xcd 0x3ff0000 64k
43
+ l2_bitmap |= QCOW_OFLAG_SUB_ALLOC_RANGE(first_sc, last_sc + 1);
172
+
44
+ l2_bitmap &= ~QCOW_OFLAG_SUB_ZERO_RANGE(first_sc, last_sc + 1);
173
+Done
45
+ set_l2_bitmap(s, l2_slice, l2_index + i, l2_bitmap);
174
+=== Test cbw-filter based fleecing ===
46
+ }
175
+
47
}
176
+--- Setting up images ---
48
177
+
178
+Done
179
+
180
+--- Launching VM ---
181
+
182
+Done
183
+
184
+--- Setting up Fleecing Graph ---
185
+
186
+{"return": {}}
187
+{"return": {}}
188
+{"return": {}}
189
+
190
+--- Setting up NBD Export ---
191
+
192
+{"return": {}}
193
+{"return": {}}
194
+
195
+--- Sanity Check ---
196
+
197
+read -P0x5d 0 64k
198
+read -P0xd5 1M 64k
199
+read -P0xdc 32M 64k
200
+read -P0xcd 0x3ff0000 64k
201
+read -P0 0x00f8000 32k
202
+read -P0 0x2010000 32k
203
+read -P0 0x3fe0000 64k
204
+
205
+--- Testing COW ---
206
+
207
+write -P0xab 0 64k
208
+{"return": ""}
209
+write -P0xad 0x00f8000 64k
210
+{"return": ""}
211
+write -P0x1d 0x2008000 64k
212
+{"return": ""}
213
+write -P0xea 0x3fe0000 64k
214
+{"return": ""}
215
+
216
+--- Verifying Data ---
217
+
218
+read -P0x5d 0 64k
219
+read -P0xd5 1M 64k
220
+read -P0xdc 32M 64k
221
+read -P0xcd 0x3ff0000 64k
222
+read -P0 0x00f8000 32k
223
+read -P0 0x2010000 32k
224
+read -P0 0x3fe0000 64k
225
+
226
+--- Cleanup ---
227
+
228
+{"return": {}}
229
+{"return": {}}
230
+{"return": {}}
231
{"return": {}}
232
233
--- Confirming writes ---
234
@@ -XXX,XX +XXX,XX @@ read -P0xdc 32M 32k
235
read -P0xcd 0x3ff0000 64k
236
237
Done
238
-=== Test filter based fleecing ===
239
+=== Test fleecing-format based fleecing ===
240
241
--- Setting up images ---
242
243
@@ -XXX,XX +XXX,XX @@ Done
244
{"return": {}}
245
{"return": {}}
246
{"return": {}}
247
+{"return": {}}
248
249
--- Setting up NBD Export ---
250
251
@@ -XXX,XX +XXX,XX @@ read -P0 0x3fe0000 64k
252
{"return": {}}
253
{"return": {}}
254
{"return": {}}
255
+{"return": {}}
256
257
--- Confirming writes ---
49
258
50
--
259
--
51
2.26.2
260
2.34.1
52
53
diff view generated by jsdifflib
[PULL 26/34] qcow2: Clear the L2 bitmap when allocating a compressed cluster
[PULL 21/23] iotests.py: add qemu_io_pipe_and_status()
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
Compressed clusters always have the bitmap part of the extended L2
3
Add helper that returns both status and output, to be used in the
4
entry set to 0.
4
following commit
5
5
6
Signed-off-by: Alberto Garcia <berto@igalia.com>
6
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
7
Reviewed-by: Max Reitz <mreitz@redhat.com>
7
Message-Id: <20220303194349.2304213-15-vsementsov@virtuozzo.com>
8
Message-Id: <04455b3de5dfeb9d1cfe1fc7b02d7060a6e09710.1594396418.git.berto@igalia.com>
8
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
9
Signed-off-by: Max Reitz <mreitz@redhat.com>
10
---
9
---
11
block/qcow2-cluster.c | 3 +++
10
tests/qemu-iotests/iotests.py | 3 +++
12
1 file changed, 3 insertions(+)
11
1 file changed, 3 insertions(+)
13
12
14
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
13
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
15
index XXXXXXX..XXXXXXX 100644
14
index XXXXXXX..XXXXXXX 100644
16
--- a/block/qcow2-cluster.c
15
--- a/tests/qemu-iotests/iotests.py
17
+++ b/block/qcow2-cluster.c
16
+++ b/tests/qemu-iotests/iotests.py
18
@@ -XXX,XX +XXX,XX @@ int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs,
17
@@ -XXX,XX +XXX,XX @@ def qemu_io(*args):
19
BLKDBG_EVENT(bs->file, BLKDBG_L2_UPDATE_COMPRESSED);
18
'''Run qemu-io and return the stdout data'''
20
qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice);
19
return qemu_tool_pipe_and_status('qemu-io', qemu_io_wrap_args(args))[0]
21
set_l2_entry(s, l2_slice, l2_index, cluster_offset);
20
22
+ if (has_subclusters(s)) {
21
+def qemu_io_pipe_and_status(*args):
23
+ set_l2_bitmap(s, l2_slice, l2_index, 0);
22
+ return qemu_tool_pipe_and_status('qemu-io', qemu_io_wrap_args(args))
24
+ }
23
+
25
qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice);
24
def qemu_io_log(*args):
26
25
result = qemu_io(*args)
27
*host_offset = cluster_offset & s->cluster_offset_mask;
26
log(result, filters=[filter_testfiles, filter_qemu_io])
28
--
27
--
29
2.26.2
28
2.34.1
30
31
diff view generated by jsdifflib
[PULL 27/34] qcow2: Add subcluster support to handle_alloc_space()
[PULL 22/23] iotests/image-fleecing: add test case with bitmap
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
The bdrv_co_pwrite_zeroes() call here fills complete clusters with
3
Note that reads zero areas (not dirty in the bitmap) fails, that's
4
zeroes, but it can happen that some subclusters are not part of the
4
correct.
5
write request or the copy-on-write. This patch makes sure that only
6
the affected subclusters are overwritten.
7
5
8
A potential improvement would be to also fill with zeroes the other
6
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
9
subclusters if we can guarantee that we are not overwriting existing
7
Message-Id: <20220303194349.2304213-16-vsementsov@virtuozzo.com>
10
data. However this would waste more disk space, so we should first
8
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
11
evaluate if it's really worth doing.
9
---
10
tests/qemu-iotests/tests/image-fleecing | 38 +++++++---
11
tests/qemu-iotests/tests/image-fleecing.out | 84 +++++++++++++++++++++
12
2 files changed, 113 insertions(+), 9 deletions(-)
12
13
13
Signed-off-by: Alberto Garcia <berto@igalia.com>
14
diff --git a/tests/qemu-iotests/tests/image-fleecing b/tests/qemu-iotests/tests/image-fleecing
14
Reviewed-by: Max Reitz <mreitz@redhat.com>
15
index XXXXXXX..XXXXXXX 100755
15
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
16
--- a/tests/qemu-iotests/tests/image-fleecing
16
Message-Id: <b3dc97e8e2240ddb5191a4f930e8fc9653f94621.1594396418.git.berto@igalia.com>
17
+++ b/tests/qemu-iotests/tests/image-fleecing
17
Signed-off-by: Max Reitz <mreitz@redhat.com>
18
@@ -XXX,XX +XXX,XX @@
18
---
19
# Creator/Owner: John Snow <jsnow@redhat.com>
19
block/qcow2.c | 9 +++++----
20
20
1 file changed, 5 insertions(+), 4 deletions(-)
21
import iotests
21
22
-from iotests import log, qemu_img, qemu_io, qemu_io_silent
22
diff --git a/block/qcow2.c b/block/qcow2.c
23
+from iotests import log, qemu_img, qemu_io, qemu_io_silent, \
24
+ qemu_io_pipe_and_status
25
26
iotests.script_initialize(
27
- supported_fmts=['qcow2', 'qcow', 'qed', 'vmdk', 'vhdx', 'raw'],
28
+ supported_fmts=['qcow2'],
29
supported_platforms=['linux'],
30
required_fmts=['copy-before-write'],
31
+ unsupported_imgopts=['compat']
32
)
33
34
patterns = [('0x5d', '0', '64k'),
35
@@ -XXX,XX +XXX,XX @@ remainder = [('0xd5', '0x108000', '32k'), # Right-end of partial-left [1]
36
('0xcd', '0x3ff0000', '64k')] # patterns[3]
37
38
def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
39
- fleece_img_path, nbd_sock_path, vm):
40
+ fleece_img_path, nbd_sock_path, vm,
41
+ bitmap=False):
42
log('--- Setting up images ---')
43
log('')
44
45
assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0
46
+ if bitmap:
47
+ assert qemu_img('bitmap', '--add', base_img_path, 'bitmap0') == 0
48
+
49
if use_snapshot_access_filter:
50
assert use_cbw
51
assert qemu_img('create', '-f', 'raw', fleece_img_path, '64M') == 0
52
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
53
54
# Establish CBW from source to fleecing node
55
if use_cbw:
56
- log(vm.qmp('blockdev-add', {
57
+ fl_cbw = {
58
'driver': 'copy-before-write',
59
'node-name': 'fl-cbw',
60
'file': src_node,
61
'target': tmp_node
62
- }))
63
+ }
64
+
65
+ if bitmap:
66
+ fl_cbw['bitmap'] = {'node': src_node, 'name': 'bitmap0'}
67
+
68
+ log(vm.qmp('blockdev-add', fl_cbw))
69
70
log(vm.qmp('qom-set', path=qom_path, property='drive', value='fl-cbw'))
71
72
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
73
for p in patterns + zeroes:
74
cmd = 'read -P%s %s %s' % p
75
log(cmd)
76
- assert qemu_io_silent('-r', '-f', 'raw', '-c', cmd, nbd_uri) == 0
77
+ out, ret = qemu_io_pipe_and_status('-r', '-f', 'raw', '-c', cmd,
78
+ nbd_uri)
79
+ if ret != 0:
80
+ print(out)
81
82
log('')
83
log('--- Testing COW ---')
84
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
85
for p in patterns + zeroes:
86
cmd = 'read -P%s %s %s' % p
87
log(cmd)
88
- assert qemu_io_silent('-r', '-f', 'raw', '-c', cmd, nbd_uri) == 0
89
+ out, ret = qemu_io_pipe_and_status('-r', '-f', 'raw', '-c', cmd,
90
+ nbd_uri)
91
+ if ret != 0:
92
+ print(out)
93
94
log('')
95
log('--- Cleanup ---')
96
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
97
log('Done')
98
99
100
-def test(use_cbw, use_snapshot_access_filter):
101
+def test(use_cbw, use_snapshot_access_filter, bitmap=False):
102
with iotests.FilePath('base.img') as base_img_path, \
103
iotests.FilePath('fleece.img') as fleece_img_path, \
104
iotests.FilePath('nbd.sock',
105
base_dir=iotests.sock_dir) as nbd_sock_path, \
106
iotests.VM() as vm:
107
do_test(use_cbw, use_snapshot_access_filter, base_img_path,
108
- fleece_img_path, nbd_sock_path, vm)
109
+ fleece_img_path, nbd_sock_path, vm, bitmap=bitmap)
110
111
112
log('=== Test backup(sync=none) based fleecing ===\n')
113
@@ -XXX,XX +XXX,XX @@ test(True, False)
114
115
log('=== Test fleecing-format based fleecing ===\n')
116
test(True, True)
117
+
118
+log('=== Test fleecing-format based fleecing with bitmap ===\n')
119
+test(True, True, bitmap=True)
120
diff --git a/tests/qemu-iotests/tests/image-fleecing.out b/tests/qemu-iotests/tests/image-fleecing.out
23
index XXXXXXX..XXXXXXX 100644
121
index XXXXXXX..XXXXXXX 100644
24
--- a/block/qcow2.c
122
--- a/tests/qemu-iotests/tests/image-fleecing.out
25
+++ b/block/qcow2.c
123
+++ b/tests/qemu-iotests/tests/image-fleecing.out
26
@@ -XXX,XX +XXX,XX @@ static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta)
124
@@ -XXX,XX +XXX,XX @@ read -P0 0x00f8000 32k
27
125
read -P0 0x2010000 32k
28
for (m = l2meta; m != NULL; m = m->next) {
126
read -P0 0x3fe0000 64k
29
int ret;
127
30
+ uint64_t start_offset = m->alloc_offset + m->cow_start.offset;
128
+--- Cleanup ---
31
+ unsigned nb_bytes = m->cow_end.offset + m->cow_end.nb_bytes -
129
+
32
+ m->cow_start.offset;
130
+{"return": {}}
33
131
+{"return": {}}
34
if (!m->cow_start.nb_bytes && !m->cow_end.nb_bytes) {
132
+{"return": {}}
35
continue;
133
+{"return": {}}
36
@@ -XXX,XX +XXX,XX @@ static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta)
134
+{"return": {}}
37
* efficiently zero out the whole clusters
135
+
38
*/
136
+--- Confirming writes ---
39
137
+
40
- ret = qcow2_pre_write_overlap_check(bs, 0, m->alloc_offset,
138
+read -P0xab 0 64k
41
- m->nb_clusters * s->cluster_size,
139
+read -P0xad 0x00f8000 64k
42
+ ret = qcow2_pre_write_overlap_check(bs, 0, start_offset, nb_bytes,
140
+read -P0x1d 0x2008000 64k
43
true);
141
+read -P0xea 0x3fe0000 64k
44
if (ret < 0) {
142
+read -P0xd5 0x108000 32k
45
return ret;
143
+read -P0xdc 32M 32k
46
}
144
+read -P0xcd 0x3ff0000 64k
47
145
+
48
BLKDBG_EVENT(bs->file, BLKDBG_CLUSTER_ALLOC_SPACE);
146
+Done
49
- ret = bdrv_co_pwrite_zeroes(s->data_file, m->alloc_offset,
147
+=== Test fleecing-format based fleecing with bitmap ===
50
- m->nb_clusters * s->cluster_size,
148
+
51
+ ret = bdrv_co_pwrite_zeroes(s->data_file, start_offset, nb_bytes,
149
+--- Setting up images ---
52
BDRV_REQ_NO_FALLBACK);
150
+
53
if (ret < 0) {
151
+Done
54
if (ret != -ENOTSUP && ret != -EAGAIN) {
152
+
153
+--- Launching VM ---
154
+
155
+Done
156
+
157
+--- Setting up Fleecing Graph ---
158
+
159
+{"return": {}}
160
+{"return": {}}
161
+{"return": {}}
162
+{"return": {}}
163
+
164
+--- Setting up NBD Export ---
165
+
166
+{"return": {}}
167
+{"return": {}}
168
+
169
+--- Sanity Check ---
170
+
171
+read -P0x5d 0 64k
172
+read -P0xd5 1M 64k
173
+read -P0xdc 32M 64k
174
+read -P0xcd 0x3ff0000 64k
175
+read -P0 0x00f8000 32k
176
+read failed: Invalid argument
177
+
178
+read -P0 0x2010000 32k
179
+read failed: Invalid argument
180
+
181
+read -P0 0x3fe0000 64k
182
+read failed: Invalid argument
183
+
184
+
185
+--- Testing COW ---
186
+
187
+write -P0xab 0 64k
188
+{"return": ""}
189
+write -P0xad 0x00f8000 64k
190
+{"return": ""}
191
+write -P0x1d 0x2008000 64k
192
+{"return": ""}
193
+write -P0xea 0x3fe0000 64k
194
+{"return": ""}
195
+
196
+--- Verifying Data ---
197
+
198
+read -P0x5d 0 64k
199
+read -P0xd5 1M 64k
200
+read -P0xdc 32M 64k
201
+read -P0xcd 0x3ff0000 64k
202
+read -P0 0x00f8000 32k
203
+read failed: Invalid argument
204
+
205
+read -P0 0x2010000 32k
206
+read failed: Invalid argument
207
+
208
+read -P0 0x3fe0000 64k
209
+read failed: Invalid argument
210
+
211
+
212
--- Cleanup ---
213
214
{"return": {}}
55
--
215
--
56
2.26.2
216
2.34.1
57
58
diff view generated by jsdifflib
[PULL 22/34] qcow2: Add subcluster support to zero_in_l2_slice()
[PULL 23/23] iotests/image-fleecing: test push backup with fleecing
1
From: Alberto Garcia <berto@igalia.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
The QCOW_OFLAG_ZERO bit that indicates that a cluster reads as
3
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
4
zeroes is only used in standard L2 entries. Extended L2 entries use
4
Message-Id: <20220303194349.2304213-17-vsementsov@virtuozzo.com>
5
individual 'all zeroes' bits for each subcluster.
5
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
6
---
7
tests/qemu-iotests/tests/image-fleecing | 125 +++++++++++++++-----
8
tests/qemu-iotests/tests/image-fleecing.out | 63 ++++++++++
9
2 files changed, 156 insertions(+), 32 deletions(-)
6
10
7
This must be taken into account when updating the L2 entry and also
11
diff --git a/tests/qemu-iotests/tests/image-fleecing b/tests/qemu-iotests/tests/image-fleecing
8
when deciding that an existing entry does not need to be updated.
12
index XXXXXXX..XXXXXXX 100755
9
13
--- a/tests/qemu-iotests/tests/image-fleecing
10
Signed-off-by: Alberto Garcia <berto@igalia.com>
14
+++ b/tests/qemu-iotests/tests/image-fleecing
11
Reviewed-by: Eric Blake <eblake@redhat.com>
15
@@ -XXX,XX +XXX,XX @@ remainder = [('0xd5', '0x108000', '32k'), # Right-end of partial-left [1]
12
Reviewed-by: Max Reitz <mreitz@redhat.com>
16
('0xdc', '32M', '32k'), # Left-end of partial-right [2]
13
Message-Id: <b61d61606d8c9b367bd641ab37351ddb9172799a.1594396418.git.berto@igalia.com>
17
('0xcd', '0x3ff0000', '64k')] # patterns[3]
14
Signed-off-by: Max Reitz <mreitz@redhat.com>
18
15
---
19
-def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
16
block/qcow2-cluster.c | 38 ++++++++++++++++++++------------------
20
- fleece_img_path, nbd_sock_path, vm,
17
1 file changed, 20 insertions(+), 18 deletions(-)
21
+def do_test(vm, use_cbw, use_snapshot_access_filter, base_img_path,
18
22
+ fleece_img_path, nbd_sock_path=None,
19
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
23
+ target_img_path=None,
24
bitmap=False):
25
+ push_backup = target_img_path is not None
26
+ assert (nbd_sock_path is not None) != push_backup
27
+ if push_backup:
28
+ assert use_cbw
29
+
30
log('--- Setting up images ---')
31
log('')
32
33
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
34
else:
35
assert qemu_img('create', '-f', 'qcow2', fleece_img_path, '64M') == 0
36
37
+ if push_backup:
38
+ assert qemu_img('create', '-f', 'qcow2', target_img_path, '64M') == 0
39
+
40
for p in patterns:
41
qemu_io('-f', iotests.imgfmt,
42
'-c', 'write -P%s %s %s' % p, base_img_path)
43
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
44
45
export_node = 'fl-access' if use_snapshot_access_filter else tmp_node
46
47
- log('')
48
- log('--- Setting up NBD Export ---')
49
- log('')
50
+ if push_backup:
51
+ log('')
52
+ log('--- Starting actual backup ---')
53
+ log('')
54
55
- nbd_uri = 'nbd+unix:///%s?socket=%s' % (export_node, nbd_sock_path)
56
- log(vm.qmp('nbd-server-start',
57
- {'addr': {'type': 'unix',
58
- 'data': {'path': nbd_sock_path}}}))
59
+ log(vm.qmp('blockdev-add', **{
60
+ 'driver': iotests.imgfmt,
61
+ 'node-name': 'target',
62
+ 'file': {
63
+ 'driver': 'file',
64
+ 'filename': target_img_path
65
+ }
66
+ }))
67
+ log(vm.qmp('blockdev-backup', device=export_node,
68
+ sync='full', target='target',
69
+ job_id='push-backup', speed=1))
70
+ else:
71
+ log('')
72
+ log('--- Setting up NBD Export ---')
73
+ log('')
74
75
- log(vm.qmp('nbd-server-add', device=export_node))
76
+ nbd_uri = 'nbd+unix:///%s?socket=%s' % (export_node, nbd_sock_path)
77
+ log(vm.qmp('nbd-server-start',
78
+ {'addr': { 'type': 'unix',
79
+ 'data': { 'path': nbd_sock_path } } }))
80
81
- log('')
82
- log('--- Sanity Check ---')
83
- log('')
84
+ log(vm.qmp('nbd-server-add', device=export_node))
85
86
- for p in patterns + zeroes:
87
- cmd = 'read -P%s %s %s' % p
88
- log(cmd)
89
- out, ret = qemu_io_pipe_and_status('-r', '-f', 'raw', '-c', cmd,
90
- nbd_uri)
91
- if ret != 0:
92
- print(out)
93
+ log('')
94
+ log('--- Sanity Check ---')
95
+ log('')
96
+
97
+ for p in patterns + zeroes:
98
+ cmd = 'read -P%s %s %s' % p
99
+ log(cmd)
100
+ out, ret = qemu_io_pipe_and_status('-r', '-f', 'raw', '-c', cmd,
101
+ nbd_uri)
102
+ if ret != 0:
103
+ print(out)
104
105
log('')
106
log('--- Testing COW ---')
107
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
108
log(cmd)
109
log(vm.hmp_qemu_io(qom_path, cmd, qdev=True))
110
111
+ if push_backup:
112
+ # Check that previous operations were done during backup, not after
113
+ # If backup is already finished, it's possible that it was finished
114
+ # even before hmp qemu_io write, and we didn't actually test
115
+ # copy-before-write operation. This should not happen, as we use
116
+ # speed=1. But worth checking.
117
+ result = vm.qmp('query-block-jobs')
118
+ assert len(result['return']) == 1
119
+
120
+ result = vm.qmp('block-job-set-speed', device='push-backup', speed=0)
121
+ assert result == {'return': {}}
122
+
123
+ log(vm.event_wait(name='BLOCK_JOB_COMPLETED',
124
+ match={'data': {'device': 'push-backup'}}),
125
+ filters=[iotests.filter_qmp_event])
126
+ log(vm.qmp('blockdev-del', node_name='target'))
127
+
128
log('')
129
log('--- Verifying Data ---')
130
log('')
131
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
132
for p in patterns + zeroes:
133
cmd = 'read -P%s %s %s' % p
134
log(cmd)
135
- out, ret = qemu_io_pipe_and_status('-r', '-f', 'raw', '-c', cmd,
136
- nbd_uri)
137
+ args = ['-r', '-c', cmd]
138
+ if push_backup:
139
+ args += [target_img_path]
140
+ else:
141
+ args += ['-f', 'raw', nbd_uri]
142
+ out, ret = qemu_io_pipe_and_status(*args)
143
if ret != 0:
144
print(out)
145
146
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
147
log('--- Cleanup ---')
148
log('')
149
150
- log(vm.qmp('nbd-server-stop'))
151
+ if not push_backup:
152
+ log(vm.qmp('nbd-server-stop'))
153
154
if use_cbw:
155
if use_snapshot_access_filter:
156
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
157
log('Done')
158
159
160
-def test(use_cbw, use_snapshot_access_filter, bitmap=False):
161
+def test(use_cbw, use_snapshot_access_filter,
162
+ nbd_sock_path=None, target_img_path=None, bitmap=False):
163
with iotests.FilePath('base.img') as base_img_path, \
164
iotests.FilePath('fleece.img') as fleece_img_path, \
165
- iotests.FilePath('nbd.sock',
166
- base_dir=iotests.sock_dir) as nbd_sock_path, \
167
iotests.VM() as vm:
168
- do_test(use_cbw, use_snapshot_access_filter, base_img_path,
169
- fleece_img_path, nbd_sock_path, vm, bitmap=bitmap)
170
+ do_test(vm, use_cbw, use_snapshot_access_filter, base_img_path,
171
+ fleece_img_path, nbd_sock_path, target_img_path,
172
+ bitmap=bitmap)
173
+
174
+def test_pull(use_cbw, use_snapshot_access_filter, bitmap=False):
175
+ with iotests.FilePath('nbd.sock',
176
+ base_dir=iotests.sock_dir) as nbd_sock_path:
177
+ test(use_cbw, use_snapshot_access_filter, nbd_sock_path, None,
178
+ bitmap=bitmap)
179
+
180
+def test_push():
181
+ with iotests.FilePath('target.img') as target_img_path:
182
+ test(True, True, None, target_img_path)
183
184
185
log('=== Test backup(sync=none) based fleecing ===\n')
186
-test(False, False)
187
+test_pull(False, False)
188
189
log('=== Test cbw-filter based fleecing ===\n')
190
-test(True, False)
191
+test_pull(True, False)
192
193
log('=== Test fleecing-format based fleecing ===\n')
194
-test(True, True)
195
+test_pull(True, True)
196
197
log('=== Test fleecing-format based fleecing with bitmap ===\n')
198
-test(True, True, bitmap=True)
199
+test_pull(True, True, bitmap=True)
200
+
201
+log('=== Test push backup with fleecing ===\n')
202
+test_push()
203
diff --git a/tests/qemu-iotests/tests/image-fleecing.out b/tests/qemu-iotests/tests/image-fleecing.out
20
index XXXXXXX..XXXXXXX 100644
204
index XXXXXXX..XXXXXXX 100644
21
--- a/block/qcow2-cluster.c
205
--- a/tests/qemu-iotests/tests/image-fleecing.out
22
+++ b/block/qcow2-cluster.c
206
+++ b/tests/qemu-iotests/tests/image-fleecing.out
23
@@ -XXX,XX +XXX,XX @@ static int zero_in_l2_slice(BlockDriverState *bs, uint64_t offset,
207
@@ -XXX,XX +XXX,XX @@ read -P0xdc 32M 32k
24
int l2_index;
208
read -P0xcd 0x3ff0000 64k
25
int ret;
209
26
int i;
210
Done
27
- bool unmap = !!(flags & BDRV_REQ_MAY_UNMAP);
211
+=== Test push backup with fleecing ===
28
212
+
29
ret = get_cluster_table(bs, offset, &l2_slice, &l2_index);
213
+--- Setting up images ---
30
if (ret < 0) {
214
+
31
@@ -XXX,XX +XXX,XX @@ static int zero_in_l2_slice(BlockDriverState *bs, uint64_t offset,
215
+Done
32
assert(nb_clusters <= INT_MAX);
216
+
33
217
+--- Launching VM ---
34
for (i = 0; i < nb_clusters; i++) {
218
+
35
- uint64_t old_offset;
219
+Done
36
- QCow2ClusterType cluster_type;
220
+
37
-
221
+--- Setting up Fleecing Graph ---
38
- old_offset = get_l2_entry(s, l2_slice, l2_index + i);
222
+
39
+ uint64_t old_l2_entry = get_l2_entry(s, l2_slice, l2_index + i);
223
+{"return": {}}
40
+ uint64_t old_l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index + i);
224
+{"return": {}}
41
+ QCow2ClusterType type = qcow2_get_cluster_type(bs, old_l2_entry);
225
+{"return": {}}
42
+ bool unmap = (type == QCOW2_CLUSTER_COMPRESSED) ||
226
+{"return": {}}
43
+ ((flags & BDRV_REQ_MAY_UNMAP) && qcow2_cluster_is_allocated(type));
227
+
44
+ uint64_t new_l2_entry = unmap ? 0 : old_l2_entry;
228
+--- Starting actual backup ---
45
+ uint64_t new_l2_bitmap = old_l2_bitmap;
229
+
46
+
230
+{"return": {}}
47
+ if (has_subclusters(s)) {
231
+{"return": {}}
48
+ new_l2_bitmap = QCOW_L2_BITMAP_ALL_ZEROES;
232
+
49
+ } else {
233
+--- Testing COW ---
50
+ new_l2_entry |= QCOW_OFLAG_ZERO;
234
+
51
+ }
235
+write -P0xab 0 64k
52
236
+{"return": ""}
53
- /*
237
+write -P0xad 0x00f8000 64k
54
- * Minimize L2 changes if the cluster already reads back as
238
+{"return": ""}
55
- * zeroes with correct allocation.
239
+write -P0x1d 0x2008000 64k
56
- */
240
+{"return": ""}
57
- cluster_type = qcow2_get_cluster_type(bs, old_offset);
241
+write -P0xea 0x3fe0000 64k
58
- if (cluster_type == QCOW2_CLUSTER_ZERO_PLAIN ||
242
+{"return": ""}
59
- (cluster_type == QCOW2_CLUSTER_ZERO_ALLOC && !unmap)) {
243
+{"data": {"device": "push-backup", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
60
+ if (old_l2_entry == new_l2_entry && old_l2_bitmap == new_l2_bitmap) {
244
+{"return": {}}
61
continue;
245
+
62
}
246
+--- Verifying Data ---
63
247
+
64
qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice);
248
+read -P0x5d 0 64k
65
- if (cluster_type == QCOW2_CLUSTER_COMPRESSED || unmap) {
249
+read -P0xd5 1M 64k
66
- set_l2_entry(s, l2_slice, l2_index + i, QCOW_OFLAG_ZERO);
250
+read -P0xdc 32M 64k
67
- qcow2_free_any_clusters(bs, old_offset, 1, QCOW2_DISCARD_REQUEST);
251
+read -P0xcd 0x3ff0000 64k
68
- } else {
252
+read -P0 0x00f8000 32k
69
- uint64_t entry = get_l2_entry(s, l2_slice, l2_index + i);
253
+read -P0 0x2010000 32k
70
- set_l2_entry(s, l2_slice, l2_index + i, entry | QCOW_OFLAG_ZERO);
254
+read -P0 0x3fe0000 64k
71
+ if (unmap) {
255
+
72
+ qcow2_free_any_clusters(bs, old_l2_entry, 1, QCOW2_DISCARD_REQUEST);
256
+--- Cleanup ---
73
+ }
257
+
74
+ set_l2_entry(s, l2_slice, l2_index + i, new_l2_entry);
258
+{"return": {}}
75
+ if (has_subclusters(s)) {
259
+{"return": {}}
76
+ set_l2_bitmap(s, l2_slice, l2_index + i, new_l2_bitmap);
260
+{"return": {}}
77
}
261
+{"return": {}}
78
}
262
+
79
263
+--- Confirming writes ---
264
+
265
+read -P0xab 0 64k
266
+read -P0xad 0x00f8000 64k
267
+read -P0x1d 0x2008000 64k
268
+read -P0xea 0x3fe0000 64k
269
+read -P0xd5 0x108000 32k
270
+read -P0xdc 32M 32k
271
+read -P0xcd 0x3ff0000 64k
272
+
273
+Done
80
--
274
--
81
2.26.2
275
2.34.1
82
83
diff view generated by jsdifflib
[PULL 24/34] qcow2: Add subcluster support to check_refcounts_l2()
Deleted patch
1
From: Alberto Garcia <berto@igalia.com>
2
1
3
The offset field of an uncompressed cluster's L2 entry must be aligned
4
to the cluster size, otherwise it is invalid. If the cluster has no
5
data then it means that the offset points to a preallocation, so we
6
can clear the offset field without affecting the guest-visible data.
7
This is what 'qemu-img check' does when run in repair mode.
8
9
On traditional qcow2 images this can only happen when QCOW_OFLAG_ZERO
10
is set, and repairing such entries turns the clusters from ZERO_ALLOC
11
into ZERO_PLAIN.
12
13
Extended L2 entries have no ZERO_ALLOC clusters and no QCOW_OFLAG_ZERO
14
but the idea is the same: if none of the subclusters are allocated
15
then we can clear the offset field and leave the bitmap untouched.
16
17
Signed-off-by: Alberto Garcia <berto@igalia.com>
18
Reviewed-by: Max Reitz <mreitz@redhat.com>
19
Message-Id: <9f4ed1d0a34b0a545b032c31ecd8c14734065342.1594396418.git.berto@igalia.com>
20
Signed-off-by: Max Reitz <mreitz@redhat.com>
21
---
22
block/qcow2-refcount.c | 16 +++++++++++-----
23
tests/qemu-iotests/060.out | 2 +-
24
2 files changed, 12 insertions(+), 6 deletions(-)
25
26
diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
27
index XXXXXXX..XXXXXXX 100644
28
--- a/block/qcow2-refcount.c
29
+++ b/block/qcow2-refcount.c
30
@@ -XXX,XX +XXX,XX @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res,
31
32
/* Correct offsets are cluster aligned */
33
if (offset_into_cluster(s, offset)) {
34
+ bool contains_data;
35
res->corruptions++;
36
37
- if (qcow2_get_cluster_type(bs, l2_entry) ==
38
- QCOW2_CLUSTER_ZERO_ALLOC)
39
- {
40
- fprintf(stderr, "%s offset=%" PRIx64 ": Preallocated zero "
41
+ if (has_subclusters(s)) {
42
+ uint64_t l2_bitmap = get_l2_bitmap(s, l2_table, i);
43
+ contains_data = (l2_bitmap & QCOW_L2_BITMAP_ALL_ALLOC);
44
+ } else {
45
+ contains_data = !(l2_entry & QCOW_OFLAG_ZERO);
46
+ }
47
+
48
+ if (!contains_data) {
49
+ fprintf(stderr, "%s offset=%" PRIx64 ": Preallocated "
50
"cluster is not properly aligned; L2 entry "
51
"corrupted.\n",
52
fix & BDRV_FIX_ERRORS ? "Repairing" : "ERROR",
53
@@ -XXX,XX +XXX,XX @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res,
54
int ign = active ? QCOW2_OL_ACTIVE_L2 :
55
QCOW2_OL_INACTIVE_L2;
56
57
- l2_entry = QCOW_OFLAG_ZERO;
58
+ l2_entry = has_subclusters(s) ? 0 : QCOW_OFLAG_ZERO;
59
set_l2_entry(s, l2_table, i, l2_entry);
60
ret = qcow2_pre_write_overlap_check(bs, ign,
61
l2e_offset, l2_entry_size(s), false);
62
diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out
63
index XXXXXXX..XXXXXXX 100644
64
--- a/tests/qemu-iotests/060.out
65
+++ b/tests/qemu-iotests/060.out
66
@@ -XXX,XX +XXX,XX @@ discard 65536/65536 bytes at offset 0
67
qcow2: Marking image as corrupt: Preallocated zero cluster offset 0x2a00 unaligned (guest offset: 0); further corruption events will be suppressed
68
write failed: Input/output error
69
--- Repairing ---
70
-Repairing offset=2a00: Preallocated zero cluster is not properly aligned; L2 entry corrupted.
71
+Repairing offset=2a00: Preallocated cluster is not properly aligned; L2 entry corrupted.
72
The following inconsistencies were found and repaired:
73
74
0 leaked clusters
75
--
76
2.26.2
77
78
diff view generated by jsdifflib

Patchew 2.1-devel-b67e4c2

© 2016 - 2025 Red Hat, Inc.