1
The following changes since commit b4fbe1f65a4769c09e6bf2d79fc84360f840f40e:
1
The following changes since commit 474f3938d79ab36b9231c9ad3b5a9314c2aeacde:
2
2
3
Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20190129' into staging (2019-01-29 12:00:19 +0000)
3
Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-jun-21-2019' into staging (2019-06-21 15:40:50 +0100)
4
4
5
are available in the Git repository at:
5
are available in the Git repository at:
6
6
7
https://git.xanclic.moe/XanClic/qemu.git tags/pull-block-2019-01-31
7
https://github.com/XanClic/qemu.git tags/pull-block-2019-06-24
8
8
9
for you to fetch changes up to 908b30164bbffad7430d551b2a03a8fbcaa631ef:
9
for you to fetch changes up to ab5d4a30f7f3803ca5106b370969c1b7b54136f8:
10
10
11
iotests: Allow 147 to be run concurrently (2019-01-31 00:44:55 +0100)
11
iotests: Fix 205 for concurrent runs (2019-06-24 16:01:40 +0200)
12
12
13
----------------------------------------------------------------
13
----------------------------------------------------------------
14
Block patches:
14
Block patches:
15
- New debugging QMP command to explore block graphs
15
- The SSH block driver now uses libssh instead of libssh2
16
- Converted DPRINTF()s to trace events
16
- The VMDK block driver gets read-only support for the seSparse
17
- Fixed qemu-io's use of getopt() for systems with optreset
17
subformat
18
- Minor NVMe emulation fixes
18
- Various fixes
19
- An iotest fix
19
20
---
21
22
v2:
23
- Squashed Pino's fix for pre-0.8 libssh into the libssh patch
20
24
21
----------------------------------------------------------------
25
----------------------------------------------------------------
22
Laurent Vivier (4):
26
Anton Nefedov (1):
23
block/ssh: Convert from DPRINTF() macro to trace events
27
iotest 134: test cluster-misaligned encrypted write
24
block/curl: Convert from DPRINTF() macro to trace events
25
block/file-posix: Convert from DPRINTF() macro to trace events
26
block/sheepdog: Convert from DPRINTF() macro to trace events
27
28
28
Li Qiang (3):
29
Klaus Birkelund Jensen (1):
29
nvme: use TYPE_NVME instead of constant string
30
nvme: do not advertise support for unsupported arbitration mechanism
30
nvme: ensure the num_queues is not zero
31
nvme: use pci_dev directly in nvme_realize
32
31
33
Max Reitz (3):
32
Max Reitz (1):
34
iotests.py: Add qemu_nbd_pipe()
33
iotests: Fix 205 for concurrent runs
35
iotests: Bind qemu-nbd to localhost in 147
36
iotests: Allow 147 to be run concurrently
37
34
38
Richard W.M. Jones (1):
35
Pino Toscano (1):
39
qemu-io: Add generic function for reinitializing optind.
36
ssh: switch from libssh2 to libssh
40
37
41
Vladimir Sementsov-Ogievskiy (2):
38
Sam Eiderman (3):
42
qapi: add x-debug-query-block-graph
39
vmdk: Fix comment regarding max l1_size coverage
43
scripts: add render_block_graph function for QEMUMachine
40
vmdk: Reduce the max bound for L1 table size
41
vmdk: Add read-only support for seSparse snapshots
44
42
45
configure | 14 ++++
43
Vladimir Sementsov-Ogievskiy (1):
46
qapi/block-core.json | 108 ++++++++++++++++++++++++
44
blockdev: enable non-root nodes for transaction drive-backup source
47
include/block/block.h | 1 +
45
48
include/qemu/osdep.h | 16 ++++
46
configure | 65 +-
49
include/sysemu/block-backend.h | 2 +
47
block/Makefile.objs | 6 +-
50
block.c | 148 +++++++++++++++++++++++++++++++++
48
block/ssh.c | 652 ++++++++++--------
51
block/block-backend.c | 5 ++
49
block/vmdk.c | 372 +++++++++-
52
block/curl.c | 29 ++-----
50
blockdev.c | 2 +-
53
block/file-posix.c | 25 ++----
51
hw/block/nvme.c | 1 -
54
block/sheepdog.c | 47 ++++-------
52
.travis.yml | 4 +-
55
block/ssh.c | 46 ++++------
53
block/trace-events | 14 +-
56
blockdev.c | 5 ++
54
docs/qemu-block-drivers.texi | 2 +-
57
hw/block/nvme.c | 15 ++--
55
.../dockerfiles/debian-win32-cross.docker | 1 -
58
qemu-img.c | 2 +-
56
.../dockerfiles/debian-win64-cross.docker | 1 -
59
qemu-io-cmds.c | 2 +-
57
tests/docker/dockerfiles/fedora.docker | 4 +-
60
block/trace-events | 47 +++++++++++
58
tests/docker/dockerfiles/ubuntu.docker | 2 +-
61
scripts/render_block_graph.py | 120 ++++++++++++++++++++++++++
59
tests/docker/dockerfiles/ubuntu1804.docker | 2 +-
62
tests/qemu-iotests/147 | 98 +++++++++++++++-------
60
tests/qemu-iotests/059.out | 2 +-
63
tests/qemu-iotests/iotests.py | 14 ++++
61
tests/qemu-iotests/134 | 9 +
64
19 files changed, 608 insertions(+), 136 deletions(-)
62
tests/qemu-iotests/134.out | 10 +
65
create mode 100755 scripts/render_block_graph.py
63
tests/qemu-iotests/205 | 2 +-
64
tests/qemu-iotests/207 | 54 +-
65
tests/qemu-iotests/207.out | 2 +-
66
20 files changed, 823 insertions(+), 384 deletions(-)
66
67
67
--
68
--
68
2.20.1
69
2.21.0
69
70
70
71
diff view generated by jsdifflib
1
From: Li Qiang <liq3ea@163.com>
1
From: Klaus Birkelund Jensen <klaus@birkelund.eu>
2
2
3
There is no need to make another reference.
3
The device mistakenly reports that the Weighted Round Robin with Urgent
4
Priority Class arbitration mechanism is supported.
4
5
5
Signed-off-by: Li Qiang <liq3ea@163.com>
6
It is not.
6
Reviewed-by: Max Reitz <mreitz@redhat.com>
7
7
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
8
Signed-off-by: Klaus Birkelund Jensen <klaus.jensen@cnexlabs.com>
8
Message-id: 20190120055558.32984-4-liq3ea@163.com
9
Message-id: 20190606092530.14206-1-klaus@birkelund.eu
10
Acked-by: Maxim Levitsky <mlevitsk@redhat.com>
9
Signed-off-by: Max Reitz <mreitz@redhat.com>
11
Signed-off-by: Max Reitz <mreitz@redhat.com>
10
---
12
---
11
hw/block/nvme.c | 8 ++++----
13
hw/block/nvme.c | 1 -
12
1 file changed, 4 insertions(+), 4 deletions(-)
14
1 file changed, 1 deletion(-)
13
15
14
diff --git a/hw/block/nvme.c b/hw/block/nvme.c
16
diff --git a/hw/block/nvme.c b/hw/block/nvme.c
15
index XXXXXXX..XXXXXXX 100644
17
index XXXXXXX..XXXXXXX 100644
16
--- a/hw/block/nvme.c
18
--- a/hw/block/nvme.c
17
+++ b/hw/block/nvme.c
19
+++ b/hw/block/nvme.c
18
@@ -XXX,XX +XXX,XX @@ static void nvme_realize(PCIDevice *pci_dev, Error **errp)
20
@@ -XXX,XX +XXX,XX @@ static void nvme_realize(PCIDevice *pci_dev, Error **errp)
19
pci_conf[PCI_INTERRUPT_PIN] = 1;
21
n->bar.cap = 0;
20
pci_config_set_prog_interface(pci_dev->config, 0x2);
22
NVME_CAP_SET_MQES(n->bar.cap, 0x7ff);
21
pci_config_set_class(pci_dev->config, PCI_CLASS_STORAGE_EXPRESS);
23
NVME_CAP_SET_CQR(n->bar.cap, 1);
22
- pcie_endpoint_cap_init(&n->parent_obj, 0x80);
24
- NVME_CAP_SET_AMS(n->bar.cap, 1);
23
+ pcie_endpoint_cap_init(pci_dev, 0x80);
25
NVME_CAP_SET_TO(n->bar.cap, 0xf);
24
26
NVME_CAP_SET_CSS(n->bar.cap, 1);
25
n->num_namespaces = 1;
27
NVME_CAP_SET_MPSMAX(n->bar.cap, 4);
26
n->reg_size = pow2ceil(0x1004 + 2 * (n->num_queues + 1) * 4);
27
@@ -XXX,XX +XXX,XX @@ static void nvme_realize(PCIDevice *pci_dev, Error **errp)
28
29
memory_region_init_io(&n->iomem, OBJECT(n), &nvme_mmio_ops, n,
30
"nvme", n->reg_size);
31
- pci_register_bar(&n->parent_obj, 0,
32
+ pci_register_bar(pci_dev, 0,
33
PCI_BASE_ADDRESS_SPACE_MEMORY | PCI_BASE_ADDRESS_MEM_TYPE_64,
34
&n->iomem);
35
- msix_init_exclusive_bar(&n->parent_obj, n->num_queues, 4, NULL);
36
+ msix_init_exclusive_bar(pci_dev, n->num_queues, 4, NULL);
37
38
id->vid = cpu_to_le16(pci_get_word(pci_conf + PCI_VENDOR_ID));
39
id->ssvid = cpu_to_le16(pci_get_word(pci_conf + PCI_SUBSYSTEM_VENDOR_ID));
40
@@ -XXX,XX +XXX,XX @@ static void nvme_realize(PCIDevice *pci_dev, Error **errp)
41
n->cmbuf = g_malloc0(NVME_CMBSZ_GETSIZE(n->bar.cmbsz));
42
memory_region_init_io(&n->ctrl_mem, OBJECT(n), &nvme_cmb_ops, n,
43
"nvme-cmb", NVME_CMBSZ_GETSIZE(n->bar.cmbsz));
44
- pci_register_bar(&n->parent_obj, NVME_CMBLOC_BIR(n->bar.cmbloc),
45
+ pci_register_bar(pci_dev, NVME_CMBLOC_BIR(n->bar.cmbloc),
46
PCI_BASE_ADDRESS_SPACE_MEMORY | PCI_BASE_ADDRESS_MEM_TYPE_64 |
47
PCI_BASE_ADDRESS_MEM_PREFETCH, &n->ctrl_mem);
48
49
--
28
--
50
2.20.1
29
2.21.0
51
30
52
31
diff view generated by jsdifflib
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2
2
3
Render block nodes graph with help of graphviz. This new function is
3
We forget to enable it for transaction .prepare, while it is already
4
for debugging, so there is no sense to put it into qemu.py as a method
4
enabled in do_drive_backup since commit a2d665c1bc362
5
of QEMUMachine. Let's instead put it separately.
5
"blockdev: loosen restrictions on drive-backup source node"
6
6
7
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
7
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
8
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
8
Message-id: 20190618140804.59214-1-vsementsov@virtuozzo.com
9
Reviewed-by: Max Reitz <mreitz@redhat.com>
9
Reviewed-by: John Snow <jsnow@redhat.com>
10
Message-id: 20181221170909.25584-3-vsementsov@virtuozzo.com
11
Signed-off-by: Max Reitz <mreitz@redhat.com>
10
Signed-off-by: Max Reitz <mreitz@redhat.com>
12
---
11
---
13
scripts/render_block_graph.py | 120 ++++++++++++++++++++++++++++++++++
12
blockdev.c | 2 +-
14
1 file changed, 120 insertions(+)
13
1 file changed, 1 insertion(+), 1 deletion(-)
15
create mode 100755 scripts/render_block_graph.py
16
14
17
diff --git a/scripts/render_block_graph.py b/scripts/render_block_graph.py
15
diff --git a/blockdev.c b/blockdev.c
18
new file mode 100755
16
index XXXXXXX..XXXXXXX 100644
19
index XXXXXXX..XXXXXXX
17
--- a/blockdev.c
20
--- /dev/null
18
+++ b/blockdev.c
21
+++ b/scripts/render_block_graph.py
19
@@ -XXX,XX +XXX,XX @@ static void drive_backup_prepare(BlkActionState *common, Error **errp)
22
@@ -XXX,XX +XXX,XX @@
20
assert(common->action->type == TRANSACTION_ACTION_KIND_DRIVE_BACKUP);
23
+#!/usr/bin/env python
21
backup = common->action->u.drive_backup.data;
24
+#
22
25
+# Render Qemu Block Graph
23
- bs = qmp_get_root_bs(backup->device, errp);
26
+#
24
+ bs = bdrv_lookup_bs(backup->device, backup->device, errp);
27
+# Copyright (c) 2018 Virtuozzo International GmbH. All rights reserved.
25
if (!bs) {
28
+#
26
return;
29
+# This program is free software; you can redistribute it and/or modify
27
}
30
+# it under the terms of the GNU General Public License as published by
31
+# the Free Software Foundation; either version 2 of the License, or
32
+# (at your option) any later version.
33
+#
34
+# This program is distributed in the hope that it will be useful,
35
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
36
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
37
+# GNU General Public License for more details.
38
+#
39
+# You should have received a copy of the GNU General Public License
40
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
41
+#
42
+
43
+import os
44
+import sys
45
+import subprocess
46
+import json
47
+from graphviz import Digraph
48
+from qemu import MonitorResponseError
49
+
50
+
51
+def perm(arr):
52
+ s = 'w' if 'write' in arr else '_'
53
+ s += 'r' if 'consistent-read' in arr else '_'
54
+ s += 'u' if 'write-unchanged' in arr else '_'
55
+ s += 'g' if 'graph-mod' in arr else '_'
56
+ s += 's' if 'resize' in arr else '_'
57
+ return s
58
+
59
+
60
+def render_block_graph(qmp, filename, format='png'):
61
+ '''
62
+ Render graph in text (dot) representation into "@filename" and
63
+ representation in @format into "@filename.@format"
64
+ '''
65
+
66
+ bds_nodes = qmp.command('query-named-block-nodes')
67
+ bds_nodes = {n['node-name']: n for n in bds_nodes}
68
+
69
+ job_nodes = qmp.command('query-block-jobs')
70
+ job_nodes = {n['device']: n for n in job_nodes}
71
+
72
+ block_graph = qmp.command('x-debug-query-block-graph')
73
+
74
+ graph = Digraph(comment='Block Nodes Graph')
75
+ graph.format = format
76
+ graph.node('permission symbols:\l'
77
+ ' w - Write\l'
78
+ ' r - consistent-Read\l'
79
+ ' u - write - Unchanged\l'
80
+ ' g - Graph-mod\l'
81
+ ' s - reSize\l'
82
+ 'edge label scheme:\l'
83
+ ' <child type>\l'
84
+ ' <perm>\l'
85
+ ' <shared_perm>\l', shape='none')
86
+
87
+ for n in block_graph['nodes']:
88
+ if n['type'] == 'block-driver':
89
+ info = bds_nodes[n['name']]
90
+ label = n['name'] + ' [' + info['drv'] + ']'
91
+ if info['drv'] == 'file':
92
+ label += '\n' + os.path.basename(info['file'])
93
+ shape = 'ellipse'
94
+ elif n['type'] == 'block-job':
95
+ info = job_nodes[n['name']]
96
+ label = info['type'] + ' job (' + n['name'] + ')'
97
+ shape = 'box'
98
+ else:
99
+ assert n['type'] == 'block-backend'
100
+ label = n['name'] if n['name'] else 'unnamed blk'
101
+ shape = 'box'
102
+
103
+ graph.node(str(n['id']), label, shape=shape)
104
+
105
+ for e in block_graph['edges']:
106
+ label = '%s\l%s\l%s\l' % (e['name'], perm(e['perm']),
107
+ perm(e['shared-perm']))
108
+ graph.edge(str(e['parent']), str(e['child']), label=label)
109
+
110
+ graph.render(filename)
111
+
112
+
113
+class LibvirtGuest():
114
+ def __init__(self, name):
115
+ self.name = name
116
+
117
+ def command(self, cmd):
118
+ # only supports qmp commands without parameters
119
+ m = {'execute': cmd}
120
+ ar = ['virsh', 'qemu-monitor-command', self.name, json.dumps(m)]
121
+
122
+ reply = json.loads(subprocess.check_output(ar))
123
+
124
+ if 'error' in reply:
125
+ raise MonitorResponseError(reply)
126
+
127
+ return reply['return']
128
+
129
+
130
+if __name__ == '__main__':
131
+ obj = sys.argv[1]
132
+ out = sys.argv[2]
133
+
134
+ if os.path.exists(obj):
135
+ # assume unix socket
136
+ qmp = QEMUMonitorProtocol(obj)
137
+ qmp.connect()
138
+ else:
139
+ # assume libvirt guest name
140
+ qmp = LibvirtGuest(obj)
141
+
142
+ render_block_graph(qmp, out)
143
--
28
--
144
2.20.1
29
2.21.0
145
30
146
31
diff view generated by jsdifflib
1
To do this, we need to allow creating the NBD server on various ports
1
From: Anton Nefedov <anton.nefedov@virtuozzo.com>
2
instead of a single one (which may not even work if you run just one
3
instance, because something entirely else might be using that port).
4
2
5
So we just pick a random port in [32768, 32768 + 1024) and try to create
3
COW (even empty/zero) areas require encryption too
6
a server there. If that fails, we just retry until something sticks.
7
4
8
For the IPv6 test, we need a different range, though (just above that
5
Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
9
one). This is because "localhost" resolves to both 127.0.0.1 and ::1.
6
Reviewed-by: Eric Blake <eblake@redhat.com>
10
This means that if you bind to it, it will bind to both, if possible, or
7
Reviewed-by: Max Reitz <mreitz@redhat.com>
11
just one if the other is already in use. Therefore, if the IPv6 test
8
Reviewed-by: Alberto Garcia <berto@igalia.com>
12
has already taken [::1]:some_port and we then try to take
9
Message-id: 20190516143028.81155-1-anton.nefedov@virtuozzo.com
13
localhost:some_port, that will work -- only the second server will be
14
bound to 127.0.0.1:some_port alone and not [::1]:some_port in addition.
15
So we have two different servers on the same port, one for IPv4 and one
16
for IPv6.
17
18
But when we then try to connect to the server through
19
localhost:some_port, we will always end up at the IPv6 one (as long as
20
it is up), and this may not be the one we want.
21
22
Thus, we must make sure not to create an IPv6-only NBD server on the
23
same port as a normal "dual-stack" NBD server -- which is done by using
24
distinct port ranges, as explained above.
25
26
Signed-off-by: Max Reitz <mreitz@redhat.com>
27
Message-id: 20181221234750.23577-4-mreitz@redhat.com
28
Reviewed-by: John Snow <jsnow@redhat.com>
29
Signed-off-by: Max Reitz <mreitz@redhat.com>
10
Signed-off-by: Max Reitz <mreitz@redhat.com>
30
---
11
---
31
tests/qemu-iotests/147 | 98 +++++++++++++++++++++++++++++-------------
12
tests/qemu-iotests/134 | 9 +++++++++
32
1 file changed, 68 insertions(+), 30 deletions(-)
13
tests/qemu-iotests/134.out | 10 ++++++++++
14
2 files changed, 19 insertions(+)
33
15
34
diff --git a/tests/qemu-iotests/147 b/tests/qemu-iotests/147
16
diff --git a/tests/qemu-iotests/134 b/tests/qemu-iotests/134
35
index XXXXXXX..XXXXXXX 100755
17
index XXXXXXX..XXXXXXX 100755
36
--- a/tests/qemu-iotests/147
18
--- a/tests/qemu-iotests/134
37
+++ b/tests/qemu-iotests/147
19
+++ b/tests/qemu-iotests/134
38
@@ -XXX,XX +XXX,XX @@
20
@@ -XXX,XX +XXX,XX @@ echo
39
#
21
echo "== reading whole image =="
40
22
$QEMU_IO --object $SECRET -c "read 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
41
import os
23
42
+import random
24
+echo
43
import socket
25
+echo "== rewriting cluster part =="
44
import stat
26
+$QEMU_IO --object $SECRET -c "write -P 0xb 512 512" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
45
import time
46
import iotests
47
-from iotests import cachemode, imgfmt, qemu_img, qemu_nbd
48
+from iotests import cachemode, imgfmt, qemu_img, qemu_nbd, qemu_nbd_pipe
49
50
-NBD_PORT = 10811
51
+NBD_PORT_START = 32768
52
+NBD_PORT_END = NBD_PORT_START + 1024
53
+NBD_IPV6_PORT_START = NBD_PORT_END
54
+NBD_IPV6_PORT_END = NBD_IPV6_PORT_START + 1024
55
56
test_img = os.path.join(iotests.test_dir, 'test.img')
57
unix_socket = os.path.join(iotests.test_dir, 'nbd.socket')
58
@@ -XXX,XX +XXX,XX @@ class QemuNBD(NBDBlockdevAddBase):
59
except OSError:
60
pass
61
62
+ def _try_server_up(self, *args):
63
+ status, msg = qemu_nbd_pipe('-f', imgfmt, test_img, *args)
64
+ if status == 0:
65
+ return True
66
+ if 'Address already in use' in msg:
67
+ return False
68
+ self.fail(msg)
69
+
27
+
70
def _server_up(self, *args):
28
+echo
71
- self.assertEqual(qemu_nbd('-f', imgfmt, test_img, *args), 0)
29
+echo "== verify pattern =="
72
+ self.assertTrue(self._try_server_up(*args))
30
+$QEMU_IO --object $SECRET -c "read -P 0 0 512" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
73
31
+$QEMU_IO --object $SECRET -c "read -P 0xb 512 512" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
74
def test_inet(self):
75
- self._server_up('-b', 'localhost', '-p', str(NBD_PORT))
76
+ while True:
77
+ nbd_port = random.randrange(NBD_PORT_START, NBD_PORT_END)
78
+ if self._try_server_up('-b', 'localhost', '-p', str(nbd_port)):
79
+ break
80
+
32
+
81
address = { 'type': 'inet',
33
echo
82
'data': {
34
echo "== rewriting whole image =="
83
'host': 'localhost',
35
$QEMU_IO --object $SECRET -c "write -P 0xa 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
84
- 'port': str(NBD_PORT)
36
diff --git a/tests/qemu-iotests/134.out b/tests/qemu-iotests/134.out
85
+ 'port': str(nbd_port)
37
index XXXXXXX..XXXXXXX 100644
86
} }
38
--- a/tests/qemu-iotests/134.out
87
- self.client_test('nbd://localhost:%i' % NBD_PORT,
39
+++ b/tests/qemu-iotests/134.out
88
+ self.client_test('nbd://localhost:%i' % nbd_port,
40
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.
89
flatten_sock_addr(address))
41
read 134217728/134217728 bytes at offset 0
90
42
128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
91
def test_unix(self):
43
92
@@ -XXX,XX +XXX,XX @@ class BuiltinNBD(NBDBlockdevAddBase):
44
+== rewriting cluster part ==
93
except OSError:
45
+wrote 512/512 bytes at offset 512
94
pass
46
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
95
96
- def _server_up(self, address, export_name=None, export_name2=None):
97
+ # Returns False on EADDRINUSE; fails an assertion on other errors.
98
+ # Returns True on success.
99
+ def _try_server_up(self, address, export_name=None, export_name2=None):
100
result = self.server.qmp('nbd-server-start', addr=address)
101
+ if 'error' in result and \
102
+ 'Address already in use' in result['error']['desc']:
103
+ return False
104
self.assert_qmp(result, 'return', {})
105
106
if export_name is None:
107
@@ -XXX,XX +XXX,XX @@ class BuiltinNBD(NBDBlockdevAddBase):
108
name=export_name2)
109
self.assert_qmp(result, 'return', {})
110
111
+ return True
112
+
47
+
113
+ def _server_up(self, address, export_name=None, export_name2=None):
48
+== verify pattern ==
114
+ self.assertTrue(self._try_server_up(address, export_name, export_name2))
49
+read 512/512 bytes at offset 0
115
50
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
116
def _server_down(self):
51
+read 512/512 bytes at offset 512
117
result = self.server.qmp('nbd-server-stop')
52
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
118
self.assert_qmp(result, 'return', {})
119
120
def do_test_inet(self, export_name=None):
121
- address = { 'type': 'inet',
122
- 'data': {
123
- 'host': 'localhost',
124
- 'port': str(NBD_PORT)
125
- } }
126
- self._server_up(address, export_name)
127
+ while True:
128
+ nbd_port = random.randrange(NBD_PORT_START, NBD_PORT_END)
129
+ address = { 'type': 'inet',
130
+ 'data': {
131
+ 'host': 'localhost',
132
+ 'port': str(nbd_port)
133
+ } }
134
+ if self._try_server_up(address, export_name):
135
+ break
136
+
53
+
137
export_name = export_name or 'nbd-export'
54
== rewriting whole image ==
138
- self.client_test('nbd://localhost:%i/%s' % (NBD_PORT, export_name),
55
wrote 134217728/134217728 bytes at offset 0
139
+ self.client_test('nbd://localhost:%i/%s' % (nbd_port, export_name),
56
128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
140
flatten_sock_addr(address), export_name)
141
self._server_down()
142
143
@@ -XXX,XX +XXX,XX @@ class BuiltinNBD(NBDBlockdevAddBase):
144
self.do_test_inet('shadow')
145
146
def test_inet_two_exports(self):
147
- address = { 'type': 'inet',
148
- 'data': {
149
- 'host': 'localhost',
150
- 'port': str(NBD_PORT)
151
- } }
152
- self._server_up(address, 'exp1', 'exp2')
153
- self.client_test('nbd://localhost:%i/%s' % (NBD_PORT, 'exp1'),
154
+ while True:
155
+ nbd_port = random.randrange(NBD_PORT_START, NBD_PORT_END)
156
+ address = { 'type': 'inet',
157
+ 'data': {
158
+ 'host': 'localhost',
159
+ 'port': str(nbd_port)
160
+ } }
161
+ if self._try_server_up(address, 'exp1', 'exp2'):
162
+ break
163
+
164
+ self.client_test('nbd://localhost:%i/%s' % (nbd_port, 'exp1'),
165
flatten_sock_addr(address), 'exp1', 'node1', False)
166
- self.client_test('nbd://localhost:%i/%s' % (NBD_PORT, 'exp2'),
167
+ self.client_test('nbd://localhost:%i/%s' % (nbd_port, 'exp2'),
168
flatten_sock_addr(address), 'exp2', 'node2', False)
169
result = self.vm.qmp('blockdev-del', node_name='node1')
170
self.assert_qmp(result, 'return', {})
171
@@ -XXX,XX +XXX,XX @@ class BuiltinNBD(NBDBlockdevAddBase):
172
except socket.gaierror:
173
# IPv6 not available, skip
174
return
175
- address = { 'type': 'inet',
176
- 'data': {
177
- 'host': '::1',
178
- 'port': str(NBD_PORT),
179
- 'ipv4': False,
180
- 'ipv6': True
181
- } }
182
+
183
+ while True:
184
+ nbd_port = random.randrange(NBD_IPV6_PORT_START, NBD_IPV6_PORT_END)
185
+ address = { 'type': 'inet',
186
+ 'data': {
187
+ 'host': '::1',
188
+ 'port': str(nbd_port),
189
+ 'ipv4': False,
190
+ 'ipv6': True
191
+ } }
192
+ if self._try_server_up(address):
193
+ break
194
+
195
filename = { 'driver': 'raw',
196
'file': {
197
'driver': 'nbd',
198
'export': 'nbd-export',
199
'server': flatten_sock_addr(address)
200
} }
201
- self._server_up(address)
202
self.client_test(filename, flatten_sock_addr(address), 'nbd-export')
203
self._server_down()
204
205
--
57
--
206
2.20.1
58
2.21.0
207
59
208
60
diff view generated by jsdifflib
1
In some cases, we may want to deal with qemu-nbd errors (e.g. by
1
From: Sam Eiderman <shmuel.eiderman@oracle.com>
2
launching it in a different configuration until it no longer throws
3
any). In that case, we do not want its output ending up in the test
4
output.
5
2
6
It may still be useful for handling the error, though, so add a new
3
Commit b0651b8c246d ("vmdk: Move l1_size check into vmdk_add_extent")
7
function that works basically like qemu_nbd(), only that it returns the
4
extended the l1_size check from VMDK4 to VMDK3 but did not update the
8
qemu-nbd output instead of making it end up in the log. In contrast to
5
default coverage in the moved comment.
9
qemu_img_pipe(), it does still return the exit code as well, though,
10
because that is even more important for error handling.
11
6
12
Signed-off-by: Max Reitz <mreitz@redhat.com>
7
The previous vmdk4 calculation:
13
Message-id: 20181221234750.23577-2-mreitz@redhat.com
8
14
Reviewed-by: John Snow <jsnow@redhat.com>
9
(512 * 1024 * 1024) * 512(l2 entries) * 65536(grain) = 16PB
15
Reviewed-by: Eric Blake <eblake@redhat.com>
10
11
The added vmdk3 calculation:
12
13
(512 * 1024 * 1024) * 4096(l2 entries) * 512(grain) = 1PB
14
15
Adding the calculation of vmdk3 to the comment.
16
17
In any case, VMware does not offer virtual disks more than 2TB for
18
vmdk4/vmdk3 or 64TB for the new undocumented seSparse format which is
19
not implemented yet in qemu.
20
21
Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com>
22
Reviewed-by: Eyal Moscovici <eyal.moscovici@oracle.com>
23
Reviewed-by: Liran Alon <liran.alon@oracle.com>
24
Reviewed-by: Arbel Moshe <arbel.moshe@oracle.com>
25
Signed-off-by: Sam Eiderman <shmuel.eiderman@oracle.com>
26
Message-id: 20190620091057.47441-2-shmuel.eiderman@oracle.com
27
Reviewed-by: yuchenlin <yuchenlin@synology.com>
28
Reviewed-by: Max Reitz <mreitz@redhat.com>
16
Signed-off-by: Max Reitz <mreitz@redhat.com>
29
Signed-off-by: Max Reitz <mreitz@redhat.com>
17
---
30
---
18
tests/qemu-iotests/iotests.py | 14 ++++++++++++++
31
block/vmdk.c | 11 ++++++++---
19
1 file changed, 14 insertions(+)
32
1 file changed, 8 insertions(+), 3 deletions(-)
20
33
21
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
34
diff --git a/block/vmdk.c b/block/vmdk.c
22
index XXXXXXX..XXXXXXX 100644
35
index XXXXXXX..XXXXXXX 100644
23
--- a/tests/qemu-iotests/iotests.py
36
--- a/block/vmdk.c
24
+++ b/tests/qemu-iotests/iotests.py
37
+++ b/block/vmdk.c
25
@@ -XXX,XX +XXX,XX @@ def qemu_nbd(*args):
38
@@ -XXX,XX +XXX,XX @@ static int vmdk_add_extent(BlockDriverState *bs,
26
'''Run qemu-nbd in daemon mode and return the parent's exit code'''
39
return -EFBIG;
27
return subprocess.call(qemu_nbd_args + ['--fork'] + list(args))
40
}
28
41
if (l1_size > 512 * 1024 * 1024) {
29
+def qemu_nbd_pipe(*args):
42
- /* Although with big capacity and small l1_entry_sectors, we can get a
30
+ '''Run qemu-nbd in daemon mode and return both the parent's exit code
43
+ /*
31
+ and its output'''
44
+ * Although with big capacity and small l1_entry_sectors, we can get a
32
+ subp = subprocess.Popen(qemu_nbd_args + ['--fork'] + list(args),
45
* big l1_size, we don't want unbounded value to allocate the table.
33
+ stdout=subprocess.PIPE,
46
- * Limit it to 512M, which is 16PB for default cluster and L2 table
34
+ stderr=subprocess.STDOUT,
47
- * size */
35
+ universal_newlines=True)
48
+ * Limit it to 512M, which is:
36
+ exitcode = subp.wait()
49
+ * 16PB - for default "Hosted Sparse Extent" (VMDK4)
37
+ if exitcode < 0:
50
+ * cluster size: 64KB, L2 table size: 512 entries
38
+ sys.stderr.write('qemu-nbd received signal %i: %s\n' %
51
+ * 1PB - for default "ESXi Host Sparse Extent" (VMDK3/vmfsSparse)
39
+ (-exitcode,
52
+ * cluster size: 512B, L2 table size: 4096 entries
40
+ ' '.join(qemu_nbd_args + ['--fork'] + list(args))))
53
+ */
41
+ return exitcode, subp.communicate()[0]
54
error_setg(errp, "L1 size too big");
42
+
55
return -EFBIG;
43
def compare_images(img1, img2, fmt1=imgfmt, fmt2=imgfmt):
56
}
44
'''Return True if two image files are identical'''
45
return qemu_img('compare', '-f', fmt1,
46
--
57
--
47
2.20.1
58
2.21.0
48
59
49
60
diff view generated by jsdifflib
1
From: Li Qiang <liq3ea@163.com>
1
From: Sam Eiderman <shmuel.eiderman@oracle.com>
2
2
3
When it is zero, it causes segv.
3
512M of L1 entries is a very loose bound, only 32M are required to store
4
Using following command:
4
the maximal supported VMDK file size of 2TB.
5
5
6
"-drive file=//home/test/test1.img,if=none,id=id0
6
Fixed qemu-iotest 59# - now failure occures before on impossible L1
7
-device nvme,drive=id0,serial=test,num_queues=0"
7
table size.
8
causes following Backtrack:
9
8
10
Thread 4 "qemu-system-x86" received signal SIGSEGV, Segmentation fault.
9
Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com>
11
[Switching to Thread 0x7fffe9735700 (LWP 30952)]
10
Reviewed-by: Eyal Moscovici <eyal.moscovici@oracle.com>
12
0x0000555555a7a77c in nvme_start_ctrl (n=0x5555577473f0) at hw/block/nvme.c:825
11
Reviewed-by: Liran Alon <liran.alon@oracle.com>
13
825     if (unlikely(n->cq[0])) {
12
Reviewed-by: Arbel Moshe <arbel.moshe@oracle.com>
14
(gdb) bt
13
Signed-off-by: Sam Eiderman <shmuel.eiderman@oracle.com>
15
0 0x0000555555a7a77c in nvme_start_ctrl (n=0x5555577473f0)
14
Message-id: 20190620091057.47441-3-shmuel.eiderman@oracle.com
16
at hw/block/nvme.c:825
15
Reviewed-by: Max Reitz <mreitz@redhat.com>
17
1 0x0000555555a7af7f in nvme_write_bar (n=0x5555577473f0, offset=20,
18
data=4587521, size=4) at hw/block/nvme.c:969
19
2 0x0000555555a7b81a in nvme_mmio_write (opaque=0x5555577473f0, addr=20,
20
data=4587521, size=4) at hw/block/nvme.c:1163
21
3 0x0000555555869236 in memory_region_write_accessor (mr=0x555557747cd0,
22
addr=20, value=0x7fffe97320f8, size=4, shift=0, mask=4294967295, attrs=...)
23
at /home/test/qemu1/qemu/memory.c:502
24
4 0x0000555555869446 in access_with_adjusted_size (addr=20,
25
value=0x7fffe97320f8, size=4, access_size_min=2, access_size_max=8,
26
access_fn=0x55555586914d <memory_region_write_accessor>,
27
mr=0x555557747cd0, attrs=...) at /home/test/qemu1/qemu/memory.c:568
28
5 0x000055555586c479 in memory_region_dispatch_write (mr=0x555557747cd0,
29
addr=20, data=4587521, size=4, attrs=...)
30
at /home/test/qemu1/qemu/memory.c:1499
31
6 0x00005555558030af in flatview_write_continue (fv=0x7fffe0061130,
32
addr=4273930260, attrs=..., buf=0x7ffff7ff0028 "\001", len=4, addr1=20,
33
l=4, mr=0x555557747cd0) at /home/test/qemu1/qemu/exec.c:3234
34
7 0x00005555558031f9 in flatview_write (fv=0x7fffe0061130, addr=4273930260,
35
attrs=..., buf=0x7ffff7ff0028 "\001", len=4)
36
at /home/test/qemu1/qemu/exec.c:3273
37
8 0x00005555558034ff in address_space_write (
38
---Type <return> to continue, or q <return> to quit---
39
as=0x555556758480 <address_space_memory>, addr=4273930260, attrs=...,
40
buf=0x7ffff7ff0028 "\001", len=4) at /home/test/qemu1/qemu/exec.c:3363
41
9 0x0000555555803550 in address_space_rw (
42
as=0x555556758480 <address_space_memory>, addr=4273930260, attrs=...,
43
buf=0x7ffff7ff0028 "\001", len=4, is_write=true)
44
at /home/test/qemu1/qemu/exec.c:3374
45
10 0x00005555558884a1 in kvm_cpu_exec (cpu=0x555556920e40)
46
at /home/test/qemu1/qemu/accel/kvm/kvm-all.c:2031
47
11 0x000055555584cd9d in qemu_kvm_cpu_thread_fn (arg=0x555556920e40)
48
at /home/test/qemu1/qemu/cpus.c:1281
49
12 0x0000555555dbaf6d in qemu_thread_start (args=0x5555569438a0)
50
at util/qemu-thread-posix.c:502
51
13 0x00007ffff5dc86db in start_thread (arg=0x7fffe9735700)
52
at pthread_create.c:463
53
14 0x00007ffff5af188f in clone ()
54
at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
55
56
Signed-off-by: Li Qiang <liq3ea@163.com>
57
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
58
Message-id: 20190120055558.32984-3-liq3ea@163.com
59
Signed-off-by: Max Reitz <mreitz@redhat.com>
16
Signed-off-by: Max Reitz <mreitz@redhat.com>
60
---
17
---
61
hw/block/nvme.c | 5 +++++
18
block/vmdk.c | 13 +++++++------
62
1 file changed, 5 insertions(+)
19
tests/qemu-iotests/059.out | 2 +-
20
2 files changed, 8 insertions(+), 7 deletions(-)
63
21
64
diff --git a/hw/block/nvme.c b/hw/block/nvme.c
22
diff --git a/block/vmdk.c b/block/vmdk.c
65
index XXXXXXX..XXXXXXX 100644
23
index XXXXXXX..XXXXXXX 100644
66
--- a/hw/block/nvme.c
24
--- a/block/vmdk.c
67
+++ b/hw/block/nvme.c
25
+++ b/block/vmdk.c
68
@@ -XXX,XX +XXX,XX @@ static void nvme_realize(PCIDevice *pci_dev, Error **errp)
26
@@ -XXX,XX +XXX,XX @@ static int vmdk_add_extent(BlockDriverState *bs,
69
int64_t bs_size;
27
error_setg(errp, "Invalid granularity, image may be corrupt");
70
uint8_t *pci_conf;
28
return -EFBIG;
71
29
}
72
+ if (!n->num_queues) {
30
- if (l1_size > 512 * 1024 * 1024) {
73
+ error_setg(errp, "num_queues can't be zero");
31
+ if (l1_size > 32 * 1024 * 1024) {
74
+ return;
32
/*
75
+ }
33
* Although with big capacity and small l1_entry_sectors, we can get a
76
+
34
* big l1_size, we don't want unbounded value to allocate the table.
77
if (!n->conf.blk) {
35
- * Limit it to 512M, which is:
78
error_setg(errp, "drive property not set");
36
- * 16PB - for default "Hosted Sparse Extent" (VMDK4)
79
return;
37
- * cluster size: 64KB, L2 table size: 512 entries
38
- * 1PB - for default "ESXi Host Sparse Extent" (VMDK3/vmfsSparse)
39
- * cluster size: 512B, L2 table size: 4096 entries
40
+ * Limit it to 32M, which is enough to store:
41
+ * 8TB - for both VMDK3 & VMDK4 with
42
+ * minimal cluster size: 512B
43
+ * minimal L2 table size: 512 entries
44
+ * 8 TB is still more than the maximal value supported for
45
+ * VMDK3 & VMDK4 which is 2TB.
46
*/
47
error_setg(errp, "L1 size too big");
48
return -EFBIG;
49
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
50
index XXXXXXX..XXXXXXX 100644
51
--- a/tests/qemu-iotests/059.out
52
+++ b/tests/qemu-iotests/059.out
53
@@ -XXX,XX +XXX,XX @@ Offset Length Mapped to File
54
0x140000000 0x10000 0x50000 TEST_DIR/t-s003.vmdk
55
56
=== Testing afl image with a very large capacity ===
57
-qemu-img: Can't get image size 'TEST_DIR/afl9.IMGFMT': File too large
58
+qemu-img: Could not open 'TEST_DIR/afl9.IMGFMT': L1 size too big
59
*** done
80
--
60
--
81
2.20.1
61
2.21.0
82
62
83
63
diff view generated by jsdifflib
1
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
1
From: Sam Eiderman <shmuel.eiderman@oracle.com>
2
2
3
Add a new command, returning block nodes (and their users) graph.
3
Until ESXi 6.5 VMware used the vmfsSparse format for snapshots (VMDK3 in
4
4
QEMU).
5
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
5
6
Message-id: 20181221170909.25584-2-vsementsov@virtuozzo.com
6
This format was lacking in the following:
7
8
* Grain directory (L1) and grain table (L2) entries were 32-bit,
9
allowing access to only 2TB (slightly less) of data.
10
* The grain size (default) was 512 bytes - leading to data
11
fragmentation and many grain tables.
12
* For space reclamation purposes, it was necessary to find all the
13
grains which are not pointed to by any grain table - so a reverse
14
mapping of "offset of grain in vmdk" to "grain table" must be
15
constructed - which takes large amounts of CPU/RAM.
16
17
The format specification can be found in VMware's documentation:
18
https://www.vmware.com/support/developer/vddk/vmdk_50_technote.pdf
19
20
In ESXi 6.5, to support snapshot files larger than 2TB, a new format was
21
introduced: SESparse (Space Efficient).
22
23
This format fixes the above issues:
24
25
* All entries are now 64-bit.
26
* The grain size (default) is 4KB.
27
* Grain directory and grain tables are now located at the beginning
28
of the file.
29
+ seSparse format reserves space for all grain tables.
30
+ Grain tables can be addressed using an index.
31
+ Grains are located in the end of the file and can also be
32
addressed with an index.
33
- seSparse vmdks of large disks (64TB) have huge preallocated
34
headers - mainly due to L2 tables, even for empty snapshots.
35
* The header contains a reverse mapping ("backmap") of "offset of
36
grain in vmdk" to "grain table" and a bitmap ("free bitmap") which
37
specifies for each grain - whether it is allocated or not.
38
Using these data structures we can implement space reclamation
39
efficiently.
40
* Due to the fact that the header now maintains two mappings:
41
* The regular one (grain directory & grain tables)
42
* A reverse one (backmap and free bitmap)
43
These data structures can lose consistency upon crash and result
44
in a corrupted VMDK.
45
Therefore, a journal is also added to the VMDK and is replayed
46
when the VMware reopens the file after a crash.
47
48
Since ESXi 6.7 - SESparse is the only snapshot format available.
49
50
Unfortunately, VMware does not provide documentation regarding the new
51
seSparse format.
52
53
This commit is based on black-box research of the seSparse format.
54
Various in-guest block operations and their effect on the snapshot file
55
were tested.
56
57
The only VMware provided source of information (regarding the underlying
58
implementation) was a log file on the ESXi:
59
60
/var/log/hostd.log
61
62
Whenever an seSparse snapshot is created - the log is being populated
63
with seSparse records.
64
65
Relevant log records are of the form:
66
67
[...] Const Header:
68
[...] constMagic = 0xcafebabe
69
[...] version = 2.1
70
[...] capacity = 204800
71
[...] grainSize = 8
72
[...] grainTableSize = 64
73
[...] flags = 0
74
[...] Extents:
75
[...] Header : <1 : 1>
76
[...] JournalHdr : <2 : 2>
77
[...] Journal : <2048 : 2048>
78
[...] GrainDirectory : <4096 : 2048>
79
[...] GrainTables : <6144 : 2048>
80
[...] FreeBitmap : <8192 : 2048>
81
[...] BackMap : <10240 : 2048>
82
[...] Grain : <12288 : 204800>
83
[...] Volatile Header:
84
[...] volatileMagic = 0xcafecafe
85
[...] FreeGTNumber = 0
86
[...] nextTxnSeqNumber = 0
87
[...] replayJournal = 0
88
89
The sizes that are seen in the log file are in sectors.
90
Extents are of the following format: <offset : size>
91
92
This commit is a strict implementation which enforces:
93
* magics
94
* version number 2.1
95
* grain size of 8 sectors (4KB)
96
* grain table size of 64 sectors
97
* zero flags
98
* extent locations
99
100
Additionally, this commit proivdes only a subset of the functionality
101
offered by seSparse's format:
102
* Read-only
103
* No journal replay
104
* No space reclamation
105
* No unmap support
106
107
Hence, journal header, journal, free bitmap and backmap extents are
108
unused, only the "classic" (L1 -> L2 -> data) grain access is
109
implemented.
110
111
However there are several differences in the grain access itself.
112
Grain directory (L1):
113
* Grain directory entries are indexes (not offsets) to grain
114
tables.
115
* Valid grain directory entries have their highest nibble set to
116
0x1.
117
* Since grain tables are always located in the beginning of the
118
file - the index can fit into 32 bits - so we can use its low
119
part if it's valid.
120
Grain table (L2):
121
* Grain table entries are indexes (not offsets) to grains.
122
* If the highest nibble of the entry is:
123
0x0:
124
The grain in not allocated.
125
The rest of the bytes are 0.
126
0x1:
127
The grain is unmapped - guest sees a zero grain.
128
The rest of the bits point to the previously mapped grain,
129
see 0x3 case.
130
0x2:
131
The grain is zero.
132
0x3:
133
The grain is allocated - to get the index calculate:
134
((entry & 0x0fff000000000000) >> 48) |
135
((entry & 0x0000ffffffffffff) << 12)
136
* The difference between 0x1 and 0x2 is that 0x1 is an unallocated
137
grain which results from the guest using sg_unmap to unmap the
138
grain - but the grain itself still exists in the grain extent - a
139
space reclamation procedure should delete it.
140
Unmapping a zero grain has no effect (0x2 will not change to 0x1)
141
but unmapping an unallocated grain will (0x0 to 0x1) - naturally.
142
143
In order to implement seSparse some fields had to be changed to support
144
both 32-bit and 64-bit entry sizes.
145
146
Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com>
147
Reviewed-by: Eyal Moscovici <eyal.moscovici@oracle.com>
148
Reviewed-by: Arbel Moshe <arbel.moshe@oracle.com>
149
Signed-off-by: Sam Eiderman <shmuel.eiderman@oracle.com>
150
Message-id: 20190620091057.47441-4-shmuel.eiderman@oracle.com
7
Signed-off-by: Max Reitz <mreitz@redhat.com>
151
Signed-off-by: Max Reitz <mreitz@redhat.com>
8
---
152
---
9
qapi/block-core.json | 108 ++++++++++++++++++++++++
153
block/vmdk.c | 358 ++++++++++++++++++++++++++++++++++++++++++++++++---
10
include/block/block.h | 1 +
154
1 file changed, 342 insertions(+), 16 deletions(-)
11
include/sysemu/block-backend.h | 2 +
155
12
block.c | 148 +++++++++++++++++++++++++++++++++
156
diff --git a/block/vmdk.c b/block/vmdk.c
13
block/block-backend.c | 5 ++
14
blockdev.c | 5 ++
15
6 files changed, 269 insertions(+)
16
17
diff --git a/qapi/block-core.json b/qapi/block-core.json
18
index XXXXXXX..XXXXXXX 100644
157
index XXXXXXX..XXXXXXX 100644
19
--- a/qapi/block-core.json
158
--- a/block/vmdk.c
20
+++ b/qapi/block-core.json
159
+++ b/block/vmdk.c
21
@@ -XXX,XX +XXX,XX @@
160
@@ -XXX,XX +XXX,XX @@ typedef struct {
22
##
161
uint16_t compressAlgorithm;
23
{ 'command': 'query-named-block-nodes', 'returns': [ 'BlockDeviceInfo' ] }
162
} QEMU_PACKED VMDK4Header;
24
163
25
+##
164
+typedef struct VMDKSESparseConstHeader {
26
+# @XDbgBlockGraphNodeType:
165
+ uint64_t magic;
27
+#
166
+ uint64_t version;
28
+# @block-backend: corresponds to BlockBackend
167
+ uint64_t capacity;
29
+#
168
+ uint64_t grain_size;
30
+# @block-job: corresonds to BlockJob
169
+ uint64_t grain_table_size;
31
+#
170
+ uint64_t flags;
32
+# @block-driver: corresponds to BlockDriverState
171
+ uint64_t reserved1;
33
+#
172
+ uint64_t reserved2;
34
+# Since: 4.0
173
+ uint64_t reserved3;
35
+##
174
+ uint64_t reserved4;
36
+{ 'enum': 'XDbgBlockGraphNodeType',
175
+ uint64_t volatile_header_offset;
37
+ 'data': [ 'block-backend', 'block-job', 'block-driver' ] }
176
+ uint64_t volatile_header_size;
38
+
177
+ uint64_t journal_header_offset;
39
+##
178
+ uint64_t journal_header_size;
40
+# @XDbgBlockGraphNode:
179
+ uint64_t journal_offset;
41
+#
180
+ uint64_t journal_size;
42
+# @id: Block graph node identifier. This @id is generated only for
181
+ uint64_t grain_dir_offset;
43
+# x-debug-query-block-graph and does not relate to any other identifiers in
182
+ uint64_t grain_dir_size;
44
+# Qemu.
183
+ uint64_t grain_tables_offset;
45
+#
184
+ uint64_t grain_tables_size;
46
+# @type: Type of graph node. Can be one of block-backend, block-job or
185
+ uint64_t free_bitmap_offset;
47
+# block-driver-state.
186
+ uint64_t free_bitmap_size;
48
+#
187
+ uint64_t backmap_offset;
49
+# @name: Human readable name of the node. Corresponds to node-name for
188
+ uint64_t backmap_size;
50
+# block-driver-state nodes; is not guaranteed to be unique in the whole
189
+ uint64_t grains_offset;
51
+# graph (with block-jobs and block-backends).
190
+ uint64_t grains_size;
52
+#
191
+ uint8_t pad[304];
53
+# Since: 4.0
192
+} QEMU_PACKED VMDKSESparseConstHeader;
54
+##
193
+
55
+{ 'struct': 'XDbgBlockGraphNode',
194
+typedef struct VMDKSESparseVolatileHeader {
56
+ 'data': { 'id': 'uint64', 'type': 'XDbgBlockGraphNodeType', 'name': 'str' } }
195
+ uint64_t magic;
57
+
196
+ uint64_t free_gt_number;
58
+##
197
+ uint64_t next_txn_seq_number;
59
+# @BlockPermission:
198
+ uint64_t replay_journal;
60
+#
199
+ uint8_t pad[480];
61
+# Enum of base block permissions.
200
+} QEMU_PACKED VMDKSESparseVolatileHeader;
62
+#
201
+
63
+# @consistent-read: A user that has the "permission" of consistent reads is
202
#define L2_CACHE_SIZE 16
64
+# guaranteed that their view of the contents of the block
203
65
+# device is complete and self-consistent, representing the
204
typedef struct VmdkExtent {
66
+# contents of a disk at a specific point.
205
@@ -XXX,XX +XXX,XX @@ typedef struct VmdkExtent {
67
+# For most block devices (including their backing files) this
206
bool compressed;
68
+# is true, but the property cannot be maintained in a few
207
bool has_marker;
69
+# situations like for intermediate nodes of a commit block
208
bool has_zero_grain;
70
+# job.
209
+ bool sesparse;
71
+#
210
+ uint64_t sesparse_l2_tables_offset;
72
+# @write: This permission is required to change the visible disk contents.
211
+ uint64_t sesparse_clusters_offset;
73
+#
212
+ int32_t entry_size;
74
+# @write-unchanged: This permission (which is weaker than BLK_PERM_WRITE) is
213
int version;
75
+# both enough and required for writes to the block node when
214
int64_t sectors;
76
+# the caller promises that the visible disk content doesn't
215
int64_t end_sector;
77
+# change.
216
int64_t flat_start_offset;
78
+# As the BLK_PERM_WRITE permission is strictly stronger,
217
int64_t l1_table_offset;
79
+# either is sufficient to perform an unchanging write.
218
int64_t l1_backup_table_offset;
80
+#
219
- uint32_t *l1_table;
81
+# @resize: This permission is required to change the size of a block node.
220
+ void *l1_table;
82
+#
221
uint32_t *l1_backup_table;
83
+# @graph-mod: This permission is required to change the node that this
222
unsigned int l1_size;
84
+# BdrvChild points to.
223
uint32_t l1_entry_sectors;
85
+#
224
86
+# Since: 4.0
225
unsigned int l2_size;
87
+##
226
- uint32_t *l2_cache;
88
+ { 'enum': 'BlockPermission',
227
+ void *l2_cache;
89
+ 'data': [ 'consistent-read', 'write', 'write-unchanged', 'resize',
228
uint32_t l2_cache_offsets[L2_CACHE_SIZE];
90
+ 'graph-mod' ] }
229
uint32_t l2_cache_counts[L2_CACHE_SIZE];
91
+##
230
92
+# @XDbgBlockGraphEdge:
231
@@ -XXX,XX +XXX,XX @@ static int vmdk_add_extent(BlockDriverState *bs,
93
+#
232
* minimal L2 table size: 512 entries
94
+# Block Graph edge description for x-debug-query-block-graph.
233
* 8 TB is still more than the maximal value supported for
95
+#
234
* VMDK3 & VMDK4 which is 2TB.
96
+# @parent: parent id
235
+ * 64TB - for "ESXi seSparse Extent"
97
+#
236
+ * minimal cluster size: 512B (default is 4KB)
98
+# @child: child id
237
+ * L2 table size: 4096 entries (const).
99
+#
238
+ * 64TB is more than the maximal value supported for
100
+# @name: name of the relation (examples are 'file' and 'backing')
239
+ * seSparse VMDKs (which is slightly less than 64TB)
101
+#
240
*/
102
+# @perm: granted permissions for the parent operating on the child
241
error_setg(errp, "L1 size too big");
103
+#
242
return -EFBIG;
104
+# @shared-perm: permissions that can still be granted to other users of the
243
@@ -XXX,XX +XXX,XX @@ static int vmdk_add_extent(BlockDriverState *bs,
105
+# child while it is still attached to this parent
244
extent->l2_size = l2_size;
106
+#
245
extent->cluster_sectors = flat ? sectors : cluster_sectors;
107
+# Since: 4.0
246
extent->next_cluster_sector = ROUND_UP(nb_sectors, cluster_sectors);
108
+##
247
+ extent->entry_size = sizeof(uint32_t);
109
+{ 'struct': 'XDbgBlockGraphEdge',
248
110
+ 'data': { 'parent': 'uint64', 'child': 'uint64',
249
if (s->num_extents > 1) {
111
+ 'name': 'str', 'perm': [ 'BlockPermission' ],
250
extent->end_sector = (*(extent - 1)).end_sector + extent->sectors;
112
+ 'shared-perm': [ 'BlockPermission' ] } }
251
@@ -XXX,XX +XXX,XX @@ static int vmdk_init_tables(BlockDriverState *bs, VmdkExtent *extent,
113
+
252
int i;
114
+##
253
115
+# @XDbgBlockGraph:
254
/* read the L1 table */
116
+#
255
- l1_size = extent->l1_size * sizeof(uint32_t);
117
+# Block Graph - list of nodes and list of edges.
256
+ l1_size = extent->l1_size * extent->entry_size;
118
+#
257
extent->l1_table = g_try_malloc(l1_size);
119
+# Since: 4.0
258
if (l1_size && extent->l1_table == NULL) {
120
+##
259
return -ENOMEM;
121
+{ 'struct': 'XDbgBlockGraph',
260
@@ -XXX,XX +XXX,XX @@ static int vmdk_init_tables(BlockDriverState *bs, VmdkExtent *extent,
122
+ 'data': { 'nodes': ['XDbgBlockGraphNode'], 'edges': ['XDbgBlockGraphEdge'] } }
261
goto fail_l1;
123
+
262
}
124
+##
263
for (i = 0; i < extent->l1_size; i++) {
125
+# @x-debug-query-block-graph:
264
- le32_to_cpus(&extent->l1_table[i]);
126
+#
265
+ if (extent->entry_size == sizeof(uint64_t)) {
127
+# Get the block graph.
266
+ le64_to_cpus((uint64_t *)extent->l1_table + i);
128
+#
267
+ } else {
129
+# Since: 4.0
268
+ assert(extent->entry_size == sizeof(uint32_t));
130
+##
269
+ le32_to_cpus((uint32_t *)extent->l1_table + i);
131
+{ 'command': 'x-debug-query-block-graph', 'returns': 'XDbgBlockGraph' }
270
+ }
132
+
271
}
133
##
272
134
# @drive-mirror:
273
if (extent->l1_backup_table_offset) {
135
#
274
+ assert(!extent->sesparse);
136
diff --git a/include/block/block.h b/include/block/block.h
275
extent->l1_backup_table = g_try_malloc(l1_size);
137
index XXXXXXX..XXXXXXX 100644
276
if (l1_size && extent->l1_backup_table == NULL) {
138
--- a/include/block/block.h
277
ret = -ENOMEM;
139
+++ b/include/block/block.h
278
@@ -XXX,XX +XXX,XX @@ static int vmdk_init_tables(BlockDriverState *bs, VmdkExtent *extent,
140
@@ -XXX,XX +XXX,XX @@ void bdrv_eject(BlockDriverState *bs, bool eject_flag);
279
}
141
const char *bdrv_get_format_name(BlockDriverState *bs);
280
142
BlockDriverState *bdrv_find_node(const char *node_name);
281
extent->l2_cache =
143
BlockDeviceInfoList *bdrv_named_nodes_list(Error **errp);
282
- g_new(uint32_t, extent->l2_size * L2_CACHE_SIZE);
144
+XDbgBlockGraph *bdrv_get_xdbg_block_graph(Error **errp);
283
+ g_malloc(extent->entry_size * extent->l2_size * L2_CACHE_SIZE);
145
BlockDriverState *bdrv_lookup_bs(const char *device,
284
return 0;
146
const char *node_name,
285
fail_l1b:
147
Error **errp);
286
g_free(extent->l1_backup_table);
148
diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h
287
@@ -XXX,XX +XXX,XX @@ static int vmdk_open_vmfs_sparse(BlockDriverState *bs,
149
index XXXXXXX..XXXXXXX 100644
288
return ret;
150
--- a/include/sysemu/block-backend.h
151
+++ b/include/sysemu/block-backend.h
152
@@ -XXX,XX +XXX,XX @@ int coroutine_fn blk_co_copy_range(BlockBackend *blk_in, int64_t off_in,
153
int bytes, BdrvRequestFlags read_flags,
154
BdrvRequestFlags write_flags);
155
156
+const BdrvChild *blk_root(BlockBackend *blk);
157
+
158
#endif
159
diff --git a/block.c b/block.c
160
index XXXXXXX..XXXXXXX 100644
161
--- a/block.c
162
+++ b/block.c
163
@@ -XXX,XX +XXX,XX @@ BlockDeviceInfoList *bdrv_named_nodes_list(Error **errp)
164
return list;
165
}
289
}
166
290
167
+#define QAPI_LIST_ADD(list, element) do { \
291
+#define SESPARSE_CONST_HEADER_MAGIC UINT64_C(0x00000000cafebabe)
168
+ typeof(list) _tmp = g_new(typeof(*(list)), 1); \
292
+#define SESPARSE_VOLATILE_HEADER_MAGIC UINT64_C(0x00000000cafecafe)
169
+ _tmp->value = (element); \
293
+
170
+ _tmp->next = (list); \
294
+/* Strict checks - format not officially documented */
171
+ (list) = _tmp; \
295
+static int check_se_sparse_const_header(VMDKSESparseConstHeader *header,
172
+} while (0)
296
+ Error **errp)
173
+
174
+typedef struct XDbgBlockGraphConstructor {
175
+ XDbgBlockGraph *graph;
176
+ GHashTable *graph_nodes;
177
+} XDbgBlockGraphConstructor;
178
+
179
+static XDbgBlockGraphConstructor *xdbg_graph_new(void)
180
+{
297
+{
181
+ XDbgBlockGraphConstructor *gr = g_new(XDbgBlockGraphConstructor, 1);
298
+ header->magic = le64_to_cpu(header->magic);
182
+
299
+ header->version = le64_to_cpu(header->version);
183
+ gr->graph = g_new0(XDbgBlockGraph, 1);
300
+ header->grain_size = le64_to_cpu(header->grain_size);
184
+ gr->graph_nodes = g_hash_table_new(NULL, NULL);
301
+ header->grain_table_size = le64_to_cpu(header->grain_table_size);
185
+
302
+ header->flags = le64_to_cpu(header->flags);
186
+ return gr;
303
+ header->reserved1 = le64_to_cpu(header->reserved1);
304
+ header->reserved2 = le64_to_cpu(header->reserved2);
305
+ header->reserved3 = le64_to_cpu(header->reserved3);
306
+ header->reserved4 = le64_to_cpu(header->reserved4);
307
+
308
+ header->volatile_header_offset =
309
+ le64_to_cpu(header->volatile_header_offset);
310
+ header->volatile_header_size = le64_to_cpu(header->volatile_header_size);
311
+
312
+ header->journal_header_offset = le64_to_cpu(header->journal_header_offset);
313
+ header->journal_header_size = le64_to_cpu(header->journal_header_size);
314
+
315
+ header->journal_offset = le64_to_cpu(header->journal_offset);
316
+ header->journal_size = le64_to_cpu(header->journal_size);
317
+
318
+ header->grain_dir_offset = le64_to_cpu(header->grain_dir_offset);
319
+ header->grain_dir_size = le64_to_cpu(header->grain_dir_size);
320
+
321
+ header->grain_tables_offset = le64_to_cpu(header->grain_tables_offset);
322
+ header->grain_tables_size = le64_to_cpu(header->grain_tables_size);
323
+
324
+ header->free_bitmap_offset = le64_to_cpu(header->free_bitmap_offset);
325
+ header->free_bitmap_size = le64_to_cpu(header->free_bitmap_size);
326
+
327
+ header->backmap_offset = le64_to_cpu(header->backmap_offset);
328
+ header->backmap_size = le64_to_cpu(header->backmap_size);
329
+
330
+ header->grains_offset = le64_to_cpu(header->grains_offset);
331
+ header->grains_size = le64_to_cpu(header->grains_size);
332
+
333
+ if (header->magic != SESPARSE_CONST_HEADER_MAGIC) {
334
+ error_setg(errp, "Bad const header magic: 0x%016" PRIx64,
335
+ header->magic);
336
+ return -EINVAL;
337
+ }
338
+
339
+ if (header->version != 0x0000000200000001) {
340
+ error_setg(errp, "Unsupported version: 0x%016" PRIx64,
341
+ header->version);
342
+ return -ENOTSUP;
343
+ }
344
+
345
+ if (header->grain_size != 8) {
346
+ error_setg(errp, "Unsupported grain size: %" PRIu64,
347
+ header->grain_size);
348
+ return -ENOTSUP;
349
+ }
350
+
351
+ if (header->grain_table_size != 64) {
352
+ error_setg(errp, "Unsupported grain table size: %" PRIu64,
353
+ header->grain_table_size);
354
+ return -ENOTSUP;
355
+ }
356
+
357
+ if (header->flags != 0) {
358
+ error_setg(errp, "Unsupported flags: 0x%016" PRIx64,
359
+ header->flags);
360
+ return -ENOTSUP;
361
+ }
362
+
363
+ if (header->reserved1 != 0 || header->reserved2 != 0 ||
364
+ header->reserved3 != 0 || header->reserved4 != 0) {
365
+ error_setg(errp, "Unsupported reserved bits:"
366
+ " 0x%016" PRIx64 " 0x%016" PRIx64
367
+ " 0x%016" PRIx64 " 0x%016" PRIx64,
368
+ header->reserved1, header->reserved2,
369
+ header->reserved3, header->reserved4);
370
+ return -ENOTSUP;
371
+ }
372
+
373
+ /* check that padding is 0 */
374
+ if (!buffer_is_zero(header->pad, sizeof(header->pad))) {
375
+ error_setg(errp, "Unsupported non-zero const header padding");
376
+ return -ENOTSUP;
377
+ }
378
+
379
+ return 0;
187
+}
380
+}
188
+
381
+
189
+static XDbgBlockGraph *xdbg_graph_finalize(XDbgBlockGraphConstructor *gr)
382
+static int check_se_sparse_volatile_header(VMDKSESparseVolatileHeader *header,
383
+ Error **errp)
190
+{
384
+{
191
+ XDbgBlockGraph *graph = gr->graph;
385
+ header->magic = le64_to_cpu(header->magic);
192
+
386
+ header->free_gt_number = le64_to_cpu(header->free_gt_number);
193
+ g_hash_table_destroy(gr->graph_nodes);
387
+ header->next_txn_seq_number = le64_to_cpu(header->next_txn_seq_number);
194
+ g_free(gr);
388
+ header->replay_journal = le64_to_cpu(header->replay_journal);
195
+
389
+
196
+ return graph;
390
+ if (header->magic != SESPARSE_VOLATILE_HEADER_MAGIC) {
391
+ error_setg(errp, "Bad volatile header magic: 0x%016" PRIx64,
392
+ header->magic);
393
+ return -EINVAL;
394
+ }
395
+
396
+ if (header->replay_journal) {
397
+ error_setg(errp, "Image is dirty, Replaying journal not supported");
398
+ return -ENOTSUP;
399
+ }
400
+
401
+ /* check that padding is 0 */
402
+ if (!buffer_is_zero(header->pad, sizeof(header->pad))) {
403
+ error_setg(errp, "Unsupported non-zero volatile header padding");
404
+ return -ENOTSUP;
405
+ }
406
+
407
+ return 0;
197
+}
408
+}
198
+
409
+
199
+static uintptr_t xdbg_graph_node_num(XDbgBlockGraphConstructor *gr, void *node)
410
+static int vmdk_open_se_sparse(BlockDriverState *bs,
411
+ BdrvChild *file,
412
+ int flags, Error **errp)
200
+{
413
+{
201
+ uintptr_t ret = (uintptr_t)g_hash_table_lookup(gr->graph_nodes, node);
414
+ int ret;
202
+
415
+ VMDKSESparseConstHeader const_header;
203
+ if (ret != 0) {
416
+ VMDKSESparseVolatileHeader volatile_header;
417
+ VmdkExtent *extent;
418
+
419
+ ret = bdrv_apply_auto_read_only(bs,
420
+ "No write support for seSparse images available", errp);
421
+ if (ret < 0) {
204
+ return ret;
422
+ return ret;
205
+ }
423
+ }
206
+
424
+
207
+ /*
425
+ assert(sizeof(const_header) == SECTOR_SIZE);
208
+ * Start counting from 1, not 0, because 0 interferes with not-found (NULL)
426
+
209
+ * answer of g_hash_table_lookup.
427
+ ret = bdrv_pread(file, 0, &const_header, sizeof(const_header));
210
+ */
428
+ if (ret < 0) {
211
+ ret = g_hash_table_size(gr->graph_nodes) + 1;
429
+ bdrv_refresh_filename(file->bs);
212
+ g_hash_table_insert(gr->graph_nodes, node, (void *)ret);
430
+ error_setg_errno(errp, -ret,
431
+ "Could not read const header from file '%s'",
432
+ file->bs->filename);
433
+ return ret;
434
+ }
435
+
436
+ /* check const header */
437
+ ret = check_se_sparse_const_header(&const_header, errp);
438
+ if (ret < 0) {
439
+ return ret;
440
+ }
441
+
442
+ assert(sizeof(volatile_header) == SECTOR_SIZE);
443
+
444
+ ret = bdrv_pread(file,
445
+ const_header.volatile_header_offset * SECTOR_SIZE,
446
+ &volatile_header, sizeof(volatile_header));
447
+ if (ret < 0) {
448
+ bdrv_refresh_filename(file->bs);
449
+ error_setg_errno(errp, -ret,
450
+ "Could not read volatile header from file '%s'",
451
+ file->bs->filename);
452
+ return ret;
453
+ }
454
+
455
+ /* check volatile header */
456
+ ret = check_se_sparse_volatile_header(&volatile_header, errp);
457
+ if (ret < 0) {
458
+ return ret;
459
+ }
460
+
461
+ ret = vmdk_add_extent(bs, file, false,
462
+ const_header.capacity,
463
+ const_header.grain_dir_offset * SECTOR_SIZE,
464
+ 0,
465
+ const_header.grain_dir_size *
466
+ SECTOR_SIZE / sizeof(uint64_t),
467
+ const_header.grain_table_size *
468
+ SECTOR_SIZE / sizeof(uint64_t),
469
+ const_header.grain_size,
470
+ &extent,
471
+ errp);
472
+ if (ret < 0) {
473
+ return ret;
474
+ }
475
+
476
+ extent->sesparse = true;
477
+ extent->sesparse_l2_tables_offset = const_header.grain_tables_offset;
478
+ extent->sesparse_clusters_offset = const_header.grains_offset;
479
+ extent->entry_size = sizeof(uint64_t);
480
+
481
+ ret = vmdk_init_tables(bs, extent, errp);
482
+ if (ret) {
483
+ /* free extent allocated by vmdk_add_extent */
484
+ vmdk_free_last_extent(bs);
485
+ }
213
+
486
+
214
+ return ret;
487
+ return ret;
215
+}
488
+}
216
+
489
+
217
+static void xdbg_graph_add_node(XDbgBlockGraphConstructor *gr, void *node,
490
static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
218
+ XDbgBlockGraphNodeType type, const char *name)
491
QDict *options, Error **errp);
219
+{
492
220
+ XDbgBlockGraphNode *n;
493
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
221
+
494
* RW [size in sectors] SPARSE "file-name.vmdk"
222
+ n = g_new0(XDbgBlockGraphNode, 1);
495
* RW [size in sectors] VMFS "file-name.vmdk"
223
+
496
* RW [size in sectors] VMFSSPARSE "file-name.vmdk"
224
+ n->id = xdbg_graph_node_num(gr, node);
497
+ * RW [size in sectors] SESPARSE "file-name.vmdk"
225
+ n->type = type;
498
*/
226
+ n->name = g_strdup(name);
499
flat_offset = -1;
227
+
500
matches = sscanf(p, "%10s %" SCNd64 " %10s \"%511[^\n\r\"]\" %" SCNd64,
228
+ QAPI_LIST_ADD(gr->graph->nodes, n);
501
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
229
+}
502
230
+
503
if (sectors <= 0 ||
231
+static void xdbg_graph_add_edge(XDbgBlockGraphConstructor *gr, void *parent,
504
(strcmp(type, "FLAT") && strcmp(type, "SPARSE") &&
232
+ const BdrvChild *child)
505
- strcmp(type, "VMFS") && strcmp(type, "VMFSSPARSE")) ||
233
+{
506
+ strcmp(type, "VMFS") && strcmp(type, "VMFSSPARSE") &&
234
+ typedef struct {
507
+ strcmp(type, "SESPARSE")) ||
235
+ unsigned int flag;
508
(strcmp(access, "RW"))) {
236
+ BlockPermission num;
509
continue;
237
+ } PermissionMap;
510
}
238
+
511
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
239
+ static const PermissionMap permissions[] = {
512
return ret;
240
+ { BLK_PERM_CONSISTENT_READ, BLOCK_PERMISSION_CONSISTENT_READ },
513
}
241
+ { BLK_PERM_WRITE, BLOCK_PERMISSION_WRITE },
514
extent = &s->extents[s->num_extents - 1];
242
+ { BLK_PERM_WRITE_UNCHANGED, BLOCK_PERMISSION_WRITE_UNCHANGED },
515
+ } else if (!strcmp(type, "SESPARSE")) {
243
+ { BLK_PERM_RESIZE, BLOCK_PERMISSION_RESIZE },
516
+ ret = vmdk_open_se_sparse(bs, extent_file, bs->open_flags, errp);
244
+ { BLK_PERM_GRAPH_MOD, BLOCK_PERMISSION_GRAPH_MOD },
517
+ if (ret) {
245
+ { 0, 0 }
518
+ bdrv_unref_child(bs, extent_file);
246
+ };
519
+ return ret;
247
+ const PermissionMap *p;
520
+ }
248
+ XDbgBlockGraphEdge *edge;
521
+ extent = &s->extents[s->num_extents - 1];
249
+
522
} else {
250
+ QEMU_BUILD_BUG_ON(1UL << (ARRAY_SIZE(permissions) - 1) != BLK_PERM_ALL + 1);
523
error_setg(errp, "Unsupported extent type '%s'", type);
251
+
524
bdrv_unref_child(bs, extent_file);
252
+ edge = g_new0(XDbgBlockGraphEdge, 1);
525
@@ -XXX,XX +XXX,XX @@ static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
253
+
526
if (strcmp(ct, "monolithicFlat") &&
254
+ edge->parent = xdbg_graph_node_num(gr, parent);
527
strcmp(ct, "vmfs") &&
255
+ edge->child = xdbg_graph_node_num(gr, child->bs);
528
strcmp(ct, "vmfsSparse") &&
256
+ edge->name = g_strdup(child->name);
529
+ strcmp(ct, "seSparse") &&
257
+
530
strcmp(ct, "twoGbMaxExtentSparse") &&
258
+ for (p = permissions; p->flag; p++) {
531
strcmp(ct, "twoGbMaxExtentFlat")) {
259
+ if (p->flag & child->perm) {
532
error_setg(errp, "Unsupported image type '%s'", ct);
260
+ QAPI_LIST_ADD(edge->perm, p->num);
533
@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
534
{
535
unsigned int l1_index, l2_offset, l2_index;
536
int min_index, i, j;
537
- uint32_t min_count, *l2_table;
538
+ uint32_t min_count;
539
+ void *l2_table;
540
bool zeroed = false;
541
int64_t ret;
542
int64_t cluster_sector;
543
+ unsigned int l2_size_bytes = extent->l2_size * extent->entry_size;
544
545
if (m_data) {
546
m_data->valid = 0;
547
@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
548
if (l1_index >= extent->l1_size) {
549
return VMDK_ERROR;
550
}
551
- l2_offset = extent->l1_table[l1_index];
552
+ if (extent->sesparse) {
553
+ uint64_t l2_offset_u64;
554
+
555
+ assert(extent->entry_size == sizeof(uint64_t));
556
+
557
+ l2_offset_u64 = ((uint64_t *)extent->l1_table)[l1_index];
558
+ if (l2_offset_u64 == 0) {
559
+ l2_offset = 0;
560
+ } else if ((l2_offset_u64 & 0xffffffff00000000) != 0x1000000000000000) {
561
+ /*
562
+ * Top most nibble is 0x1 if grain table is allocated.
563
+ * strict check - top most 4 bytes must be 0x10000000 since max
564
+ * supported size is 64TB for disk - so no more than 64TB / 16MB
565
+ * grain directories which is smaller than uint32,
566
+ * where 16MB is the only supported default grain table coverage.
567
+ */
568
+ return VMDK_ERROR;
569
+ } else {
570
+ l2_offset_u64 = l2_offset_u64 & 0x00000000ffffffff;
571
+ l2_offset_u64 = extent->sesparse_l2_tables_offset +
572
+ l2_offset_u64 * l2_size_bytes / SECTOR_SIZE;
573
+ if (l2_offset_u64 > 0x00000000ffffffff) {
574
+ return VMDK_ERROR;
575
+ }
576
+ l2_offset = (unsigned int)(l2_offset_u64);
261
+ }
577
+ }
262
+ if (p->flag & child->shared_perm) {
578
+ } else {
263
+ QAPI_LIST_ADD(edge->shared_perm, p->num);
579
+ assert(extent->entry_size == sizeof(uint32_t));
580
+ l2_offset = ((uint32_t *)extent->l1_table)[l1_index];
581
+ }
582
if (!l2_offset) {
583
return VMDK_UNALLOC;
584
}
585
@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
586
extent->l2_cache_counts[j] >>= 1;
587
}
588
}
589
- l2_table = extent->l2_cache + (i * extent->l2_size);
590
+ l2_table = (char *)extent->l2_cache + (i * l2_size_bytes);
591
goto found;
592
}
593
}
594
@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
595
min_index = i;
596
}
597
}
598
- l2_table = extent->l2_cache + (min_index * extent->l2_size);
599
+ l2_table = (char *)extent->l2_cache + (min_index * l2_size_bytes);
600
BLKDBG_EVENT(extent->file, BLKDBG_L2_LOAD);
601
if (bdrv_pread(extent->file,
602
(int64_t)l2_offset * 512,
603
l2_table,
604
- extent->l2_size * sizeof(uint32_t)
605
- ) != extent->l2_size * sizeof(uint32_t)) {
606
+ l2_size_bytes
607
+ ) != l2_size_bytes) {
608
return VMDK_ERROR;
609
}
610
611
@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
612
extent->l2_cache_counts[min_index] = 1;
613
found:
614
l2_index = ((offset >> 9) / extent->cluster_sectors) % extent->l2_size;
615
- cluster_sector = le32_to_cpu(l2_table[l2_index]);
616
617
- if (extent->has_zero_grain && cluster_sector == VMDK_GTE_ZEROED) {
618
- zeroed = true;
619
+ if (extent->sesparse) {
620
+ cluster_sector = le64_to_cpu(((uint64_t *)l2_table)[l2_index]);
621
+ switch (cluster_sector & 0xf000000000000000) {
622
+ case 0x0000000000000000:
623
+ /* unallocated grain */
624
+ if (cluster_sector != 0) {
625
+ return VMDK_ERROR;
626
+ }
627
+ break;
628
+ case 0x1000000000000000:
629
+ /* scsi-unmapped grain - fallthrough */
630
+ case 0x2000000000000000:
631
+ /* zero grain */
632
+ zeroed = true;
633
+ break;
634
+ case 0x3000000000000000:
635
+ /* allocated grain */
636
+ cluster_sector = (((cluster_sector & 0x0fff000000000000) >> 48) |
637
+ ((cluster_sector & 0x0000ffffffffffff) << 12));
638
+ cluster_sector = extent->sesparse_clusters_offset +
639
+ cluster_sector * extent->cluster_sectors;
640
+ break;
641
+ default:
642
+ return VMDK_ERROR;
264
+ }
643
+ }
265
+ }
644
+ } else {
266
+
645
+ cluster_sector = le32_to_cpu(((uint32_t *)l2_table)[l2_index]);
267
+ QAPI_LIST_ADD(gr->graph->edges, edge);
646
+
268
+}
647
+ if (extent->has_zero_grain && cluster_sector == VMDK_GTE_ZEROED) {
269
+
648
+ zeroed = true;
270
+
271
+XDbgBlockGraph *bdrv_get_xdbg_block_graph(Error **errp)
272
+{
273
+ BlockBackend *blk;
274
+ BlockJob *job;
275
+ BlockDriverState *bs;
276
+ BdrvChild *child;
277
+ XDbgBlockGraphConstructor *gr = xdbg_graph_new();
278
+
279
+ for (blk = blk_all_next(NULL); blk; blk = blk_all_next(blk)) {
280
+ char *allocated_name = NULL;
281
+ const char *name = blk_name(blk);
282
+
283
+ if (!*name) {
284
+ name = allocated_name = blk_get_attached_dev_id(blk);
285
+ }
649
+ }
286
+ xdbg_graph_add_node(gr, blk, X_DBG_BLOCK_GRAPH_NODE_TYPE_BLOCK_BACKEND,
650
}
287
+ name);
651
288
+ g_free(allocated_name);
652
if (!cluster_sector || zeroed) {
289
+ if (blk_root(blk)) {
653
if (!allocate) {
290
+ xdbg_graph_add_edge(gr, blk, blk_root(blk));
654
return zeroed ? VMDK_ZEROED : VMDK_UNALLOC;
655
}
656
+ assert(!extent->sesparse);
657
658
if (extent->next_cluster_sector >= VMDK_EXTENT_MAX_SECTORS) {
659
return VMDK_ERROR;
660
@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
661
m_data->l1_index = l1_index;
662
m_data->l2_index = l2_index;
663
m_data->l2_offset = l2_offset;
664
- m_data->l2_cache_entry = &l2_table[l2_index];
665
+ m_data->l2_cache_entry = ((uint32_t *)l2_table) + l2_index;
666
}
667
}
668
*cluster_offset = cluster_sector << BDRV_SECTOR_BITS;
669
@@ -XXX,XX +XXX,XX @@ static int vmdk_pwritev(BlockDriverState *bs, uint64_t offset,
670
if (!extent) {
671
return -EIO;
672
}
673
+ if (extent->sesparse) {
674
+ return -ENOTSUP;
291
+ }
675
+ }
292
+ }
676
offset_in_cluster = vmdk_find_offset_in_cluster(extent, offset);
293
+
677
n_bytes = MIN(bytes, extent->cluster_sectors * BDRV_SECTOR_SIZE
294
+ for (job = block_job_next(NULL); job; job = block_job_next(job)) {
678
- offset_in_cluster);
295
+ GSList *el;
296
+
297
+ xdbg_graph_add_node(gr, job, X_DBG_BLOCK_GRAPH_NODE_TYPE_BLOCK_JOB,
298
+ job->job.id);
299
+ for (el = job->nodes; el; el = el->next) {
300
+ xdbg_graph_add_edge(gr, job, (BdrvChild *)el->data);
301
+ }
302
+ }
303
+
304
+ QTAILQ_FOREACH(bs, &graph_bdrv_states, node_list) {
305
+ xdbg_graph_add_node(gr, bs, X_DBG_BLOCK_GRAPH_NODE_TYPE_BLOCK_DRIVER,
306
+ bs->node_name);
307
+ QLIST_FOREACH(child, &bs->children, next) {
308
+ xdbg_graph_add_edge(gr, bs, child);
309
+ }
310
+ }
311
+
312
+ return xdbg_graph_finalize(gr);
313
+}
314
+
315
BlockDriverState *bdrv_lookup_bs(const char *device,
316
const char *node_name,
317
Error **errp)
318
diff --git a/block/block-backend.c b/block/block-backend.c
319
index XXXXXXX..XXXXXXX 100644
320
--- a/block/block-backend.c
321
+++ b/block/block-backend.c
322
@@ -XXX,XX +XXX,XX @@ int coroutine_fn blk_co_copy_range(BlockBackend *blk_in, int64_t off_in,
323
blk_out->root, off_out,
324
bytes, read_flags, write_flags);
325
}
326
+
327
+const BdrvChild *blk_root(BlockBackend *blk)
328
+{
329
+ return blk->root;
330
+}
331
diff --git a/blockdev.c b/blockdev.c
332
index XXXXXXX..XXXXXXX 100644
333
--- a/blockdev.c
334
+++ b/blockdev.c
335
@@ -XXX,XX +XXX,XX @@ BlockDeviceInfoList *qmp_query_named_block_nodes(Error **errp)
336
return bdrv_named_nodes_list(errp);
337
}
338
339
+XDbgBlockGraph *qmp_x_debug_query_block_graph(Error **errp)
340
+{
341
+ return bdrv_get_xdbg_block_graph(errp);
342
+}
343
+
344
BlockJob *do_blockdev_backup(BlockdevBackup *backup, JobTxn *txn,
345
Error **errp)
346
{
347
--
679
--
348
2.20.1
680
2.21.0
349
681
350
682
diff view generated by jsdifflib
1
From: Laurent Vivier <lvivier@redhat.com>
1
From: Pino Toscano <ptoscano@redhat.com>
2
2
3
Signed-off-by: Laurent Vivier <lvivier@redhat.com>
3
Rewrite the implementation of the ssh block driver to use libssh instead
4
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
4
of libssh2. The libssh library has various advantages over libssh2:
5
- easier API for authentication (for example for using ssh-agent)
6
- easier API for known_hosts handling
7
- supports newer types of keys in known_hosts
8
9
Use APIs/features available in libssh 0.8 conditionally, to support
10
older versions (which are not recommended though).
11
12
Adjust the iotest 207 according to the different error message, and to
13
find the default key type for localhost (to properly compare the
14
fingerprint with).
15
Contributed-by: Max Reitz <mreitz@redhat.com>
16
17
Adjust the various Docker/Travis scripts to use libssh when available
18
instead of libssh2. The mingw/mxe testing is dropped for now, as there
19
are no packages for it.
20
21
Signed-off-by: Pino Toscano <ptoscano@redhat.com>
22
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
23
Acked-by: Alex Bennée <alex.bennee@linaro.org>
24
Message-id: 20190620200840.17655-1-ptoscano@redhat.com
5
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
25
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
6
Message-id: 20181213162727.17438-2-lvivier@redhat.com
26
Message-id: 5873173.t2JhDm7DL7@lindworm.usersys.redhat.com
7
[mreitz: Fixed type of ssh_{read,write}_return's parameter to be ssize_t
8
instead of size_t]
9
Signed-off-by: Max Reitz <mreitz@redhat.com>
27
Signed-off-by: Max Reitz <mreitz@redhat.com>
10
---
28
---
11
block/ssh.c | 46 +++++++++++++++++-----------------------------
29
configure | 65 +-
12
block/trace-events | 17 +++++++++++++++++
30
block/Makefile.objs | 6 +-
13
2 files changed, 34 insertions(+), 29 deletions(-)
31
block/ssh.c | 652 ++++++++++--------
32
.travis.yml | 4 +-
33
block/trace-events | 14 +-
34
docs/qemu-block-drivers.texi | 2 +-
35
.../dockerfiles/debian-win32-cross.docker | 1 -
36
.../dockerfiles/debian-win64-cross.docker | 1 -
37
tests/docker/dockerfiles/fedora.docker | 4 +-
38
tests/docker/dockerfiles/ubuntu.docker | 2 +-
39
tests/docker/dockerfiles/ubuntu1804.docker | 2 +-
40
tests/qemu-iotests/207 | 54 +-
41
tests/qemu-iotests/207.out | 2 +-
42
13 files changed, 449 insertions(+), 360 deletions(-)
14
43
44
diff --git a/configure b/configure
45
index XXXXXXX..XXXXXXX 100755
46
--- a/configure
47
+++ b/configure
48
@@ -XXX,XX +XXX,XX @@ auth_pam=""
49
vte=""
50
virglrenderer=""
51
tpm=""
52
-libssh2=""
53
+libssh=""
54
live_block_migration="yes"
55
numa=""
56
tcmalloc="no"
57
@@ -XXX,XX +XXX,XX @@ for opt do
58
;;
59
--enable-tpm) tpm="yes"
60
;;
61
- --disable-libssh2) libssh2="no"
62
+ --disable-libssh) libssh="no"
63
;;
64
- --enable-libssh2) libssh2="yes"
65
+ --enable-libssh) libssh="yes"
66
;;
67
--disable-live-block-migration) live_block_migration="no"
68
;;
69
@@ -XXX,XX +XXX,XX @@ disabled with --disable-FEATURE, default is enabled if available:
70
coroutine-pool coroutine freelist (better performance)
71
glusterfs GlusterFS backend
72
tpm TPM support
73
- libssh2 ssh block device support
74
+ libssh ssh block device support
75
numa libnuma support
76
libxml2 for Parallels image format
77
tcmalloc tcmalloc support
78
@@ -XXX,XX +XXX,XX @@ EOF
79
fi
80
81
##########################################
82
-# libssh2 probe
83
-min_libssh2_version=1.2.8
84
-if test "$libssh2" != "no" ; then
85
- if $pkg_config --atleast-version=$min_libssh2_version libssh2; then
86
- libssh2_cflags=$($pkg_config libssh2 --cflags)
87
- libssh2_libs=$($pkg_config libssh2 --libs)
88
- libssh2=yes
89
+# libssh probe
90
+if test "$libssh" != "no" ; then
91
+ if $pkg_config --exists libssh; then
92
+ libssh_cflags=$($pkg_config libssh --cflags)
93
+ libssh_libs=$($pkg_config libssh --libs)
94
+ libssh=yes
95
else
96
- if test "$libssh2" = "yes" ; then
97
- error_exit "libssh2 >= $min_libssh2_version required for --enable-libssh2"
98
+ if test "$libssh" = "yes" ; then
99
+ error_exit "libssh required for --enable-libssh"
100
fi
101
- libssh2=no
102
+ libssh=no
103
fi
104
fi
105
106
##########################################
107
-# libssh2_sftp_fsync probe
108
+# Check for libssh 0.8
109
+# This is done like this instead of using the LIBSSH_VERSION_* and
110
+# SSH_VERSION_* macros because some distributions in the past shipped
111
+# snapshots of the future 0.8 from Git, and those snapshots did not
112
+# have updated version numbers (still referring to 0.7.0).
113
114
-if test "$libssh2" = "yes"; then
115
+if test "$libssh" = "yes"; then
116
cat > $TMPC <<EOF
117
-#include <stdio.h>
118
-#include <libssh2.h>
119
-#include <libssh2_sftp.h>
120
-int main(void) {
121
- LIBSSH2_SESSION *session;
122
- LIBSSH2_SFTP *sftp;
123
- LIBSSH2_SFTP_HANDLE *sftp_handle;
124
- session = libssh2_session_init ();
125
- sftp = libssh2_sftp_init (session);
126
- sftp_handle = libssh2_sftp_open (sftp, "/", 0, 0);
127
- libssh2_sftp_fsync (sftp_handle);
128
- return 0;
129
-}
130
+#include <libssh/libssh.h>
131
+int main(void) { return ssh_get_server_publickey(NULL, NULL); }
132
EOF
133
- # libssh2_cflags/libssh2_libs defined in previous test.
134
- if compile_prog "$libssh2_cflags" "$libssh2_libs" ; then
135
- QEMU_CFLAGS="-DHAS_LIBSSH2_SFTP_FSYNC $QEMU_CFLAGS"
136
+ if compile_prog "$libssh_cflags" "$libssh_libs"; then
137
+ libssh_cflags="-DHAVE_LIBSSH_0_8 $libssh_cflags"
138
fi
139
fi
140
141
@@ -XXX,XX +XXX,XX @@ echo "GlusterFS support $glusterfs"
142
echo "gcov $gcov_tool"
143
echo "gcov enabled $gcov"
144
echo "TPM support $tpm"
145
-echo "libssh2 support $libssh2"
146
+echo "libssh support $libssh"
147
echo "QOM debugging $qom_cast_debug"
148
echo "Live block migration $live_block_migration"
149
echo "lzo support $lzo"
150
@@ -XXX,XX +XXX,XX @@ if test "$glusterfs_iocb_has_stat" = "yes" ; then
151
echo "CONFIG_GLUSTERFS_IOCB_HAS_STAT=y" >> $config_host_mak
152
fi
153
154
-if test "$libssh2" = "yes" ; then
155
- echo "CONFIG_LIBSSH2=m" >> $config_host_mak
156
- echo "LIBSSH2_CFLAGS=$libssh2_cflags" >> $config_host_mak
157
- echo "LIBSSH2_LIBS=$libssh2_libs" >> $config_host_mak
158
+if test "$libssh" = "yes" ; then
159
+ echo "CONFIG_LIBSSH=m" >> $config_host_mak
160
+ echo "LIBSSH_CFLAGS=$libssh_cflags" >> $config_host_mak
161
+ echo "LIBSSH_LIBS=$libssh_libs" >> $config_host_mak
162
fi
163
164
if test "$live_block_migration" = "yes" ; then
165
diff --git a/block/Makefile.objs b/block/Makefile.objs
166
index XXXXXXX..XXXXXXX 100644
167
--- a/block/Makefile.objs
168
+++ b/block/Makefile.objs
169
@@ -XXX,XX +XXX,XX @@ block-obj-$(CONFIG_CURL) += curl.o
170
block-obj-$(CONFIG_RBD) += rbd.o
171
block-obj-$(CONFIG_GLUSTERFS) += gluster.o
172
block-obj-$(CONFIG_VXHS) += vxhs.o
173
-block-obj-$(CONFIG_LIBSSH2) += ssh.o
174
+block-obj-$(CONFIG_LIBSSH) += ssh.o
175
block-obj-y += accounting.o dirty-bitmap.o
176
block-obj-y += write-threshold.o
177
block-obj-y += backup.o
178
@@ -XXX,XX +XXX,XX @@ rbd.o-libs := $(RBD_LIBS)
179
gluster.o-cflags := $(GLUSTERFS_CFLAGS)
180
gluster.o-libs := $(GLUSTERFS_LIBS)
181
vxhs.o-libs := $(VXHS_LIBS)
182
-ssh.o-cflags := $(LIBSSH2_CFLAGS)
183
-ssh.o-libs := $(LIBSSH2_LIBS)
184
+ssh.o-cflags := $(LIBSSH_CFLAGS)
185
+ssh.o-libs := $(LIBSSH_LIBS)
186
block-obj-dmg-bz2-$(CONFIG_BZIP2) += dmg-bz2.o
187
block-obj-$(if $(CONFIG_DMG),m,n) += $(block-obj-dmg-bz2-y)
188
dmg-bz2.o-libs := $(BZIP2_LIBS)
15
diff --git a/block/ssh.c b/block/ssh.c
189
diff --git a/block/ssh.c b/block/ssh.c
16
index XXXXXXX..XXXXXXX 100644
190
index XXXXXXX..XXXXXXX 100644
17
--- a/block/ssh.c
191
--- a/block/ssh.c
18
+++ b/block/ssh.c
192
+++ b/block/ssh.c
19
@@ -XXX,XX +XXX,XX @@
193
@@ -XXX,XX +XXX,XX @@
20
#include "qapi/qmp/qstring.h"
194
21
#include "qapi/qobject-input-visitor.h"
195
#include "qemu/osdep.h"
22
#include "qapi/qobject-output-visitor.h"
196
23
+#include "trace.h"
197
-#include <libssh2.h>
24
198
-#include <libssh2_sftp.h>
25
-/* DEBUG_SSH=1 enables the DPRINTF (debugging printf) statements in
199
+#include <libssh/libssh.h>
26
- * this block driver code.
200
+#include <libssh/sftp.h>
27
- *
201
28
+/*
202
#include "block/block_int.h"
29
* TRACE_LIBSSH2=<bitmask> enables tracing in libssh2 itself. Note
203
#include "block/qdict.h"
30
* that this requires that libssh2 was specially compiled with the
204
@@ -XXX,XX +XXX,XX @@
31
* `./configure --enable-debug' option, so most likely you will have
205
#include "trace.h"
32
* to compile it yourself. The meaning of <bitmask> is described
206
33
* here: http://www.libssh2.org/libssh2_trace.html
207
/*
208
- * TRACE_LIBSSH2=<bitmask> enables tracing in libssh2 itself. Note
209
- * that this requires that libssh2 was specially compiled with the
210
- * `./configure --enable-debug' option, so most likely you will have
211
- * to compile it yourself. The meaning of <bitmask> is described
212
- * here: http://www.libssh2.org/libssh2_trace.html
213
+ * TRACE_LIBSSH=<level> enables tracing in libssh itself.
214
+ * The meaning of <level> is described here:
215
+ * http://api.libssh.org/master/group__libssh__log.html
34
*/
216
*/
35
-#define DEBUG_SSH 0
217
-#define TRACE_LIBSSH2 0 /* or try: LIBSSH2_TRACE_SFTP */
36
#define TRACE_LIBSSH2 0 /* or try: LIBSSH2_TRACE_SFTP */
218
+#define TRACE_LIBSSH 0 /* see: SSH_LOG_* */
37
219
38
-#define DPRINTF(fmt, ...) \
39
- do { \
40
- if (DEBUG_SSH) { \
41
- fprintf(stderr, "ssh: %-15s " fmt "\n", \
42
- __func__, ##__VA_ARGS__); \
43
- } \
44
- } while (0)
45
-
46
typedef struct BDRVSSHState {
220
typedef struct BDRVSSHState {
47
/* Coroutine. */
221
/* Coroutine. */
48
CoMutex lock;
222
@@ -XXX,XX +XXX,XX @@ typedef struct BDRVSSHState {
49
@@ -XXX,XX +XXX,XX @@ static int check_host_key_knownhosts(BDRVSSHState *s,
223
50
switch (r) {
224
/* SSH connection. */
51
case LIBSSH2_KNOWNHOST_CHECK_MATCH:
225
int sock; /* socket */
226
- LIBSSH2_SESSION *session; /* ssh session */
227
- LIBSSH2_SFTP *sftp; /* sftp session */
228
- LIBSSH2_SFTP_HANDLE *sftp_handle; /* sftp remote file handle */
229
+ ssh_session session; /* ssh session */
230
+ sftp_session sftp; /* sftp session */
231
+ sftp_file sftp_handle; /* sftp remote file handle */
232
233
- /* See ssh_seek() function below. */
234
- int64_t offset;
235
- bool offset_op_read;
236
-
237
- /* File attributes at open. We try to keep the .filesize field
238
+ /*
239
+ * File attributes at open. We try to keep the .size field
240
* updated if it changes (eg by writing at the end of the file).
241
*/
242
- LIBSSH2_SFTP_ATTRIBUTES attrs;
243
+ sftp_attributes attrs;
244
245
InetSocketAddress *inet;
246
247
@@ -XXX,XX +XXX,XX @@ static void ssh_state_init(BDRVSSHState *s)
248
{
249
memset(s, 0, sizeof *s);
250
s->sock = -1;
251
- s->offset = -1;
252
qemu_co_mutex_init(&s->lock);
253
}
254
255
@@ -XXX,XX +XXX,XX @@ static void ssh_state_free(BDRVSSHState *s)
256
{
257
g_free(s->user);
258
259
+ if (s->attrs) {
260
+ sftp_attributes_free(s->attrs);
261
+ }
262
if (s->sftp_handle) {
263
- libssh2_sftp_close(s->sftp_handle);
264
+ sftp_close(s->sftp_handle);
265
}
266
if (s->sftp) {
267
- libssh2_sftp_shutdown(s->sftp);
268
+ sftp_free(s->sftp);
269
}
270
if (s->session) {
271
- libssh2_session_disconnect(s->session,
272
- "from qemu ssh client: "
273
- "user closed the connection");
274
- libssh2_session_free(s->session);
275
- }
276
- if (s->sock >= 0) {
277
- close(s->sock);
278
+ ssh_disconnect(s->session);
279
+ ssh_free(s->session); /* This frees s->sock */
280
}
281
}
282
283
@@ -XXX,XX +XXX,XX @@ session_error_setg(Error **errp, BDRVSSHState *s, const char *fs, ...)
284
va_end(args);
285
286
if (s->session) {
287
- char *ssh_err;
288
+ const char *ssh_err;
289
int ssh_err_code;
290
291
- /* This is not an errno. See <libssh2.h>. */
292
- ssh_err_code = libssh2_session_last_error(s->session,
293
- &ssh_err, NULL, 0);
294
- error_setg(errp, "%s: %s (libssh2 error code: %d)",
295
+ /* This is not an errno. See <libssh/libssh.h>. */
296
+ ssh_err = ssh_get_error(s->session);
297
+ ssh_err_code = ssh_get_error_code(s->session);
298
+ error_setg(errp, "%s: %s (libssh error code: %d)",
299
msg, ssh_err, ssh_err_code);
300
} else {
301
error_setg(errp, "%s", msg);
302
@@ -XXX,XX +XXX,XX @@ sftp_error_setg(Error **errp, BDRVSSHState *s, const char *fs, ...)
303
va_end(args);
304
305
if (s->sftp) {
306
- char *ssh_err;
307
+ const char *ssh_err;
308
int ssh_err_code;
309
- unsigned long sftp_err_code;
310
+ int sftp_err_code;
311
312
- /* This is not an errno. See <libssh2.h>. */
313
- ssh_err_code = libssh2_session_last_error(s->session,
314
- &ssh_err, NULL, 0);
315
- /* See <libssh2_sftp.h>. */
316
- sftp_err_code = libssh2_sftp_last_error((s)->sftp);
317
+ /* This is not an errno. See <libssh/libssh.h>. */
318
+ ssh_err = ssh_get_error(s->session);
319
+ ssh_err_code = ssh_get_error_code(s->session);
320
+ /* See <libssh/sftp.h>. */
321
+ sftp_err_code = sftp_get_error(s->sftp);
322
323
error_setg(errp,
324
- "%s: %s (libssh2 error code: %d, sftp error code: %lu)",
325
+ "%s: %s (libssh error code: %d, sftp error code: %d)",
326
msg, ssh_err, ssh_err_code, sftp_err_code);
327
} else {
328
error_setg(errp, "%s", msg);
329
@@ -XXX,XX +XXX,XX @@ sftp_error_setg(Error **errp, BDRVSSHState *s, const char *fs, ...)
330
331
static void sftp_error_trace(BDRVSSHState *s, const char *op)
332
{
333
- char *ssh_err;
334
+ const char *ssh_err;
335
int ssh_err_code;
336
- unsigned long sftp_err_code;
337
+ int sftp_err_code;
338
339
- /* This is not an errno. See <libssh2.h>. */
340
- ssh_err_code = libssh2_session_last_error(s->session,
341
- &ssh_err, NULL, 0);
342
- /* See <libssh2_sftp.h>. */
343
- sftp_err_code = libssh2_sftp_last_error((s)->sftp);
344
+ /* This is not an errno. See <libssh/libssh.h>. */
345
+ ssh_err = ssh_get_error(s->session);
346
+ ssh_err_code = ssh_get_error_code(s->session);
347
+ /* See <libssh/sftp.h>. */
348
+ sftp_err_code = sftp_get_error(s->sftp);
349
350
trace_sftp_error(op, ssh_err, ssh_err_code, sftp_err_code);
351
}
352
@@ -XXX,XX +XXX,XX @@ static void ssh_parse_filename(const char *filename, QDict *options,
353
parse_uri(filename, options, errp);
354
}
355
356
-static int check_host_key_knownhosts(BDRVSSHState *s,
357
- const char *host, int port, Error **errp)
358
+static int check_host_key_knownhosts(BDRVSSHState *s, Error **errp)
359
{
360
- const char *home;
361
- char *knh_file = NULL;
362
- LIBSSH2_KNOWNHOSTS *knh = NULL;
363
- struct libssh2_knownhost *found;
364
- int ret, r;
365
- const char *hostkey;
366
- size_t len;
367
- int type;
368
-
369
- hostkey = libssh2_session_hostkey(s->session, &len, &type);
370
- if (!hostkey) {
371
+ int ret;
372
+#ifdef HAVE_LIBSSH_0_8
373
+ enum ssh_known_hosts_e state;
374
+ int r;
375
+ ssh_key pubkey;
376
+ enum ssh_keytypes_e pubkey_type;
377
+ unsigned char *server_hash = NULL;
378
+ size_t server_hash_len;
379
+ char *fingerprint = NULL;
380
+
381
+ state = ssh_session_is_known_server(s->session);
382
+ trace_ssh_server_status(state);
383
+
384
+ switch (state) {
385
+ case SSH_KNOWN_HOSTS_OK:
386
+ /* OK */
387
+ trace_ssh_check_host_key_knownhosts();
388
+ break;
389
+ case SSH_KNOWN_HOSTS_CHANGED:
390
ret = -EINVAL;
391
- session_error_setg(errp, s, "failed to read remote host key");
392
+ r = ssh_get_server_publickey(s->session, &pubkey);
393
+ if (r == 0) {
394
+ r = ssh_get_publickey_hash(pubkey, SSH_PUBLICKEY_HASH_SHA256,
395
+ &server_hash, &server_hash_len);
396
+ pubkey_type = ssh_key_type(pubkey);
397
+ ssh_key_free(pubkey);
398
+ }
399
+ if (r == 0) {
400
+ fingerprint = ssh_get_fingerprint_hash(SSH_PUBLICKEY_HASH_SHA256,
401
+ server_hash,
402
+ server_hash_len);
403
+ ssh_clean_pubkey_hash(&server_hash);
404
+ }
405
+ if (fingerprint) {
406
+ error_setg(errp,
407
+ "host key (%s key with fingerprint %s) does not match "
408
+ "the one in known_hosts; this may be a possible attack",
409
+ ssh_key_type_to_char(pubkey_type), fingerprint);
410
+ ssh_string_free_char(fingerprint);
411
+ } else {
412
+ error_setg(errp,
413
+ "host key does not match the one in known_hosts; this "
414
+ "may be a possible attack");
415
+ }
416
goto out;
417
- }
418
-
419
- knh = libssh2_knownhost_init(s->session);
420
- if (!knh) {
421
+ case SSH_KNOWN_HOSTS_OTHER:
422
ret = -EINVAL;
423
- session_error_setg(errp, s,
424
- "failed to initialize known hosts support");
425
+ error_setg(errp,
426
+ "host key for this server not found, another type exists");
427
+ goto out;
428
+ case SSH_KNOWN_HOSTS_UNKNOWN:
429
+ ret = -EINVAL;
430
+ error_setg(errp, "no host key was found in known_hosts");
431
+ goto out;
432
+ case SSH_KNOWN_HOSTS_NOT_FOUND:
433
+ ret = -ENOENT;
434
+ error_setg(errp, "known_hosts file not found");
435
+ goto out;
436
+ case SSH_KNOWN_HOSTS_ERROR:
437
+ ret = -EINVAL;
438
+ error_setg(errp, "error while checking the host");
439
+ goto out;
440
+ default:
441
+ ret = -EINVAL;
442
+ error_setg(errp, "error while checking for known server (%d)", state);
443
goto out;
444
}
445
+#else /* !HAVE_LIBSSH_0_8 */
446
+ int state;
447
448
- home = getenv("HOME");
449
- if (home) {
450
- knh_file = g_strdup_printf("%s/.ssh/known_hosts", home);
451
- } else {
452
- knh_file = g_strdup_printf("/root/.ssh/known_hosts");
453
- }
454
-
455
- /* Read all known hosts from OpenSSH-style known_hosts file. */
456
- libssh2_knownhost_readfile(knh, knh_file, LIBSSH2_KNOWNHOST_FILE_OPENSSH);
457
+ state = ssh_is_server_known(s->session);
458
+ trace_ssh_server_status(state);
459
460
- r = libssh2_knownhost_checkp(knh, host, port, hostkey, len,
461
- LIBSSH2_KNOWNHOST_TYPE_PLAIN|
462
- LIBSSH2_KNOWNHOST_KEYENC_RAW,
463
- &found);
464
- switch (r) {
465
- case LIBSSH2_KNOWNHOST_CHECK_MATCH:
466
+ switch (state) {
467
+ case SSH_SERVER_KNOWN_OK:
52
/* OK */
468
/* OK */
53
- DPRINTF("host key OK: %s", found->key);
469
- trace_ssh_check_host_key_knownhosts(found->key);
54
+ trace_ssh_check_host_key_knownhosts(found->key);
470
+ trace_ssh_check_host_key_knownhosts();
55
break;
471
break;
56
case LIBSSH2_KNOWNHOST_CHECK_MISMATCH:
472
- case LIBSSH2_KNOWNHOST_CHECK_MISMATCH:
473
+ case SSH_SERVER_KNOWN_CHANGED:
57
ret = -EINVAL;
474
ret = -EINVAL;
475
- session_error_setg(errp, s,
476
- "host key does not match the one in known_hosts"
477
- " (found key %s)", found->key);
478
+ error_setg(errp,
479
+ "host key does not match the one in known_hosts; this "
480
+ "may be a possible attack");
481
goto out;
482
- case LIBSSH2_KNOWNHOST_CHECK_NOTFOUND:
483
+ case SSH_SERVER_FOUND_OTHER:
484
ret = -EINVAL;
485
- session_error_setg(errp, s, "no host key was found in known_hosts");
486
+ error_setg(errp,
487
+ "host key for this server not found, another type exists");
488
+ goto out;
489
+ case SSH_SERVER_FILE_NOT_FOUND:
490
+ ret = -ENOENT;
491
+ error_setg(errp, "known_hosts file not found");
492
goto out;
493
- case LIBSSH2_KNOWNHOST_CHECK_FAILURE:
494
+ case SSH_SERVER_NOT_KNOWN:
495
ret = -EINVAL;
496
- session_error_setg(errp, s,
497
- "failure matching the host key with known_hosts");
498
+ error_setg(errp, "no host key was found in known_hosts");
499
+ goto out;
500
+ case SSH_SERVER_ERROR:
501
+ ret = -EINVAL;
502
+ error_setg(errp, "server error");
503
goto out;
504
default:
505
ret = -EINVAL;
506
- session_error_setg(errp, s, "unknown error matching the host key"
507
- " with known_hosts (%d)", r);
508
+ error_setg(errp, "error while checking for known server (%d)", state);
509
goto out;
510
}
511
+#endif /* !HAVE_LIBSSH_0_8 */
512
513
/* known_hosts checking successful. */
514
ret = 0;
515
516
out:
517
- if (knh != NULL) {
518
- libssh2_knownhost_free(knh);
519
- }
520
- g_free(knh_file);
521
return ret;
522
}
523
524
@@ -XXX,XX +XXX,XX @@ static int compare_fingerprint(const unsigned char *fingerprint, size_t len,
525
526
static int
527
check_host_key_hash(BDRVSSHState *s, const char *hash,
528
- int hash_type, size_t fingerprint_len, Error **errp)
529
+ enum ssh_publickey_hash_type type, Error **errp)
530
{
531
- const char *fingerprint;
532
-
533
- fingerprint = libssh2_hostkey_hash(s->session, hash_type);
534
- if (!fingerprint) {
535
+ int r;
536
+ ssh_key pubkey;
537
+ unsigned char *server_hash;
538
+ size_t server_hash_len;
539
+
540
+#ifdef HAVE_LIBSSH_0_8
541
+ r = ssh_get_server_publickey(s->session, &pubkey);
542
+#else
543
+ r = ssh_get_publickey(s->session, &pubkey);
544
+#endif
545
+ if (r != SSH_OK) {
546
session_error_setg(errp, s, "failed to read remote host key");
547
return -EINVAL;
548
}
549
550
- if(compare_fingerprint((unsigned char *) fingerprint, fingerprint_len,
551
- hash) != 0) {
552
+ r = ssh_get_publickey_hash(pubkey, type, &server_hash, &server_hash_len);
553
+ ssh_key_free(pubkey);
554
+ if (r != 0) {
555
+ session_error_setg(errp, s,
556
+ "failed reading the hash of the server SSH key");
557
+ return -EINVAL;
558
+ }
559
+
560
+ r = compare_fingerprint(server_hash, server_hash_len, hash);
561
+ ssh_clean_pubkey_hash(&server_hash);
562
+ if (r != 0) {
563
error_setg(errp, "remote host key does not match host_key_check '%s'",
564
hash);
565
return -EPERM;
566
@@ -XXX,XX +XXX,XX @@ check_host_key_hash(BDRVSSHState *s, const char *hash,
567
return 0;
568
}
569
570
-static int check_host_key(BDRVSSHState *s, const char *host, int port,
571
- SshHostKeyCheck *hkc, Error **errp)
572
+static int check_host_key(BDRVSSHState *s, SshHostKeyCheck *hkc, Error **errp)
573
{
574
SshHostKeyCheckMode mode;
575
576
@@ -XXX,XX +XXX,XX @@ static int check_host_key(BDRVSSHState *s, const char *host, int port,
577
case SSH_HOST_KEY_CHECK_MODE_HASH:
578
if (hkc->u.hash.type == SSH_HOST_KEY_CHECK_HASH_TYPE_MD5) {
579
return check_host_key_hash(s, hkc->u.hash.hash,
580
- LIBSSH2_HOSTKEY_HASH_MD5, 16, errp);
581
+ SSH_PUBLICKEY_HASH_MD5, errp);
582
} else if (hkc->u.hash.type == SSH_HOST_KEY_CHECK_HASH_TYPE_SHA1) {
583
return check_host_key_hash(s, hkc->u.hash.hash,
584
- LIBSSH2_HOSTKEY_HASH_SHA1, 20, errp);
585
+ SSH_PUBLICKEY_HASH_SHA1, errp);
586
}
587
g_assert_not_reached();
588
break;
589
case SSH_HOST_KEY_CHECK_MODE_KNOWN_HOSTS:
590
- return check_host_key_knownhosts(s, host, port, errp);
591
+ return check_host_key_knownhosts(s, errp);
592
default:
593
g_assert_not_reached();
594
}
595
@@ -XXX,XX +XXX,XX @@ static int check_host_key(BDRVSSHState *s, const char *host, int port,
596
return -EINVAL;
597
}
598
599
-static int authenticate(BDRVSSHState *s, const char *user, Error **errp)
600
+static int authenticate(BDRVSSHState *s, Error **errp)
601
{
602
int r, ret;
603
- const char *userauthlist;
604
- LIBSSH2_AGENT *agent = NULL;
605
- struct libssh2_agent_publickey *identity;
606
- struct libssh2_agent_publickey *prev_identity = NULL;
607
+ int method;
608
609
- userauthlist = libssh2_userauth_list(s->session, user, strlen(user));
610
- if (strstr(userauthlist, "publickey") == NULL) {
611
+ /* Try to authenticate with the "none" method. */
612
+ r = ssh_userauth_none(s->session, NULL);
613
+ if (r == SSH_AUTH_ERROR) {
614
ret = -EPERM;
615
- error_setg(errp,
616
- "remote server does not support \"publickey\" authentication");
617
+ session_error_setg(errp, s, "failed to authenticate using none "
618
+ "authentication");
619
goto out;
620
- }
621
-
622
- /* Connect to ssh-agent and try each identity in turn. */
623
- agent = libssh2_agent_init(s->session);
624
- if (!agent) {
625
- ret = -EINVAL;
626
- session_error_setg(errp, s, "failed to initialize ssh-agent support");
627
- goto out;
628
- }
629
- if (libssh2_agent_connect(agent)) {
630
- ret = -ECONNREFUSED;
631
- session_error_setg(errp, s, "failed to connect to ssh-agent");
632
- goto out;
633
- }
634
- if (libssh2_agent_list_identities(agent)) {
635
- ret = -EINVAL;
636
- session_error_setg(errp, s,
637
- "failed requesting identities from ssh-agent");
638
+ } else if (r == SSH_AUTH_SUCCESS) {
639
+ /* Authenticated! */
640
+ ret = 0;
641
goto out;
642
}
643
644
- for(;;) {
645
- r = libssh2_agent_get_identity(agent, &identity, prev_identity);
646
- if (r == 1) { /* end of list */
647
- break;
648
- }
649
- if (r < 0) {
650
+ method = ssh_userauth_list(s->session, NULL);
651
+ trace_ssh_auth_methods(method);
652
+
653
+ /*
654
+ * Try to authenticate with publickey, using the ssh-agent
655
+ * if available.
656
+ */
657
+ if (method & SSH_AUTH_METHOD_PUBLICKEY) {
658
+ r = ssh_userauth_publickey_auto(s->session, NULL, NULL);
659
+ if (r == SSH_AUTH_ERROR) {
660
ret = -EINVAL;
661
- session_error_setg(errp, s,
662
- "failed to obtain identity from ssh-agent");
663
+ session_error_setg(errp, s, "failed to authenticate using "
664
+ "publickey authentication");
665
goto out;
666
- }
667
- r = libssh2_agent_userauth(agent, user, identity);
668
- if (r == 0) {
669
+ } else if (r == SSH_AUTH_SUCCESS) {
670
/* Authenticated! */
671
ret = 0;
672
goto out;
673
}
674
- /* Failed to authenticate with this identity, try the next one. */
675
- prev_identity = identity;
676
}
677
678
ret = -EPERM;
679
@@ -XXX,XX +XXX,XX @@ static int authenticate(BDRVSSHState *s, const char *user, Error **errp)
680
"and the identities held by your ssh-agent");
681
682
out:
683
- if (agent != NULL) {
684
- /* Note: libssh2 implementation implicitly calls
685
- * libssh2_agent_disconnect if necessary.
686
- */
687
- libssh2_agent_free(agent);
688
- }
689
-
690
return ret;
691
}
692
58
@@ -XXX,XX +XXX,XX @@ static int connect_to_ssh(BDRVSSHState *s, BlockdevOptionsSsh *opts,
693
@@ -XXX,XX +XXX,XX @@ static int connect_to_ssh(BDRVSSHState *s, BlockdevOptionsSsh *opts,
59
}
694
int ssh_flags, int creat_mode, Error **errp)
695
{
696
int r, ret;
697
- long port = 0;
698
+ unsigned int port = 0;
699
+ int new_sock = -1;
700
701
if (opts->has_user) {
702
s->user = g_strdup(opts->user);
703
@@ -XXX,XX +XXX,XX @@ static int connect_to_ssh(BDRVSSHState *s, BlockdevOptionsSsh *opts,
704
s->inet = opts->server;
705
opts->server = NULL;
706
707
- if (qemu_strtol(s->inet->port, NULL, 10, &port) < 0) {
708
+ if (qemu_strtoui(s->inet->port, NULL, 10, &port) < 0) {
709
error_setg(errp, "Use only numeric port value");
710
ret = -EINVAL;
711
goto err;
712
}
713
714
/* Open the socket and connect. */
715
- s->sock = inet_connect_saddr(s->inet, errp);
716
- if (s->sock < 0) {
717
+ new_sock = inet_connect_saddr(s->inet, errp);
718
+ if (new_sock < 0) {
719
ret = -EIO;
720
goto err;
721
}
722
723
+ /*
724
+ * Try to disable the Nagle algorithm on TCP sockets to reduce latency,
725
+ * but do not fail if it cannot be disabled.
726
+ */
727
+ r = socket_set_nodelay(new_sock);
728
+ if (r < 0) {
729
+ warn_report("can't set TCP_NODELAY for the ssh server %s: %s",
730
+ s->inet->host, strerror(errno));
731
+ }
732
+
733
/* Create SSH session. */
734
- s->session = libssh2_session_init();
735
+ s->session = ssh_new();
736
if (!s->session) {
737
ret = -EINVAL;
738
- session_error_setg(errp, s, "failed to initialize libssh2 session");
739
+ session_error_setg(errp, s, "failed to initialize libssh session");
740
goto err;
741
}
742
743
-#if TRACE_LIBSSH2 != 0
744
- libssh2_trace(s->session, TRACE_LIBSSH2);
745
-#endif
746
+ /*
747
+ * Make sure we are in blocking mode during the connection and
748
+ * authentication phases.
749
+ */
750
+ ssh_set_blocking(s->session, 1);
751
752
- r = libssh2_session_handshake(s->session, s->sock);
753
- if (r != 0) {
754
+ r = ssh_options_set(s->session, SSH_OPTIONS_USER, s->user);
755
+ if (r < 0) {
756
+ ret = -EINVAL;
757
+ session_error_setg(errp, s,
758
+ "failed to set the user in the libssh session");
759
+ goto err;
760
+ }
761
+
762
+ r = ssh_options_set(s->session, SSH_OPTIONS_HOST, s->inet->host);
763
+ if (r < 0) {
764
+ ret = -EINVAL;
765
+ session_error_setg(errp, s,
766
+ "failed to set the host in the libssh session");
767
+ goto err;
768
+ }
769
+
770
+ if (port > 0) {
771
+ r = ssh_options_set(s->session, SSH_OPTIONS_PORT, &port);
772
+ if (r < 0) {
773
+ ret = -EINVAL;
774
+ session_error_setg(errp, s,
775
+ "failed to set the port in the libssh session");
776
+ goto err;
777
+ }
778
+ }
779
+
780
+ r = ssh_options_set(s->session, SSH_OPTIONS_COMPRESSION, "none");
781
+ if (r < 0) {
782
+ ret = -EINVAL;
783
+ session_error_setg(errp, s,
784
+ "failed to disable the compression in the libssh "
785
+ "session");
786
+ goto err;
787
+ }
788
+
789
+ /* Read ~/.ssh/config. */
790
+ r = ssh_options_parse_config(s->session, NULL);
791
+ if (r < 0) {
792
+ ret = -EINVAL;
793
+ session_error_setg(errp, s, "failed to parse ~/.ssh/config");
794
+ goto err;
795
+ }
796
+
797
+ r = ssh_options_set(s->session, SSH_OPTIONS_FD, &new_sock);
798
+ if (r < 0) {
799
+ ret = -EINVAL;
800
+ session_error_setg(errp, s,
801
+ "failed to set the socket in the libssh session");
802
+ goto err;
803
+ }
804
+ /* libssh took ownership of the socket. */
805
+ s->sock = new_sock;
806
+ new_sock = -1;
807
+
808
+ /* Connect. */
809
+ r = ssh_connect(s->session);
810
+ if (r != SSH_OK) {
811
ret = -EINVAL;
812
session_error_setg(errp, s, "failed to establish SSH session");
813
goto err;
814
}
815
816
/* Check the remote host's key against known_hosts. */
817
- ret = check_host_key(s, s->inet->host, port, opts->host_key_check, errp);
818
+ ret = check_host_key(s, opts->host_key_check, errp);
819
if (ret < 0) {
820
goto err;
821
}
822
823
/* Authenticate. */
824
- ret = authenticate(s, s->user, errp);
825
+ ret = authenticate(s, errp);
826
if (ret < 0) {
827
goto err;
828
}
829
830
/* Start SFTP. */
831
- s->sftp = libssh2_sftp_init(s->session);
832
+ s->sftp = sftp_new(s->session);
833
if (!s->sftp) {
834
- session_error_setg(errp, s, "failed to initialize sftp handle");
835
+ session_error_setg(errp, s, "failed to create sftp handle");
836
+ ret = -EINVAL;
837
+ goto err;
838
+ }
839
+
840
+ r = sftp_init(s->sftp);
841
+ if (r < 0) {
842
+ sftp_error_setg(errp, s, "failed to initialize sftp handle");
843
ret = -EINVAL;
844
goto err;
845
}
60
846
61
/* Open the remote file. */
847
/* Open the remote file. */
62
- DPRINTF("opening file %s flags=0x%x creat_mode=0%o",
848
trace_ssh_connect_to_ssh(opts->path, ssh_flags, creat_mode);
63
- opts->path, ssh_flags, creat_mode);
849
- s->sftp_handle = libssh2_sftp_open(s->sftp, opts->path, ssh_flags,
64
+ trace_ssh_connect_to_ssh(opts->path, ssh_flags, creat_mode);
850
- creat_mode);
65
s->sftp_handle = libssh2_sftp_open(s->sftp, opts->path, ssh_flags,
851
+ s->sftp_handle = sftp_open(s->sftp, opts->path, ssh_flags, creat_mode);
66
creat_mode);
67
if (!s->sftp_handle) {
852
if (!s->sftp_handle) {
68
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn ssh_co_create_opts(const char *filename, QemuOpts *opts,
853
- session_error_setg(errp, s, "failed to open remote file '%s'",
69
/* Get desired file size. */
854
- opts->path);
70
ssh_opts->size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
855
+ sftp_error_setg(errp, s, "failed to open remote file '%s'",
71
BDRV_SECTOR_SIZE);
856
+ opts->path);
72
- DPRINTF("total_size=%" PRIi64, ssh_opts->size);
857
ret = -EINVAL;
73
+ trace_ssh_co_create_opts(ssh_opts->size);
858
goto err;
74
859
}
75
uri_options = qdict_new();
860
76
ret = parse_uri(filename, uri_options, errp);
861
- r = libssh2_sftp_fstat(s->sftp_handle, &s->attrs);
77
@@ -XXX,XX +XXX,XX @@ static void restart_coroutine(void *opaque)
862
- if (r < 0) {
78
BDRVSSHState *s = bs->opaque;
863
+ /* Make sure the SFTP file is handled in blocking mode. */
79
AioContext *ctx = bdrv_get_aio_context(bs);
864
+ sftp_file_set_blocking(s->sftp_handle);
80
865
+
81
- DPRINTF("co=%p", restart->co);
866
+ s->attrs = sftp_fstat(s->sftp_handle);
82
+ trace_ssh_restart_coroutine(restart->co);
867
+ if (!s->attrs) {
83
aio_set_fd_handler(ctx, s->sock, false, NULL, NULL, NULL, NULL);
868
sftp_error_setg(errp, s, "failed to read file attributes");
84
869
return -EINVAL;
85
aio_co_wake(restart->co);
870
}
871
@@ -XXX,XX +XXX,XX @@ static int connect_to_ssh(BDRVSSHState *s, BlockdevOptionsSsh *opts,
872
return 0;
873
874
err:
875
+ if (s->attrs) {
876
+ sftp_attributes_free(s->attrs);
877
+ }
878
+ s->attrs = NULL;
879
if (s->sftp_handle) {
880
- libssh2_sftp_close(s->sftp_handle);
881
+ sftp_close(s->sftp_handle);
882
}
883
s->sftp_handle = NULL;
884
if (s->sftp) {
885
- libssh2_sftp_shutdown(s->sftp);
886
+ sftp_free(s->sftp);
887
}
888
s->sftp = NULL;
889
if (s->session) {
890
- libssh2_session_disconnect(s->session,
891
- "from qemu ssh client: "
892
- "error opening connection");
893
- libssh2_session_free(s->session);
894
+ ssh_disconnect(s->session);
895
+ ssh_free(s->session);
896
}
897
s->session = NULL;
898
+ s->sock = -1;
899
+ if (new_sock >= 0) {
900
+ close(new_sock);
901
+ }
902
903
return ret;
904
}
905
@@ -XXX,XX +XXX,XX @@ static int ssh_file_open(BlockDriverState *bs, QDict *options, int bdrv_flags,
906
907
ssh_state_init(s);
908
909
- ssh_flags = LIBSSH2_FXF_READ;
910
+ ssh_flags = 0;
911
if (bdrv_flags & BDRV_O_RDWR) {
912
- ssh_flags |= LIBSSH2_FXF_WRITE;
913
+ ssh_flags |= O_RDWR;
914
+ } else {
915
+ ssh_flags |= O_RDONLY;
916
}
917
918
opts = ssh_parse_options(options, errp);
919
@@ -XXX,XX +XXX,XX @@ static int ssh_file_open(BlockDriverState *bs, QDict *options, int bdrv_flags,
920
}
921
922
/* Go non-blocking. */
923
- libssh2_session_set_blocking(s->session, 0);
924
+ ssh_set_blocking(s->session, 0);
925
926
qapi_free_BlockdevOptionsSsh(opts);
927
928
return 0;
929
930
err:
931
- if (s->sock >= 0) {
932
- close(s->sock);
933
- }
934
- s->sock = -1;
935
-
936
qapi_free_BlockdevOptionsSsh(opts);
937
938
return ret;
939
@@ -XXX,XX +XXX,XX @@ static int ssh_grow_file(BDRVSSHState *s, int64_t offset, Error **errp)
940
{
941
ssize_t ret;
942
char c[1] = { '\0' };
943
- int was_blocking = libssh2_session_get_blocking(s->session);
944
+ int was_blocking = ssh_is_blocking(s->session);
945
946
/* offset must be strictly greater than the current size so we do
947
* not overwrite anything */
948
- assert(offset > 0 && offset > s->attrs.filesize);
949
+ assert(offset > 0 && offset > s->attrs->size);
950
951
- libssh2_session_set_blocking(s->session, 1);
952
+ ssh_set_blocking(s->session, 1);
953
954
- libssh2_sftp_seek64(s->sftp_handle, offset - 1);
955
- ret = libssh2_sftp_write(s->sftp_handle, c, 1);
956
+ sftp_seek64(s->sftp_handle, offset - 1);
957
+ ret = sftp_write(s->sftp_handle, c, 1);
958
959
- libssh2_session_set_blocking(s->session, was_blocking);
960
+ ssh_set_blocking(s->session, was_blocking);
961
962
if (ret < 0) {
963
sftp_error_setg(errp, s, "Failed to grow file");
964
return -EIO;
965
}
966
967
- s->attrs.filesize = offset;
968
+ s->attrs->size = offset;
969
return 0;
970
}
971
972
@@ -XXX,XX +XXX,XX @@ static int ssh_co_create(BlockdevCreateOptions *options, Error **errp)
973
ssh_state_init(&s);
974
975
ret = connect_to_ssh(&s, opts->location,
976
- LIBSSH2_FXF_READ|LIBSSH2_FXF_WRITE|
977
- LIBSSH2_FXF_CREAT|LIBSSH2_FXF_TRUNC,
978
+ O_RDWR | O_CREAT | O_TRUNC,
979
0644, errp);
980
if (ret < 0) {
981
goto fail;
982
@@ -XXX,XX +XXX,XX @@ static int ssh_has_zero_init(BlockDriverState *bs)
983
/* Assume false, unless we can positively prove it's true. */
984
int has_zero_init = 0;
985
986
- if (s->attrs.flags & LIBSSH2_SFTP_ATTR_PERMISSIONS) {
987
- if (s->attrs.permissions & LIBSSH2_SFTP_S_IFREG) {
988
- has_zero_init = 1;
989
- }
990
+ if (s->attrs->type == SSH_FILEXFER_TYPE_REGULAR) {
991
+ has_zero_init = 1;
992
}
993
994
return has_zero_init;
86
@@ -XXX,XX +XXX,XX @@ static coroutine_fn void co_yield(BDRVSSHState *s, BlockDriverState *bs)
995
@@ -XXX,XX +XXX,XX @@ static coroutine_fn void co_yield(BDRVSSHState *s, BlockDriverState *bs)
996
.co = qemu_coroutine_self()
997
};
998
999
- r = libssh2_session_block_directions(s->session);
1000
+ r = ssh_get_poll_flags(s->session);
1001
1002
- if (r & LIBSSH2_SESSION_BLOCK_INBOUND) {
1003
+ if (r & SSH_READ_PENDING) {
1004
rd_handler = restart_coroutine;
1005
}
1006
- if (r & LIBSSH2_SESSION_BLOCK_OUTBOUND) {
1007
+ if (r & SSH_WRITE_PENDING) {
87
wr_handler = restart_coroutine;
1008
wr_handler = restart_coroutine;
88
}
1009
}
89
1010
90
- DPRINTF("s->sock=%d rd_handler=%p wr_handler=%p", s->sock,
1011
@@ -XXX,XX +XXX,XX @@ static coroutine_fn void co_yield(BDRVSSHState *s, BlockDriverState *bs)
91
- rd_handler, wr_handler);
1012
trace_ssh_co_yield_back(s->sock);
92
+ trace_ssh_co_yield(s->sock, rd_handler, wr_handler);
93
94
aio_set_fd_handler(bdrv_get_aio_context(bs), s->sock,
95
false, rd_handler, wr_handler, NULL, &restart);
96
qemu_coroutine_yield();
97
- DPRINTF("s->sock=%d - back", s->sock);
98
+ trace_ssh_co_yield_back(s->sock);
99
}
1013
}
100
1014
101
/* SFTP has a function `libssh2_sftp_seek64' which seeks to a position
1015
-/* SFTP has a function `libssh2_sftp_seek64' which seeks to a position
102
@@ -XXX,XX +XXX,XX @@ static void ssh_seek(BDRVSSHState *s, int64_t offset, int flags)
1016
- * in the remote file. Notice that it just updates a field in the
103
bool force = (flags & SSH_SEEK_FORCE) != 0;
1017
- * sftp_handle structure, so there is no network traffic and it cannot
104
1018
- * fail.
105
if (force || op_read != s->offset_op_read || offset != s->offset) {
1019
- *
106
- DPRINTF("seeking to offset=%" PRIi64, offset);
1020
- * However, `libssh2_sftp_seek64' does have a catastrophic effect on
107
+ trace_ssh_seek(offset);
1021
- * performance since it causes the handle to throw away all in-flight
108
libssh2_sftp_seek64(s->sftp_handle, offset);
1022
- * reads and buffered readahead data. Therefore this function tries
109
s->offset = offset;
1023
- * to be intelligent about when to call the underlying libssh2 function.
110
s->offset_op_read = op_read;
1024
- */
1025
-#define SSH_SEEK_WRITE 0
1026
-#define SSH_SEEK_READ 1
1027
-#define SSH_SEEK_FORCE 2
1028
-
1029
-static void ssh_seek(BDRVSSHState *s, int64_t offset, int flags)
1030
-{
1031
- bool op_read = (flags & SSH_SEEK_READ) != 0;
1032
- bool force = (flags & SSH_SEEK_FORCE) != 0;
1033
-
1034
- if (force || op_read != s->offset_op_read || offset != s->offset) {
1035
- trace_ssh_seek(offset);
1036
- libssh2_sftp_seek64(s->sftp_handle, offset);
1037
- s->offset = offset;
1038
- s->offset_op_read = op_read;
1039
- }
1040
-}
1041
-
1042
static coroutine_fn int ssh_read(BDRVSSHState *s, BlockDriverState *bs,
1043
int64_t offset, size_t size,
1044
QEMUIOVector *qiov)
111
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int ssh_read(BDRVSSHState *s, BlockDriverState *bs,
1045
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int ssh_read(BDRVSSHState *s, BlockDriverState *bs,
112
char *buf, *end_of_vec;
1046
113
struct iovec *i;
1047
trace_ssh_read(offset, size);
114
1048
115
- DPRINTF("offset=%" PRIi64 " size=%zu", offset, size);
1049
- ssh_seek(s, offset, SSH_SEEK_READ);
116
+ trace_ssh_read(offset, size);
1050
+ trace_ssh_seek(offset);
117
1051
+ sftp_seek64(s->sftp_handle, offset);
118
ssh_seek(s, offset, SSH_SEEK_READ);
1052
119
1053
/* This keeps track of the current iovec element ('i'), where we
1054
* will write to next ('buf'), and the end of the current iovec
120
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int ssh_read(BDRVSSHState *s, BlockDriverState *bs,
1055
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int ssh_read(BDRVSSHState *s, BlockDriverState *bs,
121
*/
1056
buf = i->iov_base;
1057
end_of_vec = i->iov_base + i->iov_len;
1058
1059
- /* libssh2 has a hard-coded limit of 2000 bytes per request,
1060
- * although it will also do readahead behind our backs. Therefore
1061
- * we may have to do repeated reads here until we have read 'size'
1062
- * bytes.
1063
- */
122
for (got = 0; got < size; ) {
1064
for (got = 0; got < size; ) {
1065
+ size_t request_read_size;
123
again:
1066
again:
124
- DPRINTF("sftp_read buf=%p size=%zu", buf, end_of_vec - buf);
1067
- trace_ssh_read_buf(buf, end_of_vec - buf);
125
+ trace_ssh_read_buf(buf, end_of_vec - buf);
1068
- r = libssh2_sftp_read(s->sftp_handle, buf, end_of_vec - buf);
126
r = libssh2_sftp_read(s->sftp_handle, buf, end_of_vec - buf);
1069
- trace_ssh_read_return(r);
127
- DPRINTF("sftp_read returned %zd", r);
1070
+ /*
128
+ trace_ssh_read_return(r);
1071
+ * The size of SFTP packets is limited to 32K bytes, so limit
129
1072
+ * the amount of data requested to 16K, as libssh currently
130
if (r == LIBSSH2_ERROR_EAGAIN || r == LIBSSH2_ERROR_TIMEOUT) {
1073
+ * does not handle multiple requests on its own.
1074
+ */
1075
+ request_read_size = MIN(end_of_vec - buf, 16384);
1076
+ trace_ssh_read_buf(buf, end_of_vec - buf, request_read_size);
1077
+ r = sftp_read(s->sftp_handle, buf, request_read_size);
1078
+ trace_ssh_read_return(r, sftp_get_error(s->sftp));
1079
1080
- if (r == LIBSSH2_ERROR_EAGAIN || r == LIBSSH2_ERROR_TIMEOUT) {
1081
+ if (r == SSH_AGAIN) {
131
co_yield(s, bs);
1082
co_yield(s, bs);
1083
goto again;
1084
}
1085
- if (r < 0) {
1086
- sftp_error_trace(s, "read");
1087
- s->offset = -1;
1088
- return -EIO;
1089
- }
1090
- if (r == 0) {
1091
+ if (r == SSH_EOF || (r == 0 && sftp_get_error(s->sftp) == SSH_FX_EOF)) {
1092
/* EOF: Short read so pad the buffer with zeroes and return it. */
1093
qemu_iovec_memset(qiov, got, 0, size - got);
1094
return 0;
1095
}
1096
+ if (r <= 0) {
1097
+ sftp_error_trace(s, "read");
1098
+ return -EIO;
1099
+ }
1100
1101
got += r;
1102
buf += r;
1103
- s->offset += r;
1104
if (buf >= end_of_vec && got < size) {
1105
i++;
1106
buf = i->iov_base;
132
@@ -XXX,XX +XXX,XX @@ static int ssh_write(BDRVSSHState *s, BlockDriverState *bs,
1107
@@ -XXX,XX +XXX,XX @@ static int ssh_write(BDRVSSHState *s, BlockDriverState *bs,
133
char *buf, *end_of_vec;
1108
134
struct iovec *i;
1109
trace_ssh_write(offset, size);
135
1110
136
- DPRINTF("offset=%" PRIi64 " size=%zu", offset, size);
1111
- ssh_seek(s, offset, SSH_SEEK_WRITE);
137
+ trace_ssh_write(offset, size);
1112
+ trace_ssh_seek(offset);
138
1113
+ sftp_seek64(s->sftp_handle, offset);
139
ssh_seek(s, offset, SSH_SEEK_WRITE);
1114
140
1115
/* This keeps track of the current iovec element ('i'), where we
1116
* will read from next ('buf'), and the end of the current iovec
141
@@ -XXX,XX +XXX,XX @@ static int ssh_write(BDRVSSHState *s, BlockDriverState *bs,
1117
@@ -XXX,XX +XXX,XX @@ static int ssh_write(BDRVSSHState *s, BlockDriverState *bs,
1118
end_of_vec = i->iov_base + i->iov_len;
142
1119
143
for (written = 0; written < size; ) {
1120
for (written = 0; written < size; ) {
1121
+ size_t request_write_size;
144
again:
1122
again:
145
- DPRINTF("sftp_write buf=%p size=%zu", buf, end_of_vec - buf);
1123
- trace_ssh_write_buf(buf, end_of_vec - buf);
146
+ trace_ssh_write_buf(buf, end_of_vec - buf);
1124
- r = libssh2_sftp_write(s->sftp_handle, buf, end_of_vec - buf);
147
r = libssh2_sftp_write(s->sftp_handle, buf, end_of_vec - buf);
1125
- trace_ssh_write_return(r);
148
- DPRINTF("sftp_write returned %zd", r);
1126
+ /*
149
+ trace_ssh_write_return(r);
1127
+ * Avoid too large data packets, as libssh currently does not
150
1128
+ * handle multiple requests on its own.
151
if (r == LIBSSH2_ERROR_EAGAIN || r == LIBSSH2_ERROR_TIMEOUT) {
1129
+ */
1130
+ request_write_size = MIN(end_of_vec - buf, 131072);
1131
+ trace_ssh_write_buf(buf, end_of_vec - buf, request_write_size);
1132
+ r = sftp_write(s->sftp_handle, buf, request_write_size);
1133
+ trace_ssh_write_return(r, sftp_get_error(s->sftp));
1134
1135
- if (r == LIBSSH2_ERROR_EAGAIN || r == LIBSSH2_ERROR_TIMEOUT) {
1136
+ if (r == SSH_AGAIN) {
152
co_yield(s, bs);
1137
co_yield(s, bs);
153
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int ssh_flush(BDRVSSHState *s, BlockDriverState *bs)
1138
goto again;
1139
}
1140
if (r < 0) {
1141
sftp_error_trace(s, "write");
1142
- s->offset = -1;
1143
return -EIO;
1144
}
1145
- /* The libssh2 API is very unclear about this. A comment in
1146
- * the code says "nothing was acked, and no EAGAIN was
1147
- * received!" which apparently means that no data got sent
1148
- * out, and the underlying channel didn't return any EAGAIN
1149
- * indication. I think this is a bug in either libssh2 or
1150
- * OpenSSH (server-side). In any case, forcing a seek (to
1151
- * discard libssh2 internal buffers), and then trying again
1152
- * works for me.
1153
- */
1154
- if (r == 0) {
1155
- ssh_seek(s, offset + written, SSH_SEEK_WRITE|SSH_SEEK_FORCE);
1156
- co_yield(s, bs);
1157
- goto again;
1158
- }
1159
1160
written += r;
1161
buf += r;
1162
- s->offset += r;
1163
if (buf >= end_of_vec && written < size) {
1164
i++;
1165
buf = i->iov_base;
1166
end_of_vec = i->iov_base + i->iov_len;
1167
}
1168
1169
- if (offset + written > s->attrs.filesize)
1170
- s->attrs.filesize = offset + written;
1171
+ if (offset + written > s->attrs->size) {
1172
+ s->attrs->size = offset + written;
1173
+ }
1174
}
1175
1176
return 0;
1177
@@ -XXX,XX +XXX,XX @@ static void unsafe_flush_warning(BDRVSSHState *s, const char *what)
1178
}
1179
}
1180
1181
-#ifdef HAS_LIBSSH2_SFTP_FSYNC
1182
+#ifdef HAVE_LIBSSH_0_8
1183
1184
static coroutine_fn int ssh_flush(BDRVSSHState *s, BlockDriverState *bs)
154
{
1185
{
155
int r;
1186
int r;
156
1187
157
- DPRINTF("fsync");
1188
trace_ssh_flush();
158
+ trace_ssh_flush();
1189
+
1190
+ if (!sftp_extension_supported(s->sftp, "fsync@openssh.com", "1")) {
1191
+ unsafe_flush_warning(s, "OpenSSH >= 6.3");
1192
+ return 0;
1193
+ }
159
again:
1194
again:
160
r = libssh2_sftp_fsync(s->sftp_handle);
1195
- r = libssh2_sftp_fsync(s->sftp_handle);
161
if (r == LIBSSH2_ERROR_EAGAIN || r == LIBSSH2_ERROR_TIMEOUT) {
1196
- if (r == LIBSSH2_ERROR_EAGAIN || r == LIBSSH2_ERROR_TIMEOUT) {
162
@@ -XXX,XX +XXX,XX @@ static int64_t ssh_getlength(BlockDriverState *bs)
1197
+ r = sftp_fsync(s->sftp_handle);
163
1198
+ if (r == SSH_AGAIN) {
164
/* Note we cannot make a libssh2 call here. */
1199
co_yield(s, bs);
165
length = (int64_t) s->attrs.filesize;
1200
goto again;
166
- DPRINTF("length=%" PRIi64, length);
1201
}
167
+ trace_ssh_getlength(length);
1202
- if (r == LIBSSH2_ERROR_SFTP_PROTOCOL &&
1203
- libssh2_sftp_last_error(s->sftp) == LIBSSH2_FX_OP_UNSUPPORTED) {
1204
- unsafe_flush_warning(s, "OpenSSH >= 6.3");
1205
- return 0;
1206
- }
1207
if (r < 0) {
1208
sftp_error_trace(s, "fsync");
1209
return -EIO;
1210
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int ssh_co_flush(BlockDriverState *bs)
1211
return ret;
1212
}
1213
1214
-#else /* !HAS_LIBSSH2_SFTP_FSYNC */
1215
+#else /* !HAVE_LIBSSH_0_8 */
1216
1217
static coroutine_fn int ssh_co_flush(BlockDriverState *bs)
1218
{
1219
BDRVSSHState *s = bs->opaque;
1220
1221
- unsafe_flush_warning(s, "libssh2 >= 1.4.4");
1222
+ unsafe_flush_warning(s, "libssh >= 0.8.0");
1223
return 0;
1224
}
1225
1226
-#endif /* !HAS_LIBSSH2_SFTP_FSYNC */
1227
+#endif /* !HAVE_LIBSSH_0_8 */
1228
1229
static int64_t ssh_getlength(BlockDriverState *bs)
1230
{
1231
BDRVSSHState *s = bs->opaque;
1232
int64_t length;
1233
1234
- /* Note we cannot make a libssh2 call here. */
1235
- length = (int64_t) s->attrs.filesize;
1236
+ /* Note we cannot make a libssh call here. */
1237
+ length = (int64_t) s->attrs->size;
1238
trace_ssh_getlength(length);
168
1239
169
return length;
1240
return length;
1241
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn ssh_co_truncate(BlockDriverState *bs, int64_t offset,
1242
return -ENOTSUP;
1243
}
1244
1245
- if (offset < s->attrs.filesize) {
1246
+ if (offset < s->attrs->size) {
1247
error_setg(errp, "ssh driver does not support shrinking files");
1248
return -ENOTSUP;
1249
}
1250
1251
- if (offset == s->attrs.filesize) {
1252
+ if (offset == s->attrs->size) {
1253
return 0;
1254
}
1255
1256
@@ -XXX,XX +XXX,XX @@ static void bdrv_ssh_init(void)
1257
{
1258
int r;
1259
1260
- r = libssh2_init(0);
1261
+ r = ssh_init();
1262
if (r != 0) {
1263
- fprintf(stderr, "libssh2 initialization failed, %d\n", r);
1264
+ fprintf(stderr, "libssh initialization failed, %d\n", r);
1265
exit(EXIT_FAILURE);
1266
}
1267
1268
+#if TRACE_LIBSSH != 0
1269
+ ssh_set_log_level(TRACE_LIBSSH);
1270
+#endif
1271
+
1272
bdrv_register(&bdrv_ssh);
170
}
1273
}
1274
1275
diff --git a/.travis.yml b/.travis.yml
1276
index XXXXXXX..XXXXXXX 100644
1277
--- a/.travis.yml
1278
+++ b/.travis.yml
1279
@@ -XXX,XX +XXX,XX @@ addons:
1280
- libseccomp-dev
1281
- libspice-protocol-dev
1282
- libspice-server-dev
1283
- - libssh2-1-dev
1284
+ - libssh-dev
1285
- liburcu-dev
1286
- libusb-1.0-0-dev
1287
- libvte-2.91-dev
1288
@@ -XXX,XX +XXX,XX @@ matrix:
1289
- libseccomp-dev
1290
- libspice-protocol-dev
1291
- libspice-server-dev
1292
- - libssh2-1-dev
1293
+ - libssh-dev
1294
- liburcu-dev
1295
- libusb-1.0-0-dev
1296
- libvte-2.91-dev
171
diff --git a/block/trace-events b/block/trace-events
1297
diff --git a/block/trace-events b/block/trace-events
172
index XXXXXXX..XXXXXXX 100644
1298
index XXXXXXX..XXXXXXX 100644
173
--- a/block/trace-events
1299
--- a/block/trace-events
174
+++ b/block/trace-events
1300
+++ b/block/trace-events
175
@@ -XXX,XX +XXX,XX @@ iscsi_xcopy(void *src_lun, uint64_t src_off, void *dst_lun, uint64_t dst_off, ui
1301
@@ -XXX,XX +XXX,XX @@ nbd_client_connect_success(const char *export_name) "export '%s'"
176
# block/nbd-client.c
1302
# ssh.c
177
nbd_read_reply_entry_fail(int ret, const char *err) "ret = %d, err: %s"
1303
ssh_restart_coroutine(void *co) "co=%p"
178
nbd_co_request_fail(uint64_t from, uint32_t len, uint64_t handle, uint16_t flags, uint16_t type, const char *name, int ret, const char *err) "Request failed { .from = %" PRIu64", .len = %" PRIu32 ", .handle = %" PRIu64 ", .flags = 0x%" PRIx16 ", .type = %" PRIu16 " (%s) } ret = %d, err: %s"
1304
ssh_flush(void) "fsync"
179
+
1305
-ssh_check_host_key_knownhosts(const char *key) "host key OK: %s"
180
+# block/ssh.c
1306
+ssh_check_host_key_knownhosts(void) "host key OK"
181
+ssh_restart_coroutine(void *co) "co=%p"
1307
ssh_connect_to_ssh(char *path, int flags, int mode) "opening file %s flags=0x%x creat_mode=0%o"
182
+ssh_flush(void) "fsync"
1308
ssh_co_yield(int sock, void *rd_handler, void *wr_handler) "s->sock=%d rd_handler=%p wr_handler=%p"
183
+ssh_check_host_key_knownhosts(const char *key) "host key OK: %s"
1309
ssh_co_yield_back(int sock) "s->sock=%d - back"
184
+ssh_connect_to_ssh(char *path, int flags, int mode) "opening file %s flags=0x%x creat_mode=0%o"
1310
ssh_getlength(int64_t length) "length=%" PRIi64
185
+ssh_co_yield(int sock, void *rd_handler, void *wr_handler) "s->sock=%d rd_handler=%p wr_handler=%p"
1311
ssh_co_create_opts(uint64_t size) "total_size=%" PRIu64
186
+ssh_co_yield_back(int sock) "s->sock=%d - back"
1312
ssh_read(int64_t offset, size_t size) "offset=%" PRIi64 " size=%zu"
187
+ssh_getlength(int64_t length) "length=%" PRIi64
1313
-ssh_read_buf(void *buf, size_t size) "sftp_read buf=%p size=%zu"
188
+ssh_co_create_opts(uint64_t size) "total_size=%" PRIu64
1314
-ssh_read_return(ssize_t ret) "sftp_read returned %zd"
189
+ssh_read(int64_t offset, size_t size) "offset=%" PRIi64 " size=%zu"
1315
+ssh_read_buf(void *buf, size_t size, size_t actual_size) "sftp_read buf=%p size=%zu (actual size=%zu)"
190
+ssh_read_buf(void *buf, size_t size) "sftp_read buf=%p size=%zu"
1316
+ssh_read_return(ssize_t ret, int sftp_err) "sftp_read returned %zd (sftp error=%d)"
191
+ssh_read_return(ssize_t ret) "sftp_read returned %zd"
1317
ssh_write(int64_t offset, size_t size) "offset=%" PRIi64 " size=%zu"
192
+ssh_write(int64_t offset, size_t size) "offset=%" PRIi64 " size=%zu"
1318
-ssh_write_buf(void *buf, size_t size) "sftp_write buf=%p size=%zu"
193
+ssh_write_buf(void *buf, size_t size) "sftp_write buf=%p size=%zu"
1319
-ssh_write_return(ssize_t ret) "sftp_write returned %zd"
194
+ssh_write_return(ssize_t ret) "sftp_write returned %zd"
1320
+ssh_write_buf(void *buf, size_t size, size_t actual_size) "sftp_write buf=%p size=%zu (actual size=%zu)"
195
+ssh_seek(int64_t offset) "seeking to offset=%" PRIi64
1321
+ssh_write_return(ssize_t ret, int sftp_err) "sftp_write returned %zd (sftp error=%d)"
1322
ssh_seek(int64_t offset) "seeking to offset=%" PRIi64
1323
+ssh_auth_methods(int methods) "auth methods=0x%x"
1324
+ssh_server_status(int status) "server status=%d"
1325
1326
# curl.c
1327
curl_timer_cb(long timeout_ms) "timer callback timeout_ms %ld"
1328
@@ -XXX,XX +XXX,XX @@ sheepdog_snapshot_create(const char *sn_name, const char *id) "%s %s"
1329
sheepdog_snapshot_create_inode(const char *name, uint32_t snap, uint32_t vdi) "s->inode: name %s snap_id 0x%" PRIx32 " vdi 0x%" PRIx32
1330
1331
# ssh.c
1332
-sftp_error(const char *op, const char *ssh_err, int ssh_err_code, unsigned long sftp_err_code) "%s failed: %s (libssh2 error code: %d, sftp error code: %lu)"
1333
+sftp_error(const char *op, const char *ssh_err, int ssh_err_code, int sftp_err_code) "%s failed: %s (libssh error code: %d, sftp error code: %d)"
1334
diff --git a/docs/qemu-block-drivers.texi b/docs/qemu-block-drivers.texi
1335
index XXXXXXX..XXXXXXX 100644
1336
--- a/docs/qemu-block-drivers.texi
1337
+++ b/docs/qemu-block-drivers.texi
1338
@@ -XXX,XX +XXX,XX @@ print a warning when @code{fsync} is not supported:
1339
1340
warning: ssh server @code{ssh.example.com:22} does not support fsync
1341
1342
-With sufficiently new versions of libssh2 and OpenSSH, @code{fsync} is
1343
+With sufficiently new versions of libssh and OpenSSH, @code{fsync} is
1344
supported.
1345
1346
@node disk_images_nvme
1347
diff --git a/tests/docker/dockerfiles/debian-win32-cross.docker b/tests/docker/dockerfiles/debian-win32-cross.docker
1348
index XXXXXXX..XXXXXXX 100644
1349
--- a/tests/docker/dockerfiles/debian-win32-cross.docker
1350
+++ b/tests/docker/dockerfiles/debian-win32-cross.docker
1351
@@ -XXX,XX +XXX,XX @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
1352
mxe-$TARGET-w64-mingw32.shared-curl \
1353
mxe-$TARGET-w64-mingw32.shared-glib \
1354
mxe-$TARGET-w64-mingw32.shared-libgcrypt \
1355
- mxe-$TARGET-w64-mingw32.shared-libssh2 \
1356
mxe-$TARGET-w64-mingw32.shared-libusb1 \
1357
mxe-$TARGET-w64-mingw32.shared-lzo \
1358
mxe-$TARGET-w64-mingw32.shared-nettle \
1359
diff --git a/tests/docker/dockerfiles/debian-win64-cross.docker b/tests/docker/dockerfiles/debian-win64-cross.docker
1360
index XXXXXXX..XXXXXXX 100644
1361
--- a/tests/docker/dockerfiles/debian-win64-cross.docker
1362
+++ b/tests/docker/dockerfiles/debian-win64-cross.docker
1363
@@ -XXX,XX +XXX,XX @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
1364
mxe-$TARGET-w64-mingw32.shared-curl \
1365
mxe-$TARGET-w64-mingw32.shared-glib \
1366
mxe-$TARGET-w64-mingw32.shared-libgcrypt \
1367
- mxe-$TARGET-w64-mingw32.shared-libssh2 \
1368
mxe-$TARGET-w64-mingw32.shared-libusb1 \
1369
mxe-$TARGET-w64-mingw32.shared-lzo \
1370
mxe-$TARGET-w64-mingw32.shared-nettle \
1371
diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
1372
index XXXXXXX..XXXXXXX 100644
1373
--- a/tests/docker/dockerfiles/fedora.docker
1374
+++ b/tests/docker/dockerfiles/fedora.docker
1375
@@ -XXX,XX +XXX,XX @@ ENV PACKAGES \
1376
libpng-devel \
1377
librbd-devel \
1378
libseccomp-devel \
1379
- libssh2-devel \
1380
+ libssh-devel \
1381
libubsan \
1382
libusbx-devel \
1383
libxml2-devel \
1384
@@ -XXX,XX +XXX,XX @@ ENV PACKAGES \
1385
mingw32-gtk3 \
1386
mingw32-libjpeg-turbo \
1387
mingw32-libpng \
1388
- mingw32-libssh2 \
1389
mingw32-libtasn1 \
1390
mingw32-nettle \
1391
mingw32-pixman \
1392
@@ -XXX,XX +XXX,XX @@ ENV PACKAGES \
1393
mingw64-gtk3 \
1394
mingw64-libjpeg-turbo \
1395
mingw64-libpng \
1396
- mingw64-libssh2 \
1397
mingw64-libtasn1 \
1398
mingw64-nettle \
1399
mingw64-pixman \
1400
diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
1401
index XXXXXXX..XXXXXXX 100644
1402
--- a/tests/docker/dockerfiles/ubuntu.docker
1403
+++ b/tests/docker/dockerfiles/ubuntu.docker
1404
@@ -XXX,XX +XXX,XX @@ ENV PACKAGES flex bison \
1405
libsnappy-dev \
1406
libspice-protocol-dev \
1407
libspice-server-dev \
1408
- libssh2-1-dev \
1409
+ libssh-dev \
1410
libusb-1.0-0-dev \
1411
libusbredirhost-dev \
1412
libvdeplug-dev \
1413
diff --git a/tests/docker/dockerfiles/ubuntu1804.docker b/tests/docker/dockerfiles/ubuntu1804.docker
1414
index XXXXXXX..XXXXXXX 100644
1415
--- a/tests/docker/dockerfiles/ubuntu1804.docker
1416
+++ b/tests/docker/dockerfiles/ubuntu1804.docker
1417
@@ -XXX,XX +XXX,XX @@ ENV PACKAGES flex bison \
1418
libsnappy-dev \
1419
libspice-protocol-dev \
1420
libspice-server-dev \
1421
- libssh2-1-dev \
1422
+ libssh-dev \
1423
libusb-1.0-0-dev \
1424
libusbredirhost-dev \
1425
libvdeplug-dev \
1426
diff --git a/tests/qemu-iotests/207 b/tests/qemu-iotests/207
1427
index XXXXXXX..XXXXXXX 100755
1428
--- a/tests/qemu-iotests/207
1429
+++ b/tests/qemu-iotests/207
1430
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.img') as disk_path, \
1431
1432
iotests.img_info_log(remote_path)
1433
1434
- md5_key = subprocess.check_output(
1435
- 'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
1436
- 'cut -d" " -f3 | base64 -d | md5sum -b | cut -d" " -f1',
1437
- shell=True).rstrip().decode('ascii')
1438
+ keys = subprocess.check_output(
1439
+ 'ssh-keyscan 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
1440
+ 'cut -d" " -f3',
1441
+ shell=True).rstrip().decode('ascii').split('\n')
1442
+
1443
+ # Mappings of base64 representations to digests
1444
+ md5_keys = {}
1445
+ sha1_keys = {}
1446
+
1447
+ for key in keys:
1448
+ md5_keys[key] = subprocess.check_output(
1449
+ 'echo %s | base64 -d | md5sum -b | cut -d" " -f1' % key,
1450
+ shell=True).rstrip().decode('ascii')
1451
+
1452
+ sha1_keys[key] = subprocess.check_output(
1453
+ 'echo %s | base64 -d | sha1sum -b | cut -d" " -f1' % key,
1454
+ shell=True).rstrip().decode('ascii')
1455
1456
vm.launch()
1457
+
1458
+ # Find correct key first
1459
+ matching_key = None
1460
+ for key in keys:
1461
+ result = vm.qmp('blockdev-add',
1462
+ driver='ssh', node_name='node0', path=disk_path,
1463
+ server={
1464
+ 'host': '127.0.0.1',
1465
+ 'port': '22',
1466
+ }, host_key_check={
1467
+ 'mode': 'hash',
1468
+ 'type': 'md5',
1469
+ 'hash': md5_keys[key],
1470
+ })
1471
+
1472
+ if 'error' not in result:
1473
+ vm.qmp('blockdev-del', node_name='node0')
1474
+ matching_key = key
1475
+ break
1476
+
1477
+ if matching_key is None:
1478
+ vm.shutdown()
1479
+ iotests.notrun('Did not find a key that fits 127.0.0.1')
1480
+
1481
blockdev_create(vm, { 'driver': 'ssh',
1482
'location': {
1483
'path': disk_path,
1484
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.img') as disk_path, \
1485
'host-key-check': {
1486
'mode': 'hash',
1487
'type': 'md5',
1488
- 'hash': md5_key,
1489
+ 'hash': md5_keys[matching_key],
1490
}
1491
},
1492
'size': 8388608 })
1493
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.img') as disk_path, \
1494
1495
iotests.img_info_log(remote_path)
1496
1497
- sha1_key = subprocess.check_output(
1498
- 'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
1499
- 'cut -d" " -f3 | base64 -d | sha1sum -b | cut -d" " -f1',
1500
- shell=True).rstrip().decode('ascii')
1501
-
1502
vm.launch()
1503
blockdev_create(vm, { 'driver': 'ssh',
1504
'location': {
1505
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.img') as disk_path, \
1506
'host-key-check': {
1507
'mode': 'hash',
1508
'type': 'sha1',
1509
- 'hash': sha1_key,
1510
+ 'hash': sha1_keys[matching_key],
1511
}
1512
},
1513
'size': 4194304 })
1514
diff --git a/tests/qemu-iotests/207.out b/tests/qemu-iotests/207.out
1515
index XXXXXXX..XXXXXXX 100644
1516
--- a/tests/qemu-iotests/207.out
1517
+++ b/tests/qemu-iotests/207.out
1518
@@ -XXX,XX +XXX,XX @@ virtual size: 4 MiB (4194304 bytes)
1519
1520
{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": {"driver": "ssh", "location": {"host-key-check": {"mode": "none"}, "path": "/this/is/not/an/existing/path", "server": {"host": "127.0.0.1", "port": "22"}}, "size": 4194304}}}
1521
{"return": {}}
1522
-Job failed: failed to open remote file '/this/is/not/an/existing/path': Failed opening remote file (libssh2 error code: -31)
1523
+Job failed: failed to open remote file '/this/is/not/an/existing/path': SFTP server: No such file (libssh error code: 1, sftp error code: 2)
1524
{"execute": "job-dismiss", "arguments": {"id": "job0"}}
1525
{"return": {}}
1526
196
--
1527
--
197
2.20.1
1528
2.21.0
198
1529
199
1530
diff view generated by jsdifflib
Deleted patch
1
From: Laurent Vivier <lvivier@redhat.com>
2
1
3
Signed-off-by: Laurent Vivier <lvivier@redhat.com>
4
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
5
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
6
Message-id: 20181213162727.17438-3-lvivier@redhat.com
7
Signed-off-by: Max Reitz <mreitz@redhat.com>
8
---
9
block/curl.c | 29 ++++++++---------------------
10
block/trace-events | 9 +++++++++
11
2 files changed, 17 insertions(+), 21 deletions(-)
12
13
diff --git a/block/curl.c b/block/curl.c
14
index XXXXXXX..XXXXXXX 100644
15
--- a/block/curl.c
16
+++ b/block/curl.c
17
@@ -XXX,XX +XXX,XX @@
18
#include "crypto/secret.h"
19
#include <curl/curl.h>
20
#include "qemu/cutils.h"
21
+#include "trace.h"
22
23
-// #define DEBUG_CURL
24
// #define DEBUG_VERBOSE
25
26
-#ifdef DEBUG_CURL
27
-#define DEBUG_CURL_PRINT 1
28
-#else
29
-#define DEBUG_CURL_PRINT 0
30
-#endif
31
-#define DPRINTF(fmt, ...) \
32
- do { \
33
- if (DEBUG_CURL_PRINT) { \
34
- fprintf(stderr, fmt, ## __VA_ARGS__); \
35
- } \
36
- } while (0)
37
-
38
#if LIBCURL_VERSION_NUM >= 0x071000
39
/* The multi interface timer callback was introduced in 7.16.0 */
40
#define NEED_CURL_TIMER_CALLBACK
41
@@ -XXX,XX +XXX,XX @@ static int curl_timer_cb(CURLM *multi, long timeout_ms, void *opaque)
42
{
43
BDRVCURLState *s = opaque;
44
45
- DPRINTF("CURL: timer callback timeout_ms %ld\n", timeout_ms);
46
+ trace_curl_timer_cb(timeout_ms);
47
if (timeout_ms == -1) {
48
timer_del(&s->timer);
49
} else {
50
@@ -XXX,XX +XXX,XX @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
51
}
52
socket = NULL;
53
54
- DPRINTF("CURL (AIO): Sock action %d on fd %d\n", action, (int)fd);
55
+ trace_curl_sock_cb(action, (int)fd);
56
switch (action) {
57
case CURL_POLL_IN:
58
aio_set_fd_handler(s->aio_context, fd, false,
59
@@ -XXX,XX +XXX,XX @@ static size_t curl_read_cb(void *ptr, size_t size, size_t nmemb, void *opaque)
60
size_t realsize = size * nmemb;
61
int i;
62
63
- DPRINTF("CURL: Just reading %zd bytes\n", realsize);
64
+ trace_curl_read_cb(realsize);
65
66
if (!s || !s->orig_buf) {
67
goto read_end;
68
@@ -XXX,XX +XXX,XX @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
69
}
70
}
71
72
- DPRINTF("CURL: Opening %s\n", file);
73
+ trace_curl_open(file);
74
qemu_co_queue_init(&s->free_state_waitq);
75
s->aio_context = bdrv_get_aio_context(bs);
76
s->url = g_strdup(file);
77
@@ -XXX,XX +XXX,XX @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
78
"Server does not support 'range' (byte ranges).");
79
goto out;
80
}
81
- DPRINTF("CURL: Size = %" PRIu64 "\n", s->len);
82
+ trace_curl_open_size(s->len);
83
84
qemu_mutex_lock(&s->mutex);
85
curl_clean_state(state);
86
@@ -XXX,XX +XXX,XX @@ static void curl_setup_preadv(BlockDriverState *bs, CURLAIOCB *acb)
87
state->acb[0] = acb;
88
89
snprintf(state->range, 127, "%" PRIu64 "-%" PRIu64, start, end);
90
- DPRINTF("CURL (AIO): Reading %" PRIu64 " at %" PRIu64 " (%s)\n",
91
- acb->bytes, start, state->range);
92
+ trace_curl_setup_preadv(acb->bytes, start, state->range);
93
curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range);
94
95
curl_multi_add_handle(s->multi, state->curl);
96
@@ -XXX,XX +XXX,XX @@ static void curl_close(BlockDriverState *bs)
97
{
98
BDRVCURLState *s = bs->opaque;
99
100
- DPRINTF("CURL: Close\n");
101
+ trace_curl_close();
102
curl_detach_aio_context(bs);
103
qemu_mutex_destroy(&s->mutex);
104
105
diff --git a/block/trace-events b/block/trace-events
106
index XXXXXXX..XXXXXXX 100644
107
--- a/block/trace-events
108
+++ b/block/trace-events
109
@@ -XXX,XX +XXX,XX @@ ssh_write(int64_t offset, size_t size) "offset=%" PRIi64 " size=%zu"
110
ssh_write_buf(void *buf, size_t size) "sftp_write buf=%p size=%zu"
111
ssh_write_return(ssize_t ret) "sftp_write returned %zd"
112
ssh_seek(int64_t offset) "seeking to offset=%" PRIi64
113
+
114
+# block/curl.c
115
+curl_timer_cb(long timeout_ms) "timer callback timeout_ms %ld"
116
+curl_sock_cb(int action, int fd) "sock action %d on fd %d"
117
+curl_read_cb(size_t realsize) "just reading %zu bytes"
118
+curl_open(const char *file) "opening %s"
119
+curl_open_size(uint64_t size) "size = %" PRIu64
120
+curl_setup_preadv(uint64_t bytes, uint64_t start, const char *range) "reading %" PRIu64 " at %" PRIu64 " (%s)"
121
+curl_close(void) "close"
122
--
123
2.20.1
124
125
diff view generated by jsdifflib
Deleted patch
1
From: Laurent Vivier <lvivier@redhat.com>
2
1
3
Signed-off-by: Laurent Vivier <lvivier@redhat.com>
4
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
5
Message-id: 20181213162727.17438-4-lvivier@redhat.com
6
Signed-off-by: Max Reitz <mreitz@redhat.com>
7
---
8
block/file-posix.c | 25 ++++++-------------------
9
block/trace-events | 7 +++++++
10
2 files changed, 13 insertions(+), 19 deletions(-)
11
12
diff --git a/block/file-posix.c b/block/file-posix.c
13
index XXXXXXX..XXXXXXX 100644
14
--- a/block/file-posix.c
15
+++ b/block/file-posix.c
16
@@ -XXX,XX +XXX,XX @@
17
#include <xfs/xfs.h>
18
#endif
19
20
-//#define DEBUG_BLOCK
21
-
22
-#ifdef DEBUG_BLOCK
23
-# define DEBUG_BLOCK_PRINT 1
24
-#else
25
-# define DEBUG_BLOCK_PRINT 0
26
-#endif
27
-#define DPRINTF(fmt, ...) \
28
-do { \
29
- if (DEBUG_BLOCK_PRINT) { \
30
- printf(fmt, ## __VA_ARGS__); \
31
- } \
32
-} while (0)
33
+#include "trace.h"
34
35
/* OS X does not have O_DSYNC */
36
#ifndef O_DSYNC
37
@@ -XXX,XX +XXX,XX @@ static int xfs_write_zeroes(BDRVRawState *s, int64_t offset, uint64_t bytes)
38
39
if (xfsctl(NULL, s->fd, XFS_IOC_ZERO_RANGE, &fl) < 0) {
40
err = errno;
41
- DPRINTF("cannot write zero range (%s)\n", strerror(errno));
42
+ trace_file_xfs_write_zeroes(strerror(errno));
43
return -err;
44
}
45
46
@@ -XXX,XX +XXX,XX @@ static int xfs_discard(BDRVRawState *s, int64_t offset, uint64_t bytes)
47
48
if (xfsctl(NULL, s->fd, XFS_IOC_UNRESVSP64, &fl) < 0) {
49
err = errno;
50
- DPRINTF("cannot punch hole (%s)\n", strerror(errno));
51
+ trace_file_xfs_discard(strerror(errno));
52
return -err;
53
}
54
55
@@ -XXX,XX +XXX,XX @@ static char *FindEjectableOpticalMedia(io_iterator_t *mediaIterator)
56
57
/* If a match was found, leave the loop */
58
if (*mediaIterator != 0) {
59
- DPRINTF("Matching using %s\n", matching_array[index]);
60
+ trace_file_FindEjectableOpticalMedia(matching_array[index]);
61
mediaType = g_strdup(matching_array[index]);
62
break;
63
}
64
@@ -XXX,XX +XXX,XX @@ static bool setup_cdrom(char *bsd_path, Error **errp)
65
if (partition_found == false) {
66
error_setg(errp, "Failed to find a working partition on disc");
67
} else {
68
- DPRINTF("Using %s as optical disc\n", test_partition);
69
+ trace_file_setup_cdrom(test_partition);
70
pstrcpy(bsd_path, MAXPATHLEN, test_partition);
71
}
72
return partition_found;
73
@@ -XXX,XX +XXX,XX @@ static bool hdev_is_sg(BlockDriverState *bs)
74
75
ret = ioctl(s->fd, SG_GET_SCSI_ID, &scsiid);
76
if (ret >= 0) {
77
- DPRINTF("SG device found: type=%d, version=%d\n",
78
- scsiid.scsi_type, sg_version);
79
+ trace_file_hdev_is_sg(scsiid.scsi_type, sg_version);
80
return true;
81
}
82
83
diff --git a/block/trace-events b/block/trace-events
84
index XXXXXXX..XXXXXXX 100644
85
--- a/block/trace-events
86
+++ b/block/trace-events
87
@@ -XXX,XX +XXX,XX @@ curl_open(const char *file) "opening %s"
88
curl_open_size(uint64_t size) "size = %" PRIu64
89
curl_setup_preadv(uint64_t bytes, uint64_t start, const char *range) "reading %" PRIu64 " at %" PRIu64 " (%s)"
90
curl_close(void) "close"
91
+
92
+# block/file-posix.c
93
+file_xfs_write_zeroes(const char *error) "cannot write zero range (%s)"
94
+file_xfs_discard(const char *error) "cannot punch hole (%s)"
95
+file_FindEjectableOpticalMedia(const char *media) "Matching using %s"
96
+file_setup_cdrom(const char *partition) "Using %s as optical disc"
97
+file_hdev_is_sg(int type, int version) "SG device found: type=%d, version=%d"
98
--
99
2.20.1
100
101
diff view generated by jsdifflib
Deleted patch
1
From: Laurent Vivier <lvivier@redhat.com>
2
1
3
Signed-off-by: Laurent Vivier <lvivier@redhat.com>
4
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
5
Message-id: 20181213162727.17438-5-lvivier@redhat.com
6
[mreitz: Fixed sheepdog_snapshot_create_inode's format string to use
7
PRIx32 for uint32_ts]
8
Signed-off-by: Max Reitz <mreitz@redhat.com>
9
---
10
block/sheepdog.c | 47 +++++++++++++++++-----------------------------
11
block/trace-events | 14 ++++++++++++++
12
2 files changed, 31 insertions(+), 30 deletions(-)
13
14
diff --git a/block/sheepdog.c b/block/sheepdog.c
15
index XXXXXXX..XXXXXXX 100644
16
--- a/block/sheepdog.c
17
+++ b/block/sheepdog.c
18
@@ -XXX,XX +XXX,XX @@
19
#include "sysemu/block-backend.h"
20
#include "qemu/bitops.h"
21
#include "qemu/cutils.h"
22
+#include "trace.h"
23
24
#define SD_PROTO_VER 0x01
25
26
@@ -XXX,XX +XXX,XX @@ static inline size_t count_data_objs(const struct SheepdogInode *inode)
27
(1UL << inode->block_size_shift));
28
}
29
30
-#undef DPRINTF
31
-#ifdef DEBUG_SDOG
32
-#define DEBUG_SDOG_PRINT 1
33
-#else
34
-#define DEBUG_SDOG_PRINT 0
35
-#endif
36
-#define DPRINTF(fmt, args...) \
37
- do { \
38
- if (DEBUG_SDOG_PRINT) { \
39
- fprintf(stderr, "%s %d: " fmt, __func__, __LINE__, ##args); \
40
- } \
41
- } while (0)
42
-
43
typedef struct SheepdogAIOCB SheepdogAIOCB;
44
typedef struct BDRVSheepdogState BDRVSheepdogState;
45
46
@@ -XXX,XX +XXX,XX @@ static coroutine_fn void reconnect_to_sdog(void *opaque)
47
Error *local_err = NULL;
48
s->fd = get_sheep_fd(s, &local_err);
49
if (s->fd < 0) {
50
- DPRINTF("Wait for connection to be established\n");
51
+ trace_sheepdog_reconnect_to_sdog();
52
error_report_err(local_err);
53
qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, 1000000000ULL);
54
}
55
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn aio_read_response(void *opaque)
56
break;
57
case AIOCB_FLUSH_CACHE:
58
if (rsp.result == SD_RES_INVALID_PARMS) {
59
- DPRINTF("disable cache since the server doesn't support it\n");
60
+ trace_sheepdog_aio_read_response();
61
s->cache_flags = SD_FLAG_CMD_DIRECT;
62
rsp.result = SD_RES_SUCCESS;
63
}
64
@@ -XXX,XX +XXX,XX @@ static int sd_open(BlockDriverState *bs, QDict *options, int flags,
65
s->discard_supported = true;
66
67
if (snap_id || tag[0]) {
68
- DPRINTF("%" PRIx32 " snapshot inode was open.\n", vid);
69
+ trace_sheepdog_open(vid);
70
s->is_snapshot = true;
71
}
72
73
@@ -XXX,XX +XXX,XX @@ static void sd_close(BlockDriverState *bs)
74
unsigned int wlen, rlen = 0;
75
int fd, ret;
76
77
- DPRINTF("%s\n", s->name);
78
+ trace_sheepdog_close(s->name);
79
80
fd = connect_to_sdog(s, &local_err);
81
if (fd < 0) {
82
@@ -XXX,XX +XXX,XX @@ static int sd_create_branch(BDRVSheepdogState *s)
83
char *buf;
84
bool deleted;
85
86
- DPRINTF("%" PRIx32 " is snapshot.\n", s->inode.vdi_id);
87
+ trace_sheepdog_create_branch_snapshot(s->inode.vdi_id);
88
89
buf = g_malloc(SD_INODE_SIZE);
90
91
@@ -XXX,XX +XXX,XX @@ static int sd_create_branch(BDRVSheepdogState *s)
92
goto out;
93
}
94
95
- DPRINTF("%" PRIx32 " is created.\n", vid);
96
+ trace_sheepdog_create_branch_created(vid);
97
98
fd = connect_to_sdog(s, &local_err);
99
if (fd < 0) {
100
@@ -XXX,XX +XXX,XX @@ static int sd_create_branch(BDRVSheepdogState *s)
101
102
s->is_snapshot = false;
103
ret = 0;
104
- DPRINTF("%" PRIx32 " was newly created.\n", s->inode.vdi_id);
105
+ trace_sheepdog_create_branch_new(s->inode.vdi_id);
106
107
out:
108
g_free(buf);
109
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn sd_co_rw_vector(SheepdogAIOCB *acb)
110
}
111
112
if (create) {
113
- DPRINTF("update ino (%" PRIu32 ") %" PRIu64 " %" PRIu64 " %ld\n",
114
- inode->vdi_id, oid,
115
- vid_to_data_oid(inode->data_vdi_id[idx], idx), idx);
116
+ trace_sheepdog_co_rw_vector_update(inode->vdi_id, oid,
117
+ vid_to_data_oid(inode->data_vdi_id[idx], idx),
118
+ idx);
119
oid = vid_to_data_oid(inode->vdi_id, idx);
120
- DPRINTF("new oid %" PRIx64 "\n", oid);
121
+ trace_sheepdog_co_rw_vector_new(oid);
122
}
123
124
aio_req = alloc_aio_req(s, acb, oid, len, offset, flags, create,
125
@@ -XXX,XX +XXX,XX @@ static int sd_snapshot_create(BlockDriverState *bs, QEMUSnapshotInfo *sn_info)
126
SheepdogInode *inode;
127
unsigned int datalen;
128
129
- DPRINTF("sn_info: name %s id_str %s s: name %s vm_state_size %" PRId64 " "
130
- "is_snapshot %d\n", sn_info->name, sn_info->id_str,
131
- s->name, sn_info->vm_state_size, s->is_snapshot);
132
+ trace_sheepdog_snapshot_create_info(sn_info->name, sn_info->id_str, s->name,
133
+ sn_info->vm_state_size, s->is_snapshot);
134
135
if (s->is_snapshot) {
136
error_report("You can't create a snapshot of a snapshot VDI, "
137
@@ -XXX,XX +XXX,XX @@ static int sd_snapshot_create(BlockDriverState *bs, QEMUSnapshotInfo *sn_info)
138
return -EINVAL;
139
}
140
141
- DPRINTF("%s %s\n", sn_info->name, sn_info->id_str);
142
+ trace_sheepdog_snapshot_create(sn_info->name, sn_info->id_str);
143
144
s->inode.vm_state_size = sn_info->vm_state_size;
145
s->inode.vm_clock_nsec = sn_info->vm_clock_nsec;
146
@@ -XXX,XX +XXX,XX @@ static int sd_snapshot_create(BlockDriverState *bs, QEMUSnapshotInfo *sn_info)
147
}
148
149
memcpy(&s->inode, inode, datalen);
150
- DPRINTF("s->inode: name %s snap_id %x oid %x\n",
151
- s->inode.name, s->inode.snap_id, s->inode.vdi_id);
152
+ trace_sheepdog_snapshot_create_inode(s->inode.name, s->inode.snap_id,
153
+ s->inode.vdi_id);
154
155
cleanup:
156
g_free(inode);
157
diff --git a/block/trace-events b/block/trace-events
158
index XXXXXXX..XXXXXXX 100644
159
--- a/block/trace-events
160
+++ b/block/trace-events
161
@@ -XXX,XX +XXX,XX @@ file_xfs_discard(const char *error) "cannot punch hole (%s)"
162
file_FindEjectableOpticalMedia(const char *media) "Matching using %s"
163
file_setup_cdrom(const char *partition) "Using %s as optical disc"
164
file_hdev_is_sg(int type, int version) "SG device found: type=%d, version=%d"
165
+
166
+# block/sheepdog.c
167
+sheepdog_reconnect_to_sdog(void) "Wait for connection to be established"
168
+sheepdog_aio_read_response(void) "disable cache since the server doesn't support it"
169
+sheepdog_open(uint32_t vid) "0x%" PRIx32 " snapshot inode was open"
170
+sheepdog_close(const char *name) "%s"
171
+sheepdog_create_branch_snapshot(uint32_t vdi) "0x%" PRIx32 " is snapshot"
172
+sheepdog_create_branch_created(uint32_t vdi) "0x%" PRIx32 " is created"
173
+sheepdog_create_branch_new(uint32_t vdi) "0x%" PRIx32 " was newly created"
174
+sheepdog_co_rw_vector_update(uint32_t vdi, uint64_t oid, uint64_t data, long idx) "update ino (%" PRIu32 ") %" PRIu64 " %" PRIu64 " %ld"
175
+sheepdog_co_rw_vector_new(uint64_t oid) "new oid 0x%" PRIx64
176
+sheepdog_snapshot_create_info(const char *sn_name, const char *id, const char *name, int64_t size, int is_snapshot) "sn_info: name %s id_str %s s: name %s vm_state_size %" PRId64 " " "is_snapshot %d"
177
+sheepdog_snapshot_create(const char *sn_name, const char *id) "%s %s"
178
+sheepdog_snapshot_create_inode(const char *name, uint32_t snap, uint32_t vdi) "s->inode: name %s snap_id 0x%" PRIx32 " vdi 0x%" PRIx32
179
--
180
2.20.1
181
182
diff view generated by jsdifflib
Deleted patch
1
From: "Richard W.M. Jones" <rjones@redhat.com>
2
1
3
On FreeBSD 11.2:
4
5
$ nbdkit memory size=1M --run './qemu-io -f raw -c "aio_write 0 512" $nbd'
6
Parsing error: non-numeric argument, or extraneous/unrecognized suffix -- aio_write
7
8
After main option parsing, we reinitialize optind so we can parse each
9
command. However reinitializing optind to 0 does not work on FreeBSD.
10
What happens when you do this is optind remains 0 after the option
11
parsing loop, and the result is we try to parse argv[optind] ==
12
argv[0] == "aio_write" as if it was the first parameter.
13
14
The FreeBSD manual page says:
15
16
In order to use getopt() to evaluate multiple sets of arguments, or to
17
evaluate a single set of arguments multiple times, the variable optreset
18
must be set to 1 before the second and each additional set of calls to
19
getopt(), and the variable optind must be reinitialized.
20
21
(From the rest of the man page it is clear that optind must be
22
reinitialized to 1).
23
24
The glibc man page says:
25
26
A program that scans multiple argument vectors, or rescans the same
27
vector more than once, and wants to make use of GNU extensions such as
28
'+' and '-' at the start of optstring, or changes the value of
29
POSIXLY_CORRECT between scans, must reinitialize getopt() by resetting
30
optind to 0, rather than the traditional value of 1. (Resetting to 0
31
forces the invocation of an internal initialization routine that
32
rechecks POSIXLY_CORRECT and checks for GNU extensions in optstring.)
33
34
This commit introduces an OS-portability function called
35
qemu_reset_optind which provides a way of resetting optind that works
36
on FreeBSD and platforms that use optreset, while keeping it the same
37
as now on other platforms.
38
39
Note that the qemu codebase sets optind in many other places, but in
40
those other places it's setting a local variable and not using getopt.
41
This change is only needed in places where we are using getopt and the
42
associated global variable optind.
43
44
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
45
Message-id: 20190118101114.11759-2-rjones@redhat.com
46
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
47
Reviewed-by: Eric Blake <eblake@redhat.com>
48
Signed-off-by: Max Reitz <mreitz@redhat.com>
49
---
50
configure | 14 ++++++++++++++
51
include/qemu/osdep.h | 16 ++++++++++++++++
52
qemu-img.c | 2 +-
53
qemu-io-cmds.c | 2 +-
54
4 files changed, 32 insertions(+), 2 deletions(-)
55
56
diff --git a/configure b/configure
57
index XXXXXXX..XXXXXXX 100755
58
--- a/configure
59
+++ b/configure
60
@@ -XXX,XX +XXX,XX @@ if compile_prog "" "" ; then
61
signalfd=yes
62
fi
63
64
+# check if optreset global is declared by <getopt.h>
65
+optreset="no"
66
+cat > $TMPC << EOF
67
+#include <getopt.h>
68
+int main(void) { return optreset; }
69
+EOF
70
+
71
+if compile_prog "" "" ; then
72
+ optreset=yes
73
+fi
74
+
75
# check if eventfd is supported
76
eventfd=no
77
cat > $TMPC << EOF
78
@@ -XXX,XX +XXX,XX @@ fi
79
if test "$signalfd" = "yes" ; then
80
echo "CONFIG_SIGNALFD=y" >> $config_host_mak
81
fi
82
+if test "$optreset" = "yes" ; then
83
+ echo "HAVE_OPTRESET=y" >> $config_host_mak
84
+fi
85
if test "$tcg" = "yes"; then
86
echo "CONFIG_TCG=y" >> $config_host_mak
87
if test "$tcg_interpreter" = "yes" ; then
88
diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
89
index XXXXXXX..XXXXXXX 100644
90
--- a/include/qemu/osdep.h
91
+++ b/include/qemu/osdep.h
92
@@ -XXX,XX +XXX,XX @@ extern int daemon(int, int);
93
#include <ctype.h>
94
#include <errno.h>
95
#include <fcntl.h>
96
+#include <getopt.h>
97
#include <sys/stat.h>
98
#include <sys/time.h>
99
#include <assert.h>
100
@@ -XXX,XX +XXX,XX @@ extern int qemu_icache_linesize_log;
101
extern int qemu_dcache_linesize;
102
extern int qemu_dcache_linesize_log;
103
104
+/*
105
+ * After using getopt or getopt_long, if you need to parse another set
106
+ * of options, then you must reset optind. Unfortunately the way to
107
+ * do this varies between implementations of getopt.
108
+ */
109
+static inline void qemu_reset_optind(void)
110
+{
111
+#ifdef HAVE_OPTRESET
112
+ optind = 1;
113
+ optreset = 1;
114
+#else
115
+ optind = 0;
116
+#endif
117
+}
118
+
119
#endif
120
diff --git a/qemu-img.c b/qemu-img.c
121
index XXXXXXX..XXXXXXX 100644
122
--- a/qemu-img.c
123
+++ b/qemu-img.c
124
@@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv)
125
return 0;
126
}
127
argv += optind;
128
- optind = 0;
129
+ qemu_reset_optind();
130
131
if (!trace_init_backends()) {
132
exit(1);
133
diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
134
index XXXXXXX..XXXXXXX 100644
135
--- a/qemu-io-cmds.c
136
+++ b/qemu-io-cmds.c
137
@@ -XXX,XX +XXX,XX @@ static int command(BlockBackend *blk, const cmdinfo_t *ct, int argc,
138
}
139
}
140
141
- optind = 0;
142
+ qemu_reset_optind();
143
return ct->cfunc(blk, argc, argv);
144
}
145
146
--
147
2.20.1
148
149
diff view generated by jsdifflib
Deleted patch
1
From: Li Qiang <liq3ea@163.com>
2
1
3
Signed-off-by: Li Qiang <liq3ea@163.com>
4
Reviewed-by: Max Reitz <mreitz@redhat.com>
5
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
6
Message-id: 20190120055558.32984-2-liq3ea@163.com
7
Signed-off-by: Max Reitz <mreitz@redhat.com>
8
---
9
hw/block/nvme.c | 2 +-
10
1 file changed, 1 insertion(+), 1 deletion(-)
11
12
diff --git a/hw/block/nvme.c b/hw/block/nvme.c
13
index XXXXXXX..XXXXXXX 100644
14
--- a/hw/block/nvme.c
15
+++ b/hw/block/nvme.c
16
@@ -XXX,XX +XXX,XX @@ static void nvme_instance_init(Object *obj)
17
}
18
19
static const TypeInfo nvme_info = {
20
- .name = "nvme",
21
+ .name = TYPE_NVME,
22
.parent = TYPE_PCI_DEVICE,
23
.instance_size = sizeof(NvmeCtrl),
24
.class_init = nvme_class_init,
25
--
26
2.20.1
27
28
diff view generated by jsdifflib
1
By default, qemu-nbd binds to 0.0.0.0. However, we then proceed to
1
Tests should place their files into the test directory. This includes
2
connect to "localhost". Usually, this works out fine; but if this test
2
Unix sockets. 205 currently fails to do so, which prevents it from
3
is run concurrently, some other test function may have bound a different
3
being run concurrently.
4
server to ::1 (on the same port -- you can bind different serves to the
5
same port, as long as one is on IPv4 and the other on IPv6).
6
7
So running qemu-nbd works, it can bind to 0.0.0.0:NBD_PORT. But
8
potentially a concurrent test has successfully taken [::1]:NBD_PORT. In
9
this case, trying to connect to "localhost" will lead us to the IPv6
10
instance, where we do not want to end up.
11
12
Fix this by just binding to "localhost". This will make qemu-nbd error
13
out immediately and not give us cryptic errors later.
14
15
(Also, it will allow us to just try a different port as of a future
16
patch.)
17
4
18
Signed-off-by: Max Reitz <mreitz@redhat.com>
5
Signed-off-by: Max Reitz <mreitz@redhat.com>
19
Message-id: 20181221234750.23577-3-mreitz@redhat.com
6
Message-id: 20190618210238.9524-1-mreitz@redhat.com
20
Reviewed-by: John Snow <jsnow@redhat.com>
21
Reviewed-by: Eric Blake <eblake@redhat.com>
7
Reviewed-by: Eric Blake <eblake@redhat.com>
22
Signed-off-by: Max Reitz <mreitz@redhat.com>
8
Signed-off-by: Max Reitz <mreitz@redhat.com>
23
---
9
---
24
tests/qemu-iotests/147 | 2 +-
10
tests/qemu-iotests/205 | 2 +-
25
1 file changed, 1 insertion(+), 1 deletion(-)
11
1 file changed, 1 insertion(+), 1 deletion(-)
26
12
27
diff --git a/tests/qemu-iotests/147 b/tests/qemu-iotests/147
13
diff --git a/tests/qemu-iotests/205 b/tests/qemu-iotests/205
28
index XXXXXXX..XXXXXXX 100755
14
index XXXXXXX..XXXXXXX 100755
29
--- a/tests/qemu-iotests/147
15
--- a/tests/qemu-iotests/205
30
+++ b/tests/qemu-iotests/147
16
+++ b/tests/qemu-iotests/205
31
@@ -XXX,XX +XXX,XX @@ class QemuNBD(NBDBlockdevAddBase):
17
@@ -XXX,XX +XXX,XX @@ import iotests
32
self.assertEqual(qemu_nbd('-f', imgfmt, test_img, *args), 0)
18
import time
33
19
from iotests import qemu_img_create, qemu_io, filter_qemu_io, QemuIoInteractive
34
def test_inet(self):
20
35
- self._server_up('-p', str(NBD_PORT))
21
-nbd_sock = 'nbd_sock'
36
+ self._server_up('-b', 'localhost', '-p', str(NBD_PORT))
22
+nbd_sock = os.path.join(iotests.test_dir, 'nbd_sock')
37
address = { 'type': 'inet',
23
nbd_uri = 'nbd+unix:///exp?socket=' + nbd_sock
38
'data': {
24
disk = os.path.join(iotests.test_dir, 'disk')
39
'host': 'localhost',
25
40
--
26
--
41
2.20.1
27
2.21.0
42
28
43
29
diff view generated by jsdifflib