python/qemu/aqmp/qmp_send.py | 229 +++++++++++++++++++++++++++++++++++ scripts/qmp/qmp-send | 11 ++ 2 files changed, 240 insertions(+) create mode 100644 python/qemu/aqmp/qmp_send.py create mode 100755 scripts/qmp/qmp-send
It takes an input file containing raw qmp commands (concatenated json
dicts) and send all commands one by one to a qmp server. When one
command fails, it exits.
As a convenience, it can also wrap the qemu process to avoid having
to start qemu in background. When wrapping qemu, the program returns
only when the qemu process terminates.
Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
---
Hi all,
Following our discussion, I've started this. What do you think ?
I tried to follow Daniel's qmp-shell-wrap. I think it is
better to have similar options (eg: logging). There is also room
for factorizing code if we want to keep them aligned and ease
maintenance.
There are still some pylint issues (too many branches in main and it
does not like my context manager if else line). But it's kind of a
mess to fix theses so I think it's enough for a first version.
I name that qmp-send as Daniel proposed, maybe qmp-test matches better
what I'm doing there ?
Thanks,
Damien
---
python/qemu/aqmp/qmp_send.py | 229 +++++++++++++++++++++++++++++++++++
scripts/qmp/qmp-send | 11 ++
2 files changed, 240 insertions(+)
create mode 100644 python/qemu/aqmp/qmp_send.py
create mode 100755 scripts/qmp/qmp-send
diff --git a/python/qemu/aqmp/qmp_send.py b/python/qemu/aqmp/qmp_send.py
new file mode 100644
index 0000000000..cbca1d0205
--- /dev/null
+++ b/python/qemu/aqmp/qmp_send.py
@@ -0,0 +1,229 @@
+#
+# Copyright (C) 2022 Greensocs
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+#
+
+"""
+usage: qmp-send [-h] [-f FILE] [-s SOCKET] [-v] [-p] [--wrap ...]
+
+Send raw qmp commands to qemu as long as they succeed. It either connects to a
+remote qmp server using the provided socket or wrap the qemu process. It stops
+sending the provided commands when a command fails (disconnection or error
+response).
+
+optional arguments:
+ -h, --help show this help message and exit
+ -f FILE, --file FILE Input file containing the commands
+ -s SOCKET, --socket SOCKET
+ < UNIX socket path | TCP address:port >
+ -v, --verbose Verbose (echo commands sent and received)
+ -p, --pretty Pretty-print JSON
+ --wrap ... QEMU command line to invoke
+
+When qemu wrap option is used, this script waits for qemu to terminate but
+never send any quit or kill command. This needs to be done manually.
+"""
+
+import argparse
+import contextlib
+import json
+import logging
+import os
+from subprocess import Popen
+import sys
+from typing import List, TextIO
+
+from qemu.aqmp import ConnectError, QMPError, SocketAddrT
+from qemu.aqmp.legacy import (
+ QEMUMonitorProtocol,
+ QMPBadPortError,
+ QMPMessage,
+)
+
+
+LOG = logging.getLogger(__name__)
+
+
+class QmpRawDecodeError(Exception):
+ """
+ Exception for raw qmp decoding
+
+ msg: exception message
+ lineno: input line of the error
+ colno: input column of the error
+ """
+ def __init__(self, msg: str, lineno: int, colno: int):
+ self.msg = msg
+ self.lineno = lineno
+ self.colno = colno
+ super().__init__(f"{msg}: line {lineno} column {colno}")
+
+
+class QMPSendError(QMPError):
+ """
+ QMP Send Base error class.
+ """
+
+
+class QMPSend(QEMUMonitorProtocol):
+ """
+ QMP Send class.
+ """
+ def __init__(self, address: SocketAddrT,
+ pretty: bool = False,
+ verbose: bool = False,
+ server: bool = False):
+ super().__init__(address, server=server)
+ self._verbose = verbose
+ self._pretty = pretty
+ self._server = server
+
+ def setup_connection(self) -> None:
+ """Setup the connetion with the remote client/server."""
+ if self._server:
+ self.accept()
+ else:
+ self.connect()
+
+ def _print(self, qmp_message: object) -> None:
+ jsobj = json.dumps(qmp_message,
+ indent=4 if self._pretty else None,
+ sort_keys=self._pretty)
+ print(str(jsobj))
+
+ def execute_cmd(self, cmd: QMPMessage) -> None:
+ """Execute a qmp command."""
+ if self._verbose:
+ self._print(cmd)
+ resp = self.cmd_obj(cmd)
+ if resp is None:
+ raise QMPSendError("Disconnected")
+ if self._verbose:
+ self._print(resp)
+ if 'error' in resp:
+ raise QMPSendError(f"Command failed: {resp['error']}")
+
+
+def raw_load(file: TextIO) -> List[QMPMessage]:
+ """parse a raw qmp command file.
+
+ JSON formatted commands can expand on several lines but must
+ be separated by an end-of-line (two commands can not share the
+ same line).
+ File must not end with empty lines.
+ """
+ cmds: List[QMPMessage] = []
+ linecnt = 0
+ while True:
+ buf = file.readline()
+ if not buf:
+ return cmds
+ prev_err_pos = None
+ buf_linecnt = 1
+ while True:
+ try:
+ cmds.append(json.loads(buf))
+ break
+ except json.JSONDecodeError as err:
+ if prev_err_pos == err.pos:
+ # adding a line made no progress so
+ # + either we're at EOF and json data is truncated
+ # + or the parsing error is before
+ raise QmpRawDecodeError(err.msg, linecnt + err.lineno,
+ err.colno) from err
+ prev_err_pos = err.pos
+ buf += file.readline()
+ buf_linecnt += 1
+ linecnt += buf_linecnt
+
+
+def report_error(msg: str) -> None:
+ """Write an error to stderr."""
+ sys.stderr.write('ERROR: %s\n' % msg)
+
+
+def main() -> None:
+ """
+ qmp-send entry point: parse command line arguments and start the REPL.
+ """
+ parser = argparse.ArgumentParser(
+ description="""
+ Send raw qmp commands to qemu as long as they succeed. It either
+ connects to a remote qmp server using the provided socket or wrap
+ the qemu process. It stops sending the provided commands when a
+ command fails (disconnection or error response).
+ """,
+ epilog="""
+ When qemu wrap option is used, this script waits for qemu
+ to terminate but never send any quit or kill command. This
+ needs to be done manually.
+ """)
+
+ parser.add_argument('-f', '--file', action='store',
+ help='Input file containing the commands')
+ parser.add_argument('-s', '--socket', action='store',
+ help='< UNIX socket path | TCP address:port >')
+ parser.add_argument('-v', '--verbose', action='store_true',
+ help='Verbose (echo commands sent and received)')
+ parser.add_argument('-p', '--pretty', action='store_true',
+ help='Pretty-print JSON')
+
+ parser.add_argument('--wrap', nargs=argparse.REMAINDER,
+ help='QEMU command line to invoke')
+
+ args = parser.parse_args()
+
+ socket = args.socket
+ wrap_qemu = args.wrap is not None
+
+ if wrap_qemu:
+ if len(args.wrap) != 0:
+ qemu_cmdline = args.wrap
+ else:
+ qemu_cmdline = ["qemu-system-x86_64"]
+ if socket is None:
+ socket = "qmp-send-wrap-%d" % os.getpid()
+ qemu_cmdline += ["-qmp", "unix:%s" % socket]
+
+ try:
+ address = QMPSend.parse_address(socket)
+ except QMPBadPortError:
+ parser.error(f"Bad port number: {socket}")
+ return # pycharm doesn't know error() is noreturn
+
+ try:
+ with open(args.file, mode='rt', encoding='utf8') as file:
+ qmp_cmds = raw_load(file)
+ except QmpRawDecodeError as err:
+ report_error(str(err))
+ sys.exit(1)
+
+ try:
+ with QMPSend(address, args.pretty, args.verbose,
+ server=wrap_qemu) as qmp:
+ # starting with python 3.7 we could use contextlib.nullcontext
+ qemu = Popen(qemu_cmdline) if wrap_qemu else contextlib.suppress()
+ with qemu:
+ try:
+ qmp.setup_connection()
+ except ConnectError as err:
+ if isinstance(err.exc, OSError):
+ report_error(f"Couldn't connect to {socket}: {err!s}")
+ else:
+ report_error(str(err))
+ sys.exit(1)
+ try:
+ for cmd in qmp_cmds:
+ qmp.execute_cmd(cmd)
+ except QMPError as err:
+ report_error(str(err))
+ sys.exit(1)
+ finally:
+ if wrap_qemu:
+ os.unlink(socket)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/scripts/qmp/qmp-send b/scripts/qmp/qmp-send
new file mode 100755
index 0000000000..8d3063797c
--- /dev/null
+++ b/scripts/qmp/qmp-send
@@ -0,0 +1,11 @@
+#!/usr/bin/env python3
+
+import os
+import sys
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.aqmp import qmp_send
+
+
+if __name__ == '__main__':
+ qmp_send.main()
--
2.35.1On Wed, Mar 16, 2022 at 10:54:55AM +0100, Damien Hedde wrote:
> +def raw_load(file: TextIO) -> List[QMPMessage]:
> + """parse a raw qmp command file.
> +
> + JSON formatted commands can expand on several lines but must
> + be separated by an end-of-line (two commands can not share the
> + same line).
> + File must not end with empty lines.
> + """
> + cmds: List[QMPMessage] = []
> + linecnt = 0
> + while True:
> + buf = file.readline()
> + if not buf:
> + return cmds
If you change this to 'break'...
> + prev_err_pos = None
> + buf_linecnt = 1
> + while True:
> + try:
> + cmds.append(json.loads(buf))
...and this to
yield json.loads(buf)
then....
> + break
> + except json.JSONDecodeError as err:
> + if prev_err_pos == err.pos:
> + # adding a line made no progress so
> + # + either we're at EOF and json data is truncated
> + # + or the parsing error is before
> + raise QmpRawDecodeError(err.msg, linecnt + err.lineno,
> + err.colno) from err
> + prev_err_pos = err.pos
> + buf += file.readline()
> + buf_linecnt += 1
> + linecnt += buf_linecnt
> +
> +
> +def report_error(msg: str) -> None:
> + """Write an error to stderr."""
> + sys.stderr.write('ERROR: %s\n' % msg)
> +
> +
> +def main() -> None:
> + """
> + qmp-send entry point: parse command line arguments and start the REPL.
> + """
> + parser = argparse.ArgumentParser(
> + description="""
> + Send raw qmp commands to qemu as long as they succeed. It either
> + connects to a remote qmp server using the provided socket or wrap
> + the qemu process. It stops sending the provided commands when a
> + command fails (disconnection or error response).
> + """,
> + epilog="""
> + When qemu wrap option is used, this script waits for qemu
> + to terminate but never send any quit or kill command. This
> + needs to be done manually.
> + """)
> +
> + parser.add_argument('-f', '--file', action='store',
> + help='Input file containing the commands')
> + parser.add_argument('-s', '--socket', action='store',
> + help='< UNIX socket path | TCP address:port >')
> + parser.add_argument('-v', '--verbose', action='store_true',
> + help='Verbose (echo commands sent and received)')
> + parser.add_argument('-p', '--pretty', action='store_true',
> + help='Pretty-print JSON')
> +
> + parser.add_argument('--wrap', nargs=argparse.REMAINDER,
> + help='QEMU command line to invoke')
> +
> + args = parser.parse_args()
> +
> + socket = args.socket
> + wrap_qemu = args.wrap is not None
> +
> + if wrap_qemu:
> + if len(args.wrap) != 0:
> + qemu_cmdline = args.wrap
> + else:
> + qemu_cmdline = ["qemu-system-x86_64"]
> + if socket is None:
> + socket = "qmp-send-wrap-%d" % os.getpid()
> + qemu_cmdline += ["-qmp", "unix:%s" % socket]
> +
> + try:
> + address = QMPSend.parse_address(socket)
> + except QMPBadPortError:
> + parser.error(f"Bad port number: {socket}")
> + return # pycharm doesn't know error() is noreturn
> +
> + try:
> + with open(args.file, mode='rt', encoding='utf8') as file:
> + qmp_cmds = raw_load(file)
> + except QmpRawDecodeError as err:
> + report_error(str(err))
> + sys.exit(1)
...change this to
fh = sys.stdin
if args.file is not None and args.file != '-':
fh = open(args.file, mode='rt', encoding='utf8')
....
> +
> + try:
> + with QMPSend(address, args.pretty, args.verbose,
> + server=wrap_qemu) as qmp:
> + # starting with python 3.7 we could use contextlib.nullcontext
> + qemu = Popen(qemu_cmdline) if wrap_qemu else contextlib.suppress()
> + with qemu:
> + try:
> + qmp.setup_connection()
> + except ConnectError as err:
> + if isinstance(err.exc, OSError):
> + report_error(f"Couldn't connect to {socket}: {err!s}")
> + else:
> + report_error(str(err))
> + sys.exit(1)
> + try:
> + for cmd in qmp_cmds:
...finally this to
for cmd in raw_load(fh)
This means we can use qmp-send in a pipeline with commands
sent to QEMU on the fly as they arrive, rather than having
to read all the commands upfront before QEMU is started.
BTW, as an example usage I was trying your impl here in the following
way to extract information about CPUs that are deprecated
echo -e '{ "execute": "query-cpu-definitions"}\n{"execute": "quit"}' | \
qmp-send -v -p --wrap ./build/qemu-system-x86_64 -nodefaults -vnc :1 | \
jq -r --slurp '.[1].return[] | [.name, .deprecated] | @csv'
> + qmp.execute_cmd(cmd)
> + except QMPError as err:
> + report_error(str(err))
> + sys.exit(1)
> + finally:
> + if wrap_qemu:
> + os.unlink(socket)
> +
> +
> +if __name__ == '__main__':
> + main()
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 5/25/22 18:06, Daniel P. Berrangé wrote:
> On Wed, Mar 16, 2022 at 10:54:55AM +0100, Damien Hedde wrote:
>
>
>> +def raw_load(file: TextIO) -> List[QMPMessage]:
>> + """parse a raw qmp command file.
>> +
>> + JSON formatted commands can expand on several lines but must
>> + be separated by an end-of-line (two commands can not share the
>> + same line).
>> + File must not end with empty lines.
>> + """
>> + cmds: List[QMPMessage] = []
>> + linecnt = 0
>> + while True:
>> + buf = file.readline()
>> + if not buf:
>> + return cmds
>
> If you change this to 'break'...
>
>> + prev_err_pos = None
>> + buf_linecnt = 1
>> + while True:
>> + try:
>> + cmds.append(json.loads(buf))
>
> ...and this to
>
> yield json.loads(buf)
>
> then....
>
>> + break
>> + except json.JSONDecodeError as err:
>> + if prev_err_pos == err.pos:
>> + # adding a line made no progress so
>> + # + either we're at EOF and json data is truncated
>> + # + or the parsing error is before
>> + raise QmpRawDecodeError(err.msg, linecnt + err.lineno,
>> + err.colno) from err
>> + prev_err_pos = err.pos
>> + buf += file.readline()
>> + buf_linecnt += 1
>> + linecnt += buf_linecnt
>> +
>> +
>> +def report_error(msg: str) -> None:
>> + """Write an error to stderr."""
>> + sys.stderr.write('ERROR: %s\n' % msg)
>> +
>> +
>> +def main() -> None:
>> + """
>> + qmp-send entry point: parse command line arguments and start the REPL.
>> + """
>> + parser = argparse.ArgumentParser(
>> + description="""
>> + Send raw qmp commands to qemu as long as they succeed. It either
>> + connects to a remote qmp server using the provided socket or wrap
>> + the qemu process. It stops sending the provided commands when a
>> + command fails (disconnection or error response).
>> + """,
>> + epilog="""
>> + When qemu wrap option is used, this script waits for qemu
>> + to terminate but never send any quit or kill command. This
>> + needs to be done manually.
>> + """)
>> +
>> + parser.add_argument('-f', '--file', action='store',
>> + help='Input file containing the commands')
>> + parser.add_argument('-s', '--socket', action='store',
>> + help='< UNIX socket path | TCP address:port >')
>> + parser.add_argument('-v', '--verbose', action='store_true',
>> + help='Verbose (echo commands sent and received)')
>> + parser.add_argument('-p', '--pretty', action='store_true',
>> + help='Pretty-print JSON')
>> +
>> + parser.add_argument('--wrap', nargs=argparse.REMAINDER,
>> + help='QEMU command line to invoke')
>> +
>> + args = parser.parse_args()
>> +
>> + socket = args.socket
>> + wrap_qemu = args.wrap is not None
>> +
>> + if wrap_qemu:
>> + if len(args.wrap) != 0:
>> + qemu_cmdline = args.wrap
>> + else:
>> + qemu_cmdline = ["qemu-system-x86_64"]
>> + if socket is None:
>> + socket = "qmp-send-wrap-%d" % os.getpid()
>> + qemu_cmdline += ["-qmp", "unix:%s" % socket]
>> +
>> + try:
>> + address = QMPSend.parse_address(socket)
>> + except QMPBadPortError:
>> + parser.error(f"Bad port number: {socket}")
>> + return # pycharm doesn't know error() is noreturn
>> +
>> + try:
>> + with open(args.file, mode='rt', encoding='utf8') as file:
>> + qmp_cmds = raw_load(file)
>> + except QmpRawDecodeError as err:
>> + report_error(str(err))
>> + sys.exit(1)
>
> ...change this to
>
> fh = sys.stdin
> if args.file is not None and args.file != '-':
> fh = open(args.file, mode='rt', encoding='utf8')
>
> ....
>
>> +
>> + try:
>> + with QMPSend(address, args.pretty, args.verbose,
>> + server=wrap_qemu) as qmp:
>> + # starting with python 3.7 we could use contextlib.nullcontext
>> + qemu = Popen(qemu_cmdline) if wrap_qemu else contextlib.suppress()
>> + with qemu:
>> + try:
>> + qmp.setup_connection()
>> + except ConnectError as err:
>> + if isinstance(err.exc, OSError):
>> + report_error(f"Couldn't connect to {socket}: {err!s}")
>> + else:
>> + report_error(str(err))
>> + sys.exit(1)
>> + try:
>> + for cmd in qmp_cmds:
>
> ...finally this to
>
> for cmd in raw_load(fh)
>
>
> This means we can use qmp-send in a pipeline with commands
> sent to QEMU on the fly as they arrive, rather than having
> to read all the commands upfront before QEMU is started.
Yes. I was not sure which way was "better" between reading on the fly or
buffering everything before. In we want pipelining, we don't have much
choice.
>
> BTW, as an example usage I was trying your impl here in the following
> way to extract information about CPUs that are deprecated
>
> echo -e '{ "execute": "query-cpu-definitions"}\n{"execute": "quit"}' | \
> qmp-send -v -p --wrap ./build/qemu-system-x86_64 -nodefaults -vnc :1 | \
> jq -r --slurp '.[1].return[] | [.name, .deprecated] | @csv'
>
>
>> + qmp.execute_cmd(cmd)
>> + except QMPError as err:
>> + report_error(str(err))
>> + sys.exit(1)
>> + finally:
>> + if wrap_qemu:
>> + os.unlink(socket)
>> +
>> +
>> +if __name__ == '__main__':
>> + main()
>
>
> With regards,
> Daniel
On Wed, Mar 16, 2022 at 5:55 AM Damien Hedde <damien.hedde@greensocs.com> wrote:
>
> It takes an input file containing raw qmp commands (concatenated json
> dicts) and send all commands one by one to a qmp server. When one
> command fails, it exits.
>
> As a convenience, it can also wrap the qemu process to avoid having
> to start qemu in background. When wrapping qemu, the program returns
> only when the qemu process terminates.
>
> Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
> ---
>
> Hi all,
>
> Following our discussion, I've started this. What do you think ?
>
> I tried to follow Daniel's qmp-shell-wrap. I think it is
> better to have similar options (eg: logging). There is also room
> for factorizing code if we want to keep them aligned and ease
> maintenance.
>
> There are still some pylint issues (too many branches in main and it
> does not like my context manager if else line). But it's kind of a
> mess to fix theses so I think it's enough for a first version.
Yeah, don't worry about these. You can just tell pylint to shut up
while you prototype. Sometimes it's just not worth spending more time
on a more beautiful factoring. Oh well.
>
> I name that qmp-send as Daniel proposed, maybe qmp-test matches better
> what I'm doing there ?
>
I think I agree with Dan's response.
> Thanks,
> Damien
> ---
> python/qemu/aqmp/qmp_send.py | 229 +++++++++++++++++++++++++++++++++++
I recommend putting this in qemu/util/qmp_send.py instead.
I'm in the process of pulling out the AQMP lib and hosting it
separately. Scripts like this I think should stay in the QEMU tree, so
moving it to util instead is probably best. Otherwise, I'll *really*
have to commit to the syntax, and that's probably a bigger hurdle than
you want to deal with.
> scripts/qmp/qmp-send | 11 ++
> 2 files changed, 240 insertions(+)
> create mode 100644 python/qemu/aqmp/qmp_send.py
> create mode 100755 scripts/qmp/qmp-send
>
> diff --git a/python/qemu/aqmp/qmp_send.py b/python/qemu/aqmp/qmp_send.py
> new file mode 100644
> index 0000000000..cbca1d0205
> --- /dev/null
> +++ b/python/qemu/aqmp/qmp_send.py
> @@ -0,0 +1,229 @@
> +#
> +# Copyright (C) 2022 Greensocs
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later. See the COPYING file in the top-level directory.
> +#
> +
> +"""
> +usage: qmp-send [-h] [-f FILE] [-s SOCKET] [-v] [-p] [--wrap ...]
> +
> +Send raw qmp commands to qemu as long as they succeed. It either connects to a
> +remote qmp server using the provided socket or wrap the qemu process. It stops
> +sending the provided commands when a command fails (disconnection or error
> +response).
> +
> +optional arguments:
> + -h, --help show this help message and exit
> + -f FILE, --file FILE Input file containing the commands
> + -s SOCKET, --socket SOCKET
> + < UNIX socket path | TCP address:port >
> + -v, --verbose Verbose (echo commands sent and received)
> + -p, --pretty Pretty-print JSON
> + --wrap ... QEMU command line to invoke
> +
> +When qemu wrap option is used, this script waits for qemu to terminate but
> +never send any quit or kill command. This needs to be done manually.
> +"""
> +
> +import argparse
> +import contextlib
> +import json
> +import logging
> +import os
> +from subprocess import Popen
> +import sys
> +from typing import List, TextIO
> +
> +from qemu.aqmp import ConnectError, QMPError, SocketAddrT
> +from qemu.aqmp.legacy import (
> + QEMUMonitorProtocol,
> + QMPBadPortError,
> + QMPMessage,
> +)
> +
> +
> +LOG = logging.getLogger(__name__)
> +
> +
> +class QmpRawDecodeError(Exception):
> + """
> + Exception for raw qmp decoding
> +
> + msg: exception message
> + lineno: input line of the error
> + colno: input column of the error
> + """
> + def __init__(self, msg: str, lineno: int, colno: int):
> + self.msg = msg
> + self.lineno = lineno
> + self.colno = colno
> + super().__init__(f"{msg}: line {lineno} column {colno}")
> +
> +
> +class QMPSendError(QMPError):
> + """
> + QMP Send Base error class.
> + """
> +
> +
> +class QMPSend(QEMUMonitorProtocol):
> + """
> + QMP Send class.
> + """
> + def __init__(self, address: SocketAddrT,
> + pretty: bool = False,
> + verbose: bool = False,
> + server: bool = False):
> + super().__init__(address, server=server)
> + self._verbose = verbose
> + self._pretty = pretty
> + self._server = server
> +
> + def setup_connection(self) -> None:
> + """Setup the connetion with the remote client/server."""
> + if self._server:
> + self.accept()
> + else:
> + self.connect()
> +
> + def _print(self, qmp_message: object) -> None:
> + jsobj = json.dumps(qmp_message,
> + indent=4 if self._pretty else None,
> + sort_keys=self._pretty)
> + print(str(jsobj))
> +
> + def execute_cmd(self, cmd: QMPMessage) -> None:
> + """Execute a qmp command."""
> + if self._verbose:
> + self._print(cmd)
> + resp = self.cmd_obj(cmd)
> + if resp is None:
> + raise QMPSendError("Disconnected")
> + if self._verbose:
> + self._print(resp)
> + if 'error' in resp:
> + raise QMPSendError(f"Command failed: {resp['error']}")
> +
> +
> +def raw_load(file: TextIO) -> List[QMPMessage]:
> + """parse a raw qmp command file.
> +
> + JSON formatted commands can expand on several lines but must
> + be separated by an end-of-line (two commands can not share the
> + same line).
> + File must not end with empty lines.
> + """
> + cmds: List[QMPMessage] = []
> + linecnt = 0
> + while True:
> + buf = file.readline()
> + if not buf:
> + return cmds
> + prev_err_pos = None
> + buf_linecnt = 1
> + while True:
> + try:
> + cmds.append(json.loads(buf))
> + break
> + except json.JSONDecodeError as err:
> + if prev_err_pos == err.pos:
> + # adding a line made no progress so
> + # + either we're at EOF and json data is truncated
> + # + or the parsing error is before
> + raise QmpRawDecodeError(err.msg, linecnt + err.lineno,
> + err.colno) from err
> + prev_err_pos = err.pos
> + buf += file.readline()
> + buf_linecnt += 1
> + linecnt += buf_linecnt
> +
> +
> +def report_error(msg: str) -> None:
> + """Write an error to stderr."""
> + sys.stderr.write('ERROR: %s\n' % msg)
> +
> +
> +def main() -> None:
> + """
> + qmp-send entry point: parse command line arguments and start the REPL.
> + """
> + parser = argparse.ArgumentParser(
> + description="""
> + Send raw qmp commands to qemu as long as they succeed. It either
> + connects to a remote qmp server using the provided socket or wrap
> + the qemu process. It stops sending the provided commands when a
> + command fails (disconnection or error response).
> + """,
> + epilog="""
> + When qemu wrap option is used, this script waits for qemu
> + to terminate but never send any quit or kill command. This
> + needs to be done manually.
> + """)
> +
> + parser.add_argument('-f', '--file', action='store',
> + help='Input file containing the commands')
> + parser.add_argument('-s', '--socket', action='store',
> + help='< UNIX socket path | TCP address:port >')
> + parser.add_argument('-v', '--verbose', action='store_true',
> + help='Verbose (echo commands sent and received)')
> + parser.add_argument('-p', '--pretty', action='store_true',
> + help='Pretty-print JSON')
> +
> + parser.add_argument('--wrap', nargs=argparse.REMAINDER,
> + help='QEMU command line to invoke')
> +
> + args = parser.parse_args()
> +
> + socket = args.socket
> + wrap_qemu = args.wrap is not None
> +
> + if wrap_qemu:
> + if len(args.wrap) != 0:
> + qemu_cmdline = args.wrap
> + else:
> + qemu_cmdline = ["qemu-system-x86_64"]
> + if socket is None:
> + socket = "qmp-send-wrap-%d" % os.getpid()
> + qemu_cmdline += ["-qmp", "unix:%s" % socket]
> +
> + try:
> + address = QMPSend.parse_address(socket)
> + except QMPBadPortError:
> + parser.error(f"Bad port number: {socket}")
> + return # pycharm doesn't know error() is noreturn
> +
> + try:
> + with open(args.file, mode='rt', encoding='utf8') as file:
> + qmp_cmds = raw_load(file)
> + except QmpRawDecodeError as err:
> + report_error(str(err))
> + sys.exit(1)
> +
> + try:
> + with QMPSend(address, args.pretty, args.verbose,
> + server=wrap_qemu) as qmp:
> + # starting with python 3.7 we could use contextlib.nullcontext
> + qemu = Popen(qemu_cmdline) if wrap_qemu else contextlib.suppress()
> + with qemu:
> + try:
> + qmp.setup_connection()
> + except ConnectError as err:
> + if isinstance(err.exc, OSError):
> + report_error(f"Couldn't connect to {socket}: {err!s}")
> + else:
> + report_error(str(err))
> + sys.exit(1)
> + try:
> + for cmd in qmp_cmds:
> + qmp.execute_cmd(cmd)
> + except QMPError as err:
> + report_error(str(err))
> + sys.exit(1)
> + finally:
> + if wrap_qemu:
> + os.unlink(socket)
> +
> +
> +if __name__ == '__main__':
> + main()
> diff --git a/scripts/qmp/qmp-send b/scripts/qmp/qmp-send
> new file mode 100755
> index 0000000000..8d3063797c
> --- /dev/null
> +++ b/scripts/qmp/qmp-send
> @@ -0,0 +1,11 @@
> +#!/usr/bin/env python3
> +
> +import os
> +import sys
> +
> +sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
> +from qemu.aqmp import qmp_send
> +
> +
> +if __name__ == '__main__':
> + qmp_send.main()
> --
> 2.35.1
>
Seems broadly fine to me, but I didn't review closely this time. If it
works for you, it works for me.
As for making QEMU hang: there's a few things you could do, take a
look at iotests and see how they handle timeout blocks in synchronous
code -- iotests.py line 696 or so, "class Timeout". When writing async
code, you can also do stuff like this:
async def foo():
await asyncio.wait_for(qmp.execute("some-command", args_etc), timeout=30)
See https://docs.python.org/3/library/asyncio-task.html#asyncio.wait_for
--js
On 4/4/22 22:34, John Snow wrote:
> On Wed, Mar 16, 2022 at 5:55 AM Damien Hedde <damien.hedde@greensocs.com> wrote:
>>
>> It takes an input file containing raw qmp commands (concatenated json
>> dicts) and send all commands one by one to a qmp server. When one
>> command fails, it exits.
>>
>> As a convenience, it can also wrap the qemu process to avoid having
>> to start qemu in background. When wrapping qemu, the program returns
>> only when the qemu process terminates.
>>
>> Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
>> ---
>>
>> Hi all,
>>
>> Following our discussion, I've started this. What do you think ?
>>
>> I tried to follow Daniel's qmp-shell-wrap. I think it is
>> better to have similar options (eg: logging). There is also room
>> for factorizing code if we want to keep them aligned and ease
>> maintenance.
>>
>> There are still some pylint issues (too many branches in main and it
>> does not like my context manager if else line). But it's kind of a
>> mess to fix theses so I think it's enough for a first version.
>
> Yeah, don't worry about these. You can just tell pylint to shut up
> while you prototype. Sometimes it's just not worth spending more time
> on a more beautiful factoring. Oh well.
>
>>
>> I name that qmp-send as Daniel proposed, maybe qmp-test matches better
>> what I'm doing there ?
>>
>
> I think I agree with Dan's response.
>
>> Thanks,
>> Damien
>> ---
>> python/qemu/aqmp/qmp_send.py | 229 +++++++++++++++++++++++++++++++++++
>
> I recommend putting this in qemu/util/qmp_send.py instead.
>
> I'm in the process of pulling out the AQMP lib and hosting it
> separately. Scripts like this I think should stay in the QEMU tree, so
> moving it to util instead is probably best. Otherwise, I'll *really*
> have to commit to the syntax, and that's probably a bigger hurdle than
> you want to deal with.
If it stays in QEMU tree, what licensing should I use ? LGPL does not
hurt, no ?
>
>> scripts/qmp/qmp-send | 11 ++
>> 2 files changed, 240 insertions(+)
>> create mode 100644 python/qemu/aqmp/qmp_send.py
>> create mode 100755 scripts/qmp/qmp-send
>>
>> diff --git a/python/qemu/aqmp/qmp_send.py b/python/qemu/aqmp/qmp_send.py
>> new file mode 100644
>> index 0000000000..cbca1d0205
>> --- /dev/null
>> +++ b/python/qemu/aqmp/qmp_send.py
>
> Seems broadly fine to me, but I didn't review closely this time. If it
> works for you, it works for me.
>
> As for making QEMU hang: there's a few things you could do, take a
> look at iotests and see how they handle timeout blocks in synchronous
> code -- iotests.py line 696 or so, "class Timeout". When writing async
> code, you can also do stuff like this:
>
> async def foo():
> await asyncio.wait_for(qmp.execute("some-command", args_etc), timeout=30)
>
> See https://docs.python.org/3/library/asyncio-task.html#asyncio.wait_for
>
> --js
>
Thanks for the tip,
--
Damien
On Tue, Apr 5, 2022, 5:03 AM Damien Hedde <damien.hedde@greensocs.com>
wrote:
>
>
> On 4/4/22 22:34, John Snow wrote:
> > On Wed, Mar 16, 2022 at 5:55 AM Damien Hedde <damien.hedde@greensocs.com>
> wrote:
> >>
> >> It takes an input file containing raw qmp commands (concatenated json
> >> dicts) and send all commands one by one to a qmp server. When one
> >> command fails, it exits.
> >>
> >> As a convenience, it can also wrap the qemu process to avoid having
> >> to start qemu in background. When wrapping qemu, the program returns
> >> only when the qemu process terminates.
> >>
> >> Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
> >> ---
> >>
> >> Hi all,
> >>
> >> Following our discussion, I've started this. What do you think ?
> >>
> >> I tried to follow Daniel's qmp-shell-wrap. I think it is
> >> better to have similar options (eg: logging). There is also room
> >> for factorizing code if we want to keep them aligned and ease
> >> maintenance.
> >>
> >> There are still some pylint issues (too many branches in main and it
> >> does not like my context manager if else line). But it's kind of a
> >> mess to fix theses so I think it's enough for a first version.
> >
> > Yeah, don't worry about these. You can just tell pylint to shut up
> > while you prototype. Sometimes it's just not worth spending more time
> > on a more beautiful factoring. Oh well.
> >
> >>
> >> I name that qmp-send as Daniel proposed, maybe qmp-test matches better
> >> what I'm doing there ?
> >>
> >
> > I think I agree with Dan's response.
> >
> >> Thanks,
> >> Damien
> >> ---
> >> python/qemu/aqmp/qmp_send.py | 229 +++++++++++++++++++++++++++++++++++
> >
> > I recommend putting this in qemu/util/qmp_send.py instead.
> >
> > I'm in the process of pulling out the AQMP lib and hosting it
> > separately. Scripts like this I think should stay in the QEMU tree, so
> > moving it to util instead is probably best. Otherwise, I'll *really*
> > have to commit to the syntax, and that's probably a bigger hurdle than
> > you want to deal with.
>
> If it stays in QEMU tree, what licensing should I use ? LGPL does not
> hurt, no ?
>
Whichever you please. GPLv2+ would be convenient and harmonizes well with
other tools. LGPL is only something I started doing so that the "qemu.qmp"
package would be LGPL. Licensing the tools as LGPL was just a sin of
convenience so I could claim a single license for the whole wheel/egg/tgz.
(I didn't want to make separate qmp and qmp-tools packages.)
Go with what you feel is best.
> >
> >> scripts/qmp/qmp-send | 11 ++
> >> 2 files changed, 240 insertions(+)
> >> create mode 100644 python/qemu/aqmp/qmp_send.py
> >> create mode 100755 scripts/qmp/qmp-send
> >>
> >> diff --git a/python/qemu/aqmp/qmp_send.py b/python/qemu/aqmp/qmp_send.py
> >> new file mode 100644
> >> index 0000000000..cbca1d0205
> >> --- /dev/null
> >> +++ b/python/qemu/aqmp/qmp_send.py
> >
> > Seems broadly fine to me, but I didn't review closely this time. If it
> > works for you, it works for me.
> >
> > As for making QEMU hang: there's a few things you could do, take a
> > look at iotests and see how they handle timeout blocks in synchronous
> > code -- iotests.py line 696 or so, "class Timeout". When writing async
> > code, you can also do stuff like this:
> >
> > async def foo():
> > await asyncio.wait_for(qmp.execute("some-command", args_etc),
> timeout=30)
> >
> > See https://docs.python.org/3/library/asyncio-task.html#asyncio.wait_for
> >
> > --js
> >
>
> Thanks for the tip,
> --
> Damien
>
Oh, and one more. the legacy.py bindings for AQMP also support a
configurable timeout that applies to most API calls by default.
see https://gitlab.com/jsnow/qemu.qmp/-/blob/main/qemu/qmp/legacy.py#L285
(Branch still in limbo here, but it should still be close to the same in
qemu.git)
I believe this is used by iotests.py when it sets up its machine.py
subclass ("VM", iirc) so that most qmp invocations in iotests have a
default timeout and won't hang tests indefinitely.
--js
>
John Snow <jsnow@redhat.com> writes: > On Tue, Apr 5, 2022, 5:03 AM Damien Hedde <damien.hedde@greensocs.com> > wrote: [...] >> If it stays in QEMU tree, what licensing should I use ? LGPL does not >> hurt, no ? >> > > Whichever you please. GPLv2+ would be convenient and harmonizes well with > other tools. LGPL is only something I started doing so that the "qemu.qmp" > package would be LGPL. Licensing the tools as LGPL was just a sin of > convenience so I could claim a single license for the whole wheel/egg/tgz. > > (I didn't want to make separate qmp and qmp-tools packages.) > > Go with what you feel is best. Any license other than GPLv2+ needs justification in the commit message. [...]
Damien Hedde <damien.hedde@greensocs.com> writes: > On 4/4/22 22:34, John Snow wrote: >> On Wed, Mar 16, 2022 at 5:55 AM Damien Hedde <damien.hedde@greensocs.com> wrote: [...] >> I recommend putting this in qemu/util/qmp_send.py instead. >> I'm in the process of pulling out the AQMP lib and hosting it >> separately. Scripts like this I think should stay in the QEMU tree, so >> moving it to util instead is probably best. Otherwise, I'll *really* >> have to commit to the syntax, and that's probably a bigger hurdle than >> you want to deal with. > > If it stays in QEMU tree, what licensing should I use ? LGPL does not > hurt, no ? GPLv2+ is the default, and for a reason. [...]
© 2016 - 2024 Red Hat, Inc.