Hi,

The main idea of this series is to be a bit more user-friendly when
using qmp-shell in a non-interactive way: with an input redirection
from a file containing a list of commands.

I'm working on dynamic qapi config of a qemu machine, this would
be very useful to provide and reproduce small examples.

This series proposes the following modifications:
+ no prompt when input is non-interactive
+ an --exit-on-error option so that the shell exits on first
  error (disconnection, command parsing error, response is an error)
+ support for comment lines and escaping eol to have more reability
  in the source files.

I tested this using QMPShell. I tried HMPShell but did not findout
how to successfully use it with qemu. How do I setup an HMPShell ?.

Another "issue" I have is the handling of integers. I
deal with a lot of addresses and reading/writing them as decimal is
a bit painful (json does not support hexadecimal integer format). Do
you think of any reasonable workaround for this ? Maybe HMP shell
support this ?

Thanks for your comments,
--
Damien

Damien Hedde (5):
  python: qmp_shell: don't prompt when stdin is non-interactive
  python: qmp_shell: refactor the parsing error handling
  python: qmp_shell: refactor disconnection handling
  python: qmp_shell: add -e/--exit-on-error option
  python: qmp_shell: handle comment lines and escaped eol

 python/qemu/aqmp/qmp_shell.py | 117 ++++++++++++++++++++++++----------
 1 file changed, 83 insertions(+), 34 deletions(-)

-- 
2.35.1

On Mon, Feb 21, 2022 at 10:55 AM Damien Hedde
<damien.hedde@greensocs.com> wrote:
>
> Hi,
>
> The main idea of this series is to be a bit more user-friendly when
> using qmp-shell in a non-interactive way: with an input redirection
> from a file containing a list of commands.
>
> I'm working on dynamic qapi config of a qemu machine, this would
> be very useful to provide and reproduce small examples.
>
> This series proposes the following modifications:
> + no prompt when input is non-interactive
> + an --exit-on-error option so that the shell exits on first
>   error (disconnection, command parsing error, response is an error)
> + support for comment lines and escaping eol to have more reability
>   in the source files.
>
> I tested this using QMPShell. I tried HMPShell but did not findout
> how to successfully use it with qemu. How do I setup an HMPShell ?.
>

Should be the same. It's just passing HMP commands through the QMP
layer using {"execute": "human-monitor-command", ...}.

> ./qmp-shell -H ../../bin/git/monitor.sock
Welcome to the HMP shell!
Connected to QEMU 6.2.50

(QEMU) help
announce_self [interfaces] [id] -- Trigger GARP/RARP announcements

... (and many more lines.)

HMP is a completely different interface, with different commands, etc.
It's meant for human users, but we are trying to remove any reasons to
use it. It has its uses for some debug commands that we don't want to
support over the QMP interface, and for some commands that aren't safe
and might freeze, etc. Dan has spent a lot of time and effort
migrating the save state commands over to QMP, for example.

> Another "issue" I have is the handling of integers. I
> deal with a lot of addresses and reading/writing them as decimal is
> a bit painful (json does not support hexadecimal integer format). Do
> you think of any reasonable workaround for this ? Maybe HMP shell
> support this ?

Take a look at `def _parse_value(cls, val: str) -> object:`

int("0x13") is going to raise an Exception, so this won't be perceived
as a numerical value. You can use int(val, base=0) to have Python
guess the base from the string given (which allows for "0x", "0b" and
maybe some others.) This might cause a regression if anyone was using
this to pass "0x13" as a string on purpose.

It might also be the case that the FuzzyJSON portion of the parser
already accepts hexadecimal numbers. You'd have to check, but if you
take a look at that piece of the code, you can see that we parse
"JSON" arguments as a Python AST and then swap out the true/false/null
literals for True/False/None. This gives some flexibility to the
syntax being entered here. I don't know off the top of my head what
happens to numbers here. This parser is only used when an argument
starts with '{' or '[', though.

This is related to another problem we have, which is that the
qom-set/qom-get scripts are pretty much extremely busted for anything
other than strings. The type safety of these interfaces is not ...
really there right now. In a beautiful future utopia, we'd be able to
get the type information we need directly from QEMU and
deterministically convert everything into the right form before
sending it out over the wire without guessing around.

--js

Damien Hedde <damien.hedde@greensocs.com> writes:

> Hi,
>
> The main idea of this series is to be a bit more user-friendly when
> using qmp-shell in a non-interactive way: with an input redirection
> from a file containing a list of commands.
>
> I'm working on dynamic qapi config of a qemu machine, this would
> be very useful to provide and reproduce small examples.

Why not use plain QMP for that?

[...]

On 2/22/22 07:10, Markus Armbruster wrote:
> Damien Hedde <damien.hedde@greensocs.com> writes:
> 
>> Hi,
>>
>> The main idea of this series is to be a bit more user-friendly when
>> using qmp-shell in a non-interactive way: with an input redirection
>> from a file containing a list of commands.
>>
>> I'm working on dynamic qapi config of a qemu machine, this would
>> be very useful to provide and reproduce small examples.
> 
> Why not use plain QMP for that?
> 
> [...]
> 
What do you mean by plain QMP ?

--
Damien

Damien Hedde <damien.hedde@greensocs.com> writes:

> On 2/22/22 07:10, Markus Armbruster wrote:
>> Damien Hedde <damien.hedde@greensocs.com> writes:
>> 
>>> Hi,
>>>
>>> The main idea of this series is to be a bit more user-friendly when
>>> using qmp-shell in a non-interactive way: with an input redirection
>>> from a file containing a list of commands.
>>>
>>> I'm working on dynamic qapi config of a qemu machine, this would
>>> be very useful to provide and reproduce small examples.
>> Why not use plain QMP for that?
>> [...]
>> 
> What do you mean by plain QMP ?

Talk straight to QEMU without a translator:

    $ cat script
    {"execute": "qmp_capabilities"}
    {"execute": "quit"}
    $ socat -t 3 STDIO UNIX-CONNECT:$HOME/work/images/test-qmp <script
    {"QMP": {"version": {"qemu": {"micro": 50, "minor": 2, "major": 6}, "package": "v6.2.0-1603-gc13b8e9973"}, "capabilities": ["oob"]}}
    {"error": {"class": "CommandNotFound", "desc": "Expecting capabilities negotiation with 'qmp_capabilities'"}}
    armbru@dusky:~/work/qemu$ echo -e '{"execute":"qmp_capabilities"}{"execute":"quit"}' >script
    armbru@dusky:~/work/qemu$ echo -e '{"execute":"qmp_capabilities"}\n{"execute":"quit"}' >script
    armbru@dusky:~/work/qemu$ socat -t 3 STDIO UNIX-CONNECT:$HOME/work/images/test-qmp <script
    {"QMP": {"version": {"qemu": {"micro": 50, "minor": 2, "major": 6}, "package": "v6.2.0-1603-gc13b8e9973"}, "capabilities": ["oob"]}}
    {"return": {}}
    {"return": {}}
    {"timestamp": {"seconds": 1645523438, "microseconds": 951702}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}

socat also supports interactive use nicely.  Try

    $ socat "READLINE,history=$HOME/.qmp_history,prompt=QMP>" UNIX-CONNECT:$HOME/path/to/socket

Helpfully blinks matching parenthesis for me.