This patch series introduces support for the Multi-Core Debug (MCD) API, a
commonly used debug interface by emulators. The MCD API, defined through a
header file, consists of 54 functions for implementing debug and trace.
However, since it is a header-file-only interface, MCD does not specify a
communication protocol. We get around this limitation by following a remote
procedure call approach using QMP. The client stub corresponding to this
implementation can be found at https://gitlab.com/lauterbach/mcdrefsrv

This series is the successor to:
"[PATCH v5 00/18] first version of mcdstub"
(https://patchew.org/QEMU/20231220162555.19545-1-nicolas.eder@lauterbach.com/)

* Architecture-independent MCD implementation
* QMP instead of custom TCP protocol

qemu-system-<arch> [options] -qmp tcp::1235,server=on,wait=off

* Architecture-independent QTest test suite

V=1 QTEST_QEMU_BINARY="./qemu-system-<arch> [options]" tests/qtest/mcd-test

* Architecture-specific tests can be found at the client stub

Mario Fleischmann (16):
  mcd: Introduce Multi-Core Debug (MCD) API
  mcd: Introduce MCD server
  mcd: Implement target initialization API
  mcd: Implement server connection API
  mcd: Implement target system query
  mcd: Implement core connection control
  mcd: Implement memory space query
  gdbstub: Expose GDBRegisterState
  mcd: Implement register query
  mcd: Implement runstate control
  mcd test: Implement core state query
  gdbstub: Expose gdb_write_register
  mcd: Implement register/memory access
  mcd: Implement single stepping
  mcd: Implement trigger control
  mcd: Implement reset control

 MAINTAINERS               |    9 +
 docs/interop/index.rst    |    1 +
 docs/interop/mcd.rst      |   58 +
 gdbstub/gdbstub.c         |   15 +-
 include/exec/gdbstub.h    |   18 +-
 mcd/libmcd_qapi.c         |  506 +++++
 mcd/libmcd_qapi.h         |   81 +
 mcd/mcd_api.h             | 3963 +++++++++++++++++++++++++++++++++++++
 mcd/mcdserver.c           | 2242 +++++++++++++++++++++
 mcd/mcdstub_qapi.c        |  974 +++++++++
 mcd/meson.build           |   12 +
 meson.build               |    1 +
 qapi/mcd.json             | 2366 ++++++++++++++++++++++
 qapi/meson.build          |    1 +
 qapi/qapi-schema.json     |    1 +
 tests/qtest/libmcd-test.c |  379 ++++
 tests/qtest/libmcd-test.h |   68 +
 tests/qtest/mcd-test.c    |  644 ++++++
 tests/qtest/meson.build   |    2 +
 19 files changed, 11331 insertions(+), 10 deletions(-)
 create mode 100644 docs/interop/mcd.rst
 create mode 100644 mcd/libmcd_qapi.c
 create mode 100644 mcd/libmcd_qapi.h
 create mode 100644 mcd/mcd_api.h
 create mode 100644 mcd/mcdserver.c
 create mode 100644 mcd/mcdstub_qapi.c
 create mode 100644 mcd/meson.build
 create mode 100644 qapi/mcd.json
 create mode 100644 tests/qtest/libmcd-test.c
 create mode 100644 tests/qtest/libmcd-test.h
 create mode 100644 tests/qtest/mcd-test.c

-- 
2.34.1

Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:

> This patch series introduces support for the Multi-Core Debug (MCD) API, a
> commonly used debug interface by emulators. The MCD API, defined through a
> header file, consists of 54 functions for implementing debug and trace.
> However, since it is a header-file-only interface, MCD does not specify a
> communication protocol. We get around this limitation by following a remote
> procedure call approach using QMP. The client stub corresponding to this
> implementation can be found at https://gitlab.com/lauterbach/mcdrefsrv
>
> This series is the successor to:
> "[PATCH v5 00/18] first version of mcdstub"
> (https://patchew.org/QEMU/20231220162555.19545-1-nicolas.eder@lauterbach.com/)
>
> * Architecture-independent MCD implementation
> * QMP instead of custom TCP protocol

Rationale?  There must be pros and cons.

How much data would you expect to flow in practical usage?  QMP isn't
designed for bulk transfer...

> qemu-system-<arch> [options] -qmp tcp::1235,server=on,wait=off
>
> * Architecture-independent QTest test suite
>
> V=1 QTEST_QEMU_BINARY="./qemu-system-<arch> [options]" tests/qtest/mcd-test
>
> * Architecture-specific tests can be found at the client stub

[...]

>  qapi/mcd.json             | 2366 ++++++++++++++++++++++

This is *massive*.  By non-blank, non-comment lines, it's the second
largest module in qapi/, almost 9% of the entire schema.  It's larger
than the entire QEMU guest agent QAPI schema.  The QAPI generator
generates some 280KiB of C code for it.

[...]

Thanks a lot for the response, I really appreciate your time.

On 07.04.2025 14:33, Markus Armbruster wrote:

> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
> 
>> This patch series introduces support for the Multi-Core Debug (MCD) API, a
>> commonly used debug interface by emulators. The MCD API, defined through a
>> header file, consists of 54 functions for implementing debug and trace.
>> However, since it is a header-file-only interface, MCD does not specify a
>> communication protocol. We get around this limitation by following a remote
>> procedure call approach using QMP. The client stub corresponding to this
>> implementation can be found at https://gitlab.com/lauterbach/mcdrefsrv
>>
>> This series is the successor to:
>> "[PATCH v5 00/18] first version of mcdstub"
>> (https://patchew.org/QEMU/20231220162555.19545-1-nicolas.eder@lauterbach.com/)
>>
>> * Architecture-independent MCD implementation
>> * QMP instead of custom TCP protocol
> 
> Rationale?  There must be pros and cons.

Assuming you're referring to the protocol of the previous patch series:
The previous TCP protocol only supported a subset of MCD. As the 
implementation progresses, the protocol eventually needs to be extended, 
possibly resulting in backwards compatibility problems.
Following an RPC approach and keeping the communication layer as close 
to the MCD API as possible results in a larger protocol at first, but 
does not need to be changed afterwards.
By directly mapping MCD functions onto QMP commands, the complexity in 
the server and client stubs can be minimized.

Assuming you're referring to the QMP choice:
QMP is being described as the "protocol which allows applications to 
control a QEMU instance".
It provides a RPC framework which automatically (de)serializes methods 
and their parameters, even inside QTests.
The whole interface is automatically documented.

> How much data would you expect to flow in practical usage?  QMP isn't
> designed for bulk transfer...

According to ifstat, the expected data rate in practical usage is around

KB/s in  KB/s out
    100      100

I fully understand your concern and agree that a JSON-based
protocol does not result in the lowest data rate.

If the data rate is the highest priority: *Before* the QMP supported was 
implemented, the MCD interface was built on a custom RPC framework, 
generated with the code generator at:

https://gitlab.com/lauterbach/mcdrefsrv/-/tree/main/codegen

The resulting header file was basically a set of functions capable of 
serializing MCD's function arguments into a byte stream and vice-versa:

https://gitlab.com/lauterbach/mcdrefsrv/-/blob/df754cef7f19ece2d00b6ce4e307ba37e91e5dcb/include/mcd_rpc.h

The QMP support was added because of the advantages listed above and in 
order to evade yet another custom communication protocol.
As a user of the MCD interface, I haven't noticed any negative impact of 
the increased data rate in realistic debugging scenarios, even when 
trying to drive the data rate up. If that would have been the case, I 
would have sent this patch request with our custom RPC protocol.

>> qemu-system-<arch> [options] -qmp tcp::1235,server=on,wait=off
>>
>> * Architecture-independent QTest test suite
>>
>> V=1 QTEST_QEMU_BINARY="./qemu-system-<arch> [options]" tests/qtest/mcd-test
>>
>> * Architecture-specific tests can be found at the client stub
> 
> [...]
> 
>>   qapi/mcd.json             | 2366 ++++++++++++++++++++++
> 
> This is *massive*.  By non-blank, non-comment lines, it's the second
> largest module in qapi/, almost 9% of the entire schema.  It's larger
> than the entire QEMU guest agent QAPI schema.  The QAPI generator
> generates some 280KiB of C code for it.

I understand your point and I think it touches on the point made above 
regarding MCD's complexity:

> mcd/mcd_api.h             | 3963 +++++++++++++++++++++++++++++++++++++

I hope that we agree that RPC is generally the right approach to 
implement MCD. As far as the implementation is concerned, I'm open to 
any suggestion you have. I've always avoided to introduce any 
unnecessary external dependencies.

Best regards,
Mario

Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:

> Thanks a lot for the response, I really appreciate your time.
>
> On 07.04.2025 14:33, Markus Armbruster wrote:
>
>> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
>> 
>>> This patch series introduces support for the Multi-Core Debug (MCD) API, a
>>> commonly used debug interface by emulators. The MCD API, defined through a
>>> header file, consists of 54 functions for implementing debug and trace.
>>> However, since it is a header-file-only interface, MCD does not specify a
>>> communication protocol. We get around this limitation by following a remote
>>> procedure call approach using QMP. The client stub corresponding to this
>>> implementation can be found at https://gitlab.com/lauterbach/mcdrefsrv
>>>
>>> This series is the successor to:
>>> "[PATCH v5 00/18] first version of mcdstub"
>>> (https://patchew.org/QEMU/20231220162555.19545-1-nicolas.eder@lauterbach.com/)
>>>
>>> * Architecture-independent MCD implementation
>>> * QMP instead of custom TCP protocol
>> 
>> Rationale?  There must be pros and cons.
>
> Assuming you're referring to the protocol of the previous patch series:
> The previous TCP protocol only supported a subset of MCD. As the 
> implementation progresses, the protocol eventually needs to be extended, 
> possibly resulting in backwards compatibility problems.
> Following an RPC approach and keeping the communication layer as close 
> to the MCD API as possible results in a larger protocol at first, but 
> does not need to be changed afterwards.
> By directly mapping MCD functions onto QMP commands, the complexity in 
> the server and client stubs can be minimized.
>
> Assuming you're referring to the QMP choice:
> QMP is being described as the "protocol which allows applications to 
> control a QEMU instance".
> It provides a RPC framework which automatically (de)serializes methods 
> and their parameters, even inside QTests.
> The whole interface is automatically documented.

Let's see whether I understand.

MCD is an established C interface.

Your goal is to provide remote MCD for QEMU, i.e. the client uses the
MCD C interface, and the interface's implementation talks to an MCD
server integrated into QEMU via some remote transport.

The previous version connects the two with a bespoke protocol via TCP.
The client software translates between the C interface and this
protocol.  QEMU implements the protocol's server side.  Designing and
maintaining a protocol is expensive.

This versions makes two changes:

1. Instead of layering a protocol on top of MCD, you use MCD directly.
This eliminates protocol design and maintenance.  Moreover, translation
becomes straightforward marshaling / unmarshaling for the transport.

2. You use QMP as a transport.  This gets you marshaling / unmarshaling
for free.  It also provides some useful infrastructure for tests,
documentation and such.

Fair?

>> How much data would you expect to flow in practical usage?  QMP isn't
>> designed for bulk transfer...
>
> According to ifstat, the expected data rate in practical usage is around
>
> KB/s in  KB/s out
>     100      100
>
> I fully understand your concern and agree that a JSON-based
> protocol does not result in the lowest data rate.
>
> If the data rate is the highest priority: *Before* the QMP supported was 
> implemented, the MCD interface was built on a custom RPC framework, 
> generated with the code generator at:
>
> https://gitlab.com/lauterbach/mcdrefsrv/-/tree/main/codegen
>
> The resulting header file was basically a set of functions capable of 
> serializing MCD's function arguments into a byte stream and vice-versa:
>
> https://gitlab.com/lauterbach/mcdrefsrv/-/blob/df754cef7f19ece2d00b6ce4e307ba37e91e5dcb/include/mcd_rpc.h
>
> The QMP support was added because of the advantages listed above and in 
> order to evade yet another custom communication protocol.
> As a user of the MCD interface, I haven't noticed any negative impact of 
> the increased data rate in realistic debugging scenarios, even when 
> trying to drive the data rate up. If that would have been the case, I 
> would have sent this patch request with our custom RPC protocol.

I see.

>>> qemu-system-<arch> [options] -qmp tcp::1235,server=on,wait=off
>>>
>>> * Architecture-independent QTest test suite
>>>
>>> V=1 QTEST_QEMU_BINARY="./qemu-system-<arch> [options]" tests/qtest/mcd-test
>>>
>>> * Architecture-specific tests can be found at the client stub
>> 
>> [...]
>> 
>>>   qapi/mcd.json             | 2366 ++++++++++++++++++++++
>> 
>> This is *massive*.  By non-blank, non-comment lines, it's the second
>> largest module in qapi/, almost 9% of the entire schema.  It's larger
>> than the entire QEMU guest agent QAPI schema.  The QAPI generator
>> generates some 280KiB of C code for it.
>
> I understand your point and I think it touches on the point made above 
> regarding MCD's complexity:
>
>> mcd/mcd_api.h             | 3963 +++++++++++++++++++++++++++++++++++++

Uh, that's a big one.

Out of curiosity, what's the size of the previous version's code to
translate between the C interface and TCP?

> I hope that we agree that RPC is generally the right approach to 
> implement MCD. As far as the implementation is concerned, I'm open to 
> any suggestion you have. I've always avoided to introduce any 
> unnecessary external dependencies.

I think you're much better qualified to judge the merits of RPC here
than I am.  That leaves the question of the RPC transport.  You want to
use QMP.

On the one hand, I'm tickled to see QAPI/QMP used for things it wasn't
designed for.

On the other hand, QMP has grown so big.  Keeping it cohesive has become
a near-impossible mission.

Hmm.

We already provide another remote debugger interface: the GDB stub.
It's optional, i.e. the user has to create it, either with command line
option -gdb, or monitor command gdbserver.

We already have two-and-a-half QMPs: qemu-system-FOO's QMP,
qemu-storage-daemon's QMP (subset of the previous, so it counts only
half), and qemu-ga's QMP-like protocol.

What about providing the MCD interface as a separate QMP-like protocol?

It gets its own QAPI schema, just like for qemu-ga.  Simplifies
compiling it out when not needed.

It gets its own socket, just like the GDB stub.  Might reduce
interference between debugging and QMP.

Thoughts?  Alex, Philippe, care to chime in?

Apologies for the line wrapping in yesterday's answer. Should be fixed now.

On 08.04.2025 09:00, Markus Armbruster wrote:
> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
> 
>> Thanks a lot for the response, I really appreciate your time.
>>
>> On 07.04.2025 14:33, Markus Armbruster wrote:
>>
>>> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
>>>
>>>> This patch series introduces support for the Multi-Core Debug (MCD) API, a
>>>> commonly used debug interface by emulators. The MCD API, defined through a
>>>> header file, consists of 54 functions for implementing debug and trace.
>>>> However, since it is a header-file-only interface, MCD does not specify a
>>>> communication protocol. We get around this limitation by following a remote
>>>> procedure call approach using QMP. The client stub corresponding to this
>>>> implementation can be found at https://gitlab.com/lauterbach/mcdrefsrv
>>>>
>>>> This series is the successor to:
>>>> "[PATCH v5 00/18] first version of mcdstub"
>>>> (https://patchew.org/QEMU/20231220162555.19545-1-nicolas.eder@lauterbach.com/)
>>>>
>>>> * Architecture-independent MCD implementation
>>>> * QMP instead of custom TCP protocol
>>>
>>> Rationale?  There must be pros and cons.
>>
>> Assuming you're referring to the protocol of the previous patch series:
>> The previous TCP protocol only supported a subset of MCD. As the 
>> implementation progresses, the protocol eventually needs to be extended, 
>> possibly resulting in backwards compatibility problems.
>> Following an RPC approach and keeping the communication layer as close 
>> to the MCD API as possible results in a larger protocol at first, but 
>> does not need to be changed afterwards.
>> By directly mapping MCD functions onto QMP commands, the complexity in 
>> the server and client stubs can be minimized.
>>
>> Assuming you're referring to the QMP choice:
>> QMP is being described as the "protocol which allows applications to 
>> control a QEMU instance".
>> It provides a RPC framework which automatically (de)serializes methods 
>> and their parameters, even inside QTests.
>> The whole interface is automatically documented.
> 
> Let's see whether I understand.
> 
> MCD is an established C interface.
> 
> Your goal is to provide remote MCD for QEMU, i.e. the client uses the
> MCD C interface, and the interface's implementation talks to an MCD
> server integrated into QEMU via some remote transport.
> 
> The previous version connects the two with a bespoke protocol via TCP.
> The client software translates between the C interface and this
> protocol.  QEMU implements the protocol's server side.  Designing and
> maintaining a protocol is expensive.
> 
> This versions makes two changes:
> 
> 1. Instead of layering a protocol on top of MCD, you use MCD directly.
> This eliminates protocol design and maintenance.  Moreover, translation
> becomes straightforward marshaling / unmarshaling for the transport.
> 
> 2. You use QMP as a transport.  This gets you marshaling / unmarshaling
> for free.  It also provides some useful infrastructure for tests,
> documentation and such.
> 
> Fair?

Couldn't have put it better myself.

>>> How much data would you expect to flow in practical usage?  QMP isn't
>>> designed for bulk transfer...
>>
>> According to ifstat, the expected data rate in practical usage is around
>>
>> KB/s in  KB/s out
>>     100      100
>>
>> I fully understand your concern and agree that a JSON-based
>> protocol does not result in the lowest data rate.
>>
>> If the data rate is the highest priority: *Before* the QMP supported was 
>> implemented, the MCD interface was built on a custom RPC framework, 
>> generated with the code generator at:
>>
>> https://gitlab.com/lauterbach/mcdrefsrv/-/tree/main/codegen
>>
>> The resulting header file was basically a set of functions capable of 
>> serializing MCD's function arguments into a byte stream and vice-versa:
>>
>> https://gitlab.com/lauterbach/mcdrefsrv/-/blob/df754cef7f19ece2d00b6ce4e307ba37e91e5dcb/include/mcd_rpc.h
>>
>> The QMP support was added because of the advantages listed above and in 
>> order to evade yet another custom communication protocol.
>> As a user of the MCD interface, I haven't noticed any negative impact of 
>> the increased data rate in realistic debugging scenarios, even when 
>> trying to drive the data rate up. If that would have been the case, I 
>> would have sent this patch request with our custom RPC protocol.
> 
> I see.
> 
>>>> qemu-system-<arch> [options] -qmp tcp::1235,server=on,wait=off
>>>>
>>>> * Architecture-independent QTest test suite
>>>>
>>>> V=1 QTEST_QEMU_BINARY="./qemu-system-<arch> [options]" tests/qtest/mcd-test
>>>>
>>>> * Architecture-specific tests can be found at the client stub
>>>
>>> [...]
>>>
>>>>   qapi/mcd.json             | 2366 ++++++++++++++++++++++
>>>
>>> This is *massive*.  By non-blank, non-comment lines, it's the second
>>> largest module in qapi/, almost 9% of the entire schema.  It's larger
>>> than the entire QEMU guest agent QAPI schema.  The QAPI generator
>>> generates some 280KiB of C code for it.
>>
>> I understand your point and I think it touches on the point made above 
>> regarding MCD's complexity:
>>
>>> mcd/mcd_api.h             | 3963 +++++++++++++++++++++++++++++++++++++
> 
> Uh, that's a big one.
> 
> Out of curiosity, what's the size of the previous version's code to
> translate between the C interface and TCP?

The previous version's protocol was very similar to GDB's remote serial
protocol which is why the size of its implementation is comparable to
that of gdbstub.c

> debug/mcdstub/mcdstub.c                  | 2481 ++++++++++++++++++++++

Note that this file contains both the transport layer as well as the
implementation and does not implement mcd_api.h which makes the two
patch sets difficult to compare.

>> I hope that we agree that RPC is generally the right approach to 
>> implement MCD. As far as the implementation is concerned, I'm open to 
>> any suggestion you have. I've always avoided to introduce any 
>> unnecessary external dependencies.
> 
> I think you're much better qualified to judge the merits of RPC here
> than I am.  That leaves the question of the RPC transport.  You want to
> use QMP.

I think that QMP is the more mature protocol than my RPC
proof-of-concept and would improve interoperability. However, our
open-source client stub supports both transport layers, so we're fine
with both.

> On the one hand, I'm tickled to see QAPI/QMP used for things it wasn't
> designed for.

Might be a language issue but I can't tell whether that's meant
positively or negatively.

> On the other hand, QMP has grown so big.  Keeping it cohesive has become
> a near-impossible mission.
> 
> Hmm.
> 
> We already provide another remote debugger interface: the GDB stub.
> It's optional, i.e. the user has to create it, either with command line
> option -gdb, or monitor command gdbserver.

We had that option in our previous patch set:

> $ qemu-system-arm -M virt -cpu cortex-a15 -mcd tcp::1235

If you'd like, we can of course add such an option again. The QMP choice
made that redundant for me because I wasn't aware of a way to only
provide a subset of QMP.

> We already have two-and-a-half QMPs: qemu-system-FOO's QMP,
> qemu-storage-daemon's QMP (subset of the previous, so it counts only
> half), and qemu-ga's QMP-like protocol.
> 
> What about providing the MCD interface as a separate QMP-like protocol?
> It gets its own QAPI schema, just like for qemu-ga.  Simplifies
> compiling it out when not needed.
> It gets its own socket, just like the GDB stub.  Might reduce
> interference between debugging and QMP.
> 
> Thoughts?  Alex, Philippe, care to chime in?

Sound reasonable to me. Keeping in mind the size of generated QAPI code,
an option to `./configure [...] --enable-mcd` is definitely advisable.

Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:

> Apologies for the line wrapping in yesterday's answer. Should be fixed now.
>
> On 08.04.2025 09:00, Markus Armbruster wrote:
>> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
>> 
>>> Thanks a lot for the response, I really appreciate your time.
>>>
>>> On 07.04.2025 14:33, Markus Armbruster wrote:
>>>
>>>> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
>>>>
>>>>> This patch series introduces support for the Multi-Core Debug (MCD) API, a
>>>>> commonly used debug interface by emulators. The MCD API, defined through a
>>>>> header file, consists of 54 functions for implementing debug and trace.
>>>>> However, since it is a header-file-only interface, MCD does not specify a
>>>>> communication protocol. We get around this limitation by following a remote
>>>>> procedure call approach using QMP. The client stub corresponding to this
>>>>> implementation can be found at https://gitlab.com/lauterbach/mcdrefsrv
>>>>>
>>>>> This series is the successor to:
>>>>> "[PATCH v5 00/18] first version of mcdstub"
>>>>> (https://patchew.org/QEMU/20231220162555.19545-1-nicolas.eder@lauterbach.com/)
>>>>>
>>>>> * Architecture-independent MCD implementation
>>>>> * QMP instead of custom TCP protocol
>>>>
>>>> Rationale?  There must be pros and cons.
>>>
>>> Assuming you're referring to the protocol of the previous patch series:
>>> The previous TCP protocol only supported a subset of MCD. As the 
>>> implementation progresses, the protocol eventually needs to be extended, 
>>> possibly resulting in backwards compatibility problems.
>>> Following an RPC approach and keeping the communication layer as close 
>>> to the MCD API as possible results in a larger protocol at first, but 
>>> does not need to be changed afterwards.
>>> By directly mapping MCD functions onto QMP commands, the complexity in 
>>> the server and client stubs can be minimized.
>>>
>>> Assuming you're referring to the QMP choice:
>>> QMP is being described as the "protocol which allows applications to 
>>> control a QEMU instance".
>>> It provides a RPC framework which automatically (de)serializes methods 
>>> and their parameters, even inside QTests.
>>> The whole interface is automatically documented.
>> 
>> Let's see whether I understand.
>> 
>> MCD is an established C interface.
>> 
>> Your goal is to provide remote MCD for QEMU, i.e. the client uses the
>> MCD C interface, and the interface's implementation talks to an MCD
>> server integrated into QEMU via some remote transport.
>> 
>> The previous version connects the two with a bespoke protocol via TCP.
>> The client software translates between the C interface and this
>> protocol.  QEMU implements the protocol's server side.  Designing and
>> maintaining a protocol is expensive.
>> 
>> This versions makes two changes:
>> 
>> 1. Instead of layering a protocol on top of MCD, you use MCD directly.
>> This eliminates protocol design and maintenance.  Moreover, translation
>> becomes straightforward marshaling / unmarshaling for the transport.
>> 
>> 2. You use QMP as a transport.  This gets you marshaling / unmarshaling
>> for free.  It also provides some useful infrastructure for tests,
>> documentation and such.
>> 
>> Fair?
>
> Couldn't have put it better myself.
>
>>>> How much data would you expect to flow in practical usage?  QMP isn't
>>>> designed for bulk transfer...
>>>
>>> According to ifstat, the expected data rate in practical usage is around
>>>
>>> KB/s in  KB/s out
>>>     100      100
>>>
>>> I fully understand your concern and agree that a JSON-based
>>> protocol does not result in the lowest data rate.
>>>
>>> If the data rate is the highest priority: *Before* the QMP supported was 
>>> implemented, the MCD interface was built on a custom RPC framework, 
>>> generated with the code generator at:
>>>
>>> https://gitlab.com/lauterbach/mcdrefsrv/-/tree/main/codegen
>>>
>>> The resulting header file was basically a set of functions capable of 
>>> serializing MCD's function arguments into a byte stream and vice-versa:
>>>
>>> https://gitlab.com/lauterbach/mcdrefsrv/-/blob/df754cef7f19ece2d00b6ce4e307ba37e91e5dcb/include/mcd_rpc.h
>>>
>>> The QMP support was added because of the advantages listed above and in 
>>> order to evade yet another custom communication protocol.
>>> As a user of the MCD interface, I haven't noticed any negative impact of 
>>> the increased data rate in realistic debugging scenarios, even when 
>>> trying to drive the data rate up. If that would have been the case, I 
>>> would have sent this patch request with our custom RPC protocol.
>> 
>> I see.
>> 
>>>>> qemu-system-<arch> [options] -qmp tcp::1235,server=on,wait=off
>>>>>
>>>>> * Architecture-independent QTest test suite
>>>>>
>>>>> V=1 QTEST_QEMU_BINARY="./qemu-system-<arch> [options]" tests/qtest/mcd-test
>>>>>
>>>>> * Architecture-specific tests can be found at the client stub
>>>>
>>>> [...]
>>>>
>>>>>   qapi/mcd.json             | 2366 ++++++++++++++++++++++
>>>>
>>>> This is *massive*.  By non-blank, non-comment lines, it's the second
>>>> largest module in qapi/, almost 9% of the entire schema.  It's larger
>>>> than the entire QEMU guest agent QAPI schema.  The QAPI generator
>>>> generates some 280KiB of C code for it.
>>>
>>> I understand your point and I think it touches on the point made above 
>>> regarding MCD's complexity:
>>>
>>>> mcd/mcd_api.h             | 3963 +++++++++++++++++++++++++++++++++++++
>> 
>> Uh, that's a big one.
>> 
>> Out of curiosity, what's the size of the previous version's code to
>> translate between the C interface and TCP?
>
> The previous version's protocol was very similar to GDB's remote serial
> protocol which is why the size of its implementation is comparable to
> that of gdbstub.c
>
>> debug/mcdstub/mcdstub.c                  | 2481 ++++++++++++++++++++++
>
> Note that this file contains both the transport layer as well as the
> implementation and does not implement mcd_api.h which makes the two
> patch sets difficult to compare.

Understood.

>>> I hope that we agree that RPC is generally the right approach to 
>>> implement MCD. As far as the implementation is concerned, I'm open to 
>>> any suggestion you have. I've always avoided to introduce any 
>>> unnecessary external dependencies.
>> 
>> I think you're much better qualified to judge the merits of RPC here
>> than I am.  That leaves the question of the RPC transport.  You want to
>> use QMP.
>
> I think that QMP is the more mature protocol than my RPC
> proof-of-concept and would improve interoperability. However, our
> open-source client stub supports both transport layers, so we're fine
> with both.
>
>> On the one hand, I'm tickled to see QAPI/QMP used for things it wasn't
>> designed for.
>
> Might be a language issue but I can't tell whether that's meant
> positively or negatively.

Positively!

>> On the other hand, QMP has grown so big.  Keeping it cohesive has become
>> a near-impossible mission.
>> 
>> Hmm.
>> 
>> We already provide another remote debugger interface: the GDB stub.
>> It's optional, i.e. the user has to create it, either with command line
>> option -gdb, or monitor command gdbserver.
>
> We had that option in our previous patch set:
>
>> $ qemu-system-arm -M virt -cpu cortex-a15 -mcd tcp::1235
>
> If you'd like, we can of course add such an option again. The QMP choice
> made that redundant for me because I wasn't aware of a way to only
> provide a subset of QMP.

Or rather several QMPs.

>> We already have two-and-a-half QMPs: qemu-system-FOO's QMP,
>> qemu-storage-daemon's QMP (subset of the previous, so it counts only
>> half), and qemu-ga's QMP-like protocol.

qemu-ga's protocol differs in certain details for historical reasons.  A
new one should not.

QMP supports introspection via query-qmp-schema.  This enables clients
to deal with a wide range of server versions.  You don't need this if
your interface never changes (a rather bold assertion), or your client
always targets one specific version.  If you need it, I'm happy to
advise on how to use it.

>> What about providing the MCD interface as a separate QMP-like protocol?
>> It gets its own QAPI schema, just like for qemu-ga.  Simplifies
>> compiling it out when not needed.
>>
>> It gets its own socket, just like the GDB stub.  Might reduce
>> interference between debugging and QMP.
>> 
>> Thoughts?  Alex, Philippe, care to chime in?
>
> Sound reasonable to me. Keeping in mind the size of generated QAPI code,
> an option to `./configure [...] --enable-mcd` is definitely advisable.

Alex, Philippe?

Markus Armbruster <armbru@redhat.com> writes:

> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
>
>> Apologies for the line wrapping in yesterday's answer. Should be fixed now.
>>
>> On 08.04.2025 09:00, Markus Armbruster wrote:
>>> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
>>> 
>>>> Thanks a lot for the response, I really appreciate your time.
>>>>
>>>> On 07.04.2025 14:33, Markus Armbruster wrote:
>>>>
>>>>> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
>>>>>
>>>>>> This patch series introduces support for the Multi-Core Debug (MCD) API, a
>>>>>> commonly used debug interface by emulators. The MCD API, defined through a
>>>>>> header file, consists of 54 functions for implementing debug and trace.
>>>>>> However, since it is a header-file-only interface, MCD does not specify a
>>>>>> communication protocol. We get around this limitation by following a remote
>>>>>> procedure call approach using QMP. The client stub corresponding to this
>>>>>> implementation can be found at https://gitlab.com/lauterbach/mcdrefsrv
>>>>>>
>>>>>> This series is the successor to:
>>>>>> "[PATCH v5 00/18] first version of mcdstub"
>>>>>> (https://patchew.org/QEMU/20231220162555.19545-1-nicolas.eder@lauterbach.com/)
>>>>>>
>>>>>> * Architecture-independent MCD implementation
>>>>>> * QMP instead of custom TCP protocol
>>>>>
>>>>> Rationale?  There must be pros and cons.
>>>>
>>>> Assuming you're referring to the protocol of the previous patch series:
>>>> The previous TCP protocol only supported a subset of MCD. As the 
>>>> implementation progresses, the protocol eventually needs to be extended, 
>>>> possibly resulting in backwards compatibility problems.
>>>> Following an RPC approach and keeping the communication layer as close 
>>>> to the MCD API as possible results in a larger protocol at first, but 
>>>> does not need to be changed afterwards.
>>>> By directly mapping MCD functions onto QMP commands, the complexity in 
>>>> the server and client stubs can be minimized.
>>>>
>>>> Assuming you're referring to the QMP choice:
>>>> QMP is being described as the "protocol which allows applications to 
>>>> control a QEMU instance".
>>>> It provides a RPC framework which automatically (de)serializes methods 
>>>> and their parameters, even inside QTests.
>>>> The whole interface is automatically documented.
>>> 
>>> Let's see whether I understand.
>>> 
>>> MCD is an established C interface.
>>> 
>>> Your goal is to provide remote MCD for QEMU, i.e. the client uses the
>>> MCD C interface, and the interface's implementation talks to an MCD
>>> server integrated into QEMU via some remote transport.
>>> 
>>> The previous version connects the two with a bespoke protocol via TCP.
>>> The client software translates between the C interface and this
>>> protocol.  QEMU implements the protocol's server side.  Designing and
>>> maintaining a protocol is expensive.
>>> 
>>> This versions makes two changes:
>>> 
>>> 1. Instead of layering a protocol on top of MCD, you use MCD directly.
>>> This eliminates protocol design and maintenance.  Moreover, translation
>>> becomes straightforward marshaling / unmarshaling for the transport.
>>> 
>>> 2. You use QMP as a transport.  This gets you marshaling / unmarshaling
>>> for free.  It also provides some useful infrastructure for tests,
>>> documentation and such.
>>> 
>>> Fair?
>>
>> Couldn't have put it better myself.
>>
<snip>
>>> What about providing the MCD interface as a separate QMP-like protocol?
>>> It gets its own QAPI schema, just like for qemu-ga.  Simplifies
>>> compiling it out when not needed.
>>>
>>> It gets its own socket, just like the GDB stub.  Might reduce
>>> interference between debugging and QMP.
>>> 
>>> Thoughts?  Alex, Philippe, care to chime in?
>>
>> Sound reasonable to me. Keeping in mind the size of generated QAPI code,
>> an option to `./configure [...] --enable-mcd` is definitely advisable.
>
> Alex, Philippe?

When I spoke to Mario at DVCon last year I liked the idea of re-using
QMP instead of inventing yet another RPC interface for QEMU. QMP
certainly has nicer properties than the gdbstub which has a very
"organic" and "serial" feel to it.

Are you suggesting we re-use the machinery but use an entirely separate
socket with just the MCD namespace in it? I don't see that being a
problem as long as we can test it properly in the CI.

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Alex Bennée <alex.bennee@linaro.org> writes:

> Markus Armbruster <armbru@redhat.com> writes:
>
>> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
>>
>>> Apologies for the line wrapping in yesterday's answer. Should be fixed now.
>>>
>>> On 08.04.2025 09:00, Markus Armbruster wrote:

[...]

>>>> What about providing the MCD interface as a separate QMP-like protocol?
>>>> It gets its own QAPI schema, just like for qemu-ga.  Simplifies
>>>> compiling it out when not needed.
>>>>
>>>> It gets its own socket, just like the GDB stub.  Might reduce
>>>> interference between debugging and QMP.
>>>> 
>>>> Thoughts?  Alex, Philippe, care to chime in?
>>>
>>> Sound reasonable to me. Keeping in mind the size of generated QAPI code,
>>> an option to `./configure [...] --enable-mcd` is definitely advisable.
>>
>> Alex, Philippe?
>
> When I spoke to Mario at DVCon last year I liked the idea of re-using
> QMP instead of inventing yet another RPC interface for QEMU. QMP
> certainly has nicer properties than the gdbstub which has a very
> "organic" and "serial" feel to it.
>
> Are you suggesting we re-use the machinery but use an entirely separate
> socket with just the MCD namespace in it? I don't see that being a
> problem as long as we can test it properly in the CI.

Yes.

"Keep them separate" is only a gut feeling, though.  While I pay
attention to my gut feelings, I know they can be wrong.  I am soliciting
opinions.

Markus Armbruster <armbru@redhat.com> writes:

> Alex Bennée <alex.bennee@linaro.org> writes:
>
>> Markus Armbruster <armbru@redhat.com> writes:
>>
>>> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
>>>
>>>> Apologies for the line wrapping in yesterday's answer. Should be fixed now.
>>>>
>>>> On 08.04.2025 09:00, Markus Armbruster wrote:
>
> [...]
>
>>>>> What about providing the MCD interface as a separate QMP-like protocol?
>>>>> It gets its own QAPI schema, just like for qemu-ga.  Simplifies
>>>>> compiling it out when not needed.
>>>>>
>>>>> It gets its own socket, just like the GDB stub.  Might reduce
>>>>> interference between debugging and QMP.
>>>>> 
>>>>> Thoughts?  Alex, Philippe, care to chime in?
>>>>
>>>> Sound reasonable to me. Keeping in mind the size of generated QAPI code,
>>>> an option to `./configure [...] --enable-mcd` is definitely advisable.
>>>
>>> Alex, Philippe?
>>
>> When I spoke to Mario at DVCon last year I liked the idea of re-using
>> QMP instead of inventing yet another RPC interface for QEMU. QMP
>> certainly has nicer properties than the gdbstub which has a very
>> "organic" and "serial" feel to it.
>>
>> Are you suggesting we re-use the machinery but use an entirely separate
>> socket with just the MCD namespace in it? I don't see that being a
>> problem as long as we can test it properly in the CI.
>
> Yes.
>
> "Keep them separate" is only a gut feeling, though.  While I pay
> attention to my gut feelings, I know they can be wrong.  I am soliciting
> opinions.

I forgot to add isn't the flexibility of the QMP API something we need
to handle for single binary anyway?

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Alex Bennée <alex.bennee@linaro.org> writes:

> Markus Armbruster <armbru@redhat.com> writes:
>
>> Alex Bennée <alex.bennee@linaro.org> writes:
>>
>>> Markus Armbruster <armbru@redhat.com> writes:
>>>
>>>> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
>>>>
>>>>> Apologies for the line wrapping in yesterday's answer. Should be fixed now.
>>>>>
>>>>> On 08.04.2025 09:00, Markus Armbruster wrote:
>>
>> [...]
>>
>>>>>> What about providing the MCD interface as a separate QMP-like protocol?
>>>>>> It gets its own QAPI schema, just like for qemu-ga.  Simplifies
>>>>>> compiling it out when not needed.
>>>>>>
>>>>>> It gets its own socket, just like the GDB stub.  Might reduce
>>>>>> interference between debugging and QMP.
>>>>>> 
>>>>>> Thoughts?  Alex, Philippe, care to chime in?
>>>>>
>>>>> Sound reasonable to me. Keeping in mind the size of generated QAPI code,
>>>>> an option to `./configure [...] --enable-mcd` is definitely advisable.
>>>>
>>>> Alex, Philippe?
>>>
>>> When I spoke to Mario at DVCon last year I liked the idea of re-using
>>> QMP instead of inventing yet another RPC interface for QEMU. QMP
>>> certainly has nicer properties than the gdbstub which has a very
>>> "organic" and "serial" feel to it.
>>>
>>> Are you suggesting we re-use the machinery but use an entirely separate
>>> socket with just the MCD namespace in it? I don't see that being a
>>> problem as long as we can test it properly in the CI.
>>
>> Yes.
>>
>> "Keep them separate" is only a gut feeling, though.  While I pay
>> attention to my gut feelings, I know they can be wrong.  I am soliciting
>> opinions.
>
> I forgot to add isn't the flexibility of the QMP API something we need
> to handle for single binary anyway?

I have no idea :)

Evolving a target-dependent interface into a target-independent
interface without breaking compatibility is always a bother.

It's likely more of a bother when the interface is binary.  Textual
interfaces tend to have less target-dependence.

Designing a target-independent interface is probably easier than
evolving it compatibly from a target-dependent one.

QMP is textual, and it's designed for certain kinds of compatible
evolution.  Using QAPI/QMP for a debugging interface may be a perfectly
sensible idea.  I don't know enough about debugging interfaces to judge.

Use of QAPI/QMP does not imply use of the QMP monitor.  We can keep the
monitor and the debugging interface separate even though both are based
on QAPI/QMP.

The monitor code is gnarly, I'm afraid.  It supports multiple monitors,
but they are not fully independent for historical reasons, chiefly
implied mutual exclusion for commands.  Adding a QAPI/QMP-based
debugging interface without undue coupling to monitors may pose a few
technical problems.  One way to find out.

On 08.04.2025 16:37, Markus Armbruster wrote:

> Alex Bennée <alex.bennee@linaro.org> writes:
> 
>> Markus Armbruster <armbru@redhat.com> writes:
>>
>>> Alex Bennée <alex.bennee@linaro.org> writes:
>>>
>>>> Markus Armbruster <armbru@redhat.com> writes:
>>>>
>>>>> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
>>>>>
>>>>>> Apologies for the line wrapping in yesterday's answer. Should be fixed now.
>>>>>>
>>>>>> On 08.04.2025 09:00, Markus Armbruster wrote:
>>>
>>> [...]
>>>
>>>>>>> What about providing the MCD interface as a separate QMP-like protocol?
>>>>>>> It gets its own QAPI schema, just like for qemu-ga.  Simplifies
>>>>>>> compiling it out when not needed.
>>>>>>>
>>>>>>> It gets its own socket, just like the GDB stub.  Might reduce
>>>>>>> interference between debugging and QMP.
>>>>>>>
>>>>>>> Thoughts?  Alex, Philippe, care to chime in?
>>>>>>
>>>>>> Sound reasonable to me. Keeping in mind the size of generated QAPI code,
>>>>>> an option to `./configure [...] --enable-mcd` is definitely advisable.
>>>>>
>>>>> Alex, Philippe?
>>>>
>>>> When I spoke to Mario at DVCon last year I liked the idea of re-using
>>>> QMP instead of inventing yet another RPC interface for QEMU. QMP
>>>> certainly has nicer properties than the gdbstub which has a very
>>>> "organic" and "serial" feel to it.
>>>>
>>>> Are you suggesting we re-use the machinery but use an entirely separate
>>>> socket with just the MCD namespace in it? I don't see that being a
>>>> problem as long as we can test it properly in the CI.
>>>
>>> Yes.
>>>
>>> "Keep them separate" is only a gut feeling, though.  While I pay
>>> attention to my gut feelings, I know they can be wrong.  I am soliciting
>>> opinions.
>>
>> I forgot to add isn't the flexibility of the QMP API something we need
>> to handle for single binary anyway?
> 
> I have no idea :)

Alex, thanks for chiming in! By "single binary", I assume you mean user
space emulation? In that case, could you elaborate whether and how it's
a concern related to MCD? Maybe I'm missing something here. MCD is
specifically designed for debugging multi-core SoCs and therefore
currently only supported in system emulation. For user-space debugging,
I don't see any reason why not to use GDB's remote serial protocol.

> Evolving a target-dependent interface into a target-independent
> interface without breaking compatibility is always a bother.
> 
> It's likely more of a bother when the interface is binary.  Textual
> interfaces tend to have less target-dependence.
> 
> Designing a target-independent interface is probably easier than
> evolving it compatibly from a target-dependent one.

Like the gdbstub, the MCD implementation does not have any
target-specific dependencies. This is also a change compared to the last
patch set and something I wanted to point out with

> Architecture-independent MCD implementation

But, again, maybe I'm missing something.

> QMP is textual, and it's designed for certain kinds of compatible
> evolution.  Using QAPI/QMP for a debugging interface may be a perfectly
> sensible idea.  I don't know enough about debugging interfaces to judge.

Even though MCD is a very stable API (developed in 2008, functions
haven't changed since then), as you've already pointed out, it's bold to
assume that it will never change in the future. For that reason, MCD
provides the mcd_initialize_f function which can be used to communicate
the requested and implemented API versions. As long as that function
stays serializable over the RPC layer, evolution should be possible.

> Use of QAPI/QMP does not imply use of the QMP monitor.  We can keep the
> monitor and the debugging interface separate even though both are based
> on QAPI/QMP.
> 
> The monitor code is gnarly, I'm afraid.  It supports multiple monitors,
> but they are not fully independent for historical reasons, chiefly
> implied mutual exclusion for commands.  Adding a QAPI/QMP-based
> debugging interface without undue coupling to monitors may pose a few
> technical problems.  One way to find out.

If I understand you correctly, when QAPI-MCD runs on a separate socket
without using a monitor, it's still coupled to the monitor code
internally? Does this have an influence on the either the usage of a
monitor or the MCD interface or is it rather an implementation detail?

Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:

> On 08.04.2025 16:37, Markus Armbruster wrote:
>
>> Alex Bennée <alex.bennee@linaro.org> writes:
>> 
>>> Markus Armbruster <armbru@redhat.com> writes:
>>>
>>>> Alex Bennée <alex.bennee@linaro.org> writes:
>>>>
>>>>> Markus Armbruster <armbru@redhat.com> writes:
>>>>>
>>>>>> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
>>>>>>
>>>>>>> Apologies for the line wrapping in yesterday's answer. Should be fixed now.
>>>>>>>
>>>>>>> On 08.04.2025 09:00, Markus Armbruster wrote:
<snip>
>>>>
>>>> "Keep them separate" is only a gut feeling, though.  While I pay
>>>> attention to my gut feelings, I know they can be wrong.  I am soliciting
>>>> opinions.
>>>
>>> I forgot to add isn't the flexibility of the QMP API something we need
>>> to handle for single binary anyway?
>> 
>> I have no idea :)
>
> Alex, thanks for chiming in! By "single binary", I assume you mean user
> space emulation? In that case, could you elaborate whether and how it's
> a concern related to MCD? Maybe I'm missing something here.

Not directly, it was aimed at Markus. As we work towards a single binary
build we have to consider how the QMP API is going to be presented when
the binary could run a number of different targets. Markus and Philippe
have discussed it before but I forget the details.

> MCD is
> specifically designed for debugging multi-core SoCs and therefore
> currently only supported in system emulation.

Eventually we hope to get to the position QEMU can emulation a
heterogeneous SoC so having multiple architectures under one binary will
present potential issues for debugging. 

> For user-space debugging,
> I don't see any reason why not to use GDB's remote serial protocol.
>
>> Evolving a target-dependent interface into a target-independent
>> interface without breaking compatibility is always a bother.
>> 
>> It's likely more of a bother when the interface is binary.  Textual
>> interfaces tend to have less target-dependence.
>> 
>> Designing a target-independent interface is probably easier than
>> evolving it compatibly from a target-dependent one.
>
> Like the gdbstub, the MCD implementation does not have any
> target-specific dependencies. This is also a change compared to the last
> patch set and something I wanted to point out with
>
>> Architecture-independent MCD implementation
>
> But, again, maybe I'm missing something.
>
>> QMP is textual, and it's designed for certain kinds of compatible
>> evolution.  Using QAPI/QMP for a debugging interface may be a perfectly
>> sensible idea.  I don't know enough about debugging interfaces to judge.
>
> Even though MCD is a very stable API (developed in 2008, functions
> haven't changed since then), as you've already pointed out, it's bold to
> assume that it will never change in the future. For that reason, MCD
> provides the mcd_initialize_f function which can be used to communicate
> the requested and implemented API versions. As long as that function
> stays serializable over the RPC layer, evolution should be possible.
>
>> Use of QAPI/QMP does not imply use of the QMP monitor.  We can keep the
>> monitor and the debugging interface separate even though both are based
>> on QAPI/QMP.
>> 
>> The monitor code is gnarly, I'm afraid.  It supports multiple monitors,
>> but they are not fully independent for historical reasons, chiefly
>> implied mutual exclusion for commands.  Adding a QAPI/QMP-based
>> debugging interface without undue coupling to monitors may pose a few
>> technical problems.  One way to find out.
>
> If I understand you correctly, when QAPI-MCD runs on a separate socket
> without using a monitor, it's still coupled to the monitor code
> internally? Does this have an influence on the either the usage of a
> monitor or the MCD interface or is it rather an implementation detail?

An implementation detail - we should try and avoid needless coupling if
we can though.

AFAIK not all monitor commands map to QMP equivalents but I'm not sure
if that's true the other way around - can you do everything you can over
QMP under HMP? If you don't have to then that implies we can over a
separate schema on a debug socket that doesn't need monitor bits tied
in.

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Alex Bennée <alex.bennee@linaro.org> writes:

> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
>
>> On 08.04.2025 16:37, Markus Armbruster wrote:

[...]

>>> Use of QAPI/QMP does not imply use of the QMP monitor.  We can keep the
>>> monitor and the debugging interface separate even though both are based
>>> on QAPI/QMP.
>>> 
>>> The monitor code is gnarly, I'm afraid.  It supports multiple monitors,
>>> but they are not fully independent for historical reasons, chiefly
>>> implied mutual exclusion for commands.  Adding a QAPI/QMP-based
>>> debugging interface without undue coupling to monitors may pose a few
>>> technical problems.  One way to find out.
>>
>> If I understand you correctly, when QAPI-MCD runs on a separate socket
>> without using a monitor, it's still coupled to the monitor code
>> internally? Does this have an influence on the either the usage of a
>> monitor or the MCD interface or is it rather an implementation detail?
>
> An implementation detail - we should try and avoid needless coupling if
> we can though.
>
> AFAIK not all monitor commands map to QMP equivalents but I'm not sure
> if that's true the other way around - can you do everything you can over
> QMP under HMP? If you don't have to then that implies we can over a
> separate schema on a debug socket that doesn't need monitor bits tied
> in.

Ideally, HMP commands are human-friendly wrappers around QMP commands.
Such commands can't do anything that could be done with QMP.

Not all the functionality of QMP needs to be exposed in HMP.  Some
functionality might only exist in QMP.  How much to expose is a
pragmatic "is it worth the bother?" decision.

Certain HMP commands make no sense in QMP.  Examples:

* HMP cpu sets the default CPU.  The concept doesn't exist in QMP.

* Ad hoc debugging commands that are used only by humans, such as
  sync-profile.

* The desk calculator[*]

My abstract mental model of MCD has it completely separate from monitors
(HMP and QMP).  Just like gdbstub.

We're considering to use QAPI/QMP as interface definition and transport
infrastructure.

Mario's patches do that in a simple way: by "embedding" MCD in the QMP
monitor.  This creates a coupling that doesn't exist in my abstract
mental model.

We could have a separate QAPI schema for interface definition, and a
separate socket for transport.  This avoids coupling.

Obviously, we'd still want to reuse existing QMP infrastructure as much
as practical.  Some of that that code is gnarly.  If we're not careful,
we get some subtle coupling deep under the hood.

Here's one possible complication.  While we can have any number of QMP
monitors, we still funnel most[**] commands to the main loop thread,
where they run one after the other.  If we reuse all that machinery for
MCD, we end up forcing MCD commands through the same funnel to the main
loop thread.  This coupling could be undesirable.

Before we contemplate how to press the existing QMP infrastructure into
service for MCD, we should first figure out how we'd want MCD to operate
if we started on a green field.  Can we have more than one MCD
connection?  Which thread should execure MCD commands?  Any locking
requirements?  Any need for asynchronous commands?


[*] Which arguably makes no sense in HMP either.

[**] Not out-of-band commands.  You probably don't want to know more.

On 09.04.2025 13:37, Markus Armbruster wrote:

> Alex Bennée <alex.bennee@linaro.org> writes:
> 
>> Mario Fleischmann <mario.fleischmann@lauterbach.com> writes:
>>
>>> On 08.04.2025 16:37, Markus Armbruster wrote:
> 
> [...]
> 
>>>> Use of QAPI/QMP does not imply use of the QMP monitor.  We can keep the
>>>> monitor and the debugging interface separate even though both are based
>>>> on QAPI/QMP.
>>>>
>>>> The monitor code is gnarly, I'm afraid.  It supports multiple monitors,
>>>> but they are not fully independent for historical reasons, chiefly
>>>> implied mutual exclusion for commands.  Adding a QAPI/QMP-based
>>>> debugging interface without undue coupling to monitors may pose a few
>>>> technical problems.  One way to find out.
>>>
>>> If I understand you correctly, when QAPI-MCD runs on a separate socket
>>> without using a monitor, it's still coupled to the monitor code
>>> internally? Does this have an influence on the either the usage of a
>>> monitor or the MCD interface or is it rather an implementation detail?
>>
>> An implementation detail - we should try and avoid needless coupling if
>> we can though.
>>
>> AFAIK not all monitor commands map to QMP equivalents but I'm not sure
>> if that's true the other way around - can you do everything you can over
>> QMP under HMP? If you don't have to then that implies we can over a
>> separate schema on a debug socket that doesn't need monitor bits tied
>> in.
> 
> Ideally, HMP commands are human-friendly wrappers around QMP commands.
> Such commands can't do anything that could be done with QMP.
> 
> Not all the functionality of QMP needs to be exposed in HMP.  Some
> functionality might only exist in QMP.  How much to expose is a
> pragmatic "is it worth the bother?" decision.
> 
> Certain HMP commands make no sense in QMP.  Examples:
> 
> * HMP cpu sets the default CPU.  The concept doesn't exist in QMP.
> 
> * Ad hoc debugging commands that are used only by humans, such as
>   sync-profile.
> 
> * The desk calculator[*]
> 
> My abstract mental model of MCD has it completely separate from monitors
> (HMP and QMP).  Just like gdbstub.
> 
> We're considering to use QAPI/QMP as interface definition and transport
> infrastructure.
> 
> Mario's patches do that in a simple way: by "embedding" MCD in the QMP
> monitor.  This creates a coupling that doesn't exist in my abstract
> mental model.
> 
> We could have a separate QAPI schema for interface definition, and a
> separate socket for transport.  This avoids coupling.
> 
> Obviously, we'd still want to reuse existing QMP infrastructure as much
> as practical.  Some of that that code is gnarly.  If we're not careful,
> we get some subtle coupling deep under the hood.
> 
> Here's one possible complication.  While we can have any number of QMP
> monitors, we still funnel most[**] commands to the main loop thread,
> where they run one after the other.  If we reuse all that machinery for
> MCD, we end up forcing MCD commands through the same funnel to the main
> loop thread.  This coupling could be undesirable.

I think now we understand each other. Let me try to rephrase: We're
trying to add a QEMU "debug" monitor which is used to debug the emulated
system. The debug monitor is built on the MCD API as the debug protocol
and QMP as the transport protocol and runs in parallel to the QEMU
monitor which is used to control the emulator. Since the QEMU monitor is
also built on QMP, we can reuse as much as possible from its QMP code
but have to be careful not to add any coupling that goes beyond the
transport layer.

If I understood you correctly, I would feel prepared to go to work and
try it out. If problems come up, I would get back to you.

> Before we contemplate how to press the existing QMP infrastructure into
> service for MCD, we should first figure out how we'd want MCD to operate
> if we started on a green field.  Can we have more than one MCD
> connection?

There are two types of connections in MCD. mcd_open_server_f opens the
connection to the MCD implementation, i.e. mcdserver, and initializes
mcdserver's core database. The actual debugging is performed over core
connections opened by mcd_open_core_f. Each core connection will be
identified by a pointer to a mcd_core_st object in the MCD API and by an
uint32_t holding a unique identifier over the RPC protocol.

In homogeneous multi-core systems, we can get away with one server
connection and multiple core connections. In heterogeneous systems, we
need multiple server connections since most likely multiple debuggers
have to be connected to QEMU simultaneously.

While there are use-cases to opening multiple connections to the same
core at once, I would propose to keep this is as a future option but not
to include this in the initial commits since it considerably complicates
things.

> Which thread should execute MCD commands?

Since both gdbserver and QMP monitor run in the main thread, I don't see
a reason why not to run the mcdserver in there as well.

> Any locking requirements?

As there are no explicit locks in gdbstub.c, I assume they are enforced
by QEMU's infrastructure.

> Any need for asynchronous commands?

No, none that I'm aware of.

> [*] Which arguably makes no sense in HMP either.
> 
> [**] Not out-of-band commands.  You probably don't want to know more.