This patch series intent is to introduce a generator that produces a Go
module for Go applications to interact over QMP with QEMU.

This is the second iteration:
 v1: https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg06734.html

I've pushed this series in my gitlab fork:
https://gitlab.com/victortoso/qemu/-/tree/qapi-golang-v2

I've also generated the qapi-go module over QEMU tags: v7.0.0, v7.1.0,
v7.2.6, v8.0.0 and v8.1.0, see the commits history here:
https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v2-by-tags

I've also generated the qapi-go module over each commit of this series,
see the commits history here (using previous refered qapi-golang-v2)
https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v2-by-patch

Cheers,
Victor

* Changes:
 - All patches were rebased using black python formatting tool, awesome.
   (John) https://black.readthedocs.io/en/stable/
 - All patches were tested with flake8 and pylint. Minor complains
   remains. (John)
 - All generated types are sorted in alphabetical order (Daniel)
 - Using utf8 instead of ascii encoding of output files
 - Improved commit logs
 - renamed QapiError -> QAPIError (Daniel)
 - QAPIError's Error() returns only the description (Daniel)
 - Used more type hints (Where I could) (John)
 - Removed typehint from self in the Class implementation (John)
 - The Go code that is generated is now well formated. gofmt -w and
   goimport -w don't change a thing.
 - Added a fix from John
   https://lists.gnu.org/archive/html/qemu-devel/2023-10/msg01305.html
 - Added a tangent fix suggestion to main.py "scripts: qapi: black
   format main.py"

John Snow (1):
  qapi: re-establish linting baseline

Victor Toso (10):
  scripts: qapi: black format main.py
  qapi: golang: Generate qapi's enum types in Go
  qapi: golang: Generate qapi's alternate types in Go
  qapi: golang: Generate qapi's struct types in Go
  qapi: golang: structs: Address 'null' members
  qapi: golang: Generate qapi's union types in Go
  qapi: golang: Generate qapi's event types in Go
  qapi: golang: Generate qapi's command types in Go
  qapi: golang: Add CommandResult type to Go
  docs: add notes on Golang code generator

 docs/devel/index-build.rst          |    1 +
 docs/devel/qapi-golang-code-gen.rst |  376 ++++++++
 scripts/qapi/gen.py                 |    2 +-
 scripts/qapi/golang.py              | 1349 +++++++++++++++++++++++++++
 scripts/qapi/main.py                |   79 +-
 scripts/qapi/parser.py              |    5 +-
 6 files changed, 1781 insertions(+), 31 deletions(-)
 create mode 100644 docs/devel/qapi-golang-code-gen.rst
 create mode 100644 scripts/qapi/golang.py

-- 
2.41.0

Hi,

Daniel & Andrea, it would be great to have your take on the Go
side of this series. If we can agree with an acceptable
'unstable' version of Go modules, we can start building on top of
this:
 - libraries/tools in Go to interact with QEMU
 - qapi specs to fix limitations (e.g: Data type names)
 - scripts/qapi library wrt to generating interfaces in other
   languages other than C

I would love to have this prior to 8.2 feature freeze if the
idea and current code meet your expectations.

Cheers,
Victor

On Mon, Oct 16, 2023 at 05:26:53PM +0200, Victor Toso wrote:
> This patch series intent is to introduce a generator that produces a Go
> module for Go applications to interact over QMP with QEMU.
> 
> This is the second iteration:
>  v1: https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg06734.html
> 
> I've pushed this series in my gitlab fork:
> https://gitlab.com/victortoso/qemu/-/tree/qapi-golang-v2
> 
> I've also generated the qapi-go module over QEMU tags: v7.0.0, v7.1.0,
> v7.2.6, v8.0.0 and v8.1.0, see the commits history here:
> https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v2-by-tags
> 
> I've also generated the qapi-go module over each commit of this series,
> see the commits history here (using previous refered qapi-golang-v2)
> https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v2-by-patch
> 
> Cheers,
> Victor
> 
> * Changes:
>  - All patches were rebased using black python formatting tool, awesome.
>    (John) https://black.readthedocs.io/en/stable/
>  - All patches were tested with flake8 and pylint. Minor complains
>    remains. (John)
>  - All generated types are sorted in alphabetical order (Daniel)
>  - Using utf8 instead of ascii encoding of output files
>  - Improved commit logs
>  - renamed QapiError -> QAPIError (Daniel)
>  - QAPIError's Error() returns only the description (Daniel)
>  - Used more type hints (Where I could) (John)
>  - Removed typehint from self in the Class implementation (John)
>  - The Go code that is generated is now well formated. gofmt -w and
>    goimport -w don't change a thing.
>  - Added a fix from John
>    https://lists.gnu.org/archive/html/qemu-devel/2023-10/msg01305.html
>  - Added a tangent fix suggestion to main.py "scripts: qapi: black
>    format main.py"
> 
> John Snow (1):
>   qapi: re-establish linting baseline
> 
> Victor Toso (10):
>   scripts: qapi: black format main.py
>   qapi: golang: Generate qapi's enum types in Go
>   qapi: golang: Generate qapi's alternate types in Go
>   qapi: golang: Generate qapi's struct types in Go
>   qapi: golang: structs: Address 'null' members
>   qapi: golang: Generate qapi's union types in Go
>   qapi: golang: Generate qapi's event types in Go
>   qapi: golang: Generate qapi's command types in Go
>   qapi: golang: Add CommandResult type to Go
>   docs: add notes on Golang code generator
> 
>  docs/devel/index-build.rst          |    1 +
>  docs/devel/qapi-golang-code-gen.rst |  376 ++++++++
>  scripts/qapi/gen.py                 |    2 +-
>  scripts/qapi/golang.py              | 1349 +++++++++++++++++++++++++++
>  scripts/qapi/main.py                |   79 +-
>  scripts/qapi/parser.py              |    5 +-
>  6 files changed, 1781 insertions(+), 31 deletions(-)
>  create mode 100644 docs/devel/qapi-golang-code-gen.rst
>  create mode 100644 scripts/qapi/golang.py
> 
> -- 
> 2.41.0
> 
> 

On Fri, Oct 27, 2023 at 07:33:30PM +0200, Victor Toso wrote:
> Hi,
>
> Daniel & Andrea, it would be great to have your take on the Go
> side of this series. If we can agree with an acceptable
> 'unstable' version of Go modules, we can start building on top of
> this:
>  - libraries/tools in Go to interact with QEMU
>  - qapi specs to fix limitations (e.g: Data type names)
>  - scripts/qapi library wrt to generating interfaces in other
>    languages other than C
>
> I would love to have this prior to 8.2 feature freeze if the
> idea and current code meet your expectations.

Apologies for not providing any feedback so far. I'll do my best to
get around to it by the end of the week.

-- 
Andrea Bolognani / Red Hat / Virtualization

On Tue, Oct 31, 2023 at 09:42:10AM -0700, Andrea Bolognani wrote:
> On Fri, Oct 27, 2023 at 07:33:30PM +0200, Victor Toso wrote:
> > Hi,
> >
> > Daniel & Andrea, it would be great to have your take on the Go
> > side of this series. If we can agree with an acceptable
> > 'unstable' version of Go modules, we can start building on top of
> > this:
> >  - libraries/tools in Go to interact with QEMU
> >  - qapi specs to fix limitations (e.g: Data type names)
> >  - scripts/qapi library wrt to generating interfaces in other
> >    languages other than C
> >
> > I would love to have this prior to 8.2 feature freeze if the
> > idea and current code meet your expectations.
>
> Apologies for not providing any feedback so far. I'll do my best to
> get around to it by the end of the week.

Layering apologies on top of apologies: I started looking into this,
but I have since realized that I need some more time to page all the
months-old context back in and digest the whole thing. I'll continue
next week.

As an appetizer, one thing that I've noticed: you seem to ignore it
when gen:false is included in a command definition, so we get

  type DeviceAddCommand struct {
      MessageId string  `json:"-"`
      Driver    string  `json:"driver"`
      Bus       *string `json:"bus,omitempty"`
      Id        *string `json:"id,omitempty"`
  }

which I don't think will work as it can't handle even the example
used to document the command

  { "execute": "device_add",
    "arguments": { "driver": "e1000", "id": "net1",
                   "bus": "pci.0",
                   "mac": "52:54:00:12:34:56" } }

This command will probably require an ad-hoc implementation.

-- 
Andrea Bolognani / Red Hat / Virtualization

Hi,

On Fri, Nov 03, 2023 at 11:34:18AM -0700, Andrea Bolognani wrote:
> On Tue, Oct 31, 2023 at 09:42:10AM -0700, Andrea Bolognani wrote:
> > On Fri, Oct 27, 2023 at 07:33:30PM +0200, Victor Toso wrote:
> > > Hi,
> > >
> > > Daniel & Andrea, it would be great to have your take on the Go
> > > side of this series. If we can agree with an acceptable
> > > 'unstable' version of Go modules, we can start building on top of
> > > this:
> > >  - libraries/tools in Go to interact with QEMU
> > >  - qapi specs to fix limitations (e.g: Data type names)
> > >  - scripts/qapi library wrt to generating interfaces in other
> > >    languages other than C
> > >
> > > I would love to have this prior to 8.2 feature freeze if the
> > > idea and current code meet your expectations.
> >
> > Apologies for not providing any feedback so far. I'll do my best to
> > get around to it by the end of the week.
> 
> Layering apologies on top of apologies: I started looking into this,
> but I have since realized that I need some more time to page all the
> months-old context back in and digest the whole thing. I'll continue
> next week.
> 
> As an appetizer, one thing that I've noticed: you seem to ignore it
> when gen:false is included in a command definition, so we get
> 
>   type DeviceAddCommand struct {
>       MessageId string  `json:"-"`
>       Driver    string  `json:"driver"`
>       Bus       *string `json:"bus,omitempty"`
>       Id        *string `json:"id,omitempty"`
>   }
> 
> which I don't think will work as it can't handle even the example
> used to document the command
> 
>   { "execute": "device_add",
>     "arguments": { "driver": "e1000", "id": "net1",
>                    "bus": "pci.0",
>                    "mac": "52:54:00:12:34:56" } }
> 
> This command will probably require an ad-hoc implementation.

Indeed, thanks for catching this.

Cheers,
Victor

On Mon, Oct 16, 2023 at 05:26:53PM +0200, Victor Toso wrote:
> This patch series intent is to introduce a generator that produces a Go
> module for Go applications to interact over QMP with QEMU.
>
> This is the second iteration:
>  v1: https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg06734.html
>
> I've pushed this series in my gitlab fork:
> https://gitlab.com/victortoso/qemu/-/tree/qapi-golang-v2
>
> I've also generated the qapi-go module over QEMU tags: v7.0.0, v7.1.0,
> v7.2.6, v8.0.0 and v8.1.0, see the commits history here:
> https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v2-by-tags
>
> I've also generated the qapi-go module over each commit of this series,
> see the commits history here (using previous refered qapi-golang-v2)
> https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v2-by-patch

I've provided feedback on the various facets of the API in response
to the corresponding patch. Note that I've only addressed concerns
about the consumer-facing API: I have some notes about the
implementation as well, but that's something that we should be able
to easily change transparently so I didn't give it priority.

Overall, I think that the current API is quite close to being a solid
PoC that can be used as a starting point for further development.

-- 
Andrea Bolognani / Red Hat / Virtualization

Hi,

On Thu, Nov 09, 2023 at 10:35:07AM -0800, Andrea Bolognani wrote:
> On Mon, Oct 16, 2023 at 05:26:53PM +0200, Victor Toso wrote:
> > This patch series intent is to introduce a generator that produces a Go
> > module for Go applications to interact over QMP with QEMU.
> >
> > This is the second iteration:
> >  v1: https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg06734.html
> >
> > I've pushed this series in my gitlab fork:
> > https://gitlab.com/victortoso/qemu/-/tree/qapi-golang-v2
> >
> > I've also generated the qapi-go module over QEMU tags: v7.0.0, v7.1.0,
> > v7.2.6, v8.0.0 and v8.1.0, see the commits history here:
> > https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v2-by-tags
> >
> > I've also generated the qapi-go module over each commit of this series,
> > see the commits history here (using previous refered qapi-golang-v2)
> > https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v2-by-patch
> 
> I've provided feedback on the various facets of the API in response
> to the corresponding patch. Note that I've only addressed concerns
> about the consumer-facing API: I have some notes about the
> implementation as well, but that's something that we should be able
> to easily change transparently so I didn't give it priority.

Sure thing.
 
> Overall, I think that the current API is quite close to being a
> solid PoC that can be used as a starting point for further
> development.

Happy o hear it. Many thanks for the review, I really appreciate
it.

Cheers,
Victor