Hi again,

This patch series intent is to introduce a generator that produces a Go

module for Go applications to interact over QMP with QEMU.

Previous version (10 Jan 2025)

https://lists.gnu.org/archive/html/qemu-devel/2025-01/msg01530.html

The generated code was mostly tested using existing examples in the QAPI

documentation, 192 instances that might have multiple QMP messages each.

You can find the the tests and the generated code in my personal repo,

main branch:

https://gitlab.com/victortoso/qapi-go

If you want to see the generated code from QEMU's master but per patch:

https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v4-by-patch

If you rather see the diff between v9.1.0, v9.2.0 and latest:

https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v4-by-tags

#################

# Changes in v4 #

#################

1. Daniel wrote a demo on top of v3 and proposed changes that would

result in more interesting module to build on top:

https://lists.gnu.org/archive/html/qemu-devel/2025-01/msg03052.html

I've implemented all the suggestions that are relevant for this

introductory series, they are:

a. New struct type Message, that shall be used for a 1st level

unmarshalling of the JSON message.

b. Removal of Marshal/Unmarshal code in both Events and Comands,

together with utility code that is not relevant anymore.

c. Declaration of 3 new interfaces:

i. Events

ii. Commands

iii. CommandsAsync

2. I've moved the code to a new folder: scripts/qapi/golang. This

allowed me to move templates out of golang.py, keeping go related

code self-contained in the new directory.

3. As mentioned in (2), created protocol.go and utils.go that are 100%

hand generated Go code. Message mentioned in (1a) is under

protocol.go

4. Defined license using SPDX-License-Identifier.

a. Every Go source code written by hand is 100% MIT-0

b. Every Go source code generated is dual licensed as MIT-0 and

GPL-2.0-or-later

c. The binary code is expected to be MIT-0 only but not really

relevant for this series.

If you want more information, please check the thread:

https://lists.gnu.org/archive/html/qemu-devel/2024-11/msg01621.html

5. I've renamed the generated files.

a. Any type related file is now prefixed with "gen_type_"

b. Any interface related file is prefixed as "gen_iface_"

6. Relevant changes were made to the doc but it is not complete. I plan

that follow-up proposals would add to the documentation.

7. Improvements to the generator were made to.

8. Also worth to mention that resulting generated code does not have any

diff with gofmt and goimport tools, as requested in the past.

################

# Expectations #

################

As is, this still is a PoC that works. I'd like to have the generated

code included in QEMU's gitlab [0] in order to write library and tools

on top. Initial version should be considered alpha. Moving to

beta/stable would require functional libraries and tools, but this work

needs to be merged before one commit to that.

[0] https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg07024.html

##################

# Follow-up work #

##################

When this is merged we would need to:

1. Create gitlab's repo

2. Add unit test and CI to new repos

3. Have first alpha relase/tag

4. Start working on top :)

Thanks for the time looking at this. I appreciate it.

Victor Toso (11):

qapi: golang: first level unmarshalling type

qapi: golang: Generate enum type

qapi: golang: Generate alternate types

qapi: golang: Generate struct types

qapi: golang: structs: Address nullable members

qapi: golang: Generate union type

qapi: golang: Generate event type

qapi: golang: Generate Event interface

qapi: golang: Generate command type

qapi: golang: Generate Command sync/async interfaces

docs: add notes on Golang code generator

docs/devel/index-build.rst | 1 +

docs/devel/qapi-golang-code-gen.rst | 420 ++++++++

scripts/qapi/golang/__init__.py | 0

scripts/qapi/golang/golang.py | 1444 +++++++++++++++++++++++++++

scripts/qapi/golang/protocol.go | 48 +

scripts/qapi/golang/utils.go | 38 +

scripts/qapi/main.py | 2 +

7 files changed, 1953 insertions(+)

create mode 100644 docs/devel/qapi-golang-code-gen.rst

create mode 100644 scripts/qapi/golang/__init__.py

create mode 100644 scripts/qapi/golang/golang.py

create mode 100644 scripts/qapi/golang/protocol.go

create mode 100644 scripts/qapi/golang/utils.go

2.48.1

This first patch introduces protocol.go. It introduces the Message Go

struct type that can unmarshall any QMP message.

It does not handle deeper than 1st layer of the JSON object, that is,

with:

1. {

"execute": "query-machines",

"arguments": { "compat-props": true }

}

2. {

"event": "BALLOON_CHANGE",

"data": { "actual": 944766976 },

"timestamp": {

"seconds": 1267020223,

"microseconds": 435656

}

}

We will be able to know it is a query-machine command or a

balloon-change event. Specific data type to handle arguments/data will

be introduced further in the series.

This patch also introduces the Visitor skeleton with a proper write()

function to copy-over the protocol.go to the target destination.

Note, you can execute any patch of this series with:

python3 ./scripts/qapi-gen.py -o /tmp/out qapi/qapi-schema.json

Signed-off-by: Victor Toso <victortoso@redhat.com>

---

scripts/qapi/golang/__init__.py | 0

scripts/qapi/golang/golang.py | 135 ++++++++++++++++++++++++++++++++

scripts/qapi/golang/protocol.go | 48 ++++++++++++

scripts/qapi/main.py | 2 +

4 files changed, 185 insertions(+)

create mode 100644 scripts/qapi/golang/__init__.py

create mode 100644 scripts/qapi/golang/golang.py

create mode 100644 scripts/qapi/golang/protocol.go

diff --git a/scripts/qapi/golang/__init__.py b/scripts/qapi/golang/__init__.py

new file mode 100644

index XXXXXXX..XXXXXXX

diff --git a/scripts/qapi/golang/golang.py b/scripts/qapi/golang/golang.py

new file mode 100644

index XXXXXXX..XXXXXXX

--- /dev/null

+++ b/scripts/qapi/golang/golang.py

@@ -XXX,XX +XXX,XX @@

+"""

+Golang QAPI generator

+"""

+

+# Copyright (c) 2025 Red Hat Inc.

+#

+# Authors:

+# Victor Toso <victortoso@redhat.com>

+#

+# This work is licensed under the terms of the GNU GPL, version 2.

+# See the COPYING file in the top-level directory.

+

+# Just for type hint on self

+from __future__ import annotations

+

+import os, shutil

+from typing import List, Optional

+

+from ..schema import (

+ QAPISchema,

+ QAPISchemaBranches,

+ QAPISchemaEnumMember,

+ QAPISchemaFeature,

+ QAPISchemaIfCond,

+ QAPISchemaObjectType,

+ QAPISchemaObjectTypeMember,

+ QAPISchemaType,

+ QAPISchemaVariants,

+ QAPISchemaVisitor,

+)

+from ..source import QAPISourceInfo

+

+

+def gen_golang(schema: QAPISchema, output_dir: str, prefix: str) -> None:

+ vis = QAPISchemaGenGolangVisitor(prefix)

+ schema.visit(vis)

+ vis.write(output_dir)

+

+

+class QAPISchemaGenGolangVisitor(QAPISchemaVisitor):

+ # pylint: disable=too-many-arguments

+ def __init__(self, _: str):

+ super().__init__()

+ gofiles = ("protocol.go",)

+ self.schema: QAPISchema

+ self.golang_package_name = "qapi"

+ self.duplicate = list(gofiles)

+

+ def visit_begin(self, schema: QAPISchema) -> None:

+ self.schema = schema

+

+ def visit_end(self) -> None:

+ del self.schema

+

+ def visit_object_type(

+ self,

+ name: str,

+ info: Optional[QAPISourceInfo],

+ ifcond: QAPISchemaIfCond,

+ features: List[QAPISchemaFeature],

+ base: Optional[QAPISchemaObjectType],

+ members: List[QAPISchemaObjectTypeMember],

+ branches: Optional[QAPISchemaBranches],

+ ) -> None:

+ pass

+

+ def visit_alternate_type(

+ self,

+ name: str,

+ info: Optional[QAPISourceInfo],

+ ifcond: QAPISchemaIfCond,

+ features: List[QAPISchemaFeature],

+ variants: QAPISchemaVariants,

+ ) -> None:

+ pass

+

+ def visit_enum_type(

+ self,

+ name: str,

+ info: Optional[QAPISourceInfo],

+ ifcond: QAPISchemaIfCond,

+ features: List[QAPISchemaFeature],

+ members: List[QAPISchemaEnumMember],

+ prefix: Optional[str],

+ ) -> None:

+ pass

+

+ def visit_array_type(

+ self,

+ name: str,

+ info: Optional[QAPISourceInfo],

+ ifcond: QAPISchemaIfCond,

+ element_type: QAPISchemaType,

+ ) -> None:

+ pass

+

+ def visit_command(

+ self,

+ name: str,

+ info: Optional[QAPISourceInfo],

+ ifcond: QAPISchemaIfCond,

+ features: List[QAPISchemaFeature],

+ arg_type: Optional[QAPISchemaObjectType],

+ ret_type: Optional[QAPISchemaType],

+ gen: bool,

+ success_response: bool,

+ boxed: bool,

+ allow_oob: bool,

+ allow_preconfig: bool,

+ coroutine: bool,

+ ) -> None:

+ pass

+

+ def visit_event(

+ self,

+ name: str,

+ info: Optional[QAPISourceInfo],

+ ifcond: QAPISchemaIfCond,

+ features: List[QAPISchemaFeature],

+ arg_type: Optional[QAPISchemaObjectType],

+ boxed: bool,

+ ) -> None:

+ pass

+

+ def write(self, outdir: str) -> None:

+ godir = "go"

+ targetpath = os.path.join(outdir, godir)

+ os.makedirs(targetpath, exist_ok=True)

+

+ # Content to be copied over

+ srcdir = os.path.dirname(os.path.realpath(__file__))

+ for filename in self.duplicate:

+ srcpath = os.path.join(srcdir, filename)

+ dstpath = os.path.join(targetpath, filename)

+ shutil.copyfile(srcpath, dstpath)

diff --git a/scripts/qapi/golang/protocol.go b/scripts/qapi/golang/protocol.go

new file mode 100644

index XXXXXXX..XXXXXXX

--- /dev/null

+++ b/scripts/qapi/golang/protocol.go

@@ -XXX,XX +XXX,XX @@

+/*

+ * Copyright 2025 Red Hat, Inc.

+ * SPDX-License-Identifier: MIT-0

+ *

+ * Authors:

+ * Victor Toso <victortoso@redhat.com>

+ * Daniel P. Berrange <berrange@redhat.com>

+ */

+package qapi

+

+import (

+    "encoding/json"

+    "time"

+)

+

+/* Union of data for command, response, error, or event,

+ * since when receiving we don't know upfront which we

+ * must deserialize */

+type Message struct {

+    QMP *json.RawMessage `json:"QMP,omitempty"`

+    Execute string `json:"execute,omitempty"`

+    ExecOOB string `json:"exec-oob,omitempty"`

+    Event string `json:"event,omitempty"`

+    Error *json.RawMessage `json:"error,omitempty"`

+    Return *json.RawMessage `json:"return,omitempty"`

+    ID string `json:"id,omitempty"`

+    Timestamp *Timestamp `json:"timestamp,omitempty"`

+    Data *json.RawMessage `json:"data,omitempty"`

+    Arguments *json.RawMessage `json:"arguments,omitempty"`

+}

+

+type QAPIError struct {

+    Class string `json:"class"`

+    Description string `json:"desc"`

+}

+

+func (err *QAPIError) Error() string {

+    return err.Description

+}

+

+type Timestamp struct {

+    Seconds int `json:"seconds"`

+    MicroSeconds int `json:"microseconds"`

+}

+

+func (t *Timestamp) AsTime() time.Time {

+    return time.Unix(int64(t.Seconds), int64(t.MicroSeconds)*1000)

+}

diff --git a/scripts/qapi/main.py b/scripts/qapi/main.py

index XXXXXXX..XXXXXXX 100644

--- a/scripts/qapi/main.py

+++ b/scripts/qapi/main.py

@@ -XXX,XX +XXX,XX @@

from .error import QAPIError

from .events import gen_events

from .features import gen_features

+from .golang import golang

from .introspect import gen_introspect

from .schema import QAPISchema

from .types import gen_types

@@ -XXX,XX +XXX,XX @@ def generate(schema_file: str,

gen_commands(schema, output_dir, prefix, gen_tracing)

gen_events(schema, output_dir, prefix)

gen_introspect(schema, output_dir, prefix, unmask)

+ golang.gen_golang(schema, output_dir, prefix)

def main() -> int:

--

2.48.1

This patch handles QAPI enum types and generates its equivalent in Go.

We sort the output based on enum's type name.

Enums are being handled as strings in Golang.

1. For each QAPI enum, we will define a string type in Go to be the

assigned type of this specific enum.

2. Naming: CamelCase will be used in any identifier that we want to

export, which is everything.

Example:

qapi:

| ##

| # @DisplayProtocol:

| #

| # Display protocols which support changing password options.

| #

| # Since: 7.0

| ##

| { 'enum': 'DisplayProtocol',

| 'data': [ 'vnc', 'spice' ] }

go:

| // Display protocols which support changing password options.

| //

| // Since: 7.0

| type DisplayProtocol string

|

| const (

|     DisplayProtocolVnc DisplayProtocol = "vnc"

|     DisplayProtocolSpice DisplayProtocol = "spice"

| )

Signed-off-by: Victor Toso <victortoso@redhat.com>

---

scripts/qapi/golang/golang.py | 185 +++++++++++++++++++++++++++++++++-

1 file changed, 183 insertions(+), 2 deletions(-)

diff --git a/scripts/qapi/golang/golang.py b/scripts/qapi/golang/golang.py

index XXXXXXX..XXXXXXX 100644

--- a/scripts/qapi/golang/golang.py

+++ b/scripts/qapi/golang/golang.py

@@ -XXX,XX +XXX,XX @@

# Just for type hint on self

from __future__ import annotations

-import os, shutil

+import os, shutil, textwrap

from typing import List, Optional

from ..schema import (

@@ -XXX,XX +XXX,XX @@

)

from ..source import QAPISourceInfo

+TEMPLATE_GENERATED_HEADER = """

+/*

+ * Copyright 2025 Red Hat, Inc.

+ * SPDX-License-Identifier: (MIT-0 and GPL-2.0-or-later)

+ */

+

+/****************************************************************************

+ * THIS CODE HAS BEEN GENERATED. DO NOT CHANGE IT DIRECTLY *

+ ****************************************************************************/

+package {package_name}

+"""

+

+TEMPLATE_GO_IMPORTS = """

+import (

+{imports}

+)

+"""

+

+TEMPLATE_ENUM = """

+type {name} string

+

+const (

+{fields}

+)

+"""

+

+

+# Takes the documentation object of a specific type and returns

+# that type's documentation and its member's docs.

+def qapi_to_golang_struct_docs(doc: QAPIDoc) -> (str, Dict[str, str]):

+ if doc is None:

+ return "", {}

+

+ cmt = "// "

+ fmt = textwrap.TextWrapper(

+ width=70, initial_indent=cmt, subsequent_indent=cmt

+ )

+ main = fmt.fill(doc.body.text)

+

+ for section in doc.sections:

+ # TODO is not a relevant section to Go applications

+ if section.tag in ["TODO"]:

+ continue

+

+ if main != "":

+ # Give empty line as space for the tag.

+ main += "\n//\n"

+

+ tag = "" if section.tag is None else f"{section.tag}: "

+ text = section.text.replace(" ", " ")

+ main += fmt.fill(f"{tag}{text}")

+

+ fields = {}

+ for key, value in doc.args.items():

+ if len(value.text) > 0:

+ fields[key] = " ".join(value.text.replace("\n", " ").split())

+

+ return main, fields

+

def gen_golang(schema: QAPISchema, output_dir: str, prefix: str) -> None:

vis = QAPISchemaGenGolangVisitor(prefix)

@@ -XXX,XX +XXX,XX @@ def gen_golang(schema: QAPISchema, output_dir: str, prefix: str) -> None:

vis.write(output_dir)

+def qapi_to_field_name_enum(name: str) -> str:

+ return name.title().replace("-", "")

+

+

+def fetch_indent_blocks_over_enum_with_docs(

+ name: str, members: List[QAPISchemaEnumMember], docfields: Dict[str, str]

+) -> Tuple[int]:

+ maxname = 0

+ blocks: List[int] = [0]

+ for member in members:

+ # For simplicity, every time we have doc, we add a new indent block

+ hasdoc = member.name is not None and member.name in docfields

+

+ enum_name = f"{name}{qapi_to_field_name_enum(member.name)}"

+ maxname = (

+ max(maxname, len(enum_name)) if not hasdoc else len(enum_name)

+ )

+

+ if hasdoc:

+ blocks.append(maxname)

+ else:

+ blocks[-1] = maxname

+

+ return blocks

+

+

+def generate_content_from_dict(data: dict[str, str]) -> str:

+ content = ""

+

+ for name in sorted(data):

+ content += data[name]

+

+ return content.replace("\n\n\n", "\n\n")

+

+

+def generate_template_imports(words: List[str]) -> str:

+ if len(words) == 0:

+ return ""

+

+ if len(words) == 1:

+ return '\nimport "{words[0]}"\n'

+

+ return TEMPLATE_GO_IMPORTS.format(

+ imports="\n".join(f'\t"{w}"' for w in words)

+ )

+

+

class QAPISchemaGenGolangVisitor(QAPISchemaVisitor):

# pylint: disable=too-many-arguments

def __init__(self, _: str):

super().__init__()

gofiles = ("protocol.go",)

+ # Map each qapi type to the necessary Go imports

+ types = {

+ "enum": [],

+ }

+

self.schema: QAPISchema

self.golang_package_name = "qapi"

self.duplicate = list(gofiles)

+ self.enums: dict[str, str] = {}

+ self.docmap = {}

+

+ self.types = dict.fromkeys(types, "")

+ self.types_import = types

def visit_begin(self, schema: QAPISchema) -> None:

self.schema = schema

+ # iterate once in schema.docs to map doc objects to its name

+ for doc in schema.docs:

+ if doc.symbol is None:

+ continue

+ self.docmap[doc.symbol] = doc

+

+ for qapitype, imports in self.types_import.items():

+ self.types[qapitype] = TEMPLATE_GENERATED_HEADER[1:].format(

+ package_name=self.golang_package_name

+ )

+ self.types[qapitype] += generate_template_imports(imports)

+

def visit_end(self) -> None:

del self.schema

+ self.types["enum"] += generate_content_from_dict(self.enums)

def visit_object_type(

self,

@@ -XXX,XX +XXX,XX @@ def visit_enum_type(

members: List[QAPISchemaEnumMember],

prefix: Optional[str],

) -> None:

- pass

+ assert name not in self.enums

+ doc = self.docmap.get(name, None)

+ maindoc, docfields = qapi_to_golang_struct_docs(doc)

+

+ # The logic below is to generate QAPI enums as blocks of Go consts

+ # each with its own type for type safety inside Go applications.

+ #

+ # Block of const() blocks are vertically indented so we have to

+ # first iterate over all names to calculate space between

+ # $var_name and $var_type. This is achieved by helper function

+ # @fetch_indent_blocks_over_enum_with_docs()

+ #

+ # A new indentation block is defined by empty line or a comment.

+

+ indent_block = iter(

+ fetch_indent_blocks_over_enum_with_docs(name, members, docfields)

+ )

+ maxname = next(indent_block)

+ fields = ""

+ for index, member in enumerate(members):

+ # For simplicity, every time we have doc, we go to next indent block

+ hasdoc = member.name is not None and member.name in docfields

+

+ if hasdoc:

+ maxname = next(indent_block)

+

+ enum_name = f"{name}{qapi_to_field_name_enum(member.name)}"

+ name2type = " " * (maxname - len(enum_name) + 1)

+

+ if hasdoc:

+ docstr = (

+ textwrap.TextWrapper(width=80)

+ .fill(docfields[member.name])

+ .replace("\n", "\n\t// ")

+ )

+ fields += f"""\t// {docstr}\n"""

+

+ fields += f"""\t{enum_name}{name2type}{name} = "{member.name}"\n"""

+

+ if maindoc != "":

+ maindoc = f"\n{maindoc}"

+

+ self.enums[name] = maindoc + TEMPLATE_ENUM.format(

+ name=name, fields=fields[:-1]

+ )

def visit_array_type(

self,

@@ -XXX,XX +XXX,XX @@ def write(self, outdir: str) -> None:

srcpath = os.path.join(srcdir, filename)

dstpath = os.path.join(targetpath, filename)

shutil.copyfile(srcpath, dstpath)

+

+ # Types to be generated

+ for qapitype, content in self.types.items():

+ gofile = f"gen_type_{qapitype}.go"

+ pathname = os.path.join(targetpath, gofile)

+

+ with open(pathname, "w", encoding="utf8") as outfile:

+ outfile.write(content)

--

2.48.1

This patch handles QAPI alternate types and generates data structures

in Go that handles it.

Alternate types are similar to Union but without a discriminator that

can be used to identify the underlying value on the wire.

1. Over the wire, we need to infer underlying value by its type

2. Pointer to types are mapped as optional. Absent value can be a

valid value.

3. We use Go's standard 'encoding/json' library with its Marshal

and Unmarshal interfaces.

4. As an exceptional but valid case, there are types that accept

JSON NULL as value. Due to limitations with Go's standard library

(point 3) combined with Absent being a possibility (point 2), we

translante NULL values to a boolean field called 'IsNull'. See the

second example and docs/devel/qapi-golang-code-gen.rst under

Alternate section.

* First example:

qapi:

| ##

| # @BlockdevRef:

| #

| # Reference to a block device.

| #

| # @definition: defines a new block device inline

| #

| # @reference: references the ID of an existing block device

| #

| # Since: 2.9

| ##

| { 'alternate': 'BlockdevRef',

| 'data': { 'definition': 'BlockdevOptions',

| 'reference': 'str' } }

go:

| // Reference to a block device.

| //

| // Since: 2.9

| type BlockdevRef struct {

|     // defines a new block device inline

|     Definition *BlockdevOptions

|     // references the ID of an existing block device

|     Reference *string

| }

|

| func (s BlockdevRef) MarshalJSON() ([]byte, error) {

| ...

| }

|

| func (s *BlockdevRef) UnmarshalJSON(data []byte) error {

| ...

| }

usage:

| input := `{"driver":"qcow2","data-file":"/some/place/my-image"}`

| k := BlockdevRef{}

| err := json.Unmarshal([]byte(input), &k)

| if err != nil {

| panic(err)

| }

| // *k.Definition.Qcow2.DataFile.Reference == "/some/place/my-image"

* Second example:

qapi:

| { 'alternate': 'StrOrNull',

| 'data': { 's': 'str',

| 'n': 'null' } }

| // This is a string value or the explicit lack of a string (null

| // pointer in C). Intended for cases when 'optional absent' already

| // has a different meaning.

| //

| // Since: 2.10

| type StrOrNull struct {

|     // the string value

|     S *string

|     // no string value

|     IsNull bool

| }

|

| // Helper function to get its underlying Go value or absent of value

| func (s *StrOrNull) ToAnyOrAbsent() (any, bool) {

| ...

| }

|

| func (s StrOrNull) MarshalJSON() ([]byte, error) {

| ...

| }

|

| func (s *StrOrNull) UnmarshalJSON(data []byte) error {

| ...

| }

Signed-off-by: Victor Toso <victortoso@redhat.com>

---

scripts/qapi/golang/golang.py | 306 +++++++++++++++++++++++++++++++++-

scripts/qapi/golang/utils.go | 26 +++

2 files changed, 329 insertions(+), 3 deletions(-)

create mode 100644 scripts/qapi/golang/utils.go

diff --git a/scripts/qapi/golang/golang.py b/scripts/qapi/golang/golang.py

index XXXXXXX..XXXXXXX 100644

--- a/scripts/qapi/golang/golang.py

+++ b/scripts/qapi/golang/golang.py

@@ -XXX,XX +XXX,XX @@

from __future__ import annotations

import os, shutil, textwrap

-from typing import List, Optional

+from typing import List, Optional, Tuple

from ..schema import (

QAPISchema,

+ QAPISchemaAlternateType,

QAPISchemaBranches,

QAPISchemaEnumMember,

QAPISchemaFeature,

@@ -XXX,XX +XXX,XX @@

)

from ..source import QAPISourceInfo

+FOUR_SPACES = " "

+

TEMPLATE_GENERATED_HEADER = """

/*

* Copyright 2025 Red Hat, Inc.

@@ -XXX,XX +XXX,XX @@

)

"""

+TEMPLATE_ALTERNATE_CHECK_INVALID_JSON_NULL = """

+ // Check for json-null first

+ if string(data) == "null" {{

+ return errors.New(`null not supported for {name}`)

+ }}"""

+

+TEMPLATE_ALTERNATE_NULLABLE_CHECK = """

+ }} else if s.{var_name} != nil {{

+ return *s.{var_name}, false"""

+

+TEMPLATE_ALTERNATE_MARSHAL_CHECK = """

+ if s.{var_name} != nil {{

+ return json.Marshal(s.{var_name})

+ }} else """

+

+TEMPLATE_ALTERNATE_UNMARSHAL_CHECK = """

+ // Check for {var_type}

+ {{

+ s.{var_name} = new({var_type})

+ if err := strictDecode(s.{var_name}, data); err == nil {{

+ return nil

+ }}

+ s.{var_name} = nil

+ }}

+

+"""

+

+TEMPLATE_ALTERNATE_NULLABLE_MARSHAL_CHECK = """

+ if s.IsNull {

+ return []byte("null"), nil

+ } else """

+

+TEMPLATE_ALTERNATE_NULLABLE_UNMARSHAL_CHECK = """

+ // Check for json-null first

+ if string(data) == "null" {

+ s.IsNull = true

+ return nil

+ }"""

+

+TEMPLATE_ALTERNATE_METHODS = """

+func (s {name}) MarshalJSON() ([]byte, error) {{

+{marshal_check_fields}

+ return {marshal_return_default}

+}}

+

+func (s *{name}) UnmarshalJSON(data []byte) error {{

+{unmarshal_check_fields}

+ return fmt.Errorf("Can't convert to {name}: %s", string(data))

+}}

+"""

+

# Takes the documentation object of a specific type and returns

# that type's documentation and its member's docs.

@@ -XXX,XX +XXX,XX @@ def gen_golang(schema: QAPISchema, output_dir: str, prefix: str) -> None:

vis.write(output_dir)

+def qapi_to_field_name(name: str) -> str:

+ return name.title().replace("_", "").replace("-", "")

+

+

def qapi_to_field_name_enum(name: str) -> str:

return name.title().replace("-", "")

+def qapi_schema_type_to_go_type(qapitype: str) -> str:

+ schema_types_to_go = {

+ "str": "string",

+ "null": "nil",

+ "bool": "bool",

+ "number": "float64",

+ "size": "uint64",

+ "int": "int64",

+ "int8": "int8",

+ "int16": "int16",

+ "int32": "int32",

+ "int64": "int64",

+ "uint8": "uint8",

+ "uint16": "uint16",

+ "uint32": "uint32",

+ "uint64": "uint64",

+ "any": "any",

+ "QType": "QType",

+ }

+

+ prefix = ""

+ if qapitype.endswith("List"):

+ prefix = "[]"

+ qapitype = qapitype[:-4]

+

+ qapitype = schema_types_to_go.get(qapitype, qapitype)

+ return prefix + qapitype

+

+

+# Helper for Alternate generation

+def qapi_field_to_alternate_go_field(

+ member_name: str, type_name: str

+) -> Tuple[str, str, str]:

+ # Nothing to generate on null types. We update some

+ # variables to handle json-null on marshalling methods.

+ if type_name == "null":

+ return "IsNull", "bool", ""

+

+ # On Alternates, fields are optional represented in Go as pointer

+ return (

+ qapi_to_field_name(member_name),

+ qapi_schema_type_to_go_type(type_name),

+ "*",

+ )

+

+

+def fetch_indent_blocks_over_args(

+ args: List[dict[str:str]],

+) -> Tuple[int, int]:

+ maxname, maxtype = 0, 0

+ blocks: tuple(int, int) = []

+ for arg in args:

+ if "comment" in arg or "doc" in arg:

+ blocks.append((maxname, maxtype))

+ maxname, maxtype = 0, 0

+

+ if "comment" in arg:

+ # They are single blocks

+ continue

+

+ if "type" not in arg:

+ # Embed type are on top of the struct and the following

+ # fields do not consider it for formatting

+ blocks.append((maxname, maxtype))

+ maxname, maxtype = 0, 0

+ continue

+

+ maxname = max(maxname, len(arg.get("name", "")))

+ maxtype = max(maxtype, len(arg.get("type", "")))

+

+ blocks.append((maxname, maxtype))

+ return blocks

+

+

def fetch_indent_blocks_over_enum_with_docs(

name: str, members: List[QAPISchemaEnumMember], docfields: Dict[str, str]

) -> Tuple[int]:

@@ -XXX,XX +XXX,XX @@ def fetch_indent_blocks_over_enum_with_docs(

return blocks

+# Helper function for boxed or self contained structures.

+def generate_struct_type(

+ type_name,

+ type_doc: str = "",

+ args: List[dict[str:str]] = None,

+ indent: int = 0,

+) -> str:

+ base_indent = FOUR_SPACES * indent

+

+ with_type = ""

+ if type_name != "":

+ with_type = f"\n{base_indent}type {type_name}"

+

+ if type_doc != "":

+ # Append line jump only if type_doc exists

+ type_doc = f"\n{type_doc}"

+

+ if args is None:

+ # No args, early return

+ return f"""{type_doc}{with_type} struct{{}}"""

+

+ # The logic below is to generate fields of the struct.

+ # We have to be mindful of the different indentation possibilities between

+ # $var_name $var_type $var_tag that are vertically indented with gofmt.

+ #

+ # So, we first have to iterate over all args and find all indent blocks

+ # by calculating the spaces between (1) member and type and between (2)

+ # the type and tag. (1) and (2) is the tuple present in List returned

+ # by the helper function fetch_indent_blocks_over_args.

+ inner_indent = base_indent + FOUR_SPACES

+ doc_indent = inner_indent + "// "

+ fmt = textwrap.TextWrapper(

+ width=70, initial_indent=doc_indent, subsequent_indent=doc_indent

+ )

+

+ indent_block = iter(fetch_indent_blocks_over_args(args))

+ maxname, maxtype = next(indent_block)

+ members = " {\n"

+ for index, arg in enumerate(args):

+ if "comment" in arg:

+ maxname, maxtype = next(indent_block)

+ members += f""" // {arg["comment"]}\n"""

+ # comments are single blocks, so we can skip to next arg

+ continue

+

+ name2type = ""

+ if "doc" in arg:

+ maxname, maxtype = next(indent_block)

+ members += fmt.fill(arg["doc"])

+ members += "\n"

+

+ name = arg["name"]

+ if "type" in arg:

+ namelen = len(name)

+ name2type = " " * max(1, (maxname - namelen + 1))

+

+ type2tag = ""

+ if "tag" in arg:

+ typelen = len(arg["type"])

+ type2tag = " " * max(1, (maxtype - typelen + 1))

+

+ gotype = arg.get("type", "")

+ tag = arg.get("tag", "")

+ members += (

+ f"""{inner_indent}{name}{name2type}{gotype}{type2tag}{tag}\n"""

+ )

+

+ members += f"{base_indent}}}\n"

+ return f"""{type_doc}{with_type} struct{members}"""

+

+

+def generate_template_alternate(

+ self: QAPISchemaGenGolangVisitor,

+ name: str,

+ variants: Optional[QAPISchemaVariants],

+) -> str:

+ args: List[dict[str:str]] = []

+ nullable = name in self.accept_null_types

+ if nullable:

+ # Examples in QEMU QAPI schema: StrOrNull and BlockdevRefOrNull

+ marshal_return_default = """[]byte("{}"), nil"""

+ marshal_check_fields = TEMPLATE_ALTERNATE_NULLABLE_MARSHAL_CHECK[1:]

+ unmarshal_check_fields = TEMPLATE_ALTERNATE_NULLABLE_UNMARSHAL_CHECK

+ else:

+ marshal_return_default = f'nil, errors.New("{name} has empty fields")'

+ marshal_check_fields = ""

+ unmarshal_check_fields = (

+ TEMPLATE_ALTERNATE_CHECK_INVALID_JSON_NULL.format(name=name)

+ )

+

+ doc = self.docmap.get(name, None)

+ content, docfields = qapi_to_golang_struct_docs(doc)

+ if variants:

+ for var in variants.variants:

+ var_name, var_type, isptr = qapi_field_to_alternate_go_field(

+ var.name, var.type.name

+ )

+ args.append(

+ {

+ "name": f"{var_name}",

+ "type": f"{isptr}{var_type}",

+ "doc": docfields.get(var.name, ""),

+ }

+ )

+ # Null is special, handled first

+ if var.type.name == "null":

+ assert nullable

+ continue

+

+ skip_indent = 1 + len(FOUR_SPACES)

+ if marshal_check_fields == "":

+ skip_indent = 1

+ marshal_check_fields += TEMPLATE_ALTERNATE_MARSHAL_CHECK[

+ skip_indent:

+ ].format(var_name=var_name)

+ unmarshal_check_fields += TEMPLATE_ALTERNATE_UNMARSHAL_CHECK[

+ :-1

+ ].format(var_name=var_name, var_type=var_type)

+

+ content += string_to_code(generate_struct_type(name, args=args))

+ content += string_to_code(

+ TEMPLATE_ALTERNATE_METHODS.format(

+ name=name,

+ marshal_check_fields=marshal_check_fields[:-6],

+ marshal_return_default=marshal_return_default,

+ unmarshal_check_fields=unmarshal_check_fields[1:],

+ )

+ )

+ return "\n" + content

+

+

def generate_content_from_dict(data: dict[str, str]) -> str:

content = ""

@@ -XXX,XX +XXX,XX @@ def generate_content_from_dict(data: dict[str, str]) -> str:

return content.replace("\n\n\n", "\n\n")

+def string_to_code(text: str) -> str:

+ DOUBLE_BACKTICK = "``"

+ result = ""

+ for line in text.splitlines():

+ # replace left four spaces with tabs

+ limit = len(line) - len(line.lstrip())

+ result += line[:limit].replace(FOUR_SPACES, "\t")

+

+ # work with the rest of the line

+ if line[limit : limit + 2] == "//":

+ # gofmt tool does not like comments with backticks.

+ result += line[limit:].replace(DOUBLE_BACKTICK, '"')

+ else:

+ result += line[limit:]

+ result += "\n"

+

+ return result

+

+

def generate_template_imports(words: List[str]) -> str:

if len(words) == 0:

return ""

@@ -XXX,XX +XXX,XX @@ class QAPISchemaGenGolangVisitor(QAPISchemaVisitor):

# pylint: disable=too-many-arguments

def __init__(self, _: str):

super().__init__()

- gofiles = ("protocol.go",)

+ gofiles = ("protocol.go", "utils.go")

# Map each qapi type to the necessary Go imports

types = {

+ "alternate": ["encoding/json", "errors", "fmt"],

"enum": [],

}

@@ -XXX,XX +XXX,XX @@ def __init__(self, _: str):

self.golang_package_name = "qapi"

self.duplicate = list(gofiles)

self.enums: dict[str, str] = {}

+ self.alternates: dict[str, str] = {}

+ self.accept_null_types = []

self.docmap = {}

self.types = dict.fromkeys(types, "")

@@ -XXX,XX +XXX,XX @@ def __init__(self, _: str):

def visit_begin(self, schema: QAPISchema) -> None:

self.schema = schema

+ # We need to be aware of any types that accept JSON NULL

+ for name, entity in self.schema._entity_dict.items():

+ if not isinstance(entity, QAPISchemaAlternateType):

+ # Assume that only Alternate types accept JSON NULL

+ continue

+

+ for var in entity.alternatives.variants:

+ if var.type.name == "null":

+ self.accept_null_types.append(name)

+ break

+

# iterate once in schema.docs to map doc objects to its name

for doc in schema.docs:

if doc.symbol is None:

@@ -XXX,XX +XXX,XX @@ def visit_begin(self, schema: QAPISchema) -> None:

def visit_end(self) -> None:

del self.schema

self.types["enum"] += generate_content_from_dict(self.enums)

+ self.types["alternate"] += generate_content_from_dict(self.alternates)

def visit_object_type(

self,

@@ -XXX,XX +XXX,XX @@ def visit_alternate_type(

features: List[QAPISchemaFeature],

variants: QAPISchemaVariants,

) -> None:

- pass

+ assert name not in self.alternates

+ self.alternates[name] = generate_template_alternate(

+ self, name, variants

+ )

def visit_enum_type(

self,

diff --git a/scripts/qapi/golang/utils.go b/scripts/qapi/golang/utils.go

new file mode 100644

index XXXXXXX..XXXXXXX

--- /dev/null

+++ b/scripts/qapi/golang/utils.go

@@ -XXX,XX +XXX,XX @@

+/*

+ * Copyright 2025 Red Hat, Inc.

+ * SPDX-License-Identifier: MIT-0

+ *

+ * Authors:

+ * Victor Toso <victortoso@redhat.com>

+ */

+package qapi

+

+import (

+    "encoding/json"

+    "strings"

+)

+

+// Creates a decoder that errors on unknown Fields

+// Returns nil if successfully decoded @from payload to @into type

+// Returns error if failed to decode @from payload to @into type

+func strictDecode(into interface{}, from []byte) error {

+    dec := json.NewDecoder(strings.NewReader(string(from)))

+    dec.DisallowUnknownFields()

+

+    if err := dec.Decode(into); err != nil {

+        return err

+    }

+    return nil

+}

--

2.48.1

This patch handles QAPI struct types and generates the equivalent

types in Go. The following patch adds extra logic when a member of the

struct has a Type that can take JSON Null value (e.g: StrOrNull in

QEMU)

The highlights of this implementation are:

1. Generating a Go struct that requires a @base type, the @base type

fields are copied over to the Go struct. The advantage of this

approach is to not have embed structs in any of the QAPI types.

Note that embedding a @base type is recursive, that is, if the

@base type has a @base, all of those fields will be copied over.

2. About the Go struct's fields:

i) They can be either by Value or Reference.

ii) Every field that is marked as optional in the QAPI specification

are translated to Reference fields in its Go structure. This

design decision is the most straightforward way to check if a

given field was set or not. Exception only for types that can

take JSON Null value.

iii) Mandatory fields are always by Value with the exception of QAPI

arrays, which are handled by Reference (to a block of memory) by

Go.

iv) All the fields are named with Uppercase due Golang's export

convention.

Example:

qapi:

| ##

| # @BlockdevCreateOptionsFile:

| #

| # Driver specific image creation options for file.

| #

| # @filename: Filename for the new image file

| #

| # @size: Size of the virtual disk in bytes

| #

| # @preallocation: Preallocation mode for the new image (default: off;

| # allowed values: off, falloc (if CONFIG_POSIX_FALLOCATE), full

| # (if CONFIG_POSIX))

| #

| # @nocow: Turn off copy-on-write (valid only on btrfs; default: off)

| #

| # @extent-size-hint: Extent size hint to add to the image file; 0 for

| # not adding an extent size hint (default: 1 MB, since 5.1)

| #

| # Since: 2.12

| ##

| { 'struct': 'BlockdevCreateOptionsFile',

| 'data': { 'filename': 'str',

| 'size': 'size',

| '*preallocation': 'PreallocMode',

| '*nocow': 'bool',

| '*extent-size-hint': 'size'} }

go:

| // Driver specific image creation options for file.

| //

| // Since: 2.12

| type BlockdevCreateOptionsFile struct {

|     // Filename for the new image file

|     Filename string `json:"filename"`

|     // Size of the virtual disk in bytes

|     Size uint64 `json:"size"`

|     // Preallocation mode for the new image (default: off; allowed

|     // values: off, falloc (if CONFIG_POSIX_FALLOCATE), full (if

|     // CONFIG_POSIX))

|     Preallocation *PreallocMode `json:"preallocation,omitempty"`

|     // Turn off copy-on-write (valid only on btrfs; default: off)

|     Nocow *bool `json:"nocow,omitempty"`

|     // Extent size hint to add to the image file; 0 for not adding an

|     // extent size hint (default: 1 MB, since 5.1)

|     ExtentSizeHint *uint64 `json:"extent-size-hint,omitempty"`

| }

Signed-off-by: Victor Toso <victortoso@redhat.com>

---

scripts/qapi/golang/golang.py | 193 +++++++++++++++++++++++++++++++++-

1 file changed, 192 insertions(+), 1 deletion(-)

diff --git a/scripts/qapi/golang/golang.py b/scripts/qapi/golang/golang.py

index XXXXXXX..XXXXXXX 100644

--- a/scripts/qapi/golang/golang.py

+++ b/scripts/qapi/golang/golang.py

@@ -XXX,XX +XXX,XX @@ def gen_golang(schema: QAPISchema, output_dir: str, prefix: str) -> None:

vis.write(output_dir)

+def qapi_name_is_base(name: str) -> bool:

+ return qapi_name_is_object(name) and name.endswith("-base")

+

+

+def qapi_name_is_object(name: str) -> bool:

+ return name.startswith("q_obj_")

+

+

def qapi_to_field_name(name: str) -> str:

return name.title().replace("_", "").replace("-", "")

@@ -XXX,XX +XXX,XX @@ def qapi_to_field_name_enum(name: str) -> str:

return name.title().replace("-", "")